https://doi.org/10.19053/01227238.12833

Artículos

 

 

e-Learning Integrated STEMEducation Center (eLISE) in Asia: A Reflection Case Study of Taiwanand Vietnam Research Project

Centro integrado de e-Learning y educación en STEM en Asia: una reflexión desde la experiencia de investigación en Taiwan y Vietnam

Centro integrado de e-Learning e educação em STEM na Ásia: uma reflexão a partir da perspectiva de pesquisa em Taiwan e Vietnã

 

 

Chun-Yen Chang[1]* https://orcid.org/0000-0003-2373-2004

Pei-Ling Lin[2]** https://orcid.org/0000-0003-3028-9231

Nguyln Thj To Khuyên[3]*** https://orcid.org/0000-0003-3592-0969

 

* Science Education Center, National Taiwan Normal University, Taiwan

** Chung Yuan Christian University, Taiwan

*** Graduate Institute of Science Education, National Taiwan Normal University, Taiwan

 

Abstract

Objective:This article describes the implementation plan, advance and future directions of the academic and educational research center eLISE (e-Learning Integrated STEM Education Center) whose foundation intends to narrow the collaboration between Taiwan and Vietnam in e-Learning and STEM (Science, Technology, Engineering, Mathematics) in the framework of the New Southbound Policy[4], a long-term project announced by the Taiwanese government to strengthen the partnership with South Asian countries.

Originality / contribution:This article contributes to the reflection about the promotion of international cooperation in innovation, scientific and technological research as well as the analysis of public policies guided towards e-Learning and STEM innovation.

Method/ strategies: The stages of the research and innovation process were (1) Innovation e-Learning and STEM Instruction teaching material and module: test application, teacher workshops and interviews (2) Innovation e-Learning and STEM assessment through CloudClassRoom observation with Gamified Electronic Audio Response System and Google Bert.

Conclusions: The description of the implementation of eLISE, the explanation of the development of innovative curriculums and teacher workshops, as well as the talent exchange and the cooperation between industry and academy, highlights the importance of cultivate talents and workforces educated through STEM and e-Learning, seeking regional development and prosperity for both, Taiwan and Vietnam.

Keywords: STEM; Asia; New Southbound Policy; Cross-border cooperation; e-Learning.

Resumen

Objetivo: Este artículo describe el plan de implementación, avance y direcciones futuras del centro de investigación académica y educativa eLISE (e-Learning Integrated STEM Education Center), cuya fundación pretende estrechar la colaboración entre Taiwán y Vietnam en e-Learning y educación en STEM (Science, Technology, Engineering, Mathematics) en el marco de3 vs

la Nueva Política hacia el Sur, un proyecto a largo plazo propuesto por el Gobierno Taiwanés que busca el fortalecimiento de las relaciones entre los países surasiáticos.

Originalidad/aporte: Aporta a la reflexión sobre la promoción de la cooperación internacional en innovación, investigación científica y tecnológica, así como en el análisis de políticas públicas guiadas hacia la innovación en e-Learning y educación en STEM.

Método/estrategias de recolección de información: Las etapas del proceso de investigación e innovación fueron: (1) Módulo y material de innovación e-Learning y educación: aplicación de test, talleres de formación de profesores y entrevistas (2) Innovación e-Learning y evaluación STEM mediante observación de CloudClassRoom, además de Gamified Electronic Audio Response System y Google Bert.

Conclusiones: La descripción de la implementación de eLISE, la explicación del desarrollo de currículos innovadores y la implementación de los talleres de profesores, así como el intercambio de talentos y la cooperación entre industria y academia releva la importancia de cultivar talentos y fuerzas de trabajo educados en e-Learning y STEM que busquen el desarrollo regional y la prosperidad para Taiwán y Vietnam.

Palabras clave: Educación en STEM; Asía; nueva política hacia el Sur; cooperación transfronteriza, e-learning.

Resumo

Objetivo: Este artigo descreve o plano de implementação, progresso e direções futuras do centro de pesquisa acadêmica e educativa eLISE (e-Learning Integrated STEM Education Center), cuja fundação pretende fortalecer a colaboração entre Taiwan e Vietnã no e-Learning e educação em STEM (Science, Technology, Engineering, Mathematics) no marco da Nova Política para o Sul, um projeto de longo prazo proposto pelo governo taiwanês que busca o fortalecimento das relações entre os países do sul asiático.

Originalidade/contribuição: contribui para a reflexão sobre a promoção da cooperação internacional na inovação, pesquisa científica e tecnológica, assim como na análise de políticas voltadas para a inovação em e-Learning e educação em STEM.

Método/estratégias de coleta de dados: As etapas do processo de pesquisa e inovação foram: (1) Módulo e material de inovação e-Learning e educação:

aplicação de testes, oficinas de formação de professores e entrevistas, (2) Inovação e-Learnig e avaliação STEM mediante observação de CloudClassRoom, além do Gamifield Electronic Audio Response System e Google Bert.

Conclusões: A descrição da implementação de eLISE, a explicação do desenvolvimento de currículos inovadores e a implementação das oficinas para professores, bem como o intercâmbio de talentos e a cooperação entre a indústria e a academia destacam a importância de cultivar talentos e forças de trabalho formadas em e-Learning e STEM que busquem o desenvolvimento regional e a prosperidade para Taiwan e Vietnã.

Palavras-chave: Educação em STEM; Ásia; nova política para o Sul; cooperação transfronteiriça; e-leaming.

 

Recibido: 10/11/2020

Evaluado: 23/01/2021

Aprobado: 20/01/2021

 

Introduction

 

“One STEM teacher, one thousand STEM minds (talents)”- The aim of e-Learning Integrated STEM Education Center (eLISE)

To develop, extend cooperation and partnerships among New Southbound Countries (Thailand, Indonesia, Philippines, Malaysia, Singapore, Brunei, Vietnam, Myanmar, Cambodia, Laos, India, Pakistan, Bangladesh, Nepal, Sri Lanka, Bhutan, as well as New Zealand 18 and Australia) has become a vital national goal for Taiwan. In 2016, Taiwan’s government announced a long-term project called the “New Southbound Policy”, aiming to strengthen the partnership with South Asian countries. Thus, to establish a crossed-countries educational and academic research center for close collaboration towards e-Learning and STEM (Science, Technology Engineering, and Mathematics) education, aptly named the e-Learning Integrated STEM Education Center (eLISE), is the core purpose of this project proposal. The establishment of eLISE will expand and enhance Taiwan’s international influence in innovative e-learning and STEM education fields.

Redefined as a “New Asia” for the next generation, this policy emerges as the immediate and future vision of the region. According to a report from the Taiwanese Ministry of education, at the elementary and middle school level, one in ten of the Taiwanese students’ parents come from a New Southbound country[5]. Ten percent of the next generation of Taiwanese’s science and technology enlightenment come from New Asian countries; accordingly connecting and cooperating with New Asian countries for e-learning and STEM education investigation and research is a vital goal for national compulsory education in Taiwan. The need to develop a new paradigm of e-Learning and STEM from local views, to fit the needs of students in New Asia countries, is an emergent and valuable goal for Asia STEM education. We believe the establishment of eLISE will help cultivate the STEM and e-Learning workforce and further raise national industries’ competitive levels for both Taiwan and Vietnam. By using Vietnam as a bridgehead, the impact of eLISE greatly extends out to other associations of Southeast Asian Nations (ASEAN) countries.

Alongside the rapid changes and developments in our society the problems and issues we face today have become increasingly complex and require knowledge and understanding from many subjects, often linked together and connected[6]. In today’s modern society there are continual innovations in electrical engineering and technology for applications in our daily life. The need for contextual, competency-oriented innovative e-learning, STEM curriculum, assessment, and training that helps students develop the competencies to respond to the challenges in todays society, is rising[7]. How to cultivate the talents in e-Learning and STEM fields and attract more students and talents form New Asian countries in pursuing academic degrees in Taiwan, based on Taiwan’s existing science and technology achievements, is a core target for eLISE.

The goals of eLISE include:

1.   To promote international cooperation in scientific research and technological innovation in the areas of e-learning and STEM (Science, Technology, Engineering, and Mathematics) education;

2.   To encourage two-way exchange and cultivation of talents in innovative e-Learning and STEM education fields;

3.   To establish a database of local scientific research talents in innovative e-Learning and STEM education fields;

4.   To promote industry-academia cooperation and technological development in innovative e-Learning and STEM education fields;

5.   To research and analyze Taiwan’s and Vietnam’s government policies toward innovative e-Learning and STEM education;

6.   By using Vietnam as a bridgehead, and cooperating with “Science and Technology Division in Hanoi, Vietnam, Ministry of Science and Technology,” eLISE can further enter and impact e-Learning and STEM education in Laos, Cambodia, Thailand, Indonesia, Malaysia, and Myanmar, extending out across the IndoChina peninsula.r

The core teams of eLISE will be The Science Education Center at National Taiwan Normal University (SEC, NTNU) and Hanoi National University of Education (HNUE). NTNU and HNUE are both the leading university of teacher cultivating in Taiwan and Vietnam, respectively.

SEC, NTNU, is known for its contributions to research and development in science education. As the major think tank for the Ministry of Education of Taiwan. The Science Education Center accommodates experienced researchers of science and science education, mainly coming from the College of Sciences at the National Taiwan Normal University. SEC, NTNU advises on strategic planning for the Ministry of Education and carries out research projects funded by the Ministry of Education and Ministry of Science and Technology of Taiwan. Our research team in Taiwan is deeply involved in the development and implementation of interdisciplinary courses in science and technology. From 2017 to 2020, the SEC, NTNU has been running a three-year joint project, funded by the Taiwanese government’s new south boundary policy: The STEM for 2TV (Science, Technology, Engineering, and Mathematics for Taiwan, Thailand, and Vietnam) collaborative project between NTNU, Kasetsart University (KU) in Thailand and Hanoi National University of Education (HNUE) in Vietnam.

HNUE is a leading university in promoting e-Learning and STEM education in Vietnam (ranked 47 th in Vietnam)[8]. The HNUE faculty members involved in this project are experts in designing and implementing integrated themes to teach science in an interdisciplinary manner. HNUE is also the main organization to prepare teachers of integrated science teaching in Vietnam.

Our previous cooperation and results of the STEM2TV project show the effects of adopting 20 a Taiwanese based curriculum design toward Vietnamese and Thailand in-service teachers’ and students’ needs. The teacher-training workshop(s) includes both STEM theoretical and practical content and innovative ICT tools. The in-person interaction with local teachers further allowed researchers to investigate the well-designed STEM pedagogy and gather Œ feedback from different countries’ teachers for New-Asia STEM education development.

Based on the process and results of implementing a STEM curriculum in a different country and context, the framework for designing suitable STEM curriculum to fit local needs is clarified. According to the empirical and practical results of this project, the process of developing a STEM curriculum framework is a reference for designing in-service teachers’ STEM education training.

Our experiences and results of the STEM2TV project can also contribute to how to improve international collaboration, to leverage different experiences, and to guide local e-learning and STEM education research in obtaining the comprehensive and contextual understanding among different southbound countries for promoting e-learning and STEM education.

SEC, NTNU, and HNUE have built up substantial and close cooperative researches and events since the STEM2TV project starting. We believe this joint venture of the e-Learning Integrated STEM Education Center is an indispensable opportunity to create extensive cooperation in achieving regional development and prosperity for Vietnam and Taiwan. The related STEM2TV academic publications and invited/keynote speeches are as follows.

 

Literature review-The strength and solid foundation ofcrossed-countries cooperation between NTNU and HNUE

 

The SEC, NTNU, has rich experiences, fruitful cooperation, and alliance with EiNUE as well as middle schools in Vietnam. From 2017 to 2020, the SEC, NTNU held a three- year joint project, funded by the Taiwanese government’s new south boundary policy: the STEM for 2TV (Science, Technology, Engineering, and Mathematics for Taiwan, Thailand, and Vietnam) project, in conjunction with Kasetsart University (KU) in Thailand and Hanoi National University of Education (HNUE) in Vietnam. The STEM2TV project aims to investigate and develop an innovative approach to STEM education in this region. The project goal focuses on how to develop modules, assessment tools, and professional teacher training to fit students’ needs and help teachers build their STEM curriculum and pedagogies in New Asian countries. Figure 1 below shows the project framework.

 

Figure 1. The project framework and vision for STEM2TV

 

From an international perspective, Taiwan and Vietnam both support STEM education as a means of increasing global competitiveness and economic growth. The governments (Ministry of Science and Technology in Taiwan and Ministry of Education and Training in Vietnam, have initiated substantial funding toward STEM education developments[9]. Evenwith the policy supports from these two countries encouraging more communities to join and explore vast possibilities of improving STEM education within the context of STEM education, Taiwan and Vietnam share many similarities, including the same challenges of implementing STEM modules and e-leamingin classrooms. The high-stake examination for enrolling high schools, and universities and large class sizes (around 30-50 students in one class) are significant and shared challenges. Table 1 summarize contexts between Taiwan and Vietnam.

 

Table 1. The summary of the similarities and differences of STEM contexts between Taiwan and Vietnam.

 

Taiwan

Vietnam

Government STEM initiatives and funding

Ministry of Science and Technology

X

High stake examination

Joint College Entrance Examination (JCEE)

University Entrance Examination (UEE)

Large class size

V

V

Located in the Chinese (Confucianism) cultural sphere

V

V

PISA results: High performance with low interest in science

V

(ranked 4th)

V

(ranked 8th)

Pre-service Teachers' professional Developing

Theoretical trainingtoward Science education

Practical training toward Science classroom

 

 

The need to develop suitable e-Teaming in Taiwan and Vietnam base on another common issue relates to the so-called Confucianism culture and the silent classroom. Although Taiwanese, and perhaps Asian, students usually remain quiet in classrooms, they might not share the same reason for being silent. They may even not prefer to be quiet in class at all. As indicated by recent research on students’ preferences towards science classroom participation[10], students have more diverse preferences towards science classroom participation if verbal and non-verbal classroom participatory practices are seen as being coexistent, ratherthan dichotomous. In previous studies[11] student engagement in verbal practices, such as asking or responding to questions, are seen as verbal participation.

On the other hand, student engagement in non-verbal practices, such as attentive listening, thinking, doing practical work, and taking notes, is seen as non-verbal participation. The studies revealed a variety of classroom participation preferences; with, only about 40% of students preferring non-verbal participatory practices[12]. As suggested by learning environment research[13], the mismatch between students preferred and actual learning environments can have a negative impact on students’ learning outcomes. Thus, exploring and conducting the multivariate curriculum and assessments’ designs of e-Learning, which can fit the needs of silent and verbal participatory students, will be another important consideration for our project goal.

With our close alliance and fruitful results with the HNUE in the STEM2TV and the similarities and differences of STEM contexts between Taiwan and Vietnam, we believe that HNUE is the most suitable institute to cooperate to establish the eLISE center.

Cuadro de texto: 23Up to now, the development of related innovative e-Learning and STEM curriculum modules, assessments, teaching material from the SEC, NTNU is becoming mature and has obtained several patents (show as figure 2). Through the establishment of the eLISE center, Taiwan can help HNUE to develop the local innovative e-Learning and STEM curriculum modules, assessments, teaching material of Vietnam, and our eLSE network layout can also link to both the industrial and academic ends of Vietnam. In terms of academic collaborations, the eLISE will cooperate with the numbers of teacher education universities in all parts of Vietnam (for example, the Thai Nguyen University of Education, University of Technical Education, Ho Chi Minh City, etc.) to promote the results of the eLISE to local primary and secondary schools in Vietnam. In terms of industry-school cooperation, the eLISE will modularize our innovative e-Learning and STEM curriculum modules, assessments, teaching material, and cooperate with local commercial companies of scientific games and teaching material in Vietnam, in order to assist the technology transfer of academic research results successfully.

The international cooperation between Taiwan and Vietnam through eLISE will help Taiwan promote further and impact the e-leaming and STEM education in other ASEN countries. Using Vietnam as a bridgehead, eLISE can further enter and impact the e-Leaming and STEM education in Laos, Cambodia, Thailand, Indonesia, Malaysia, and Myanmar and across the entire Indo-China peninsula.

 

 

Figure 2. The related obtained patents of the SEC, NTNU

 

Research plan

To bring a clear picture of eLISEs establishment and development. The objectives of the cooperative events of eTISE will be: (1) to develop an interdisciplinary, innovative e-Learning and STEM curriculum, implemented in both Taiwan and Vietnam, (2) to coordinate sub-projects to ensure the effectiveness of the innovative e-Learning and STEM curriculum, and (3) to cultivate talents in innovative e-Leaming and STEM education fields. The academic researches and classroom practicing in eTISE will encompass the following four subprojects: (f ) Curriculum and Instruction- using innovative technology to facilitate innovative e-Leaming and STEM teaching, (2) Innovative assessment for the next generation- establishing innovative e-Leaming and STEM standards, and assessing students’ competencies (3) Teacher Education- developing teachers’ capacity of innovative e-Leaming and STEM teaching. (4) To research and analyze Taiwan’s and Vietnam’s government policy toward innovative e-Leaming, and STEM education and (5) To advise on planning and strategy to the Ministry of Education of Taiwan and Vietnam and cany out research projects funded by the Ministry of Education and Ministry of Science and Technology in Taiwan. Presented below, in the following figure 3, is the researching and practicing framework of eTISE.

 

Figure 3. The researching and practicing framework of eLISE

 

 

The Administrative and industry-academic cooperation Group will be responsible for the related administrative matters of hosting workshops, communicating with local schools and industries. The Administrative and industry-academic cooperation Group will include two full-time research assistants and numbers part-time research assistants in Taiwan and two full-time research assistants and numbers part-time research assistant in Vietnam.

The regular meetings and research mobility from two sides will be held in order to cooperate on the cross-countries academic events and exchanging the local researching and practicing experiences.

 

Figure 4. The organization of the eLISE center

 

By taking advantage of the eLISE centeris location (HNUE is only 2 km away from the Science and Technology Division Taipei Economic and Cultural Office in Hanoi, Vietnam), we will work closely with the office to connect with major universities in Vietnam. The eLISE will play a connector to introduce Taiwans achievements in science and technology and help 26 expose and promote existing science and education media of the Ministry of Science and Technology of Taiwan to Vietnam.

 

Expecting results

For the first mission of the eLISE project, we focus on developing innovative e-Learning and STEM curriculum modules, assessments, and teaching material_to further the project phase. Based on SEC, NTNU’s previous research results and experiences, the development of the curriculum modules, assessments, and teaching material will include multiple elements, including (1) Curriculum design framework: Integrated STEM Instructional Module (6E Learning by DeSIGN™ Model), Question-Driven Instruction (QDI) model, (2) Teaching materials of XR (Augmented reality, Virtual reality, or Mixed reality) and Social scientific issue game-based learning materials, and (3) the Assessment tool: ICT platform: CloudClassRoom.

The first version of all innovative e-Learning and STEM curriculum modules, assessments, teaching material was developed in English and then translated into Vietnamese. The researching mobility will also focus on a two-side research team’s academic meeting to research and analyze Taiwan’s and Vietnams government policy toward innovative e-Learning and STEM education to bring a comprehensive context for developing the localizing innovative e-Learning and STEM curriculum modules, assessments, and teaching materials.

For the curriculum design framework of the e-Leaming and STEM modules development, the Integrated STEM Instructional Module follows the 6E Learning by DeSIGN™ Model, and the Question-Driven Instruction (QDI) model will be used.

By using the 6E model, students will engage in exploring, explaining, engineering, enriching, and evaluating their solutions to context-based real-world STEM issues, which required an application of mathematical and scientific concepts. Mathematical modeling was incorporated while students were designing, testing, and revising their solutions to context-based real-world problems.

In QDI, the teacher continually poses questions to challenge students. By answering the questions and discussing answers with peers, students are engaged in constructing, testing, and refining their knowledge. The iterative process of question-answering forms a tight feedback loop to help the teacher monitor and guide students’ performance over time in multiple problem-solving situations. The teaching effectiveness of QDI is validated by numerous studies conducted in western classrooms[14]. QDI is an ideal starting point to design useful methods to implement our STEM module. The process of QDI is well aligned with the research-based principles of how people learn. QDI is assessment-centered, leveraging a tight assessment-and-feedback loop to stimulate students’ metacognition to monitor, revise, and continually improve their learning. It also helps teachers diagnose and address students’ learning difficulties and needs in a timely manner.

Cuadro de texto: 27Moreover, QDI is both learner and knowledge-centered. If appropriately used, QDI is a highly efficient method for teachers to gauge students’ changes in knowledge, skills, attitudes, beliefs, and cultural practices. Thus, based on QDI information, teachers can adjust and reorganize instructional activities more effectively. QDI is also community-centered, requiring students to solve problems together and prompting a sense of affiliation and community in the classroom. Context-based problems can be used in QDI to motivate and guide students’ inquiries into context-based real-world STEM issues. The process of QDI implementation documented in the literature serves as a concrete template for us to develop modules. QDI is easy for teachers to employ, extend, and modify in their regular school programs. Engineering design and mathematical e-modeling can be inserted between each context-based problem, as follow-up activities to test students’ solutions to problems empirically, and to make Engineering and Technology more apparent.

For teaching materials of the e-Learning and STEM modules development, XR (Augmented reality, Virtual reality, or Mixed reality) and Social scientific issue game-based learning materials.

As technology continues to advance, people worldwide are eager to take advantage of its rapid development. With the help of the media, more people are becoming interested in Augmented Reality (AR) and Virtual Reality (VR) or Mixed Reality (MR). AR is defined as “a real-world context that is dynamically overlaid with coherent location or context-sensitivevirtual information” (Klopfer & squire, 2008; p205). It carries the properties of combining real and augmented objects, running interactively in real times, and registering real and augmented objects with each other[15] XR technology can provide a simulated learning context for students to explore the situations and problems in the real world. Despite the challenges of the effectiveness and appeal of XR technology in education, learning, and training (ELT), XR technology is still considered to have great potential in this area of application. XR technology allows students to understand the tools and technologies they will face after entering the industry through simulation, reduce the gap between academics and industry, and accelerate the development of innovative technologies for XR. Bringing a “multiplier effect” can also reduce the cost of purchasing teaching tools and equipment. XR can make cross-disciplinary education integrating science, technology engineering, and math (STEM) more interesting, more attractive, and more productive. XR can create a more effective learning environment, enabling STEM to emphasize interactive, inquiry, and goal-oriented learning methods. Students will get a more engaging learning experience. The main reason is that XR can immerse students in the learning environment; virtual collaboration with international students can also be carried out through distance learning to reduce barriers to students’ access to world-class scientific expertise through cross-border collaboration.

According to SSI & STEM game-based E-learning material (SSGELM), there are various issues in our daily life affected by the social diversity and global communication, additional, due to the need of individual life and the development of a sustainable society, it is important in terms of issue education as well as issue learning.

The SSGELMs, designed by our group, provide the advantage of issue learning. First, the entertaining environment of a board game can improve students’ learning motivation, and the systemic rules of a game can assist students in constructing the concepts of issue. Second, the simulated social interaction in a game can help students transfer their learning experience got in the game to daily life. Third, the functions of 3R (VREMREAR)-assisted board games (e.g., Increasing the amount of information, presenting various representations) can model game structure precisely and promote the students’ inquiry abilities, such as information gathering, factor exploring, and action planning. Final, board games are easy to carry and operate, which can facilitate education promotion.

The SSGELMs can guild the student’s awareness of the issues in life, and analyze the contexts of the issue according to different perspectives and disciplines, additionally bringing up their problem-solving skills, responsible value, and dynamic behaviors. SSGELMs has English and Vietnamese language versions.

 

1. “SAVIOURS,” a board game of climate change adaptation issues

The game system, active learning, and inquiry learning: defining problems, analyzing risks, evaluating strategies, executing strategies, monitoring responses, and correcting actions.

The E-system: The simulation of disaster-causing factors, the interactions of disaster and disaster prevention: Augmented reality for exploring the cause and impact of a disaster. Finally, the disaster response system is developed for the response of disaster prevention strategies.

 

 

2. “Water Ark,” a board game of sustainable water resources management

The game system: Multiplayer social interaction learning: Public benefit, water distribution, conflict negotiation.

The E-system: The simulation of the interaction and resource consumption in a multipla¬yer society: Interactive response system for professional behavior and resource distribution.

 

 

3. “Electric current war,” a board game with electrochemistry and circuit physics

Cuadro de texto: 30

The game system: STEM learning: Electricity concept, battery detection, circuit connection, metal reactivity calculation.

The E-system: Arduino for detecting and exporting electric current; arithmetic exploration program for calculating metal reactivity.

(Playable situation) (Electrochemistry) (Circuit physics

Our group has also implemented teacher workshops to train the teachers to design the SSGELM with an open-source. The content of the workshop includes the idea and design process of SSGELM, the application by technology, and the teaching and evaluation of SSI or STEM education. For example, the application by technology are real-time effects (by Recorder app), information collection (by VR), inquiry disclosure (by AR), factor calculation (by Excel), real-time statistics (by CCR), environmental processing (by MR).

In order to develop an appropriate formative assessment tool for the e-Learning and STEM curriculum modules, the ICT platform called CloudClassRoom (CCR) will be used. CCR is a powerful formative assessment platform for our e-Learning and STEM curriculum module, which can record student’s learning achievements and perceptions in the STEM class. The basic idea of CCR is using mobile devices to enable every student in a real classroom to respond to the teacher’s questions in a prompt and fully collective manner. CCR is written in HTML 5.0 and works on all mobile devices (such as laptops, PDAs, smartphones, and tablets) without additional software or plug-ins installation. Once a teacher uses his or her devices to connect to CCR, he/she can pose various types of questions to the whole class. Students are then enabled to use their personal devices to respond with digits, text, pictures, animations, or even multimedia. By this means, silent students may feel it is safer or more comfortable to participate because there is no pressure to speak an answer out loud or raisetheir hands to volunteer to speak. Verbal students’ needs for interacting with teachers and peers may be met as well[16].

At present, CCR has combined with the Short Text Analysis Module to use natural language processing technology. CCR can automate key vocabulary retrieval, topic analysis, organization, and grouping for students’ text so that teachers can quickly understand the class students’ text short answer content and carry out appropriate teaching activities accordingly. CCR currently contains three different text Analysis Algorithms, including Hierarchical Agglomerative Clustering and Latent Semantic Analysis, as well as the Latent Dirichlet Allocation. CCR is now using the pre-training module provided by Google BERT. The deep two-way unsupervised AI learning method is used to obtain deeper concepts beyond students’ answers, and make more accurate grouping by using broad concept labels, which also enables teachers to interpret the general situation absorbed by students quickly.

Cuadro de texto: 31In 2019, CCR entered a new milestone. It developed the IRS game mode (Gamified Electronic Audience Response System, GEARS). Teachers can shorten or lengthen the permitted response time, provide rewards for students’ correct answers, and set available props. Students can use props that teachers open, such as: increasing the number of seconds in the time limit, deleting the choices in multiple-choice questions, watching the current class answer, and so on. There is also a “hero list” in the system, which will announce the scoring situation at any time, and let the first-place student express their thoughts to the class. CCR makes traditional classroom teaching easier, and the new game mode adds fun, anticipation, and excitement to activities. The CCR system allows teachers to make effective-in-class modifications to enrich both students’ learning interests and achievements. Results show that students can learn while fully engaging in entertainment as they play games. CCR be selected as “An Exemplar Institution” in the 2019 EDUCAUSE Horizon Report. CCR was selected from over 60 submissions from various institutions in the United States and internationally.

Using the idea of Question-Driven Instruction[17], each E phase of the 6E Integrated STEM Instructional Module will start with the teacher continually posing related questions on CCR to challenge students and raise students’ attentions to join the teaching-learning process. Following the activity of each E phase, students need to answer the designed formative assessment questions on CCR.

The second mission of eLISE is talent cultivation of innovative e-Learning and STEM education in Vietnam and Taiwan. _The talents and teacher-training workshops will be held in Vietnam and Taiwan both to present our first-phase results: innovative e-Learning and STEM curriculum modules, assessments, teaching material, and provide the training of the application and designing opportunities to the workshop participants. The participants will include university faculties, under and graduate students, school teachers both in Taiwan and Vietnam. Workshops will be designed to popularize the ideas of using innovative e-Learning and STEM curriculum modules, assessments, teaching material, and to equip the participantswith the skills needed in the incorporated course. The pilot test of our innovative e-Leaming and STEM curriculum modules, assessments, teaching materials will be applied to local classrooms in both Vietnam and Taiwan.

Also, the two-side visiting scholars, students, and staff exchanging will be essential. The talents exchanging and visiting and pilot studies will facilitate the workshop talents’ training and provide deeper observations and investigation of e-Learning and STEM education in Vietnam and Taiwan. By connecting with the resources of the International Doctoral Program in Integrative STEM Education of NTNU, which aim to assist countries around the world in training STEM innovators for the future of the technology industry the eLISE can further offer the opportunity to attract more e-learning and STEM students/talents from Vietnam to pursue academic degrees in Taiwan.

The third mission of eLISE combines the results of the first mission (innovative e-Learning and STEM curriculum modules, assessments, teaching material) and the second mission (talent cultivation of innovative e-Learning and STEM education). Full-implementation and evaluation of the e-Learning and STEM curriculum modules will be organized in local classrooms in Vietnam and Taiwan by the in/pre-teachers, which attend our workshop training. Empirical studies will be conducted to assess teachers’ perceptions and teaching effectiveness towards using the e-Learning and STEM curriculum modules. Teachers will be assessed by the experts on the effectiveness of their teachings with their self-developed e-Learning and STEM curriculum modules. Teachers’ perceptions of implementing Integrated STEM Instructional Module (6E Learning by DeSIGN™ Model, Question-Driven Instruction (QDI) model), Social scientific issue game-based learning materials, and CCR, XR technology into their teaching will be assessed through the quantitative and qualitative instrumentations. Encourage and facilitate in-service teachers with outstanding innovative e-Learning and STEM curriculum modules, assessments, teaching material to participate in related innovative teaching module design competitions.

The eLISE will play a connecting role to cooperate with science game-based companies, help to the commercialization of our innovative e-Learning and STEM curriculum modules, assessments, teaching material. Thus, further to promote industry-academia cooperation and technological development in innovative e-Learning and STEM education fields in Taiwan and Vietnam.

The goals of the first year of the eLISE project will focus on (1) the translation of the innovative e-Learning and STEM curriculum modules, assessments, teaching material from SEC, NTNU in English version and Vietnamese version, (2) the workshops of cultivation of innovative e-Learning and STEM technical talents and (3) the research mobility and local classroom observations investigating the similarity and difference of Taiwan’s and Vietnam’s government policy and local classroom needing of innovative e-Learning and STEM education.

Based on the first year’s results, the eLISE will place emphases on (1) the developing of Vietnamese localized innovative e-Learning and STEM curriculum modules, assessments, (2) the holding the STEM teacher training workshops, and (3) actual classroom operations of Vietnamese localized innovative e-Learning and STEM curriculum modules, assessments in Vietnam schools in the second year.

The eLISE will extend our effectiveness to industry-school cooperation in the final year of the project. In this phase, the eLISE will hold the international academic conferences/ forums and investment fairs to promote, discuss, and further advise our project results toward both the academic community and commercial companies. Also, the core idea of research mobility in the final year will be investigating how to modularize and commercialize our innovative e-Learning and STEM curriculum modules and assessments, teaching material with Vietnamese commercial companies in order to realize the technology transfer of academic research results successfully.

 

Conclusions

Based on the idea of “one thousand STEM teacher and ten thousand STEM minds” with eLISE, the (1) developing an innovative e-Learning and STEM curriculum modules, assessments, teaching material, (2) the STEM and e-Learning teachers and talents training workshops, (3) talents exchanging and (4) industry-academia cooperation and (5) e-learning and STEM modules implements in local classrooms in Taiwan and Vietnam, we believe the project of eLISE can cultivate future e-learning and STEM talents and workforces in achieving regional development and prosperity for both Taiwan and Vietnam. According to our future results and experiences of the eLISE, we can further advise on planning and strategy for the Ministry of Education and carry out research projects funded by the Ministry of Education and Ministry of Science and Technology in Taiwan and Vietnam.

Cuadro de texto: 33The leadership team of the eLISE center is divided into two levels: “director” and “task force (Research and Development Group and Administrative and industry-academic cooperation Group),” as figure 4. The directors of eLISE are Prof. Chun-Yen, Chang from SEC, NTNU, leading the Taiwan team, with Vietnams team led by Prof. Nguyen Van Hien from HNUE.

The main tasks of the Research and Development Group will be researching and development of (1) the innovative e-Learning and STEM curriculum modules, assessments, teaching materials, and (2) talents and teacher training workshop materials. Also, (3) data collection and analysis and academic publishing of our project results. The Research and Development Group will include one Postdoctoral fellow and several part-time research assistants in Taiwan and one Postdoctoral fellow or full-time research assistant and several part-time research assistants in Vietnam.

By using Vietnam as a bridgehead and cooperating with “Science and Technology Division in Hanoi, Vietnam, Ministry of Science and Technology” eLISE can further enter and impact the e-Learning and STEM education in Laos, Cambodia, Thailand, Indonesia, Malaysia, and Myanmar and across the entire Indo-China peninsula.

 

4. Future Directions of the eLISE

The eLISE project has already received the seed funding from NTNU. The HNUE has now started to apply for further project funding support from the Ministry of Education and Training (MoET) of Vietnam and the Ministry of Science and Technology (MoST) of Vietnam. In the future, the eLISE project can obtain independent funding from the MoET and the MoST of Vietnam. We believe that after this three-year cooperation of eLISE, the project results and experiences can further help EfNUE to operate the eLISE center independently. The SEC, NTNU, will be EfNUE’s long-term partner in the future to assist eLISE in the stable operation, research, and dissemination of the center’s results.

 

5. Conflicts of interest statement

The authours claimed we have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational

grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.

 

6. Acknowledgements

This project was supported by the Ministry of Science and Technology(MOST), Taiwan, by the number of grant 106-2511-S-003-050-MY3, AJoint Adventure in Science Education Research & Practice (STEM 2TV).

 

Reference

Ahn, W., Chu, H. E., Martin, S. N., Chien, Y. T., Jen, C. H., & Chang, C. Y, "Development of an instrument to examine Engagement and Participation in Classroom-Science (EPIC-S)". In 2016 International Conference of East-Asian Association for Science Education (EASE 2016): Tokyo, Japan.

Beatty, Ian D., William J. Leonard, William J. Gerace & Robert J. Dufresne"Question Driven Instruction: Teaching science (well) with an audience response system ".Audience Response Systems in Higher Education Applications and Cases, 0, 96-115 (2006): https://doi.org/10.4018/978-1-59140-947-2.ch007

Beck, Ulrich, (Germany: SAGE, 1992). Risk Society: Towards a New Modernity.

Bransford, John D. et al, (Washington, DC: National Academy Press, 2000). How people learn: Brain, mind, experience, and school.

Breiner, Jonathan M., Shelly Sheats Harkness, Carla. C Johnson & Catherine M. Koehler, "What Is STEM? A Discussion About Conceptions of STEM in Education and Partnerships" School Science and Mathematics, 112(1), (2012): 3-11 https://doi.Org/10.1111/j.1949-8594.2011.00109.x

Boonruang, Sasiwimori'A Stem education A new method of teaching science, technology, engineering and mathematics in an applied approach is being promoted by the IPST" Bangkok: 14 January, 2015. https://www.bangkokpost.com/tech/local-news/456725/a-stem-education

Burke, Barry N. "6E Learning by Design Model" Technology and Engineering Teacher, (2014): 14-19.

Bybee, Rodger W., (Virginia: National Sciende Teaching Association, 2013). The Case for STEM Education: Challenges and Opportunities. National Science Teachers Association - NSTA Press.

Bybee, Rodger W. (Washington DC: American Association for the Advancement of Science, 2010) What is STEM education? Science, 329(5995), 996 (2010). https://doi.org/10.1126/science.1194998

Caldwell, Jane E. "Clickers in the Large Classroom: Current Research and Best-Practice Tips", CBE-Life Sciences Education, 6(1 (2007): 9-20 https://doi.org/10.1187/cbe.06-12-0205

Cannady, Matthew A., Eric Greenwald & Kimberly N. Harris, "Problematizingthe STEM Pipeline Metaphor: Is the STEM Pipeline Metaphor Serving Our Students and the STEM Workforce?", Science Education, 98(3) (2014): 443-460. https://doi.org/10.1002/sce.21108

Council, British. (2016). STEM Education Programme-Vitnem. https://www.britishcouncil.vn/en/ programmes/education/science-innovation/newton-programme-vietnam/stem

Crouch, Catherine H., Eric Mazur, (Maryland: American Journal of Physics, 2001). Peer Instruction: Ten years of experience and results. American Journal of Physics, 69(9), (2001): 970-977. https://doi.org/10.1119/1.1374249

Chang, Chun-Yen, Chien-Hua Hsiao, James P. Barufaldi, "Preferred-actual learning environment ‘spaces' and earth science outcomes in Taiwan" Science Education, 90(3), (2006): 420-433. https://doi.org/10.1002/sce.20125

Chang, Chun-Yen, Yeh, T. K., Lin, C. Y, Chang, Y. H., & Chen, C. L. D, "The impact of congruency between preferred and actual learning environments on tenth graders" science literacy in Taiwan Journal of Science Education and Technology, 19(4), (2010): 332-340 https://doi.org/10.1007/s10956-010-9203-1

Chien, Y. T., Chang, Y. H., & Chang, C. Y, "Do we click in the right way? A meta-analytic review of clicker-integrated instruction. Educational Research Review", 17, (2016): 1-18 https://doi.Org/10.1016/j.edurev.2015.10.003

Cuadro de texto: 35Chung, Chih-Chao. "Analysis of the learning effectiveness of the STEAM-6E special course-a case study about the creative design of loT assistant devices for the elderly". Sustainability (Switzerland), 10(9), (2018): 1-16. https://doi.org/10.3390/su10093040

Deboer, George E., "Scientific Literacy: Another Look at Its Historical and Contemporary Meanings and Its Relationship to Science Education Reform" Journal of Research in Science Teaching, 37(6), (2000): 582-601. https://doi.org/10.1002/1098-2736(200008)37:6<582::aid-tea5>3.0.co;2-l

Deslauriers, Louis, Ellen Schelew & Carl Wieman, "Improved learning in a large-enrollment physics class" Science, 332(6031), (2011): 862-864. https://doi.org/10.1126/science.1201783

Dufresne, Robert J., William J. Gerace & William J. Leonard "Solving physics problems with multiple representations" The Physics Teacher, 35(5) (1997): 270-275 https://doi.Org/10.1119/1.2344681

Engineering and National Research Council, National Academy of (Washington, DC: The National Academies Press, 2014). STEM integration in K-12 Education: Status, prospects, and an agenda for research.

English, Lyn. D, "STEM education K-12: perspectives on integration", International Journal of STEM Education, 3(1), 3 (2016). https://doi.org/10.1186/s40594-016-0036-1

Freeman, Brigid, Simon Marginson & Russell Tyler. (Eds.). (UK: Taylor and Francis Group, 2014). The age of STEM: Educational policy and practice across the world in science, technology, engineering and mathematics. New York: Routledge.

Jolly, Anne. "Six characteristics of a great STEM lesson" Education Week, 2014. https://www.edweek.org/tm/articles/2014/06/17/ctq Jolly_stem.html

Jen, C. H., Chien, Y. T, Martin, S. N., Chu, H. E., & Chang, C. Y, Student participation and perception of social environment in the science classroom. In The European Science Education Research Association (ESERA) 2017 International Conference. Dublin, Ireland.

Kelley, Todd R. & J. Geoff Knowles, "A conceptual frameworkfor integrated STEM education" International Journal of STEM Education, 3(1), 11 (2016). https://doi.org/10.1186/s40594-016-0046-z

Kolstp, Stein D. "Socioscientific Issues" Science Education, 85(3), (2001): 291-310. https://doi.org/10.1002/ sce.1011

Lee, Min-Hsien, Ching Sing Chai & Huang-YaoHong, "STEM Education in Asia Pacific: Challenges and Development" Asia-Pacific Education Researcher, 28(1), 1-4 (2019): 1-4 https://doi.org/10.1007/S40299-018-0424-z

Lucas, Bill, Janet Hanson & Guy Claxton, (London, UK: Royal Academy of Engineering, 2014). Thinking like an engineer: Implications for the education system.

LX, Quang, Hoang, L.H., Chuan, V., Nam, N., Anh, N. and Nhung, V.T "Integrated Science, Technology, Engineering and Mathematics (STEM) Education through Active Experience of Designing Technical Toys in Vietnamese Schools" British Journal of Education, Society & Behavioural Science, 11(2), 1-12 (2015). https://doi.org/10.9734/bjesbs/2015/19429

M.-H, Hsu. (2016). From knowledge to practice-Analysis of science education implementation plan (In Chinese). Taipei.

Miller, Jon D."Scientific Literacy : A Conceptual and Empirical Review" Daedalus, 112(2), (1983) 29-48.

Ministry of Education, Taiwan. The distribution of the new inhabitants children in elementary and middle school (in Chinese). (Taipei, 2017).

OECD, PISA2015 results (Volume l): Excellence and equity in education. (Paris, France: OECD Publishing, 2016)

Office of the Chief Scientist, Benchmarking Australian science, technology, engineering and mathematics. (Canberra, Australia: Australian Government, 2014)

Office of Trade Negotiations, executive Yuan, Bureau of Foreign Trade, T. M. of E. A. New Southbound Policy Guidelines and Action Plan. (Taipei, 2017).

Paez, Tobias Martin, David Aguilera, Francisco Javier Perales -Palacios & José Miguel Vílchez-Gónzales (2019). "What are we talking about when we talk about STEM education? A review of literature" Science Education, 103(1), (2019): 799-822 https://doi.org/10.1002/sce.21522

Painprasert, Niramis. "Factors Supporting the STEM Education Learning Management of Leader Teachers in the STEM Education Network of Thailand” In the Twelfht International Conferece on eLearning for Knowledge-Based Society (2015): 36.1-36.6.

Royal Society Science Policy Centre, Vision for science and mathematics education. (London, UK: The Royal Society, 2014).

Roberts, Amanda "A justification for STEM education", Technology and Engineering Teacher, 72(8), (2012): 1-5.

Smith, Karl A., Tamara J. Moore, "Advancing the State of the Art of STEM Integration" Journal of STEM Education, 15(1), (2014): 5-10.

Smith, Michelle, William B. Wood, Wendy Adams & C.E Wieman "Why peer discussion improves student performance on in-class concept questions", Science, 323(5910), (2009): 122-124. https://doi.org/10.1016/j.ydbio.2009.05.104

Srikoom, Wachira, Chatree Faikhamta & Deborah L. Hanuscin, "Dimensions of Effective STEM Integrated Teaching Practice", K-12 STEM Education, 4(2), (2018): 313-330.

Taiwan Ministry of Education, 12 Basic Education Curriculum Guideline: Natural sciences, (Taipei, 2018).

Tips", CBE-Life Sciences Education, 6(1 (2007): 9-20 https://doi.org/10.1187/cbe.06-12-0205

Vasquez, Jo Anne. "STEM—Beyond the Acronym" Educational Leadership, 72(4), (2015): 10-15.13.

Cómo citar: Chun-Yen Chang, Pei-Ling Lin y Nguyễn Thị Tố Khuyên, “e-Learning Integrated STEM Education Center (eLISE) in Asia: A Reflection Case Study of Taiwan and Vietnam Research Project” Revista Historia de la Educación Latinoamericana. Vol . 2 3 no. 36 (2021).



[1]  Science education scholar in Taiwan. Currently, he serves as Chair Professor In the National Taiwan Normal University (NTNU), Director of the Science Education Center (NTNU), Professor of the Graduate Institute of Science Education and the Department of Earth Sciences (NTNU). Over the past few years, he has been a Visiting Professor at the Taipei Medical University, The Education University of Hong Kong, and the Paris 8 University. He is Editor-in-chief of the Eurasia Journal of Mathematics, Science and Technology Education; Co-editor of the International Journal of Educational Methodology and Editorial Board Member of the Studies in Science Education, Learning, Media & Technology, & Journal of Science Education and Technology. changcy@ ntnu.edu.tw

[2]                                         Researcher, Science Education Center, National Taiwan Normal University peilinglinOI 24@gmail.com

[3]                                         Ph.D. student at the Graduate Institute of Science Education of National Taiwan Normal University She holds a bachelor's and master's certification from the Faculty of Physics at Hanoi National University of Education. Her Master thesis was related to teaching and learning STEM, namely developing STEM activities on Nano Technology for lower secondary pupils. Currently her research areas focus on teacher professional development in STEM education, ntkhuyen.aliza.231 @ gmail.com

[4]   Office of Trade Negotiations, executive Yuan, Bureau of Foreign Trade, T. M. of E. A. New Southbound Policy Guidelines and Action Plan. (Taipei, 2017).

[5]                       Taiwan Ministry of Education, The distribution of the new inhabitants children in elementary and middle school (In Chinese). (Taipei, 2017).

[6]                       Rodger W, Bybee, (Washington DC: American Association for the Advancement of Science, 2010) What is STEM education? Science, 329(5995), 996 (2010). https://doi.org/10.1126/science.1194998; Todd R. Kelley & J. Geoff Knowles, “A conceptual framework for integrated STEM education" International Journal of STEM Education, 3(1), 11 (2016). https://doi.org/10.1186/s40594-016-0046-z; Wachira Srikoom, Chatree Faikhamta & Deborah L. Hanuscin, "Dimensions of Effective STEM Integrated Teaching Practice", K-12 STEM Education, 4(2), (2018): 313-330. https:// doi.org/10.14456/k12stemed.2018.4; Jo Anne Vasquez, “STEM--Beyond the Acronym" Educational Leadership, 72(4), (2015): 10-15.

[7] Jonathan M. Breiner, Shelly Sheats Harkness, Carla. C Johnson & Catherine M. Koehler, “What Is STEM? A Discussion About Conceptions of STEM in Education and Partnerships" School Science and Mathematics, 112(1), 3-11 (2012) https://doi.org/10.11 n/j.1949-8594.2011.00109.x; Rodger W, Bybee, (Virginia: National Science Teaching Association, 2013). The Case for STEM Education: Challenges and Opportunities. National Science Teachers Association - NSTA Press.

[8] Top Universities in Vietnam (Vietnam, 2021) https://www.4icu.org/vn/.  

[9]   Sasiwimon Boonruang "A Stem education A new method of teaching science, technology, engineering and mathematics in an applied approach is being promoted by the IPST" Bangkok: 14 January, 2015. https://www.bangkokpost.com/tech/local-news/456725/a-stem-education; British Council. (2016). STEM Education Programme-Vitnem https:// www.britishcouncil.vn/en/programmes/education/science-innovation/newton-programme-vietnam/stem;      Hsu, M.-H. (2016). From knowledge to practice-Analysis of science education implementation plan (In Chinese). Taipei; Niramis Painprasert, “Factors Supporting the STEM Education Learning Management of Leader Teachers in the STEM Education Network of Thailand" In The Twelfth International Conference on eLearning for Knowledge-Based Society (2015): 36.1-36.6; Quang, L.X., Hoang, L.H., Chuan, V, Nam, N., Anh, N. and Nhung, V.T “Integrated Science, Technology, Engineering and Mathematics (STEM) Education through Active Experience of Designing Technical Toys in Vietnamese Schools" British Journal of Education, Society & Behavioral Science, 11(2), 1-12 (2015). https://doi. org/10.9734/bjesbs/2015/19429.

[10]        Ahn, W., Chu, H. E., Martin, S. N., Chien, Y. T., Jen, C. H., & Chang, C. Y, “Development of an instrument to examine Engagement and Participation in Classroom-Science (EPIC-S)". In 2016 International Conference of East-Asian Association for Science Education (EASE 2016): Tokyo, Japan; Chien, Y. T., Chang, Y H., &Chang, C. Y, “Do we click in the right way? A meta-analytic review of clicker-integrated instruction. Educational Research Review", 17, (2016): 1-18 https://doi.Org/10.1016/j.edurev.2015.10.003; Jen, C. H., Chien, Y T., Martin, S. N., Chu, H. E., & Chang, C.

Y, Student participation and perception of social environment in the science classroom. In The European Science Education Research Association (ESERA) 2017 International Conference. Dublin, Ireland.

[11] Ibid.

[12]        Ahn, W., Chu, H. E., Martin, S. N., Chien, Y. T., Jen, C. H., & Chang, C. Y, “Development of an instrument to examine Engagement and Participation in Classroom-Science (EPIC-S)". In 2016 International Conference of East-Asian Association for Science Education (EASE 2016): Tokyo, Japan; Chien, Y. T., Chang, Y H., &Chang, C. Y, “Do we click in the right way? A meta-analytic review of clicker-integrated instruction. Educational Research Review", 17, (2016): 1-18 https://doi.Org/10.1016/j.edurev.2015.10.003; Jen, C. H., Chien, Y T., Martin, S. N., Chu, H. E., & Chang, C. Y, Student participation and perception of social environment in the science classroom. In The European Science Education Research Association (ESERA) 2017 International Conference. Dublin, Ireland.

13 Chun-Yen Chang, Chien-Hua Hsiao, James P. Barufaldi, “Preferred-actual learning environment ‘spaces’ and earth science outcomes in Taiwan” Science Education, 90(3), (2006): 420–433. https://doi.org/10.1002/sce.20125; Chang, C. Y., Yeh, T. K., Lin, C. Y., Chang, Y. H., &Chen, C. L. D, “The impact of congruency between preferred and actual learning environments on tenth graders” science literacy in Taiwan. Journal of Science Education and Technology, 19(4), (2010): 332–340. https://doi.org/10.1007/s10956-010-9203-1

[14]             Beatty , Ian D., William J. Leonard, William J. Gerace & Robert J. Dufresne“Question Driven Instruction: Teaching science (well) with an audience response system". Audience Response Systems in Higher Education Applications and Cases, 0, 96-11 5 (2006): https://doi.org/10.4018/978-1-59140-947-2.ch007; Jane E. Caldwell, “Clickers in the Large Classroom: Current Research and Best-Practice Tips", CBE-Life Sciences Education, 6(1 (2007): 9-20 https:// doi.org/10.1187/cbe.06-12-0205

[15]             Ronald Azuma, Yohan Baillot, Reinhold Behringer, Steven Feiner “Recent advances in augmented reality" IEEE Computer Graphics and Applications, 21 (06) (2001): 34 - 47

[16]        Chien, Y. T, Chang, Y H., & Chang, C. Y, "Do we click in the right way? A meta-analytic review of clicker-integrated instruction. Educational Research Review" 17, (2016): 1-18 https://doi.Org/10.1016/j.edurev.2015.10.003

[17] Beatty , Ian D., William J. Leonard, William J. Gerace & Robert J. Dufresne"Question Driven Instruction: Teaching science (well) with an audience response system". Audience Response Systems in Higher Education Applications and Cases, 0, 96-115 (2006): https://doi.org/10.4018/978-1-59140-947-2.ch007