1. Evaluation of Course Design and Student Comprehension in the International Learning Environment: A Panel Data Analysis Pavasajjanant Natcha Yuji Akematsu and Masatsugu Tsuji Graduate School of Applied Informatics University of Hyogo

2. Outline  Course design: three stages How the course was designed based on constructivism theory and Bloom’s taxonomy.  Evaluation of the course Using panel data analysis on students replies How students acquired knowledge? How smoothly they perceive course contents, instructional methods and media usage? Analyze how three stages of the course design achieve original objectives

3. Distance Learning Project Learning content MX Flash software Learning Methods - Understand content well - Can use learning media Computer-based system

4. Course Design Constructivism Bloom’s Taxonomy 1 st Recognition to acquire Basic facts and information 2 st Understanding to understand the basic use of the acquired knowledge 3 st Application to apply this knowledge in other situations 4 st Analysis to construct their own new knowledge based on knowledge and information obtained Collaborative Learning Traditional Lecture Self Learning Connecting new and old information Interrelating the learning style and learning achievements. Learning basic knowledge Learning by experience and being advised Learning by experience at the more explicit level Our three stages

5. Course design: three stages of the learning process Teaching stages Traditional lectureSelf-learning Collaborative learning Learning Objectives Understand the Basic concept of MX Flash Learn to use MX Flash to create animation by themselves Exchange idea and create a short animation story for presentation Learning content and activity -Basic usage of MX Flash -ICT media used as online text posted on the school website -Workshop in making a simple animation using MX Flash -Introduction on how to use WBT/BBS -Grouping to create a animation -Inter-group communication via BBS ICT MediaOn-line TextWBTBBS and TV phone Other mediaOral, Print-out

6. Homepage Sharing Learning materials & Web board

7. Sample of the WBT

8. Web board ： Thai Students

9. Web board ： Japanese Students

10. Evaluation: Panel Data Analysis

11. ① Comprehension Q1. Do you understand today’s class? ② Cognitive load Q2. Do you feel difficult in learning today’s content and Flash MX? Q3. Is today’s class activity difficult? ③ Motivation Q4. Do you want to learn more about Flash MX? Q5. Do you want to search for related information from the Internet or any other on-line sources? Students are asked to reply four degree. Learning Feedback and Evaluation

12. Estimation Method (Previous Study*) Multiple regression: how Comprehension was affected by Cognitive load & Motivation Traditional Lecture Comprehension: Dependent variable Cognitive load & Motivation: Independent variables regard to teaching contents teaching methods regard to teaching contents media and Information literacy Self Learning Collaborative Learning For each stage *Natcha, P., “Effectiveness of International Distance Learning on Computer Software in High Schools between Thailand and Japan,” Discussion Paper No. AIDP0903, Graduate School of Applied Informatics, University of Hyogo, 2009.

13. Dependent variable: Q1- ComprehensionJapanese StudentsThai Students Traditional lectureCoef.S.E. Coef.S.E. Q2 Cognitive load in teaching contents 0.060.07 0.220.06 *** Q3 Cognitive load in teaching methods 0.370.07 *** 0.310.06 *** Q4 Motivation towards teaching contents 0.190.06 *** 0.110.05 ** Q5 Motivation towards media and information literacy 0.050.04 0.090.04 ** Constant 1.500.20 *** 1.170.15 *** N=267, Adj. R2=0.29N=244, Adj. R2=0.42 Self-LearningCoef.S.E. Coef.S.E. Q2 Cognitive load in teaching contents 0.170.06 *** 0.320.06 *** Q3 Cognitive load in teaching methods 0.140.06 ** 0.250.06 *** Q4 Motivation towards teaching contents 0.280.05 *** 0.120.04 *** Q5 Motivation towards media and information literacy -0.070.03 ** 0.060.04 Constant 1.710.18 *** 0.830.12 *** N=377, Adj. R2=0.30N=237, Adj. R2=0.53 Collaborative LearningCoef.S.E. Coef.S.E. Q2 Cognitive load in teaching contents 0.220.07 *** 0.420.08 *** Q3 Cognitive load in teaching methods 0.110.07 * 0.300.07 *** Q4 Motivation towards teaching contents 0.360.05 *** 0.060.05 Q5 Motivation towards media and information literacy -0.030.04 0.060.05 Constant 1.430.20 *** 0.710.15 *** N=348, Adj. R2=0.26N=200, Adj. R2=0.51 ***1%, **5%, *10% of the significance level, respectively. Estimation Results (Previous Study*) *Natcha, P., “Effectiveness of International Distance Learning on Computer Software in High Schools between Thailand and Japan,” Discussion Paper No. AIDP0903, Graduate School of Applied Informatics, University of Hyogo, 2009.

14. Estimation Method Panel data approach: how Comprehension was affected by Cognitive load & Motivation and Comprehension in the previous stage Traditional Lecture Comprehension: Dependent variable Cognitive load & Motivation: Comprehension in the previous stage(s): Self Learning Collaborative Learning Enhancement of comprehension of the previous stage = evaluation of course design Independent variables

15. Effect of the learning styles Appropriateness of Course Design Dependent variable: Q1Comprehension Thai studentsJapanese students Traditional lectureCoef.S.E. Coef.S.E. Q2 Cognitive load in teaching contents0.230.07***0.050.08 Q3 Cognitive load in teaching methods0.250.07***0.340.08*** Q4 Motivation towards teaching contents0.070.04*0.220.06*** Q5 Motivation towards media and information literacy-0.030.020.04 Constant1.300.21***1.540.22*** Self-LearningCoef.S.E. Coef.S.E. Comprehension in Traditional lecture0.09-0.070.43-0.16*** Q2 Cognitive load in teaching contents0.440.06***0.020.08 Q3 Cognitive load in teaching methods0.150.06**0.150.07** Q4 Motivation towards teaching contents0.020.030.360.07*** Q5 Motivation towards media and information literacy0.010.02-0.040.03 Constant0.680.23***0.310.58 Collaborative LearningCoef.S.E. Coef.S.E. Comprehension in Traditional lecture0.260.10***0.340.24 Comprehension in Self-Learning0.050.100.610.31* Q2 Cognitive load in teaching contents0.440.06***0.000.16 Q3 Cognitive load in teaching methods0.090.060.300.16* Q4 Motivation towards teaching contents0.03 0.120.10 Q5 Motivation towards media and information literacy-0.020.02-0.080.08 Constant0.310.23 1.390.95 ***1%, **5%, *10% of the significance level, respectively.

16. Appropriateness of Course Design: Case of Thailand Traditional Lecture Self Learning Collaborative Learning 1 2 3 Thai students Prior learning stage Self-learningTraditional Self-learningn.a.1 Collaborative learning23***

17. Self Learning Collaborative Learning 1 2 3 Japanese students Prior learning stage Self-learningTraditional Self-learningn.a.1*** Collaborative learning2*3 Appropriateness of Course Design: Case of Japan Traditional Lecture

18. From the analysis (1) Each learning stage Thai students : Cognitive load ( Difficulty) is important Japanese student: Motivation is important (2) Corse Design This course design is more fitted to Japanese students than Thai Reasons for Thai students Language barrier Unfamiliarity with teaching methods Less experience with ICT learning environment Readiness to distance learning Less sufficient self-organization They failed to perform assign role of communication They depended much on teachers and TA Conclusions

19. -Combination of various instructional methods enhance effective learning in School Education The class should place a greater emphasis on two-way interactive communication and more real time. Suggestion: Using BBS >> more compulsory Interaction among teachers and students >> more real-time -role play, etc) Classroom discipline Class period is too short for self-learning and collaborative learning 50 minutes = moving to the computer classroom +actual learning time (10-15 min) (35-40 min) Suggestion: Shorter period of each class and longer term in the whole course Lesson learned from This Project

20. Thank you