  

Media-Assisted Learning Pedagogical Evaluation of CONNECT Runs Franz X. Bogner University of Bayreuth Centre of Math & Science Education

Digital Media in Science Education

Our evaluation is designed to address the following research questions: * Does the CONNECT technology - platform and augmented reality (AR) - add value to the field trip experience? [Is an augmented field trip experience that is supported by a specific internet platform superior to a similar field trip experience that does not make use of advanced technology to support the link between classrooms and science centres?] * Do the educational scenarios provide added value for teachers and students, and if so, in what ways? [Is CONNECT superior to traditional classroom teaching and if, under what circumstances?]

Classroom alone – Similar Curriculum, No Museum Visit Museum Visit w/ Pre & Post Activities (No Platform/AR) No Connect Scenarios Museum Visit w/ Pre & Post Activities (with Platform/AR) Connect Scenarios A quasi-experimental design, including 3 main treatment groups 1) This group serves as a control group, consisting of classes that study the content material without the museum visit. 2) This group consists of classes who visit the museum and complete pre- and post-visit activities, but all events are completed without the use of the CONNECT technology (the platform & AR) 3) This group consists of classes who visit the museum and complete pre- and post-visit activities making use of the CONNECT technology (the platform & AR) 1 2 3

Classroom alone – Similar Curriculum, No Museum Visit Museum Visit w/ Pre & Post Activities (No Platform/AR) No Connect Scenarios Museum Visit w/ Pre & Post Activities (with Platform/AR) Connect Scenarios How these treatment groups test the hypothesis: * The comparison of groups 2 & 3 will test the first hypothesis- added value of the technology. * The comparison of group 1 to group 2 will test the 2nd hypothesis- added value of the educational scenarios

Methodology & Instruments Quantitative methodologies: * Pre & Post Questionnaires Audiences There are 2 primary audiences: teachers and students be monitored. Qualitative methodologies: * Selected Testing Run Observations * Focus Groups * Open-ended questionnaire questions * Possible video analysis

Teachers What assessed?MethodologyInstrumentWhen AttitudesQuestionnaire AT (to be developed), Likert type scales [15]All Testing Runs a. field tripQuestionnaire All Testing Runs b. museumsQuestionnaire All Testing Runs c. IST, ARQuestionnaire All Testing Runs MotivationQuestionnaire3 dimensions [30]All Testing Runs Postgrading the visitQuestionnaire [1]All Testing Runs Feedback by teacherQuestionnaire Visit Assessment Tool/PNIQ (includes assessment of platform contribution) (10’)All Testing Runs Overall changeFocus Groups Interview questions (includes assessment of platform contribution) Testing Runs 1 & 3

Students What assessed?MethodologyInstrument (# of questions)When background variablesQuestionnaireRavens (25) All Testing Runs CognitiveQuestionnaire Knowledge and skills test (exhibit dependant, developed from scenarios, TIMSS items) (15) All Testing Runs AttitudesQuestionnaire AT (to be developed), Likert type scales (15) All Testing Runs a. field tripQuestionnaire All Testing Runs b. museumsQuestionnaire All Testing Runs c. IST, ARQuestionnaire All Testing Runs MotivationQuestionnaire3 dimensions (30) All Testing Runs Postgrading the visitQuestionnaire1 item All Testing Runs Overall changePortfolios All testing runs

Goals and Targets

Study Design

Achievement Gain Wilcoxon: Z = -9.0 p< 0.001

Achievement Gain (Biology) Wilcoxon: Z = -7.5 p< 0.001

Achievement Gain (Subject Integrated) Wilcoxon: Z = -7.9 p< 0.001

Learning in Dyads (= Pair of Pupils)

homogeneous vs. heterogeneous dyads homogeneous: difference of teacher assessment < 1 Md = 6 Md = 7 heterogeneous: difference of teacher assessment > 1

assessment gain per dyad

There is … … no relationship between the cognitive ability (within a dyad) and the individual learning gain. … no relationship between the dyad consistence and the individual learning gain. … a relationship between the individual learning gain within a dyad.

Intrinsic Motivation Inventory (IMI) ( Deci & Ryan 2001) Interest/Enjoyment Perceived Competence Effort/Importance Pressure/Tension Perceived Choice Value/Usefulness

Interest/Enjoyment I enjoyed doing this activity very much This activity was fun to do. I thought this was a boring activity.(R) This activity did not hold my attention at all.(R) I would describe this activity as very interesting. I thought this activity was quite enjoyable. While I was doing this activity, I was thinking about how much I enjoyed it. Perceived Competence I think I am pretty good at this activity. I think I did pretty well at this activity, compared to other students. After working at this activity for awhile, I felt pretty competent. I am satisfied with my performance at this task. I was pretty skilled at this activity. This was an activity that I couldn’t do very well.(R) Effort/Importance I put a lot of effort into this. I didn’t try very hard to do well at this activity. (R) I tried very hard on this activity. It was important to me to do well at this task. I didn’t put much energy into this.(R)

Pressure/Tension I did not feel nervous at all while doing this. (R) I felt very tense while doing this activity. I was very relaxed in doing these.(R) I was anxious while working on this task. I felt pressured while doing these. Perceived Choice I believe I had some choice about doing this activity. I felt like it was not my own choice to do this task.(R) I didn’t really have a choice about doing this task.(R) I felt like I had to do this.(R) I did this activity because I had no choice.(R) I did this activity because I wanted to. I did this activity because I had to.(R) Value/Usefulness I believe this activity could be of some value to me. I think that doing this activity is useful for _____________________ I think this is important to do because it can ____________________ I would be willing to do this again because it has some value to me. I think doing this activity could help me to _____________________ I believe doing this activity could be beneficial to me. I think this is an important activity.

***p< 0,001; **p< 0,01 Results Motivation

