Dr. Wendy L. Keeney-Kennicutt Presidential Professor for Teaching Excellence, Piper Professor of 2010 Associate Director, First Year Chemistry Program, TAMU Ms. Zahira Merchant Educational Psychology, TAMU 1 Using Virtual 3-D Technology to Improve Learners’ Spatial and Science Skills: Teaching VSEPR ® Theory in Second Life ® A STUDY

Premise 2 to investigate if Second Life can enhance to investigate if Second Life can enhance –learning of a chemistry concept: 3-D shapes of molecules (VSEPR* theory) –Spatial ability – translating 3-D images into 2-D pictures and back again * Valence Shell Electron Pair Repulsion Theory 

My Second Life Space on 12 th Man Island (maps.secondlife.com/secondlife/12th%20Man/215/237/26) My Second Life Space on 12 th Man Island (maps.secondlife.com/secondlife/12th%20Man/215/237/26) 3

Classroom Use 4 Classroom Area – for reviews and office hours Blackboard Movie Screen Streaming Desktop Clicker System Quiz System Seats that allow students to raise their hands Notice Board

TAMU Islands 5 Dr. K’s Chemistry Corner Aggie Orientation TAMU Faculty are using SL to do  Meetings  Simulations  HSC Nursing*  Ag Crisis Comm*  Ecosystems*  Student Talks*  Guest speakers  Tech Writing  Office hours & Reviews  Learning modules  Pre-Teacher training*  Architecture Projects  Streaming live events  Viz lab student films * Funded projects TAM Health Science Center (4 Islands) Agri- Culture TAMU- Galveston (2 Islands) Eco- system Communication Island - coming

Study Design Study Design Quasi-experimental pre-posttest control group research design study Quasi-experimental pre-posttest control group research design study –2 classes of Chem 101 students with same instructor (me) –~ 240 students in each class – all assessments with HW worth 40/700 pts. Experimental group: 3 activities in SL (6wks) Experimental group: 3 activities in SL (6wks) Control group: same 3 activities using two 2-D rotated images (SL screen shots) Control group: same 3 activities using two 2-D rotated images (SL screen shots) See blog: chemist-in-sl.blogspot.com See blog: chemist-in-sl.blogspot.com 6

Study Design Study Design Blog: chemist-in-sl.blogspot.com Blog: chemist-in-sl.blogspot.com 7

Pre-Testing Spatial Ability Spatial Ability –Card Rotations Test (2-D) from ETS: paper Educational Testing Service Educational Testing Service –Purdue Spatial Visualization Test (3-D): online Bodner, George M, and Roland B Guay. “The Purdue Visualization of Rotations Test.” The Chemical Educator 2.4 (1997) : Chemistry Content Chemistry Content –VSEPR theory: online Science logic Science logic – Test of Logical Thinking (TOLT): online 8

Pre-Testing On-line tests prepared by us with Qualtrics On-line tests prepared by us with Qualtrics PSVT (20 items, 10 min) TOLT (10 items, 38 min) PSVT (20 items, 10 min) TOLT (10 items, 38 min) 9 Free! Lots of research uses them!

Pre-Testing CRT CRT –20 items with 8 parts each –10 items per page, 3 min per page –Students must identify if each figure, when compared to the main figure on the left, is the  Same - S (rotated in the plane), or  Different – D (flipped over and rotated) 10

Pre-Testing VSEPR test – developed by me VSEPR test – developed by me –Tested on 3 faculty and ~100 past students –36 questions, 4 parts, 45 min (angle, content knowledge, shape id, chemical examples of shape) –Reliability: 0.90 (Cronbach’s alpha) based on post test data 11

Post-Testing Spatial Ability Spatial Ability –Card Rotations Test –Purdue Spatial Visualization Test Chemistry Content Chemistry Content –VSEPR theory on-line test Presence questionnaire + Demographics Presence questionnaire + Demographics –Included qualitative aspect HW & Laboratory module on VSEPR theory HW & Laboratory module on VSEPR theory Exam questions - same for both groups Exam questions - same for both groups 12

Intervention #1 The Molecule Game The Molecule Game Experimental Experimental –Develop SL skills beyond SL &TAMU orientations (inventory, chat, interacting with objects, taking photos) –Begin seeing molecules in 3-D Control Control –Answer same questions from SL screen shots (2 views) 13

Intervention #1 The Molecule Game Control Control –Students answer the same questions using SL screen shots (2 views) 14

Intervention #1 Feedback The Molecule Game 15 87%written directions were clear (N=71) 69%watched video & found it helpful (N=67) 86%MG is useful to learn SL skills (N=49) 85%MG ran smoothly (N=13)

The Chemist as Artist The Chemist as Artist Experimental Experimental –Develop more SL skills –Interact and draw 2 views each of 3molecules in sandbox with photos (link, move, copy, rotate) ⇒ ⇒ Control Control –Draw given 2 views of 3 molecules (5 different sets – screen shots) 16 Intervention #2

The Chemist as Artist The Chemist as Artist Control - Draw given 2 views of 3 molecules (5 different sets – screen shots) 17 Intervention #2

Intervention #3 Tower of VSEPR Experimental Experimental –Rez 11 molecules –Measure bond angles, determine geometries, Lewis dot structures, draw structures 18 Control Control –Do same homework from 2 views of each SL molecule (5 sets)

Intervention #3 Tower of VSEPR HW 19

Student Acceptance Student Issues  Solutions Student Issues  Solutions –On-line test compliance  gave pts/reminders – SL learning curve  videos/ppt in SL/RL – SL learning curve  videos/ppt in SL/RL –not virtual world savvy  SL/RL office hours –some think SL is creepy  TAMU Island 20

Before interventions, the control (C) and experimental (E) groups were the same as measured by: Before interventions, the control (C) and experimental (E) groups were the same as measured by: Preliminary Results 21 TEST C (N=137) Mean (SD) E (N=153) Mean (SD) TOLT (max=10)6.43 (2.10)6.14 (2.24) PVRT (max=20)12.36 (3.48)11.86 (3.70) VSEPR Content Test (max=45) 7.08 (3.56)7.39 (3.77)

The two groups were different on CRT pre-test but NOT post-test. The E group showed a small The two groups were different on CRT pre-test but NOT post-test. The E group showed a small but significantly larger gain. Preliminary Results 22 CRT (max=160) Control Mean (SD) Experimental Mean (SD) p Pretest107.9 (30.4)100.3 (27.9)0.027 (2-tailed) Posttest127.2 (29.5)123.8 (22.5)0.28 Gain (over 6 wks) 19.3 (20.4) p<0.001 Cohen’s d= (17.8) p<0.001 Cohen’s d= (1-tailed) Cohen’s d=0.22

PVRT – no significant differences were seen between the pretest and posttest for either group. PVRT – no significant differences were seen between the pretest and posttest for either group. Exam 3 score on VSEPR – no significant differences Exam 3 score on VSEPR – no significant differences VSEPR HW and Laboratory – no significant differences VSEPR HW and Laboratory – no significant differences VSEPR content test VSEPR content test –Overall, no significant differences were seen on posttest Control: ± 7.60 Experimental: ± 8.73 Preliminary Results 23

More on VSEPR test: On Part 1 (determining angle from drawing), the E group did significantly better (max=4) More on VSEPR test: On Part 1 (determining angle from drawing), the E group did significantly better (max=4) Preliminary Results 24 VSEPR Pt 1 (max=4) Control Mean (SD) Experimental Mean (SD) p Pretest1.01 (0.57)1.07 (0.47)0.26 (2-tailed) Posttest1.74 (1.20)2.01 (1.19)0.03 (1-tailed) Cohen’s d=0.24 Gain (over 6 wks) 0.74 (1.20) p=0.004 Cohen’s d= (1.14) p<0.001 Cohen’s d= (1-tailed) Cohen’s d=0.18

Even more on VSEPR test: If you include all pre/post students, the results are even more significant, BUT Even more on VSEPR test: If you include all pre/post students, the results are even more significant, BUT On average, students are recognizing <2/4 of angles On average, students are recognizing <2/4 of angles. Preliminary Results 25 VSEPR Pt 1 (max=4) Control Mean (SD) Experimental Mean (SD) p Pretest1.03 (0.58)1.04 (0.51)0.82 (2-tailed) Posttest1.55 (1.17)1.82 (1.19)0.019 (2-tailed) Cohen’s d=0.24 Gain (over 6 wks) 0.52 (1.17) P<0.001 Cohen’s d= (1.16) p<0.001 Cohen’s d= (2-tailed) Cohen’s d=0.22

Let’s look at the questions one by one. Let’s look at the questions one by one. Here is the Here is the question verbage: Preliminary Results 26

Gain-C (-0.04 ± 0.49; N=194) Gain-C (-0.04 ± 0.49; N=194) Gain-E (0.005 ± 0.44; N=209) Gain-E (0.005 ± 0.44; N=209) Question 1 of 4 27 p < 0.16 (1-tailed) Confidence–Pre C=74% E=75% Confidence–Post C=77% E=82%

Gain-C (0.25 ± 0.47) Gain-C (0.25 ± 0.47) Gain-E (0.33 ± 0.49) Gain-E (0.33 ± 0.49) Question 2 of 4 28 p < (1-tailed) Confidence–Pre C=53% E=72% Confidence–Post C=75% E=76% Confidence–Pre C=58% E=75% Confidence–Post C=63% E=60%

Gain-C (0.08± 0.44) Gain-C (0.08± 0.44) Gain-E (0.15 ± 0.49) Gain-E (0.15 ± 0.49) Question 3 of 4 29 p < (1-tailed) Confidence–Pre C=61% E=59% Confidence–Post C=65% E=62% Confidence–Pre C=46% E=57% Confidence–Post C=74% E=75%

Gain-C (0.23± 0.49) Gain-C (0.23± 0.49) Gain-E (0.29 ± 0.47) Gain-E (0.29 ± 0.47) Question 4 of 4 30 p < (1-tailed) Confidence–Pre C=34% E=42% Confidence–Post C=75% E=74% Confidence–Pre C=68% E=66% Confidence–Post C=70% E=68%

Disheartening!! Comments 31

We analyzed two questions posed to the Experimental group: We analyzed two questions posed to the Experimental group: 1. Is it a good idea to use Second Life in future chemistry classes? (Quantitative analysis) 2. Please explain your answer. (Qualitative analysis) More Preliminary Results 32

33 Is it a good idea to use Second Life in future classes? Is it a good idea to use Second Life in future classes? 23% 18% N = 219 Percentage

Please Explain Your Answer…. Main Theme - Perceived Usefulness Please Explain Your Answer…. Main Theme - Perceived Usefulness Percentage N = % 35% 7% Subthemes: 1. Visualization (38%) 2. Time Consumption (29%) 3. Interactivity (17%) 4. Other methods such as text books, 2-D images, paper pencil were just as useful (16%) 34 Other themes: Perceived ease of use & gaming experience

Results - Perceived Usefulness Results - Perceived Usefulness Comments 1. It helped a lot with the VSEPR theory by allowing me to visualize the molecules so when I’m drawing them on paper I can see the molecule in my mind’s eye. This allowed me to draw 3D molecule faster and easier. 2. I found that it took a lot of time to learn how to manipulate the molecules in Second Life. 3. It allows you actually see and interact with a molecule 4. I feel it would be easier to lecture about this material and show students more examples. 35

Future Work/Questions  Did my attempt to show the correct molecular shape by giving 2 views of a molecule to the control group skew the CRT results?  Do students taking a regular general chem class increase their CRT scores naturally?  If so, do they keep their higher CRT scores over time?  Can we really assume that students can see that this angle is 90 o, can see that this angle is 90 o, without training them? Not! without training them? Not! 36

Acknowledgements  Zahira and I thank all my students who had to endure all our assessments….  Thank you and any questions? 37