1 Advanced Educational Interfaces: Teaching and Learning with Augmented Reality Brett E. Shelton Ph.C. Educational Technology College of Education University of Washington PETTT – Program for Educational Transformation Through Technology

2 Presentation Overview  The Problem  Research Overview Focus Methods  Results Learning Assessment Interaction Analysis  Discussion of Trends  Future Directions  Doctoral Consortium Issue – Video Ex.

3 The Problem  Traditional instruction uses many different representations, always involving a significant level of abstraction  Interface constraints Students have difficulty accommodating spatially related knowledge Instructors are challenged to represent spatial phenomena

4 The Augmented Reality (AR) Interface  Interaction with the real environment  Augmented with virtual 3D objects  Exist in 3D space  Consists of: Display Mode of manipulation for 3D objects Accompanying HW and SW

5 Research Overview  Building on recent work in AR, cognition, vision science, educational technology  Examine AR for viewing and manipulating 3D objects for classroom-based curriculum  What is happening?  In what ways do students’ understandings change?  Resources?  Interface design  Cognitive advantages

6 Research Focus  Introductory physical science (Geography, Astronomy, etc.) Working knowledge of physical landscape and environmental processes Exist at different spatial scales Essential for more sophisticated concepts  Ex. A Private Universe film Earth closest to sun –> summer

7 Phase 1 Methods  Introductory Geography at a 4-year university 34 students, summer to 6 animated models of earth, sun, supporting models and annotations Replaced traditional instruction covering this material

8 Model #2

9

10 Model #5

11 Phase 1 Methods: Procedure and Sample  Students participated one at a time over a 2 day period  Pre-assessment and post-assessment worksheets 3 questions, 1 per topic Written and sketch explanation Assessed on a 0-5, novice-expert scale  Students received participation grades, examination the following week

12 Phase 1 Methods: Data Gathering  Pre-assessment, intervention, post- assessment  Intervention: Subjects given a brief introduction of the interface and how to manipulate virtual objects View each model, ask questions, answer questions  Sessions were audio/videotaped from 2 perspectives

13 3 rd Person Perspective

14 1 st Person Perspective

15 Assessment of Learning with AR  How did students’ performance change from pre-assessment to post-assessment?  Which students improved and which did not?  Which topics was student performance affected?

16 Phase 1 Results: Assessment Sheets  General improvement of conceptual and factual understanding  Misrepresentation of factual information was greatly reduced

17 Phase 1 Results: Interaction analysis  Short adjustment time to interface familiarity  “Explorers” were more comfortable with content, “Watchers” less, regardless of interface familiarity

18 Results – Interaction analysis  Physical inspection of the content was a key to understanding how multiple elements worked together Able to switch their attention to different things happening in the same model Either moving the model, or depending on the chosen angle, switching their area of focus

19 Results – Interaction analysis  Control over content Students had difficulty in articulating this as a factor Use a “diagram” in a way they could control what they were looking at

20 Discussion of Trends  Getting beyond abstraction No need for orange-earth  AR may provide more powerful learning opportunities We can represent 3D objects as they exist in relative 3D space for learning  Augmented reality allowed students to view and interact with sophisticated phenomena in real- world setting Time, position, movements and angles In the classroom may benefit quality in curriculum involving complex 3D spatial phenomena and concepts

21 Future Directions (Phase 2: Oct-Nov 2002)  Interview data gathering for pre- and post-assessment  Videotape analysis of experts for assessment comparison  Methodologically tie student learning through their activity with AR  Help inform new studies of AR  Help inform AR design for use with specific learning applications

22 Issue Focus  Video analysis of augmented reality activity  Link visual behaviors with information acquisition 1 st and 3 rd perspectives together Tracing specific student action  After specific instruction  As an “overall” activity

23 Interaction Analysis (Jordan & Henderson, 1995)  Considers: Physical activity with interface Cognitive activity with content Verbal exchanges  Within (in addition to) IA How might I identify those resources within the interface the student is using? In what ways do these resources impact the cognitive processes involved in learning? How can I further trace student activity to derive meaningful insights about what and how students are learning?

24 Video Samples…

25 Issue Revisited  Further ideas in how I might… Link students’ visual behaviors with information acquisition Identify useful interface resources Link these resources to cognitive learning processes