UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing The Potential of Games & Simulations for Learning and Assessment Richard Wainess: Presented Monday, January 22, 2007 at the 2007 CRESST Conference

2/30 Overview 1. Do games & simulations foster learning? 2. Issues affecting learning from games & simulations 3. Assessments using games & simulations 4. Embedding instructional methods into games & simulations 5. The bottom line

Do games and simulations foster learning?

4/30 The promise of game/sim-based learning General belief is games & simulations improve instruction Provide complex and diverse approaches to learning Provide learner control Can address both cognitive and affective needs Motivate learners

5/30 What the evidence shows Positive outcomes? Numerous knowledge outcomes are attributed to games and simulations Studies also warn about anecdotal and descriptive evidence (Leemkuil et al., 2003; Wolfe, 1997) Generalizable skills outcomes have been cited (Day et al. 2001; Green & Bavelier, 2003; Greenfield et al., 1994) Negative or null outcomes? Reviews and meta-analyses cite mixed or negative reviews (Dekkers & Donatti, 1981; Druckman, 1995) Positive attitude toward games (i.e., motivation) doesn’t necessarily equal learning (Brougere, 1999; Salas et al., 1998; Salomon, 1984)

6/30 What the evidence shows (cont’d) So, do games & simulations foster learning? Learning is related to Instructional Methods, not media (games/simulations, book, lectures, etc.) (de Jong & van Joolingen, 1998; Garris et al., 2002; Gredler, 1996; Leemkuil et al., 2003; Thiagarajan, 1998) NO!!!!

7/30 Implied model of games & simulations Increased Learning based on erroneous and/or unsupported research conclusions Games and simulations

8/30 True model of games & simulations Games and simulations Increased Learning? Studying Prior Knowl. Student Attitude Teacher Training Teacher Attitude Student Ability Instructional Components based on empirical evidence

Issues affecting learning from games and simulations

10/30 Cognitive load = burden placed on our working memory Affected by instructional components Number and nature of elements needing processing (Cooper, 1998; Paas et al., 2003) Message complexity, stimulus features (Daniels & Moore, 2000) Particular to computer media & interactivity Additional media-related cognitive demands (Daniels & Moore, 2000) Level of learner control and autonomy (Dias, Gomes, & Correia, 1999)

11/30 Task difficulty Relationship of task difficulty to mental effort is an inverted U (Clark, 1999; Malone & Lepper, 1987) Too easy = player gets bored Too hard = player disengages Optimal cognitive load = neither too hard nor too easy Effort Task difficulty Less More Enjoyment of a game/sim is linked mental effort Mental effort is linked to task difficulty

12/30 Sources of cognitive load Intrinsic Cognitive Load Germane Cognitive Load Extraneous Cognitive Load Learner Control

13/30 Sources of cognitive load Intrinsic Cognitive Load Load incurred by the to-be-learned material (Brunken et al., 2003; Paas et al., 2003) (i.e., metacognition, forming schema, chunking, automating) Germane Cognitive Load Cognitive load imposed by instruction; methods and strategies required to process intrinsic load (Renkl & Atkinson, 2003) (e.g., instruction, examples, reading, searching, problem solving, interface elements)

14/30 Sources of cognitive load Extraneous Cognitive Load Imposed by an unnecessary stimuli (e.g., interface artwork, extraneous sounds) (Brunken et al. 2003) Seductive details (Mayer et al., 2001; Schraw, 1998) Learner Control Pacing & navigation (Barab et al., 1999; Cutmore et al., 2000) Metacognitive processes (Clark, 2003; Daniels & Moore, 2000; Dillon & Gabbard, 1998) Goal setting and planning, monitoring, selecting, organizing, etc.)

15/30 Issues pertaining to games Extraneous details, split attention, learner control Tom Clancy’s Ghost Recon (Red Storm Entertainment, 2002)

16/30 Issues pertaining to simulations Extraneous & seductive details, split attention, learner control (cognitive load issues)

17/30 Issues pertaining to simulations Experiential/Discovery-based Learning (metacognitive and cognitive load issues) Developed by the University of Southern California’s Center for Cognitive Technology (USC/CCT) using iRidesAuthor

18/30 What’s bad about games/sims? Extraneous and Seductive details Learner Control Mindless manipulation Potential for unguided learning

19/30 What’s good about games/sims? Authentic tasks Personalization Presence/immersion Motivation (and persistence)

20/30 What game/sim elements are detrimental? Rich, engaging content Immersive environments Impressive sound tracks Spectacular effects Compelling fantasy Gives learner control Can promote mindless manipulation

21/30 What game/sim elements are beneficial? Rich, engaging content? Immersive environments? Impressive sound tracks? Spectacular effects? Compelling fantasy? Gives learner control? Opportunity to experiment? Potential for adaptive learning

Assessment using games and simulations

23/30 Computer-based assessments Requirements Ability of computer to assess and report performance Ability of computer to use outcomes to adapt learning Ability to assess meaningful learning Applying knowledge and skills Authentic task performance

24/30 Computer-based assessments (cont’d) Examples Retention assessments (assess memory) Fill-in-the-blank Knowledge maps Performance based assessments (assess problem solving, decision-making) Knowledge maps Fill-in-the-blank Process maps Authentic tasks

25/30 Embedding assessments Pre-assessments (diagnostic assessment) Adaptive instruction Assess learning gains In-course assessments (formative assessment) Signaling/Priming/Cueing Feedback/Monitoring Authentic tasks Post-assessments (summative assessment) Assess learning outcomes Provide predictions of future performance (transfer) Developed by the University of Southern California’s Center for Cognitive Technology (USC/CCT) using iRidesAuthor

Embedding Instructional Methods into Games and Simulations

27/30 Embedded Guidance Instruction Worked examples Practice opportunities With goals Signaling/cueing Other learning scaffolds (instructional methods/strategies) Developed by the University of Southern California’s Center for Cognitive Technology (USC/CCT) using iRidesAuthor

The bottom line

29/30 The bottom line Can we learn from games and simulations? Can games and simulations provide effective assessments? YES With proper instructional methods YES Particularly if they exploit the capabilities of computers

30/30