Richard Wainess Dissertation Proposal v.3Presented 04/27/04 The Effect of Navigation Maps on Problem Solving Tasks Instantiated in a Computer-Based Video Game Committee Members: Dr. Harold O’Neil (Chair) Dr. Richard Clark Dr. Edward Kazlauskas Dr. Janice Schafrik Dr. Yanis Yortsos (Outside Member) Richard Wainess Dissertation Proposal Presented to the Faculty of the Graduate School University of Southern California April 27, 2004 Slide 1 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Research Questions Will the problem solving performance of participants who use a navigation map in a 3-D, occluded, computer-based video game (i.e., SafeCracker®) be better than the problem solving performance of those who do not use the map (the control group)? Will the continued motivation of participants who use a navigation map in a 3-D, occluded, computer-based video game (i.e., SafeCracker®) be greater than the continued motivation of those who do not use the navigation map (the control group)? Slide 2 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Cognitive Load Theory Auditory/Verbal and Visual/Spatial Channels/Memory (Baddeley,1986; Mayer & Moreno, 2003) Limited Working Memory (Brunken et al., 2003) –7 +/- 2 (Miller, 1956) –Possibly only 2 or 3 novel elements (Paas et al., 2003) Unlimited Long-Term Memory (Mousavi et al., 1995) Cognitive Load = mental capacity imposed on working memory (Sweller & Chandler, 1994) –Controlled by schema development & Automation Words Pictures Prior Knowledge Selecting Words Working MemoryAudio/Visual Info Ears Eyes Sensory Memory Selecting Images Integrating Images SoundsVerbal Model Pictorial Model Organizing Words Organizing Images Mayer & Moreno, 2001 Slide 3 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Cognitive Load Theory (cont’d) Intrinsic Cognitive Load: The process of integrating new information with existing knowledge : e.g., working memory, long term memory, metacognition (Brunken et al., 2003; Paas et al., 2003). Germane Cognitive Load: The cognitive load required to process intrinsic load (Renkl & Atkinson, 2003). –Imposed by the environmental requirements related to learning (e.g., instruction, reading, searching, problem solving, interface elements) Extraneous Cognitive Load: Imposed by an unnecessary stimuli: e.g., interface artwork, extraneous sounds (Brunken et al. 2003). –Seductive Details (Mayer et al., 2001; Schraw, 1998). Learner Control: pacing & navigation (Barab et al., 1999; Cutmore et al., 2000) –Mixed or Negative Results (Bernard et al., 2003; Niemiec et al., 1996) Slide 4 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Games and Simulations Games: rules, constraints/privileges, imaginative, linear (Gredler, 1996) Simulation-Games: combination of games and simulations (Gredler, 1996) Motivation in games: fantasy, control & manipulation, challenge & complexity, curiosity, competition, feedback, fun Slide 5 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Games and Simulations Outcomes with Games and Simulations Positive outcomes: –Numerous knowledge outcomes attributed to games and simulations Warning about anecdotal and descriptive evaluations (Leemkuil et al., 2003; Wolfe, 1997) –Generalizable skills outcomes (Day et al. 2001; Green & Bavelier, 2003; Greenfield et al., 1994) Negative or null outcomes: –Reviews and meta-analyses cite mixed or negative reviews (Dekkers & Donatti, 1981; Druckman, 1995) –Positive attitude toward games doesn’t necessarily equal learning (Brougere, 1999; Salas et al., 1998; Salomon, 1984) Outcomes related to Instructional Design, not games/simulations (de Jong & van Joolingen, 1998; Garris et al., 2002; Gredler, 1996; Leemkuil et al., 2003; Thiagarajan, 1998) Slide 6 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Scaffolding (including Graphical Scaffolding) Instructional methods –Should keep cognitive load low (Clark, 2003) –External methods which replace internal processes (Clark, 2001) Scaffolding is an instructional method Scaffolding provides support during learning (Allen, 1997; Chalmers, 2003; van Merrienboer et al., 2002, 2003) Graphical Scaffolding –Includes maps and menus as advance organizers (Jones et al., 1995) –Maps supported by researchers as visual aids and organizers (Benbasat & Todd, 1993; Chou & Lin, 1998; Ruddle et al., 1999) –Should be used for visual tasks (Mayer et al., 2002) Slide 7 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Navigation Maps Navigation maps effective in 3-D, occluded, environments with simple problem solving tasks (Galimberti, 2001) Not yet examined Navigation in 3D, occluded, environments with complex problem solving tasks Slide 8 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Navigation Map Floor Plan of Mansion’s First Floor from SafeCracker® Slide 9 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Research Hypotheses Problem Solving –Hypothesis 1: Navigation maps will produce a significant increase in content understanding compared to the control group. –Hypothesis 2: Navigation maps will produce a significant increase in problem solving strategy retention compared to the control group. –Hypothesis 3: Navigation maps will produce a significant increase in problem solving strategy transfer compared to the control group. –Hypothesis 4: There will be no significant difference in self-regulation between the navigation map group and the control group. However, it is expected that higher levels of self-regulation will be associated with better performance. Motivation –Hypothesis 5: Navigation maps will produce a significantly greater amount of optional continued game play compared to the control group. Slide 10 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Method Design –Experimental: Random Assign to Treatment or Control Group –Treatment receives navigation map; Control Group doesn’t –Segregated group sessions: all Treatment or all Control Participants –60 males & females, ages 18-25, undergraduates at USC –No prior experience playing SafeCracker® Procedure ( 90 minutes ) –Demographic Information & Self-Regulation Questionnaire –Introduced to Knowledge Map –Handout Navigation Map to treatment group only –Introduced to the game ( interface, opening locks, finding objects ) –Pretest: Knowledge map Slide 11 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Method (cont’d) Procedure (cont’d) –Play game (first set of rooms) –Intermediate Test Knowledge Map; Retention/Transfer Questions; Task Check-List –Play Game (second set of rooms) –Post Test: Knowledge Map; Retention/Transfer Questions, Task Check-List –Debriefing –Optional playing time (up to 30 minutes) Measures –Problem Solving (O’Neil, 1999) Content Understanding Problem Solving Strategies Self Regulation –Motivation Verbalized desire to continue playing Amount of free play minutes Slide 12 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 key books crack room tool desk direction map safe clue brochure part of used for contains used for causes results from contains results from causes used for contains results from part of contains uses causes Sample SafeCracker ® Knowledge Map contains Slide 13 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Instrument: Problem Solving Retention & Transfer Questions Retention questions: 1.List how you opened the safe in the first room. 2.List how you opened the safe in the second room. Transfer questions: 1.List some ways to improve the way you solved opening the safe in room 1. 2.List some ways to improve the way you solved opening the safe in room 2. 3.List some ways to improve the way you navigated from room 1 to room 2. Slide 14 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Instrument: Self-Regulation Self-Regulation Questionnaire Based on O’Neil (1999) Problem Solving Model Trait self-regulation questionnaire (O’Neil & Herl, 1998). 32 Questions: 8 each of four measures –planning –self-checking/monitoring –self-efficacy –effort Slide 15 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Data Analysis Descriptive statistics: Means, Standard Deviation, etc., for all measures T-Tests for the following –Hypothesis 1: Navigation maps will produce a significant increase in content understanding compared to the control group. –Hypothesis 2: Navigation maps will produce a significant increase in problem solving strategy retention compared to the control group. –Hypothesis 3: Navigation maps will produce a significant increase in problem solving strategy transfer compared to the control group. –Hypothesis 5: Navigation maps will produce a significantly greater amount of optional continued game play compared to the control group. Pearson’s Correlation: Effect of the four self-regulation variables (planning, self-checking/monitoring, self-efficacy, and effort) on content understanding and problem solving strategies. –Hypothesis 4: There will be no significant difference in self-regulation between the navigation map group and the control group. However, it is expected that higher levels of self-regulation will be associated with better performance. Slide 16 of 16

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 BACK UP Slide 17

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Problem Solving Assessment Content Understanding Problem-Solving Strategies Self-Regulation Domain Specific Domain Independent MetacognitionMotivation PlanningSelf- Monitoring Effort Self- Efficacy Problem Solving O’Neil (1999) Problem Solving Model Slide 18

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Navigation Maps Navigation: Tracking one’s position in an environment to arrive at a destination (Cutmore et al., 2000) Occlusion: when a path is blocked visually (Cutmore et al., 2000) Navigation maps effective for occluded 3-D navigation (Cutmore et al. 2000; Dempsey, 2002) Navigation maps effective in 2-D environments with complex problem solving tasks (Baylor, 2001; Chou & Lin, 1998; Chou et al., 2000) Navigation maps effective in 3-D occluded environments with simple problem solving tasks (Galimberti, 2001) Not yet examined Navigation in 3D, occluded, environments with complex problem solving tasks Slide 19

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Instrument: Content Understanding Knowledge Mapping Software Slide 20

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Problem Solving Assessment Based on O’Neil’s (1999) Problem Solving Model –Requires content understanding, problem solving strategies, self- regulation Transfer questions are alternative to transfer tasks (Moreno & Mayer, 1998) Declarative knowledge measured by retention (Day et al., 2001) Knowledge map: concepts and links (Schau & Mattern, 1997) –Reflects the organization of knowledge (Day et al., 2001) –Reliable and efficient measure of Content Understanding (Herl et al., 1999; O’Neil, 1999; Ruiz-Primo et al., 1997; Schacter et al., 1999) –Predictive of both retention and transfer (Day et al., 2001) –Reliable measure of Problem Solving Strategies (Baker & Mayer, 1999) Slide 21

Richard Wainess Dissertation Proposal v.3Presented 04/27/04 Problem Solving Assessment (cont’d) Measuring Problem Solving Strategies –Domain general and specific (Alexander, 1992; Bruning et al., 1999) –Knowledge Mapping (Baker & Mayer, 1999) –Problem solving questions Positively correlated with retention and transfer (Mayer & Baker, 1998) Measuring Self-Regulation –Includes Metacognition & Self-efficacy (O’Neil, 1999) Metacognition: planning & self-checking (Pintrich & DeGroot, 1990) Self-efficacy: mental effort and self-efficacy (Zimmerman, 1994, 2000) –Trait self-regulation questionnaire (Hong & O’Neil, 2001) Includes planning, self-checking, mental effort, and self-efficacy 32 questions: 8 for each of the four sub-categories Slide 22