1. Dynamically altering the learning trajectories of novices with pedagogical agents Carole R. Beal, USC Ronald H. Stevens, UCLA Cognition & Student Learning R305H050052

2. Project focus: Scientific problem solving Learning to do science –hypothesis generation –effective use of resources & content knowledge –strategy selection & progress checking Skills apply across science domains –“Investigation & experimentation” in CA standards Stevens et al. developed IMMEX to provide problem solving practice opportunities IMMEX: Online multimedia problem solving environment providing simulations of case- type scientific problems www.immex.ucla.edu Ron Stevens

3. IMMEX example: Duck Run Domain: Chemistry Goal: Identify element dumped in pond Available resources: –Experts –Characteristics –Periodic table –Glossary Multiple cases

4. IMMEX: Strategy assessment Solution strategies: –searching resources –balancing cost of search & info. value, time Students’ actions classified by ANN –“nodes” represent: what problem resources are viewed relation with solve rates 8% of cases students look at all resources solve rate only 47% Stevens et al. (1999). Artificial neural network-based performance assessments. Computers in Human Behavior, 15, 295-314. Duck Run

5. IMMEX: Prior work on problem solving strategies Stabilization –approaches tend to persist over time –case repetition does not always lead to improved performance Student strategies related to teacher implementation –better performance with teachers who “frame” activity well PedAgents goal: Enhance IMMEX to encourage good strategies “red” teacher’s students using resources well & solving problems Stevens, R. H., & Thadani, V. (2007). A value-based approach for quantifying scientific problem solving effectiveness. Journal of Technology, Instruction, Cognition & Learning, 5, 325-337.

6. PedAgents: Strategy intervention Integrate strategy guidance into case prologues –e.g., “Sometimes students just dive in to a problem and try out lots of different things. But it’s better to make sure you know what the problem is asking first. If you don’t, go back and read the prologue again, or ask your teacher for help.” Prologues also address motivational issues –e.g., “Sometimes, students give up too fast on IMMEX problems. They think the answer should come in just a couple of minutes. But hard problems take a lot of mental effort. Stick with it and you’ll see results!” Prologues are not content- specific –delivered randomly across cases strategy suggestion delivered at prologue

7. PedAgents: Strategy intervention Epilogues encourage self-reflection & strategy review: –e.g., “Look at your notes for this case. Are they organized? Can you see the path you took to get to your answer? If not, can you think of one or two ways of making them more organized? (If you didn’t take notes, take them – it will help!)” Epilogues also target motivational issues in problem solving: –e.g., “Sometimes students get discouraged by this case. If you solved it, that’s great! If you didn’t, don’t be too hard on yourself. You can learn even when you don’t get the right answer. Make failure work for you by figuring out what to do differently next time.” strategy hint also delivered at case epilogue

8. PedAgents: Intervention results Improved first-try solve rates: from 64% to 75% Reduction in never-solved cases Higher IRT scores across five cases Decrease in use of “exhaustive” strategies –students who still look at all resources more likely to solve –hints may be promoting better integration of information decrease to 4.5% of cases; solve rate increases to 61% Duck Run

9. PedAgents: Interventions Message content is key –Prior study showed generic academic advice messages were not helpful: e.g., “Successful students use a planner to keep track of notes and assignments” “It’s a good idea to keep up with your class reading. Don’t let yourself get too far behind!” Current study: Role of message delivery –Text messages compared with animated agents –Human or non-human agent characters e.g., teacher or duck

10. Next steps Goal: integrate strategy guidance into complete IMMEX middle school curriculum –earth, life & physical sciences –match message content with cases e.g., link motivation- oriented messages with more difficult cases –support for effective implementation by teachers “Phyto Phiasco” “Get organized” “dashboard” for teachers to track student progress