1. The Open Learning Initiative (OLI) Candace Thille, Director, Open Learning Initiative

2. What is Carnegie Mellon’s Open Learning Initiative? Scientifically-based open online learning environments designed to improve both quality and productivity in higher education

3. Goal-Directed Practice and Targeted Feedback Improve Learning Effectiveness

4. What is a Cognitive Tutor? A computerized learning environment whose design is based on cognitive principles and whose interaction with students is based on that of a (human) tutor—i.e., making comments when the student errs, answering questions about what to do next, and maintaining a low profile when the student is performing well.

5. Practice Synthesizing and Applying Skills & Knowledge

6. Feedback: Changing the Productivity of Learners and Teachers

10. What Are the Affordances of the Technology? The real key is:

11. “The Killer App” Feedback Loops for Continuous Improvement

12. Learning Curve Analysis DataShop: Pittsburgh Science of Learning Center

13. Cliques (sub-communities) The teacher Institute for Knowledge Innovation and Technology : University of Toronto

14. The teacher Institute for Knowledge Innovation and Technology : University of Toronto

15. OLI Review: Apply learning science research and scientific method to course development, implementation and evaluation Develop Learning Environments collaboratively. (teams of content experts and novices, learning scientists, HCI, software engineers) Feedback loops for continuous improvement What Difference Does it Make ?

16. Accelerated Learning Results OLI students completed course in half the time with half the number of in-person course meetings OLI students showed significantly greater learning gains (on the national standard “CAOS” test for statistics knowledge) and similar exam scores No significant difference between OLI and traditional students in the amount of time spent studying statistics outside of class No significant difference between OLI and traditional students in follow-up measures given 1+ semesters later

17. Quotes Student Quote: “This is so much better than reading a textbook or listening to a lecture! My mind didn’t wander, and I was not bored while doing the lessons. I actually learned something.” Instructor Quote: “The format [of the accelerated learning study] was among the best teaching experiences I’ve had in my 15 years of teaching statistics.”

18. “The Killer App” Feedback Loops for Continuous Improvement

19. Science of Learning Methods Ideally, learning research is both scientifically rigorous and realistically applicable Scientific “gold standard” of randomized trials Random assignment to condition (treatment vs. control) Other variables controlled to enable more valid inferences Multiple learning measures can be captured (with detail) But, such experiments tend to lack realism Short in duration Inauthentic context Unrepresentative participants

20. Methods continued Realistic classroom studies offer benefits Real students learning in real classrooms Duration of study less constrained Practical issues automatically included (time, teacher…) But, such studies have their difficulties too Lack of control of relevant variables Changing many variables limits potential inferences Compliance issues

21. Fine-grain data, collected over longer durations Rich, interaction-level data Automatic logging makes data easily accessible Rigorous studies with real content in real classrooms with real students OLI resources can be as controlled as lab environment Combine theory and practice to develop effective educational technology and refine learning theory OLI Learning and Research Environments: Best of Both Worlds

22. PSLC LearnLab: Like a research hospital for learning Tech enhanced courses in Science, Math & Language –Intelligent tutors, virtual lab simulations, language dialogs, multimedia, … Data on what works Researchers Schools Physics intelligent tutor (VanLehn, van de Sande) Researchers Schools Learn Lab OLI Statistics (Meyer, Lovett, Thille) OLI Chemistry (Yaron, Greeno, )

23. Strategy for Educational Improvement Data Theory OER/EdTech

24. Learning Research -> Learning Principles Eberly Center for Teaching Excellence

25. “Improvement in Post Secondary Education will require converting teaching from a ‘solo sport’ to a community based research activity.” —Herbert Simon OLI receives generous financial and intellectual support from: The William and Flora Hewlett Foundation, The Bill and Melinda Gates Foundation, The Lumina Foundation, The Spencer Foundation, The Kresge Foundation, The Walter S. Johnson Foundation, The National Science Foundation and Carnegie Mellon University cthille@cmu.edu

26. Thank you! Learning Principles: www.cmu.edu/teaching/eberly/ Eberley Center for Teaching Excellence: Susan Ambrose, Marsha Lovett, Michael Bridges, Michele DiPietro), Marie Norman Statistics Example Marsha Lovett, Oded Meyer, Joel Greenhouse, Ross Strader Chemistry Example David Yaron, Michael Karabinos, Gaea Leinhardt, Pittsburgh Science of Learning Center: www.learnlab.org Ken Koedinger, Charles Perfetti, David Klahr, Lauren Resnick, Vincent Aleven, Maxine Eskenazi, Carolyn Rose, All 200+ past & current members! Group Knowledge Building and Analysis Marlene Scardamalia Institute for Knowledge Innovation and Technology : University of Toronto