1. Crowdsourcing Personalized Online Education
Closing Comments
Eric Horvitz

2. Crowdsourcing Personalized Online Education
Great people, conversations, presentations!
Thanks for investing the time & travel.
Thanks for sharing your creative ideas, insights, directions, and enthusiasm.

3. Crowdsourcing Personalized Online Education
Inflection point: Methods, connectivity, scale, interest in online ed.
Evolving ideas & prototypes
New directions in the air

4. Crowdsourcing Personalized Online Education
Multiple communities: Opportunities and challenges
Scaling Education Online
Data!!
Goals
Lab!
Computer Science
Education

5. Crowdsourcing Personalized Online Education
Multiple communities: Opportunities and challenges
Scaling Education Online
Methods
Models
Insights
Designs
Computer Science
Education
DataLearnPredictAct
Designs for interaction
Just build it.
Automate!
Graphical models
POMDPs
Machine learning
Optimize!
Mechanism design
Practices & themes
Small n studies
Cognitive models
Daily challenges
In vivo studies
Protocol analysis
Tutor learning
ITS
AI-ED
CAI
Cognitive psychology
HCI AI
HCOMP
CogSci

6. Crowdsourcing Personalized Online Education
Seeking synthesis and deep understanding – building new community
Education
Computer Science
Cognitive psychology
Practices & themes
Small n studies
Cognitive models
Daily challenges
In vivo studies
Protocol analysis
Tutor learning
DataLearnPredictAct
Designs for interaction
Just build it.
Automate!
Graphical models
POMDPs
Machine learning
Optimize!
Mechanism design
ITS
AI-ED
CAI
HCI: Mechanisms, de & Studies
AI: Data  Learn  Predict  Act
Psych/education: Cognitive models
HCI AI
HCOMP
CogSci

7. Crowdsourcing Personalized Online Education
Seeking synthesis and deep understanding – building new community
Education
Computer Science
Cognitive psychology
Practices & themes
Small n studies
Cognitive models
Daily challenges
In vivo studies
Protocol analysis
Tutor learning
DataLearnPredictAct
Designs for interaction
Just build it.
Automate!
Graphical models
POMDPs
Machine learning
Optimize!
Mechanism design
ITS
AI-ED
CAI
HCI: Mechanisms, de & Studies
AI: Data  Learn  Predict  Act
Psych/education: Cognitive models
HCI AI
HCOMP
CogSci

8. Crowdsourcing Personalized Online Education
Seeking synthesis and deep understanding – building new community
Education
Computer Science
Cognitive psychology
Practices & themes
Small n studies
Cognitive models
Daily challenges
In vivo studies
Protocol analysis
Tutor learning
DataLearnPredictAct
Designs for interaction
Just build it.
Automate!
Graphical models
POMDPs
Machine learning
Optimize!
Mechanism design
HCOMP
ITS
AI-ED
CAI
Building community
HCI: Mechanisms, de & Studies
AI: Data  Learn  Predict  Act
Psych/education: Cognitive models
HCI AI
CogSci

9. Crowdsourcing Personalized Online Education
Seeking synthesis and deep understanding – building new community
Education
Computer Science
Cognitive psychology
Practices & themes
Small n studies
Cognitive models
Daily challenges
In vivo studies
DataLearnPredictAct
Designs for interaction
Just build it.
Automate!
Graphical models
POMDPs
Machine learning
Optimize!
Mechanism design
ITS
AI-ED
HCI AI
HCOMP
CogSci

10. Crowdsourcing Personalized Online Education
Seeking synthesis and deep understanding – building new community
Education
Computer Science
Cognitive psychology
Practices & themes
Small n studies
Cognitive models
Daily challenges
In vivo studies
DataLearnPredictAct
Designs for interaction
Just build it.
Automate!
Graphical models
POMDPs
Machine learning
Optimize!
Mechanism design
ITS
AI-ED
HCI AI
HCOMP
CogSci

11. Synthesis to understand mysteries of great teaching
Richard Feynman on his beloved Mr. Bader:
“When I was in high school, my physics teacher—whose name was Mr. Bader—called me down one day after physics class and said,
"You look bored; I want to tell you something interesting."
Then he told me something which I found absolutely fascinating, and have, since then, always found fascinating the principle of least action.”
(Chapter 19, Vol.2, Feynman Lectures on Physics).

12. Mysteries of affect, attention, engagement
Herb Simon, Reason in Human Affairs, Harry Camp Lecture, Stanford, 1982

13. Keynote at ITS
by an outsider
in 2000…

14. Keynote at ITS
by an outsider
in 2000…

15. Keynote at ITS
by an outsider
in 2000…

16. Keynote at ITS
by an outsider
in 2000…

17. Keynote at ITS by an outsider in 2000…
Model assessed manually with a psychologist.
Predict kids’ forthcoming loss of engagement with Microsoft childrens’ software applications (1996).
Might this approach be employed in online tutoring systems?
K. Risden, H. 1996

18. Performance, & engagement
19962011: Opportunities to learn rich models from data
e.g., CrowdSynth: 34m votes, 100k participants
Machine vision
Current activity
User’s Long-term activities
Performance, & engagement
System
actions
E. Kamar, S. Hacker, H. Combining Human and Machine Intelligence in Large-scale Crowdsourcing, 2012.

19. Crowdsourcing Personalized Online Education
Directions & Follow up
Research directions
Community
HCI: Mechanisms, de & Studies
AI: Data  Learn  Predict  Act
Psych/education: Cognitive models

20. HCI: Mechanisms, de & Studies
AI: Data  Learn  Predict  Act
Psych/education: Cognitive models