Sung Young’s comments Start up was rough. Asked Mr. Davis SY was not able to enter whole content Didn’ thave to learn whole content; could look up insead; would recommend closed book second session. Learned the material by reading; The role of map was to force me to read ; not different from traditional

Kurt’ scomments Semantics of the nodes “heat energy” Asking Betty what a term means is not helpful I didn’t feel like I was teaching

Daren’s comments Darren: slow start. Hates the voice Felt good that Betty (not me) failing Tried to put quiz into the net The voice is a nice relief from (boring) reading, but the voice has to be better

Andre’s comments Installatoin difficulties Without knowing content, had hard start Study guide was key Switching screens was inconvenient Gaming: Give her the quiz then put in corrective links Needs direct instruction before Believe that teaching reveals gaps

Robert’s comments Another slow start Read library; didn’t notice study guide Putting underelined terms from library into graph Found mattrix tab that showed % knowledge Copied rows to graph; that didn’t work Found that Mr. Davis’s suggestons weren’t helpful Liked Betty’s logic; showing explanations Ended up focussing on wrong answers from quiz Cluttered graph; no zooming; auto format Moving links insead of delete/receate Popups annoying How did they recognize terms e.g., mist vs. water vapor; spelling Granularity of nodes wasn’t clear Struggled with the “atmosphere” node’s meaning Had to consciouisly engage in the teaching role; it supported that ; became more engaging Mac vioice absent

Maria’s comments Rought beginning Need instruction on how to build a map Pop ups annoying One of Mr. Davis’ suggestons was incomprehensible When Betty asks “does my answer seem right?” there is no way to answer. Mr. Davis was not letting me do my own learning plan – draw first then later ask. Betty keeps asking me to expand inappropriately Spelling check was nice Nice that it checks for duplicates; but handles it poorly What does the green line outside a node mean? What is the purpose of the notes field on a node—where does it appear? Saving and returning later? Took labels from the matrix into the map nodes; Worrying too much about passing quizes than learning climate change. Betty’s is a tool/notation

Javier’s comments Pop ups annoying, especially when not using Betty and when doing a task Transferred labels from quiz to map; relationships too. Began to focus on answewring the quiz rather than learning Quiz words were not the ones that I used; it needs to know synonyms. Workspace is too small. Clutter. Losing nodes How to control curves Voice annoying; turned it off Feel’s like Betty is teaching; reviewing my work as tutor Betty’s comments about “now I’m underwstadning” feels like positive feedback.

Teachable agents: A mini-lit review (implemented systems only) Kurt VanLehn CPI 494, March 19, 2009

(Near) synonyms for “teachable agent” Learning by teaching/tutoring system – Synonymous, but may not have image & persona Simulated student – May be used for non-pedagogical purposes Learning companion, Co-learner – User & agent are both learners Knowledge acquisition system – User is assumed to be an expert already

Framework for comparing teachable agents Same old nested loop structure: – Outer loop: select task, do task, repeat – Inner loop: prep step, do step, feedback, repeat The human user plays role of tutor – Inner loop over steps – always – Outer loop over tasks – sometimes not The agent plays role of tutee – May or may not have hidden domain knowledge Role reversal? If yes, called reciprocal tutoring Their communication is either or both of: – Unnatural: User can edit agent’s knowledge – Natural: Only observable actions and talk

Betty’s Brain Task domains: stream ecology, climate change, body temperature regulation… Reciprocal: No Communication: Unnatural, Persona – User edits agent’s KR directly – User can ask questions, ask for traces of reasoning – Agent takes test Expert knowledge: Yes – The scoring of the test Evaluations: Next class meeting

Steps Ur, S. & VanLehn, K. (1994). STEPS: A preliminary model of learning from a tutor. The Proceedings of the 16th Annual Conference of the Cognition Science Society. Hillsdale, NJ: Erlbaum. Tasks: Solving physics problems Reciprocal: No Communication: Natural – User demonstrates solution while agent self-explains – Agent solves problems while user gives feedback on steps, including bottom-out hints Expert knowledge: None Evaluation: None

Palthepu, Greer & McCalla 1991 Palthepu, S., Greer, J., & McCalla, G. (1991). Learning by teaching. The Proceedings of the International Conference on the Learning Sciences, AACE, Task: populating an inheritance hierarchy e.g., animals, dophins, legs, mamals, live births… Reciprocal: No Communication: Natural – Student tells agent facts – Agent asks the student questions “Do dolphins have legs?” Expert knowledge: None Evaluation: None

PAL Reif, F., & Scott, L. A. (1999). Teaching scientific thinking skills: Students and computers coaching each other. American Journal of Physics, 67(9), Task: Solving physics problems Reciprocal: Yes Communication: Natural, no persona – Agent solves problems & user catches mistakes – User solves problems and agent catches mistakes Expert knowledge: Yes – When user is tutoring, system corrects user’s tutoring – When agent is tutoring, then standard step-based tutoring Evaluation: Yes – No tutoring < PAL = human tutoring

PAL’s evaluation Class: did homework at home like usual PAL: did homework in lab on PAL Tutoring: did homework in lab with human tutors Class < PAL d=1.01; Class < Tutoring d=1.31 PAL = Tutoring Confound? Class may not have been taught Reif’s method