Embodied Conversational Agents: A Case Study of Freudbot Bob Heller, PhD Athabasca University November 3, 2004.

Slides:



Advertisements
Similar presentations
ETL339: E-Learning Is it all just smoke and mirrors... bells and whistles?
Advertisements

Learning at Lanyon High B Davies. Groups Previously students were seated around the classroom and were not participating well in the lesson. The structure.
School of something FACULTY OF OTHER School of Computing FACULTY OF ENGINEERING Fostering language learner autonomy via adaptive conversation tutors by.
Cognitive-metacognitive and content-technical aspects of constructivist Internet-based learning environments: a LISREL analysis 指導教授:張菽萱 報告人:沈永祺.
A GUIDE TO CREATING QUALITY ONLINE LEARNING DOING DISTANCE EDUCATION WELL.
UDL BOOT CAMP Overview: The introduction provides a framework for applying Universal Design for Learning (UDL) principles to meeting the instructional.
The Computer as a Tutor. With the invention of the microcomputer (now also commonly referred to as PCs or personal computers), the PC has become the tool.
Designing Learning Programs. Ambition in Action Topics /Definition /Components of a learning program /Defining the purpose and focus.
Pandorabots The Leader in Chatbot Software. Overview Quick Tour of: History of Chatterbots Introduction to Basic Technology Applications Gallery Pandorabots.
E-Learning Can it replace the traditional classroom? A Case Study at A Private High School Jennelle Irene Spurlock-Johnson Wendy Zhang Leeta Allen-Haynes.
DESIGN AND ONLINE DELIVERY OF A COASTAL SCIENCE MODULE USING A CONSTRUCTIVIST APPROACH. RAMESSUR R.T. 1 and SANTALLY M.I. 2 1.Faculty of Science, University.
Facilitating Online Discussions Jason D. Baker. Topics Discussion Value Discussion Tools Discussion Tips.
Online Career Assessment: Matching Profiles and Training Programs Bryan Dik, Ph.D. Kurt Kraiger, Ph.D.
Characteristics of on-line formation courses. Criteria for their pedagogical evaluation Catalina Martínez Mediano, Department of Research Methods and Diagnosis.
Intelligent Robotics Project “A CUSTOMIZED ALICE” By Anupama Seshagiri.
Teacher’s Attitudes and Perceptions Toward the Use of Inspiration 6 Software in Inclusive World History Classes at the Secondary Level Summary by Katie.
Second Life at Athabasca University Dave Annand Gunnar Schwede Bob Heller Colin Elliott.
Introduction to Web-Based Learning. Defining Web-Based Instruction Instruction via Internet and Intranet only. Synonymous with online learning.
Master Online Teacher Competencies eLearning Conference 2007 Virgil E. Varvel Jr.
 Chat has enormous potential to link students around the world, in real time. It is technology that many learners will often be familiar with and will.
Three-way blending with learning objects: A versatile approach to supporting international and British students using reusable learning objects Julie Watson.
Margaret J. Cox King’s College London
Recognition of Prior Learning and Experience The Case of the Ditsela FETC:TUP Learnership.
Chatterbots Go Native: Considerations for an Eco-system Fostering the Development of Artificial Life Forms in a Human World (3/03 - Japan) Authors Dr.
GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004.
DEVELOPMENT OF LEARNING MATERIAL FOR USER EDUCATION – BASED ON A SURVEY RÓZA FRANK BERZSENYI COLLEGE, SZOMBATHELY LIS DEPARTMENT
Dissertation Theme “The incidence of using WebQuests on the teaching-learning process of English Foreign Language (EFL) for students attending the seventh.
“ Sociology of Education and Didactics- theory of Education ” Thelma de Jager (Educational Studies )
Proposal Presentation. Introduction - Not much studied - Topical - Likely to continue into future - High level of political interference - Problem w/
1 Politeness Effect: Pedagogical Agents and Learning Gains 報 告 人:張純瑋 Wang, N., Johnson, W.L., Mayer, R.E., Rizzo, P., Shaw, E., and Collins, H. (2005).
INTERACTION, ENGAGEMENT AND MOTIVATION. WELCOME o Facilitator name Position at university Contact info.
The Interpersonal Mode
The strategy using in communication between Japanese and Taiwanese university student MA3C0210 王明君 Chelsea.
Peer-to-Peer learning using a learning management system in an ODL environment.
Projects Project 1: PSU CHATBOT Language: AIML Group: Omar and Ahmed
Web-based Technologies in Teaching and Learning Chapters 9 & 11 October 13, 2005 Professor Paul Kim By Piya Sorcar.
Lecture 15 – Social ‘Robots’. Lecture outline This week Selecting interfaces for robots. Personal robotics Chatbots AIML.
Evaluation of Course Design and Student Comprehension in the International Learning Environment: A Panel Data Analysis Pavasajjanant Natcha Yuji Akematsu.
1 Meta-Analysis of the Effectiveness of Pedagogical Agent 報 告 人:張純瑋 Kim, M. & Ryu, J. (2003). Meta-analysis of the effectiveness of pedagogical agent.
The effects of captions on deaf students’contents comprehension, cognitive load and motivation in online contents 21 June 2010 Joong-O Yoon
The product of evaluation is knowledge. This could be knowledge about a design, knowledge about the user or knowledge about the task.
Dialogue systems Volha Petukhova Saarland University 03/07/2015 Einführung in Diskurs and Pragmatik, Sommersemester
State University of New York An Emerging Model for Online Learning MERLOT International Conference – August A Systemic Approach to Online Learning.
An Evaluation of an On-line Anatomy Course by Lab Instructors: Building on Instructional Design Guo, X., Katz, L., & Maitland, M. The University of Calgary,
Effects of an online problem based learning course on content knowledge acquisition and critical thinking skills Presenter: Han, Yi-Ti Adviser: Chen, Ming-Puu.
E VALUATING YOUR E - LEARNING COURSE LTU Workshop 11 March 2008.
Programme design and student assessment David Baume 1.
Surveying instructor and learner attitudes toward e-learning Presenter: Jenny Tseng Professor: Ming-Puu Chen Date: April 12, 2008 Liaw, S., Huang, H.,
QuizMASter – A Multi-Agent Game-Style Learning Activity Mark Dutchuk Vancouver Island University, Canada Khalid Aziz Muhammadi Government of Alberta, Canada.
THE PERCEPTIONS OF ENGLISH LANGUAGE TEACHING STUDENTS ON ELT WEBSITES Assist. Prof. Dr. Hasan Bedir/ Cukurova University Inst Emsal Ates Ozdemir/Mersin.
Rebecca Castro Katrina Coker Design Presentation II Everything You Always Wanted To Know About Science Fair But Were Afraid To Ask.
Instructional Design Course The Analysis Phase Roger Powley, CD PhD.
Conversational Agents and Learning Outcomes: An Experimental Investigation Bob Heller Mike Proctor Athabasca University Research supported by Mission Critical.
Virtual Tutor Application v1.0 Ruth Agada Dr. Jie Yan Bowie State University Computer Science Department.
ISECON (San Antonio, TX) November 1, Student Perceptions of Online Learning: A Comparison of Two Different Populations Kitty Daniels and Susan Feather.
A Multi-Agent and Service-Oriented Architecture for Developing Integrated and Intelligent WBES Computing and Information Systems Oscar Lin Peter Holt.
Animated Pedagogical Agents: Historical Figure Applications Bob Heller, Athabasca University Mike Procter, Athabasca University
Strategies for blended learning in an undergraduate curriculum Benjamin Kehrwald, Massey University College of Education.
1 Multimedia Development Team. 2 To discuss phases of MM production team members Multimedia I.
Design Evaluation Overview Introduction Model for Interface Design Evaluation Types of Evaluation –Conceptual Design –Usability –Learning Outcome.
SUPPORT, PROGRESS, & MOTIVATION RAY POWERS EDU 601 DR. WILLIAM REEVES MAY 30, 2016 Ray Powers EDU 601 Dr. William Reeves April 18, 2016.
TEMPUS IV MASTS Master curriculum design workshop in Nis Subject Teacher Education Master Programme at University of Belgrade.
SIG Writing Conference, Liverpool – July 2016
CHAN Ka Wing, LAM Chung Man and YEUNG Yau Yeun
7/17/ :54 AM Conveying User Facial Expressions into Virtual World E-Learning Applications Mike Procter GSRDD 2014 © 2007 Microsoft Corporation. All.
Moving blended learning beyond learning management systems
Personalizing conversational agent based e-learning applications
Nives Mikelic Preradovic, Sanja Kisicek, Damir Boras
Teaching Course 1, Session 7, Day Two
Presentation transcript:

Embodied Conversational Agents: A Case Study of Freudbot Bob Heller, PhD Athabasca University November 3, 2004

Mike Proctor – AIML programmer Dean Mah – Web implementation Billy Cheung – Graphics, test chatter Lisa Jewell – Chat log analysis, content developer, test chatter Julianna Charchun – Chat log analysis Jude Onuh – AIML programmer Acknowledgements

Embodied Conversational Agents Definitions Embodiment in Conversational Interfaces: REA (Cassel et al., 1999) Embodied Conversational Agents (Cassel, Sullivan, Prevost, & Churchill, 2000) –FMTB model Vos (2002) offers 5 features of ECA –Human like appearance –Body used for communication purposes –Natural communication protocols –Multimodality –Social role

Embodied Conversational Agents Anthropomorphic Agents Animated Interface Agents Animated Pedagogical agents Pedagogical Agent Persona Intelligent Tutoring Systems -AutoTutor (Graesser et al) Chatterbots or Chatbots - Weizenbaum’s (1966) Eliza

Why? -primacy of conversation -Constructivist theory -The Media Equation -Persona effect -cognitive load Embodied Conversational Agents

Richard Wallace and A.L.I.C.E. Artificial Linguistic Internet Computer Entity 3 time winner of the Loebner Contest (the holy grail for chatbots) AIML – Artificial Intelligence Markup Language PandoraBots

Embodied Conversational Agents ‘Theory’ behind ALICE -pattern matching -Zipf distribution -Iterative

Freudbot 1 Why Freud? Initial plan of deployment The famous personality application –Emile –Shakespeare –Hans Christian Anderson –John Lennon

Freudbot 1 Developing the AIML Narrative structure Test chatters How much ALICE?

Research Questions Is it worth it? Is ‘chattiness’ related to the subjective evaluation of chat experience? Are there individual difference variables that are related to measures of chat performance/experience? Freudbot 1

Online Recruitment – restricted to psychology students – Incentive (1/30 chance at $300) Random assignment to bot type Controlled Chat –automatically directed to questionnaire after 10 mins of chat Freudbot 1: Methodology

nPercent GenderMen1218% Women5582% Age Distribution % % % % % % Student StatusFull-time2740% Part-time3552% Non-student58% Self-rated academic Below avg00% abilityAverage1319% Above avg3958% Excellent1522% Freudbot 1: Participants (N=67)

Is it worth it? self-report data* Mean Useful 2.2 Recommend2.4 Overall2.4 Enjoyable 2.6 Engaging2.7 Memorable2.8 Expansion3.4 * 5 point scale Would you chat again? Yes No (n=30) (n=35)

Best Features Interactivity16 Able to ask questions with answers16 Learning about Freud & theories13 Simplicity/ease of use5 Entertaining/humorous5 Thought provoking5 No good features5 Technological features of Freudbot4 Potential to Freudbot4 Alternative learning style3 Novelty/uniqueness of Freudbot3 Tricking Freudbot2 Unpredictable2 Worst Features Repetition33 Unable to answer questions23 Conversation did not flow12 Limited knowledge base10 User needed prior knowledge 3 User was uncertain about what to do 3 Not an effective learning tool3 Conversation was too short1 No sound1 Is it worth it?

MeanRange Number of Exchanges Chat logs Mean Proportion of on-task responses by participant*.60 questions.37 comments*.23 * correlated with a composite measure of self rated chat experience Proportion of repetitions by Freudbot.25 Proportion of non-sensical by Freudbot.39

FreudAlice JustFreud n=35n=32 Useful Recommend Overall Enjoyable Memorable Engaging Expansion Chattiness? # of Exchanges On task Response* * -significant difference btw groups

Individual difference variables? demographic –Gender –Age –Student status* –Self-rated academic ability computer experience & self-rated skill academic background –# of university courses –# of distance ed courses* –# of psychology courses –Rated importance of Freud*

Individual difference variables? attitudes towards technology and education –Positive aspects of on-line activities –Independent Learner –negative aspects of on-line activities*

Is it worth it? –worth another look Is ‘chattiness’ related to the subjective evaluation of chat experience? –‘Chattiness’ is not the right level –Nass and Reeves (1998) Are there individual difference variables that are related to measures of chat performance/experience? –some relations that make sense and others that don’t Freudbot 1 Summary

Research Goals 1. Improve Performance Fix repetition problem Topic tags More content 2. Replication 3. Instructional Set 4. Future Development Freudbot 2

online recruitment, incentive, & controlled chat identical to Freudbot 1 random assignment to instructional set similar questionnaire with additional questions on applications and improvements Freudbot 2:Methodology

nPercent GenderMen1018% Women4582% Age Distribution % % % % % % Student StatusFull-time2647% Part-time2851% Non-student12% Self-rated academic % ability % % % % Participants (N=55)

Improvement? self-report data (5 point scale) Freudbot 1 Freudbot 2 Useful** Recommend** Overall** Enjoyable Engaging** Memorable Expansion** ** - statisically significant Would you chat again? Yes No (n=37) (n=18)

Improvement? MeanRange Number of Exchanges Chat logs Mean Proportion of on-task responses by participant*.90 questions.36 comments.48 * correlated with a composite measure of self rated chat experience Proportion of appropriate responses by Freud.60

Replication? Demographic –Gender* –Age –Student status* –Self-rated academic ability computer experience academic background –# of university courses –# of distance ed courses –# of psychology courses –Rated importance of Freud*

Replication? attitudes towards technology and education –Positive aspects of on-line activities –Independent Learner –negative aspects of on-line activities*

Instructional Set? Brief Set Elaborate Set n=27n=28 Useful Recommend Overall Enjoyable Memorable Engaging Expansion # of Exchanges On task Response.90.90

Future Development? Freudbot Improvements Mean* Chat behaviour4.2 Audio Response3.1 Voice Recognition2.6 Synchronization2.5 Animation/movment2.3 * 5-point scale Other Applications Mean* Practice quizbot4.1 Famous personality4.1 Course content3.4 Chatroom3.3 Course Admin3.2

1. Improvement - yes, but clearly room for more 2. Replication - some 3. Instructional Set - no effects 4. Development Freudbot 2: Summary

Future Direction Haptek Freud –Animacy/agency hypothesis Piagetbot (Support from MCR) –learning outcomes Skinnerbot (Lyle Grant) Coursebot Quizbot

Questions?