Proactive Acquisition Dialogues Jihie Kim Yolanda Gil www.isi.edu/expect/projects/rkf/

Slides:



Advertisements
Similar presentations
The Cost of Authoring with a Knowledge Layer Judy Kay and Lichao Li School of Information Technologies The University of Sydney, Australia.
Advertisements

A didactic plan for a communicative translation class Dr. Constanza Gerding Salas Leipzig Universität - Universidad de Concepción May 2012.
Sue Sears Sally Spencer Nancy Burstein OSEP Directors’ Conference 2013
CRITICAL THINKING The Discipline The Skill The Art.
Dialogic teaching in language classrooms. Do you know what RHINOs are? Really Here In Name Only Do you discover any ‘Rhinos’ in your classrooms?
DARPA SHAKEN Virus-Invades-Cell Invade VirusCell Attach Penetrate Release Move invader thing invaded barrier Cell-membrane has-part subevent penetrator.
Training for OCAN Users Day 2
The Systems Analysis Toolkit
The “Highly Effective” Early Childhood Classroom Environment
Curriculum Instruction & Assessment Part I - Alignment By Tina Waddy.
Dialogue in Intelligent Tutoring Systems Dialogs on Dialogs Reading Group CMU, November 2002.
Planning for Inquiry The Learning Cycle. What do I want the students to know and understand? Take a few minutes to observe the system to be studied. What.
Principles and Methods
Lesson planning? It can’t be that difficult! Svetla Tashevska, NBU.
Planning, Instruction, and Technology
Science Inquiry Minds-on Hands-on.
Basics of Lesson Planning
31 st October, 2012 CSE-435 Tashwin Kaur Khurana.
What is Business Analysis Planning & Monitoring?
Welcome!. How to teach what you know… Paula Williams Mike Taylor Neil Watkins.
New Advanced Higher Subject Implementation Events
AICT5 – eProject Project Planning for ICT. Process Centre receives Scenario Group Work Scenario on website in October Assessment Window Individual Work.
What should teachers do in order to maximize learning outcomes for their students?
1 USC Information Sciences Institute Jihie Kim Yolanda Gil Jim Blythe Intelligent Systems Division USC/Information Sciences Institute
“We will lead the nation in improving student achievement.” CLASS Keys TM Module 1: Content and Structure Spring 2010 Teacher and Leader Quality Education.
1 Professional Teacher Evaluation Program. 2 What is the PTEP? Assessment program designed to determine the quality of teacher performance in relation.
Curriculum development A brief guide to the construction of relevant curricula.
EARTO – working group on quality issues – 2 nd session Anneli Karttunen, Quality Manager VTT Technical Research Centre of Finland This presentation.
Guided Reading Workshop for SENCOs March Replaces the individualised teaching of reading with group teaching; Provides a significantly higher.
Multimedia Specification Design and Production 2013 / Semester 1 / week 9 Lecturer: Dr. Nikos Gazepidis
Object-Oriented Software Engineering Practical Software Development using UML and Java Chapter 7: Focusing on Users and Their Tasks.
Iasi 25 – 26 June 2009 Creativity and innovation to promote multilingualism and intercultural dialogue.
HOW ASSESSMENT SUPPORTS RTI 2 AND CLOSING THE ACHIEVEMENT GAP Clinch-Powell Cooperative Presenters:
THE PROCESS OF PLANNING The “Back Story” for teachers…..
Assessment Workshop Title of the Project (date). Project Title Assessment Workshop October 25, 2015© Company Name All rights reserved2 Agenda Purpose.
1 USC Information Sciences Institute Yolanda GilFebruary 2001 Knowledge Acquisition as Tutorial Dialogue: Some Ideas Yolanda Gil.
Understanding Our Learners: Teaching Mathematics for Inclusion & Equity February Bus Cluster Professional Development
Evaluating a UI Design Expert inspection methods Cognitive Walkthrough
Teaching Reading Comprehension
Summative vs. Formative Assessment. What Is Formative Assessment? Formative assessment is a systematic process to continuously gather evidence about learning.
CHAPTER 15 Your Personal Classroom Management Philosophy.
Case Based Learning (CBL) Training Slides
SLICK: Proactive Acquisition Dialog Jihie Kim Yolanda Gil Varun Ratnakar.
CRITICAL THINKING AND THE NURSING PROCESS Entry Into Professional Nursing NRS 101.
1 USC INFORMATION SCIENCES INSTITUTE EXPECT TEMPLE: TEMPLate Extension Through Knowledge Acquisition Yolanda Gil Jim Blythe Information Sciences Institute.
“Teaching”…Chapter 11 Planning For Instruction
1 USC INFORMATION SCIENCES INSTITUTE Gil & Kim Interactive Knowledge Acquisition Tools: A Tutoring Perspective Yolanda Gil Jihie Kim USC/Information Sciences.
RECIPROCAL TEACHING: IN AN ESL CLASSROOM Melissa Dye EDBE /11/2014.
Scaffolding Cognitive Coaching Reciprocal Teaching Think-Alouds.
November 12-13, 20098th International ILIAS Conference, Budapest, Hungary Organizational Integration of ILIAS Services at Dennis Gabor Applied University.
KANAL (Knowledge ANALysis) Status Jihie Kim Yolanda Gil Jim Blythe Varun Ratnakar
Designing Quality Assessment and Rubrics
COMPREHENSION STRATEGIES
Setting Objectives and Providing Feedback
Theories in Formation (TIF)
Dialog Manager for COA entry
KANAL: Knowledge ANALysis
OSEP Leadership Conference July 28, 2015 Margaret Heritage, WestEd
KANAL: Knowledge ANALysis
Chapter 3: Curriculum © VAN SCHAIK PUBLISHERS Chapter 3: Curriculum.
Next Step #2: Acquisition Dialogue
ELT. General Supervision
Prepared by: Toni Joy Thurs Atayoc, RMT
Deriving Acquisition Principles from Tutoring Principles
Discussion and Vote to Amend the Regulations
Assessment Workshop Title of the Project (date)
CP Storyboard Proposal
Unit 7: Instructional Communication and Technology
United Nations Voluntary Fund on Disability (UNVFD)
AICT5 – eProject Project Planning for ICT
Presentation transcript:

Proactive Acquisition Dialogues Jihie Kim Yolanda Gil

Feedback from Summer-01 Evaluations User’s comments on SHAKEN: “The system had to be taken by the hand” "I do not really know whether there is a possibility of standardizing the entire [KA] process. But it would be better to document some of the [KA] processes which you think are standardized". Proposed solution: Extend SHAKEN with proactive dialogue capabilities

Research Issues How to turn a KA tool into a good student, how to help a user be a good teacher –Tutoring & educational literature Assess competence and confidence in the new body of knowledge Dialogue planning –Meta-level knowledge about KA tasks Collaborative dialogue User modeling Utility of system’s interventions

Deriving Acquisition Principles from Tutoring Principles Instructional System Acquisition Tool teaches ? ? Good Tutoring Principles Good Learning Principles SOFTWAREUSER

Deriving Acquisition Principles from Tutoring Principles (II) Instructional System Acquisition Tool teaches ? ? Good Tutoring Principles Good Learning Principles SOFTWAREUSER

14 Tutoring and Learning Principles [Kim & Gil 02] Teaching/Learning principleTutoring literature Start by introducing lesson topics and goals Atlas-Andes, Meno-Tutor, Human tutorial dialog Use topics of the lesson as a guide BE&E, UMFE Subsumption to existing cognitive structure Human learning, WHY, Atlas- Andes Immediate FeedbackSOPHIE, Auto-Tutor, Lisp tutor, Human tutorial dialog, human learning Generate educated guessesHuman tutorial dialog, QUADRATIC, PACT Keep on trackGUIDON, SHOLAR, TRAIN-Tutor Indicate lack of understandingHuman tutorial dialog, WHY

Teaching/Learning principleTutoring literature Detect and fix “buggy” knowledgeSCHOLAR, Meno-Tutor, WHY, Buggy, CIRCSIM Learn deep modelPACT, Atlas-Andes Learn domain languageAtlas-Andes, Meno-Tutor Keep track of correct answersAtlas-Andes Prioritize learning tasksWHY Summarize what was learnedEXCHECK, TRAIN-Tutor, Meno-Tutor Provide overall assessment of learning knowledge WEST, Human tutorial dialog Tutoring and Learning Principles (cont)

Competence and Confidence: Learning Awareness Capable of assessing: –Competence : What is known, what is unknown –Confidence : What has been tested, what has been checked by the user Steer the dialogue to improve KB in both counts

Awareness Annotations 1) Annotations to the new body of knowledge: –For each lesson: purpose, assumed background, sub-lessons, overall competence and confidence –For each k item: connection to lesson, relation to other items, identity wrt other items, possible analogies and generalizations, domain terminology details, competence, confidence –For each axiom of a k item: required information, generality, completeness, confidence 2) Annotations to the dialogue history: –For each user action: changes to the annotations to the new knowledge, acquisition goals achieved and/or activated, possible future KA strategies

Dialogue Planning: Viewing KA Activities as Lessons 1) SET UP LESSON AND CHECK BACKGROUND 2) ACCEPT AND RELATE NEW DEFINITIONS 3) TEST AND FIX 4) FIT WITH EXISTING KNOWLEDGE STRUCTURES: 5) ACHIEVE PROFICIENCY 6) REACH CLOSURE

Dialogue Planning that Exploits Acquisition Principles SET UP LESSON AND CHECK BACKGROUND: –Get the overall topic and purpose of the lesson. –Acquire any assumed prior knowledge before pursuing the lesson. ACCEPT AND RELATE NEW DEFINITIONS: –Accept new definitions –Ensure that new knowledge is specific as possible. –Ask the user to be complete when enumerating items in terms of the elements and in terms of the significance of the order given. –Get all the information required when existing knowledge indicates it must be provided. –Make all new definitions consistent with existing knowledge. –Connect all new items with the topic of the lesson. TEST AND FIX: –Test the new body of knowledge and generate tests for the aspects that have not been thoroughly tested. –Fix problems that result from self-checks or from user's indications. –Ensure user checks the reason for the answers, not just the answers themselves. –Confirm new answers that change in light of new knowledge over what the user had seen the answer to be earlier.

FIT WITH EXISTING KNOWLEDGE STRUCTURES: –Establish identity of new objects by checking if existing objects appear to be the same. –Generalize definitions if analogous things exist and there could be plausible generalizations. ACHIEVE PROFICIENCY: –Acquire domain terms to describe new knowledge. –Learn to reason/generate answers efficiently and with shorter explanations. REACH CLOSURE: –Ensure that the purpose/topics of the lesson were covered and the test questions appropriately answered. Acquisition Principles (cont)

SHAKEN’01 UI (SHAKEN client) User command logs SHAKEN server KANAL KB Q/A

Acquisition Dialogues in SHAKEN’02 UI (SHAKEN client) User command logs SHAKEN server KANAL KB Q/A KA dialog Window Acquisition Strategies Active Acquisition strategy State & History … … … Dialogue Manager

Bacterial Transcription: A process model in biology Scenario called Bact-Txn1 CollideMove-ThroughRecognizeMake-Contact Bacterial-Polymerase Base-Pair Promoter object base object base subevent next Base-Pair path first-subevent Tangible-Entity DNA-Melting next object Bacterial-DNA structural-part-of

Gral acquisition principle Specific acquisition goal Educated guesses

Awareness Annotations: 1) State

Awareness Annotations: 2) History Shows user’s actions and their effects in accomplishing acquisition goals or raising new ones User can view changes to the state

INSTRUCTION ASSIMILATION TRIGGER GOALS & STRATEGIES PROPOSE STRATEGIES PRIORITIZE GOALS & STRATEGIES PRESENTATION DESIGN History Awareness Annotations Active goals & candidate strategies Immediate goals & strategies STATUS Acqu. goals Acqu. strats Guess Generators Interaction Guidelines Operational Principles Priority Schemes General Tutoring & Learning Principles KA Tool GUI Dialogue GUI -Goals & Strats -State - Suggestions - History Knowledge Base Input command Five Main Functions of KA Tools

Tutoring and Learning principles used in KA tools [Gil & Kim 02] KSSnAssess learned knowledge Summarize what is learned EXPECTPrioritize learned tasks SEEK2Keep track of answers Learn domain language Learn deep models EXPECT,CHIMERATAQLDetect and fix “buggy” K INSTRUCTO- SOAR Indicate lack of understanding Keep on track EXPECTTEIREISIASGenerate educated guesses EXPECTTEIREISIASINSTRUCTO-SOARPROTOSImmediate feedback PROTOS, SALT TEIREISIAS PROTOSSubsumption to existing cog. structure SALTEXPECTSEEK2SALTUse topics of the lesson as a guide EXPECT, SEEK2Introduce topics & goals Design Presentation Prioritize Goals & Strats Propose Strategies Trigger Goals Assimilate Instruction Tutoring/Learning principle

Future Work Dialogue Planning –Designing a library of dialogue plans for KA tasks –Focus and attention (track subdialogues, detect thrashing) Collaborative dialogue techniques Integration with SHAKEN –Longer term: redesign overall user interaction Integration of other modules to operationalize more principles (CHIMAERA, Analogy, etc.)