What can CTAT do for you? Overview of the CTAT track Vincent Aleven, Bruce McLaren and the CTAT team 3rd Annual PSLC LearnLab Summer School Pittsburgh,

Slides:



Advertisements
Similar presentations
Introduction to AuthorIT April 10, 2006 Symposium on Knowledge Representation TICL SIG Joseph M. Scandura, Ph.D. Chairman, Board Scientific Advisors, MERGE.
Advertisements

CONCEPTUAL WEB-BASED FRAMEWORK IN AN INTERACTIVE VIRTUAL ENVIRONMENT FOR DISTANCE LEARNING Amal Oraifige, Graham Oakes, Anthony Felton, David Heesom, Kevin.
Modelling with expert systems. Expert systems Modelling with expert systems Coaching modelling with expert systems Advantages and limitations of modelling.
Advances in the PARCC Mathematics Assessment August
Towards Adaptive Web-Based Learning Systems Katerina Georgouli, MSc, PhD Associate Professor T.E.I. of Athens Dept. of Informatics Tempus.
© Chinese University, CSE Dept. Software Engineering / Software Engineering Topic 1: Software Engineering: A Preview Your Name: ____________________.
Educational data mining overview & Introduction to Exploratory Data Analysis Ken Koedinger CMU Director of PSLC Professor of Human-Computer Interaction.
1 LearnLab: Bridging the Gap Between Learning Science and Educational Practice Ken Koedinger Human-Computer Interaction & Psychology, CMU PI & CMU Director.
Project Proposal.
Improving learning by improving the cognitive model: A data- driven approach Cen, H., Koedinger, K., Junker, B. Learning Factors Analysis - A General Method.
Collaborative Warrior Tutoring Tom Livak Neil Heffernan 8/24/06.
Building Intelligent Tutoring Systems with the Cognitive Tutor Authoring Tools (CTAT) Vincent Aleven, Jonathan Sewall, and the CTAT team.
An Individualized Web-Based Algebra Tutor D.Sklavakis & I. Refanidis 1 An Individualized Web-Based Algebra Tutor Based on Dynamic Deep Model Tracing Dimitrios.
Supporting (aspects of) self- directed learning with Cognitive Tutors Ken Koedinger CMU Director of Pittsburgh Science of Learning Center Human-Computer.
Toward the extraction of production rules for solving logic proofs Tiffany Barnes, John Stamper Computer Science.
1 CS 430 / INFO 430 Information Retrieval Lecture 24 Usability 2.
CLT Conference Heerlen Ron Salden, Ken Koedinger, Vincent Aleven, & Bruce McLaren (Carnegie Mellon University, Pittsburgh, USA) Does Cognitive Load Theory.
Educational Data Mining Overview John Stamper PSLC Summer School /25/2011 1PSLC Summer School 2011.
CS 1 with Robots CS1301 – Where it Fits Institute for Personal Robots in Education (IPRE)‏
Click to edit the title text format An Introduction to TuTalk: Developing Dialogue Agents for Learning Studies Pamela Jordan University of Pittsburgh Learning.
Educational Data Mining and DataShop John Stamper Carnegie Mellon University 1 9/12/2012 PSLC Corporate Partner Meeting 2012.
ADL Slide 1 December 15, 2009 Evidence-Centered Design and Cisco’s Packet Tracer Simulation-Based Assessment Robert J. Mislevy Professor, Measurement &
31 st October, 2012 CSE-435 Tashwin Kaur Khurana.
Building Intelligent Tutoring Systems with the Cognitive Tutor Authoring Tools (CTAT) Vincent Aleven and the CTAT team 7th Annual PSLC Summer School Pittsburgh,
1 Source: Bruce McLarenEducational Technologies WS 2006/07 Educational Technologies WS2006 Authoring Tools - CTAT Bruce M. McLaren Senior Researcher, DFKI.
Cookies, Spreadsheets, and Modeling: Dynamic, Interactive, Visual Science and Math Scott A. Sinex Prince George’s Community College Presented at Network.
1 Unit 4: One-Step Equations The Georgia Performance Standards Website.
PSLC DataShop Introduction Slides current to DataShop version John Stamper DataShop Technical Director.
Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team.
System Design: Designing the User Interface Dr. Dania Bilal IS582 Spring 2009.
Learning SQL with a Computerized Tutor (Centered on SQL-Tutor) Antonija Mitrovic (University of Canterbury) Presented by Danielle H. Lee.
CSA3212: User Adaptive Systems Dr. Christopher Staff Department of Computer Science & AI University of Malta Lecture 9: Intelligent Tutoring Systems.
CSCI 1101 Intro to Computers 3. Common Productivity Software.
Tuteurs cognitifs: La théorie ACT-R et les systèmes de production Roger Nkambou.
1 FlexTraining in a Nutshell Welcome to a brief introduction of the FlexTraining Total e- Learning Solution. This short sample course will outline the.
Extreme/Agile Programming Prabhaker Mateti. ACK These slides are collected from many authors along with a few of mine. Many thanks to all these authors.
Simulated Student: Building Cognitive Model by Demonstration Noboru Matsuda School of Computer Science Carnegie Mellon University.
Database Design and Management CPTG /23/2015Chapter 12 of 38 Functions of a Database Store data Store data School: student records, class schedules,
Evidence-based Practice Chapter 3 Ken Koedinger Based on slides from Ruth Clark 1.
Object-Oriented Analysis and Design Fall 2009.
Noboru Matsuda Human-Computer Interaction Institute
Using the Right Method to Collect Information IW233 Amanda Murphy.
1 USC Information Sciences Institute Yolanda GilFebruary 2001 Knowledge Acquisition as Tutorial Dialogue: Some Ideas Yolanda Gil.
Tutoring & Help System CSE-435 Nicolas Frantzen CSE-435 Nicolas Frantzen.
Spring 2011 Tutor Training Modern Learning Theories and Tutoring Designed and Presented by Tem Fuller.
Vincent Aleven & Kirsten Butcher Robust Learning in Visual/Verbal Problem Solving: Contiguity, Integrated Hints, and Elaborated Explanations.
Design Process … and some design inspiration. Course ReCap To make you notice interfaces, good and bad – You’ll never look at doors the same way again.
SimStudent: A computational model of learning for Intelligent Authoring and beyond Noboru Matsuda Human-Computer Interaction Institute Carnegie Mellon.
711: Intelligent Tutoring Systems Week 1 – Introduction.
M= Math in STEM College and Career Ready- Conference Summer, 2015.
SimStudent: Building a Cognitive Tutor by Teaching a Simulated Student Noboru Matsuda Human-Computer Interaction Institute Carnegie Mellon University.
Teaching Abstract Data Type Semantics with Multimedia* Glenn D. Blank, Edwin J. Kay, William M. Pottenger Jeffrey J. Heigl, Soma Roy and Shreeram A. Sahasrabudhe.
Data mining with DataShop Ken Koedinger CMU Director of PSLC Professor of Human-Computer Interaction & Psychology Carnegie Mellon University.
RULES Patty Nordstrom Hien Nguyen. "Cognitive Skills are Realized by Production Rules"
Mass Producing Example- Tracing Tutors Bruce McLaren Human-Computer Interaction Institute Carnegie Mellon University.
1 The Software Development Process ► Systems analysis ► Systems design ► Implementation ► Testing ► Documentation ► Evaluation ► Maintenance.
George Goguadze, Eric Andrès Universität des Saarlandes Johan Jeuring, Bastiaan Heeren Open Universiteit Nederland Generation of Interactive Exercises.
Helpful hints for planning your Wednesday investigation.
July 8, 2008In vivo experimentation: 1 Step by Step In Vivo Experimentation Lecture 3 for the IV track of the 2011 PSLC Summer School Philip Pavlik Jr.
#1 Make sense of problems and persevere in solving them How would you describe the problem in your own words? How would you describe what you are trying.
Learning Analytics isn’t new Ways in which we might build on the long history of adaptive learning systems within contemporary online learning design Professor.
Design Evaluation Overview Introduction Model for Interface Design Evaluation Types of Evaluation –Conceptual Design –Usability –Learning Outcome.
Mathematical Practice Standards
What is a CAT? What is a CAT?.
Inquiry learning and SimQuest
Intelligent Tutoring Systems
Big Data, Education, and Society
Simulated Student: Building Cognitive Model by Demonstration
CS1301 – Where it Fits Institute for Personal Robots in Education
Philip Pavlik Jr. University of Memphis
Presentation transcript:

What can CTAT do for you? Overview of the CTAT track Vincent Aleven, Bruce McLaren and the CTAT team 3rd Annual PSLC LearnLab Summer School Pittsburgh, June 18-22, 2007

What can CTAT do for you? - 2 3rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 If you are not in the CTAT track, why would you listen to this talk? CTAT relevant to most other tracks: –TuTalk: TuTalk dialogue tutor agent and CTAT intelligent tutor would make an awesome duo! –In Vivo: could do an in vivo experiment with CTAT-based tutors (happens all the time!) –Data Shop: many data sets in the Data Shop were generated using CTAT-built tutors –TagHelper: Hmm…. maybe you are naturally curious about CTAT Track hopping is allowed! –E.g., if in the In Vivo track, could attend (some) CTAT sessions

What can CTAT do for you? - 3 3rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Overview What is “a tutor?” –And would you want to build one? Why do you need CTAT to build one? What more precisely can you do with CTAT? –Short movie of authoring with CTAT –Examples of projects that have used CTAT –Key choices to make when authoring with CTAT Planned activities in the CTAT track

What can CTAT do for you? - 4 3rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 A simple example: Fraction Addition Interact with the fraction addition tutor on the CTAT web site …

What can CTAT do for you? - 5 3rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Key elements of interactivity in CTAT Tutors 1.Problem-solving environment, often with interactive tools 2.Tutorial guidance, in the form of a. implicit yes-no feedback on correctness of problem steps b. feedback messages for commonly-occurring errors c. next-step hints (on demand or tutor initiated) ==> can handle multiple student strategies for solving problems 3.Adaptive problem selection based on student performance solving problems with the tutor Koedinger, K. R., & Aleven V. (in press). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review.

What can CTAT do for you? - 6 3rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Kinds of Computer Tutors Intelligent tutoring systems e.g., Sherlock Model-tracing tutors e.g., Andes Cognitive Tutors e.g., Algebra Tutoring systems CAI e.g., Microsoft’s Personal Tutor Constraint- based tutors e.g., SQL Tutor Example-tracing tutors e.g., Stoichiometry, French Culture Tutor Can be built with CTAT

What can CTAT do for you? - 7 3rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Cognitive Tutors: Real-world success of Intelligent Tutoring Systems technology Cognitive Tutor Courses –Goal: Improve quantitative literacy & fluency –Computer-based one-to-one tutoring Based on computational models of student thinking Based on the concept of Model Tracing Used by many students in many schools –Algebra Cognitive Tutor: In over 2,000 schools in the USA, 300,000 students per year –Geometry Cognitive Tutor: In 350 schools –Most widely used intelligent tutoring system? University created a company, Carnegie Learning, to disseminate

Algebra Cognitive Tutor Sample Use graphs, graphics calculator Analyze real world problem scenarios Use table, spreadsheet Use equations, symbolic calculator Tutor learns about each student Tutor follows along, provides context-sensitive Instruction

What can CTAT do for you? - 9 3rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Replicated Field Studies Full year classroom experiments Replicated over 3 years in urban schools In Pittsburgh & Milwaukee Results: % better on problem solving & representation use % better on standardized tests. Koedinger, Anderson, Hadley, & Mark (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 ACT-R: A Cognitive Theory of Learning and Performance Big theory … key tenets: –Learning by doing, not by listening or watching –Production rules represent performance knowledge: These units are: Instruction implications: modular context specific isolate skills, concepts, strategies address "when" as well as "how" Anderson, J.R., & Lebiere, C. (1998). The Atomic Components of Thought. Erlbaum.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Cognitive Model: A system that can solve problems in the various ways students can Strategy 1: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + ac = d Strategy 2: IF the goal is to solve a(bx+c) = d THEN rewrite this as bx + c = d/a Misconception: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + c = d Cognitive Tutor Technology: Use ACT-R theory to individualize instruction

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, (2x - 5) = 9 6x - 15 = 92x - 5 = 36x - 5 = 9 Cognitive Tutor Technology: Use ACT-R theory to individualize instruction Cognitive Model: A system that can solve problems in the various ways students can If goal is solve a(bx+c) = d Then rewrite as abx + ac = d If goal is solve a(bx+c) = d Then rewrite as abx + c = d If goal is solve a(bx+c) = d Then rewrite as bx+c = d/a Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, (2x - 5) = 9 6x - 15 = 92x - 5 = 36x - 5 = 9 Cognitive Tutor Technology: Use ACT-R theory to individualize instruction Cognitive Model: A system that can solve problems in the various ways students can If goal is solve a(bx+c) = d Then rewrite as abx + ac = d If goal is solve a(bx+c) = d Then rewrite as abx + c = d Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction Hint message: “Distribute a across the parentheses.” Bug message: “You need to multiply c by a also.” Knowledge Tracing: Assesses student's knowledge growth -> individualized activity selection and pacing Known? = 85% chanceKnown? = 45%

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 CTAT motivation: Make tutor development easier and faster! Development costs of instructional technology are high –E.g., ~300 dev hours per hour of instruction for Computer Aided Instruction (Murray, 1999) Cognitive Tutors: –Large student learning gains as a result of detailed cognitive modeling –~200 dev hours per hour of instruction (Koedinger et al, 1997) –Requires PhD level cog scientists and AI programmers Solution: Easy to use Cognitive Tutor Authoring Tools (CTAT) Murray, T. (1999). Authoring Intelligent Tutoring Systems: An Analysis of the state of the art. The International Journal of Artificial Intelligence in Education, 10, Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. The International Journal of Artificial Intelligence in Education, 8,

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 CTAT goal: broaden the group of targeted authors Instructional technology developers (e.g., instructional media dept. at university, or developers of on-line courses) Researchers interested in intelligent tutoring systems Instructors (e.g., computer-savvy college professors) Learning sciences researchers interested in using computer-based tutors in their experiments –Within the PSLC, CTAT-based tutors are often used in in vivo experiments

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 How to reduce the authoring cost? Less programming, more automation –Drag & drop interface construction –Demonstration-based programming –New: Machine learning and data mining Human-Computer Interaction (HCI) methods –User studies, summer schools, informal & formal comparison studies Exploit tools already in use –Component-based architecture with “standard” tool-tutor protocol –Off-the-shelf tools and languages (e.g., Netbeans, Eclipse, Flash, Jess)

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 CTAT supports two types of tutors Cognitive Tutors and Example-Tracing Tutors Behavior is similar (though not identical), both support the same elements of interactivity: –problem-solving environment with tutorial guidance: –yes/no feedback on steps (can handle multiple student strategies!) –Specific feedback messages for common errors –Next-step hints Technology and development are very different –Example-Tracing Tutors: built through “programming by demonstration” - authors creates examples of how the students should solve problems –Cognitive Tutors: built by AI programming (much harder) - author creates rule-based cognitive model

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Movie Showing How an Example- Tracing Tutor is built

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Authoring with CTAT and Flash Behavior Recorder Flash authoring environment with tutor interface shown Web browser with Example-Tracing tutor embedded in HTML

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Authoring an Example-Tracing Tutor Step 1: Create a User Interface –Create the graphical user interface (GUI) used by the student Step 2: Demonstrate Behavior – Demonstrate correct, alternative correct, and incorrect solutions Step 3: Generalize – Specify how demonstrated behavior could vary within given problem allowed order of steps allowed variants for a given step Step 4: Annotate the Graph –Annotate solutions steps in the resulting “behavior graph” with: hint messages, error messages, labels for concepts or skills associated with actions Test and Iterate on Steps 1-4 … Step 5: Publish Tutor to Web

Behavior Recorder Data Shop CTAT’s Modular Architecture Student Interface (or external problem-solving environment) Cool Modes CyclePad Java Swing Flash Learner Management System LMS Tutor Shop Cognitive Model Development Tools TDK (Lisp-based) Jess tools (Java- based) Editor Eclipse Tutor Engine Tertle (Lisp-based) + Model Tracer Jess (Java-based) + Model Tracer Example Tracer (Java-based) Example Tracer (Flash-based) GUI Builder IntelliJ Dreamweaver Code Warrior Netbeans Flash MX 2004

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 DataShop - Inspection & Evaluation of CTAT Data Learning Curve Report

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 DataShop - Inspection & Evaluation of CTAT Data Error Report

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 CTAT’s Track Record CTAT-based tutors have been used in (approx. 15) experiments in Geometry, Chemistry, Chinese, and French Over 300 users (you not included) Four releases in past year (1.6, 2.0, 2.1, 2.2) CTAT Web site –35,000 unique visitors in 2006, 580,000 hits CTAT Downloads –2,200 in 2006 (14% CMU & Pitt)

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team,

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Development time comparison Past estimates of ITS development 200:1 for robust beta versions used in real classrooms Example-Tracing Tutor Development 25:1 for initial alpha versions: (Koedinger et al, 2004) If creating a robust beta doubles dev time, we would have ~50:1 ratio -- a reduction of 4 times! Koedinger, K., Aleven, V., Heffernan, N., McLaren, B. M., and Hockenberry, M (2004). Opening the Door to Non-Programmers: Authoring Intelligent Tutor Behavior by Demonstration;. In the Proceedings of the Seventh International Conference on Intelligent Tutoring Systems (ITS-2004).

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Where does CTAT fit into the Realm of Authoring Tools? Murray survey (1999) discusses 7 categories of authoring tools for intelligent tutoring systems (ITS), including: Device Simulation - RIDES (Munro et al 1997) Tutoring Strategies - REDEEM (Ainsworth et al 2003; Major et al 1997) Domain Expert System - Demonstr8 (Blessing 1997) Cognitive Tutor Authoring Tools: “Domain Expert System” category of authoring systems No commercially successful tools for authoring model-tracing, Cognitive Tutors First to provide functionality to simplify & speed cog. tutor development

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Other Example-Tracing Tutors built with CTAT Chinese - tone study Chinese - listen and read French culture French Chemistry (stoichiometry) Geometry (self-assessment) Geometry (on-line testing)

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Chinese: tone study Wang, Perfetti, Liu, et al.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Stoichiometry study McLaren et al.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Geometry self-assessment tutor Roll et al.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 French culture Ogan et al.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Help seeking Aleven, McLaren, Roll, et al.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 French Jones et al.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Chinese: listen and read Mei, Haney, et al.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Division Tutor (Stefan King)

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Mass Production: Create template behavior graph and spreadsheet with data for each problem Each row contains values for one variable in the Behavior Graph template Each column contains data for 1 problem A Behavior Graph is created automatically for each problem by merging the spreadsheet data into the template

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 What is Mass Production? Many learning situations require repetition of similar problems, but authoring many similar Example-Tracing Tutors is hard –It is time-consuming to demonstrate individual problems –Inconsistency between problems is often introduced Examples: –Practice fraction addition problems in a math class –Drill on proper use of articles in a language class CTAT mass production allows us to use a single demonstration as the basis for many tutored problems of the same type

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 The Mass Production process

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Self-explanation of incorrect examples Julie Booth et al.

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Another example - Thermodynamics Simulator hooked up with CTAT CTAT provides tutoring for analyzing power plant efficiency

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 What type of tutor shall I build? Cognitive Tutors v. Example-Tracing Tutors Cognitive Tutor (e.g., Algebra Cognitive Tutor, Logic Tutor) –Technology: uses rule-based cognitive model to monitor students (implemented in Jess) –Development: Difficult to build; requires AI programming –Advantages: General for a class of problems; can deal with subtle dependencies of steps, many solution paths, etc. –Sometimes an advantage: SimStudents module writes rules for you (experimental use of machine learning; tried on a small scale) Example-Tracing Tutors (the other examples in this talk) –Technology: uses examples of problem solutions to monitor students –Development: programming by demonstration, “Mass Production” facilitates authoring of many isomorphic problems (in Excel) –Advantages: Much easier to build; for non-programmers

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 How to make the choice? Often, example-tracing tutors are a good choice –Easier to build –At this point in time, their web delivery is better supported Cognitive Tutors only if –You have (AI) programming experience (possibly gained during this summer school), and –You deal with problems that have subtle dependencies among steps, or many different solutions paths Best to build an example-tracing tutor first –Rapid prototype! And supports cognitive task analysis –Then decide whether to move on to Cognitive Tutor –Example-Tracing Tutors re-usable when developing a Cognitive Tutor: Examples guide planning, serve as semi-automated test cases

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 During the summer school Learning to create Example-Tracing Tutors is easy, learning to create Cognitive Tutors is hard The CTAT track will cover development of Cognitive Tutors and Example-Tracing Tutors –In your project you could decide to focus mainly on example- tracing tutors –(Then again, this is your chance to get some mentoring as you build a Cognitive Tutor) If you are not in the CTAT track, but interested in learning to build tutors in limited time, it is best to focus on example-tracing tutors

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Options for developing a user interface for your tutor (no programming!) Java with Netbeans GUI Builder –Freely available –Web delivery via WebStart (we are still gaining experience with this) Flash (from Adobe) –Easy to deploy on the web: With plug-in, compatible with many browsers –Not free

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Choices, choices, … $$$ may be a factor … Whether you know Flash or Java need not be a factor –Unless you need to program custom behaviors Web delivery of Cognitive Tutors only supported with Java, but web delivery of Example-Tracing tutors is easier in Flash Sets of widgets/components are different See also : Our recommendation: use Flash if you can

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 During the summer school Examples shown in demos and lectures will come mainly from Java / Netbeans For your projects, you are free to choose – If the long-term prospects for your project are uncertain, then the available widgets/components should be decisive –Otherwise, all factors mentioned on the previously slide should probably be considered

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Summary of choices to be made when using CTAT for real Cognitive Tutor or Example-Tracing Tutors? –If Cognitive Tutors, use SimStudents? –If Example-Tracing Tutors, use Mass Production? Tutor interface in Flash or Java? –Also possible to program your own

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Overview of CTAT track activities

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Demo of CTAT

What can CTAT do for you? rd LearnLab Summer School, Pittsburgh, PA, June 18-22, 2007© Vincent Aleven, Bruce McLaren, & the CTAT Team, 2007 Thank you!