Modeling Student Benefits from Illustrations and Graphs Michael Lipschultz Diane Litman Computer Science Department University of Pittsburgh.

Slides:

Advertisements

Similar presentations

Detecting Statistical Interactions with Additive Groves of Trees

Advertisements

CPSC 502, Lecture 15Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 15 Nov, 1, 2011 Slide credit: C. Conati, S.

The Q-matrix method: A new artificial intelligence tool for data mining Dr. Tiffany Barnes Kennedy 213, PhD - North Carolina State University.

Prediction with Regression

Data Mining Methodology 1. Why have a Methodology  Don’t want to learn things that aren’t true May not represent any underlying reality ○ Spurious correlation.

Ryan S.J.d. Baker Adam B. Goldstein Neil T. Heffernan Detecting the Moment of Learning.

Uncertainty Corpus: Resource to Study User Affect in Complex Spoken Dialogue Systems Kate Forbes-Riley, Diane Litman, Scott Silliman, Amruta Purandare.

Week 8 Video 4 Hidden Markov Models.

January 6, afternoon session 1 Statistics Micro Mini Multiple Regression January 5-9, 2008 Beth Ayers.

Studying Behavior. Variables Any event, situation, behavior or individual characteristic that varies In our context, the “things” that make up an experiment.

Chan & Chou’s system Chan, T.-W., & Chou, C.-Y. (1997). Exploring the design of computer supports for reciprocal tutoring. International Journal of Artificial.

Taxonomy of Effortless Creation of Algorithm Visualizations Petri Ihantola, Ville Karavirta, Ari Korhonen and Jussi Nikander HELSINKI UNIVERSITY OF TECHNOLOGY.

Contrasting Examples in Mathematics Lessons Support Flexible and Transferable Knowledge Bethany Rittle-Johnson Vanderbilt University Jon Star Michigan.

Lasso regression. The Goals of Model Selection Model selection: Choosing the approximate best model by estimating the performance of various models Goals.

+ Doing More with Less : Student Modeling and Performance Prediction with Reduced Content Models Yun Huang, University of Pittsburgh Yanbo Xu, Carnegie.

CLT Conference Heerlen Ron Salden, Ken Koedinger, Vincent Aleven, & Bruce McLaren (Carnegie Mellon University, Pittsburgh, USA) Does Cognitive Load Theory.

Topics = Domain-Specific Concepts Online Physics Encyclopedia ‘Eric Weisstein's World of Physics’ Contains total 3040 terms including multi-word concepts.

1 of 27 PSYC 4310/6310 Advanced Experimental Methods and Statistics © 2013, Michael Kalsher Michael J. Kalsher Department of Cognitive Science Adv. Experimental.

1. An Overview of the Data Analysis and Probability Standard for School Mathematics? 2.

Author: Fang Wei, Glenn Blank Department of Computer Science Lehigh University July 10, 2007 A Student Model for an Intelligent Tutoring System Helping.

Evaluating Adaptive Generation of Problems in Programming Tutors – Two Studies Amruth KUMAR Ramapo College of New Jersey, Mahwah, NJ 07430, USA.

Modeling User Satisfaction and Student Learning in a Spoken Dialogue Tutoring System with Generic, Tutoring, and User Affect Parameters Kate Forbes-Riley.

Chapter 4 Correlation and Regression Understanding Basic Statistics Fifth Edition By Brase and Brase Prepared by Jon Booze.

Section Copyright © 2014, 2012, 2010 Pearson Education, Inc. Lecture Slides Elementary Statistics Twelfth Edition and the Triola Statistics Series.

Irkutsk State Medical University Department of Faculty Therapy Correlations Khamaeva A. A. Irkutsk, 2009.

Discovering the Intrinsic Cardinality and Dimensionality of Time Series using MDL BING HU THANAWIN RAKTHANMANON YUAN HAO SCOTT EVANS1 STEFANO LONARDI EAMONN.

Statistics in Applied Science and Technology Chapter 13, Correlation and Regression Part I, Correlation (Measure of Association)

1 Multimedia-Supported Metaphors for Meaning Making in Mathematics Moreno & Mayer (1999)

A Meta-Study of Algorithm Visualization Effectiveness Christopher Hundhausen, Sarah Douglas, John Stasko Presented by David Burlinson 8/10/2015.

Memory and Analogy in Game-Playing Agents Jonathan Rubin & Ian Watson University of Auckland Game AI Group

What is science? an introduction to life science.

Exploiting Context Analysis for Combining Multiple Entity Resolution Systems -Ramu Bandaru Zhaoqi Chen Dmitri V.kalashnikov Sharad Mehrotra.

Incorporating Self Regulated Learning Techniques into Learning by Teaching Environments Biswas, G., Leelawong, K. and Belynne, K. The twenty sixth Annual.

Developing Learning by Teaching Environments that support Self-Regulated Learning Gautam Biswas, Krittaya Leelawong, Kadira Belynne, Karun Viswanath, Daniel.

Learning Gains in Physics in Relation to Students' Mathematics Skills David E. Meltzer Department of Physics and Astronomy Iowa State University.

© Copyright McGraw-Hill Correlation and Regression CHAPTER 10.

The Persistence of the Gender Gap in Introductory Physics Lauren Kost Steven Pollock, Noah Finkelstein Department of Physics, University of Colorado at.

Predicting Student Emotions in Computer-Human Tutoring Dialogues Diane J. Litman&Kate Forbes-Riley University of Pittsburgh Department of Computer Science.

Modeling Student Benefits from Illustrations and Graphs Michael Lipschultz Diane Litman Intelligent Tutoring Systems Conference (2014)

Paired Sampling in Density-Sensitive Active Learning Pinar Donmez joint work with Jaime G. Carbonell Language Technologies Institute School of Computer.

Effects of an online problem based learning course on content knowledge acquisition and critical thinking skills Presenter: Han, Yi-Ti Adviser: Chen, Ming-Puu.

Boundary Detection in Tokenizing Network Application Payload for Anomaly Detection Rachna Vargiya and Philip Chan Department of Computer Sciences Florida.

Speech and Language Processing for Educational Applications Professor Diane Litman Computer Science Department & Intelligent Systems Program & Learning.

Section Copyright © 2014, 2012, 2010 Pearson Education, Inc. Chapter 10 Correlation and Regression 10-2 Correlation 10-3 Regression.

1 Hypermedia learning and prior knowledge: domain expertise vs. system expertise. Timothy J. F. Mitchell, Sherry Y. Chen & Robert D. Macredie. (2005) Hypermedia.

1 Fuzzy Versus Quantitative Association Rules: A Fair Data-Driven Comparison Shih-Ming Bai and Shyi-Ming Chen Department of Computer Science and Information.

A comparative approach for gene network inference using time-series gene expression data Guillaume Bourque* and David Sankoff *Centre de Recherches Mathématiques,

Learning Photographic Global Tonal Adjustment with a Database of Input / Output Image Pairs.

Metacognition and Learning in Spoken Dialogue Computer Tutoring Kate Forbes-Riley and Diane Litman Learning Research and Development Center University.

A Tutorial Dialogue System that Adapts to Student Uncertainty Diane Litman Computer Science Department & Intelligent Systems Program & Learning Research.

Improving (Meta)cognitive Tutoring by Detecting and Responding to Uncertainty Diane Litman & Kate Forbes-Riley University of Pittsburgh Pittsburgh, PA.

12b - 1 © 2000 Prentice-Hall, Inc. Statistics Multiple Regression and Model Building Chapter 12 part II.

Using Natural Language Processing to Analyze Tutorial Dialogue Corpora Across Domains and Modalities Diane Litman, University of Pittsburgh, Pittsburgh,

Prosodic Cues to Disengagement and Uncertainty in Physics Tutorial Dialogues Diane Litman, Heather Friedberg, Kate Forbes-Riley University of Pittsburgh.

Slide Slide 1 Chapter 10 Correlation and Regression 10-1 Overview 10-2 Correlation 10-3 Regression 10-4 Variation and Prediction Intervals 10-5 Multiple.

EXPERT SYSTEMS BY MEHWISH MANZER (63) MEER SADAF NAEEM (58) DUR-E-MALIKA (55)

Predicting Emotion in Spoken Dialogue from Multiple Knowledge Sources Kate Forbes-Riley and Diane Litman Learning Research and Development Center and Computer.

Automated feedback in statistics education

Effects of User Similarity in Social Media Ashton Anderson Jure Leskovec Daniel Huttenlocher Jon Kleinberg Stanford University Cornell University Avia.

Data Mining CAS 2004 Ratemaking Seminar Philadelphia, Pa.

Towards Emotion Prediction in Spoken Tutoring Dialogues

Dialogue-Learning Correlations in Spoken Dialogue Tutoring

Mingyu Feng Neil Heffernan Joseph Beck

Chapter 10 Correlation and Regression

Correlation and Regression

Unit: Science & Technology

Educational Data Mining Success Stories

CS639: Data Management for Data Science

Dr. Fowler  AFM  Unit 8-5 Linear Correlation

Simulation-driven Enterprise Modelling: WHY ?

Presentation transcript:

Modeling Student Benefits from Illustrations and Graphs Michael Lipschultz Diane Litman Computer Science Department University of Pittsburgh

(VanLehn et al, 05) (Rau et al, 09) (Kozhevnikov et al, 07) (Graesser et al, 05) Some ITSes use visuals [Graesser et al, 05; VanLehn et al, 05; Rau et al, 09] Benefits of – Illustrations – Graphs Visually Conveying Concepts in Intelligent Tutoring Systems 2 Graphs Illustrations

Visuals used to convey concepts [Graesser et al, 05; VanLehn et al, 05; Rau et al, 09] Benefits of – Illustrations – Graphs Visually Conveying Concepts in Intelligent Tutoring Systems 3 Graphs Illustrations

Visuals used to convey concepts [Graesser et al, 05; VanLehn et al, 05; Rau et al, 09] Relatable [Kozhevnikov et al, 07; Goldstone et al, 12] Better transfer [Leelawong & Biswas, 08; Van Heuvelen & Zou, 01] Introduction 4 Graphs Illustrations Frequent switching [Rau et al., 09; Rau et al., 12a,b] Concreteness fading [Goldstone & Son, 05; McNeil & Fyfe, 12]

Relatable [Kozhevnikov et al, 07; Goldstone et al, 12] Better transfer [Leelawong & Biswas, 08; Van Heuvelen & Zou, 01] Illustrations vs. Graphs 5 GraphsIllustrations Frequent switching [Rau et al., 09; Rau et al., 12a,b] Concreteness fading [Goldstone & Son, 05; McNeil & Fyfe, 12]

Best representation varies – Gender [Meltzer, 05] – Knowledge [Kozhevnikov et al, 07; Maries et al, 12] – Skills [Dancy & Beichner, 06; Kozhevnikov et al, 07; Velez et al, 05; Conati & Maclaren, 08] Adapt to student & context – Successful in uncertainty & motivation -> persistence & better learning gains [Aist et al, 02; Forbes-Riley & Litman, 11] In this work: develop modeling approach – Model better than baseline – Model with interesting properties Adapt to Student 6

Outline Introduction Data Modeling – Augmented Stepwise Regression Quantitative Results Best Model Conclusions and Future Work 7

Data Prior study [Lipschultz & Litman, 13] – Problem-solving [VanLehn et al, 05] + post-problem discussion [Katz et al, 11] – Saw either illustrations only or graphs only – Pretest & Post-test – to measure learning gains – 29 subjects: 2,042 data points (dialogue answers) Features: – Student information [Arroyo et al, 00; Chi et al, 11; Lipschultz & Litman, 11] – Student skill [Chi et al, 11; Lipschultz & Litman, 11] – Domain information [Baschera et al, 11; Lipschultz & Litman, 11; Ward & Litman, 06] – Contextual information [Baker et al; Beck, 06; Drummond & Litman, 10; D'Mello & Graesser, 10] 8

Modeling Don’t know best visual – Know: visual seen & amount learned Infer best visual from learning gains Regression to predict learning – Syntactic constraints for useful model – Goal: good model that is useful 9

Modeling with Stepwise Regression Algorithm 1.Stepwise Linear Regression – Convert features to binary 2.Identify Problematic Rules – Mutually Exclusive – Non-Adaptive 3.Handle Problematic Rules – Remove Lesser Rule in Pair 4.Relearn Model – Regular Regression 5.Rank by absolute value of coefficient 10

Modeling with Stepwise Regression 1. Stepwise Linear Regression Postscore = terms + prescore terms: β i *representation*(tutoring context) β i *representation*partition*rule – β i * Illustration * (PreScore=High) * (ResponseTime=Fast) – For high pretesters, when ResponseTime=Fast, show illustrations – Binary features Learns coefficients (β i ’s) Keeps only predictive terms 11

Modeling with Stepwise Regression Example Starting Set of Terms (72) Illustration*(PreScore=High)*(WalkThruPct=Low) Illustration*(PreScore=High)*(WalkThruPct=High) Illustration*(PreScore=High)*(RQPctCorrect=Low) Illustration*(PreScore=High)*(RQPctCorrect=High) Illustration*(PreScore=High)*(PctSituationCorrect=Low) Illustration*(PreScore=High)*(PctSituationCorrect=High) Illustration*(PreScore=High)*(PctSessionCorrect=Low) Illustration*(PreScore=High)*(PctSessionCorrect=High) Illustration*(PreScore=High)*(PctThruSituation=Early) Illustration*(PreScore=High)*(PctThruSituation=Late) Illustration*(PreScore=High)*(PctThruSession=Early) Illustration*(PreScore=High)*(PctThruSession=Late) Illustration*(PreScore=High)*(KCUsage=State) Illustration*(PreScore=High)*(KCUsage=Apply) Illustration*(PreScore=High)*(ItemDiff=Easy) Illustration*(PreScore=High)*(ItemDiff=Hard) Illustration*(PreScore=High)*(ResponseTime=Fast) Illustration*(PreScore=High)*(ResponseTime=Slow) Graph*(PreScore=High)*(WalkThruPct=Low) Graph*(PreScore=High)*(WalkThruPct=High) Graph*(PreScore=High)*(RQPctCorrect=Low) Graph*(PreScore=High)*(RQPctCorrect=High) Graph*(PreScore=High)*(PctSituationCorrect=Low) Graph*(PreScore=High)*(PctSituationCorrect=High) Graph*(PreScore=High)*(PctSessionCorrect=Low) Graph*(PreScore=High)*(PctSessionCorrect=High) Graph*(PreScore=High)*(PctThruSituation=Early) Graph*(PreScore=High)*(PctThruSituation=Late) Graph*(PreScore=High)*(PctThruSession=Early) Graph*(PreScore=High)*(PctThruSession=Late) Graph*(PreScore=High)*(KCUsage=State) Graph*(PreScore=High)*(KCUsage=Apply) Graph*(PreScore=High)*(ItemDiff=Easy) Graph*(PreScore=High)*(ItemDiff=Hard) Graph*(PreScore=High)*(ResponseTime=Fast) Graph*(PreScore=High)*(ResponseTime=Slow) Illustration*(PreScore=Low)*(WalkThruPct=Low) Illustration*(PreScore=Low)*(WalkThruPct=High) Illustration*(PreScore=Low)*(RQPctCorrect=Low) Illustration*(PreScore=Low)*(RQPctCorrect=High) Illustration*(PreScore=Low)*(PctSituationCorrect=Low) Illustration*(PreScore=Low)*(PctSituationCorrect=High) Illustration*(PreScore=Low)*(PctSessionCorrect=Low) Illustration*(PreScore=Low)*(PctSessionCorrect=High) Illustration*(PreScore=Low)*(PctThruSituation=Early) Illustration*(PreScore=Low)*(PctThruSituation=Late) Illustration*(PreScore=Low)*(PctThruSession=Early) Illustration*(PreScore=Low)*(PctThruSession=Late) Illustration*(PreScore=Low)*(KCUsage=State) Illustration*(PreScore=Low)*(KCUsage=Apply) Illustration*(PreScore=Low)*(ItemDiff=Easy) Illustration*(PreScore=Low)*(ItemDiff=Hard) Illustration*(PreScore=Low)*(ResponseTime=Fast) Illustration*(PreScore=Low)*(ResponseTime=Slow) Graph*(PreScore=Low)*(WalkThruPct=Low) Graph*(PreScore=Low)*(WalkThruPct=High) Graph*(PreScore=Low)*(RQPctCorrect=Low) Graph*(PreScore=Low)*(RQPctCorrect=High) Graph*(PreScore=Low)*(PctSituationCorrect=Low) Graph*(PreScore=Low)*(PctSituationCorrect=High) Graph*(PreScore=Low)*(PctSessionCorrect=Low) Graph*(PreScore=Low)*(PctSessionCorrect=High) Graph*(PreScore=Low)*(PctThruSituation=Early) Graph*(PreScore=Low)*(PctThruSituation=Late) Graph*(PreScore=Low)*(PctThruSession=Early) Graph*(PreScore=Low)*(PctThruSession=Late) Graph*(PreScore=Low)*(KCUsage=State) Graph*(PreScore=Low)*(KCUsage=Apply) Graph*(PreScore=Low)*(ItemDiff=Easy) Graph*(PreScore=Low)*(ItemDiff=Hard) Graph*(PreScore=Low)*(ResponseTime=Fast) Graph*(PreScore=Low)*(ResponseTime=Slow) Terms from Step 1 (~10) *Illustration*(PreScore=High)*(ResponseTime=Fast) *Graph*(PreScore=High)*(ResponseTime=Fast) *Illustration*(PreScore=Low)*(PctThruTutoring=High) *Graph*(PreScore=Low)*(PctThruTutoring=High) *Illustration*(PreScore=High)*(PctSessionCorrect=High) *Illustration*(PreScore=High)*(PctSessionCorrect=Low) *Illustration*(PreScore=High)*(KCUsage=Apply) *Illustration*(PreScore=High)*(ItemDiff=Easy) *Graph*(PreScore=Low)*(PctThruSituation=High) *Graph*(PreScore=Low)*(PctWalkThru=Low) 12

Modeling with Stepwise Regression Example Starting Set of Terms (72) Illustration*(PreScore=High)*(WalkThruPct=Low) Illustration*(PreScore=High)*(WalkThruPct=High) Illustration*(PreScore=High)*(RQPctCorrect=Low) Illustration*(PreScore=High)*(RQPctCorrect=High) Illustration*(PreScore=High)*(PctSituationCorrect=Low) Illustration*(PreScore=High)*(PctSituationCorrect=High) Illustration*(PreScore=High)*(PctSessionCorrect=Low) Illustration*(PreScore=High)*(PctSessionCorrect=High) Illustration*(PreScore=High)*(PctThruSituation=Early) Illustration*(PreScore=High)*(PctThruSituation=Late) Illustration*(PreScore=High)*(PctThruSession=Early) Illustration*(PreScore=High)*(PctThruSession=Late) Illustration*(PreScore=High)*(KCUsage=State) Illustration*(PreScore=High)*(KCUsage=Apply) Illustration*(PreScore=High)*(ItemDiff=Easy) Illustration*(PreScore=High)*(ItemDiff=Hard) Illustration*(PreScore=High)*(ResponseTime=Fast) Illustration*(PreScore=High)*(ResponseTime=Slow) Graph*(PreScore=High)*(WalkThruPct=Low) Graph*(PreScore=High)*(WalkThruPct=High) Graph*(PreScore=High)*(RQPctCorrect=Low) Graph*(PreScore=High)*(RQPctCorrect=High) Graph*(PreScore=High)*(PctSituationCorrect=Low) Graph*(PreScore=High)*(PctSituationCorrect=High) Graph*(PreScore=High)*(PctSessionCorrect=Low) Graph*(PreScore=High)*(PctSessionCorrect=High) Graph*(PreScore=High)*(PctThruSituation=Early) Graph*(PreScore=High)*(PctThruSituation=Late) Graph*(PreScore=High)*(PctThruSession=Early) Graph*(PreScore=High)*(PctThruSession=Late) Graph*(PreScore=High)*(KCUsage=State) Graph*(PreScore=High)*(KCUsage=Apply) Graph*(PreScore=High)*(ItemDiff=Easy) Graph*(PreScore=High)*(ItemDiff=Hard) Graph*(PreScore=High)*(ResponseTime=Fast) Graph*(PreScore=High)*(ResponseTime=Slow) Illustration*(PreScore=Low)*(WalkThruPct=Low) Illustration*(PreScore=Low)*(WalkThruPct=High) Illustration*(PreScore=Low)*(RQPctCorrect=Low) Illustration*(PreScore=Low)*(RQPctCorrect=High) Illustration*(PreScore=Low)*(PctSituationCorrect=Low) Illustration*(PreScore=Low)*(PctSituationCorrect=High) Illustration*(PreScore=Low)*(PctSessionCorrect=Low) Illustration*(PreScore=Low)*(PctSessionCorrect=High) Illustration*(PreScore=Low)*(PctThruSituation=Early) Illustration*(PreScore=Low)*(PctThruSituation=Late) Illustration*(PreScore=Low)*(PctThruSession=Early) Illustration*(PreScore=Low)*(PctThruSession=Late) Illustration*(PreScore=Low)*(KCUsage=State) Illustration*(PreScore=Low)*(KCUsage=Apply) Illustration*(PreScore=Low)*(ItemDiff=Easy) Illustration*(PreScore=Low)*(ItemDiff=Hard) Illustration*(PreScore=Low)*(ResponseTime=Fast) Illustration*(PreScore=Low)*(ResponseTime=Slow) Graph*(PreScore=Low)*(WalkThruPct=Low) Graph*(PreScore=Low)*(WalkThruPct=High) Graph*(PreScore=Low)*(RQPctCorrect=Low) Graph*(PreScore=Low)*(RQPctCorrect=High) Graph*(PreScore=Low)*(PctSituationCorrect=Low) Graph*(PreScore=Low)*(PctSituationCorrect=High) Graph*(PreScore=Low)*(PctSessionCorrect=Low) Graph*(PreScore=Low)*(PctSessionCorrect=High) Graph*(PreScore=Low)*(PctThruSituation=Early) Graph*(PreScore=Low)*(PctThruSituation=Late) Graph*(PreScore=Low)*(PctThruSession=Early) Graph*(PreScore=Low)*(PctThruSession=Late) Graph*(PreScore=Low)*(KCUsage=State) Graph*(PreScore=Low)*(KCUsage=Apply) Graph*(PreScore=Low)*(ItemDiff=Easy) Graph*(PreScore=Low)*(ItemDiff=Hard) Graph*(PreScore=Low)*(ResponseTime=Fast) Graph*(PreScore=Low)*(ResponseTime=Slow) Terms from Step 1 (~10) *Illustration*(PreScore=High)*(ResponseTime=Fast) *Graph*(PreScore=High)*(ResponseTime=Fast) *Illustration*(PreScore=Low)*(PctThruTutoring=Later) *Graph*(PreScore=Low)*(PctThruTutoring=Later) *Graph*(PreScore=Low)*(ResponseTime=Fast) *Graph*(PreScore=Low)*(ResponseTime=Slow) *Illustration*(PreScore=High)*(KCUsage=Apply) *Illustration*(PreScore=High)*(ItemDiff=Easy) *Graph*(PreScore=Low)*(PctThruSituation=High) *Graph*(PreScore=Low)*(PctWalkThru=Low) 13 But some of these rules don’t make sense…

Modeling with Stepwise Regression Algorithm 1.Stepwise Linear Regression – Convert features to binary 2.Identify Problematic Rules – Mutually Exclusive – Non-Adaptive 3.Handle Problematic Rules – Remove Lesser Rule in Pair 4.Relearn Model – Regular Regression 5.Rank by absolute value of coefficient 14

Modeling with Stepwise Regression 2. Identify Problematic Rules Mutually Exclusive Rules Non-Adaptive Rules *Illustration*(PreScore=High)*(ResponseTime=Fast) 0.789*Graph *(PreScore=High)*(ResponseTime=Fast) Same context Different Representation 0.456*Graph*(PreScore=Low)*(ResponseTime=Fast) 0.123*Graph*(PreScore=Low)*(ResponseTime=Slow) Same Representation Opposite Contexts

Modeling with Stepwise Regression Algorithm 1.Stepwise Linear Regression – Convert features to binary 2.Identify Problematic Rules – Mutually Exclusive – Non-Adaptive 3.Handle Problematic Rules – Remove Lesser Rule in Pair 4.Relearn Model – Regular Regression 5.Rank by absolute value of coefficient 16

Modeling with Stepwise Regression 3. Handling Problematic Rules Mutually Exclusive Rules Non-Adaptive Rules *Illustration*(PreScore=High)*(ResponseTime=Fast) 0.789*Graph *(PreScore=High)*(ResponseTime=Fast) 0.456*Graph*(PreScore=Low)*(ResponseTime=Fast) 0.123*Graph*(PreScore=Low)*(ResponseTime=Slow)

Modeling with Stepwise Regression Algorithm 1.Stepwise Linear Regression – Convert features to binary 2.Identify Problematic Rules – Mutually Exclusive – Non-Adaptive 3.Handle Problematic Rules – Remove Lesser Rule in Pair 4.Relearn Model – Regular Regression 5.Rank by absolute value of coefficient 18

Modeling the Best Representation: Experiment Model Types – Baseline: just show one kind (illustration) – 1 Factor: 1 Tutoring Context factor in term – 2 Factors: Partition data along 1 variable High pretesters vs. Low pretesters – 3 Factors: Partition along 2 variables 10-fold cross validation 19 Partition variables: 1.Gender 2.SpatialReason 3.PreScore 4.PctThruProblem 5.PctThruSession

Modeling the Best Representation: Results 20 Higher is Better Significantly better than baseline 2 Factors3 Factors (PreScore and …)

Modeling the Best Representation: PreScore Model 21 High Pretesters (n=11) 1.If many correct answers during session, show illustrations 2.If few correct answers in problem, show illustrations 3.If later in tutoring, show illustrations Low Pretesters (n=18) 1.If few correct answers during walk throughs or reflection dialogues, show graphs 2.If many correct answers during session, show illustrations 3.If later in tutoring, show illustrations 4.If few correct answers in problem, show graphs

Modeling the Best Representation: Results 22 Higher is Better Significantly better than baseline 2 Factors3 Factors (PreScore and …)

Interpreting the Model PreScore*Gender 23 (n=8) 1.If few correct answers in walk throughs or reflections, show graphs 2.If many correct answers in session or problem, show illustrations 3.If early in problem or session, show graphs (n=9) 1.If many correct answers in session, show graphs 2.If early in session, show illustrations 3.If many correct answers in problem, show illustrations 4.If early in problem, show illustrations 5.If few correct answers in reflections, show illustrations (n=3) 1.If few correct in reflections, show illustrations 2.If many correct in session, show illustrations 3.If few correct in walk throughs, show illustrations (n=9) 1.If few correct answers in walk throughs or reflections, show illustrations 2.If many correct answers in session, show illustrations 3.If early in session or problem, show graphs 4.If many correct answers in problem, show illustrations Females Males High Pretesters Low Pretesters

Conclusion Developed modeling technique – Best visual unknown – Handles “problematic” rules 5 models outperform baseline – Possible to model benefit Partitioning Useful: PreScore & Gender 24

Future Work Empirical Evaluation of Model – Is adapting visual representation helpful? Develop method of selecting partition features – Partial correlation with postscore (covars=existing partitions)? 25

Thank you Thanks to IES for supporting this research. 26