Intelligent Tutoring System for CS-I and II Laboratory Middle Tennessee State University J. Yoo, C. Pettey, S. Yoo J. Hankins, C. Li, S. Seo Supported by NSF CCLI-AI grant: DUE

Middle Tennessee State University Adaptive Tutor for Online Learning : AtoL Create lab environment to provide personalized assistance. Dynamically adapts to the needs of each student. Provides fast learners with more challenging questions. Gives slow learners a chance to review basic concepts. Course management component Web-based system

Middle Tennessee State University Architecture of AToL

Middle Tennessee State University The design of the tutors Question tutor Select a question from the database Grades the student’s answer Display the correct answer along with the explanation for it. Program tutor Select a programming exercise from the database Displays an incomplete program along with instructions for completing the program. A web-browser based programming environment : to code, compile, execute, and submit Incomplete program Solution program Test data files Explanation and more T/F, multiple choice, short answer Pre-assigned difficulty level Solutions Explanation of the solution

Middle Tennessee State University The design of the lab page Existing layout One long page Pre-set exercises Answer sheet for answering questions Unix environment for programming exercises Wait for about two weeks for your grade

Middle Tennessee State University New design (1) Smaller pages One page per concept Question tutor/ program tutor buttons

Middle Tennessee State University New design (1) Smaller pages One page per concept Question tutor/ program tutor buttons Programming environment

Middle Tennessee State University New design (1) Smaller pages One page per concept Question tutor/ program tutor buttons Programming environment Monitoring grade

Middle Tennessee State University New design (2) Structured page a single web page that contains a structured outline of the concepts Link to another page for the concept

Middle Tennessee State University Designing the new lab page Existing labNew lab design (1)New lab design (2) One Web page / labEach lab is partitioned into several web pages a single web page that contains a structured outline of the concepts Each page addressing a specific concept. Link to another page for the concept An answer sheet to record answers and print for submission Tutor buttons are provided so that a student can initiate a tutoring session Programming in Unix environment with hardcopy for submission Program tutor in Web browser with automatic grading About two week waiting for grade Monitoring his/her grade online

Middle Tennessee State University Student progress monitoring tool: for student and teacher Class performance monitoring for teacher

Middle Tennessee State University Design of the adaptive component Two-step semi-supervised learning system for strategy adaptation 1. C4.5 is used to learn the classification rules 2. Markov chain clustering to maximize learning

Middle Tennessee State University Identifying student model Three student models : challenge type regular type reinforcement type Collect student information Student activities with the tutor Student learning style Learn classification rules using C4.5  Identify student model using student activities

Middle Tennessee State University Strategy adjustment Bayesian Markov Chain Clustering algorithm : refined groups that share similar behavior patterns using data collected previously Revises the learning strategy of the student using the recent performance of the student  selects an appropriate problem based on the revised, individualized strategy

Middle Tennessee State University Tutor Student Model Identifier  GUI Strategy Adjustment Module Adaptation components challenge type? regular type? reinforcement type ? Refined groups of similar behavior patterns Student behaviors Student type Selected question

Middle Tennessee State University Experimentation New labs: Three new design (1): paged Two new design (2): structured Eight classes (5 different teachers) Two groups in each classroom Testing Pre-test, lab, and post-test  difference between the pre-test and post-test Exit survey

Middle Tennessee State University Improvement comparisons in post-test scores LabDesignMean %Sample SizeSignificant File I/O New (1)3461 NoNo Old2965 Selection New (1)1968 Yes Old1363 Loop New (1)2760 Yes Old1550 Functions New (2)3762 Yes Old2153 Arrays New (2)4656 Yes Old2652

Middle Tennessee State University Lab completion rate File I/OSelectionLoopFunctionsArrays New90%99%92%94%98% Old86%90%71%77%78% Diff4%9%21%17%20%

Middle Tennessee State University Analysis of student satisfaction Group one: 4 old labs, five new labs (1,2) Group two: 9 old labs Both groups agree that the labs were helpful in understanding the concepts better. 77% (group one) vs. 57% (group two) The new style labs were better than old style labs: 74% of group one Immediate feedback, online grading, online submission  aided their learning more effectively Rank three labs : 1. New design (2) : structured 2. New design (1): paged 3. old design: current Marked either 4 or 5

Middle Tennessee State University Analysis of student satisfaction: The survey result for the new design(2) Survey question Agree (4 or 5) Disagree (1 or 2) Solving similar questions multiple times helped you understand the concept 78%9% I like solving the similar questions multiple times 66%14% Displaying the concepts in a hierarchical structure helped you understand the overall picture of the material. 66%9% Displaying your progress in a tree view with green/yellow/red/gray color nodes helped you monitor your understanding. 64%17%

Middle Tennessee State University Analysis of teacher satisfaction for the new design (2) All the teachers found the system to be very useful. comments included that the new lab helped them: focus on specific topics by collapsing the tree determine concepts that needed more class discussion determine inappropriate difficulty levels of existing questions determine where more questions were needed in the lab flexible in assigning the lower bounds on colors for honors classes as opposed to regular classes

Middle Tennessee State University Conclusions A Web-based adaptive tutoring system for CS-I and CS-II closed labs Question tutor: help a student learn a concept by solving a series of questions Program tutor: help a student learn programming skills as well as the corresponding concepts AtoL employs a two-step semi-supervised learning system for strategy adaptation The Web-based course management system Statistically significant improvement in understanding of concepts Students preferred AtoL because it provided immediate feedback, online grading, and an online submission Teachers found AtoL very useful