U.S. Army Research, Development and Engineering Command Use of Evidence-based Strategies to Enhance the Extensibility of Adaptive Tutoring Technologies Benjamin Goldberg, Ron Tarr, Dr. Deborah Billings, Naomi Malone, Keith Brawner, and Dr. Robert Sottilare

Push for Tailored Training Computer-based tutoring systems (CBTS) have demonstrated significant promise in tutoring individuals in well-defined domains, but… Fifty years of research have been unsuccessful in making CBTS ubiquitous in military training… Why? CBTS are expensive to author and are insufficiently adaptable to support the tailored, self-regulated , individual & small unit tutoring experiences required to support: U.S. Army Learning Model (ALM) for 2015 (TRADOC, 2011) U.S. Air Force (AETC, 2008) U.S. Navy STEM Grand Challenge (ONR, 2012) OSD R&T Vision for PAL NATO HFM RTG 237 (Advanced ITS) TTCP HUM TP-2 (Training Panel) Few open-source components, methods or standards for CBTS Few authoring tools or standards to promote reuse CBTS do not support small unit training/tutoring experiences CBTS do not translate well beyond desktop learning (e.g., mobile learning, mixed reality or live training) and thereby do not support ALM U.S. Air Education and Training Command (2008). On Learning: The Future of Air Force Education and Training. Randolph AFB, TX. http://www.aetc.af.mil/shared/media/document/AFD-080130-066.pdf 

Computer-Based Tutoring Systems (CBTS) ITSs apply Artificial Intelligence tools and methods to individualize instruction Based on benefits associated with one-on-one expert tutoring (2-Sigma Problem; Bloom, 1984) Mediates learning by providing feedback when appropriate and adjusting difficulty levels to maintain desired challenge Facilitated by 4 common components

Generalized Intelligent Framework for Tutoring (GIFT)

Pedagogical Modeling Designed to balance the level of guidance a learner needs with the goal of maintaining engagement and motivation

Macro-Adaptive Strategies Organized and sequential set of tactics to be implemented online What to adapt and how to adapt Addresses four instructional design areas: Selection Sequencing Synthesizing Summarizing Based on metrics collected prior to the commencement of instruction

Macro-Adaptation: Sources of Adaptation Source of Adaptation (Individual Differences & Task Characteristics) Student Performance/ Achievement Levels Working Memory Capacity (WMC) Knowledge Type MATERIAL: Task Difficulty/ Complexity Learner’s Prior Knowledge/ Expertise Learner’s Traits/Ability/ Attributes Learning Styles/Cognitive Styles IMI Levels Task Learning Category: Cognitive, Affective, Psychomotor, Social “Sources of adaptation” refer to the factors that prompt, or trigger, adaptation to occur, namely the characteristics of learners that elicit specific instructional tactics to be implemented.

Macro-Adaptation Targets of Adaptation Target of Adaptation (Instructional Tactics) Sequence of Instruction Presentation of Information (Graphics, Animations, etc.) Degree of Learner Control Feedback (Frequency, Content) Problem Difficulty Pace of Instruction Different sources of adaptation appear to be linked with different targets of adaptation (as shown) in the literature. Based on: - The instructional elements that are adapted - Elements are adapted according to the specific sources of adaptation

To start a search, click on the Strategies tab at the top. Home Tutorial Instructor Training Developer The Instructional Strategies Indicator (ISI) is a computer-based searchable tool for selecting individualized instructional strategies based on the type of knowledge being learned, the domain in which the learning occurs, the expertise level of the learner, and the size of the group being taught. Our goal is to inform the selection of instructional strategies in order to improve learning effectiveness and efficiency across a wide range of domains. Ultimately, the ISI is intended to provide recommendations regarding how, what, and when to embed instructional components into simulation-based training systems. To start a search, click on the Strategies tab at the top. Log in: User name ______________ Password _________________ The following slides show the ISI prototype, which will be demonstrated to the SMEs. This is the home page from which the user logs in and can select either the Tutorial, the Instructor Search or the Developer Search.

Application in GIFT: Model Development

Illustrative Example PRE-TRAINING [Human] Instructor Input Trainee Characteristics Assessment of Trainee Goal Orientation/Motivation Expertise Level Self-Efficacy Journeyman Expert Novice Conceptual Procedural Declarative Integrative Instructional Content Knowledge Type Learning Objectives GIFT’s Pedagogical Model Higher Level Instructional Tactic EXAMPLE: Increase Scenario Complexity Increase “fog” in Scenario Remake this slide: Highlight information that comes from learner and domain modules…Helps distinguish information that is dependent to the tasks being trained.

Extensibility The need for Empirical Evaluation of a Generalized Pedagogical Model to determine its utility across multiple domains and training platforms Goal of Experimentation Determine best practices for “Best-in-Class” model to be incorporated in GIFT Determine influence Individual Differences have on training outcomes Determine how instructors and trainers will use prescribed outputs from model implementation

Road-Ahead Development of a complementary Micro-Adaptive Pedagogical Model for use in GIFT Used to inform ‘in-situ’ strategies and tactics on a general level (e.g., provide hint, adjust difficulty, perform assessment, etc.) Requires techniques to monitor reactive states (i.e., cognitive and affective) and strategies to mitigate negative states Soldier Centered Army Learning Environment (SCALE) Utilize GIFT as SCALE’s pedagogical management engine Provide mechanisms of personalized/tailored instruction with distributable training content

QUESTIONS