Evaluation of Training Effectiveness: Tactical Decision Making Systems and Training of Instructors Amela Sadagic, PhD

Agenda Objective Overarching Approaches Hypothesis Train the Trainers Pilot Study Final TEE Study Additional Contributions Q & A

Research Question & Answer Issues of availability of good training simulations cannot be treated separately from the issues of availability of equally good training methodologies and pedagogies. Q: When we work with the training systems, should we even consider doing studies that do not employ well designed training methodologies and pedagogies? A: We believe we should not.

Objectives Primary: TEE study of simulations used to train tactical decision-making combat skills. Preliminary study of combining different training systems and tools. Develop the guidelines for optimal training approaches and pedagogies that most effectively serve selected training objectives and audience. Identify training opportunities and potential for extended use of TDS-s.

Glossary TDG: Tactical Decision Games pen-and-paper technique TDS: Tactical Decision-making Simulation computer-based training simulations that use gaming technologies CCM: Close Combat Marine Multiuser TDS, navigation through 2D world with top-down view of the terrain IOC: Infantry Officers Course, USMC TBS: The Basic School, USMC

Overarching Approaches Directly connect our research with: Institution that defines the training requirements and makes system recommendations: TECOM (USMC), Institutions that provide the training: The Basic School - TBS and Infantry Officers Course - IOC (both USMC). Support USMC strategy of Distributed Operations, Address specific training needs of the next generation of instructors (‘train the trainers’), Explore novel training approaches and pedagogies, Allow for a synergetic mix of old systems and methods (TDG) and new systems and methods (TDS).

Hypothesis Primary: The strategic use of virtual training simulations does contribute towards higher effectiveness in training, affecting learning results and motivation. ‘Early majority’ and ‘late majority’ groups of users will need more than just having the physical access to training systems and tools.

Train the Trainers Acquire personal skills and knowledge but also know how to teach / instruct other people: gradually increase the complexity (crawl-walk-run): fight alone -> ‘buddy-on-buddy’ -> ‘force-on-force’ role changing: be both team leader and a team member, evaluate someone else’s performance, multiple AAR opportunities, add a motivation ingredient. Issue: a ‘gaming’ behavior solution: instead of fighting the ‘destructive’ tendencies (use of non-doctrinal techniques), redirect them to constructive direction

Pilot Study: 12 candidates from IOC

Pilot Study: 12 candidates from IOC av. 24 years old, all 2nd Lt., av. 29.5 months in service Prior use of training systems: 8 YES (5.3 hours total) Game experience: 10 YES (most used several times a year), first person shooters (9), racing (5), other sports (5) Own personal computer: everyone! (av. 5.3 years) What do they use it for: email + web browsing Other applications: Microsoft Office Suite (4) but rarely used Skills they have most difficulties with: Direct a CAS (12!), Employ support arms (4), Direct the employment of 60mm/81mm mortars for offensive ops.(4) and defensive ops (3)

Pilot Study: 12 candidates from IOC Post-session results: Majority agreed that 2 hours of training was not enough, Majority reported no change in skill level for most of the skills, For some skills they reported higher initial proficiency level than the level they reported at the end… yet they thought they improved those skills with training they just received: initial skill level overstated Could we use TDS as a validation tool (‘How well are we doing with our training?’)

Final TEE Study (IOC): a plan Select training objectives that can be effectively conquered in a given time-frame, Integrate selected training approaches and pedagogies, One group of IOC candidates to use TDGs and another group TDS. Then they switch. Exposure: 2 training sessions, 3 hours each. Conduct evaluation of learned skills after each session. Collect both subjective and objective performance measurements.

Additional Contributions Assist IOC in optimizing current training program with TDS, Assist TBS in defining new POI (a part associated with the use of TDSs), Advise both institutions on how to record and follow up the performances exhibited by all candidates that will go through the schools. identify performance trends and if needed address them with appropriate training interventions.

To sum up If one wants to serve the needs of majority of users: Do not just ‘thick the box’ - dedicate a solid amount of time to this type of training, and offer it to the people that need it. Explore a synergy of the old (proven) and new (proven & perhaps exciting) training techniques and systems. Insert motivation elements. Fully integrate technologies/systems with best training approaches and pedagogies. Provide a ‘full package’.

Special thanks to: Sponsor: Navy Modeling & Simulation Management Office (NAVMSMO) Partners in the project: USMC TECOM, IOC and TBS, Quantico, VA Maj McDonough

Q & A Contact: asadagic@nps.edu / ph: 831.656.3819 www.movesinstitute.org/~amela