1. Toward quantifying the effect of prior training on task performance MURI Annual Review September 26-27, 2006 Bill Raymond

2. Overview Project goal: Quantify the effects on performance of different training methods for complex military tasks. Project goal: Quantify the effects on performance of different training methods for complex military tasks. Feature decomposition: Feature decomposition: 1.Task type 2.Training method 3.Performance assessment (context & measures) 4.Training principles Planning matrix: Planning matrix: - Capture where we know of, and can quantify in terms of performance measures, effects of training method and performance context on task components. Quantify principles: Quantify principles: - Derive performance functions for points in the feature space using empirical data from laboratory tasks. - Generalize performance functions for implementation in IMPRINT modeling tool to simulate training effects on task performance.

3. Decomposition issues Constraints on decompositions Constraints on decompositions Features must relate to experimental designs  Must be able to describe all experimental tasks.  Task, training, and performance context features can be no finer than experimental manipulations. Features may be different for research and IMPRINT  Can’t control training in the real world as carefully as in the laboratory  Not all experimental results will be major effects.  IMPRINT task categories are already defined.  Planning features should converge to final IMPRINT features, diverging from research features

4. Planning matrix issues What will the matrix construction provide? What will the matrix construction provide?  Current and planned research coverage of space  May be used by us or others for future planning  Approximation of final IMPRINT training features  Initial step in determining the generality of performance functions in the space

5. Training variables - during skill learning: Training variables - during skill learning:  How was the skill taught?  What kind of practice did learners get?  How did practice relate to the way the skill will be used? Performance context variables - at skill use: Performance context variables - at skill use:  How does expected performance relate to training?  How long has it been since training?  Did learners get refresher training? Starting point: Analyzing training and performance Pedagogy Practice Performance } }

6. Task, training, and performance matrix Task components Training features Performanc e context PedagogyPractice Visual Numerical Analysis Information processing Fine motor - discrete Fine motor - continuous Gross motor - light Gross motor - heavy Communication (reading & writing) Communication (oral) IMPRINT task taxons Data entry

7. Pedagogy parameters Method Method  Instruction (=default)  Demonstration  Simulation  Discovery  Modeling/mimicking  Immersion Learning location (local = default, remote/distance) Learning location (local = default, remote/distance) Discussion/Question and answer (no = default, yes) Discussion/Question and answer (no = default, yes) Individualization (no = default, yes) Individualization (no = default, yes)

8. Task by Pedagogy parameters Task components Pedagogy Method Learning location Discussion /Q&A? Individualized? Visual Numerical Analysis Information processing Fine motor - discrete Fine motor - continuous Gross motor - light Gross motor - heavy Communication (reading & writing) Communication (oral) IMPRINT task taxons Data entry (Instruction) Classificatio n Inst/Discovery

9. Practice parameters Scheduling of trials and sessions Scheduling of trials and sessions  Number  Spacing (massed = default, spaced, expanding/contracting)  Distribution (mixed = default, blocked) Scope of practiced task (partial, whole = default, whole + supplemental) Scope of practiced task (partial, whole = default, whole + supplemental) Depth of processing (no = default, yes) Depth of processing (no = default, yes) Processing mediation (no = default, yes) Processing mediation (no = default, yes) Stimulus–response compatibility (yes = default, no) Stimulus–response compatibility (yes = default, no) Time pressure (no = default, yes) Time pressure (no = default, yes) Feedback - presence (no = default, all trials, periodic) Feedback - presence (no = default, all trials, periodic) Context of practice Context of practice  Distractor (no = default, yes)  Secondary activity (no = default, yes)

10. Task by practice Task components Practice SchedulingScopeProcessing depth Processing mediation Stimulus- response compatibility Time pressure FeedbackContext Visual Numerical Analysis Information processing Item repetition, # Sessions, Spacing Part/ whole Yes (presentation format) Yes (prior knowledge) No (Input- output Format) Yes (response & accuracy) Distractor/2nda ry activity (vocal activity) Fine motor - discrete Item repetition, # Sessions, Spacing Part/ whole Yes (response & accuracy) Distractor/2nda ry activity (vocal activity) Fine motor - continuous Gross motor - light Gross motor - heavy Communication (reading & writing) Communication (oral) Data entry IMPRINT task taxons

11. Performance context parameters Transfer Transfer  New context (relative to training)  New task (relative to training) Delay interval (default = none, time period) Delay interval (default = none, time period) Refresher training (default = no, schedule) Refresher training (default = no, schedule)

12. Task by performance parameters Task components Performance context New context New task Delay interval Refresher training Visual Numerical Analysis Information processing Yes (typing hand, output configuration) Yes Fine motor - discrete Yes (typing hand, output configuration) Yes Fine motor - continuous Gross motor - light Gross motor - heavy Communication (reading & writing) Communication (oral) IMPRINT task taxons Data entry

13. Quantifying training principles Data Entry used as an example Data Entry used as an example Consider two principles Consider two principles  Practice  Learning (Power law of practice)  Skill practice - no item repetition  Specific learning - item repetition  Prolonged work  Diminished performance Quantify effects for each taxon Quantify effects for each taxon  Cognitive (“Information processing”)  Physical (“Fine motor - discrete”) …and performance context …and performance context  Transfer to new items (similarity dimension)  Retention of learned skill (refresher training)

14. Skill practice: Quantifying learning Skill practice improves performance.5 msec/item Skill practice improves performance.5 msec/item  Mean decreases 300 msec in 640 (unique) items Where does skill practice come from? Where does skill practice come from?  Repetition of individual digits (and pairs of digits?)  Cognitive or physical learning?  Individual differences?

15. Skill practice: Origin or learning Pair repetition? Subjects appear to “chunk” digits 1 & 2, digits 3 & 4 Subjects appear to “chunk” digits 1 & 2, digits 3 & 4  so they may be learning something about pairs of digits

16. Skill practice: Origin of learning Pair repetition? Effect of 2-digit chunk practice appears minimal Effect of 2-digit chunk practice appears minimal  Skill practice is general facility at number typing

17. Skill practice: Type of learning Physical or cognitive? Speed improvement occurs on digits 1 and 3 Speed improvement occurs on digits 1 and 3  Learning is cognitive

18. Skill practice: Individual differences “ Chunkers ” are 15% slower than “ non-chunkers ” “ Chunkers ” are 15% slower than “ non-chunkers ”  Appears to be a strategy choice  Pedagogy - advantage for instruction over “discovery”?

19. Specific learning: Quantifying learning Repetitious practice improves performance faster initially Repetitious practice improves performance faster initially  Power law of practice

20. General learning functions Performance as a function of number of repetitions Performance as a function of number of repetitions  Planned experiment...?...?

21. General learning functions Transfer and retention Transfer and retention  Planned experiment... New items?Old items? TransferRetentionLearning

22. Prolonged practice Prolonged work results in an increase in errors Prolonged work results in an increase in errors  Accuracy rate decline of about 1% over 320 items Where does the decline in accuracy originate? Where does the decline in accuracy originate?  Cognitive or physical fatigue?

23. Prolonged practice: Type of performance decline Two types of errors: Two types of errors:  Stimulus adjacency errors: 1234  1244  Key adjacency errors: 1234  1264 90% of errors are of these two types 90% of errors are of these two types Origin of errors Origin of errors  Stimulus adjacency = cognitive  Key adjacency =motor phase, which could be motor output planning (cognitive) or motor execution (execution)

24. Prolonged practice: Type of performance decline Practice results in an increase in key adjacency errors Practice results in an increase in key adjacency errors  Accuracy decline occurs during the motor phase (which may be both cognitive and physical)

25. Prolonged practice: Type of performance decline Feedback eliminates the speed-accuracy trade-off Feedback eliminates the speed-accuracy trade-off  If feedback is cognitive, then the relevant processes in the motoric phase must be cognitive

26. Summary Task components Training features Performance context PedagogyPractice Information processing (Cognitive) Method: Instruction - strategy instruction may improve speed “Discovery” - some Ss 15% slower Scheduling: no reps - speed decrease linear (.5 msec/item) item reps - power law (parameters to be determined) Feedback: no feedback - accuracy decline (1%/300 items) typing/accuracy feedback - no decline Transfer: Retention: (planned experiment) Fine motor - discrete (Physical) Transfer: Retention: (planned experiment) IMPRINT task taxons