1. J.S. Bresee Conor McKenna iFest – 08/01/17
Benefits of Using Machine-generated Human Factors Performance Data for Training and Assessment
J.S. Bresee
Conor McKenna
iFest – 08/01/17

2. Bottom Line Up Front
Adaptive training depends on continuous assessment
Typical assessment methods are intrusive and not suitable for team tasks
Bio-signal data enables data gathering that is continuous and unobtrusive
Start saving audio: to make this useful, it’s time to start collecting, baselining and validating these data.

3. Customized Learning: Unrealized Potential
Historically:
Customized learning = 1 teacher + 1 student
Practically:
Customized learning depends upon automation

4. Two key requirements: Continuous assessment
Selecting an alternative learning option requires knowledge of student progress
Alternative learning pathways and events
Alternate paths designed in advance and pre-loaded
Alternate paths generated by gaming structures and logic.

5. Challenges
Anticipating (or generating) - the number and type of alternate learning pathways that will be needed.
Dealing with both the creative and financial challenges of providing and selecting alternate pathways.
Accurately assessing student progress across a range of dimensions.
Performance
Engagement
Collaboration
Communication

6. Historic assessment methods
Verbal information and cognitive tasks:
Periodic insertion of “test questions”
Easily designed
Inexpensively implemented
Obtrusive; interrupts the flow of learning
Not suitable for assessing continuous performance

7. Historic assessment methods
Continuous performance/team tasks:
Instructor observation
Record/replay and critique
After-the-fact evaluation forms and questionnaires
Requires either recall of performance or informed interpretation
Resulting data is usually non-numeric and based on informed opinion (subjective).

8. Assessment with Bio-signal data
Unobtrusive
Continuously collected
Objective numeric ratio-scale data
Useful for individual feedback or aggregate analysis
Available technology

9. Eye-tracking
Used in cockpits for decades for weapons control
Common human factors research variable
Used in simulation visual systems

10. Facial expression recognition
Spin-off of facial recognition technology
Widely used in market research
Demonstrated use in gathering learner feedback
Body of research findings is substantial and expanding
e.g., Dr. Sidney D’Mello, Notre Dame

11. Speech Signal analysis
Based on well-established speech science
Prosodic accommodation research is well established
e.g.; Celine De Loose, Brian Vaughan, Trinity College, Dublin
Non-verbal prosodic analysis works with any language

12. Sample use case for speech signal analysis

13. Applications: Any team performance task:
Crew performance in flight simulation.
Bio-signal data correlated with digital flight data recording and playback
Applicable to multiple platforms (formations), crew-served platforms, remotely-piloted platform teams.
Medical/first responder team performance
Command center/CIC team training
Consultation/negotiation training

14. The Path Ahead
DATA
Machine learning (AI)
Adaptive
Personalised
Evidenced
Develop individual performance and behavioural data to learners.
Gather bio-signal data and associated meta-data in significant quantities
Apply Big Data techniques to identify useful trends for adaptive learning
Uncover useful adaptive learning paradigms for use with LMS or game-driven syllabus control

15. Company Overview >>
Vocavio provides speech signal processing software to augment simulation, training and operations in aerospace & defense.
Technology applications include;
Monitoring and assessing crew communications in simulation environments i.e. integrating our middleware into LVC (live, virtual, constructive) training environments to measure and validate mission critical skills.
Our software uses speech signal processing (processing non-verbal aspects of speech) to output a range of performance data relating to communication dynamics [e.g. communication effectiveness, leadership, cohesiveness, intensity etc.. ]
We are a campus company of Trinity College Dublin. (incorporated in 2014).
Patented technology (US, EUR, CAN) that was developed and scientifically validated at Trinity College Dublin (with assistance of military and commercial pilots).
Industry achievements
Modsim entrepreneur award 2016 by NDIA (Virginia Beach, USA)
Top 20 cloud technology company 2016 (Clearwater M&A, UK)