1. WP 3: Human Performance Metrics 10 Oct 2002 Brian Hilburn

2. Goals Extract set of ASAS applicable HumPerf Metrics Outline application methods (guidance) Provide primary references Mapped onto: - Application (eg Airborne Sep.) - Validation Type (eg Realtime, etc) - Agent (ATC vs Flightdeck) - Human Performance Area

3. Human Performance Areas –Workload –Situation Awareness –System Monitoring –Teamwork –Trust –Usability and User Acceptance –Human Error

4. Parallel WP2 (System Metrics) –SYSTEM versus HUMAN performance? Distinction sometimes arbitrary –Performance Areas differ –Implicit links to Goals human performance influences SAFETY etc

5. WP 3 Deliverable ASAS f 1 Air Ground Wrkld SA etc Obj Subj Fast t Real t ASAS f n....

6. Methods Literature review - ASAS sources (transatlantic FreeFlight) - ATM validation efforts (INTEGRA, RHEA,…) - Aviation Human Factors - Human Factors general Distillation of methods and metrics Guidelines Including general Measurement Issues

7. General Measurement Issues Validation test subjects Experimental design in validation Analytical techniques Balancing validation realism & data collection

8. Format of Metrics

9. Guidance: Literature Consensus and Expert Opinion... EXAMPLE

10. NASA Task Load Index Heart Rate Variability Pupil Diameter Human Computer Trust Scale Auditory Choice Secondary Task Response Time Situation Awareness Rating Technique Blink latency Choice Reaction Time secondary task etc………... 100+ Human Performance Metrics

11. Choosing a Metric –ObjectivityObjective / Subjective –IntrusiveHigh / Low –CostHigh / Low –ReliabilityHigh / Low –ValidityHigh / Low –UtilityHigh / Low –ExpertiseHigh / Low –ResourceHigh / Low

12. A Worked Example... Given: Airborne Spacing application (in-descent spacing scenario) Controlled airspace Study: Real time Question: Transient workload peaks? Issues: Time resolution, intrusiveness, cost

13. Applying the Framework

14. General Guidance

15. l Link between System and Human Performance metrics l What are my goals? l Relationship to higher validation goals l Guidance, not a cookbook l Ideally used in conjunction w SYSTEM metrics Conclusions