WP 3: Human Performance Metrics 10 Oct 2002 Brian Hilburn

Goals Extract set of ASAS applicable HumPerf Metrics Outline application methods (guidance) Provide primary references Mapped onto: - Application (eg Airborne Sep.) - Validation Type (eg Realtime, etc) - Agent (ATC vs Flightdeck) - Human Performance Area

Human Performance Areas –Workload –Situation Awareness –System Monitoring –Teamwork –Trust –Usability and User Acceptance –Human Error

Parallel WP2 (System Metrics) –SYSTEM versus HUMAN performance? Distinction sometimes arbitrary –Performance Areas differ –Implicit links to Goals human performance influences SAFETY etc

WP 3 Deliverable ASAS f 1 Air Ground Wrkld SA etc Obj Subj Fast t Real t ASAS f n....

Methods Literature review - ASAS sources (transatlantic FreeFlight) - ATM validation efforts (INTEGRA, RHEA,…) - Aviation Human Factors - Human Factors general Distillation of methods and metrics Guidelines Including general Measurement Issues

General Measurement Issues Validation test subjects Experimental design in validation Analytical techniques Balancing validation realism & data collection

Format of Metrics

Guidance: Literature Consensus and Expert Opinion... EXAMPLE

NASA Task Load Index Heart Rate Variability Pupil Diameter Human Computer Trust Scale Auditory Choice Secondary Task Response Time Situation Awareness Rating Technique Blink latency Choice Reaction Time secondary task etc……… Human Performance Metrics

Choosing a Metric –ObjectivityObjective / Subjective –IntrusiveHigh / Low –CostHigh / Low –ReliabilityHigh / Low –ValidityHigh / Low –UtilityHigh / Low –ExpertiseHigh / Low –ResourceHigh / Low

A Worked Example... Given: Airborne Spacing application (in-descent spacing scenario) Controlled airspace Study: Real time Question: Transient workload peaks? Issues: Time resolution, intrusiveness, cost

Applying the Framework

General Guidance

l Link between System and Human Performance metrics l What are my goals? l Relationship to higher validation goals l Guidance, not a cookbook l Ideally used in conjunction w SYSTEM metrics Conclusions