Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-1A l Humans smarter, meaner, more strategic, emotional, variable, etc. l Will superconflict solving deteriorate or improve? Superconflict n=8

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-2A Who are the bots and who are the humans?

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-3A l Phase I: Web experiment (2000) l Phase II: Classroom experiment (2001) Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR Jacco Hoekstra, Head of Training & Concept Validation department NLR Airborne Separation Assurance Validation with Multiple Humans-in-the-loop

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-4A Human interaction experiments l Before these experiments: - micro: human-in-the-loop only 1 or 2 crews - macro: traffic simulations with all a/c controlled by ‘bots’

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-5A Human interaction experiments l Goal HIE: - Investigate effect of multiple humans in the loop l Research issues: - Validations bots behaviour vs. human behaviour - micro: difference 1 on 1 - macro: complex geometries - Competition effect

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-6A Distributed simulation via Internet DirectXTMX as Game server

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-7A Webexperiment set-up l Freesim: downloadable flight simulator with ASAS as Game Client (MS DirectX protocol)

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-8A FreeSim flight simulator l Downloadable with on-line user manual l Two modes: –Stand-alone with basic training scenarios –Online with Traffic Manager as traffic server l Still available on website: freeflight/fsmanual/freesim2.zip

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-9A FreeSim on-line l Two modes open (demos) and secured (experiments): - using unique address - valid session name - IP number for login l User UTC time synchronized via US Navy clock on web l System time asynchronous, data logging centralized l Performance data and events are sent to TMX by Freesim

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-10A Web experiment l Validation of the bots Competition effect l Two types of conflicts: –human-human –human-bot l Both in traffic and conflict: participants could not tell difference bot/human in normal (realistic) flight scenario l Competition effect not tested due to low number of participants (max 12 simultaneously)

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-11A Result: Conflict rate variation per a/c high l Conflict rate distribution can be odd-shaped and cause problems # a/c

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-12A Classroom experiment Set-up l 24 participants (nearly or just grad) of KLM pilot school l Game effect: goal is time or safety l Two types of scenarios: normal (realistic) & superconflicts

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-13A Classroom experiment results l Workload ratings for normal scenarios with bottlenecks: - higher than before (training?) - multi a/c conflicts driving factor workload

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-14A Classroom experiment results l Preventive effect in superconflict scenarios visible on ‘no conflict’ manoeuvres l Minor effect of strategy superconflict normal no conflict in conflict

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-15A Classroom experiment results: Intrusions in superconflicts N >= 12 l Number of intrusions (all scrapings) - Bots lower in small superconflicts and higher in larger superconflicts l Larger margin required between resozone and cdzone Average Δt = 17 sec about 2% intrusion

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-16A Classroom experiment results: number of conflicts alerts in superconflict l Number of conflict alerts (excl. scrapings) Pilots significantly better

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-17A Classroom experiment Conclusions l Some human effects: - going for the edge - playing chicken (based on observation of one-sided resolutions) l Humans slightly better at solving superconflicts than the bots (lower number of conflicts per a/c)

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-18A Overall conclusions l Bots seem able to simulate humans in normal traffic pattern behaviour l Humans smarter and meaner overall slightly better in superconflicts l Conflict rate and especially bottlenecks are driving factor for workload l Internet technically feasible but requires extra organisation. Classroom allows more control over e.g. training and nr of participants

Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-19A Questions? No problem! We have right of way.