1. Normal Operations Safety Survey (NOSS)
Chris Henry
The University of Texas Human Factors Research Project
The University of Texas at Austin
First ICAO TEM & NOSS Week
Luxembourg – 10 November 2005
UT Human Factors Research Project © Copyright

2. Aviation Safety Envelope
Accidents
Incidents
Normal Operations
Safety

3. Safety Data Coverage Accidents Incidents Normal Operations
Voluntary Incident Reports
NOSS
On any given day airlines may collect information from
This diagram represents sources of aviation safety data
We collect 4 main types of informaiton with the safety envelope:
Accident investigations – rarest, non-voluntary, reactive investigations
FOQA – technical flight parameters
ASAP – self-report data resulting from voluntary crew reporting, proactive, center on FAR violations
LOSA – normal line operations, info from SME’s on normal flights
At UT – we have developed a multi-project research program that covers assessment and analysis of 3 of these data collection programs
- reaching each level of the safety envelope
Accident Investigation
Mandatory Incident Reports
UT Human Factors Research Project © Copyright

4. Aviation Safety Envelope
Accidents
Incidents
Normal Operations
NOSS Rationale
Proactive snapshot of system / controller performance strengths and weaknesses
in normal operations (just like a health checkup)
On any given day airlines may collect information from
This diagram represents sources of aviation safety data
We collect 4 main types of informaiton with the safety envelope:
Accident investigations – rarest, non-voluntary, reactive investigations
FOQA – technical flight parameters
ASAP – self-report data resulting from voluntary crew reporting, proactive, center on FAR violations
LOSA – normal line operations, info from SME’s on normal flights
At UT – we have developed a multi-project research program that covers assessment and analysis of 3 of these data collection programs
- reaching each level of the safety envelope
UT Human Factors Research Project © Copyright

5. NOSS Success Factors
NOSS success is dependent upon methodology and execution
Formal Check
Regulator
Nobody
Angel
Performance
NOSS Observer
Natural
Performance
- Controller Trust + - +
NOSS value - +
Low controller trust = Low quality data because there will be no differentiation between NOSS and proficiency checks

6. NOSS: Gaining Controller Trust
Over-the-shoulder observations during normal shifts
Joint management / association sponsorship
Voluntary Participation
Trusted and trained observers
Anonymous, confidential, and non-punitive data collection
Trusted and secure data collection site
Systematic data collection instrument
Data verification process
Data-derived targets for safety enhancement
Feedback results to controllers

7. NOSS Operating Characteristics
Over-the-shoulder observations during normal shifts
No observations of controllers who are undergoing training
No observations while checks are being conducted
Joint management / association sponsorship
Letter signed by management & association representatives sent to all controllers
Voluntary controller participation
Controllers have a right to decline a NOSS observation
Denial rates in NOSS trials were low – 1 per 50 observations

8. NOSS Operating Characteristics
Trusted and trained observers
Trust and Credibility
Selection process – management / association approval of candidates
Most observers should be line controllers – but diversity of interested parties is good.
Training
Classroom training, test observations, recalibration session
Curriculum
TEM principles and exercises
NOSS observation protocols
Narrative guidelines

9. NOSS Operating Characteristics
Anonymous, confidential, and non-punitive data collection
No names, employee numbers, dates, experience, or other identifying information
Data used for safety purposes only – no punitive actions
Observers identity is known only by the third party facilitator
Trusted and secure data collection site
Third party or controller association gatekeeper
Controllers and observers must be comfortable with the data collection site
Systematic data collection instrument
No judgments – observers record data based on TEM events

10. NOSS Operating Characteristics
Data verification process
TEM data checked to ensure coding accuracy and consistency with SOP
Data analysis does not begin until verification has been completed
Data-derived targets for safety enhancement
Serve as benchmarks for safety change
Initial NOSS, safety change process, follow-up NOSS
Feedback results to controllers
Results summarized for controllers
Information on how organization intends to respond to the data

11. NOSS Defined
The 10 characteristics that differentiate NOSS (LOSA) from other methodologies have been endorsed by
ICAO
IATA
IFATCA (6)
IFALPA
US ALPA UT
NOSS must have all ten characteristics

12. Methodological Considerations
At which level were the NOSS trial observations focused?
Groups
Individual Controllers
Positions
Interviews
Threats, errors, and undesired states must be observable

13. Demographics

14. General Narrative

15. Briefings

16. Threat Management Worksheet

17. Error Management Worksheet

18. TEM Countermeasures

19. Threat Codebook - I

20. Threat Codebook - II

21. Error Codebook

22. US Codebook

23. NOSS Report and Raw Data
Most frequent and mismanaged threats
Most frequent and mismanaged errors
Strengths and weaknesses of Threat and Error Countermeasures
Facility to facility or sector to sector differences
Comparison to similar airspace of other air traffic service providers
NOSS Report
NOSS Raw Data
Text from all narratives
Listing and coding of every threat, error, and undesired state observed

24. Sample Final Report Charts

25. Raw Data: Narratives - Handovers
NOSS Observation #17 Handover/Takeover #1 Time Stamp 14: XYZ Radar Sector Positions Staffed: Radar Controller Position Relieved: Radar Controller
The oncoming controller was at the start of their shift, when they arrived they walked straight up to the console and started a conversation with the XYZ Controller about non operational matters, this distracted the controller whilst aircraft were calling resulting in a number of missed calls from aircraft being made to the XYZ Controller. The briefing commenced with the oncoming controller not reading any of the requisite briefing material. The controller giving the HO/TO did it from memory, the checklist provided was not referred to and not all the pertinent information was covered. The controller giving the HO/TO did not mention the RWY configuration in XXX or the fact that they were on ILS approaches. At the end of the HO/TO, the relieving controller plugged in and did not voice log on.

26. NOSS Raw Data - Narratives
ABC47 crossed into the XYZ sector but did not contact the controller. At time 19:42, BB TMA called the XYZ sector controller requesting a non-standard level of FL350 for a flight, which was approved and properly noted. During this conversation, ABC47 called but was not heard due to the ongoing conversation with BB TMA. At this point the A/C was already 40 nm into the sector (this was a very late call). A minute later, ABC47 called again and this time the controller heard the call. At this point it was discovered that ABC47 was east of track by 3nm without prior authorization leading to a potential conflict with EDF54 who was in the vicinity climbing to FL160. ABC47 given a right turn to manage the situation and both A/C were given traffic information. Additionally, CVR310 was descending through the level of ABC47 while it was not on the appropriate frequency (inside the XYZ sector).

27. Raw Data: Threats
NOSS Observation #: 27 Threat # 3 XYZ Radar Sector Time Stamp: 17:42
Threat Description
AAA4103,04,4003, and BBB405 were operating in the XYZ airspace at the same time with similar callsigns.
Threat Management Description
The XYZ controller made a general broadcast to the aircraft advising of the potential for confusion and was particularly careful to ensure the correct read-back and callsign with instructions. All calls where made to the appropriate a/c and the correct a/c responded to the calls each time.
Threat Type Airborne
Threat Code Similar Call Signs
Threat Outcome: Inconsequential

28. NOSS Summary Primary features
Purely diagnostic – no solutions – “operations health check”
Captures system safety and controller performance issues prior to the occurrence of incidents
Identifies system safety and controller performance strengths and weaknesses in normal operations using observations
Analysis
Aggregated TEM data provides a safety snapshot of operations on a typical day
Organizational change
Measure the effectiveness safety solutions with a follow-up NOSS

29. NOSS – Future UT Activities
NOSS & LOSA: What can each add to the other?
NAV CANADA radar trial
Airservices Australia tower trial
Multi-stage reliability studies in conjunction with the FAA & NOSS trial partners
European trials?