L.Connell 3/04 Aviation Safety Reporting System (ASRS) and Related Systems Pre-Solicitation Conference NNA J-GVW March 19, 2004 Hosted by NASA Ames Research Center

L.Connell 3/04 Overview Background & Program Status ASRS Report Processing IT Development Analysts Workbench Query Tool Digital Data Transmission from ASAP airlines Products (Alert Messages CALLBACK, Search Requests, Website, Research) Special Projects Cross-Industry Application of the ASRS Model NASA/VA Patient Safety Reporting System (PSRS) Security Incident Reporting Project (SIRS)

L.Connell 3/04 Background and Program Status

L.Connell 3/04 Overview Summary ASRS established in 1976 as an independent, confidential, voluntary reporting system for aviation at NASA with FAA offering “immunity” to reporters. One of the first lines of defense in identifying safety issues NASA chosen as “Honest Broker” Located at NASA Ames Research Center due to Aviation Human Factors expertise and research—it was recognized that approximately 70% of all accidents are human factors related. ASRS database is a national asset of U.S. aviation safety data Utilization, efficiencies, and improvements of the system have been accomplished progressively over the years.

L.Connell 3/04 Unique balance of contractual and financial arrangements between NASA and other Gov’t Agencies ASRS has become a model for other aviation systems internationally and also for other operational domains throughout the world ASRS significantly related to other NASA, FAA, and aviation industry programs PSRS significantly related to VA National Center for Patient Safety and medical healthcare in general SIRS significantly related to DHS, TSA, aviation security stakeholders, and other environments. Overview Summary

L.Connell 3/04 Code IH Human Factors Research and Technology Division Patricia Jones, Chief (Acting) Code IHH Human Information Processing Research Branch Code IHI Human-Automation Integration Research Branch Code IHS System Safety Research Branch Mary Connors, Chief 7/25/03 ASRS Program PSRS SIRS Linda Connell Jeff Bixler ARC Code I Division

L.Connell 3/04 ASRS Genesis of Human Factors Research at NASA Ames

Human Factors, Risk Identification, Safety Assessment, Insights Alerting Messages, Special Studies

L.Connell 3/04 Aviation Safety Reporting System (ASRS)

L.Connell 3/04 Guiding Principles VOLUNTARY PARTICIPATION Aviation personnel voluntarily submit reports concerning events related to safety for the purpose of system alerting, understanding and learning CONFIDENTIALITY PROTECTION Protection of identity is provided by NASA through de- identification of persons, companies, and any other information NON-PUNITIVE FAA will not use, nor will NASA provide, any report submitted for inclusion under ASRS guidelines or information derived therein for use in any disciplinary or other adverse action. (Advisory Circular 0046D)

L.Connell 3/04 ASRS PURPOSE Identify Deficiencies and Discrepancies Provide Data for Planning and Improvements ALERTSPRODUCTS

L.Connell 3/04 Stakeholders

Governing Documents Federal Register Notices ‘75 & ‘76 Federal Air Regulation - FAR FAA Advisory Circulars (00-46A, B, C, & D) Defines immunity provisions for pilots and others FAA Facility Operation and Administration Handbook, T (Air Traffic Controllers) Defines immunity provision for air traffic controllers MoA between FAA and NASA (signed by NASA and FAA Administrators) Interagency Agreement (signed in ‘99 and renewed in’04 for an additional 5 years)

L.Connell 3/04 Predominant Statements Contained in the Governing Documents ASRS is voluntary, confidential, and non-punitive safety reporting system with NASA as honest broker Reports will not be used by or made available to the FAA for disciplinary or enforcement purposes Limited immunity provisions present from first release of the Advisory Circular AC 00-46A Accidents by NTSB criteria and DOJ criminal offenses wholly excluded Time-critical information alerted to FAA and others Advisory Committee appointed from all elements involved in the operational aspects of the National Aviation System Reporting of findings to the public, aviation community, and the FAA

L.Connell 3/04 Normal Operations Incidents Injury Accidents Fatal Accidents ASRS is Complementary to Other Systems of Reporting Event Occurrences { FAA & NTSB { ASRS Precursors

L.Connell 3/04 Program Reporting Reporting to FAA, as specified in Interagency Agreement Bi-weekly telecons Annual Report Semi-annual reviews NASA ASRS Subcommittee of Headquarters Aeronautics Technical Advisory Committee (ATAC) Semi-annual meetings NASA HQ ATAC Chairman of ASRS Subcommittee invited to present usually once a year or on request

L.Connell 3/04 Annual ASRS Report Intake ASRS Report Intake An Increase of 70% Since 1988 Averaging 2,900 Reports Per Month 145 per working day Total 2003 Report Intake = 34,043 Reports Intake projected to exceed 34,000 in 2004

L.Connell 3/04 INCIDENT REPORTER DISTRIBUTION January December 2002

L.Connell 3/04 Significant Item Quantity Incident Reports Received 605,133 Safety Alert Messages Issued 3,467 Search Requests/Quick Responses 6,864 CALLBACK Safety Bulletins 291 DIRECTLINE Issues 10 Major Research Studies 59 ASRS METRICS April 1976 – December 2003

L.Connell 3/04 Report Processing

L.Connell 3/04 Report Processing Report Processing Report Processing Mail Pickup Mail Pickup Date/Time Stamp Date/Time Stamp Initial Screen Initial Screen Alerting Messages Alerting Messages Match Multiples Match Multiples Analyst Coding Analyst Coding Telephone Callback Telephone Callback Quality Check Quality Check De- identification De- identification Data Entry Data Entry Destruction Of Originals Destruction Of Originals

L.Connell 3/04 ASRS Expert Analyst Staff Aviation Operational Expert Analysts perform initial screening, identification of alerts, initial analyses, database coding, and special studies (e.g., structured callback interviews and consultation on special safety topics). Expert Analysts are retired air carrier pilots (Part 121 and 135), retired air traffic controllers, retired maintenance technicians, general aviation pilots, and flight attendants. A minimum of 10 years aviation experience required. Years of aviation experience of the current analyst staff totals to approx. 390 yrs. An average of 35.5 yrs experience in aviation.

L.Connell 3/04 IT Development

Expert Analyst

L.Connell 3/04 Mix and Match Database Database has 2 Levels of restricted access Three users are allowed to access database Report Processing Coordinator has passwords of 2 Support Staff Database Password required to open file

L.Connell 3/04 Current IT Plan Full Form Process Automation Analyst Workbench - custom browser-based, cross-platform software application used for report processing; deployed October 2001 Track report electronically through complete report production process Minimize paper report flow Evaluation of IT Security related to ERS Electronic De-Identification Prompt analyst for possible airline Ids, names, etc. Allow analyst to replace with text, e.g., ACFT X Computer assisted prompts for analyst coding Data analytic & graphic tools Public Query Wizard

L.Connell 3/04 System Hardware 43 Workstations 11 MAC’s 32 Window PC’s 13 Windows Servers 1 SGI Server 13 Printers 3 Automated Back up tape drives Cost efficient, relatively low investment in hardware

L.Connell 3/04 System Software Oracle 8i - relational database platform for ASRS database utilities; Oracle 9i for PSRS Access, Filemaker Database Search Tools Brio Oracle Based - Query Wizard QUORUM/Perilog (new NASA developed textual search engine) Microsoft office Desktop publishing and website utilities Adobe products, Macromedia products Majority of products - COTS

L.Connell 3/04 Network Entire network is connected with Category 5 ethernet cable All servers connected to network at 100 megabits per second T1 connection to Ames for incoming and outgoing network traffic Firewall technology applied

L.Connell 3/04 Digital Data Transmission Airline ASAP Program Coordination with ASRS

L.Connell 3/04 ASAP Report Submission Update ASRS began receiving ASAP reports in 1997 Since June 2002, the ASRS has received a total of 11,209 ASAP reports Receiving Paper Reports from 8 Airlines Beginning Process of Digital Data Transmission from ASAP to ASRS Use of state-of-art SSH Secure Shell and Private Key technology One ASAP airline transmitting data electronically Meeting of ASAP Managers and ASRS, June 18-19, 2003 Hosted by United Airlines, Denver Attended by 13 ASAP programs, both flight and maintenance Attended by FAA-AFS, FAA-ASY, University of Texas, MITRE, UTRS

L.Connell 3/04 Digital Data Transfer Coordinate with airline ASAP programs and developers Digitally receive ASAP reports over a secure connection Integrate received data with ASRS system

L.Connell 3/04 Digital Data Transfer Process Airline formats ASAP data into XML 128 bit “Secret Key” encryption Data received in XML format BIZTALK performs data translation Data stored in ASRS database Data enters ASRS screening process

BIZTALK Software Translation

L.Connell 3/04 Products

ASRS Database Alert Messages Researc h FAA & NTSB Quick Responses Monthly Safety Newsletter Quarterly Safety Bulletin Incident Reports Database Search Requests CD ROM ASRS PRODUCTS & SERVICES FOR THE AVIATION COMMUNITY

L.Connell 3/04 ALERT MESSAGE PYRAMID For Your Information Notices Alert Bulletins The majority of de-identified alerting messages are discussed in telecons with FAA and other agencies.

L.Connell 3/04 Alert Message Overview When ASRS receives a report describing a hazardous situation (e.g., defective navigation aid, an aircraft system anomaly, a confusing procedure, or any other circumstance which might compromise safe flight), an alert message is issued using de-identified information provided in the reports. There are two levels of alerts: Alert Bullletins and FYI Notices Alert messages have a single purpose: To relay safety information to individuals in a position of authority so that they can evaluate the information and take the needed corrective actions. ASRS has no direct operational or regulatory authority. It acts through and with the cooperation of other gov’t agencies and the aviation industry.

L.Connell 3/04 ASRS Alert Messages January 1997 – December 2003

L.Connell 3/04 PUBLICATIONS CALLBACK is ASRS's award-winning monthly safety bulletin that began publication in It has an estimated readership of more than 150,000. Over 260 issues have been published and distributed throughout the U.S. and to the international aviation community. All issues are available for download at the ASRS website. DIRECTLINE Is another award-winning ASRS publication. This safety journal has an estimated distribution / readership of 20,000. Ten issues have been published since 1991 with a baseline of three to five articles per issue. All issues are available for download at the ASRS website.

ASRS Home Page ASRS Database Access at FAA NASDAC Site:

L.Connell 3/04 ASRS Advisory Subcommittee

L.Connell 3/04 ASRS Advisory Committee Initial ASRS Advisory Committee established in 1976 following agreements between NASA and FAA. Is crucial feature of the ASRS and its success because it provides the forum for industry, FAA and NASA to assure ASRS integrity. Members include representatives from AFA, AIA, ALPA (Chair), AOPA, ATA, GAMA,HAI, IAM, NATCA, NBAA, RAA, and FAA, Office of System Safety. ASRS Advisory Committee has a Security Group that will visit the facility unannounced to check report security provisions.

L.Connell 3/04 International Confidential Aviation Safety Systems ICASS

L.Connell 3/04 International Confidential Aviation Safety Systems (ICASS) Membership dependent on three criteria: Voluntary submission Confidential protections Non-punitive Nine countries are operating systems based in large part on the ASRS model UK, Canada, Australia, Russia, Taiwan, Korea, Japan, China, and U.S. New Zealand, Germany, and others continue to work to establish these systems Recently, ICASS was recognized by ICAO as the source for assisting other countries globally who are interested in creating these systems within their own aviation systems. (Statement being added to Annex 13, Accident Prevention).

L.Connell 3/04 Special Projects

L.Connell 3/04 Additions to ASRS Core Program FAA Runway Safety Office – Identification of Airport Surface Movement Event locations at 21 major airports NASA Aviation Safety and Security Program – Incident Reporting Enhancements (IT Development, ASAP Coordination, etc) – Modeling of ASRS to a NASA Security Incident Reporting System (SIRS) VA National Center for Patient Safety – Modeling of ASRS to a NASA Patient Safety Reporting System (PSRS) Special Projects

L.Connell 3/04

ASRS - Model for Other Operational Systems There has been growing interest from a variety of disciplines desiring the safety benefits realized in aviation. Numerous organizations have requested assistance from the ASRS in creating reporting systems designed to help them understand and learn about their own systems.

L.Connell 3/04 CROSS-INDUSTRY APPLICATION Patient Safety Reporting System (PSRS)

INSTITUTE OF MEDICINE REPORT Released 1999

Medical Performance Factors

VA and NASA Launch Agreement for PSRS May ‘00 - Interagency Agreement signed between NASA and Department of Veterans Affairs –Dr. Thomas Garthwaite, Under Secretary of Health –Dr. Henry McDonald, Director, NASA Ames Research Center NASA Ames Research Center, Moffett Field, CA and VA National Center for Patient Safety, Ann Arbor, MI are collaborating to maximize patient safety efforts –Dr. Jim Bagian, Director of NCPS –Ms. Linda Connell, Director of PSRS

VA Staff Invited to Participate Any VA staff or personnel working in VA facilities are invited to submit voluntary reports concerning medical safety Physicians Nurses Laboratory Radiology Pharmacy Rehabilitation Dietitians Support Staff Others

Total Report Intake *As of 8/26/2003 Total Reports = 305

L.Connell 3/04 CROSS-INDUSTRY APPLICATION Security Incident Reporting System (SIRS)

L.Connell 3/04Outline Background Sub-Project Goals & Objectives Summary of Planning Roadmap SIRS Consortium

L.Connell 3/04Background Shortly after September 11, 2001, the NASA Aviation Safety Reporting System (ASRS) began receiving reports from pilots, flight attendants, and mechanics on aviation security events. NASA ASRS, as the trusted “honest broker” to these communities, has received greater than 500 security reports. The ASRS promptly established a communications link with the FAA Office of Civil Aviation Security to receive de-identified aviation security reports. With the transfer of Civil Aviation Security responsibilities to the newly established Transportation Security Administration (TSA), the ASRS established a new link for these security incidents after the transition. NASA and the industry/FAA representatives to the ASRS recognized the need to establish an independent, separate reporting system that focused on the sensitive security information. The TSA workforce and other air transportation personnel who experience security incidents or concerns are expected to contribute. NASA ASRS performed a review of the security reports and conducted a structured telephone interview to provide an early gap analysis concerning the security data. This work supported by CICT funds provides a strong basis for the continued development of SIRS.

Security Incidents Location Airport Terminal

Security Incidents Location Flight Phase Boarding Pre-flight Servicing Taxi-out Takeoff Climb Cruise Decent Landing Taxi-in Deplanning

L.Connell 3/04 Sub-Project Goals and Objectives GOAL: Develop and create the ability to provide a national confidential Security Incident Reporting System that will serve as a data resource and early warning system for security stakeholders. OBJECTIVES: Provide a confidential, non-punitive reporting environment Gather information on security system events from TSA personnel, airport employees, airline employees, and other aviation system stakeholders Identify vulnerabilities that may not be discernable through other reporting avenues Alert decision makers to potential problems for appropriate follow-up action

L.Connell 3/04 Summary of Planning SCOPE: Build a consortium of industry/government participants for SIRS development Leverage off established relationships with ASRS Advisory Committee representatives and new contacts within the aviation security community. First Stakeholders Consortium meeting was held on September 30-October 1, 2003 –Forty representatives from various stakeholder organizations attended the meeting and endorsed the SIRS concept. ASSUMPTIONS: SIRS draws from the success of the ASRS model and its spin-off to medicine (NASA/VA PSRS). SIRS can utilize existing technology and skills established under ASRS and PSRS. SIRS would have the support of key TSA and “alphabet group” stakeholders. SIRS would replicate the success criteria applied to ASRS and its 27-year history.

L.Connell 3/04Roadmap FY Technical Working Group Stakeholders Consortium 10 airports demonstration 3 airports project Mature SIRS Product Development of Partnering Agreements Roadmap to a Mission Ready SIRS Prototype Nationwide 2 Consortium Continued Field Test and Partnering Agreements TWG 12 airports demonstration 4 5

L.Connell 3/04 Stakeholder Attendees SIRS Consortium Meeting September 30 – October 1, 2003 TSA DHS FAA Volpe ARC GRC ACI-NA Airport Security Managers Local Law Enforcement Airport Law Enforcement ALPA, AFA, AMFA, AOPA, APA, APFA, ATA, CAPA, HAI, IAM, IPA, NATCA, TWU

L.Connell 3/04 SIRS Prototype Development SIRS is expected to be initiated progressively. The ASRS model will be evaluated and replicated as it applies to the environment of aviation security. * * * * * FY’03-08 Development of prototype system for event and close call incident reporting related to security Validation of reporting methodology Confirmation of utility of providing systems view Proof-of-concept for the aviation security community

L.Connell 3/04 The Model Can Provide - “The First Line of Defense” A confidential reporting system can hear about and identify safety events and issues for early warning of potential serious occurrences... ASRS Model is Well-Established, Proven, and Trusted... from the “human element” - the people interacting directly with the medical system... who describe events having face validity - honest, accurate, professional, introspective, and... from the reporter’s unique point of view

L.Connell 3/04 Confidential Reporting Can - Provide the “Big Picture” Most other data sources lack the ability to capture the information that this type of reporting receives. -Reporting from a broad population of system users -Reporting covers the full spectrum of safety concerns Reporting system can target specific populations for special study and expanded information gathering -Structured Callback Studies -Routine telephone callback -Special Studies

L.Connell 3/04 Thank you for your interest and attention

L.Connell 3/04