1. Presented to: By: Date: Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Investment Analysis Readiness Decision (IARD) Independent Evaluation Review (IER) Mark House Jonathan Green, Marty Sobol, John Sullivan Aug. 24, 2010

2. 2 Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Aug. 24, 2010 This IER presentation is notable for its length. Currently all alternatives result in a negative NPV. One of the questions facing the analyst is “At what point is further analysis warranted?” The results in this instance are stated completely in only a few slides. To analyze further, or to expand the review to fit a preconceived notion of a correct number of pages, only serves to muddy the issue. The communication of any analysis should only be as long and as complex as it needs to be. The goal is to provide the decision maker the tools to make a decision. Introduction

3. 3 Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Aug. 24, 2010 NAVAID Control, Interlock and Monitoring Equipment (NCIME) consists of two subsystems and two corresponding sets of Performance Specifications NAVAID Control and Monitoring Equipment (NCME) –Standardized NAVAID Control and Monitoring System to be installed at a maximum of 47 airports (based on the 35 OEP airports plus 5 non-OEP airports included in the top-50 busiest airports and 7 non- OEP airports with at least five ILSs) Universal Interlock Controller (UIC) –Standardized Interlock Controller to be installed at a maximum of 215 airports, which would include all airports with two or more localizers. Definition of Program

4. 4 Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Aug. 24, 2010 Current Operations Air Traffic Control Towers currently have numerous NAVAID monitor, status, and control displays of various implementations. –As directed by Congressional earmarks, 16 airports have Integrated Control and Monitoring Systems (ICMSs) and were acquired outside of the FAA’s system development process. Airports with multiple Instrument Landing Systems (ILSs) currently have various nonstandard interlock devices Resulting Shortfalls Efficiency issues resulting from ICMS outages Possible safety hazard likelihood reduction at ICMS locations Standardization is lacking in both NAVAID monitor, status and control displays as well as interlock devices Problem Statement

5. 5 Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Aug. 24, 2010 Alternatives and Legacy Comparison NAVAID Control and Monitoring Equipment Interlock Control Legacy Case 16 ICMS-equipped airports and other equipment at other airports Various IC at airports Alt. 1Replace ICMS with NCME at all 16 ICMS airports; no change at other airports 16 UIC + various IC at other airports Alt. 2Replace ICMS with NCME at all 16 ICMS airports; replace other equip with NCME at 31 add. airports; no change at other airports 47 UIC + various IC at other airports Alt. 316 ICMS equipped airports and other equipment at other airports (Legacy Case) 215 UIC at airports with 2 or more ILS Alt. 4Replace ICMS with NCME at all 16 ICMS airports; replace other equip with NCME at 31 more airports; no change at other airports 215 UIC at airports with 2 or more ILS

6. 6 Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Aug. 24, 2010 Alternatives and Legacy Comparison: Enumeration of Equipment by Category

7. 7 Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Aug. 24, 2010 Benefits Analysis (related to shortfalls) Safety –ICMS Security Risk Management Document (SRMD) identified two medium risks (identified in the report as H1 and H3) that could affect safety –The interlock violation hazard (H1) could produce a catastrophic event at an extremely improbable likelihood during CAT III operations Implementing the UIC will only transfer this hazard from ICMS to the UIC Since the hazard will not be eliminated, there are no resulting safety benefits –The loss of approach guidance hazard (H3) could result in a hazardous event at an extremely remote likelihood during CAT III operations The H3 likelihood could be further reduced by employing a mechanism to completely isolate ICMS from the ILS facilities (e.g., a specific mechanism, “KC2081 Option 5,” developed by the FAA Kansas City NavAids Engineering Center) –Similar events were extracted from the NTSB accident database –Estimated $/event from injuries, fatalities, and aircraft damage –Benefits were calculated based on the difference between the current risk (extremely remote) and the predicted residual risk (extremely improbable) Further reduction is possible by conducting reliability analysis and testing, system safety assessment, and employing design assurance techniques Efficiency –Identified all possible ICMS-related outages from OPSNET and extrapolated the associated delays Equipment Standardization –Standardization benefit unknown until inventory of current NAVAID control and monitor equipment and interlock controllers is provided by program office

8. 8 Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Aug. 24, 2010 Benefits Analysis (WBS 5.0 cost avoidance) Cost Avoidance Benefits for In Service Management (WBS 5.0) –Alternatives 1, 2, and 4 all have lower overall In Service Management costs (WBS 5.0 costs) than legacy case Lower combined program support, logistics, in service training, and second level engineering costs (including transition costs from legacy case) –Alternative 3 has slightly higher overall In Service Management costs (WBS 5.0 costs) than legacy case Slightly higher combined program support, logistics, in service training, and second level engineering costs due primarily to legacy costs associated with ICMS subsystem which is not replaced in Alt. 3 WBS 3.0WBS 4.0WBS 5.0Total Legacyx.x Alt. 1x.x Alt. 2x.x Alt. 3x.x Alt. 4x.x Cost Comparisons (in PV$M)

9. 9 Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Aug. 24, 2010 Preliminary PV Benefits, Costs and NPVs Shortfall analysis shows PV benefits of $xxM based on xx-year life cycle –Safety PV benefit of $x.xM from reduced hazard-related risk –Efficiency PV benefit of $x.xM from reduced delays caused by ICMS outages Since the present value of each alternative’s benefit stream is only $x.xM, and incremental costs of each alternative are greater than this, none of the alternatives would provide net benefits over legacy case –Legacy Case: PV costs are $xx.xM –Alternative 1: PV costs $xx.xM  incremental cost = $xx.xM  NPV= -$xx.xM –Alternative 2: PV costs $xx.xM  incr. cost = $xx.xM  NPV= -$xx.xM –Alternative 3: PV costs $xx.xM  incr. cost = $xx.xM  NPV= -$xx.xM –Alternative 4: PV costs $xx.xM  incr. cost = $xx.xM  NPV= -$x.xM Legacy-case and alternative costs based on ROMs –Legacy does not include all relevant interlock control equipment –Costs for legacy case appear high relative to alternatives, especially second- level engineering costs –Legacy case second level engineering PV costs are $x.xM –Alternative 1 second level engineering PV costs are $x.0M –More rigorous second level engineering costs required (both legacy and alternatives) for better comparison All Alternatives have a negative NPV.

10. 10 Federal Aviation Administration NAVAID Control, Interlock and Monitoring Equipment (NCIME) Aug. 24, 2010 Next Steps Proceed to IARD under following recommendations: Do not enter into investment analysis as there is no business case Further evaluate the safety issue –Review each alternative regarding safety issue –Consider additional alternatives which could be more cost effective without any decrease in potential benefits Medium risk safety hazard only possible at airports with CAT III ILS No CAT III ILS at 2 ICMS airports – NY-LaGuardia and Phoenix –Determine impact of interlock violation risk mitigation efforts A failure modes effects and criticality analysis (FMECA) of the KC2081 Option 5 demonstrates an implemented reliability of greater than 1-(2 x 10E-8) Program office to return to EC with discussion regarding alternatives and their impact on safety –New risk adjusted costs for each alternative and legacy case need to be performed; new risk adjusted benefits for each alternative as well