1 st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Probabilistic NAS Platform George Hunter, Fred Wieland Ben Boisvert, Krishnakumar Ramamoorthy Sensis Corporation December 10, 2008

2 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Outline What is PNP? Team and development history Example uses of the model Software processes and testing Validation

3 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Outline What is PNP? Team and development history Example uses of the model Software processes and testing Validation

4 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 What is PNP? An fast-time and flexible NAS-wide simulation tool –Real-time or fast-time modes Half-hour runtime on a laptop, to simulate a day in the NAS –Physics-based: trajectories computed through integrating aerodynamic energy balance equations by varying the time-step size –System uncertainties (weather, security, operations …) –Plug-and-play architecture Dynamic clients (TFM, DAC, AOC, …) –An ATC community resource –Formal software development processes in place –Adaptable to current system or NextGen future concepts Uses –Environment in which to design, build and test decision support tools TFM, DAC, AOC, … Fast-time, real-time, shadow-mode –Potential NAS tool Service provider, operator, collaborative uses –Benefits assessment tool Fast-time tool to evaluate improved infrastructure, technology, procedures … Evaluates historic and future traffic scenarios in weather

5 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 PNP Architecture Probabilistic NAS Platform (PNP) Weather Data Reports MATLAB ® Scripting Interface MATLAB ® Scripting Interface NAS Database NAS Database NAS Simulation Performance Data Flight Data Graphical User Interface Plan View Display Graphical User Interface Plan View Display A fast-time physics-based (trajectory-based) NAS-wide modeling tool

6 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 PNP Architecture Probabilistic NAS Platform (PNP) Weather Data Reports SimObjects MATLAB ® Scripting Interface MATLAB ® Scripting Interface NAS Database NAS Database MATLAB ® Client External Client (Any Language) External Client (Any Language) Client As Middleware Client As Middleware Java Client Decision making NAS Simulation Performance Data Flight Data Graphical User Interface Plan View Display Graphical User Interface Plan View Display Prob-TFM A fast-time physics-based (trajectory-based) NAS-wide modeling tool

7 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 PNP Client Development TFM client development –ProbTFM (Sensis internal development) TFM client integration –C 2 (algorithms from and used with permission of Bob Hoffman, Metron) –Constrained LP (algorithms from and used with permission of NASA, Joey Rios) in progress DAC client integration –MxDAC (algorithms from and used with permission of Min Xue, NASA/UARC) AOC client development –Gaming behaviors (collaboration with GMU/Lance Sherry) in progress

8 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Capabilities Summary Real-time Fast-time Airport weather impact models Airspace weather impact models Weather-integrated decision making Probabilistic modeling / decision making Traffic flow management Dynamic airspace configuration Surface traffic modeling Terminal area modeling Super density operations Fuel burn modeling Emissions modeling Trajectory-based operations Separation assurance Plug-n-play Fast run-time ExistingCan Support√ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √

9 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Outline What is PNP? Team and development history Example uses of the model Software processes and testing Validation A fast-time physics-based (trajectory-based) NAS-wide modeling tool

10 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Team and Development History People Env’nment Data Funct’ality Projects Users Matlab Java/real-time WSI collaboration for real-time weather feed Dynamic clients NAS-wide, probabilistic Wx modeling and routing Client architecture InternalNWAGMU Krishnakumar Ramamoorthy Ben Boisvert Diego Escala Tae Lee Michelle Lu Huina Gao George Hunter JPDOFAA NASPAC NASA NRAs Web 2.0 ProjectSystemSystem leadSystemSoftware leadSoftware

11 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Outline What is PNP? Team and development history Example uses of the model Software processes and testing Validation A fast-time physics-based (trajectory-based) NAS-wide modeling tool

12 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Example Project Uses JPDO Modeling and Analysis –NextGen performance evaluation with weather FAA NASPAC Weather Modeling –Convection impact modeling for NASPAC NASA Gaming NRA –Evaluation of NextGen gaming with AOC clients NASA MetaSimulation NRA –Investigation of TFM + DAC interactions NASA SLDAST RFA –Evaluation of NextGen TFM concepts and models NASA Market-Based TFM NRA –Evaluation of NextGen market-based TFM concepts

13 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 NextGen Sensitivity Studies NextGen Performance Sensitivity Analysis Benefit of Improved Wx Forecasts Benefit of Using Clear Weather Forecasts Persistence forecast 11/16/06 Case 2: No distinction between clear and heavy weather forecast accuracy Case 1: Take advantage of improved forecast accuracy in clear weather NAS Performance Sensitivity Kris Ramamoorthy, George Hunter, "Evaluation of National Airspace System Performance Improvement With Four Dimensional Trajectories," AIAA Digital Avionics Systems Conference (DASC), Dallas, TX, October, 2007 George Hunter, Fred Wieland " Sensitivity of the National Airspace System Performance to Weather Forecast Accuracy," Integrated Communications, Navigation and Surveillance Conference (ICNS), Herndon, VA, May, 2008

14 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Market-Based TFM Studies UAL233 Delay Cost NAS Access Valuation Models SCC Delay George Hunter, et. al., "Toward an Economic Model to Incentivize Voluntary Optimization of NAS Traffic Flow," AIAA ATIO Conference, Anchorage, AK, September, 2008.

15 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Dynamic Airspace Configuration Nov 12, 2006, LAT=2, #Gen=40 ZFW FAA sectors George Hunter, "Preliminary Assessment of Interactions Between Traffic Flow Management and Dynamic Airspace Configuration Capabilities," AIAA Digital Avionics Systems Conference (DASC), St. Paul, MN, October, 2008.

16 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 AOC Dispatch Use Case Reroute with low probability of delay

17 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Outline What is PNP? Team and development history Example uses of the model Software processes and testing Validation A fast-time physics-based (trajectory-based) NAS-wide modeling tool

18 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Unit and System Testing Regression Testing Trunk Configuration Management Branch Configuration Management Project Monitoring & Control Development Tracking Quantitative Project Management Processes and Testing Cycle

19 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Project Monitoring and Control JIRA is used to track issues –Project Manager and Lead Software Engineer assign task priorities, due dates, and personnel. Weekly telecoms keep distributed team apprised of PNP and communications open Project Manager maintains a master schedule in MS- Project

20 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Development Tracking Software engineers use JIRA to track and status development efforts.

21 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Branch Configuration Management Software Engineers are responsible for creating branches from the trunk to develop fixes/enhancements. The Configuration Management of the software is accomplished with Subversion –Subversion is an open source version control system (

22 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Unit and System Testing Software Engineers are responsible for creating unit tests to verify the correctness of their code. The JIRA issue number is to be used throughout the code and unit tests for tracking purposes. Software Engineers are responsible for running their own system/function tests to verify their software. Once testing is validated, code is merged back on to the trunk.

23 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Trunk Configuration Management Once all validated JIRA issues are merged unto the trunk, regression testing is performed.

24 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Regression Testing Regression testing Aggregate results –Total delay –Total congestion –Traffic volume –#TFM initiatives –Runtime Different scenarios –Truncated demand set –Full demand set –Weather Automated –Weekly or as required Archived Graphical quick-look

25 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Quantitative Project Management Regression testing validation is performed and the release letter is updated. Release is tagged in Subversion. JIRA issues are closed. Documentation is updated to reflect changes in software.

26 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Outline What is PNP? Team and development history Example uses of the model Software processes and testing Validation A fast-time physics-based (trajectory-based) NAS-wide modeling tool

27 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 System-Level Engineering Validation ASPM / ETMS verification tests –Compare ASPM/ETMS data with simulation data Calibrate concept to match aggregate field observations –Models Trajectory data Airport capacities (VMC / IMC) Sector capacities in weather –Aggregate performance Mean flight delay Sector and airport overloadings –Detailed performance Flight delay by airport and time of day Overloading and delay patterns (Spatial and temporal)  Delays by airport and time of day  Sector and airport loading by time of day  Spatial loading patterns –Light and heavy weather days √

28 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 System-Level Software Verification Cross check sums –  Flights =  Operations at all airports –  Flight time =  Minutes from sector loads –  Sector load by sector =  Sector load by time –  Airport ops =  Flights using the airport in demand set –  Delays by flight =  Delays by time; and reroutes Weather data checks –Compare PNP/Metar airport capacity with ASPM AAR/ADR –Compare PNP/Metar airport capacity with ASPM IFR periods –Ensure  En route convection versus time of day is smooth –Ensure WxMAP ≤ MAP for all sector time bins Graphical –Ensure reroutes overlaid on weather make sense TFM Performance –Number of delays per flight, min and max flight delay –Maximum airport and sector overloading (ensure are reasonable)

29 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 System-Level Engineering Validation ASPM / ETMS verification tests –Compare ASPM/ETMS data with simulation data Calibrate concept to match aggregate field observations –Models Trajectory data Airport capacities (VMC / IMC) Sector capacities in weather –Aggregate performance Mean flight delay Sector and airport overloadings –Detailed performance Flight delay by airport and time of day Overloading and delay patterns (Spatial and temporal)  Delays by airport and time of day  Sector and airport loading by time of day  Spatial loading patterns –Light and heavy weather days √

30 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Trajectory Model Validation Compared to ETMS flight data (May 2008) N: 316 Mean: min Std dev: min Mean: 0.80 min Std dev: 6.51 min R 2 : Detrended for Range George Hunter, Ben Boisvert, Kris Ramamoorthy, "Advanced Traffic Flow Management Experiments for National Airspace Performance Improvement," 2007 Winter Simulation Conference, Washington, DC, December, 2007

31 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 ProbTFM Performance ASPM / ETMS verification tests –Compare ASPM/ETMS data with simulation data Calibrate concept to match aggregate field observations –Models Trajectory data Airport capacities (VMC / IMC), actual and forecasted Sector capacities in weather, actual and forecasted –Aggregate performance Mean flight delay Sector and airport loadings –Detailed performance Flight delay by airport and time of day Overloading and delay patterns (Spatial and temporal)  Delays by airport and time of day  Sector and airport loading by time of day  Spatial loading patterns –Light and heavy weather days √

32 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Compare With Field Observations January 7, 2007 (Similar results with other days) LAT = 0 LAT = 60 minutes LAT = 30 mins (14.5,1657) Compare to ETMS/ASPM –Forecast accuracies, Decision making horizon, Delay distribution

33 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Verification of Results ASPM / ETMS verification tests –Compare ASPM/ETMS data with simulation data Calibrate concept to match aggregate field observations –Models Trajectory data Airport capacities (VMC / IMC), actual and forecasted Sector capacities in weather, actual and forecasted –Aggregate performance Mean flight delay Sector and airport loadings –Detailed performance Flight delay by airport and time of day Overloading and delay patterns (Spatial and temporal)  Delays by airport and time of day  Sector and airport loading by time of day  Spatial loading patterns –Light and heavy weather days √

34 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 System Loading Patterns ProbTFM predicted, 14:45 GMT ETMS Actual, 14:45 GMT ETMS ProbTFM ETMS Underloading Overloading ProbTFM loading

35 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Verification of Results ASPM / ETMS verification tests –Compare ASPM/ETMS data with simulation data Calibrate concept to match aggregate field observations –Models Trajectory data Airport capacities (VMC / IMC), actual and forecasted Sector capacities in weather, actual and forecasted –Aggregate performance Mean flight delay Sector and airport loadings –Detailed performance Flight delay by airport and time of day Overloading and delay patterns (Spatial and temporal)  Delays by airport and time of day  Sector and airport loading by time of day  Spatial loading patterns –Light and heavy weather days, control days √

36 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Conclusion The development of PNP has benefited from lessons learned over past two decades in NAS system wide modeling –Plug and play simulation architecture –Supports both analytical and HITL studies –Adaptable to simulate current system as well as NextGen future concepts –Fast-time, physics-based –Formal software development processes in place –Probabilistic decision making and extensive weather modeling explicitly incorporated in tool

37 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Publications 1.George Hunter, "Preliminary Assessment of Interactions Between Traffic Flow Management and Dynamic Airspace Configuration Capabilities," AIAA Digital Avionics Systems Conference (DASC), St. Paul, MN, October, George Hunter, et. al., "Toward an Economic Model to Incentivize Voluntary Optimization of NAS Traffic Flow," AIAA ATIO Conference, Anchorage, AK, September, George Hunter, Fred Wieland " Sensitivity of the National Airspace System Performance to Weather Forecast Accuracy," Integrated Communications, Navigation and Surveillance Conference (ICNS), Herndon, VA, May, George Hunter, Kris Ramamoorthy, "Integration of terminal area probabilistic meteorological forecasts in NAS-wide traffic flow management decision making," 13th Conference on Aviation, Range and Aerospace Meteorology, New Orleans, LA, January, Kris Ramamoorthy, George Hunter, "The Integration of Meteorological Data in Air Traffic Management: Requirements and Sensitivities," 46th AIAA Aerospace Sciences Meeting and Exhibit, Reno, NV, January, George Hunter, Ben Boisvert, Kris Ramamoorthy, "Advanced Traffic Flow Management Experiments for National Airspace Performance Improvement," 2007 Winter Simulation Conference, Washington, DC, December, Kris Ramamoorthy, George Hunter, "Evaluation of National Airspace System Performance Improvement With Four Dimensional Trajectories," AIAA Digital Avionics Systems Conference (DASC), Dallas, TX, October, Kris Ramamoorthy, Ben Boisvert, George Hunter, "Sensitivity of Advanced Traffic Flow Management to Different Weather Scenarios," Integrated Communications, Navigation and Surveillance Conference (ICNS), Herndon, VA, May, George Hunter, Ben Boisvert, Kris Ramamoorthy, "Use of automated aviation weather forecasts in future NAS," The 87th American Meteorological Society Annual Meeting, San Antonio, TX, January, Kris Ramamoorthy, George Hunter, "Probabilistic Traffic Flow Management in the Presence of Inclement Weather and Other System Uncertainties," INFORMS Annual Meeting, Pittsburgh, PA, November, Kris Ramamoorthy, Ben Boisvert, George Hunter, "A Real-Time Probabilistic TFM Evaluation Tool," AIAA Digital Avionics Systems Conference (DASC), Portland, OR, October, George Hunter, Kris Ramamoorthy, Alexander Klein "Modeling and Performance of NAS in Inclement Weather," AIAA Aviation Technology, Integration and Operations (ATIO) Forum, Wichita, KS, September Kris Ramamoorthy, George Hunter, "A Trajectory-Based Probabilistic TFM Evaluation Tool and Experiment," Integrated Communications, Navigation and Surveillance Conference (ICNS), Baltimore, MD, May, Kris Ramamoorthy, George Hunter, "Avionics and National Airspace Architecture Strategies for Future Demand Scenarios in Inclement Weather," AIAA Digital Avionics Systems Conference (DASC), Crystal City, VA, October, George Hunter, Kris Ramamoorthy, Joe Post, "Evaluation of the Future National Airspace System in Heavy Weather," AIAA Aviation Technology, Integration and Operations (ATIO) Forum, Arlington, VA, September James D. Phillips, “An Accurate and Flexible Trajectory Analysis,” World Aviation Congress (SAE Paper ), Anaheim, CA, October 13-16, 1997.

38 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Questions?

39 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Backup

40 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 PNP Systems Requirements System requirements –PNP is a Java application –Hardware Memory: minimum 1GB, preferred 2GB CPU: Pentium (4) 3.2 GHz or better Video card: 128MB memory, preferred 256MB –Software Java JDK 6 MySQL Server –Third party licenses Eurocontrol BADA usage license

41 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Weather Days Ten weather days, two control days

42 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Weather Days Weather days –Spectrum of weather days Variation in weather type and intensity Variation in season –Support real-world comparison Support same sector data Variation in traffic demand volume and structure  Different days of week, holidays Control days

43 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08

44 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 NextGen Performance Sensitivity Analysis

45 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 En Route and Terminal Area Combined Sensitivities

46 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08

47 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Benefit of Improved Convection Forecasts

48 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Investment Analysis

49 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08

50 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Benefit of Using Clear Weather Forecasts

51 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Benefit Evaluation Persistence forecast, 11/16/06 Case 2: No distinction between clear and heavy weather forecast accuracy Case 1: Take advantage of improved forecast accuracy in clear weather

52 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08

53 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Market-Based TFM: Valuation of NAS Access

54 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Congestion-Delay Relationship Unconstrained sector congestion cost (SCC) for zero lookahead time (blue) and PNP-ProbTFM simulated delay (black) time histories for all en route NAS sectors and flights, respectively. SCC Delay

55 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Aggregate Delay Model Hypothesize a first-order lag transfer function Simulated delay Modeled delay

56 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Hypothesize a second-order transfer function Simulated delay Modeled delay Aggregate Delay Model

57 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Transfer Functions Summary

58 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Explicit Cost Model Evaluate cost of NAS access by removing the flight Remove one flight –11/16/06, UAL233, A320 –Morning departure from Bradley International (KBDL) to Chicago O’Hare airport (KORD) –Relatively high cost flight SCC

59 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Remove UAL233 Delay reduction by time bin in simulation run –Delay reduction of 8141 minutes

60 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08

61 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 NAS Performance Sensitivity Studies Performance sensitivity to: Delay distribution policy (most important factor) TFM system agility System forecasts (least important factor) Nov 12, 2006 ETMS/ASPM Minimum Delay Non Agile Minimum Delay Delay Distribution Non Agile Delay Distribution

62 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08

63 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Dynamic Airspace Configuration

64 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 NAS Sectorization Nov 12, 2006

65 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 MxDAC Afternoon Sectorization Nov 12, 2006, LAT=6, #Gen=20

66 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 MxDAC Midday Sectorization Nov 12, 2006, LAT=2, #Gen=40 Coeff_peak_ac_var=0.0 Coeff_avg_ac_var=0.0 Coeff_crossings=0.0 Coeff_transition_time=0.0 Coeff_residual_capacity=1.0

67 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Delay-Congestion Performance MxDAC off MxDAC on, LAT = 4 hrs MxDAC on, LAT = 2 hrs

68 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08

69 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Equity Analysis: Cost of Delay Distribution

70 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Cost of Distributing Delay RMS delay can be reduced by spreading delay to more flights –But at the cost of increased total delay Nov 12, 2006 $65/minute Increased delay distribution

71 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08

72 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 AOC Dispatch Use Case

73 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Dispatcher Successfully Finds a Reroute Reroute with low probability of delay

74 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08

75 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Project Monitoring and Control JIRA is used to track issues –Project Manager and Lead Software Engineer assign task priorities, due dates, and personnel. Weekly telecoms keep distributed team apprised of PNP and communications open Project Manager maintains a master schedule in MS- Project

76 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Development Tracking Software engineers use JIRA to track and status development efforts.

77 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Branch Configuration Management Software Engineers are responsible for creating branches from the trunk to develop fixes/enhancements. The Configuration Management of the software is accomplished with Subversion –Subversion is an open source version control system (

78 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Unit and System Testing Software Engineers are responsible for creating unit tests to verify the correctness of their code. The JIRA issue number is to be used throughout the code and unit tests for tracking purposes. Software Engineers are responsible for running their own system/function tests to verify their software. Once testing is validated, code is merged back on to the trunk.

79 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Trunk Configuration Management Once all validated JIRA issues are merged unto the trunk, regression testing is performed.

80 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Regression Testing Regression testing Aggregate results –Total delay –Total congestion –Traffic volume –#TFM initiatives –Runtime Different scenarios –Truncated demand set –Full demand set –Weather Automated –Weekly or as required Archived Graphical quick-look

81 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Quantitative Project Management Regression testing validation is performed and the release letter is updated. Release is tagged in Subversion. JIRA issues are closed. Documentation is updated to reflect changes in software.

82 1st Annual Workshop NAS-Wide Simulation in Support of NextGen, 12/10/08 Risk Management Lessons learned analysis –A wrap up meeting is held to discuss all issues on a project in which proactive steps can be documented to avoid the same mistakes