1. Jairus Hihn Jet Propulsion Laboratory, California Institute of Technology Domain-Oriented Modeling, Estimation and Improvement for Aerospace November 2, 2011 26 th International Forum on COCOMO and Systems/Software Cost Modeling

2. Background  JPL is a FFRDC for NASA operated by Caltech  JPL is a highly decentralized organization that allows great autonomy to its project managers at all levels  JPL’s Engineering and Science Directorate (3000 people) has twice been certified at CMMI ML 3 (2007, 2010)  Approximately 1000 people ($253M/year budget) in the software community  This raises the question  How do we do it  How do we measure it?  We turn process sins into virtues 2

3. Data Sources  Data Sources Metrics Collection at Key Milestones Software Inventory (2006, 2007, 2009 and 2011) Process performance measures (Tailoring Record, Work Product Checklist) Defect tracking systems (PRS, ISA, AAMS, local databases) SQI Surveys and Contact Log Product and Process Quality Assurance Activities Customer Feedback  Communicate results as widely as possible State of Software Report Noontime seminars Section manager meetings 3

4. Measuring Process Performance  Overall performance is used to establish baselines  Identifying the least performed activities provides a short list from which stakeholders can be engaged to discuss process improvement opportunities.  Metrics use is one of the least performed activities 4

5. JPL Cost and Planning Models  Collect data at major milestones for mission critical and mission support software  Labor intensive requiring numerous interviews  MSL flight software required 30-35 interviews  Enables us to  Calibrate cost models  Develop effort by role  use to do schedule models but they no longer work  Effort growth from PDR  Cost to repair defects  Size growth  Enables cost research  Data mining models developed with Tim Menzies (WVU) 5

6. Baselines and Trends  How are we doing? Trends Suggest Improvements in process discipline Projects following a disciplined process exhibit below average budget growth and higher productivity rates 66 Project Flight Software

7. NASA Software Cost improvement Task Year 1: Establish Process and Product Baseline  Product: Report that documents current state of practice at NASA with set of improvement recommendations and recommended metrics. Year 2:  Core team works to facilitate the implementation of high priority recommendations across the Agency through a tailorable Agency-wide process. Institute a cost metrics repository to gather information on cost estimate accuracy and cost estimation processes. Year 3:  Initiate a series of annual reports on NASA software cost estimation accuracy and cost estimation practices. Continue to monitor and make recommendations 7