9/17/2002 COSYSMO Usage Experience Panel: What is Happening at Lockheed Martin Garry Roedler, Lockheed Martin Engineering Process Improvement Center Pete McLoone, Lockheed Martin Integrated Systems and Solutions Garry Roedler, Lockheed Martin Engineering Process Improvement Center Pete McLoone, Lockheed Martin Integrated Systems and Solutions

Page 2 Talking Points  Collecting data points for the industry COSYSMO Effort across Lockheed Martin Business Areas  Electronics Systems has completed a usage pilot  Integrated Systems & Solutions has done some preliminary evaluation of the model Lockheed Martin is deeply involved in extending COSYSMO to Cover Risk and Schedule, distributing effort across the life cycle, and addressing Reuse Intending to do more local calibration and evaluation Promoting COSYSMO internally via Lockheed Martin Measurement Workshop and other communications mechanisms Contributing to the review of the COSYSMO User Manual and other aspects of the industry effort

Page 3 Collecting Data Reasonable success obtaining data points; need to capture many more Starting to obtain data relevant to the COSYSMO Risk Module Difficult to validate effort and size data without a lot of effort –Suspect very uneven methods for counting interfaces –“Engineered” system requirements seem to work better than customer system requirements –Labor hours are probably somewhat uneven in terms of “rules” utilized; cannot easily supply a EIA 632 breakdown –Cost drivers are sometimes misinterpreted but not a big problem; helps if a team of systems engineers is used and a COSYSMO expert can facilitate their determination

Page 4 Electronic Systems Pilot Had to adjust inputs because of reuse; did not calibrate to local data; default parameters generated high estimates compared to actual effort May be useful as a sanity check for SE cost estimates –Reuse and local calibration need to be added Academic COSYSMO is bare bones spreadsheet –Does not have features of other parametric models Reuse, local calibration capability, Class of cost, Schedule, Risk, ranges for parameters, Cost particular phase(s), labor distributions, summary reports, graphical outputs Commercial version SystemStar has some of these features Need a comprehensive User’s Guide (in development) –Better guidance on how to combine multiple questionnaire attributes into a single model parameter –Better definitions of attributes, with examples –Definitions of phases covered Next steps: more data points, local calibration, use reuse and risk profile and evaluate

Page 5 Integrated Systems and Solutions Did not do local calibration Results with four data points showed reasonable results But data collection was “quick and dirty” so not much confidence in the results Nonetheless, it appears with local calibration the model should be usable The road ahead involves more careful data collection to obtain sufficient data points to do local calibration properly

Page 6 Final Thoughts Local calibration is only as good as the data that goes into it Don’t underestimate the effort needed to obtain good data Experiment! –Is accuracy related to size driver weights? –Is accuracy related to cost driver weights? –Does accuracy get better if I throw away a size parameter? Provide quality data points to Ricardo

Page 7