Applying Cost Improvement Theory to the Production of Military Space Systems— Where Do We Go From Here? Paul Killingsworth SCEA National Conference June, 2002 Bridging Engineering and Economics Since 1973

1/96 Improvement Rates Have Become Critical Parameters of Space Program Cost Estimates For a satellite program with 20+ units, the assumed CIC slope is a far more important determinant of total production cost than T1.

1/96 Application of Cost Improvement Theory to Spacecraft Has Not Kept Pace with Other Product Types Missile, avionics estimators have well-developed cost improvement methodologies –Long product histories, many programs –Long production runs –In other words—lots of hard empirical data Spacecraft estimators have less historical data to rely on. In addition— –Few units –Very slow production rates Where We Are Today— –A reliance on industry surveys and engineering judgment –Cost improvement rates are subject to gaming and compromise

1/96 Current Approaches Make Estimation of Curves for Specific Programs Difficult Focus is on fitting CI curves to limited spacecraft cost/quantity data for individual programs. Results are often averaged to derive a recommended slope for all satellite programs Or a consensus slope is used based on expert opinion (e.g. 95 %) Tailoring CICs to particular programs is highly subjective Variance around CIC estimates is clearly large, but unquantified Lot Cost Satellite Program X Ln(Lot Qty) Lot Cost Satellite Program Y

1/96 Where Do We Go From Here? More rigorous methodologies are needed for predicting feasible cost improvement rates for spacecraft As is the case for unit estimates, (T 1, T 100, etc), risk analysis on cost improvement assumptions should be possible

1/96 We Investigated Whether a Parametric Approach to Specifying CICs is Possible An effort to avoid using a single “average” or “accepted” rate—instead, tailor the CIC to the program We leveraged data from other product types (missiles, avionics, aircraft) We attempted to formulate “CIRs”—Cost Improvement Relationships CIC (%) Program X Program B Program A Program Z Program Y Multiple Parameters Related to Production

1/96 CIR Feasibility Study Overview Underlying Assumptions Identification and Specification Data Analysis Final CIRs and Caveats Conclusions and Recommendations

1/96 General Assumptions Underlying the Analysis Product type (spacecraft, aircraft, avionics) is not as important as production environment in estimating feasible CI rates –This hypothesis is statistically testable CI rates can be estimated based on system parameters more related to their production –Not specific to product types –This is also a statistically-testable hypothesis

1/96 Parameters Were Chosen in Three Categories Design Stability Low Stability High Stability System Complexity Low Complexity High Complexity System Scale Small Scale Large Scale Increasing potential for cost improvement in production

1/96 Parameters That Were Considered

1/96 INSTAB Parameter A somewhat subjective parameter defined as follows: INSTAB VALUE 5Significant upgrades/mods on every production unit 4Significant upgrades/mods with each lot 3Small upgrades/mods between lots 2Assembly line production with few changes 1Large-scale assembly line production with no changes

1/96 Specification Effects of parameters on CI rate were assumed to be multiplicative Error term assumed to be multiplicative Exponential form with log-linear OLS regression used CIC% = b o X 1 b1 X 2 b2 X 3 b3 e XnXn CIC

1/96 A Database of Spacecraft, Missiles, Avionics, and Aircraft Was Assembled 6 spacecraft programs: 9 small missile programs: 7 avionics programs: 5 aircraft programs: 1 large missile program DMSP DSP 1 (14-22) GPS II/IIA (14-40)DSCS III (B4-B14) MILSTAR IAdd’l program AV8BF-16 F-14F/A-18 F-15 AN/ALR-45D EOAN/ALR-67 RWRAN/APQ-128 TFR Apache TADS/PNVSAN/AAS-38 FLIR AN/ALR-45D RWRAN/APG-65 FCR PatriotStingerPhoenix HARMMLRSJavelin AMRAAMHellfireALCM Titan IV

1/96 Some General Results from the Data Analysis Avionics and missile dummy variables were not significant –Indicates similarity to spacecraft Aircraft dummy variable was moderately significant –However, added little to the regression fit –Magnitude of aircraft weight and $/lb parameters were much different from spacecraft, missile, and avionics data –Fit improved substantially with exclusion of aircraft data Conclusion: spacecraft, missiles, and avionics production processes are similar enough to consider together. Aircraft are not.

1/96 CIRs Were Developed With and Without the INSTAB Parameter INSTAB was highly correlated with CIC slope, accounting for most of the variation –Statistics indicated an excellent fit to the data –However, INSTAB, a subjective parameter, controlled the result –We kept looking A CIR using RATE, T1$/lb, and weight (lb) as drivers was developed –RATE had a fair correlation with CIC –Other parameters were significant with little multicollinearity –Statistics were acceptable CIR incorporates all three categories of desired parameters: design stability ( RATE ), complexity ( T1$/lb ), and scale ( weight )

1/96 CIR Results CIC% = * Rate ^ ( ) * ln(wgt) ^ ( ) * T1$/lb ^ ( ) T1$ = $K FY02

1/96 CIR Actuals vs Predicted Plot (with RATE parameter)

1/96 Conclusions CIR results were good enough to indicate that a more thorough data collection and analysis is warranted Spacecraft, missiles, and avionics can be considered part of the same population for purposes of CIC analysis Development of CIRs would allow for statistically- based risk analysis on production CI assumptions, similar to our unit cost estimates More research is needed to develop accepted CIRs to support spacecraft acquisition decision-making A reminder: This was a feasibility study.

1/96 BACKUP VUGRAPHS

1/96 CIR Results (with INSTAB) CIC% = 88.5 * Instab ^ * ln(wgt) ^ ( ) * T1$/lb ^ ( ) T1$ = $K FY02

1/96 Actual vs Predicted Plot (with INSTAB parameter)