1. June 17, 2004 Allison Wiley, Megan Dameron, Sarah Grinnell Brian Brophy, Dick Coleman, Jessica Summerville Transitioning from Parametric to Buildup Estimates

2. allison.wiley@ngc.com, 703.961.34322 Outline Introduction Explanation of the Challenge Developing a Buildup Estimate Parametric Pullout Using a Toy Problem

3. allison.wiley@ngc.com, 703.961.34323 Introduction Purpose of this Presentation – Discuss the challenge presented in transitioning from parametric to buildup cost estimates – Present some of the ideas and approaches considered to address the challenge – Generate ideas and discussion with the audience in order to advance our thinking

4. allison.wiley@ngc.com, 703.961.34324 The Challenge The transition from a parametric to a buildup cost estimate – Develop cost estimates specific to a subsystem – Calculate the appropriate amount to remove from the parametric estimate for the system, in order to insert the buildup estimate Utility – Provides detailed information so that the cost of operations and support or specific production items can influence the system design – Buildups are instructive and can lead to improvements in parametric estimates

5. allison.wiley@ngc.com, 703.961.34325 Developing a Buildup Estimate Why the transition? – There could be subsystems that behave in a way that is inherently different from the legacy subsystems in the cost data – It may not be possible to estimate the entire system at a detailed level, but there may be detailed information available for one or more subsystems Defining “buildup” – In this presentation, the word buildup refers to an estimate specific to a subsystem – Intuitively the authors had in mind traditional buildup estimates as well as analogies – Parametric estimates of a subsystem alone could also be considered

6. allison.wiley@ngc.com, 703.961.34326 Parametric Pullout The term “parametric pullout” refers to the amount of the parametric estimate that is attributable to a specific subsystem – in short, the amount of the parametric estimate that is “pulled out” so that the buildup estimate may be inserted. The idea of parametric pullout and some of the methods considered to address this problem are explained in the next several slides with the use of a toy problem.

7. allison.wiley@ngc.com, 703.961.34327 Parametric Pullout Introduction to the toy problem – You have a CER that estimates the cost of a car based on its weight – To reduce confusion, let’s call the car the TE-4 – You have recently developed a buildup estimate for the transmission alone, and you hope to eventually have buildup estimates for most of the major parts of the car – Let’s call the transmission the BE-2 – The challenge: Incorporate the buildup estimate for the BE-2 into the total cost estimate for the TE-4

8. allison.wiley@ngc.com, 703.961.34328 Parametric Pullout (cont’d) Method 1: Historical Percentage – If the transmission of a car is historically 8% of car cost, remove 8% of the TE-4 estimated cost – Advantages: Easy to execute, also works for costs estimated using historical averages instead of CERs – Disadvantages: Method requires specific historical data that may not be available in all cases

9. allison.wiley@ngc.com, 703.961.34329 Parametric Pullout Method 1: Historical Percentage – Toy Problem Historical Data $25,680 - $4,015 + $3,650 $25,680 x 15.6%

10. allison.wiley@ngc.com, 703.961.343210 Parametric Pullout Method 1: Historical Percentages – Concerns – Consider the case where the BE-2 comprises a much larger percent of TE-4 weight than any legacy transmission and car. There is a concern that since the BE-2 weighs so much more, the percentage method may cause the removal of an inadequate amount of cost.

11. allison.wiley@ngc.com, 703.961.343211 Tip: It is frequently best to take the CER result from the parameters of the whole system, and the CER result from the parameters of the whole system minus the subsystem and note the difference. This is especially important when CERs have multiple variables and/or are non-linear. Parametric Pullout Method 2: Parameter-based – Use the parameters of the CER to determine the correct piece to pullout – Advantage: Requires little data and is easily executed – Disadvantage: Subsystem may not have parameters comparable to the parent system. For example, there is not an intuitive way to pull the cost of a tire out of a car CER that uses the weight of the car’s electrical system.

12. allison.wiley@ngc.com, 703.961.343212 Parametric Pullout Method 2: Parameter-based – Toy Problem – Run the car CER on the weight of the entire car – Run the car CER on the weight of the entire car, less the weight of the transmission – Subtracting the latter from the former yields the amount to be removed – Add in the buildup estimate for the transmission

13. allison.wiley@ngc.com, 703.961.343213 Parametric Pullout Method 2: Parameter-based – Concerns – In a CER for a total system model, the parameters may mask or interplay with other parameters – For example, in a car CER based on electrical system weights, the electrical system weight acts as a proxy for other system weights (like the tires, the seats, and the frame of the car). – Intuitively, removing a few pounds of electrical system weight removes an electrical component, but mathematically it also removes all of the frame of the car that supports the electrical component. – It is important to understand the meaning of the CER.

14. allison.wiley@ngc.com, 703.961.343214 Parametric Pullout Method 3: Obtain new CER – Example: – Y = legacy car cost – legacy transmission cost – X = (legacy car weight – legacy transmission weight)) – Results should yield reasonable F and t statistics, and a reasonable R 2 – Use TE-4 minus BE-2 weights in the new equation – Subtract the new result for a “transmission-less car” from the existing result for all of TE-4 – this is the pullout amount – Advantages: Provides a good check of the existing CER – Disadvantages: Requires significant resources and data to accomplish, some CERs will not respond well to this method without being completely reworked

15. allison.wiley@ngc.com, 703.961.343215 Toy Problem (cont’d) Method 3: Re-run CER – Return to the original data that produced the CER – Subtract the cost of the transmission from the cost of the car, and the weight of the transmission from the weight of the car Car Cost = 4,499 + 4.75(Car Weight) R 2 = 0.99 t and F significant Estimated weight of new car: 4,000 lbs Estimated cost of new car: $23,496

16. allison.wiley@ngc.com, 703.961.343216 Toy Problem (cont’d) Method 3: Re-run CER – Rerun the regression (ensure that t and F statistics are still significant, and R 2 is reasonable) – Run the new CER on the weight of car to be estimated, minus the weight of the transmission “Transmission-less Car”Cost = 4,167 + 4.46(Car Weight – Transmission Weight) R 2 = 0.99 t and F significant Estimated weight of new car w/o transmission: 4,000 – 500 = 3,500 lbs Estimated cost of new car w/o transmission: $19,791

17. allison.wiley@ngc.com, 703.961.343217 Parametric Pullout (cont’d) Method 3: Obtain new CER (cont’d) – Concerns – This method may not behave intuitively – Changes are observed in the coefficients of unchanged parameters (when multiple parameters are present) – What is the expectation for behavior? Case 1: Method 2 and Method 3 yield the same result Car CER Possible “Transmission-less Car” CER If Method 2 and Method 3 are to yield the same result, the new CER must intersect the Car CER at the weight of the “transmission-less car”

18. allison.wiley@ngc.com, 703.961.343218 Parametric Pullout (cont’d) Method 3: Obtain new CER (cont’d) – Concerns (cont’d) – What is the expectation for behavior? Car CER Possible “Transmission-less Car” CER Case 2: Method 3 removes more cost than Method 2 but the new CER behaves intuitively

19. allison.wiley@ngc.com, 703.961.343219 Parametric Pullout (cont’d) Method 3: Obtain new CER (cont’d) – Concerns (cont’d) – What is the expectation for behavior? Case 3: Method 3 removes less cost than Method 2, the new CER may not behave intuitively Car CER Possible “Transmission-less Car” CER

20. allison.wiley@ngc.com, 703.961.343220 Parametric Pullout (cont’d) Method 3: Obtain new CER (cont’d) – Concerns (cont’d) – What is the expectation for behavior? Case 4: The new CER yields an estimate higher than the existing CER – the method does not produce a parametric pullout Car CER Possible “Transmission-less Car” CER

21. allison.wiley@ngc.com, 703.961.343221 Parametric Pullout Second order effects – In some models, particularly in operating and support cost, some elements are estimated using a parametric relationship to another cost element – If this is the case, it is important to keep careful track of order of operations, etc. … to be certain that the appropriate results are captured Warning: It is easy to get caught in the trap of thinking that the element which is being pulled out and put back in is “causing” changes to other elements in the model. This is not the case - second order changes to elements are attributable to a total change in the cost element value, not to the system being incorporated into the model.

22. allison.wiley@ngc.com, 703.961.343222 Parametric Pullout Complexities beyond the toy problem – The toy problem demonstrates an example of a CER that is linear and uses just one variable – There are many complexities of the problem that occur in non-linear and/or multivariable CERs

23. allison.wiley@ngc.com, 703.961.343223 Lessons Learned Buildups are useful because of the insight they may provide. The following has occurred during the authors’ experience with this topic: – A significant improvement in understanding of the nuances of a main source of historical data resulted – Led to an examination of several existing CERs and led directly to improvements in at least one – Allowed system designers to focus future cost and CAIV resources on specific areas of interest – The details of the process generated discussions with the system designers that provided further insight into design and cost issues of all types