June 17, 2004 Allison Wiley, Megan Dameron, Sarah Grinnell Brian Brophy, Dick Coleman, Jessica Summerville Transitioning from Parametric to Buildup Estimates

Outline Introduction Explanation of the Challenge Developing a Buildup Estimate Parametric Pullout Using a Toy Problem

Introduction Purpose of this Presentation – Discuss the challenge presented in transitioning from parametric to buildup cost estimates – Present some of the ideas and approaches considered to address the challenge – Generate ideas and discussion with the audience in order to advance our thinking

The Challenge The transition from a parametric to a buildup cost estimate – Develop cost estimates specific to a subsystem – Calculate the appropriate amount to remove from the parametric estimate for the system, in order to insert the buildup estimate Utility – Provides detailed information so that the cost of operations and support or specific production items can influence the system design – Buildups are instructive and can lead to improvements in parametric estimates

Developing a Buildup Estimate Why the transition? – There could be subsystems that behave in a way that is inherently different from the legacy subsystems in the cost data – It may not be possible to estimate the entire system at a detailed level, but there may be detailed information available for one or more subsystems Defining “buildup” – In this presentation, the word buildup refers to an estimate specific to a subsystem – Intuitively the authors had in mind traditional buildup estimates as well as analogies – Parametric estimates of a subsystem alone could also be considered

Parametric Pullout The term “parametric pullout” refers to the amount of the parametric estimate that is attributable to a specific subsystem – in short, the amount of the parametric estimate that is “pulled out” so that the buildup estimate may be inserted. The idea of parametric pullout and some of the methods considered to address this problem are explained in the next several slides with the use of a toy problem.

Parametric Pullout Introduction to the toy problem – You have a CER that estimates the cost of a car based on its weight – To reduce confusion, let’s call the car the TE-4 – You have recently developed a buildup estimate for the transmission alone, and you hope to eventually have buildup estimates for most of the major parts of the car – Let’s call the transmission the BE-2 – The challenge: Incorporate the buildup estimate for the BE-2 into the total cost estimate for the TE-4

Parametric Pullout (cont’d) Method 1: Historical Percentage – If the transmission of a car is historically 8% of car cost, remove 8% of the TE-4 estimated cost – Advantages: Easy to execute, also works for costs estimated using historical averages instead of CERs – Disadvantages: Method requires specific historical data that may not be available in all cases

Parametric Pullout Method 1: Historical Percentage – Toy Problem Historical Data $25,680 - $4,015 + $3,650 $25,680 x 15.6%

Parametric Pullout Method 1: Historical Percentages – Concerns – Consider the case where the BE-2 comprises a much larger percent of TE-4 weight than any legacy transmission and car. There is a concern that since the BE-2 weighs so much more, the percentage method may cause the removal of an inadequate amount of cost.

Tip: It is frequently best to take the CER result from the parameters of the whole system, and the CER result from the parameters of the whole system minus the subsystem and note the difference. This is especially important when CERs have multiple variables and/or are non-linear. Parametric Pullout Method 2: Parameter-based – Use the parameters of the CER to determine the correct piece to pullout – Advantage: Requires little data and is easily executed – Disadvantage: Subsystem may not have parameters comparable to the parent system. For example, there is not an intuitive way to pull the cost of a tire out of a car CER that uses the weight of the car’s electrical system.

Parametric Pullout Method 2: Parameter-based – Toy Problem – Run the car CER on the weight of the entire car – Run the car CER on the weight of the entire car, less the weight of the transmission – Subtracting the latter from the former yields the amount to be removed – Add in the buildup estimate for the transmission

Parametric Pullout Method 2: Parameter-based – Concerns – In a CER for a total system model, the parameters may mask or interplay with other parameters – For example, in a car CER based on electrical system weights, the electrical system weight acts as a proxy for other system weights (like the tires, the seats, and the frame of the car). – Intuitively, removing a few pounds of electrical system weight removes an electrical component, but mathematically it also removes all of the frame of the car that supports the electrical component. – It is important to understand the meaning of the CER.

Parametric Pullout Method 3: Obtain new CER – Example: – Y = legacy car cost – legacy transmission cost – X = (legacy car weight – legacy transmission weight)) – Results should yield reasonable F and t statistics, and a reasonable R 2 – Use TE-4 minus BE-2 weights in the new equation – Subtract the new result for a “transmission-less car” from the existing result for all of TE-4 – this is the pullout amount – Advantages: Provides a good check of the existing CER – Disadvantages: Requires significant resources and data to accomplish, some CERs will not respond well to this method without being completely reworked

Toy Problem (cont’d) Method 3: Re-run CER – Return to the original data that produced the CER – Subtract the cost of the transmission from the cost of the car, and the weight of the transmission from the weight of the car Car Cost = 4, (Car Weight) R 2 = 0.99 t and F significant Estimated weight of new car: 4,000 lbs Estimated cost of new car: $23,496

Toy Problem (cont’d) Method 3: Re-run CER – Rerun the regression (ensure that t and F statistics are still significant, and R 2 is reasonable) – Run the new CER on the weight of car to be estimated, minus the weight of the transmission “Transmission-less Car”Cost = 4, (Car Weight – Transmission Weight) R 2 = 0.99 t and F significant Estimated weight of new car w/o transmission: 4,000 – 500 = 3,500 lbs Estimated cost of new car w/o transmission: $19,791

Parametric Pullout (cont’d) Method 3: Obtain new CER (cont’d) – Concerns – This method may not behave intuitively – Changes are observed in the coefficients of unchanged parameters (when multiple parameters are present) – What is the expectation for behavior? Case 1: Method 2 and Method 3 yield the same result Car CER Possible “Transmission-less Car” CER If Method 2 and Method 3 are to yield the same result, the new CER must intersect the Car CER at the weight of the “transmission-less car”

Parametric Pullout (cont’d) Method 3: Obtain new CER (cont’d) – Concerns (cont’d) – What is the expectation for behavior? Car CER Possible “Transmission-less Car” CER Case 2: Method 3 removes more cost than Method 2 but the new CER behaves intuitively

Parametric Pullout (cont’d) Method 3: Obtain new CER (cont’d) – Concerns (cont’d) – What is the expectation for behavior? Case 3: Method 3 removes less cost than Method 2, the new CER may not behave intuitively Car CER Possible “Transmission-less Car” CER

Parametric Pullout (cont’d) Method 3: Obtain new CER (cont’d) – Concerns (cont’d) – What is the expectation for behavior? Case 4: The new CER yields an estimate higher than the existing CER – the method does not produce a parametric pullout Car CER Possible “Transmission-less Car” CER

Parametric Pullout Second order effects – In some models, particularly in operating and support cost, some elements are estimated using a parametric relationship to another cost element – If this is the case, it is important to keep careful track of order of operations, etc. … to be certain that the appropriate results are captured Warning: It is easy to get caught in the trap of thinking that the element which is being pulled out and put back in is “causing” changes to other elements in the model. This is not the case - second order changes to elements are attributable to a total change in the cost element value, not to the system being incorporated into the model.

Parametric Pullout Complexities beyond the toy problem – The toy problem demonstrates an example of a CER that is linear and uses just one variable – There are many complexities of the problem that occur in non-linear and/or multivariable CERs

Lessons Learned Buildups are useful because of the insight they may provide. The following has occurred during the authors’ experience with this topic: – A significant improvement in understanding of the nuances of a main source of historical data resulted – Led to an examination of several existing CERs and led directly to improvements in at least one – Allowed system designers to focus future cost and CAIV resources on specific areas of interest – The details of the process generated discussions with the system designers that provided further insight into design and cost issues of all types