COSYSMO Delphi Round 2 Results Compiled by: Ricardo Valerdi rvalerdi@sunset.usc.edu December 2003 (updated Jan 2, 2004)

Size Driver Results These four drivers help us determine how “big” a system is. The effort required for “nominal” # of system requirements serves as a basis for comparison to the other three size drivers. For example, the systems engineering effort required for difficult algorithms is 16.64 times greater than for nominal requirements. Easy Nominal Difficult # of System Requirements 0.49 1.00 4.23 # of Interfaces 1.50 3.92 8.21 # of Critical Algorithms 3.01 5.84 16.64 # of Operational Scenarios 10.31 24.58 53.85

Cost Driver Results Note 1: The Effort Multiplier Ratio (EMR) is the ratio of the large value over the small one (i.e., Requirements Understanding EMR is 1.71/.65 = 2.63) Note 2: Cost drivers are listed in order of appearance on the Delphi survey Note 3: Intermediate values (Low, High, and Very High) were updated as geometric ratios rather than arithmetic differences. EMRs did not change. Very Low Low Nominal High Very High Extra High EMR Requirements Understanding 1.71 1.31 1.00 0.81 0.65   2.63 Architecture Understanding 1.51 1.23 0.84 0.71 2.13 Level of Service Requirements 0.66 1.36 1.84 2.79 Migration Complexity 1.20 1.43 Technology Maturity 1.75 1.32 0.82 0.68 2.57 Documentation 0.77 0.88 1.15 # and diversity of installations/platforms 1.21 1.47 1.78 # of recursive levels in the design 0.67 1.22 1.48 2.21 Stakeholder team cohesion 1.45 0.86 0.74 1.96 Personnel/team capability 1.50 Personnel experience/continuity 1.54 1.24 2.26 Process capability 0.91 0.83 0.76 1.74 Multisite coordination 1.34 1.16 0.75 1.79 Tool support 1.39 1.18 1.88

Note: Cost drivers are listed in order of EMR value (or influence)

Profile of respondents (n=40) Average years of SW/SE experience: 18 Average years of cost modeling experience: 6 Application domains (most respondents selected more than one category): Other highlights: 22% of respondents also participated in Round 1 of the Delphi in January 2002 55% of the respondents were from Raytheon, Lockheed Martin, and Northrop Grumman