1. University of Southern California Center for Systems and Software Engineering 1 Mauricio E. Peña Dr. Ricardo Valerdi March 8, 2011 Characterizing the Impact of Requirements Volatility on Systems Engineering Effort

2. University of Southern California Center for Systems and Software Engineering 2 Agenda 1:30 – 1:45 pmIntroductions & objectives 1:45 – 2:00 pmCOSYSMO overview and updates 2:00 - 2:30 pmCOSYSMO requirements volatility extension: introduction and research results 2:30 – 3:30 pmRequirements volatility measures: definitions & counting rules 3:30 – 3:45 pmBreak 3:45 – 4:45 pmSurvey Exercise: expected level of volatility and impact on systems engineering effort 4:45 – 5:00 pmDiscussion

3. University of Southern California Center for Systems and Software Engineering 3 Objectives of the Workshop Learn about COSYSMO and the latest research updates Explore the causes of requirements volatility and its impact on systems engineering effort Obtain feedback on a proposed requirements volatility extension to COSYSMO Reach agreement on definitions and rating scales for requirements volatility Provide an opportunity for participants to exchange lessons learned on requirements volatility and influence the direction of future research

4. University of Southern California Center for Systems and Software Engineering 4 COSYSMO Overview & Updates

5. University of Southern California Center for Systems and Software Engineering 5 CSSE Parametric Cost Models The Constructive Systems Engineering Cost Model (COSYSMO) was developed by the USC Center for Software and Systems Engineering (CSSE) in collaboration with INCOSE and Industry affiliates COSYSMO is the first generally-available parametric cost model designed to estimate Systems Engineering effort Built on experience from COCOMO 1981, COCOMO II –Most widely used software cost models worldwide –Developed with Affiliate funding, expertise, data support

6. University of Southern California Center for Systems and Software Engineering 6 COSYSMO Size* Drivers Effort Multipliers Effort Calibration # Requirements # Interfaces # Scenarios # Algorithms - Application factors -8 factors - Team factors -6 factors WBS guided by EIA/ANSI 632 COSYSMO Operational Concept Source: Valerdi, R. (2005) *Each weighted by complexity: easy, nominal, and difficult

7. University of Southern California Center for Systems and Software Engineering 7 Scope of the Model ISO/IEC 15288 – Common framework for describing the life cycle of systems EIA/ANSI 632 – Integrated set of fundamental systems engineering processes Source : Draft Report ISO Study Group May 2, 2000

8. University of Southern California Center for Systems and Software Engineering 8 Life Cycle Phase Definition Conceptualize phase focuses on identifying stakeholder needs, exploring different solution concepts, and proposing candidate solutions. The Development phase involves refining the system requirements, creating a solution description, and building a system. The Operational Test & Evaluation Phase involves verifying/validating the system and performing the appropriate inspections before it is delivered to the user The Transition to Operation Phase involves the transition to utilization of the system to satisfy the users’ needs. Source: Valerdi, R. (2005)

9. University of Southern California Center for Systems and Software Engineering 9 Parametric Cost Estimating Relationship Where: PM NS = effort in Person Months (Nominal Schedule) A = constant derived from historical project data Size = determined by computing the weighted average of the (4) size drivers E = represents diseconomy of scale n = number of cost drivers (14) EM = effort multiplier for the ith cost driver. The geometric product results in an overall effort adjustment factor to the nominal effort Source: Valerdi, R. (2005)

10. University of Southern California Center for Systems and Software Engineering 10 COSYSMO Cost Drivers Application Factors –Requirements understanding –Architecture understanding –Level of service requirements –Migration complexity –Technology Maturity –Documentation Match to Life Cycle Needs –# and Diversity of Installations/Platforms –# of Recursive Levels in the Design Team Factors –Stakeholder team cohesion –Personnel/team capability –Personnel experience/continuity –Process maturity –Multisite coordination –Tool support Source: Valerdi, R. (2005)

11. University of Southern California Center for Systems and Software Engineering 11 Cost Driver Rating Scales Very LowLowNominalHighVery HighExtra HighEMR Requirements Understanding1.871.371.000.770.60 3.12 Architecture Understanding1.641.281.000.810.65 2.52 Level of Service Requirements0.620.791.001.361.85 2.98 Migration Complexity 1.001.251.551.93 Technology Risk0.670.821.001.321.75 2.61 Documentation0.780.881.001.131.28 1.64 # and diversity of installations/platforms 1.001.231.521.87 # of recursive levels in the design0.760.871.001.211.47 1.93 Stakeholder team cohesion1.501.221.000.810.65 2.31 Personnel/team capability1.501.221.000.810.65 2.31 Personnel experience/continuity1.481.221.000.820.67 2.21 Process capability1.471.211.000.880.770.682.16 Multisite coordination1.391.181.000.900.800.721.93 Tool support1.391.181.000.850.72 1.93 Source: Valerdi, R. (2005)

12. University of Southern California Center for Systems and Software Engineering 12

13. University of Southern California Center for Systems and Software Engineering 13 COSYSMO 2.0 Operational Concept Source: Fortune (2009)

14. University of Southern California Center for Systems and Software Engineering 14 COSYSMO 2.0 Model Form Source: Fortune (2009)

15. University of Southern California Center for Systems and Software Engineering 15 Reuse Category Weights Source: Fortune (2009)

16. University of Southern California Center for Systems and Software Engineering 16 COSYSMO 2.0 Implementation Results Across 44 projects at 1 diversified organization Using COSYSMO: –PRED(.30) = 14% –PRED(.40) = 20% –PRED(.50) = 20% –R 2 = 0.50 Using COSYSMO 2.0: –PRED(.30) = 34% –PRED(.40) = 50% –PRED(.50) = 57% –R 2 = 0.72 Result: 36 of 44 (82%) estimates improved Source: Fortune (2009)

17. University of Southern California Center for Systems and Software Engineering 17 COSYSMO Extension: Requirements Volatility

18. University of Southern California Center for Systems and Software Engineering 18 Importance of Understanding Requirements Volatility Requirements volatility has been identified by numerous research studies as a risk factor and cost-driver of systems engineering projects 1 Requirements changes are costly, particularly in the later stages of the lifecycle process because the change may require rework of the design, verification and deployment plans 2 The Government Accountability Office (GAO) concluded in a 2004 report on the DoD’s acquisition of software-intensive weapons systems that missing, vague, or changing requirements are a major cause of project failure 3 System developers often lack effective methods and tools to account for and manage requirements volatility Source: 1- Boehm (1991), 2- Kotonya and Sommerville (1995), 3- GAO-04-393

19. University of Southern California Center for Systems and Software Engineering 19 Requirements Volatility is Expected Changes to requirements are a part of our increasingly complex systems & dynamic business environment –Stakeholders needs evolve rapidly –The customer may not be able to fully specify the system requirements up front –New requirements may emerge as knowledge of the system evolves –Requirements often change during the early phases of the project as a result of trades and negotiations Sources: Kotonya and Sommerville (1995); Reifer (2000) Requirements volatility must be anticipated and managed

20. University of Southern California Center for Systems and Software Engineering 20 Questions that will be covered How do we account for the impact of requirements volatility on functional size and effort estimates? What counting rules should be used for requirements volatility? –How do we distinguish between typical changes/iterations and unplanned changes that require more effort than originally projected? –How do we prevent from counting the elaboration of requirements as volatility (added requirements)? Is it feasible to track the volatility of other size drivers (interfaces, operational scenarios, algorithms)?

21. University of Southern California Center for Systems and Software Engineering 21 Requirements Volatility Definitions Requirements volatility is typically defined as the change in requirements (added, deleted, and modified) over a given time interval Also known as: Requirements creep: An increase in scope and number of system requirements Requirements churn: Instability in the requirements set – requirements are modified or re-worked without necessarily resulting in an increase in the total number of requirements Source: MIL-STD-498

22. University of Southern California Center for Systems and Software Engineering 22 Requirements Volatility Metrics (1 of 3) Systems Engineering Leading Indicators Guide, Version 2.0, 2010 Volatility expressed as a % of the total number of requirements

23. University of Southern California Center for Systems and Software Engineering 23 Requirements Volatility Metrics (2 of 3) Source: Hammer, T., Huffman, L., and Rosenberg, L. (1998). Volatility metrics track the number and type of changes over time

24. University of Southern California Center for Systems and Software Engineering 24 Requirements Volatility Metrics (3 of 3)

25. University of Southern California Center for Systems and Software Engineering 25 Systems Engineering Reviews and Acquisition Lifecycle Phases Source: DoD Instruction 5000.02 (2008) A Requirements baseline is established when the functional requirements and characteristics of the system are formally documented and placed under configuration control

26. University of Southern California Center for Systems and Software Engineering 26 Requirements and Evolutionary Acquisition Process Source: DoD Instruction 5000.02 (2008)

27. University of Southern California Center for Systems and Software Engineering 27 Requirements Trends as a Leading Indicator of Project Performance Evaluates trends in the growth, change, completeness and correctness of the system requirements. It helps to determine the stability and completeness of the system requirements which could potentially impact project performance Source: Systems Engineering Leading Indicators Guide, Version 2.0, 2010 Planned # of Requirements Actual # of Requirements Projected # of Requirements Number of Requirements Requirements Growth Trends Time SRRPDRCDR Corrective Action Taken

28. University of Southern California Center for Systems and Software Engineering 28 Implications to COSYSMO During the development of COSYSMO, volatility was identified as a relevant adjustment factor to the model’s size drivers However, there was insufficient data to incorporate volatility effects into the initial version of the model The primary objective of the research is to complete the requirements volatility extension to COSYSMO within the existing structure and scope of the model Source: Valerdi (2005)

29. University of Southern California Center for Systems and Software Engineering 29 COSYSMO Volatility Factor Source: Fortune (2009)

30. University of Southern California Center for Systems and Software Engineering 30 Overview of Research to Date Data were collected through surveys and interviews of subject-matter experts in four different workshops A variety of industries were represented with an emphasis on aerospace & defense Exploratory survey administered at: –Workshop # 1: 2010 USC-CSSE Annual Research Review –Workshop # 2: 2010 LAI Knowledge Exchange Event Survey exercises were conducted at: –Workshop # 3: 2010 Practical Software and Systems Measurement (PSM) Users Group Conference –Workshop # 4: University of Southern California 25 th Annual COCOMO Forum

31. University of Southern California Center for Systems and Software Engineering 31 Observations from the Literature and Exploratory Research 1.Requirements volatility is correlated with an increase in project size and cost 2.The level of requirements volatility is a function of the system life cycle phase and does not necessarily behave linearly 3.The cost / effort impact of a requirements change increases the later the change occurs in the system life cycle 4.The impact of requirements volatility varies depending on the type of change: added, deleted, or modified 5.Removing a requirement may not necessarily result in a net decrease in engineering effort and cost A causal model diagram was developed that relates technical, organizational and contextual project factors to requirements volatility

32. University of Southern California Center for Systems and Software Engineering 32 Requirements Volatility Causal Model Diagram Contextual / Environmental Changes Experienced Staff Changes in COTS Products Changes in Org / structure/ Process Poor Understanding of the System & Customer Needs Customer-driven Scope Change Changes in Co-dependent Systems Requirements Process Maturity Technology Maturity Requirements Volatility Sources: Kotonya and Sommerville (1995); Hammer et al. (1998); Malaiya and Denton (1999); Stark et al. (1999); Houston (2000); Zowghi and Nurmuliani (2002); Kulk and Verhoef 2008; Ferreira, S., Collofello, J., Shunk, D., and Mackulak, G. (2009)

33. University of Southern California Center for Systems and Software Engineering 33 Impact of Requirements Volatility Project Effort Rework / Defects Number of System Requirements Customer Satisfaction Quality Project Cost Project Schedule Requirements Volatility +/- Sources: Kotonya and Sommerville (1995); Hammer et al. (1998); Malaiya and Denton (1999); Stark et al. (1999); Houston (2000); Zowghi and Nurmuliani (2002); Kulk and Verhoef 2008; Ferreira, S., Collofello, J., Shunk, D., and Mackulak, G. (2009)

34. University of Southern California Center for Systems and Software Engineering 34 Exploratory Survey Developed to gather the perspectives of subject- matter experts on the causes, impacts, and expected level of requirements volatility for a given system of interest Organizations represented: –The Aerospace Corporation, Northrop Grumman Corporation –The Boeing Company, Softstar Systems, Raytheon –United Launch Alliance, Massachusetts Institute of Technology, University of Southern California, and –United States Army –Unites States Navy

35. University of Southern California Center for Systems and Software Engineering 35 Exploratory Survey Participant Background System Application Domain System H/W –S/W breakdown 22 respondents 23 years average industry experience Primarily from a Military/Defense and Space Systems Background Experienced on Systems with a fairly balanced H/W and S/W work content

36. University of Southern California Center for Systems and Software Engineering 36 Expected Level of Volatility Across Lifecycle Phases Most respondents expect >20% volatility during the conceptualize phase of the project, decreasing to <5% in the transition to operation phase Source: LAI Knowledge Exchange / CSSE ARR Survey (2010)

37. University of Southern California Center for Systems and Software Engineering 37 Impact of Hardware/Software Project Breakdown on Expected Volatility Operational Test & Evaluation Lifecycle Phase Transition to Operation Lifecycle Phase Respondents from S/W intensive projects tend to expect more volatility later in the lifecycle Source: LAI Knowledge Exchange / CSSE ARR Survey (2010)

38. University of Southern California Center for Systems and Software Engineering 38 % of respondents Impacts of Volatility In general, results of the survey support observations from the literature and causal model Most respondents stated that requirements volatility will cause a moderate to large increase in the number of system requirements and rework Moderate decreaseModerate Increase No impactLarge Increase

39. University of Southern California Center for Systems and Software Engineering 39 Ranked Causes of Requirements Volatility (Exploratory Survey) Contextual / Environmental Changes Experienced Staff Changes in COTS Products Changes in Org / structure/ Process Poor Understanding of the System & Customer Needs Customer-driven Scope Change Changes in Co-dependent Systems Requirements Process Maturity Technology Maturity Requirements Volatility 1 2 34 5 678 9 Sources: Kotonya and Sommerville (1995); Hammer et al. (1998); Malaiya and Denton (1999); Stark et al. (1999); Houston (2000); Zowghi and Nurmuliani (2002); Kulk and Verhoef 2008; Ferreira, S., Collofello, J., Shunk, D., and Mackulak, G. (2009)

40. University of Southern California Center for Systems and Software Engineering 40 Workshop # 3: PSM Conference Survey Exercise administered during the 2010 Practical Software and Systems Measurement Conference Attended by 9 participants from diverse engineering organizations and academic institutions Organizations Represented: –Distributed Management, Northrop Grumman –Lockheed Martin, Ericsson España –Samsung SDS, U.S. Navy –Massachusetts Institute of Technology (MIT)

41. University of Southern California Center for Systems and Software Engineering 41 Workshop # 3: Survey Exercise Participants were asked to: 1.Draw a requirements volatility profile across the lifecycle phases covered by COSYSMO 2.Draw an “ease of change” profile across the same life cycle phases to determine the volatility weighting factor 3.Discuss variation in 1 and 2 above for: 1.Large and small projects 2.Hardware and software projects 3.Development and recurring projects

42. University of Southern California Center for Systems and Software Engineering 42 Sample Volatility Profiles Source: Practical Software & Systems Measurement Conference Survey (2010)

43. University of Southern California Center for Systems and Software Engineering 43 Ease of Change Profile Cost Penalty defined as 1 / ease of change Average Ease of Change Factor (Estimated) Average Cost Penalty (Estimated) 0.8 0.40.10.05 1.25 2.510 20

44. University of Southern California Center for Systems and Software Engineering 44 Workshop #4: COCOMO Forum Attended by 15 participants from 10 different engineering organizations and academic institutions The participants were asked to estimate: Expected level of volatility –The cumulative volatility (%) over the four life cycle phases covered by COSYSMO –How those changes in requirements would be partitioned per life cycle phase –The breakdown of volatility per type of change (added, deleted, modified) Cost Penalty –Provided a change cost penalty as a function of lifecycle phase –Estimated the cost penalty per type of change

45. University of Southern California Center for Systems and Software Engineering 45 Expected Volatility per Lifecycle Phase Requirements Volatility (% changes) Expected Cumulative Volatility = 28%, STD = 14%

46. University of Southern California Center for Systems and Software Engineering 46 Weighting Factor per Lifecycle Phase 1 = no penalty

47. University of Southern California Center for Systems and Software Engineering 47 Breakdown per type of Change Source: COCOMO Forum Survey Exercise (2010) Expected Breakdown of Volatility per type of change: 34% added, 51% modified, 15% deleted

48. University of Southern California Center for Systems and Software Engineering 48 Proposed Requirements Volatility Extension

49. University of Southern California Center for Systems and Software Engineering 49 Requirements Volatility Size Driver Adjustment Factor REVL is defined as the % of the baseline set of requirements that changes throughout the lifecycle A weighting factor is added to account for the effort penalty due to rework This relationship is expressed through the following equation: Where, R x,r = Baseline number of requirements (easy, nominal, difficult) R eff = Effective number of requirements at the end of the project w v = Requirements volatility weighting factor Source: Boehm (2000)

50. University of Southern California Center for Systems and Software Engineering 50 Aggregate Weighting Factor As previously discussed, the effort penalty due to requirements volatility is likely to increase across lifecycle phases In order to capture this potential lifecycle phase dependency in the model, the following relationship is proposed: Where w v = Requirements volatility weighting factor w x,l = Weighting factor for added, deleted, or modified Θ x,l = % of total changes that were added, deleted or modified l = lifecycle phases

51. University of Southern California Center for Systems and Software Engineering 51 Proposed Model (2) As previously discussed, requirements volatility may be decomposed into added, modified or deleted requirement; each with a different weighting factor R v = w a · R added + w m · R modified + w d · R deleted Where, R v = Effective number of requirements changes w x = Weighting factor for added, deleted, or modified R x = Number of requirements added, modified, deleted

52. University of Southern California Center for Systems and Software Engineering 52 Requirements Volatility Measures

53. University of Southern California Center for Systems and Software Engineering 53 Requirements Size Driver Definition This driver represents the number of requirements for the system-of-interest at a specific level of design. The quantity of requirements includes those related to the effort involved in system engineering the system interfaces, system specific algorithms, and operational scenarios. Requirements may be functional, performance, feature, or service-oriented in nature depending on the methodology used for specification. They may also be defined by the customer or contractor. Each requirement may have effort associated with is such as V&V, functional decomposition, functional allocation, etc. System requirements can typically be quantified by counting the number of applicable shalls/wills/shoulds/mays in the system or marketing specification Source: Valerdi (2005)

54. University of Southern California Center for Systems and Software Engineering 54 Requirements Counting Rules 1.Determine the system of interest 2.Decompose system objectives, capabilities, or measures of effectiveness into requirements that can be tested, verified, or designed Different systems will exhibit different levels of requirements decomposition depending on the application domain, customer’s ability to write good system requirements, and the functional size of the system 3.Provide a graphical or narrative representation of the system of interest and how it relates to the rest of the system 4.Count the number of requirements in the system/marketing specification or the verification test matrix for the level of design in which systems engineering is taking place in the desired system of interest 5.Determine the complexity, volatility, and reuse of requirements Source: Valerdi (2005)

55. University of Southern California Center for Systems and Software Engineering 55 Requirements Volatility Measures Base Measures –Number of requirements changes –Number of added, deleted or modified requirements Measurement methods 1.Count the number of requirements changes from Engineering Change Proposals (ECPs) 2.Count the number and type of changes using a requirements management tool such as the Dynamic Object Oriented Requirements System (DOORS) 3.Use a text differencing tool to determine the number and type of requirements changes between two versions of a specification

56. University of Southern California Center for Systems and Software Engineering 56 Requirements Volatility Counting Rules We need to formulate specific counting rules for the volatility of the requirements size driver Questions for discussion: –How do we distinguish between typical changes/iterations and unplanned changes that require more effort than originally projected? –How do we prevent from counting the elaboration of requirements as volatility (added requirements)? –How do we avoid counting editorial changes (typos, etc.)? –How do we identify major changes in scope that could disrupt the results?

57. University of Southern California Center for Systems and Software Engineering 57 Cone of Uncertainty Source: Boehm et al., 2004

58. University of Southern California Center for Systems and Software Engineering 58 Requirements Decomposition Framework Sky level –Top-level objective –Summary use case Kite level –Additional information as to how the sky-level objective is to be will be satisfied Seal level –User level task –Environment in which the developer interacts with the stakeholder Underwater level –Detailed design and implementation Sources: Cockburn (2001); Valerdi (2005)

59. University of Southern California Center for Systems and Software Engineering 59 Requirements Elaboration (S/W Example) Capability Requirements Use Cases Use Case Steps expansion Contractor Customer expansion SLOC expansion Capability Goal Sources: Cockburn (2001); Malik (2010)

60. University of Southern California Center for Systems and Software Engineering 60 Elaboration Factors AverageStd. Dev. Study 1Study 2Study 1Study 2 Capability Goal to Capability Requirement 1.952.460.670.94 Capability Requirement to Use Case 4.890.891.490.55 Use Case to Use Case Steps 5.677.061.602.97 Use Case Steps to SLOC 266.9366.9172.7764.46 Source: Malik (2010)

61. University of Southern California Center for Systems and Software Engineering 61 Requirements Decomposition Example Objective: Broadcast television signals over the Continental United States (CONUS) compatible with 18 inch receive antennas 1.The system shall be able to receive a Ku-band signal from the customer ground station and downlink the signal to CONUS coverage with a minimum Equivalent isotropically radiated power (EIRP) of 30 dBm 1.1The system shall have a pointing accuracy of 0.5 degrees 1.2The system shall radiate a minimum of 3000 W of RF power –1.2.1 The system shall be able to produce 4000 W of DC power –1.2.2 The system shall be able to store 500 W-Hr of energy –1.2.3 The system shall be able to dissipate 2000 W of heat 1.2.3.1The radiator surface area shall be greater than… 1.2.3.2The radiator surface properties shall be……. 1.2.3.3Metal surfaces shall be blanketed……. Too High? Too Low? Just Right

62. University of Southern California Center for Systems and Software Engineering 62 Proposed Guidelines Count only requirements changes after the requirement set is baselined and under configuration control (SRR, PDR) Count only changes to requirements for the system- of-interest at a specific level of design (use COSYSMO requirements counting rules) Identify disruptive changes in scope / re-baselining in the data collection instrument Use counting logic rules to exclude multiple changes to a requirement in one day (avoid counting editorial changes)

63. University of Southern California Center for Systems and Software Engineering 63 Feedback on Data Collection Instrument Data Collection Instrument Link

64. University of Southern California Center for Systems and Software Engineering 64 Questions on the Approach Should we attempt to count changes per type of requirement (easy, nominal, difficult)? –Is it feasible? –Does it add value to the analysis? Do you keep metrics on the volatility of the other size drivers? –Operational scenarios –System interfaces –Algorithms Would measuring the impact of requirements volatility envelope the impact of changes to the other size drives?

65. University of Southern California Center for Systems and Software Engineering 65 Survey Exercise

66. University of Southern California Center for Systems and Software Engineering 66 Thank you!

67. University of Southern California Center for Systems and Software Engineering 67 References Boehm, B. (1991). “Software Risk Management: Principles and Practices.” IEEE Software. Vol 8 (No.1).pp 32-41 Boehm, B., Abts, C., Brown, A.W., Chulani, S., Clark, B., Horowitz, E., Madachy, R., Reifer, D.J., and Steece, B. (2000). Software Cost Estimation with COCOMO II. Prentice Hall B. Boehm et al., Guidelines for Model-Based (System) Architecting and Software Engineering (MBASE) Version 2.4.1, USC-CSE, 2004. Cockburn, A. (2001). Writing Effective Use Cases, Addison-Wesley. EIA (2002). EIA-731.1 - Systems Engineering Capability Model Ferreira, S., Collofello, J., Shunk, D., and Mackulak, G. (2009). “Understanding the effects of requirements volatility in software engineering by using analytical modeling and software process simulation.” The Journal of Systems and Software. Vol. 82, pp 1568-1577. Fortune, J. (2009) Estimating Systems Engineering Reuse with the Constructive Systems Engineering Cost Model (COSYSMO 2.0). Doctoral Dissertation. University of Southern California General Accounting Office (2004). Stronger Management Practices are Needed to Improve DOD’s Software-intensive Weapon Acquisitions (GAO-04-393). Defense Acquisitions. Hammer, T., Huffman, L., and Rosenberg, L. (1998). “Doing requirements right the first time.” Crosstalk, the Journal of Defense Software Engineering. Pp 20-25. Houston, Dan X. (2000). A Software Project Simulation Model for Risk Management, Ph.D. Dissertation, Arizona State University IEEE (1990). IEEE Standard Glossary of Software Engineering Terminology (ANSI), [1-55937-067-X] [SH13748-NYF], IEEE INCOSE (2010). Systems Engineering Handbook. INCOSE Version 3.2 Kotonya, G., Sommerville, I., (1998). Requirements Engineering: Processes and Techniques. John Wiley and Sons, Ltd. Malik, A. (2010). Quantitative and Qualitative Analyses of Requirements Elaboration for Early Software Size Estimation. Doctoral Dissertation. University of Southern California. MIL-STD-498. 1994. Software Development and Documentation. U.S. Department of Defense. Reifer, Donald J. 2000. “Requirements Management: The Search for Nirvana.” IEEE Software. Vol 17 (No. 3), pp 45-47 Roedler, G. and Rhodes, D. (2007). Systems engineering leading indicators guide. Version 1. Massachusetts Institute of Technology, INCOSE, and PSM Valerdi, R. (2005). The constructive systems engineering cost model (COSYSMO). Doctoral Dissertation. University of Southern California Weinberg, G. (1993). Quality Software Management: Volume 2 First-Order Measurement. Dorset House. Zowghi, D. and Nurmuliani, N. (2002). A Study of the Impact of Requirements Volatility on Software Project Performance. Proceedings of the Ninth Asia-Pacific Software Engineering Conference

68. University of Southern California Center for Systems and Software Engineering 68 Call for Participation In order to complete the requirements volatility extension of COSYSMO, we are seeking industry data for engineering projects in terms of: –Systems engineering effort actuals (labor hours) –Requirements volatility: the number of requirements, added, deleted, and modified added after the requirements baseline By providing these data your organization will benefit by: –Improving its ability to estimate the impact of requirements changes on project cost –Calibrating and tailoring the updated Model for your application domain USC-CSSE and LAI at MIT have proven processes in place to ensure the confidentiality and protection of the data with its Corporate Affiliates and Consortium Members Contact: Mauricio E. Pena [mauricip@usc.edu] Ricardo Valerdi [rvalerdi@mit.edu]