An Empirical Study of Requirements-to-Code Elaboration Factors Ali Afzal Malik, Supannika Koolmanojwong, Barry W. Boehm {alimalik, koolmano, boehm} @usc.edu 24th International Forum on COCOMO and Systems/Software Cost Modeling 9/22/2018 ©USC-CSSE

Outline Introduction Motivation Related Work Empirical Setting Method Results Case Study Conclusions & Future Work 9/22/2018 ©USC-CSSE

Introduction Requirements elaboration Elaboration factor 9/22/2018 ©USC-CSSE

Requirements Elaboration Requirements Level Cockburn Metaphor Objectives Cloud Shall Statements Kite Use Cases Sea Level Use Case Steps Fish SLOC Clam 9/22/2018 ©USC-CSSE

Elaboration Factor Set of elaboration factors for each project {EF(1, 2), EF(2, 3), …, EF(n-1, n)} 9/22/2018 ©USC-CSSE

Motivation Importance of size in cost estimation Cost estimation models COCOMO II: SLOC, FP Problem: Not available during inception COSYSMO: # of Requirements Problem: Inconsistency in level of detail Elaboration factors solve these problems Enable early cost estimation Achieve consistency in level of detail 9/22/2018 ©USC-CSSE

Related Work (Cockburn, 2001) (Antón, 1996) Hierarchical refinement of use cases (Antón, 1996) GBRAM (Letier and van Lamsweerde, 2002) Agent-based tactics (Malik and Boehm, 2008) Quantitative analysis of requirements elaboration 9/22/2018 ©USC-CSSE

Empirical Setting 25 e-services projects done at USC Similarities Real-client SAIV: 24-weeks Architected-agile process Custom-development Differences Domains Types 9/22/2018 ©USC-CSSE

Method Data collection process Define mapping with Cockburn metaphor Identify relevant artifacts Retrieve requirements data from artifacts Normalize requirements data Calculate elaboration factors 9/22/2018 ©USC-CSSE

Step 1: Mapping Cockburn Metaphor Requirements Level Cloud Capability goals Kite Capability requirements Sea Level Use cases Fish Use case steps Clam SLOC 9/22/2018 ©USC-CSSE

Step 2: Relevant Artifacts Requirements Level Artifact Capability goals Operational Concept Description Capability requirements System and Software Requirements Definition Use cases System and Software Architecture Description Use case steps SLOC Source Code 9/22/2018 ©USC-CSSE

Step 3: Data Retrieval SLOC Rest Code counting tools Inspection 9/22/2018 ©USC-CSSE

Step 4: Normalization Improves Examples Consistency in level of detail Accuracy of analysis Examples CG: merge low-level goals CR: split aggregate requirements SLOC: filter auto-generated code 9/22/2018 ©USC-CSSE

Elaboration Factor Formula Step 5: Calculate EFs Elaboration Stage Elaboration Factor Formula Cockburn Metaphor E-Services Projects Cloud-to-Kite Capability Goals-to-Capability Requirements Kite-to-Sea Level Capability Requirements-to-Use Cases Sea Level-to-Fish Use Cases-to-Use Case Steps Fish-to-Clam Use Case Steps-to-SLOC 9/22/2018 ©USC-CSSE

Results Distribution of EFs EF statistics Utility of EFs 9/22/2018 ©USC-CSSE

Distribution of EFs 1 3 2 4 9/22/2018 ©USC-CSSE

EF Statistics Statistic Elaboration Factors Average 2.46 0.89 7.06 66.91 Median 2.75 6.57 40.72 Standard Deviation 0.94 0.55 2.97 64.46 9/22/2018 ©USC-CSSE

Utility Early SLOC estimation of similar new projects Most likely SLOC and 1-sd confidence interval Example: New e-services project; CG = 5 Most Likely SLOC = 5 * 2.46 * 0.89 * 7.06 * 66.91 ≈ 5171 Minimum SLOC = 5 * (2.46 – 0.94) * (0.89 – 0.55) * (7.06 – 2.97) * (66.91 – 64.46) ≈ 26 Maximum SLOC = 5 * (2.46 + 0.94) * (0.89 + 0.55) * (7.06 + 2.97) * (66.91 + 64.46) ≈ 32, 256 9/22/2018 ©USC-CSSE

Number of Requirements Case Study Large IBM projects Ball park figures from (Hopkins & Jenkins, 2008) Requirements Level Number of Requirements Business events 50 Use cases 200 Main steps 1, 500 Detailed operations 20, 000 SLOC 1, 500, 000 9/22/2018 ©USC-CSSE

Case Study cont… Elaboration Stage Elaboration Factor Cockburn Metaphor Large IBM Projects Cloud-to-Kite Business Events-to-Use Cases 4.00 Kite-to-Sea Level Use Cases-to-Main Steps 7.50 Sea Level-to-Fish Main Steps-to-Detailed Operations 13.33 Fish-to-Clam Detailed Operations-to-SLOC 75.00 9/22/2018 ©USC-CSSE

Case Study cont… EFs helpful in early sizing and cost estimation of similar large projects Example: New project with 60 business events Rough size = 60 * 4 * 7.5 * 13.33 * 75 ≈ 1, 800 KSLOC Caveat: code reuse Best approach is to obtain confidence intervals for SLOC estimates using EF distributions 9/22/2018 ©USC-CSSE

Conclusions & Future Work Entire process of requirements elaboration can be quantified using a set of EFs Sets of EFs helpful in early sizing and cost estimation of similar projects Same approach can be used to study elaboration of different types of project Future work Framework for predicting variance in EFs Other mappings More elaboration data of real projects! 9/22/2018 ©USC-CSSE

Further Information Details and a complete list of references available in accompanying paper 9/22/2018 ©USC-CSSE