Productivity Data Analysis and Issues Brad Clark, Thomas Tan USC CSSE Annual Research Review March 8, 2010

USC CSSE Annual Research Review - Mar 2010 Table of Contents Background Productivity Data Analysis by Application Domain Reducing the number of domains: Application Difficulty Topics for further discussion This work is sponsored by the Air Force Cost Analysis Agency USC CSSE Annual Research Review - Mar 2010

USC CSSE Annual Research Review - Mar 2010 Background DoD has been collecting Software resource data for a number of years Product and development description Product size Resources and schedule Product quality Analyzing ~140 records out of ~300 Additional data is coming in Objective: Improved cost estimation of future DoD software-intensive systems, as well as to the DoD cost community. Characterize different Application Domains within DoD Analyze collected data for simple cost estimating relationships within each domain Develop rules-of-thumb for missing data Make collected data useful to oversight and management entities USC CSSE Annual Research Review - Mar 2010

Software Resources Data Report USC CSSE Annual Research Review - Mar 2010

USC CSSE Annual Research Review - Mar 2010 SRDR Data Missing Domains: Internet, Maintenance and Diagnostics, Spacecraft bus Notes: SRDR: Software Resources Data Report USC CSSE Annual Research Review - Mar 2010

USC CSSE Annual Research Review - Mar 2010 Preliminary Results - Do Not Use! USC CSSE Annual Research Review - Mar 2010

Simple Cost Estimating Relationships PM = A * (EKSLOC)B Preliminary Results - Do Not Use! Notes: PM: Person Months (152 labor hours / month) EKSLOC: Equivalent Thousands of Source Lines of Code USC CSSE Annual Research Review - Mar 2010

USC CSSE Annual Research Review - Mar 2010 Sizing Issues -1 Multiple SLOC counting methods Physical: total number of lines in a file Non-commented Source: no blank or comment lines Logical No Deleted Code Counts SLOC Conversion Experiment Use the results of USC’s Code Count Tool to find conversion ratios Physical to Logical NCSS to Logical Results segregated by programming language USC CSSE Annual Research Review - Mar 2010

NCSS to Logical Conversion Ada: 45% C/C++: 61% C#: 61% Java: 72% USC CSSE Annual Research Review - Mar 2010

USC CSSE Annual Research Review - Mar 2010 Sizing Issues -2 No Modified Code parameters Percent Design Modified (DM) Percent Code Modified (CM) Percent Integration and Test Modified (IM) Software Understanding (SU) Programmer Unfamiliarity (UNFM) Program interviews provided parameters for some records USC CSSE Annual Research Review - Mar 2010

USC CSSE Annual Research Review - Mar 2010 Effort Issues Missing effort reporting for different lifecycle phases Software requirements analysis (REQ) Software architectural design (ARCH) Software coding and testing (CODE) Software integration (INT) Software qualification testing (QT) Software management, CM, QA, etc. (Other – very inconsistent) USC CSSE Annual Research Review - Mar 2010

Collapsing Application Domains Propose to reduce the number of application domains Currently have a “sparse” data table Use a model-independent approach 5-level scale to capture the “difficulty” (and therefore impact) of an application domain on productivity USC CSSE Annual Research Review - Mar 2010

Software Application Difficulties Difficulty would be described in terms of required software reliability, database size, product complexity, integration complexity, information assurance, real-time requirements, different levels of developmental risks, etc. USC CSSE Annual Research Review - Mar 2010

Simulation and Modeling Weapon Delivery and Control Application Difficulty Issues Application Domains Very Easy Easy Nominal Challenging Very Challenging Business Systems   Large biz system Trillion $/day transaction Internet Simple web pages Web application (shopping) Mega-web application Tools and Tool Systems Verification tools Safety critical Scientific Systems Offline data reduction Large dataset Simulation and Modeling Low fidelity simulator Physical phenomenon Test and Evaluation Usual Distributed debugging Training Set of screens Simulation network Command and Control Taxi-cab dispatch SOS (C4ISR) Mission Management Multi-level security and safety Weapon Delivery and Control Weapon space Safety Communications Noise, anomalies handling Radio Safety/Security Frequency-hopping USC CSSE Annual Research Review - Mar 2010

USC CSSE Annual Research Review - Mar 2010 For more information, contact: Thomas Tan thomast@usc.edu 626-617-1128 Or Brad Clark bkclark@usc.edu 703-754-0115 Questions? USC CSSE Annual Research Review - Mar 2010