Demonstrating Early Value in Software System Projects Using Software and System Cost Models Anca-Juliana Stoica, IT Dept, UU 26th International Forum on COCOMO and Systems/Software Cost Modeling University of Southern California, Los Angeles, CA November 2-4, 2011

Informationsteknologi Institutionen för informationsteknologi | Agenda Research Background Value-Based Software Engineering Software and System Cost Models Balanced Scorecard Method Algorithm for Applying VBSE Collecting Project Information and Questionnaire Experience Database Case Study Final Remarks and Future Work

Informationsteknologi Institutionen för informationsteknologi | What is value? -Value: worth in usefulness or importance to the possessor; utility or merit -Value of a system/software project: elicited by the project stakeholders’ values -Early value: value generated early in the project lifecycle

Informationsteknologi Institutionen för informationsteknologi | Value-Based Software Engineering - Traditional Software Engineering (SE) is not enough to make projects succeed -Value-Based Software Engineering (VBSE) is the integration a software system’s stakeholder value propositions into the system’s definition, design, development, deployment, evolution Includes aspects not covered by SE that are critical to the system’s success -VBSE integrates value considerations into current and emerging software engineering principles and practices while developing an overall framework in which these techniques compatibly reinforce each other.

Informationsteknologi Institutionen för informationsteknologi | VBSE Key Elements Key components: -Benefits Realization Analysis -Stakeholder Value Proposition Elicitation and Reconciliation -Business Case Analysis -Continuous Risk and Opportunity Management -Concurrent Software and System Engineering -Value-Based Monitoring and Control -Change as Opportunity

Informationsteknologi Institutionen för informationsteknologi | Problem/Question/Goal -Validating VBSE compared to SE by demonstrating early value in software system projects -Can early value be demonstrated in software system projects, if so, how? -Demonstrate value  Using data collection and projects developed in academic environment for educational and commercial users

Informationsteknologi Institutionen för informationsteknologi | COCOMO II Model -Helps planning and reasoning about the cost and schedule implications of a software project Software investment decisions Project budgets and schedules Cost/schedule/performance tradeoffs Software improvement decisions –Reuse, tools, process maturity, outsourcing

Informationsteknologi Institutionen för informationsteknologi | COSYSMO Model -COnstructive SYStems engineering cost MOdel -COSYSMO helps plan the systems engineering costs of a project

Informationsteknologi Institutionen för informationsteknologi | Balanced Scorecard Method -How do customers see us? (Customer perspective) -What must we excel at? (Internal perspective) -Can we continue to improve and create value? (Innovation and learning perspective) -How do we look to shareholders? (Financial perspective)

Informationsteknologi Institutionen för informationsteknologi | Algorithm for Applying VBSE 1.Elaborate the Results Chain based on Benefit Realization Approach 2.Identify success-critical stakeholders, their roles and value propositions 3.Analyze the Model Clash Spider Web based on the previous two steps 4. Verify that the project uses the prototyping techniques mentioned and/or a relevant process model 5. Identify milestones according to the process model used ( like: iterative process model with: i) major and minor milestones; ii) objectives to be tested against at each milestone; iii) risk-driven). 6. Estimate costs using COCOMO II and/or COSYSMO models for software and/or systems 7. Make market projections for the next time horizon (ASA, APR) (years) 8. Calculate Expected Benefits (EB) 9. Calculate Cumulative Expected Benefit (CEB) and Return On Investment (ROI) 10. Compare expected and actual values using the measures presented above in steps Identify the critical risks and try to figure out how much risk is enough by applying for example sweet points, utility functions and people factors 12. Use the Results Chain and the milestones in order to monitor the project progress and see if the project is progressing according to plan. Goal-Question-Metrics (GQM) or the Balanced Scorecard Method (BSM) can also be used. Apply corrective actions in case value is not realized 13. Apply architecture-based and refactoring techniques can be used in order to validate Change as Opportunity

Informationsteknologi Institutionen för informationsteknologi | Experience Database -In the database all the information gathered from the projects in order to perform the analysis is stored -Online access via website

Informationsteknologi Institutionen för informationsteknologi | Database Model 1 x 1 1 x 1 x x x x Project ID Name Location Description Role ID Name Deliverables Id, name, filename, project_id, deliverabletype _id, keyword Stakeholder ID Name Deliverable_ty pe ID Name Project metrics ID Name Project_id

Informationsteknologi Institutionen för informationsteknologi | Case Study: Web-Based Time-Reporting System Developed for a company that provides other companies with consultants Online, anytime, from anywhere Keeps track of consultants working with several customers at the same time Security issues- no one could break it from outside

Informationsteknologi Institutionen för informationsteknologi | Contribution OUTCOME INITIATIVE Implement an Online System for Consultants’ Time Reporting Reduces time to work Saves time for all system users OUTCOME Contribution Increased system efficiency Saves operational costs ASSUMPTION Implementing a new online IT System is an important efficiency criterion Step 1 – Results Chain based on BRA

Informationsteknologi Institutionen för informationsteknologi | Step 2 – Stakeholders Step 4: Process Model Stakeholders: -Acquirers (Users) - R2M -Mentors -Project Group -KTH Process Model: -The Rational Unified Process (RUP)  Iterative  Major and minor milestones  Formal objectives to be tested against  Risk-driven  Prototyping technique: HTML prototype of the system

Informationsteknologi Institutionen för informationsteknologi | Step 3 – Model Clash Spider Web PP/S PD/S PP/PD PC/PC PP/S PD/PD PD/S PP/PD PC/PC PD/PD Limited development budget, schedule Development visibility and control RUP Process Acquirers/Use rsrs Mentors and KTH Project group Applications compatibility Changeable requirements Many features Early availability Project cost/effectiveness Stable requirements Ease of meeting budget and schedule Freedom of choice COTS/reuse Freedom of choice process PC: Process PD: Product PP: Property S: Success

Informationsteknologi Institutionen för informationsteknologi | Step 5 – Major Milestones MilestoneDue dateMilestone details Inception readiness Oct 1, Year1 - Life Cycle Objective (LCO)Oct 26, Year1 Win- Win Negotiation, Vision, Glossary, Software and System Requirement Specification, Use Case Specification, Supplementary Specification Life Cycle Architecture (LCA)Dec 21, Year1 System Architecture, Software Architecture Document, Rose Analysis and Design Models, Data Base Model, User Interface Prototype (HTML), Architectural Prototype Initial operational capability (IOC): hardware Dec 21, Year1 System up and running Core capability drivethrough (CCD) Feb 1, Year2 Major Use Cases up and running IOC: software Feb 25, Year2 All preliminary Use Cases up and running Developed IOC March 1, Year2 - Responsive IOC March 5, Year2 -

Informationsteknologi Institutionen för informationsteknologi | Step 6 – Budget, COCOMO, COSYSMO -Overall estimated development budget: 22 PM (11 persons: 7 programmers, 4 networking engineers; 2 full-time months) -COCOMO II results for the project - COSYSMO results System Engineering Person-Months Estimated: 26 System Engineering Hours Estimated:3951 Total Normalized SE Size (expressed as EREQ): 553

Informationsteknologi Institutionen för informationsteknologi | Step 7 – Market Projections Yea r ASI j (Annual market SIze) MSEK ASH j (Annual market SHare) % ASA j (Annual market SAles) MSEK APR j Annual market PRofits MSEK

Informationsteknologi Institutionen för informationsteknologi | Step 8 – Expected Benefits Year APR j,new (Annual market PRofits) MSEK APR j,old (Annual market PRofits) MSEK CPR j (Change in PRofits) MSEK OCS j (Operational Cost Savings) MSEK EB j (Expected Benefits) MSEK

Informationsteknologi Institutionen för informationsteknologi | Step 9 – Cumulative Expected Benefits and ROI Year EB j (Expected Benefits) MSEK CEB j (Cumulative Expected Benefits) MSEK CC j (Cumulative Cost) MSEK ROI j (Return On Investment) % , , CEB k = MSEK ROI k = 318.2%

Informationsteknologi Institutionen för informationsteknologi | Return on Investment

Informationsteknologi Institutionen för informationsteknologi | Step 10 – Expected and Actual Values -Expected and Actual Values are compared using the measures presented in steps 6-9 -System up and running -Analyses performed show good ROI, COSYSMO and COCOMO results -Future expansion has a lot of possibilities if they were exploited

Informationsteknologi Institutionen för informationsteknologi | Step 11 – Risks/Opportunities at Milestones Major risks: -Team member withdrawal -Inadequate resources, personnel -Personnel competence -Time management during exams

Informationsteknologi Institutionen för informationsteknologi | Risk Examples

Informationsteknologi Institutionen för informationsteknologi | Steps 12 and 13 – Project progress and corrective actions To monitor project progress, the deadlines for milestones, their pass-fail criteria, and risk management are reviewed MilestoneDue dateRisks Eliminated Inception readiness Oct 1, Year1 Life Cycle Objective (LCO) Oct 26, Year1 Life Cycle Architecture (LCA) Dec 21, Year1 R1,R2, R6, R7, R9 Initial operational capability (IOC): hardware Dec 21, Year1 R11 Core capability drivethrough (CCD) Feb 1, Year2 R8 IOC: software Feb 25, Year2 R10 Developed IOC March 1, Year2 Responsive IOCMarch 5, Year2 R3, R4, R5, R12, R13

Informationsteknologi Institutionen för informationsteknologi | Step 14 – Architecture Is only applicable if there is a high-level architectural design like patterns and views

Informationsteknologi Institutionen för informationsteknologi | Discussion -Algorithm and method for value-based software and system monitoring and control applied for a large number of commercial and educational applications developed in an academic environment -Projects data were collected in the Experience Database -Value-based questionnaire was designed and the results were analyzed -Questionnaire results not included, but were used for validation purposes -Projects developed for commercial applications like the presented case study showed larger ROI than those for educational applications -Applying the above approach helped demonstrating early value in software and system projects motivating the need for further research in the area.

Informationsteknologi Institutionen för informationsteknologi | Conclusion Our approach for demonstrating early value in software and system projects is based on: -Survey questionnaire -Real data collection via market research -Algorithm for applying VBSE  Software and system cost estimation models  Other models / metrics -Case studies -Experience database -Website

Informationsteknologi Institutionen för informationsteknologi | Future work -Monitoring value creation in modern software processes -Applying patterns and views -Extended metrics set -Strategic IT decision-making -Models and model integration -Use project follow-up -Further case studies -Education and communication of scientific and practical results

Informationsteknologi Institutionen för informationsteknologi | References Biffl, S., Aurum A., Boehm, B., Erdogmus, H., Gruenbacher P.(Eds.), “Value-Based Software Engineering”, Springer, 2006 Boehm, B.,“Value-Based Software Engineering”, ACM SIGSOFT Software Engineering Notes vol 28 no 2, Boehm, B., and Huang, L., “Value-Based Software Engineering: A Case Study”, IEEE Computer, March, Jacobson, I., Booch, G., and Rumbaugh, J., “The Unified Software Development Process”, Addison Wesley, “RUP 2000: Rational Unified Process”, Rational Software, Stoica, A.J., “IT Project Management, Methods and Tools”. Lecture Notes, KTH, Stockholm, Sweden, Stoica, A.J., “An Internet-Intranet Solution for Software System Estimation with Use Cases”, Proceedings of the 18th International Forum on COCOMO and Software Cost Modeling, Stoica, A.J., “Software Engineering and Security Architecture”. Lecture Notes, KTH-DSV, Stockholm, Sweden, Stoica, A.J., “Aspects of Value-Based and Strategic Software Engineering Research”. Invited technical presentation, SINTEF Research Institute, Oslo, Norway, Sept Stoica, A.J., “Value-Based Software Engineering”. Lecture Notes, KTH-DSV Stockholm, Sweden, 2007.