University of Southern California Center for Systems and Software Engineering Reducing Estimation Uncertainty with Continuous Assessment: Tracking the.

Slides:



Advertisements
Similar presentations
Cost as a Business Driver 1 John Brown C Eng MIEE mr_ Software Cost Estimation.
Advertisements

Early Effort Estimation of Business Data-processing Enhancements CS 689 November 30, 2000 By Kurt Detamore.
University of Southern California Center for Systems and Software Engineering Shrinking the Cone of Uncertainty with Continuous Assessment Pongtip Aroonvatanaporn.
University of Southern California Center for Systems and Software Engineering Design-Code Review Preparation Pongtip Aroonvatanaporn CSCI577b Spring 2012.
Automated Software Cost Estimation By James Roberts EEL 6883 Spring 2007.
A Sizing Framework for DoD Software Cost Analysis Raymond Madachy, NPS Barry Boehm, Brad Clark and Don Reifer, USC Wilson Rosa, AFCAA
Copyright 2000, Stephan Kelley1 Estimating User Interface Effort Using A Formal Method By Stephan Kelley 16 November 2000.
Using UML, Patterns, and Java Object-Oriented Software Engineering Royce’s Methodology Chapter 16, Royce’ Methodology.
University of Southern California Center for Systems and Software Engineering 2012 COCOMO Forum 1 October 18, 2012 Mauricio E. Peña Ricardo Valerdi Quantifying.
COCOMO Suite Model Unification Tool Ray Madachy 23rd International Forum on COCOMO and Systems/Software Cost Modeling October 27, 2008.
Cocomo II Constructive Cost Model [Boehm] Sybren Deelstra.
University of Southern California Center for Systems and Software Engineering ©USC-CSSE1 Ray Madachy, Ricardo Valerdi USC Center for Systems and Software.
3/14/2006USC-CSE1 Ye Yang, Barry Boehm Center for Software Engineering University of Southern California COCOTS Risk Analyzer and Process Usage Annual.
USC 21 st International Forum on Systems, Software, and COCOMO Cost Modeling Nov 2006 University of Southern California Center for Software Engineering.
University of Southern California Center for Software Engineering CSE USC System Dynamics Modeling of a Spiral Hybrid Process Ray Madachy, Barry Boehm,
University of Southern California Center for Systems and Software Engineering 3/18/2008©USC-CSSE1 Requirements Elaboration Ali Afzal Malik, Barry Boehm,
Neuro-Fuzzy Algorithmic (NFA) Models and Tools for Estimation Danny Ho, Luiz F. Capretz*, Xishi Huang, Jing Ren NFA Estimation Inc., London, Ontario, Canada.
Integration of Software Cost Estimates Across COCOMO, SEER- SEM, and PRICE-S models Tom Harwick, Engineering Specialist Northrop Grumman Corporation Integrated.
University of Southern California Center for Systems and Software Engineering Integrating Systems and Software Engineering (IS&SE) with the Incremental.
Smi COCOMO II Calibration Status COCOMO Forum October 2004.
Integrated COCOMO Suite Tool for Education Ray Madachy 24th International Forum on COCOMO and Systems/Software Cost Modeling November.
University of Southern California Center for Systems and Software Engineering © 2010, USC-CSSE 1 COCOMO II Maintenance Model Upgrade Vu Nguyen, Barry Boehm.
10/25/2005USC-CSE1 Ye Yang, Barry Boehm USC-CSE COCOTS Risk Analyzer COCOMO II Forum, Oct. 25 th, 2005 Betsy Clark Software Metrics, Inc.
University of Southern California Center for Systems and Software Engineering ©USC-CSSE1 Ray Madachy, Barry Boehm USC Center for Systems and Software Engineering.
University of Southern California Center for Systems and Software Engineering 1 © USC-CSSE A Constrained Regression Technique for COCOMO Calibration Presented.
University of Southern California Center for Systems and Software Engineering Assessing the IDPD Factor: Quality Management Platform Project Thomas Tan.
University of Southern California Center for Systems and Software Engineering 10/16/2009©USC-CSSE1 Software Sizing Ali Afzal Malik & Vu Nguyen USC-CSSE.
University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 1 Assessing and Estimating Corrective, Enhancive, and Reductive.
Copyright USC-CSSE 1 Quality Management – Lessons of COQUALMO (COnstructive QUALity MOdel) A Software Defect Density Prediction Model AWBrown.
Comparison and Assessment of Cost Models for NASA Flight Projects Ray Madachy, Barry Boehm, Danni Wu {madachy, boehm, USC Center for Systems.
Technische Universität München The influence of software quality requirements on the suitability of software cost estimation methods 24th International.
COCOMO II Database Brad Clark Center for Software Engineering Annual Research Review March 11, 2002.
University of Southern California Center for Systems and Software Engineering Software Cost Estimation Metrics Manual 26 th International Forum on COCOMO.
University of Southern California Center for Systems and Software Engineering Decision Support for Value-Based Software Testing Framework Qi Li, Barry.
April 13, 2004CS WPI1 CS 562 Advanced SW Engineering General Dynamics, Needham Tuesdays, 3 – 7 pm Instructor: Diane Kramer.
University of Southern California Center for Systems and Software Engineering ©USC-CSSE1 Ray Madachy USC Center for Systems and Software Engineering
University of Southern California Center for Systems and Software Engineering © 2009, USC-CSSE 1 An Analysis of Changes in Productivity and COCOMO Cost.
University of Southern California Center for Systems and Software Engineering 1 WikiWinWin: Rapid Collaborative Requirements Negotiation Using Wiki and.
Copyright © 2001, Software Productivity Consortium NFP, Inc. SOFTWARE PRODUCTIVITY CONSORTIUM SOFTWARE PRODUCTIVITY CONSORTIUM COSYSMO Overview INCOSE.
By K Gopal Reddy.  Metrics in software are of two types.direct and indirect.  Function points as indirect metrics.  Function points are used to measure.
Slide 1 Project Management Chapter 4. Slide 2 Objectives ■ Become familiar with estimation. ■ Be able to create a project workplan. ■ Become familiar.
University of Southern California Center for Software Engineering C S E USC Using COCOMO for Software Decisions - from COCOMO II Book, Section 2.6 Barry.
COCOMO CO nstructive CO st Mo del II Copyright © 2007 Patrick McDermott UC Berkeley Extension It’s a Name Game, Don’t Blame Boehm! (rhymes)
University of Southern California Center for Systems and Software Engineering Vu Nguyen, Barry Boehm USC-CSSE ARR, May 1, 2014 COCOMO II Cost Driver Trends.
University of Southern California Center for Systems and Software Engineering COCOMO Suite Toolset Ray Madachy, NPS Winsor Brown, USC.
Ali Afzal Malik, Supannika Koolmanojwong, Barry Boehm USC-CSSE
Proposed Metrics Definition Highlights Raymond Madachy Naval Postgraduate School CSSE Annual Research Review March 8, 2010.
UNCLASSIFIED Approved for Public Release 07-MDA-2965 (26 OCT 07) Load Bearing Walls: Early Sizing Estimation In The NOSTROMO Tool (U) Dan Strickland Dynetics.
Cost Estimation Cost Estimation “The most unsuccessful three years in the education of cost estimators appears to be fifth-grade arithmetic. »Norman.
Local Calibration: How Many Data Points are Best? Presented by Barry Boehm on behalf of Vu Nguyen, Thuy Huynh University of Science Vietnam National University.
University of Southern California Center for Systems and Software Engineering 1 © USC-CSSE Integrating Case-Based, Analogy-Based, and Parameter-Based Estimation.
University of Southern California Center for Systems and Software Engineering Reducing Estimation Uncertainty with Continuous Assessment Framework Pongtip.
University of Southern California Center for Systems and Software Engineering Core Capability Drive-Through Preparation Pongtip Aroonvatanaporn CSCI 577b.
University of Southern California Center for Systems and Software Engineering 26 th Annual COCOMO Forum 1 November 2 nd, 2011 Mauricio E. Peña Dr. Ricardo.
University of Southern California Center for Systems and Software Engineering Barry Boehm, USC CS 510 Fall 2010 Software Planning Guidelines.
COCOMO Software Cost Estimating Model Lab 4 Demonstrator : Bandar Al Khalil.
1 Agile COCOMO II: A Tool for Software Cost Estimating by Analogy Cyrus Fakharzadeh Barry Boehm Gunjan Sharman SCEA 2002 Presentation University of Southern.
Developing the schedule – Tracking Gantt charts Critical path method – Longest path, earliest time Schedule trade-offs using CPM – Free slack, total slack.
Software Sizing, Estimation, and Tracking
Ivan Marsic Rutgers University
SKILL ASSESSMENT OF SOFTWARE TESTERS Case Study
Software Engineering: A Practitioner’s Approach, 6/e Chapter 23 Estimation for Software Projects copyright © 1996, 2001, 2005 R.S. Pressman & Associates,
An Empirical Study of Requirements-to-Code Elaboration Factors
SLOC and Size Reporting
COCOMO Model Basic.
Software Engineering: A Practitioner’s Approach, 6/e Chapter 23 Estimation for Software Projects copyright © 1996, 2001, 2005 R.S. Pressman & Associates,
Core Capability Drive-Through Workshop
Relating Effort Reporting to Project Estimation
Faster Cost Estimate Convergence with SLOC-Check
Chapter 26 Estimation for Software Projects.
Presentation transcript:

University of Southern California Center for Systems and Software Engineering Reducing Estimation Uncertainty with Continuous Assessment: Tracking the “Cone of Uncertainty” Pongtip Aroonvatanaporn, Chatchai Sinthop, Barry Boehm {aroonvat, sinthop, November 2, 2010

University of Southern California Center for Systems and Software Engineering Outline Introduction and Motivation Framework Model Experiment Results Conclusion and future work 11/02/20102© USC-CSSE

University of Southern California Center for Systems and Software Engineering The Cone of Uncertainty 11/02/20103© USC-CSSE Inexperienced teams Experienced teams Also applies to project estimation accuracy

University of Southern California Center for Systems and Software Engineering Definition Inexperience –Inexperienced in general –Experienced, but in a new domain –Anything that is new with little knowledge or experience 11/02/2010© USC-CSSE4

University of Southern California Center for Systems and Software Engineering The Problem Experienced teams can produce better estimates –Use “yesterday’s weather” –Past projects of comparable size –Past data of team’s productivity –Knowledge of accumulated problems and solutions Inexperienced teams do not have this luxury No tools or data that monitors project’s progression within the cone of uncertainty 11/02/20105© USC-CSSE

University of Southern California Center for Systems and Software Engineering The Problem Imprecise project scoping –Overestimate vs. underestimate Manual assessments are tedious –Complex and discouraging Project estimation not revisited –Insufficient data to perform predictions –Project’s uncertainties not adjusted Limitations in software cost estimation –Models cannot fully compensate for lack of knowledge and understanding 11/02/20106© USC-CSSE

University of Southern California Center for Systems and Software Engineering The Goal Develop a framework to address mentioned issues Help unprecedented projects track project progression Reduce the uncertainties in estimation –Achieve eventual convergence of estimate and actual Must be quick and easy to use 11/02/20107© USC-CSSE

University of Southern California Center for Systems and Software Engineering Benefits Improve project planning and management –Resources and goals Improved product quality control Actual project progress tracking –Better understanding of project status –Actual progress reports 11/02/20108© USC-CSSE

University of Southern California Center for Systems and Software Engineering Outline Introduction and Motivation Framework Model Experiment Results Conclusion and future work 11/02/20109© USC-CSSE

University of Southern California Center for Systems and Software Engineering Estimation Model Integration of the Unified Code Count tool and COCOMO II estimation model 11/02/201010© USC-CSSE Adjusted with REVL

University of Southern California Center for Systems and Software Engineering Outline Introduction and Motivation Framework Model Experiment Results Conclusion and future work 11/02/201011© USC-CSSE

University of Southern California Center for Systems and Software Engineering Experiment Setup Performed simulation on 2 projects from USC software engineering course Project similarities –Real-client –SAIV: 24-weeks –Architected agile process, 8-member team –Size, type, and complexities –Product E-services Web content management system JSP, MySQL, Tomcat 11/02/201012© USC-CSSE

University of Southern California Center for Systems and Software Engineering Obtaining Data Source code files retrieved from Subversion server Simulation of assessment done weekly Both teams were closely involved –Provide estimation of module completion –Rationale 11/02/201013© USC-CSSE

University of Southern California Center for Systems and Software Engineering Outline Introduction and Motivation Framework Model Experiment Results Conclusion and future work 11/02/201014© USC-CSSE

University of Southern California Center for Systems and Software Engineering Results Accumulated effort Initial estimate Adjusted estimate ~50 % ~18% 11/02/201015© USC-CSSE

University of Southern California Center for Systems and Software Engineering Results Project progress reaches 100% –Reflects reality Estimation errors reduced to 0% 11/02/201016© USC-CSSE

University of Southern California Center for Systems and Software Engineering Outline Introduction and Motivation Framework Model Experiment Results Conclusion and future work 11/02/201017© USC-CSSE

University of Southern California Center for Systems and Software Engineering Conclusion Both teams demonstrated the same phenomenon –Gaps in estimation errors decrease –Representation of “cone of uncertainty” Estimation framework reflects the reality of project’s progress Assessment process was quick and simple –Requires few inputs –Little analysis needed Assessment framework help inexperienced team improve project tracking and estimation 11/02/201018© USC-CSSE

University of Southern California Center for Systems and Software Engineering Future Work Tool development currently in progress Determine the frequencies of assessments required –The sweet spot Observe prediction accuracies –Experiment on projects of larger scale –Experiment on projects of different types Use concept of value-based –Apply weights to calculation of each software module based on priorities and criticalities –How to adjust COCOMO parameters 11/02/201019© USC-CSSE

University of Southern California Center for Systems and Software Engineering References 1.Boehm, B., Abts, C., Brown, A. W., Chulani, S., Clark, B. K., Horowitz, E., Madachy, R., Reifer, D. J., and Steece, B. Software Cost Estimation with COCOMO II, Prentice-Hall, Cohn, M. Agile Estimating and Planning, Prentice-Hall, DeMarco, T. Controlling Software Projects: Management, Measurement, and Estimation, Yourdon Press, Fleming, Q. W. and Koppelman, J. M. Earned Value Project Management, 2 nd edition, Project Management Institute, Jorgensen, M. and Boehm, B. “Software Development Effort Estimation: Formal Models or Expert Judgment?” IEEE Software, March-April 2009, pp Nguyen, V., Deeds-Rubin, S., Tan, T., and Boehm, B. "A SLOC Counting Standard," COCOMO II Forum Stutzke, R. D. Estimating Software-Intensive Systems, Pearson Education, Inc, /02/201020© USC-CSSE

University of Southern California Center for Systems and Software Engineering Backup Slides 11/02/2010© USC-CSSE21

University of Southern California Center for Systems and Software Engineering Related Work Software estimation methods –Estimating Software-Intensive Systems [Stutzke, 2005] –Expert-judgement vs. parametric-model [Jorgensen, 2007] –Agile estimation [Cohn, 2005] Software estimation uncertainty –PERT sizing methods [Nguyen, 2007] –Wideband Delphi estimate distributions [Boehm, 2000] Software project tracking methods –Controlling Software Projects [DeMarco, 1982] –Earned Value Management [Fleming, 2000] 11/02/201022© USC-CSSE