“Budgeting of IT-projects: Standards and best practices for cost and schedule planning.” Galorath Incorporated Daniel D. Galorath blog: Copyright Galorath Incorporated 2013

Cutter Consortium Software Project Survey: 62% overran original schedule by more than 50% 64% more than 50% over budget 70% had critical product quality defects after release Standish Group CHAOS Report 46% challenged 19% failed 35% successful ~$875 billion spent on IT ~$300 billion spent on IT projects ~$57 billion wasted annually ROI of better planning huge IT Failures Are Pervasive: And Even Successful Projects May Not Be Some suggest 80% of systems NEVER produce a positive ROI

An Estimate Defined An estimate is the most knowledgeable statement you can make at a particular point in time regarding: Effort / Cost Schedule Staffing Risk Reliability Estimates more precise with progress A WELL FORMED ESTIMATE IS A DISTRIBUTION © 2013 Copyright Galorath Incorporated 3

Human Nature: Humans Are Optimists HBR Article explains this Phenomenon: Humans seem hardwired to be optimists We routinely exaggerate benefits and discount costs Delusions of Success: How Optimism Undermines Executives' Decisions (Source: HBR Articles | Dan Lovallo, Daniel Kahneman | Jul 01, 2003)Dan LovalloDaniel Kahneman 4 Solution - Temper with “outside view”: Past Measurement Results, traditional forecasting, and statistical parametrics can help Don’t remove optimism, but balance optimism and realism

Estimation Methods 1 of 2 MethodDescriptionAdvantagesLimitations GuessingOff the cuff estimates Quick Can obtain any answer desired No Basis or substantiation No Process Usually Wrong Analogy Compare project with past similar projects. Estimates are based on actual experience. Truly similar projects must exist. Expert Judgment Consult with one or more experts. Little or no historical data is needed; good for new or unique projects. Experts may be biased; may not be consistent. Top Down Estimation Hierarchical decomposition into progressively smaller components to size software components Provides an estimate linked to requirements and allows common libraries to size lower level components. Need valid requirements. Difficult to track architecture; engineering bias may lead to underestimation. Bottom Up Use a WBS and identify every task, then sum these Forces thinking about the work Can be optimistic with items missing © 2013 Copyright Galorath Incorporated 5

Estimation Methods 2 of 2 Model CategoryDescriptionAdvantagesLimitations Design To Cost Uses expert judgment to determine how much functionality can be provided for given budget. Easy to get under stakeholder number Little or no engineering basis. Simple CER’s Equation with one or more unknowns that provides cost / schedule estimate Some basis in data Simple relationships may not tell the whole story Historical data may not tell the whole story Comprehensive Parametric Models Perform overall estimate using design parameters and mathematical algorithms. Models are usually fast and easy to use, and useful early in a program; they are also objective and repeatable. Models can be inaccurate if not properly calibrated and validated; historical data may not be relevant to new programs; optimism in parameters may lead to underestimation. © 2013 Copyright Galorath Incorporated 6

Reasons To For Viable / Repeatable Estimates (Adapted from CEBok ) To support good decisions To schedule work To determine how long the project should take and its cost To determine whether the project is worth doing To develop cash flow needs To determine how well the project is progressing To develop time-phased budgets and establish the project baseline © 2013 Copyright Galorath Incorporated 7

Best Practice: Estimation Process: Consistent Processes = Reliable Estimates 1.Establish Estimate Scope 2.Establish Technical Baseline, Ground Rules, Assumptions 3.Collect Data 4.Estimate and Validate Software Size 5.Prepare Baseline Estimates 7.Quantify Risks and Risk Analysis 6.Review, Verify and Validate Estimate 8.Generate a Project Plan 9.Document Estimate and Lessons Learned 10.Track Project Throughout Development

Estimate When Project Changes Or Information Is Available Traditional Estimate Phases During Feasibility At Concept After Requirements After Design After Drops if Incremental Agile Estimate Phases At Requirements (Use Cases, User Stories, etc.) Before Each Release The Development Method Is Part Of The Solution Not The Problem Estimates typically become more accurate and less uncertain as the project progresses… © 2013 Copyright Galorath Incorporated 9

Example: Project Cost Alone Is not The Cost of IT Failure (Source: HBR) Case Study: Levi Strauss $5m ERP deployment contracted Risks seemed small Difficulty interfacing with customers systems Had to shut down production Unable to fill orders for 3 weeks $192.5M charge against earnings on a $5m IT project failure © 2011 Copyright Galorath Incorporated 10 “IT projects touch so many aspects of organization they pose a new singular risk”

© 2013 Galorath Incorporated Manual Estimates: Human Reasons For Error (Metrics Can Help) Manual Task estimates yield SIGNIFICANT error Desire for “credibility” motivates overestimate behavior (80% probability?) –So must spend all the time to be “reliable” –Better approach: force 50% probability & have “buffer” for overruns Technical pride sometimes causes underestimates © 2013 Copyright Galorath Incorporated 11

Best Practice: Estimate Entire Total Ownership Cost Software Development (SEER-SEM) Software Maintenance (SEER-SEM) IT Infrastructure (SEER-IT) IT Services (SEER-IT) © 2013 Copyright Galorath Incorporated 12

© 2013 Galorath Incorporated Best Practice: Understand & Manage Project Realities For a given Size, Complexity and Technology Impossible Minimum Time To Complete (Effort Increases to Reduce Schedule ) Work Expands To Fill Time (Effort Increases due to lack of pressure ) Inefficient Minimum TimeOptimal Effort (Lower Effort for Longer Schedule) Calendar Time Effort Months Effort Increase due to Longer Schedule © 2013 Copyright Galorath Incorporated 13

© 2013 Galorath Incorporated Best Practice: Every Estimate needs to be Substantiated & Benchmarked SEER-SEM Estimate Your Data Regression Trend Line Galorath Benchmark Trend Line Your History Data Why Should We Care: Variances can identify estimation issues. Benchmarking can be path to improvement 14

© 2013 Galorath Incorporated Best Practice Probability based Estimates 15 What is likely to happen Feel lucky? Firm Fixed Price? Understand the risk before you commit!

© 2013 Galorath Incorporated Best Practice: Understand What Drives Productivity People Process Technology Effective Technology

© 2013 Galorath Incorporated Best Practice: Consider Total Cost of Ownership, Not Just Development Why Should We Care: Impacts ROI & development decisions can impact maintenance costs. 17

© 2013 Galorath Incorporated Best Practice: Multi-Dimensional Progress Tracking (Earned Value) Track defect discovery and removal rates against expected rates Increased defect reporting rate shows a worsening trend Heath and Status Indicator shows status and trends from the previous snapshot Thresholds are user definable Why Should We Care: Catching deviations from plan early allows adjustment or replanning 18

© 2013 Galorath Incorporated © 2013 Copyright Galorath Incorporated 19 Best Practice: Relative Estimates For Early Estimates Why Should We Care: Generate viable estimates even before any requirements analysis. Provide crosschecks & portfolio analysis Generate Sizing even before details are known based on relative analysis Access corporate knowledge from historical database Use domain specific analogies Configurable for Different Needs - estimate any type of variable

Best Practice: Establish A Project Repository © 2013 Copyright Galorath Incorporated 20

Conclusions Viable Estimates are critical to project success With proper estimation processes and tools estimation should SAVE FAR MORE THAN THEY COST Knowledge is power… Making the appropriate decisions can have a huge return © 2013 Copyright Galorath Incorporated 21

Backup Slides 22

Mythical Man-Month Rules (Source: MIT/ Brooks) “Cost varies as product of people and months, progress does not.” “Hence the man-month as a unit for measuring the size of job is a dangerous and deceptive myth” The myth of additional manpower (Brooks Law) “Adding manpower to a late project makes it later” Communication & Training Cost Brooks original formula: n(n-1)/2 12 month project schedule 1 person: 12 months 2 persons = 7 months (2 man-months extra) 3 persons = 5 months (3 man-months extra Models Like SEER Refined The Mathematics… The Phenomena Continues

High Quality Estimate Characteristics Accurate (within a viable range) Comprehensive (includes all costs or identifies exclusions) Replicable and Auditable (not just guessss) Traceable (to the source of the requirement) Credible (Believable) Timely (Available when needed) 24

25 Estimate Model Components and Characteristics (adapted from CEBOK) Inputs Technical and programmatic parameters Applicable rates Detail (Model structure) Varying level of detail depending on need Applies parametric, analogy, or build-up techniques to inputs Outputs Costs, & cost risk Schedule & schedule risk Methodology Outputs Inputs Data Groundrules & Assumptions

Frederick Brooks Classic Paper “No Silver Bullets” “There is no single development, in either technology or management technique, which by itself promises even one order-of magnitude improvement within a decade in productivity, in reliability, in simplicity.” -- Fred Brooks, 1986 i.e. There is no magical cure for the “software crisis” Not software measurement Not better tools Not ---- (fill in the blank… the current great hope)

Key Components Of A Software Project That Uses COTS COTS Software: Purchased functionality Direct Cost component of COTS integration COTS Cognition: Required functionality within the COTS software that must be understood “Glue” Code: Code written to bind COTS to developmental software Development effort must be captured 27

© 2013 Galorath Incorporated Process For Combining Estimation, Planning & Control, Measurement & Analysis 28 Prepare Estimate Prepare Estimate Baseline Approved Estimate Baseline Approved Estimate Collect In Progress Data Collect In Progress Data Snapshot Point in Time Progress Effort Schedule Size Growth Defect Rates Snapshot Point in Time Progress Effort Schedule Size Growth Defect Rates Effort Progress Schedule Progress Size Growth Defect Insert/Remove Progress Multi-Dimension Earned Value

© 2013 Galorath Incorporated Packaged Applications Are Costly To Organizations “Commercial application program or collection of programs developed to meet the needs of a variety of users, rather than custom designed for a specific organization” Many are enterprise applications Often allows / requires customization Examples: SAP; Rational PPM, SEER for Software; Microsoft Excel, CA Clarity, Oracle Business Suite Why should we care: Packages sometimes comprise solutions for parts of complicated systems and can be trouble "One-third [of the budget] has to go to testing. Don’t ever short change testing. Everyone always underestimates it, and says it’s the last thing to worry about. Don’t do that!“ - Jim Larson, consultant for communications solutions provider "One-third [of the budget] has to go to testing. Don’t ever short change testing. Everyone always underestimates it, and says it’s the last thing to worry about. Don’t do that!“ - Jim Larson, consultant for communications solutions provider 29

© 2013 Galorath Incorporated Understand Project Total Ownership Costs Up Front 30 Most Projects Spend Low During Maintenance