Joel Semeniuk CEO, Imaginet Resources Microsoft Regional Director Microsoft MVP – Team System DPR205

What Is an Estimate? Estimate <> Target or Commitment Estimate Prediction Target Desire Commitment Promise

Estimate <> Planning Estimation Analytical Unbiased Process Goal = Accuracy Planning Biased Goal-Seeking Process Goal = Seek Result

Plans Need Good Estimates Estimation Scheduling Identifying Critical Path Breaking Down Work Prioritization Defining Iterations

Common Scenario How long do you think this project should take? We need to have it ready in 3 months with these features…. My estimates say we can do this in 5 months 5 Months? Didn’t you hear me? I need to have this in 3 months for the trade show! The manager wasn’t asking for an estimate but a plan to solve a problem… Tip: When asked for an estimate – determine if it should be an estimate or a plan to hit a target

Single Point Estimates Rarely is a single point estimate an estimate Estimates should include some level of probability Tip: When you see a single point estimate – question to see if this is really a target Tip: When you see a single point estimate – you should question the probability Tip: Don’t use a planning tool to make estimates (e.g., MS Project)

Using MS Project and Single Point Estimates Really?

Estimation Challenges Provide a 90% Confident Estimate for the following: How many people are at this conference? What is the size of the largest whale ever discovered? How many bugs will you have in your next software project? What is the distance between Mars and the Sun? What is the surface temperature of the sun? What is the total volume of the Great Lakes? Question: What makes you think you are 90% confident? Question: Did you use an artificially narrow range? Question: Where does the pressure to use a narrow range come from?

Two Important Laws of Nature Parkinson’s Law If you give a person 5 days to deliver a task that could be delivered in 4, the person will find something to do with that extra day The Student Syndrome If given too much time, people procrastinate until late in the project, at which point they rush to complete the work, and likely won’t finish on time These are two common arguments against Overestimation

Uncertainty and Software Estimation Not all requirements are known until development time in most cases

Narrowing the Cone Tip: The cone does not narrow itself; it narrows when you continually make decisions that narrow variability Sources of Error 1.Chaotic Development Process 2.Unstable Requirements 3.Omitted Activities 4.Unfounded Optimism 5.Subjectivity 6.Off-the-Cuff Estimates Sources of Error 1.Chaotic Development Process 2.Unstable Requirements 3.Omitted Activities 4.Unfounded Optimism 5.Subjectivity 6.Off-the-Cuff Estimates

Things that Influence Estimates Project Size Type of Software Being Developed Personnel Factors Programming Languages and LOTS of others such as: Complexity Constraints Turnover Experience Tools Ego Team Cohesion Process maturity Architecture Risk Technology Maturity I’m sure you can name 10 more…

Estimation Practices Count, Compute, JudgeCalibration and Historical DataExpert JudgmentDecomposition and RecompositionEstimation by AnalogyProxy-Based EstimatesExpert Judgment in GroupsUse of Multiple Approaches

Count, Compute, Judge

If you can count something – count it Use “expert” judgment when there’s nothing to count What to count? Something correlated with size of software Something we can count sooner rather than later Something that will produce a statistically meaningful average Something that you understand Something you can count with minimal effort

Count, Compute, Judge Examples of things you can count… List of thingsMore things RequirementsWeb Pages FeaturesBugs Use casesReports StoriesDialog boxes Function PointsDatabase Tables Change RequestsClasses TestsCucumbers

Calibration and Historical Data Helps to transform Count into Estimate Best way to predict today’s weather is to look at yesterday’s Ways to calibrate: Industry Data Historical Data Project Data

Industry Data

Historic Project Data Example Team SizeAverage Stories Delivered per Sprint 15 2–312 4–522 6–731 8No Data Tip: Using Time and “Actual Effort” as historical data is difficult

Current Project Data Use Data from the Current Project to Predict the Rest of the Project Data for Sprint 1 27 story points (sp) delivered 12 staff weeks (sw) 3 calendar weeks (cw) Preliminary Calibration Effort = 27sp / 12sw = 2.25 sp per sw Schedule = 27sp / 3w = 9 sp per cw Remainder of Project Effort = 2.25 sp per staff week (spsw) Schedule = 9 sp per calendar week (spcw) Project size = 180 sp Preliminary Whole-Project Estimate Effort = 180sp/2.25 spsw = 80 staff weeks Schedule = 180sp / 9 spcw = 20 calendar weeks

Individual Expert Judgment Two Types Intuitive Expert Judgment (Spidey Senses) – not very accurate Structured Expert Judgment – almost as accurate as model-based estimates Who Estimates? People who are doing the work – most accurate Granularity End up with estimates that are around ¼–1 day large Use Ranges Best Case, Most Likely Case, Worst Case Use Formulas Optimistic PERT = [BestCase+(4XMostLikely)+WorstCase]/6 Pessimistic PERT = [BestCase+(3xMostLikely)+(2xWorstCase)]/6

Decomposition and Recomposition AKA – Bottom Up The Law of Large Numbers Single Large Estimates are usually Largely wrong in a single direction (+/-) Breaking into smaller pieces – some are above, some are lower The error at the detail level balances out the error on the whole How Small to Break Things Down? You need 5–10 smaller things to start seeing benefits Use Common Work Breakdown to Avoid Missing Activities CategoryCreate/DoPlanManageReviewReworkDefect IntegrationXXXX DocumentationXXXXX EtcXXX

Big Problem with Bottom UP Aggregating Estimates Doesn’t Work Well Use complex standard deviation formulas to aggregate when you have more than 10 tasks This can get VERY complicated if you want to be accurate Take a statistics course for more information

Estimation by Analogy Check for congruent assumptions Create an effort based on the size of the new project compared to old Build up estimate of new project as a % of the old project size Compare size of new project piece by piece Get details of previous project decomposed by feature area, work breakdown or some other scheme

Proxy-based Estimates Find a Proxy that correlates to what you are estimating that is easier to estimate/count E.g., Estimation of # of Test Cases is correlated to # of Requirements Some use Proxies to estimate lines of code What do lines of code really give you? Some great techniques Story Points T-Shirt Sizing Good for estimating test cases, defects, documentation, etc.

Story Points Assign a size to each story you are building Story Point ScaleSpecific Points to the Scale Powers of 21, 2, 4, 8, 16, 32 Fibonacci1, 2, 3, 5, 8, 13, 21

Story Points Example StoryPoints Story 12 Story 21 Story 34 Story 48 … Story 602 Total180

Need to Calibrate an Iteration Data for Iteration 1 27 Story Points Delivered 12 Staff weeks 3 Calendar Weeks Calibration: Effort = 27 story points / 12 staff weeks = 2.25 story points/staff week Schedule = 27 story points / 3 calendar weeks = 9 story points/calendar week Projecting Further Assumptions from Calibration Effort = 2.25 story points/staff week Schedule = 9 story points/calendar week Project size = 180 story points Preliminary Whole-Project Estimate Effort = 180 story points / 2.25 story points/staff week = 80 staff weeks Schedule = 180 story points / 9 story points/calendar week = 20 calendar weeks

T-Shirt Sizing How do Sales people and Delivery people work together? Sales people need to know relative size to help determine priority Estimator won’t estimate until further decomposed Early in the Cone of Uncertainty Remember – goal isn’t accuracy, but accurate enough to support control and decisions Can use T-shirt sizing to help

T-Shirt Sizing FeatureBusiness ValueDev Cost Feature ALargeSmall Feature BSmallLarge Feature CLarge Feature DMedium

Using for Prioritization Business ValueExtra LargeLargeMediumSmall Extra Large0467 Large-4023 Medium Small-7-3- FeatureBus. ValueDev CostNet Value Feature ALS3 Feature BLM2 Feature CLL0 Feature DMM0

Expert Judgment in Groups Group reviews can be fun Have each team estimate pieces individually, compare results Don’t average estimates Arrive at consensus Techniques Wideband Delphi Technique (quite formal) Estimation Poker (quite fun)

Mixing Things Up E.g., Expert Proxy Judgment in Groups

Quick Demos Feature Prioritization Worksheet – T-Shirt Sizing (kinda) Feature Point Worksheet Multi-Point Worksheet Estimation Poker

Some Parting Thoughts Keep Estimation Simple Time Boxing is your friend Do things that minimize uncertainty Don’t be afraid to re-estimate and thus, re-plan Estimation via consensus is harder than you think Sometimes you have to commit; I’m sorry Be careful when recording historic data – time-based data is all a lie and lines of code are meaningless when technology changes Find a method that works for you – then continually improve

Reference

Sessions On-Demand & Community Resources for IT Professionals Resources for Developers Microsoft Certification and Training Resources Microsoft Certification & Training Resources Resources

Complete an evaluation on CommNet and enter to win! Required Slide

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. Required Slide