1. Agile Philly: Estimating Vs Forecasting Using a Monte Carlo Tool
March 21, 2017
Charlie Villare
Joe Berner

2. Agenda
Who we are / how we work Estimating Traditional Estimating Early Agile Estimating Forecasting Concepts Tools

3. Who we are / How we work Cerner (formerly Siemens)
2005 – started Agile using Scrum Implementation
2012 – Started to migrate to Kanban Implementation
Malvern – Approx. 50 distributed teams - 5 to 25 / team
Malvern teams mostly Kanban
KC – some teams use Scrum and some use Kanban

4. Tonight’s Focus Forecasting as opposed to Estimating
Scientific predictions rather than subjective guessing
Tool: KanbanSim and ScrumSim from FocusedObjectives
Forecasting - no preference as to implementation method
Traditional estimation - same faults regardless of implementation method

5. Terms we’ll use below Project = Feature = Deliverable
A single thing that you could market to your users (not down to the user story level)
Program = Release
Typically a set of the above that we market and deliver to clients

6. Traditional Estimates
Waterfall Estimating
Task
Estimate
# hours of analysts
160
# hours detail design
100
# hours coding
250
# hours testing
125
Total hours
635
= # days, weeks, months (5 people/30hr week)
5 weeks
How much time to integrated testing – multiple iterations
3 weeks
How much time to implement into a beta
6 weeks
How much time to implement into Prod
2 weeks
Finish “date”
16 weeks
Typically lots of spreadsheets where you might list out the things that will have to change and then multiply some # hours by the # of things

7. Another look at traditional estimating
Work Effort
16 weeks
Actual Time
24 weeks
160
Analysis 2w
Designer Unavail
100
Detail Design 30
Scope rework
260
Coding 60
Extra coding
125
Testing 60
Bugs & testing 3w
Intg testing (IT)
In IT what did we do?
Approaches: Try to list all the things that will need to be changed or created. Guess how long each might take. Add it up and then add a “fudge” factor.
Red (SH) – Stuff Happens 1w
Fixes from IT 6w
Beta Testing 2w
Transfer to Prod

8. Early Agile Estimating
Scrum
Story Points – attempts to size stories relative to each other.
Equates a story point with a unit of time or how many points can the team do in a week/month
Estimates when stories will be done
Add them together to estimate the project/feature
SH = stuff happens
Still did not account for SH – stuff happens
How accurate are story points?

10. Forecasting Concepts
Looks at
Entire System

11. --- SH --- Forecasting Concepts Monte Carlo Technique
Simulates various possibilities for completing work and calculates “likelihoods” to complete - in %
Using Team’s Historical Data (Cycle Times)
Cycle Times account for:
Touch time (actual work time)
Queued times
Impediments (blocks)
Resource capacity
Variations Throughout Project
--- SH ---

12. Why do we Forecast (or Estimate)?
Someone requested it.
What do they really want to know?
The number of hours, days, weeks?
The cost in dollars?
Yes, but typically the most important things are
When will it be done and how confident are you with that date?

13. One Forecasting Tool
Focused Objectives – Troy
Free and licensed stuff.
Poor man’s spreadsheets – lots of capabilities
Monte Carlo tool - KanbanSim and ScrumSim
Let’s see how it works ...

14. Define the Roles and Set a WIP Ratio
Set a start date
How many of each role and how many items are you willing to let them work on at one time? Less is better.

15. Calculate WIP and Set Up Cost
Calculate WIP based upon resources and ratios
Set the cost and the work days

16. Repeat for Coding and Testing
Historical Data Needed
Separate cycle times: analysis, coding, testing
Repeat for Coding and Testing
You don’t have breakdown?
Just use total cycle time and apply a % for each
* Going forward, start tracking separate cycle times

17. Additional Scope Allows you to add a percentage of estimated scope
Each run will use a percentage data point in that range

18. Feature Entry Count = Story Count
Name = Deliverable (What shows on the board)
Define Comment if desirable

19. Single Deliverable Forecast w/Cost

20. Defining Board Columns

21. Skip Percentage Account for stories that don’t play in every column…
For example,
Spike stories – maybe only played in Analysis or Coding?
A testing only story

22. Forecasting Multiple Deliverables
Why would you forecast multiple projects together?
Order = Define priority (Random or Sequential)
Count = Story Count
Name = Deliverable (What shows on the board)
Define Comment if desirable

23. Forecasting for Capacity Fluctuations
Vacation
Ramp up
Resource reallocation

24. General Forecast
Uses historical data in numerous cycles to forecast date
Lists a range of likelihoods (85% is highlighted in Yellow)

25. Permutations View forecast by individual deliverable
Helps to see full picture – potential re-prioritization

26. Resource Capacity Evaluates where there are short comings in flow
Provides recommendations where increased WIP would help

27. Explore the Tool
Let’s dive in and explore

28. Recap Point estimation Focuses on touch time
No correlation between point value and time
No SH
Forecasting
Focuses on system thinking
Uses Monte Carlo simulation to produce a forecast
Relies on teams historical data as input to the simulation

29. Additional Questions / Discussion