1. Click to edit Master title style Mile High Agile 2013 April 2013 Frank Vega frank.vega@vissinc.com www.vissinc.com VISS Kanban Metrics: Where to Start? Todd Sheridan tsheridan@rallydev.comwww.rallydev.com

2. Click to edit Master title style Introductions April 2013Vega Information System Services, Inc. © Do Not Reproduce 2 Lean-agile Principles, Kanban Method, XP Practices Frank Vega Software Development Process Coach

3. Click to edit Master title style Introductions April 2013Vega Information System Services, Inc. © Do Not Reproduce 3 Todd Sheridan Agile Coach Agile Coachtsheridan@rallydev.com @agile_samurai @agile_samurai

4. Click to edit Master title style Introductions Vega Information System Services, Inc. © Do Not Reproduce 4 New to lean? New to kanban? Who are you? April 2013

5. Click to edit Master title style Assumptions :: Agile Experience Vega Information System Services, Inc. © Do Not Reproduce 5 April 2013

6. Click to edit Master title style Assumptions :: Value of Teams Vega Information System Services, Inc. © Do Not Reproduce 6 April 2013

7. Click to edit Master title style Vega Information System Services, Inc. © Do Not Reproduce 7 April 2013 Assumptions :: Value of Teams

8. Click to edit Master title style Assumptions :: Relative Sizing Vega Information System Services, Inc. © Do Not Reproduce 8 April 2013

9. Click to edit Master title style Metrics :: Good & Bad Vega Information System Services, Inc. © Do Not Reproduce 9 Performance based = good Performance based = good Ex. Cycle Time, Throughput, Quality Ex. Cycle Time, Throughput, Quality Passively collected Passively collected Compliance based = bad Compliance based = bad Setting a target leads to gaming the system Setting a target leads to gaming the system Individual metrics kill team work Individual metrics kill team work Ex. LoC (Lines of Code), Personal Burndowns Ex. LoC (Lines of Code), Personal Burndowns April 2013

10. Click to edit Master title style 1. What’s a “normal” size story for us? 2. How do we find problem areas? 3. Is this a really “different” story (outlier)? 4. How do we predict when we will deliver? 5. Do we really need to limit work-in-progress? 6. How do CFDs really help? 7. How do we set WIP limits? 1. What’s a “normal” size story for us? 2. How do we find problem areas? 3. Is this a really “different” story (outlier)? Kanban Metrics – Where to Start? Vega Information System Services, Inc. © Do Not Reproduce 10 A Few Kanban Metrics Questions We Hear… April 2013

11. Click to edit Master title style Data Analysis Overview Vega Information System Services, Inc. © Do Not Reproduce 11 The BIG Picture Averages Scatter Plots Percentiles Distributions Tables T-Shirt Sizes Run (Time Series) Charts April 2013

12. Click to edit Master title style Basic Terms April 2013Vega Information System Services, Inc. © Do Not Reproduce 12 The time it takes a workitem to move from “end-to-end” in this workflow that you influence is often of interest to you! Typically this will represent what you’ll refer to as the “lead time” for your workflow. Lead Time

13. Click to edit Master title style Basic Terms April 2013Vega Information System Services, Inc. © Do Not Reproduce 13 Sometimes the time it takes a workitem to move through a single workflow state or consecutive states will also be of interest to you! Typically this will represent what you’ll refer to as the “cycle time” for this state (or states) of your workflow. Cycle Time

14. Click to edit Master title style Cycle Time Basic Terms April 2013Vega Information System Services, Inc. © Do Not Reproduce 14 Depending on context you’ll often hear and see one used for the other. CONTEXT, CONTEXT, CONTEXT! CONTEXT, CONTEXT, CONTEXT!

15. Click to edit Master title style Cycle Time Basic Terms April 2013Vega Information System Services, Inc. © Do Not Reproduce 15 Your “lead time” is simply someone’s “cycle time” depending on the frame of reference. CONTEXT, CONTEXT, CONTEXT! CONTEXT, CONTEXT, CONTEXT! Lead Time

16. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 16 Average (Mean) Lead Time April 2013

17. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 17 Average (Mean) Lead Time Average (Mean) Lead Time = 7 Days Number of Stories = 591 Is the Mean a good SLA? Why? April 2013

18. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 18 Average (Mean) Lead Time Mean (Average) Lead Time = 7 Days Minimum Lead Time = 1 Day Maximum Lead Time = ? 14 Days? Number of Stories = 591 Is the Mean a good SLA? Why? 28 Days? 112 Days! April 2013

19. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 19 Average (Mean) Lead Time Mean (Average) Lead Time = 7 Days Minimum Lead Time = 1 Day Maximum Lead Time = ? 14 Days? Number of Stories = 591 What is a good SLA with this information? Why? 28 Days? 112 Days! April 2013

20. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 20 Visualize to Access Data More Fully BasicStatistics “Never interpret a mean value without knowing the underlying distribution (or the sample size).” April 2013

21. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 21 Perspective - Time How might adding a time perspective help? What additional insights might we see? April 2013

22. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 22 Scatter Plot – A Temporal Perspective April 2013 Red = Functional Stories Blue = Infrastructure Stories # of Days (lead time) 10 Day Increments Time Intervals 7 Day Increments

23. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 23 Scatter Plot – Recognizing Issues Visually Anything Interesting Here? April 2013

24. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 24 Scatter Plot – Adding Common Percentiles 50 th <= 4 days 85 th <= 13 days 95 th <=20 days April 2013 How Does These Help?

25. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 25 Scatter Plot – Adding Common Percentiles 50 th <= 4 days 85 th <= 13 days 95 th <=20 days April 2013 Outliers? Special Cause? External Variability?

26. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 26 Scatter Plot – Adding Common Percentiles 50 th <= 4 days 85 th <= 13 days 95 th <=20 days April 2013 Is this a good thing?

27. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 27 Another Perspective – Shape How might adding a data shape perspective help? What additional insights might we see? April 2013

28. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 28 Data Distribution Table April 2013

29. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 29 (Psuedo) Histogram & Pareto Chart April 2013

30. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 30 Scatter Plot – Adding T-shirt Percentiles 27% <= 1 day 54% <= 4 days 73% <= 7 days 90% <= 14 days 97% <= 28 days April 2013

31. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 31 Another Perspective – Trends How do trends or times series help? What additional insights might we see? April 2013

32. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 32 Run Chart (Time Series) April 2013

33. Click to edit Master title style How Long Do Stories Take? Vega Information System Services, Inc. © Do Not Reproduce 33 Run Chart (Time Series) April 2013

34. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 34 Little’s Law avg # customers = avg arrival rate x avg time in system John D. Little April 2013 L = λ W L = Queue Length λ = Arrival Rate W = Wait Time

35. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 35 A “Little” Renaming April 2013 L = λ W L = Queue Length W = Wait Time λ = Arrival Rate

36. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 36 April 2013 L = λ W L = Queue Length -> Work In Progress (WIP) W = Wait Time λ = Arrival Rate A “Little” Renaming

37. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 37 April 2013 WIP = λ W L = Queue Length -> Work In Progress (WIP) W = Wait Time λ = Arrival Rate A “Little” Renaming

38. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 38 April 2013 WIP = λ W L = Queue Length -> Work In Progress (WIP) W = Wait Time λ = Arrival Rate -> Departure Rate (Throughput) A “Little” Renaming

39. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 39 April 2013 WIP = Throughput x W L = Queue Length -> Work In Progress (WIP) W = Wait Time λ = Arrival Rate -> Departure Rate (Throughput) A “Little” Renaming

40. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 40 April 2013 WIP = Throughput x W L = Queue Length -> Work In Progress (WIP) W = Wait Time -> Lead Time λ = Arrival Rate -> Departure Rate (Throughput) A “Little” Renaming

41. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 41 April 2013 WIP = Throughput x Lead Time L = Queue Length -> Work In Progress (WIP) W = Wait Time -> Lead Time λ = Arrival Rate -> Departure Rate (Throughput) A “Little” Renaming

42. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 42 April 2013 WIP = Throughput x Lead Time A “Little” Algebra Remember algebra? Divide both sides by throughput…

43. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 43 April 2013 A “Little” Algebra WIP Lead Time = Throughput You get this! Now “flip” it around…

44. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 44 April 2013 A “Little” Algebra WIP Lead Time = Throughput Something more familiar in our context!

45. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 45 April 2013 Average WIP Average Lead Time Average Lead Time = Average Throughput A “Little” Queuing Theory average lead time: average lead time: average WIP: average throughput: average throughput: average time it takes a work item to move thru the system from start to end average # of work items in the system at a point in time average # of work items completely thru the system per unit of time

46. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 46 Key Assumptions for Little’s Law Same units Long running averages Stable System April 2013

47. Click to edit Master title style Little’s Law Basics Vega Information System Services, Inc. © Do Not Reproduce 47 Key Assumptions for Little’s Law Revisited April 2013 Over the time period of interest. How do you achieve the above? Policies? All three numbers measured in consistent units AAR equals ACR (or ADR) All items depart (complete), none lost or never exit WIP size (queue length) roughly same at start & end WIP average age (flow or lead time) is not changing

48. Click to edit Master title style Closing Thoughts Vega Information System Services, Inc. © Do Not Reproduce 48 April 2013 “Errors using inadequate data are much less than those using no data at all.” Charles Babbage, English mathematician, philosopher, inventor, mechanical engineer, invented the first mechanical computer (1791 – 1871) Good Data Enables Good Decisions

49. Click to edit Master title style Closing Thoughts Vega Information System Services, Inc. © Do Not Reproduce 49 April 2013 “The possession of tools, techniques, and technology is not the competitive advantage…it’s the learning you wrap around them before anyone else.” “The possession of tools, techniques, and technology is not the competitive advantage…it’s the learning you wrap around them before anyone else.” – Steven J. Spear, Senior Lecturer – MIT Sloan School of Management & Engineering Systems Division – May, LSSC 2012, Boston Learn and Use the Basic Metric Tools Well, First

50. Click to edit Master title style Time for Questions August 2012Vega Information System Services, Inc. © Do Not Reproduce 50