1. Agile Metrics Case Studies Let’s Play Agile Coach! Data -> Insight -> Action Jim Damato Solutions Architect PMI-ACP, SA, CSP, CSM, PMP, ITIL Mid-Atlantic Region (local to DC, MD, VA) james.damato@ca.com 3/25/2016 Rally is now CA Technologies!

2. 2 © 2016 CA. ALL RIGHTS RESERVED. Memes!

3. 3 © 2016 CA. ALL RIGHTS RESERVED.

4. 4

5. 5 How long will it take a ScrumMaster to screw in a lightbulb?

6. 6 © 2016 CA. ALL RIGHTS RESERVED. Two Points!

7. 7 © 2016 CA. ALL RIGHTS RESERVED. Me (without gray hair)

8. 8 © 2016 CA. ALL RIGHTS RESERVED. You?

9. 9 © 2016 CA. ALL RIGHTS RESERVED. Why a presentation on metrics?

10. 10 © 2016 CA. ALL RIGHTS RESERVED. Why a presentation on metrics? Too many organizations don’t measure anything

11. 11 © 2016 CA. ALL RIGHTS RESERVED. Why do we measure?

12. 12 © 2016 CA. ALL RIGHTS RESERVED. Why do we measure? If you can’t measure it, you can’t manage it

13. 13 © 2016 CA. ALL RIGHTS RESERVED. What are metrics for?

14. 14 © 2016 CA. ALL RIGHTS RESERVED. What are metrics for? Finding your problems (aka Continuous Improvement)

15. 15 © 2016 CA. ALL RIGHTS RESERVED. Virtue vs Sin As a lever to drive someone else's behavior For feedback to improve your own performance

16. 16 © 2016 CA. ALL RIGHTS RESERVED. What’s the most important metric in Agile?

17. 17 © 2016 CA. ALL RIGHTS RESERVED. Working software is the primary measure of progress. What’s the most important metric in Agile?

18. 18 © 2016 CA. ALL RIGHTS RESERVED. 9,192,631,770

19. 19 © 2016 CA. ALL RIGHTS RESERVED. 9,192,631,770 Don’t be fooled by the fallacy of precision

20. 20 © 2016 CA. ALL RIGHTS RESERVED. Let’s look at some practice metrics

21. 21 © 2016 CA. ALL RIGHTS RESERVED. Total Task Hours To Do Total Points Accepted Ideal Task Hours Burndown

22. 22 © 2016 CA. ALL RIGHTS RESERVED. Unfinished iteration planning Accepted half of user stories planned Accepted user stories late in iteration Remaining work has peaked Burndown is flat period in the middle Consistent acceptance once started Problem Solved?

23. 23 © 2016 CA. ALL RIGHTS RESERVED. Accepted DefinedIn-Progress Completed

24. 24 © 2016 CA. ALL RIGHTS RESERVED. Late acceptance Incomplete acceptance Drop in work on 7th day Lots of Work in Progress Work carried over from previous iteration

25. 25 © 2016 CA. ALL RIGHTS RESERVED.

26. 26 © 2016 CA. ALL RIGHTS RESERVED. Considerations –You’re an Agile coach –Arrived today –Know no history –Have the metrics –What do we see in the data? –What would we recommend? Agile

27. 27 © 2016 CA. ALL RIGHTS RESERVED. Scenario 1 Company 1 –Large engineering program (15 Teams) –Adopted SAFe (Scaling Agile) Team 1 Data: –Velocity - Iteration 2.3 thru 5.3 –Iteration 5.1, 5.2 & 5.3

28. 28 © 2016 CA. ALL RIGHTS RESERVED. Scenario 1 - Team 1 - Velocity

29. 29 © 2016 CA. ALL RIGHTS RESERVED. Scenario 1 - Team 1 - Velocity

30. 30 © 2016 CA. ALL RIGHTS RESERVED. Scenario 1 - Team 1 - Iteration 5.1

31. 31 © 2016 CA. ALL RIGHTS RESERVED. Scenario 1 - Team 1 - Iteration 5.2

32. 32 © 2016 CA. ALL RIGHTS RESERVED. Scenario 1 - Team 1 - Iteration 5.3

33. 33 © 2016 CA. ALL RIGHTS RESERVED. Scenario 1 - Observations Team 1 Inconsistent Velocity Acceptance work at iteration end Lots of WIP Not planning iteration well Adding scope during iteration Changing iteration length

34. 34 © 2016 CA. ALL RIGHTS RESERVED. Scenario 2 Same Company 1 –Large engineering program (15 Teams) –Adopted SAFe (Scaling Agile) Team 2 Data: –Velocity - Iteration 2.3 thru 5.3 –Iteration 5.1, 5.2 & 5.3

35. 35 © 2016 CA. ALL RIGHTS RESERVED. Scenario 2 - Team 2 - Velocity

36. 36 © 2016 CA. ALL RIGHTS RESERVED. Scenario 2 - Team 2 - Velocity

37. 37 © 2016 CA. ALL RIGHTS RESERVED. Scenario 2 - Team 2 - Iteration 5.1

38. 38 © 2016 CA. ALL RIGHTS RESERVED. Scenario 2 - Team 2 - Iteration 5.2

39. 39 © 2016 CA. ALL RIGHTS RESERVED. Scenario 2 - Team 2 - Iteration 5.3

40. 40 © 2016 CA. ALL RIGHTS RESERVED. Scenario 2 - Observations Team 2  Good velocity, trending upward  Not entering defects  Early acceptance of work  Better control of WIP  Good planning of iteration  Consistent scope during iteration  Changing iteration length

41. 41 © 2016 CA. ALL RIGHTS RESERVED. Scenario 3 Same Company 1 –Large engineering program (15 Teams) –Adopted SAFe (Scaling Agile) Team 1 vs 2 Data: –Impact of changing timebox length

42. 42 © 2016 CA. ALL RIGHTS RESERVED. When are these the same?

43. 43 © 2016 CA. ALL RIGHTS RESERVED. / hour / hour and a half When are these the same?

44. 44 © 2016 CA. ALL RIGHTS RESERVED. 35 28 35 28

45. 45 © 2016 CA. ALL RIGHTS RESERVED. Scenario 3 - Observations Team 1 vs 2  DON’T CHANGE TIMEBOX!!!

46. 46 © 2016 CA. ALL RIGHTS RESERVED. Scenario 4 New Company!!! –Single Agile Team –Practicing Agile for 2 years –Dedicated ScrumMaster –Distributed team –2 wk sprints; 3 mo releases Single Team Data: –Single Iteration –Entire Release

47. 47 © 2016 CA. ALL RIGHTS RESERVED. Scenario 4 - Iteration

48. 48 © 2016 CA. ALL RIGHTS RESERVED. Scenario 4 - Iteration

49. 49 © 2016 CA. ALL RIGHTS RESERVED. Scenario 4 - Release

50. 50 © 2016 CA. ALL RIGHTS RESERVED. Scenario 4 - Observations Iteration  Task actuals increase  Task burndown flat  Late acceptance  Work carrying over from previous iterations  Unfinished work moved to next iteration  Iterations are consistently over committed  Iterations velocity is consistent, but obviously lower than what the scope of the release is. Release  Shows the inconsistent nature of acceptance and late acceptance.  Release burndown depicts that work is getting done at the same rate as it is being added.  However, there is a gap between remaining and scope at the end of the release. What would we recommend?

51. 51 © 2016 CA. ALL RIGHTS RESERVED. Scenario 5 - Expert Level New-New Company!!! –Single Agile Team –Practicing Agile for 2 years –Dedicated ScrumMaster –2 wk sprints; 3 mo releases Single Team Data: –Single Iteration –Entire Release

52. 52 © 2016 CA. ALL RIGHTS RESERVED. Scenario 5 - Expert Level

53. 53 © 2016 CA. ALL RIGHTS RESERVED. Scenario 5 - Performance Analytics

54. 54 © 2016 CA. ALL RIGHTS RESERVED. Scenario 5

55. 55 © 2016 CA. ALL RIGHTS RESERVED. Scenario 5 - Observations What would we recommend? ?

56. 56 © 2016 CA. ALL RIGHTS RESERVED. How do I get metrics like that?

57. 57 © 2016 CA. ALL RIGHTS RESERVED. The first key to success?

58. 58 © 2016 CA. ALL RIGHTS RESERVED. The first key to success? Estimation!

59. 59 © 2016 CA. ALL RIGHTS RESERVED.

60. 60 © 2016 CA. ALL RIGHTS RESERVED.

61. 61 © 2016 CA. ALL RIGHTS RESERVED.

62. 62 © 2016 CA. ALL RIGHTS RESERVED.

63. 63 © 2016 CA. ALL RIGHTS RESERVED. The second key to success?

64. 64 © 2016 CA. ALL RIGHTS RESERVED. The second key to success? Daily updates of progress

65. 65 © 2016 CA. ALL RIGHTS RESERVED. (I moustache you a question!)

66. 66 © 2016 CA. ALL RIGHTS RESERVED. The End

67. Solutions Architect Jim Damato