1. Copyright 1995-2009, Dennis J. Frailey CSE7315 – Software Project Management CSE7315 M27 - Version 9.01 SMU CSE 7315 Planning and Managing a Software Project Module 27 Project Tracking and Oversight

2. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 2 Objectives of This Module  To present an overview of tracking and oversight  To discuss some basic terminology regarding measurement

3. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 3 Tracking and Oversight Occur Continuously Manage Risks Define the Approach Generate Detailed Plans Understand the Need Execute and Monitor Futrell, chapters 21, 25; Humphrey, chapters 6, 15

4. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 4 SW Project Monitoring Typical Symptoms of a Problem We’re six months behind schedule, and nobody knew it! Why did it take us so long to find out? January

5. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 5 SW Project Monitoring Symptoms of Another Problem They’ve been working on that module for eight weeks and everyone else is waiting for it. Did the developers in charge know how many people depend on the module?

6. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 6 SW Project Monitoring Other Examples  The project manager promised a new feature to the customer -- but never told any of the programmers! –“I thought you knew about this!”  The software takes up too much disk space. –“Nobody ever thought it would get that big.”

7. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 7 SW Project Tracking and Oversight Purpose To provide adequate visibility into actual progress so that management can take effective actions when the software project’s performance deviates significantly from the software plans

8. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 8 SW Tracking and Oversight Goals from the SEI CMM/CMMI 1) Actual results and performance are tracked against software plans –Plans are revised to reflect actual performance and changes in requirements or commitments 2) Corrective actions are taken and managed to closure when actual performance deviates significantly from software plans

9. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 9 3PM Today SW Tracking and Oversight Goals from the SEI CMM/CMMI 3) Changes to software commitments are communicated to all affected groups and individuals

10. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 10 SW Project Tracking & Oversight Practices Recommended by SEI  Use a software development plan for tracking and communicating status  Track schedule, size, effort, computer resources, technical activities and risks  Hold periodic reviews and take corrective actions  Revise plans and schedules to reflect changes -- using a defined procedure  Review customer commitments on a regular basis

11. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 11 Tracking Progress

12. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 12 Some Things You Can Estimate and Track  Costs  Sizes  Quality  Reliability  Schedules  Staffing  etc.

13. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 13 Establish a Data Base  Know about your organization –Performance on past projects –Lessons Learned  Know about your industry and competitors –What is best in class? –Improvement rates Historical Data Base - Data - Lessons - etc. Facts to help you manage

14. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 14 Example of Experience vs. History  History: for C++ doing your kind of software, you should be generating –25 lines of code per day during the coding phase, with –3 errors per 1000 lines of code during module test  Actual experience on your project: –40 lines of code per day, with –0.5 errors per 1000 lines of code during module test

15. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 15 Optimist’s Conclusion Do you have a solid reason to explain this difference? Ask questions. Why are you better? –Is the process different? –Are the people a lot better? –Are the tools better? We are doing much better than in the past!

16. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 16 Pessimist’s Conclusion Ask questions. Find out what is really happening. –Are the tests being performed? –Is the coverage adequate? –Are there higher rates of customer complaints after shipment? Our testing is no good (perhaps because it is being rushed due to deadlines)

17. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 17 Knowing the Competition Can Give You Insights  But what if the norm in your industry is an improvement of 25%?  And what if your competitors have all switched to Java and are 50% more productive as a result? We improved our “C” language productivity by 15%

18. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 18 Measurement

19. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 19 Every Measure Should Have a Purpose -- You Want to Get Information Data Analysis Information

20. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 20 But When You Measure Anything you Change it  Measurement takes time away from the task being performed, so it slightly reduces productivity  When people are measured they change their behavior to make the measurements look more favorable Try to measure in ways that minimize disruption Understand the built-in bias when you measure people Try to measure in ways that minimize disruption Understand the built-in bias when you measure people

21. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 21 For Every Analysis there are Two Possible Results Information - tells you something right –We are (or are not) on schedule –Our risks are (or are not) under control Misinformation - tells you something wrong –We are (or are not) on schedule –Our risks are (or are not) under control It takes careful analysis to assure that you are getting genuine information

22. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 22 Key Issues for Effective Measurement  Define how to interpret measurements –To form a basis of consistent analysis  Choose consistent display (graph) techniques –So people know how to interpret the data  Define how to use each measurement –You must also demonstrate that you are using each measurement that way, so people will believe you –Given any measure, people will change to make it look to their advantage  you want to make their behavior change in a positive way

23. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 23 Organizational Framework  There must be an Organizational Framework for understanding the importance of measurements –i.e., people do not sabotage the data collection effort –And people do not abuse measures –And people do not draw wrong conclusions

24. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 24 Achieving an Effective Organizational Framework  Educate everyone in the proper use of measurements  Develop the right measures –Involve those who are being measured –Measure only what you can benefit from  Use the measurements –To make decisions about the product and the process –But NOT to make decisions about people

25. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 25 You Want to Achieve Optimal Performance  Don’t over-measure or under-measure  Don’t over-test or under-test  Don’t over-inspect or under-inspect  etc. Track the things that represent your greatest risks and concerns Remember that it costs time and money to track - make it worthwhile Track the things that represent your greatest risks and concerns Remember that it costs time and money to track - make it worthwhile

26. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 26 Basic Definition and Terminology Issues  Levels of Measurement  What to Measure  Who Cares about What Measures

27. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 27 Levels of Measurement  What do we mean by a measure?  Does everyone mean the same thing?  How do we resolve discrepancies?  Consider the example on the next slide:

28. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 28 Information may be Measured in Many Ways! Cost is best measured in staff days! Why not dollars ? I measure it by stock price. However they measure it, they had better figure out how to reduce cost!

29. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 29 Information, Measures and Data Process in Execution Data Information Need Measure

30. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 30 Example Process in Execution Units Produced Head- count Lines of Code $ SpentMonths Productivity Units Per Month LOC per Staff Month $ per Line of Code

31. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 31 Data (or Primitive Measures) The fundamental, factual quantities that characterize a process or product –Specific facts that are countable or otherwise obtainable –Minimum amount of analysis Definitions: Data

32. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 32 Key Issues: Data  Consistent definitions  Effective collection Examples: –Hours worked –Number of employees –Lines of code

33. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 33 Consistent Definitions  What do you mean by a “day” of work? Does everyone mean the same thing?  Such questions should be asked about any data you propose to collect, such as lines of code, people, dollars, defects, variance, tests completed, complaints You must define what you want … -- How many hours per day? -- What is a “line” of code? -- Which lines to include in LOC? You must define what you want … -- How many hours per day? -- What is a “line” of code? -- Which lines to include in LOC?

34. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 34 How Long Is a Staff Day? Salaried Staff Staff Day OvertimeRegular Time Hourly StaffSalaried StaffHourly Staff UnpaidPaidUnpaidPaid

35. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 35 Effective Collection  Can you collect accurate and consistent data?  Can you collect it efficiently?  How will the organization and the process change when you collect the data?

36. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 36 Definitions: Measures Measure (sometimes called Compound Measure) The result of counting or otherwise quantifying an attribute of a process or product –How we quantify –Something we can analyze and interpret –Generally associated with specific graphs and rules of interpretation

37. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 37 Key Issues: Measures  Proper interpretation  How measures are displayed  Effective use of measures Examples: –lines of code per hour –ratio of current to historical test performance –turnover rate

38. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 38 Display and Interpretation: Consider the Following Graph

39. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 39 Effective Use  You must define how you will use each measure –A measure can typically be used many ways –Some of these ways can be ineffective or even harmful –People tend to fear measurement because of past experience with misuse  You must also demonstrate that you are using it as planned, so people will believe you  Given any measure, people will change to make it look to their advantage –So you want to make their behavior change in a positive way

40. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 40 Example: A Measure & Its Impact Information Need: Productivity Measure 1: Lines of code per day Use: reward those who produce the most lines of code per day Result: people may produce bloated, inflated code in order to look good Measure 2: requirements met and tested, weighted by complexity of requirement Use: track against history and use to identify process bottlenecks Result: people may use the data to make the process more efficient, resulting in lower cost

41. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 41 A Less Common Measure That May be Very Effective Information Need: Productivity Measure 3: Number of customer complaints Use: reward those who produce the code that is responsible for the fewest customer complaints Result: developers may pay particular care to the customer’s need, resulting in future sales and business success

42. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 42 Definitions: Information Information A measured quantity that provides insight into a software issue or concept or goal. –What and why we measure –An indicator –A standard of measurement that ties to a goal or purpose

43. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 43 Key Issues: Information  Selecting a measure that provides the desired insight or information regarding the goal or purpose Examples: –Comparisons (planned vs. actual) –Indices (performance ratio, productivity, return on investment)

44. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 44 Focusing on the Goals and Needs  Customer satisfaction can be measured in terms of response to a survey that biases customer response and hides their real views  Or it can be measured in terms of something that accurately reflects customer views  The same applies to measures of productivity, quality, cycle time, defect density, etc. Each measure should provide information that addresses some higher level objective, such as “reducing cost” or “on time delivery”

45. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 45 Select the Right Measure  There are many ways to measure a desired quantity and different organizations measure differently  The key is to spend time to select the right measure, so you satisfy the information needs in support of the higher level objective(s)

46. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 46 Good and Not So Good Measures Goal: Produce software more efficiently Information Need: Efficiency Measure 1: tests completed per week Result: easy tests done first; corners cut in testing; hard problems ignored or deferred Measure 2: rework Result: process and methods are improved to reduce rework, resulting in more efficient software development

47. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 47 Rework Is an Excellent Measure of Efficiency or Productivity It tells you where you are wasting resources However … Rework is a lagging indicator. It does not spot problems when they are happening or in advance

48. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 48 What Should We Measure? ProductProjectProcess determines success of determines quality of root causes Process Measures –Effectiveness of the process –How well are we following the process? –Risk monitoring Product Measures –Performance and quality –How well is the product meeting its requirements? Project Measures –The state of the project –How are we doing relative to cost, schedule, staffing, etc.?

49. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 49 ProductProjectProcess Attributes What Resources Quality Time Are We On Schedule? Expenses vs. Budget? How Fast can we Manufacture? What Is our Cycle Time? Post-release Defects? What will it Cost? What is our Productivity? Customer Satisfaction? In-process Defects? Performance Meets Perfor- mance Goals? Meets Mgt. Goals? Does it Work? What Attributes Can We Measure?  We want attributes that relate to our goals –time, resources, performance, quality etc.  The following type of matrix can help:

50. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 50 Who Cares About What?  Managers usually care about project measures - that’s how they are judged -- But if the project is in trouble they need to know more  Developers usually care about product measures -- that’s what they are evaluated by  Both should care about process measures -- this is usually where you learn the reasons why a project or a product is in trouble

51. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 51 Summary of Module  Measure to know what is really happening  Avoid misinterpreting the data  Define how to measure, interpret, and display the data  There are many ways to measure a given quantity – choose one that works best for you

52. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 52 References  Baumert, John H., and Mark S. McWhinney, Software Measures and the Capability Maturity Model, CMU/SEI-92-TR-25, ESC-TR- 92-025, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa., 1992.  DeMarco, Tom, Controlling Software Projects: Management, Measurement, and Estimation, New York, Yourdon Press, 1982.  Grady, Robert B. Practical Software Metrics for Project Management and Process Improvement. Englewood Cliffs, N.J., Prentice-Hall, Inc., 1992. ISBN 0-13- 720384-5.

53. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 53 Possible Exam Questions  Explain why tracking and oversight are important for project management  Explain the difference between an information need and a measure.  Explain several risks with defining and collecting data properly

54. Copyright 1995-2009, Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version 9.01 54 END OF MODULE 27