Copyright , Dennis J. Frailey CSE7315 – Software Project Management CSE7315 M27 - Version 9.01 SMU CSE 7315 Planning and Managing a Software Project Module 27 Project Tracking and Oversight

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Objectives of This Module  To present an overview of tracking and oversight  To discuss some basic terminology regarding measurement

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Tracking and Oversight Occur Continuously Manage Risks Define the Approach Generate Detailed Plans Understand the Need Execute and Monitor Futrell, chapters 21, 25; Humphrey, chapters 6, 15

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version SW Project Monitoring Typical Symptoms of a Problem We’re six months behind schedule, and nobody knew it! Why did it take us so long to find out? January

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version SW Project Monitoring Symptoms of Another Problem They’ve been working on that module for eight weeks and everyone else is waiting for it. Did the developers in charge know how many people depend on the module?

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version SW Project Monitoring Other Examples  The project manager promised a new feature to the customer -- but never told any of the programmers! –“I thought you knew about this!”  The software takes up too much disk space. –“Nobody ever thought it would get that big.”

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version SW Project Tracking and Oversight Purpose To provide adequate visibility into actual progress so that management can take effective actions when the software project’s performance deviates significantly from the software plans

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version SW Tracking and Oversight Goals from the SEI CMM/CMMI 1) Actual results and performance are tracked against software plans –Plans are revised to reflect actual performance and changes in requirements or commitments 2) Corrective actions are taken and managed to closure when actual performance deviates significantly from software plans

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version PM Today SW Tracking and Oversight Goals from the SEI CMM/CMMI 3) Changes to software commitments are communicated to all affected groups and individuals

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version SW Project Tracking & Oversight Practices Recommended by SEI  Use a software development plan for tracking and communicating status  Track schedule, size, effort, computer resources, technical activities and risks  Hold periodic reviews and take corrective actions  Revise plans and schedules to reflect changes -- using a defined procedure  Review customer commitments on a regular basis

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Tracking Progress

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Some Things You Can Estimate and Track  Costs  Sizes  Quality  Reliability  Schedules  Staffing  etc.

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Establish a Data Base  Know about your organization –Performance on past projects –Lessons Learned  Know about your industry and competitors –What is best in class? –Improvement rates Historical Data Base - Data - Lessons - etc. Facts to help you manage

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Example of Experience vs. History  History: for C++ doing your kind of software, you should be generating –25 lines of code per day during the coding phase, with –3 errors per 1000 lines of code during module test  Actual experience on your project: –40 lines of code per day, with –0.5 errors per 1000 lines of code during module test

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Optimist’s Conclusion Do you have a solid reason to explain this difference? Ask questions. Why are you better? –Is the process different? –Are the people a lot better? –Are the tools better? We are doing much better than in the past!

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Pessimist’s Conclusion Ask questions. Find out what is really happening. –Are the tests being performed? –Is the coverage adequate? –Are there higher rates of customer complaints after shipment? Our testing is no good (perhaps because it is being rushed due to deadlines)

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Knowing the Competition Can Give You Insights  But what if the norm in your industry is an improvement of 25%?  And what if your competitors have all switched to Java and are 50% more productive as a result? We improved our “C” language productivity by 15%

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Measurement

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Every Measure Should Have a Purpose -- You Want to Get Information Data Analysis Information

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version But When You Measure Anything you Change it  Measurement takes time away from the task being performed, so it slightly reduces productivity  When people are measured they change their behavior to make the measurements look more favorable Try to measure in ways that minimize disruption Understand the built-in bias when you measure people Try to measure in ways that minimize disruption Understand the built-in bias when you measure people

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version For Every Analysis there are Two Possible Results Information - tells you something right –We are (or are not) on schedule –Our risks are (or are not) under control Misinformation - tells you something wrong –We are (or are not) on schedule –Our risks are (or are not) under control It takes careful analysis to assure that you are getting genuine information

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Key Issues for Effective Measurement  Define how to interpret measurements –To form a basis of consistent analysis  Choose consistent display (graph) techniques –So people know how to interpret the data  Define how to use each measurement –You must also demonstrate that you are using each measurement that way, so people will believe you –Given any measure, people will change to make it look to their advantage  you want to make their behavior change in a positive way

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Organizational Framework  There must be an Organizational Framework for understanding the importance of measurements –i.e., people do not sabotage the data collection effort –And people do not abuse measures –And people do not draw wrong conclusions

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Achieving an Effective Organizational Framework  Educate everyone in the proper use of measurements  Develop the right measures –Involve those who are being measured –Measure only what you can benefit from  Use the measurements –To make decisions about the product and the process –But NOT to make decisions about people

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version You Want to Achieve Optimal Performance  Don’t over-measure or under-measure  Don’t over-test or under-test  Don’t over-inspect or under-inspect  etc. Track the things that represent your greatest risks and concerns Remember that it costs time and money to track - make it worthwhile Track the things that represent your greatest risks and concerns Remember that it costs time and money to track - make it worthwhile

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Basic Definition and Terminology Issues  Levels of Measurement  What to Measure  Who Cares about What Measures

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Levels of Measurement  What do we mean by a measure?  Does everyone mean the same thing?  How do we resolve discrepancies?  Consider the example on the next slide:

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Information may be Measured in Many Ways! Cost is best measured in staff days! Why not dollars ? I measure it by stock price. However they measure it, they had better figure out how to reduce cost!

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Information, Measures and Data Process in Execution Data Information Need Measure

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Example Process in Execution Units Produced Head- count Lines of Code $ SpentMonths Productivity Units Per Month LOC per Staff Month $ per Line of Code

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Data (or Primitive Measures) The fundamental, factual quantities that characterize a process or product –Specific facts that are countable or otherwise obtainable –Minimum amount of analysis Definitions: Data

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Key Issues: Data  Consistent definitions  Effective collection Examples: –Hours worked –Number of employees –Lines of code

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Consistent Definitions  What do you mean by a “day” of work? Does everyone mean the same thing?  Such questions should be asked about any data you propose to collect, such as lines of code, people, dollars, defects, variance, tests completed, complaints You must define what you want … -- How many hours per day? -- What is a “line” of code? -- Which lines to include in LOC? You must define what you want … -- How many hours per day? -- What is a “line” of code? -- Which lines to include in LOC?

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version How Long Is a Staff Day? Salaried Staff Staff Day OvertimeRegular Time Hourly StaffSalaried StaffHourly Staff UnpaidPaidUnpaidPaid

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Effective Collection  Can you collect accurate and consistent data?  Can you collect it efficiently?  How will the organization and the process change when you collect the data?

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Definitions: Measures Measure (sometimes called Compound Measure) The result of counting or otherwise quantifying an attribute of a process or product –How we quantify –Something we can analyze and interpret –Generally associated with specific graphs and rules of interpretation

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Key Issues: Measures  Proper interpretation  How measures are displayed  Effective use of measures Examples: –lines of code per hour –ratio of current to historical test performance –turnover rate

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Display and Interpretation: Consider the Following Graph

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Effective Use  You must define how you will use each measure –A measure can typically be used many ways –Some of these ways can be ineffective or even harmful –People tend to fear measurement because of past experience with misuse  You must also demonstrate that you are using it as planned, so people will believe you  Given any measure, people will change to make it look to their advantage –So you want to make their behavior change in a positive way

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Example: A Measure & Its Impact Information Need: Productivity Measure 1: Lines of code per day Use: reward those who produce the most lines of code per day Result: people may produce bloated, inflated code in order to look good Measure 2: requirements met and tested, weighted by complexity of requirement Use: track against history and use to identify process bottlenecks Result: people may use the data to make the process more efficient, resulting in lower cost

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version A Less Common Measure That May be Very Effective Information Need: Productivity Measure 3: Number of customer complaints Use: reward those who produce the code that is responsible for the fewest customer complaints Result: developers may pay particular care to the customer’s need, resulting in future sales and business success

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Definitions: Information Information A measured quantity that provides insight into a software issue or concept or goal. –What and why we measure –An indicator –A standard of measurement that ties to a goal or purpose

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Key Issues: Information  Selecting a measure that provides the desired insight or information regarding the goal or purpose Examples: –Comparisons (planned vs. actual) –Indices (performance ratio, productivity, return on investment)

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Focusing on the Goals and Needs  Customer satisfaction can be measured in terms of response to a survey that biases customer response and hides their real views  Or it can be measured in terms of something that accurately reflects customer views  The same applies to measures of productivity, quality, cycle time, defect density, etc. Each measure should provide information that addresses some higher level objective, such as “reducing cost” or “on time delivery”

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Select the Right Measure  There are many ways to measure a desired quantity and different organizations measure differently  The key is to spend time to select the right measure, so you satisfy the information needs in support of the higher level objective(s)

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Good and Not So Good Measures Goal: Produce software more efficiently Information Need: Efficiency Measure 1: tests completed per week Result: easy tests done first; corners cut in testing; hard problems ignored or deferred Measure 2: rework Result: process and methods are improved to reduce rework, resulting in more efficient software development

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Rework Is an Excellent Measure of Efficiency or Productivity It tells you where you are wasting resources However … Rework is a lagging indicator. It does not spot problems when they are happening or in advance

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version What Should We Measure? ProductProjectProcess determines success of determines quality of root causes Process Measures –Effectiveness of the process –How well are we following the process? –Risk monitoring Product Measures –Performance and quality –How well is the product meeting its requirements? Project Measures –The state of the project –How are we doing relative to cost, schedule, staffing, etc.?

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version ProductProjectProcess Attributes What Resources Quality Time Are We On Schedule? Expenses vs. Budget? How Fast can we Manufacture? What Is our Cycle Time? Post-release Defects? What will it Cost? What is our Productivity? Customer Satisfaction? In-process Defects? Performance Meets Perfor- mance Goals? Meets Mgt. Goals? Does it Work? What Attributes Can We Measure?  We want attributes that relate to our goals –time, resources, performance, quality etc.  The following type of matrix can help:

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Who Cares About What?  Managers usually care about project measures - that’s how they are judged -- But if the project is in trouble they need to know more  Developers usually care about product measures -- that’s what they are evaluated by  Both should care about process measures -- this is usually where you learn the reasons why a project or a product is in trouble

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Summary of Module  Measure to know what is really happening  Avoid misinterpreting the data  Define how to measure, interpret, and display the data  There are many ways to measure a given quantity – choose one that works best for you

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version References  Baumert, John H., and Mark S. McWhinney, Software Measures and the Capability Maturity Model, CMU/SEI-92-TR-25, ESC-TR , Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa.,  DeMarco, Tom, Controlling Software Projects: Management, Measurement, and Estimation, New York, Yourdon Press,  Grady, Robert B. Practical Software Metrics for Project Management and Process Improvement. Englewood Cliffs, N.J., Prentice-Hall, Inc., ISBN

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version Possible Exam Questions  Explain why tracking and oversight are important for project management  Explain the difference between an information need and a measure.  Explain several risks with defining and collecting data properly

Copyright , Dennis J. Frailey CSE7315- Software Project Management CSE7315 M27 - Version END OF MODULE 27