CEN th Lecture CEN 4021 Software Engineering II Instructor: Masoud Sadjadi Software Project Planning (POMA) Goals and Measurement

12 th LectureCEN 4021: Software Engineering II Acknowledgements  Dr. Onyeka Ezenwoye  Dr. Peter Clarke 2

12 th LectureCEN 4021: Software Engineering II Agenda  Organizing (POMA) –Goals and Measurement

12 th LectureCEN 4021: Software Engineering II Goals and Measurements  During the planning activities, a number of goals and measurements to gauge and validate the attainment of goals were considered and identified.  The project team still needs to be organized and motivated to both understand and accept these goals and measurements.  Managers are responsible for transforming the plan into executable items during the organizing of POMA.

12 th LectureCEN 4021: Software Engineering II Goals and Measurement  During the planning phase the goals for and measurements of, the key attributes of the product and services are determined.  Many of the goals for the system are deduced from the functional and nonfunctional requirements i.e. obtained from the customer.  Measurable attribute: An attribute for which there is a well-defined metric and a methodology for its measurements.

12 th LectureCEN 4021: Software Engineering II Overview of Goals and Measurements cont Example goals: –A secure system –A fully functional system –A high-quality system –A user-friendly and attractive system –A cost-effective project –A project that meets the schedule

12 th LectureCEN 4021: Software Engineering II  Validation of goal: –Comparing a stated goal for an attribute with the actual measurement taken for the attribute.  Verification of measurement: –Ensuring that the measurement of an attribute is properly taken and recorded.  Metric: –The unit used to characterize an attribute.  Measurement: The act of characterizing an attribute, which may involve multiple steps, utilizing the agreed upon metric for that attribute. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II Deliverable-related metrics and measurements  Some s/w deliverable attributes include: –Quality –Usability –Functional completeness –Maintainability –Modifiability –Reliability –Installability  Requires some thought and insight before a reasonable goal and metric can be designed

12 th LectureCEN 4021: Software Engineering II Project and project-related metrics and measurements  manager is usually required to describe the project in terms of the following: –Schedule integrity –Cost minimization –Productivity –Efficiency –Cost-effectiveness –Employee morale

12 th LectureCEN 4021: Software Engineering II Examples of measurement data  Estimated or planned versus actual data of software size, cost and schedule  Productivity data  Software reliability measures  Test coverage  Number and severity of defects found in requirements  Number and severity of defects found in code

12 th LectureCEN 4021: Software Engineering II Examples of metrics used to determined progress towards the goal of minimizing defects and managing quality testing  Average time to fix a service request  Cumulative defect found  Defect analysis by code module  Defect fix times, estimated versus actual  Mean time between defects

12 th LectureCEN 4021: Software Engineering II Examples questions used to characterize goals and metrics used to measure them  What is the relationship between module reuse and reliability? 1.Metric: for all modules: what is the level of reuse? 2.For top faulty modules: what is the level of reuse? 3.For all modules: what is the fault density? 4.For non-reused modules: what is the fault density?

12 th LectureCEN 4021: Software Engineering II How do you measure these? 1. Implementation is done smoothly and well documented 2. The final product functions properly and has been built to reflect any changes in the initial requirements. 3. The system will need to be reliable in the sense that it can handle multiple users. This will be measured by having multiple users log into the system and attempt to browse and book.

12 th LectureCEN 4021: Software Engineering II How do you measure these? 4. The interface will need to be easy to learn. An average user should be able to learn the basics of the system within a few minutes of use. 5. The inventory should be updated within seconds. 6. A manager should be able to make adjustments to the inventory with no errors.

12 th LectureCEN 4021: Software Engineering II  The project plan identifies important attributes and their goals. The metrics and measurement scheme for these goals were also conceived.  In the organizing phase several other important notions must also be considered by managers. –Are the goals and their associated measurement schemes clearly defined? –Has the organization embraced the measuring scheme? –Has the cost of measuring been taken into account? Goals and Measurement

12 th LectureCEN 4021: Software Engineering II  It is important that the project team understand the definitions of goals and measurements of the project.  To alleviate the problem of the team not understanding the goals and measurement scheme, the project manager should: –Review the goals set for the product and project attributes –Review the measurement scheme and modify if necessary –Build an “operational” plan for the measurement schemes Goals and Measurement

12 th LectureCEN 4021: Software Engineering II  The desired goals of a project should be stated in a quantitative way.  The classic example of a goal is “easy to use”.  What does “easy to use” mean???  For goals that can be ambiguous or vague.  Decompose them into subgoals.  If the initial requirement was “The product should be easy to use” one decomposed subgoal can be: –Every function in the product should be completed by the user without external intervention. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II  After identifying subgoals, ask the following questions: –Is the subgoal more specific than the original goal? –How would we gauge the transformed attribute? –Is there a need for a specific activity that will allow us to gauge its attainment?  The precise metric and measurement methodology should be defined in terms of a particular test if possible. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II  For “ease of use” example usability test steps might be: 1.A test case is designed for each function in the product. 2.A numerical count is kept of the number of test cases that are successfully executed by a test subject without any external intervention. 3.This test is repeated with the predetermined number of test subjects to ensure the results’ statistical relevancy. 4.All the unsuccessful test cases are summed. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II  A reasonable goal for such a test is to have 5% unsuccessful or 95% successful test cases.  Some measurements can be potentially misleading. –See text book for discussion on the meaning: totally complete, mostly complete, partially complete, and not complete.  If you are using categories to measure if goals are achieved make sure that: –Create categories that exhaustively cover the range of metrics –Each category is mutually exclusive of any other category Goals and Measurement

12 th LectureCEN 4021: Software Engineering II Building a measurement operational plan:  Each item to be measured in the plan may require a slightly different operational plan.  When refining the operational plan the following refinements steps should be considered: –Steps to ensure that the process and methodology are modified to include the details needed to implement each plan. –Steps to ensure that proper resources are made available in a timely manner. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II –Steps to ensure that necessary metrics and measurement schemes are defined for each plan item –Steps to ensure that goals are defined for the implementation and that the achievement of the goals is validated.  Operational Plan: A plan that contains all the details of how to implement what is contained in a general project plan. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II  Operational plan allows for successful measurement during the monitoring phase of the project.  During the organization of the project some elements of the plan may have to be revisited therefore a further refinement of the plan is essential. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II  The following items should be considered during the organizing and preparation phase of the POMA life cycle: –Any additional goal clarification and decomposition –Well-defined goal validation –Specific measurement techniques and schemes –Any process extensions and modifications needed to accommodate measurements –Additional software/hardware tools needed for measurement activities Goals and Measurement

12 th LectureCEN 4021: Software Engineering II Embracing the measurement scheme:  It is very important to get the team on board with the organizational plan for the measurements of the goals.  Some measurement schemes may take on quite a bit of complexity. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II  Manager should not perform all the analysis and justification studies by themselves. Other team members should participate for the following reasons: –More team members would understand the goals and measurements. –More non-management team members would feel committed to the goals and measurements. –Some team members may be counted on to “spread the message” and educate other team members. –Sharing the burden would lessen the workload of the manager. –Distributing the knowledge would lessen the general fear of being measured. –Team ownership of the goals would be more likely to be achieved. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II Goal Attainability  Measurement of the process will make some employer feel that they are being continually watched.  Sometime the measurements are used to improve the process for future projects.  A spmr should be realistic about the expectation of including the measurements of goals in the process.  Before collecting any data explain how the data will be used to the team members.  The goals and measurements activity should be properly planned, and the team should be included in the development of some of the goals. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II Goal Attainability  The team must embrace the goals and measurements as their own.  During the organization and preparation phase the spmr must actively and positively communicate the goals, measurements and measurement scheme to the team.  All team members must be included in the communication about the goals and measurements. Goals and Measurement

12 th LectureCEN 4021: Software Engineering II Goal Attainability  To win acceptance and positive reception of the goals and measurements spmr must ensure the following: –A well-defined goal and measurement scheme –Attainable goals –The team’s participation in the setting of the goals –The team’s understanding and belief in the goals and measurements –The commitment of qualified resources for measurement Goals and Measurement

12 th LectureCEN 4021: Software Engineering II Measurement cost:  Goals and measurements are important yet few s/w projects do it. Why? –“Success” is often gauged by only a single goal e.g., a deliverable’s due date. –The organization may not see the value of setting goals and measurements or may fear the process. –Management may not supply enough resources for the goals and measurements activities. –Team may oppose the goals and therefore do not measure them. Goals and Measurement