1. Copyright © 2001, Software Productivity Consortium NFP, Inc. SOFTWAR E PRODUCTIVITY CONSORTIUM Integrating Information Models: the essential element of successful measurement programs COCOMO Forum October 2004 Chris Miller Software Productivity Consortium

2. 2 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Motivation Popular emphasis on goal-based measurement, but —Anyone can have goals —Goals often are not consistent —Organizational goals may be independent of project- specific issues Measurement initiatives often implemented as disjoint collections of data Measurement systems across the enterprise must be aligned to provide a true enterprise perspective Quantification of casual relationships provide the basis for predictive analysis and forecasting performance

3. 3 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Sales Customer Satisfaction Market INNOVATION TIME-TO- MARKET PRICE QUALITY Information Needs Enterprise Process Project Product

4. 4 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Enterprise Process Project Product Types of Information Needs Measurement should support business goals and objectives Information needs flow downward Data and information are passed upward Business performance Efficiency and effectiveness of production Accomplishment of project budget and schedule Satisfaction of customer requirements

5. 5 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Enterprise View Need to measure business performance —Focusing on strategic goals —Reporting profitability (program, business unit) —Providing information for marketing Enterprise may ask questions such as: —Should we invest in a new product line? When present, an enterprise policy for managing and measuring drive measurement behavior and actions

6. 6 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Balanced Scorecard *Source: Robert S. Kaplan, and David P. Norton. “The Balanced Scorecard – Measures that Drive Performance,” Harvard Business Review, January 1992.

7. 7 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Process View Concurrent processes exist in a typical enterprise (systems, acquisition, development, operations, logistics, marketing, …) Focus on the process capability —Definition of processes –Including measurement points and vehicles —Adherence to defined processes –Self-audits —Improvement of processes –Effectiveness –Efficiency

8. 8 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Product View The system itself is the primary product —Component quality characteristics —Need to assess impact of changes across the system components —Customer satisfaction with product Products and deliverables —System architecture, specifications, software, interface control documents, test cases & procedures, users’ guides, …

9. 9 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models …Applied Across the Life Cycle Critical process performance measured in process, during process execution Product quality characteristics measured and addressed at product transition points throughout the life cycle Defects Maintenance PhasesDevelopment Phases

10. 10 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Project Views Focus on the activities and tasks as a project —Meeting product delivery dates —Planning effort —Monitoring and controlling effort –Adherence to management plan —Customer satisfaction with performance

11. 11 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models PSM Analysis Model

12. 12 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Integration of Views Enterprise Info. Needs Process Info. Needs Project Info. Needs Product Info. Needs Enterprise Info. Models Process Info. Models Project Info. Models Product Info. Models Systems Eng. Software Eng. Systems Software Senior Mgmt Commitment Plan Analyze Evaluate Automate Common Process Tailored Support Level-Specific Information

13. 13 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Integration Activities 1.Each view maintains it’s own set information models for monitoring their own information needs 2.Senior management commitment —Buy-in is needed at the most senior level to facilitate and drive an integrated measurement program (after all they will benefit the most) 3.Use common base measures —When planning to do measurement use a standard —Standard consists of common ways to analyze and evaluate the data and measurement process 4.Build automated tool support around the standard base measures and analysis techniques —Requires tailoring for each view

14. 14 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Step 1: Information Models by View Measurement Information Model Defines measurement product Links measurable entities to information needs Provides rigorous definitions of basic concepts Measurement construct consists of : —Base Measures —Derived Measures —Indicators Explains prevalence of three-level measurement models (e.g., Goal-Question-Metric [GQM]) Information Model Measurement Construct Entities Information Needs Information Product Measurable Concept Attribute

15. 15 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models INFORMATION NEEDS ANALYSIS RESULTS ANALYSIS RESULTS AND PERFORMANCE MEASURES IMPROVEMENT ACTIONS Adapted from ISO/IEC CD 15939, Information Technology – Software Measurement Process USER FEEDBACK Establish Commitment Plan Evaluate Technical & Management Processes Core Measurement Process Experience Base MEAS UREMENT PLAN Perform Database Measurement Process Model

16. 16 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Step2: Buy-in with Business Case Engineering Management Project ManagementExecutive Management Business Goals Quality Customer Satisfaction Cycle Time Cost Reduced Rework Fewer Failures Lower Price Lower support costs Productivity Increased Capacity Reduced Effort Market Share Profit Shorter schedules Time-to- market Higher Margin Low-cost provider Retention Referrals Award fees Bonuses Image SPI Activities

17. 17 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Map Goals/Measures/Activities (Cycle Time Example) Quality Cycle Time Reduced Rework Productivity Increased Capacity Process Engineering Defined Process Efficient Process Predictable Schedules Better Estimates Stable Requirements Historical Data Estimating Model Measurement Program Change Control Requirements Management Defined Process Shorter Schedules (Cust. Sat.) Reduced Labor (Cost) Early to Market Process Mapping Process Initiatives Desired Results of Activities Fewer Steps, Delays Business Goals, Objectives Engineering Analysis, Response

18. 18 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Step 2 Exit Criteria Business case needs to: —Obtain senior management commitment —Show information flow down —Highlight key activities Provide business-oriented information for each view —Potentially new information needs —Highlight linkage to other views

19. 19 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Step 3: Unify Definitions Base measures need to remain consistent across all measurement activities: —Uniform base measures —Data formats —Identify common entities and attributes Analysis techniques are tailored to generate unique indicator base on individual information needs: —Define derived measures —Consistent analysis techniques —Identify common reporting standards and graphic for related indicators

20. 20 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Uniform Base Measures Consistent use of —Data labels (e.g., SLOC) —Data formats Base measures are the lowest common denominator —Validate across all information models Avoid redundant data collection methods Maintain integrity of scales and units Base Measure Name Release 01.03 Lines of code Attribute C++ language statements Measurement Method Count semicolons in all program units Type of Measurement Method Objective Scale (including precision) Whole numbers from 0 to infinity Type of Scale Ratio Unit of Measure Line of code Base Measure Definition/Template

21. 21 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Uniform Analysis Mitigate redundant naming conventions between indicators —(e.g., Productivity) Set standard analysis approach, techniques, and methodologies Ensure decision criteria in consistent across information models (and views) Use consistent graphic representations for similar data ID Cost Performance Index for Project XX Information Need Enter ID of information need Measures Used Derived Measure(s) Cost Performance Index (CPI) Base Measures 1.Actual Cost of Work Performed 2.Budgeted Cost of Work Performed Function(s) Calculate CPI: For any given project, perform the following calculation: Actual Cost of Work Performed (ACWP) Budgeted Cost of Work Performed (BCWP) Analysis Model Simple CPI calculation. Optimal performance is to keep actuals close to or at the center line value of 1.0. Decision Criteria Expected value is ‘1.0’. Decision needed if actual value is +/– 10%. Graphical Representation. Standard CPI line graph with value of 1 represented as a horizontal line, positive values above and negative values below. CPI = Indicator Definition/Template

22. 22 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Step 4: Tool Support Automated tools support is required to handle the mass of data collection and analysis associated with an integrated measurement program across many views Base measure definitions and analysis techniques provide a basis for selection of tool suite Understanding the information flow in the business case helps during trade-off decisions among different tools

23. 23 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Summary An effectively designed measurement program must consider the interrelationships of the views An integrated measurement program will need to establish: —Uniform base measures —Common tools and databases Each view uses the base measures to: —Define indicators Allows for the flexibility for each view to address their information needs Integration is an impossible task without senior management support

24. 24 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Back-up Slides

25. 25 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models Interpretation Estimate or evaluation that provides a basis for decision making Indicator Model Derived Measure Value resulting from applying the algorithm to two or more measures Algorithm combining measures and decision criteria Derived Measure Operations mapping an attribute to a scale Method Base Measure Function Algorithm combining two or more base measures Value resulting from applying the method to one attribute Attribute Property relevant to information needs Entities Information Needs Information Product Source: Adapted from ISO/IEC 15939, Software Measurement Process Framework Measurement Construct Method Base Measure

26. 26 Copyright © 2004, Software Productivity Consortium NFP, Inc. Integrating Information Models 0.5 > Organization average of 0.3 defects per page Indicator Model Derived Measure Compare defect density to SE organizational data Interpretation Compare defect density to organizational average, perform causal analysis if > 1.0 defects per page Information Product Indicator summarized for all SE documents in monthly status Measurement Construct Example Derived Measure Function Divide (defects per page) 0.5 defects per page Attribute Entities Count only bullets in minutes Method Base Measure 50 defects found Defects per peer review Method Base Measure Count only technical pages 100 pages reviewed Pages reviewed SE Document Peer Review Information Needs Systems Engineering performance – documentation quality? Source: Adapted from ISO/IEC 15939, Software Measurement Process Framework