1. SM CMM Integration, SCAMPI, SCAMPI Lead Assessor, SCAMPI Lead Appraiser, and SEI are service marks of Carnegie Mellon University.  CMM and CMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. Transdyne Corporation http://transdynecorp.com SCAMPI Class A, B, C ID No. 0100145-01 Contributing Factors Determining the Sufficiency of COCOMO-Based Estimation Processes in SCAMPI SM Appraisals 23 rd International Forum on COCOMO and Systems/Software Cost Modeling and ICM Workshop 3 Dr. Mary Anne Herndon Certified SCAMPI A, B and C Appraiser Transdyne Corporation, San Diego, CA October 27 2008

2. 2 Agenda Overview of the SCAMPI Family Three Phases of a SCAMPI Appraisal The Rules for Verification of Appraisal Evidence Expected SCAMPI Lead Appraiser Competencies and Behaviors Overview of CMMI-DEV v1.2 Model and COCOMO-based Estimation Processes Examples of Contributing Factors in Process Areas Benefits and Summary Contributing Factors Determining the Sufficiency of COCOMO-Based Estimation Processes in SCAMPI SM Appraisals Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum

3. 3 SEI SCAMPI Reference Documents SCAMPI MethodReference Documents Class A, B, and C Appraisal Requirements for CMMI (ARC) v1.2 (CMU/SEI-2006-TR-011), 2006 SCAMPI Lead Appraiser SM Body of Knowledge (SLA BOK) (CMU/SEI-2007-TR-019), 2007 Class A Standard CMMI Appraisal Method for Process Improvement Method Definition Document (MDD) v1.2 (CMU/SEI-2006-HB-002), 2006 Class B and C Handbook for Conducting Standard CMMI Appraisal Method for Process Improvement (SCAMPI) B and C Appraisals (CMU/SEI-2005-HB-005), 2005 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum

4. 4 Three Classes of SCAMPI Appraisal Methods C B A Depth of investigation Breadth of tailoring SCAMPI C provides a wide range of options, including characterization of planned approaches to process implementation according to a scale defined by the user. Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum Plans Deployment Institutionalization

5. 5 SCAMPI Phase ActivitiesRelevant When Implementing COCOMO-Based Estimation Processes There are three phases of a SCAMPI appraisal Phase 1 - Plan and Prepare for the Appraisal Phase 2 - Conduct Appraisal Phase 3 - Report Results Yes Reporting results obtained in Phases 1 and 2 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum SCAMPI Method Appraisal Phases

6. 6 Phase 1 Requirements – Plan and Prepare for the Appraisal Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum * Requirement Relevant When Implementing COCOMO-Based Estimation Processes Class C Class A Class B

7. 7 Phase 1 Requirements – Plan and Prepare for the Appraisal (Continued) Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum * Requirement Relevant When Implementing COCOMO-Based Estimation Processes Class C Class A Class B

8. 8 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum Phase 2 Requirements - Conduct Appraisal * Requirement Relevant When Implementing COCOMO-Based Estimation Processes Class C Class A Class B

9. 9 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum SCAMPI Phase 2 “Conduct Appraisal” Processes ProcessDescription Examine objective evidence Document objective evidence (data sufficiency) Verify objective evidence ( practice strengths & weakness) Practice Characterization Class A – Fully Implemented (FI) Largely Implemented (LI) Partially Implemented (PI) Not Implemented (NI) Not Yet (NY) Class B – Green, Yellow, Red Class C – High, Medium, Low (optional) Validate preliminary appraisal outputs Generate appraisal results Class A - Goal Rating, Maturity and Capability Levels, final findings, Appraisal Disclosure Statement (ADS) Class B and C – characterizations (optional in C), final findings, ADS Appraisal Team

10. Activities Common requirements for all SCAMPI Classes: Designated appraisal results shall be provided to the stakeholders identified in the appraisal plan The appraisal record shall be delivered to the appraisal sponsor The appraisal data package shall be submitted to the CMMI Steward Appraisal Requirements for CMMI (ARC) v1.2 (CMU/SEI-2006-TR-011), 2006 The following items must be submitted to the CMMI steward: Appraisal plan, including the appraisal input Appraisal findings – strength and/or weakness statements Appraisal Disclosure Statement (ADS) Record of Entry 10 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum Phase 3 Requirements – Report Results SAS

11. 11 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum The Types of Evidence used in SCAMPI Appraisals Process Implementation Indicators (Evidence) Description and Examples Direct ArtifactThe tangible outputs associated with implementation of a CMMI practice. Example: COCOMO output data provided by a project participating in the appraisal Indirect ArtifactEvidence that is a consequence of implementing a practice, but not the primary purpose of the practice. Example: Minutes of project planning meeting discussing the use of COCOMO. AffirmationsOral/written evidence confirming/supporting implementation of a practice (or lack of implementation) typically obtained by interviews or presentations and provided usually by the person performing the work. Example: During a presentation, the project manager states that “he/she was responsible for implementing COCOMO on the project”.

12. 12 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum Rules for Types of Evidence used in SCAMPI Appraisals Process Implementation Indicators (Evidence) Data Coverage and Sufficiency Class AEach practice must have at least one direct artifact and indirect/affirmation. For each goal, at least 50% of the cells in the participating projects must have at least one affirmation. Class BEach practice must have at least direct artifact and there must be at least one affirmation from the set of projects participating in the appraisal for each practice. Class CEach practice must have at least one direct or indirect or affirmation as evidence. ReferencesAppraisal Requirements for CMMI (ARC) v1.2 (CMU/SEI-2006-TR-011), 2006 Standard CMMI Appraisal Method for Process Improvement Method Definition Document (MDD) v1.2 (CMU/SEI-2006-HB-002), 2006 Handbook for Conducting Standard CMMI Appraisal Method for Process Improvement (SCAMPI) B and C Appraisals (CMU/SEI-2005-HB-005), 2005

13. 13 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum Rules for Corroboration of Evidence SCAMPI ClassCorroboration Requirement Class A and BThe objective evidence is obtained from at least two different sources. At least one of the two sources must reflect work actually being done (e.g. process area implementation) Class CNot required ReferencesAppraisal Requirements for CMMI (ARC) v1.2 (CMU/SEI-2006-TR-011), 2006 Standard CMMI Appraisal Method for Process Improvement Method Definition Document (MDD) v1.2 (CMU/SEI-2006-HB-002), 2006 Handbook for Conducting Standard CMMI Appraisal Method for Process Improvement (SCAMPI) B and C Appraisals (CMU/SEI-2005-HB-005), 2005

14. 14 Competency Cluster COCOMO Relevant MDD Knowledge Area (KA) Competency Cluster COCOMO Relevant MDD Knowledge Area (KA) 1. Achieving & Managing Agreements KA 3 Managing Objective Evidence6. Understanding & Adapting to Organizational Context* KA 1 Appraisal Planning KA 4 Judging Processes 2. Decision Making and Problem Solving KA 2 Preparing Team & Participants KA 3 Managing Objective Evidence 7. Model Interpretation* KA 1 Appraisal Planning KA 2 Preparing Team & Participants KA 3 Managing Objective Evidence KA 4 Judging Processes KA 5 Reporting Outcomes 3. Project Planning & Management* KA 2 Preparing Team & Participants KA 3 Managing Objective Evidence 8. SCAMPI Method Tailoring, Adaptation* KA 3 Managing Objective Evidence KA 4 Judging Processes 4. Interpersonal Communication & facilitation KA 3 Managing Objective Evidence KA 4 Judging Processes 9. ProfessionalismKA 4 Judging Processes 5. Integration, Articulation, and Expression of Information* KA 3 Managing Objective Evidence KA 4 Judging Processes * Higher Maturity addressed in these Competency Clusters 14 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum SCAMPI Lead Appraiser SM Body of Knowledge (SLA BOK) Competency Cluster - MDD Knowledge Area (KA) Mapping Appraisal team Lead appraiser

15. 15 Project Management Process Areas Including IPPD Category Requirements Management Requirements Development Technical Solution Product Integration Verification Validation Engineering Configuration Management Process and Product Quality Assurance Measurement and Analysis Causal Analysis and Resolution Decision Analysis and Resolution Support Project Planning Project Monitoring and Control Supplier Agreement Management Integrated Project Management + IPPD Risk Management Quantitative Project Management Organizational Process Focus Organizational Process Definition + IPPD Organizational Training Organizational Process Performance Organizational Innovation and Deployment Process Management CMMI v1.2-DEV: Continuous and Staged Representations Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum

16. 16 Overview: CMMI-DEV v1.2 Process Areas (PAs) and COCOMO-Based Estimation Processes Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum COCOMO Model Estimating Process Generic Goals (Institutionalization ) Process Management PAs Support PAs Project Management PAs Engineering PAs Organizational Process Focus Organizational Process Definition Organizational Training Organizational Process Performance Organizational Innovation and Deployment Project Planning Project Monitoring and Control Supplier Agreement Management Integrated Project Management Risk Management Quantitative Project Management Configuration Management Process and Product Quality Assurance Measurement and Analysis Causal Analysis and Resolution Decision Analysis and Resolution Requirements Management Requirements Development Technical Solution Product Integration Verification Validation Appraisal team verifying COCOMO-based estimation evidence Organization planning improvement of estimation process

17. 17 Process AreasPhase 1: Appraisal PlanningPhase 2: Conduct Appraisal (Evidence) OPD, OPF, OT Verbal Affirmations (interviews & presentations) Project managers, process engineers, higher level managers, COCOMO experienced staff and trainers Sufficient time and resources for institutionalization of organizational and defined project estimation processes, including tailoring criteria and any staff COCOMO training Historical sufficiency of COCOMO- based estimation measurements for relevant projects Identification of business goals for improving project estimation Improvement estimates for estimation A “thread” of planned defined estimation process and implementation, including estimation data, completed project plans, COCOMO outputs, thresholds for tracking project completion, analysis of “significant variances” Improvement information of estimation process, such as lessons learned, measurements, action plans,.etc. Peer reviews of estimation process QA audits of estimation process Relevant appraisal findings Training plan, record of staff/project managers responsible for COCOMO; measures of effectiveness Examples of Contributing Factors in SCAMPI Appraisal Phases 1 and 2 for Process Management Process Areas (Capability/Maturity Levels 1 - 3) Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum

18. 18 Process AreasPhase 1: Appraisal PlanningPhase 2: Conduct Appraisal (Evidence) CAR, OID, OPP, QPM Verbal Affirmations (interviews & presentations): Project managers, process engineers, higher level managers, COCOMO experienced staff Business goals linked to estimation process performance model Historical sufficiency of COCOMO-based estimation measurements Sufficient time for statistical stabilization of estimation process Candidate realistic (process and innovative) improvements in estimation process linked to documented business goals Adequate resources to develop process performance models to improve project estimation A “thread” of projects using estimation process and implementation of process model, defined process/subprocess including ranges for tracking project completion, analysis of special and common cause variations Improvement information of estimation process, such as lessons learned, measurements, action plans, etc. Peer reviews of process performance model Causal analysis reports of defects/improvements in current COCOMO- based estimation process Action and deployment plans and results of improvements Examples of Contributing Factors in SCAMPI Appraisal Phases 1 and 2 for Process, Project Management and Support Process Areas (Higher Capability/Maturity Levels) Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum

19. 19 Process AreasPhase 1: Appraisal PlanningPhase 2: Conduct Appraisal (Evidence) IPM, PP, PMC, RSKM, SAM Verbal Affirmations (interviews & presentations) Project managers, process engineers, higher level managers, COCOMO experienced staff Sufficient time and resources for institutionalization of organizational and defined project estimation processes, including tailoring plan and any staff COCOMO training Historical sufficiency of COCOMO- based estimation measurements Appropriate risk management strategy A “thread” of planned defined estimation process and implementation, including estimation data, completed project plans, COCOMO outputs, thresholds for tracking project completion, analysis of “significant variances” Tracking and completion of project action items, as appropriate Appropriate risk identification and management of projects Improvement information of estimation process, such as lessons learned, measurements, completed action plans Peer reviews & QA audits of estimation process Training plan and record of staff/project managers responsible for COCOMO Examples of Contributing Factors in SCAMPI Appraisal Phases 1 and 2 for Project Management Process Areas (Capability/Maturity Levels 1 - 3) Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum

20. 20 Process AreasPhase 1: Appraisal PlanningPhase 2: Conduct Appraisal (Evidence) RD, REQM PI, TS, VAL, VER CM, DAR, MA, PPQA Verbal Affirmations (interviews & presentations) Project managers, process engineers, higher level managers, COCOMO experienced staff Sufficient time and resources for institutionalization of organizational and defined project estimation processes, including any staff COCOMO training Historical sufficiency of COCOMO-based estimation measurements Implementation of life cycle models Selection guidance for project complexity values A “thread” of planned defined estimation process and implementation, including estimation data, completed project plans, complexity measures, COCOMO outputs, thresholds for tracking project completion, analysis of “significant variances” Tracking and completion of project action items, as appropriate,in product life cycle phases Relevant project planning and tracking measurements for life cycle phases Relevant implementations of decision analysis process, if appropriate Peer reviews of estimation process QA audits of estimation process Revision histories of COCOMO outputs Examples of Contributing Factors for SCAMPI Appraisal Phases 1 and 2 for Engineering and Support Process Areas (Capability/Maturity Levels 1 - 3) Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum

21. 21 Generic GoalPhases 1 and 2: Plan and Conduct Appraisal GG 1: Achieve Specific Goals GG 2: Institutionalize a Managed Process GG 3: Institutionalize a Defined Process GG 4: Institutionalize a Quantitatively Managed Process GG 5: Institutionalize an Optimizing Process COCOMO-related evidence for each PA in model scope And COCOMO-related evidence for PP, PMC,CM, MA, PPQA, OT* IPM* And COCOMO-related evidence for IPM, OPD, OPF PAs And COCOMO-related evidence for OPP, QPM PAs And COCOMO-related evidence for CAR, OID, PAs Contributing Factors for SCAMPI Appraisal Phases 1 and 2 for Institutionalization (Generic Goals (GG) 1 – 5) Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum *If PA is in the model scope of appraisal

22. 22 SUMMARY Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum

23. The End 23 Transdyne Corporation http://transdynecorp.com COCOMO 2008 Forum