Use of Tailored PIIs November 17, 2003 The Use of Tailored Practice Implementation Indicators for Process Definition and Assessment Preparation Presented to National Defense Industrial Association 3rd Annual CMMI Technology Conference and User Group Denver, Colorado Ben Berauer, Raytheon November 17-20, 2003 Ben Berauer, Raytheon

Quick Review of SCAMPI PIIs Use of Tailored PIIs November 17, 2003 Quick Review of SCAMPI PIIs SCAMPI V1.1 Method Definition Document describes the process of verifying practice implementation and the role of something called Practice Implementation Indicators (PIIs) PIIs are: Direct artifacts, the tangible outputs resulting directly from implementation of a specific or generic practice Indirect artifacts, that are a consequence of performing a specific or generic practice or that substantiate its implementation, but which are not the purpose for which the practice is performed Affirmations, oral or written statements confirming or supporting implementation of a specific or generic practice The SCAMPI is a verification focused method, and organizations must have well documented evidence of direct and indirect artifacts Affirmations, though important and required, are largely corroboration CMMI typical work products were never intended to specify the direct work products that the SCAMPI now expects, so additional guidance is needed. TWPs are today incomplete, and a mixture of direct and indirect 12/4/2018 Ben Berauer, Raytheon

Practice Implementation Indicator Descriptions and Guidance Use of Tailored PIIs November 17, 2003 Practice Implementation Indicator Descriptions and Guidance Organizations now must provide as input to the appraisal a PII database that can be used to verify practice implementation, and this is often in the form of PII Descriptions (PIIDs) SCAMPI requires projects to document their PIIs Appraisals are preceded by at least one readiness review that focuses on whether the documented evidence is available, and a verification based appraisal can be accomplished A set of generic expected PIIDs were developed as a part of the effort to develop the SCAMPI V1.1 Clearer description of the typically expected direct and indirect documented artifacts characterizing each practice Generic and not tailored for any specific company or business need Did not get published as part of the method or incorporated into the CMMI model typical work products CMMI typical work products were never intended to specify the direct work products that the SCAMPI now expects, so additional guidance is needed. TWPs are today incomplete, and a mixture of direct and indirect. CMMI Objective Evidence, Rev A 12/4/2018 Ben Berauer, Raytheon

Practice Implementation Indicator Description (PIID) Use of Tailored PIIs November 17, 2003 Practice Implementation Indicator Description (PIID)     The SCAMPI PIID definitions are not part of the model or Method 12/4/2018 Ben Berauer, Raytheon

Practice Implementation Indicator Description (PIID) Use of Tailored PIIs November 17, 2003 Practice Implementation Indicator Description (PIID)     The SCAMPI PIID definitions are not part of the model or Method 12/4/2018 Ben Berauer, Raytheon

Use of Org’s Standard Process & PIIDs Use of Tailored PIIs November 17, 2003 Use of Org’s Standard Process & PIIDs Org’s standard processes may define customary and expected work products in site specific and precise terms, interpreting the direct (and some indirect) work products and artifacts produced. Project tailoring of the org’s standard processes may entail tailoring and describing the project specific work products The project’s PIIs point to specific work products that the project has or will produce and place in a PII database PIIs may be handled in various ways Org may support a standard project PII repository database Org may have a PAL that accommodates project data Projects may use a standard PIIDs template/tool PIIs may be incorporated in the OU’s process description Directive System may include process descriptions that include Expected Work Products. i.e. The standard directive for Project Planning may include an element on project estimating that specifies that a System Breakdown Structure, Organizational Structure Description, Work Breakdown Structure and WBS Dictionary are expected work products that will be reviewed. Process Description may embody an ETVX form and specify the expected work products during and at exit from each activity. The directive would clearly define the customary, or expected, standard work products. Tailoring guidelines may be established 12/4/2018 Ben Berauer, Raytheon

SCAMPI V1.1 Method Process Flow Use of Tailored PIIs November 17, 2003 SCAMPI V1.1 Method Process Flow From SCAMPI V1.1 Method Lead Appraiser engaged in process step 1.1 The first phase in the SCAMPI method focus on planning and preparing for the appraisal The lead appraiser is engaged early-on in step 1.1 The next phase of the appraisal is the actual “onsite” conduct of the appraisal 1 Plan and Prepare for Appraisal 1.1 Analyze Requirements Understand the business needs of the organizational unit for which the appraisal is being requested. The appraisal team leader will collect information and help the appraisal sponsor match appraisal objectives with their business objectives. 1.1.1 Determine Appraisal Objectives 1.1.2 Determine Appraisal Constraints 1.1.3 Determine Appraisal Scope 1.1.4 Determine Outputs 1.1.5 Obtain Commitment to Appraisal Input 1.2 Develop Appraisal Plan Document requirements, agreements, estimates, risks, method tailoring, and practice considerations (e.g., schedules, logistics, and contextual information about the organization) associated with the appraisal. Obtain, record, and make visible the sponsor’s approval of the appraisal plan. 1.2.1 Tailor Method 1.2.2 Identify Needed Resources 1.2.3 Determine Cost and Schedule 1.2.4 Plan and Manage Logistics 1.2.5 Document and Manage Risks 1.2.6 Obtain Commitment to Appraisal Plan 1.3 Select and Prepare Team Ensure that an experienced, trained, appropriately qualified team is available and prepared to execute the appraisal process. 1.3.1 Identify Team Leader 1.3.2 Select Team Members 1.3.3 Prepare Team 1.4 Obtain and Analyze Initial Objective Evidence Obtain information that facilitates site-specific preparation. Obtain data on model practices used. Identify potential issue areas, gaps, or risks to aid in refining the plan. Get preliminary understanding of the organizational unit’s operations and processes. 1.4.1 Prepare Participants 1.4.2 Administer Instruments 1.4.3 Obtain Initial Objective Evidence 1.4.4 Inventory Objective Evidence 1.5 Prepare for Collection of Objective Evidence Plan and document specific data collection strategies including sources of data, tools and technologies to be used, and contingencies to manage risk of insufficient data. 1.5.1 Perform Readiness Review 1.5.2 Prepare Data Collection Plan 1.5.3 Replan Data Collection (if needed) 12/4/2018 Ben Berauer, Raytheon

PIIDs for Appraisal Preparation Use of Tailored PIIs November 17, 2003 PIIDs for Appraisal Preparation Determining if an project’s PIIs are appropriate and adequate is left up to the CMMI appraisers But developing appropriate PIIDs are key to an organizational understanding of: The model’s intent How to document compliance with the model in preparation for an appraisal PIIDs are central in how we improve processes They should be incorporated in a site’s process descriptions They should be central to data collection and documentation practices The criteria for selecting the 'right' objective evidence include identifying appropriate 'Direct' and 'Indirect' artifacts from each focus project Collection and documenting PIIs should follow a disciplined data collection and scorecarding process … for Process Definition and Assessment Preparation There are different approaches, but one will be recommended The approach to developing PIIs should closely parallel the Appraisal Method 12/4/2018 Ben Berauer, Raytheon

Pre-Appraisal Scorecarding Use of Tailored PIIs November 17, 2003 Pre-Appraisal Scorecarding Scope & Prepare for Scorecarding SCORECARDING GUIDELINES Team Preparation SCORECARD EVIDENCE GUIDE Obtain & Analyze Objective Evidence - Indentify PIIDs - Collect PIIs - Review & Characterize PIIs Scope and prepare for scorecarding Team preparations A team of evidence collectors familiar with the project’s defined process and typical work products is assembled Process SMEs familiar with the Org’s Standard Processes and CMMI model and method are available for mentoring and verification of evidence. All team participants review and agree to the evidence collection, orgnanizational standard PIIDs, and scorecarding process. Identify appropriate PIIs for instantiations within scope of effort Collect PIIs for instantiations within scope of effort Review and characterize PIIs collected, identifying appropriate artifacts and defects, and observations Develop findings for organizational process implementation Characterize CMMI practices, goals, PAs Scorecard instantiations against the CMMI model Develop appropriate actions plans Implement actions plans to correct deficiencies Develop Action Plans Implement Action Plans 12/4/2018 Ben Berauer, Raytheon

Value Stream Current Objective Evidence Gathering method Get What You Think is the Right OE, Gathering Everything, Hope it is right Review it Try Again Review it Again Try Again Hope the SCAMPI Appraisers are nice to you Improved Objective Evidence Gathering method Use OE Expert Guidelines to Determine Appropriate OE Gather Correct OE, Review with CMMI SMEs SCAMPI Appraisers Quickly review OE 12/4/2018

Organizational Process Enablers Use of Tailored PIIs November 17, 2003 Organizational Process Enablers CMMI SCORECARDING GUIDELINES Scorecarding Effort Preparation Scorecarding Team Preparation Review of Project Artifacts Characterization of the Practices Rating of Goals Rating the Process Area CMMI SCORECARD EVIDENCE GUIDE Tailored PIID defining organizational work product expectations for each process in its Organizational Assets May be a standalone document or integrated into org’s standard process descriptions supports PII data gatherers, auditors of the processes, and the internal EPG or other process entity tasked with scorecarding activities Main take-away message (copy and paste) 12/4/2018 Ben Berauer, Raytheon

Internal PIID, Scorecarding Tool Use of Tailored PIIs November 17, 2003 Internal PIID, Scorecarding Tool 12/4/2018 Ben Berauer, Raytheon

Use of Tailored PIIs November 17, 2003 In Summary The CMMI appraisal method is PII-based, and organizations must develop tailored PIIDs One of the most difficult and costly components of preparing for a SCAMPI is collecting the 'right' objective evidence The approach to developing PIIs should closely parallel the Appraisal Method, and begin long before a SCAMPI It is difficult to do without a clear and detailed understanding about the model and appraisal method Experts in the model and method need to work hand-in-hand with organizational and project personnel to prepare in an efficient and cost effective manner Different approaches to PIIDs and preparing for appraisals may be taken An approach emphasizing PII collection and scorecarding using experience in the SCAMPI method is key Different approaches to PIIDs and preparing for appraisals are taken. The approach depends on many factors, such as initial maturity, time line goal, and internal expertise Scorecarding Class C and B appraisals (prepare and readiness assessment) 12/4/2018 Ben Berauer, Raytheon