Proposed Way Forward for SERC EM Task Barry Boehm, USC-CSSE 30 January 2009

Proposed Way Forward Iterate coverage matrix Add Personnel Competency EM Revise, circulate survey; analyze results – Follow up with experience interviews Revise EM evaluation template, use to evaluate EMs – Perhaps using Coverage Matrix categories Also evaluate EMs vs. Systemic Analysis Database Develop OpCon(s) for using EM instrument(s) Determine preferred candidate EM combinations for OpCons – Possible result: Artifact-monitoring; evidence taxonomies; specialties (teaming, weapons platforms) Prepare EM instruments for pilot use – Possible result: Refined INCOSE LIs; Refined Macro Risk Tool 01/30/20092

Iterate Coverage Matrix Originators review current ratings Expand rating scale – ** - Strong – * - Adequate – o - Partial – - Missing Reorganize categories – Try for 5x5 – 7x7 range 01/30/20093

Candidate Personnel Competency EMs INCOSE Systems Engineering Certification Framework GsWERC Body of Knowledge SysE portions Nidiffer source 01/30/20094

GsWERC BoK - Requirements 01/30/20095

EM Usage OpCons Planning review vs. execution monitoring Life cycle: Greenfield, Brownfield, O&M Domain: Weapons platform, System of systems, Net- centric services EM type: Artifact-monitoring; evidence taxonomies; specialties (teaming, weapons platforms) 01/30/20096

1/27/04©USC-CSE7 Trilateral Working Group Addressing Early Warning Indicators US, UK, and Australian participants Mix of traditional and emerging indicators Current version a work in progress; seeking community feedback Organized around program goals and critical success factors Focused on readiness for milestone decision reviews

1/27/04©USC-CSE8 Software Acquisition Goals Goal 1. System and software objectives and constraints have been adequately defined and validated. Goal 2. The system and software acquisition strategies are appropriate and compatible. Goal 3. The success-critical stakeholders have committed adequate software capability to perform their software- related tasks. Goal 4. The software product plans and process plans are feasible and compatible. A Feasibility Rationale provides convincing evidence that: Goal 5. Software progress with respect to plans is satisfactory.

1/27/04©USC-CSE9 Critical Success Factors: Goal 1 Goal 1. System and software objectives and constraints have been adequately defined and validated. 1.1System and software functionality and performance objectives have been defined and prioritized. 1.2 The system boundary, operational environment, and system and software interface objectives have been defined. 1.3System and software flexibility and evolvability objectives have been defined and prioritized. 1.4System and software environmental, resource, infrastructure, and policy constraints have been defined. 1.5System and software objectives have been validated for overall achievability within the system and software constraints.

Proposed Way Forward Iterate coverage matrix Add Personnel Competency EM Revise, circulate survey; analyze results – Follow up with experience interviews Revise EM evaluation template, use to evaluate EMs – Perhaps using Coverage Matrix categories Also evaluate EMs vs. Systemic Analysis Database Develop OpCon(s) for using EM instrument(s) Determine preferred candidate EM combinations for OpCons – Possible result: Artifact-monitoring; evidence taxonomies; specialties (teaming, weapons platforms) Prepare EM instruments for pilot use – Possible result: Refined INCOSE LIs; Refined Macro Risk Tool 01/30/200910

Macro Risk Tool Demo 01/30/200911

6/18/08©USC-CSSE12 Macro Risk Model Interface 01/30/200912