1. UNCLASSIFIED NDIA CPM 01/13/10 Page-1 Streamlining Program Reviews Terry Jaggers Principle Director, Systems Engineering Office of the Director, Defense Research and Engineering NDIA Committee on Program Management 13 January 2010

2. UNCLASSIFIED NDIA CPM 01/13/10 Page-2 AF Problem 2 ½ Years Ago… OSD PSR (ACAT IDs) Space IPA (NSS 03-01) Late-to-Need SEPs OSD Independent TRAs Emerging MRAs PEO Sufficiency Reviews Emerging LHAs and ILAs Integration Readiness/D&SWS SEAM/CMMI Process Tools Other (MIRT, CAIG & SRA) SAE Review Board Mtg (IC/D) (Independent) (Integrated/Independent) THEN (Unsustainable, begin NRC study) Issues & Concenrs:  Specialty reviews proliferating at an alarming rate (TRAs, MRAs, IRAs, PSRs, IPAs, PMAG, D&SWS, SEPs, PESHEs, RMAs, etc.)  OSD PSR focused on assessing ACAT ID programs only  SAE’s AF Review Board lacked underpinning process to assess and prepare SAE like PSR did DAE  PEO sufficiency reviews not validated, transparent nor accepted by others  PMs burdened due to lack of synchronization to key program events and multiple review teams  TRAs and other specialty reviews out of context of systems engineering and integration assessment  Training courses being developed ad hoc with no system level view on “integrated” training & education to assess/review  AF missing golden opportunities to enhance, grow and train organic acquisition workforce

3. UNCLASSIFIED NDIA CPM 01/13/10 Page-3 AF Commissioned NRC Study Statement of Task  Review the program technical reviews and assessments that space and non-space system acquisition programs are required to undergo  Assess each review and the resources required to accomplish it.  Assess the role and contribution that each review and the combined reviews make to successful acquisition.  Identify cases where different reviews have shared, common, or overlapping goals, objectives, content, or requirements.  Identify and evaluate options for streamlining, tailoring, integrating, or consolidating reviews.  Recommend changes that the Air Force and DOD should make to the reviews.

4. UNCLASSIFIED NDIA CPM 01/13/10 Page-4 NRC Study Outbrief Study & Recommendations to be Briefed by Rand Fisher Here

5. UNCLASSIFIED NDIA CPM 01/13/10 Page-5 AF [Chief Engineer] Actions Conduct joint DDR&E-SAF/AQR PSRs on ACAT 1Ds and run pilot on SDB II Program Adopt/standardize OSD PSR methodology (DAPS) for use across AF (ACAT 1C, II and IIIs) Incorporate specialty technical reviews (TRAs, MRAs, LHAs, etc) under overarching AF PSR review process Added a “mentoring/help” phase at front end of AF PSR process (start when TRA starts) Added cost estimation (CAIG), PM center of excellence (schedules) and contracting (MIRT) POCs to AF PSR team 5

6. UNCLASSIFIED NDIA CPM 01/13/10 Page-6 Challenges & Opportunities for DoD Independent Milestone Reviews (PSRs, TRAs, CAIG, Space IPAs, etc) As-Required Program Assistance (Prog Assist, Peer Reviews, Space PMAG, etc) Continuous Execution Reviews (DAES, EVMS, POPs, Etc) DAES DABs CSBs SRCA  Synchronizing/coupling reviews for effectiveness and efficiencies (PM currently tries…)  Architecting the data capture for SRCA (will PARCA do this?)  Standardizing reviews between OSD and Services (AF attempting to synergize OSD-AF tech effort in overall program review initiatives)  Pushing reviews earlier per 5000 and WSARA (MDD and MS A voids)  Reviewing gaps / overlaps in data capture between DAPS, TRAs, POPs, EVMS, etc (Univ of Tenn started)

7. UNCLASSIFIED NDIA CPM 01/13/10 Page-7 Summary DDR&E synergizing independent technical reviews –Specialty reviews (IRA, TRA, MRA, etc) –Helpful/mentoring opportunities (early engagement at TRA start) –Colloaboration with Component reviews/SMEs –Pushing to the left for MS A and MDD prep More needs to be done to synergize other reviews –Functionals (cost, contracting, test, logistics, etc) –Type (program assist, execution, decision prep) –Data collection/disposition (SRCA and PARCA) Opportunities exist to “architect” reviews smartly –Reduce program office resources –Reduce review team resources –Optimize SRCA data and resources

8. UNCLASSIFIED NDIA CPM 01/13/10 Page-8 Backup

9. UNCLASSIFIED NDIA CPM 01/13/10 Page-9 ACB MDD O&S FRP DR PDRCDR ACAT 3 Development Development Planning AC CBA B ICD Technology Development Engineering and Manufacturing Development Production and Deployment Materiel Solution Analysis MDD CPDCDD O&S FRP DR PDRCDR Historical: majority of PSRs performed around EMD for ACAT 1 Current focus: Pushing to left Desire end state: (1) Extend review and analysis capability to Services (2) Get ACAT 2&3 participation (3) Push PSRs “to the left” because that’s where program problems are born (4) Link Competency Model to SRCA results ACB MDD O&S FRP DR PDRCDR ACAT 1 ACAT 2 Policy/Guidance Education & Training Best Practices Other Processes (JCIDS, etc) Oversight (DAB/ITAB) Competency Model AOA SRCA Results Show SE’s PSR & SRCA Vision