Perspectives on Transforming DT and OT Industry-Government Roundtable Mr. Rick Lockhart Deputy Director, DT&E OUSD(AT&L) March 3, 2004

Roundtable Rick Lockhart Pete Adolph Brian Simmons Mike Toole

Defense Systems Acquisition World Today Joint Capabilities (Jt. Cap. Integration & Development System, 3170.01C) Assess as-is capability to contribute to future joint concepts Address gaps with appropriate solutions Evolutionary Acquisition (5000) Usable capability to warfighter – may lack full functionality Increments and spirals Emphasis on systems engineering Systems More complex Net-centric functionality PMs Deliver capability Within schedule & cost Fix issues in next spiral S-o-S Based on integrated architectures Enable mission capabilities “Net-Ready” KPP Users – Warfighters Deployed, or training to deploy Need better capability ASAP, to counter asymmetric threat Scarce forces for OT

Today’s Test Model T&E Model Defense Acquisition Management Framework IOC B A Technology Development System Development & Demonstration Production & Deployment Systems Acquisition Operations & Support C Sustainment FRP Decision Review FOC Design Readiness Pre - (Program Initiation) Concept Refinement Defense Acquisition Management Framework IOT&E LRIP BLRIP Rpt T&E Model Contractor DT&E OT&E Risk Reduction ACTDs Analysis Demo’s & Exp’s Technology Maturation TEMP DT OT FOTE OA TES Government

T&E Needs Transforming to Fit Today’s Acquisition World Areas to explore: IOT&E DT Integration with OT Mission Capability Testing JDEP M&S

Can We Make IOT&E Concept More Useful? IOC B System Development & Demonstration Production & Deployment Systems Acquisition C Sustainment FRP Decision Review Design Readiness (Program Initiation) IOT&E LRIP Today IOT&E is now conducted after MS C and (often) multiple LRIPs. Is pass-fail event after MS C the best contribution? What type of OT or Evaluation needed before MS C? Do we need an independent test for an independent evaluation? Does a single pass-fail event make sense with today’s spiral development? IOT&E – as structured today, does it still make sense? Multiple LRIPs...what decision does IOT&E support? Forces are scarce IOT&E is too late to impact design; adds cost; delays capability fielding Why not establish an independent evaluation, prior to MS C, based on all testing to date. Use more combined DT - OT May not need an independent OT

Can DT Community Better Integrate with OT&E? More operationally realistic DT can offset need for subsequent OT Can DT include more realistic environments? User representative operators? Introduce simulated operational conditions earlier? Connect S-o-S for sub-system level testing? Target DT objectives in OAs and FOT&Es? Is independent OT required prior to capability fielding? Joint Testing – how to test a system for joint mission capabilities? Testing is system centric (including TEMP), in a net centric S-o-S world Early joint testing is essential to cost-effective systems engineering PM does not control access to other systems Integration costs are significant S-o-S engineering is a new art

How to Support S-o-S Mission Capability Decisions? Acquisition programs are stove-pipe funded by program Test data is system-centric (except for interoperability, which is largely unknown until MS C) Should we focus on joint mission capability testing, prior to milestone decisions? But, PM does not control other S-o-S programs And, PM is not funded for S-o-S testing System acquisition stovepipes Acquisition is system centric, in a net centric world PMs lack control over other systems Many different standards, terms, processes, definitions exist – complicating developing joint capabilities Proprietary industry solutions

How to Mature JDEP? (Joint Distributed Engineering Plant) Shown to be an effective, value-added capability for A&MD Could be effective tool for joint S-o-S T&E across other mission areas Need long-term strategic plan to expand user base to other mission areas

How to Better Leverage M&S in Acquisition? Essential to assess early the joint mission capability of individual systems M&S can facilitate collaborative S-o-S engineering and test But, PM doesn’t control other system M&S OSD examining approaches to enable collaborative S-o-S engineering processes with M&S M&S – how to leverage for best advantage? JDEP has demonstrated value to A&MD – how to extend to other users? PMs resources limited for M&S development Different Service approaches Lack top-level M&S infrastructure program for acquisition programs

Recommendation An independent study on Transforming T&E Defense Science Board, National Academy of Science, etc Consider today’s T&E practices in the context of today’s defense systems acquisition world A fresh outsider look at processes, tools, policies that could benefit defense systems acquisition & test