1. Perspectives on Transforming DT and OT Industry-Government Roundtable
Mr. Rick Lockhart
Deputy Director, DT&E
OUSD(AT&L)
March 3, 2004

2. Roundtable
Rick Lockhart
Pete Adolph
Brian Simmons
Mike Toole

3. Defense Systems Acquisition World Today
Joint Capabilities (Jt. Cap. Integration & Development System, C)
Assess as-is capability to contribute to future joint concepts
Address gaps with appropriate solutions
Evolutionary Acquisition (5000)
Usable capability to warfighter – may lack full functionality
Increments and spirals Emphasis on systems engineering
Systems
More complex
Net-centric functionality
PMs
Deliver capability
Within schedule & cost
Fix issues in next spiral
S-o-S
Based on integrated architectures
Enable mission capabilities
“Net-Ready” KPP
Users – Warfighters
Deployed, or training to deploy
Need better capability ASAP, to
counter asymmetric threat
Scarce forces for OT

4. Today’s Test Model T&E Model Defense Acquisition Management Framework
IOC B A
Technology
Development
System Development
& Demonstration
Production &
Deployment
Systems Acquisition
Operations &
Support C
Sustainment
FRP
Decision
Review
FOC
Design
Readiness
Pre -
(Program
Initiation)
Concept
Refinement
Defense Acquisition Management Framework
IOT&E
LRIP
BLRIP
Rpt
T&E Model
Contractor
DT&E
OT&E
Risk Reduction
ACTDs Analysis
Demo’s & Exp’s
Technology Maturation
TEMP DT OT
FOTE OA
TES
Government

5. T&E Needs Transforming to Fit Today’s Acquisition World
Areas to explore:
IOT&E
DT Integration with OT
Mission Capability Testing
JDEP
M&S

6. Can We Make IOT&E Concept More Useful?
IOC B
System Development
& Demonstration
Production &
Deployment
Systems Acquisition C
Sustainment
FRP
Decision
Review
Design
Readiness
(Program
Initiation)
IOT&E
LRIP
Today
IOT&E is now conducted after MS C and (often) multiple LRIPs.
Is pass-fail event after MS C the best contribution?
What type of OT or Evaluation needed before MS C?
Do we need an independent test for an independent evaluation?
Does a single pass-fail event make sense with today’s spiral development?
IOT&E – as structured today, does it still make sense?
Multiple LRIPs...what decision does IOT&E support?
Forces are scarce
IOT&E is too late to impact design; adds cost; delays capability fielding
Why not establish an independent evaluation, prior to MS C, based on all testing to date.
Use more combined DT - OT
May not need an independent OT

7. Can DT Community Better Integrate with OT&E?
More operationally realistic DT can offset need for subsequent OT
Can DT include more realistic environments?
User representative operators?
Introduce simulated operational conditions earlier?
Connect S-o-S for sub-system level testing?
Target DT objectives in OAs and FOT&Es?
Is independent OT required prior to capability fielding?
Joint Testing – how to test a system for joint mission capabilities?
Testing is system centric (including TEMP), in a net centric S-o-S world
Early joint testing is essential to cost-effective systems engineering
PM does not control access to other systems
Integration costs are significant
S-o-S engineering is a new art

8. How to Support S-o-S Mission Capability Decisions?
Acquisition programs are stove-pipe funded by program
Test data is system-centric (except for interoperability, which is largely unknown until MS C)
Should we focus on joint mission capability testing, prior to milestone decisions?
But, PM does not control other S-o-S programs
And, PM is not funded for S-o-S testing
System acquisition stovepipes
Acquisition is system centric, in a net centric world
PMs lack control over other systems
Many different standards, terms, processes, definitions exist – complicating developing joint capabilities
Proprietary industry solutions

9. How to Mature JDEP? (Joint Distributed Engineering Plant)
Shown to be an effective, value-added capability for A&MD
Could be effective tool for joint S-o-S T&E across other mission areas
Need long-term strategic plan to expand user base to other mission areas

10. How to Better Leverage M&S in Acquisition?
Essential to assess early the joint mission capability of individual systems
M&S can facilitate collaborative S-o-S engineering and test
But, PM doesn’t control other system M&S
OSD examining approaches to enable collaborative S-o-S engineering processes with M&S
M&S – how to leverage for best advantage?
JDEP has demonstrated value to A&MD – how to extend to other users?
PMs resources limited for M&S development
Different Service approaches
Lack top-level M&S infrastructure program for acquisition programs

11. Recommendation An independent study on Transforming T&E
Defense Science Board, National Academy of Science, etc
Consider today’s T&E practices in the context of today’s defense systems acquisition world
A fresh outsider look at processes, tools, policies that could benefit defense systems acquisition & test