1. 4 March 2004
A Framework for OT&E Transformation NDIA 20th Annual Test and Evaluation Conference
Dr. William G. Lese
Vice President
Simulation, Analysis & Training Systems Northrop Grumman Information Technology

2. Transformation is Underway OT&E Transformation Complexities
Discussion Topics
Transformation is Underway
OT&E Transformation Complexities
OT&E Transformation Framework
Conclusions

3. Transformation is Underway
“We must transform not only the capabilities at our disposal, but also the way we think, the way we train, the way we exercise, and the way we fight.” – Secretary of Defense Rumsfeld
Jointness
Office of Force Transformation Established
Services Transformation Plans
Training Transformation (e.g. JNTC)
Defense Readiness Reporting System (DRRS)
Relating readiness to mission metrics
Joint Experimentation
Knowledge Management
OT&E Transformation is Inevitable

4. OT&E Transformation Complexities
Joint OT&E
Concept Definition
Facilities
People
Process (Sequential vs. Iterative)
Test Objectives
Measures and metrics
Data collection
Data Analysis
Cost
Responsibility
OT&E Ability to Address Adaptability and Agility
Moving away from Pass-Fail Mentality

5. OT&E Transformation Complexities
Missing the Mission Focus – Not technology focus
Size of test resources not sufficient for system of systems test
Training Transformation Relationship
Stronger link between joint experimentation and OT&E
MEM (Model-Experiment-Model)
Mission Metrics
Joint Context
Service and Joint OT&E Relationship
Facilities, resources, connectivity
Culture – Knowledge Management
“In view of the challenges we face now and in the future, I have decided to change how we develop and execute programs to ensure that our programs serve joint needs and effectively balance current and future risks.” Sec Def. Oct 2003 Memo for Mil Dept Secretaries

6. FRAMEWORK FOR TRANSFORMATION-BASED TESTING
DOD Acquisition Cycle
MNS
C&T Dev
System Dev and Demo
Production and Deployment
Support A B C
FRP
CONCEPTS
LRIP ITEM
EDM
FRP ITEM
EOA
OA (1&2)
DEM-VAL
USER RQMT
ASSESSMENT
JOINT OPERATIONAL
ANALYSIS I N D U S T R Y T E S
IOT&E M
& S
Continuous Testing, Improving, and Learning Spiral
FOT&E
DEM-VAL
USER RQMT
ASSESSMENT
JOINT OPERATIONAL
ANALYSIS I N D U S T R Y
TRANSFORMATIONAL CHANGE (DOTMLPF)
JT Staff
KM GRID
Services
Academia
Operational Requirements
Collaborative
Environment
JNTC
Industry
COCOMs
Internet/Intranet
SIPRNet
Identify & Link SMEs
DT & OT
Processes
Test and Evaluation Enablers
Training
Data Mart
Data Archives
Applications
Vectoring Information
Smart
Push-Pull
Network Agents
& Search Engines
Experimentation
M & S
Virtual Collaborative Teams and Communities of Practice

7. Key Features of the Framework
KM Impact on OTE - COPs
Access to experts – databases
Portals/ web based/connectivity
Link to JNTC
Open architecture
Testing is JTF mission based
Link to experimentation
Testing supported by M&S
Testing is spiral and continuous
Feedback to requirements

8. The Framework is a Roadmap for Successfully Transforming OT&E
Joint OT&E
Adaptability and agility
Pass-Fail Mentality
Mission focus
Training Transformation Relationship
Joint Experimentation Link
Service and Joint OT&E Relationship
Culture – Knowledge Management

9. Conclusions OT&E transformation is inevitable
KM processes and procedures play a critical role in addressing OT&E transformation complexities
Proposed framework is a roadmap for successfully transforming OT&E