1. 4/22/2017
Test and Evaluation Working Integrated Product Team (T&E WIPT) An Integral partner on the PM’s Team
A T&E WIPT is an integral partner on the PM’s team for the successful execution of an acquisition program.
Services/Agencies may have different tiles for this group, such as: integrated test team (ITT), and combined test team (CTT), but the purpose is the same – Provide T&E expertise and support to the PM and PMO
It is the group of T&E subject matter experts from the PMO, sponsoring Service/Agency, T&E community, OSD, user stakeholders, and as appropriate, contractors.
This group will work the necessary T&E issues and actions necessary for a test and evaluation effort to support the program schedule and engineering effort and provide the necessary data to the PM and others as to product/system demonstrated performance and risks.
As of 8/17/09 1

2. 4/22/2017
Purpose
The purpose of this presentation is to provide PMs, and or program T&E leads, a tool to prepare and establish a T&E WIPT
This presentation may be used as is, or adjusted with Service/Agency direction and guidance regarding establishing a T&E WIPT
This presentation is meant to be used by whomever is establishing and kicking-off a T&E WIPT.
It should be modified as necessary to reflect the respective Service/Agency direction and guidance regarding T&E WIPTs.
Presentation contents are taken from the listed references and best practices. 2

3. Outline What is a T&E WIPT Why is a T&E WIPT Needed
4/22/2017
Outline
What is a T&E WIPT
Why is a T&E WIPT Needed
T&E WIPT Membership
T&E WIPT Charter
What are the T&E WIPT Organizational Relationships
What are the T&E WIPT Products
References
Point of Contact
There are eight specific areas in this presentation.
The presentation focus is on “WHY” a T&E WIPT is needed, and “WHAT” is its organization and products. 3 3

4. 4/22/2017
What is a T&E WIPT
The T&E WIPT is a defined forum serving the PM, and PMO, as the test and evaluation subject matter experts responsible for supporting the PM, and other program WIPTs, on all aspects of a program’s test and evaluation effort. This effort includes T&E program strategy, design, development, oversight, and the analysis, assessment and reporting of test results.
The T&E WIPT should be established and chartered as early as possible during the Material Solutions Analysis phase so that it is involved in program strategy discussions and plans.
The T&E WIPT is a specific functional focused and organized group of individuals deemed necessary and appropriate to address the T&E effort for an acquisition program.
They are to provide their respective expertise to the T&E effort and advise the PM.
The T&E WIPT should be established as early as possible so as to begin understanding the operational, acquisition, and engineering strategies which will influence the test and evaluation strategy.
Key T&E strategy framing discussions revolve around such documents as: the acquisition strategy, the system engineering plan, and the ICD. These documents and others will influence significant T&E planning considerations such as: the requirement for prototypes; test bed; incremental development, T&E, and fielding; block development T&E, and fielding; and if a pilot effort is required. 4

5. What is a T&E WIPT (Continued)
4/22/2017
What is a T&E WIPT (Continued)
The PM may form lower level functional working groups, who report to the T&E WIPT, to focus on specific areas such as integrated test planning, information assurance, SW T&E, reliability scoring, M&S development and Verification, Validation, and Accreditation (VV&A), and threat support.
The T&E WIPTs primary focus areas through a program’s acquisition process and phases are identified on the next chart
Sub-WIPTs may be established and are usually focused on a specific issue/topic, and upon task completion are dissolved.
A key WIPT member requirement is understanding the acquisition process, phases, and exit and entrance criteria, and specifically, the T&E role and responsibilities for each phase. 5 5

6. T&E WIPT Focus Areas in the Acquisition Process & Phases
4/22/2017
T&E WIPT Focus Areas in the Acquisition Process & Phases
TES
TEMP
TEMP
TEMP
CDD
CPD
ICD
T&E WIPT Focus Areas
Technology Development Strategy (TDS)
Test Evaluation Strategy (TES)
ID emerging T&E capability requirements
ID T&E resources
Develop T&E requirements in RFP
Annual Report
Test and Evaluation Master Plan (TEMP)
Execute T&E Program
Provide T&E results for OIPT/DAB
CDD requirements for testability and evaluation
TRL Evaluation
T&E requirements in RFP
Support SE Tech Review (PDR)
CPD requirements for testability and evaluation
Define system capabilities and limitations
Discovery and deficiencies
Test and Evaluation Master Plan
Verification of corrections for deficiencies
T&E results for OIPT/DAB
OTRR
IOT&E
Follow-on DT and OT
Develop T&E programs to support upgrades, modifications, increments
The five blocks list the major T&E items or areas for each of the five acquisition phases which a T&E WIPT must be involved.
The items listed are not the only items of T&E WIPT concern.
Items may be added depending on the nature of the product/system under development. For example, if the product/system is for multi-Service/Agency use there may have to MOAs/MOUs regarding integrated testing, evaluation, and assessments.

7. 4/22/2017
T&E WIPT Membership
WIPT Chair: PM, DPM, or Govt. T&E Lead. Must have T&E experience relative to the product/system
HQs/Sponsor/PEO representatives. Can speak for sponsoring Service/Agency
OSD DT&E & DOT&E representatives
Responsible and participating T&E organizations
Invited SMEs relative to the WIPT issue/topic
OTA(s) and other organizations involved in the program, such as JFCOM, especially if a SoS/FoS program
The T&E WIPT is chaired by a program office representative, and include representatives from the PMO, OT&A, operational users, the OSD staff, DoD Component staffs, the intelligence community, and other agencies (e.g., JFCOM) as necessary to assist in the T&E effort.
Other may be invited depending on the issue/topics; such as specifics on personnel safety requirements, environmental certifications
Functional representatives such as: NII, SW, M&S, IA, and JITC
Contractor / vendor as appropriate for type of discussions/decisions 7 7

8. Why is a T&E WIPT Needed A T&E WIPT is needed to:
4/22/2017
Why is a T&E WIPT Needed
A T&E WIPT is needed to:
Provide a forum for all key organizations to be involved in the T&E effort.
Develop and maintain the program T&E strategy.
Develop and manage an integrated test program for M&S, developmental tests, and operational tests to support evaluations.
Develop and maintain the Integrated Test Schedule.
Identify and resolve issues.
Document a quality TES/TEMP/Schedule as quickly and as efficiently as possible – necessitates that all interested parties are afforded an opportunity to contribute to the TES/TEMP development.
These are the six main reasons for a T&E WIPT. All f which focus on sound management practices.
Overall, the T&E WIPT provides a single forum for the PM, and PMO members, for all T&E issues. 8 8

9. T&E WIPT “Do’s” and “Don’ts”
4/22/2017
Do invite Service headquarters offices to participate in the T&E WIPT
Do obtain appropriate OSD office(s) involvement early, especially DT&E and DOT&E
Do know T&E WIPT issues & be empowered
Do hold program to the rigorous T&E process
Do base positions on facts, vice emotions
Do elevate an issue if unable to resolve within the T&E WIPT
Don’t dismiss T&E WIPT meetings as not important
Don’t dismiss T&E WIPT members’ comments without reasoned debate
Don't allow unresolved issues to fester within the T&E WIPT
(that is, take extra effort to resolve one-on-one and face-to-face)
Don’t disband the T&E WIPT after TEMP approval
This is a listing of essentially best practices regarding the T&E WIPT.
The six specific “Do’s” and the four specific “Don’ts” are key to a successful and productive T&E WIPT .
A T&E WIPT charter is necessary to capture the necessary roles, responsibilities, and products of the T&E WIPT.

10. 4/22/2017
T&E WIPT Charter
Getting WIPTs off to a good start is critical. A key activity is developing a charter.
The charter documents the WIPT mission, membership, roles and responsibilities, products, and establishes the timeframe in which the effort is to be completed.
The WIPT chairperson should draft the charter, be ratified by the membership at the outset of the WIPT, and approved by the program manager.
The first WIPT document is its charter.
Formats vary and depends on Service/Agency direction and guidance regarding content. A good overall guide on integrated product teams and listed as a reference is the guide – “Rules of the Road: A Guide for Leading Successful Integrated Product Teams.”
There are six required topics in a good charter – mission, membership, roles, responsibilities, products, and schedule.
At the first WIPT meeting the working draft charter should be available for discussion and coordination.
The next four charts go into detail on the composition of a complete T&E WIPT charter.

11. T&E WIPT Charter (Continued)
4/22/2017
T&E WIPT Charter (Continued)
A charter should:
contain a clear mission statement, to include the specific purpose and objectives of the WIPT;
provide recognition of the purpose of the WIPT in a larger context;
identify the output product, process, or service to be provided;
identify the customer or recipient of the output product, process, or service;
identify the timeframe by which the product is to be produced, the process completed, or the service provided;
identify WIPT membership, to include all the cross-functional disciplines necessary to achieve the objectives of the WIPT and to produce the product, complete the process, or provide the service;
consider any need for training of the WIPT membership, particularly those new to the WIPT process;
address membership performance objectives that characterize a high-performance WIPT;
address product ownership and membership accountability;
address the use of metrics as a means of creating and maintaining team focus;
Establishing a T&E WIPT may be a new learning process for members.
The charter must be developed in a deliberate manner with each member fully understanding what the WIPT is and expected to accomplish.
Some members may be focused on one aspect of product/system development, such as flight certification.

12. T&E WIPT Charter (Continued)
4/22/2017
T&E WIPT Charter (Continued)
A charter should (Continued):
provide for membership coordination and buy-in;
embody:
the three basic tenets of WIPTs:
The PM is in charge of the program.
WIPTs are advisory bodies to the PM.
Direct communication between the program office and all levels in the acquisition oversight and review process is expected as a means of exchanging information and building trust.
the roles and responsibilities applicable to all WIPTs:
Assist the PM in developing strategies and in program planning, as requested by the PM
Establish IPT plan of action and milestones (POA&M)
Propose tailored documentation and milestone requirements
Review and provide early input to documents
Coordinate WIPT activities with the OIPT members
Resolve or elevate issues in a timely manner
Assume responsibility to obtain principals’ concurrence on issues, as well as with applicable documents or portions of documents
The charter must have every member’s concurrence and understanding the importance of its contents for a successful WIPT and T&E program.
The top three understandings are:
PM is in charge – WIPT chair is his/her direct T&E advisor.
WIPT is advisory – SME collective body on best T&E effort.
WIPT must be able to coordinate and communicate with any PMO or other s regarding the T&E effort and plans – A successful T&E effort or plan can not be developed in a vacuum.

13. T&E WIPT Charter (Continued)
4/22/2017
T&E WIPT Charter (Continued)
Six operating principles for implementing success-oriented WIPTs:
CHARTERING, LAUNCH, INITIATION
Getting teams off to a good start is critical. Key activities include charter preparation, conducting WIPT training, and preparing a POA&M. When combined, these activities contribute to early and sustained success.
GOAL ALIGNMENT
The team leader should ensure the goals and objectives of team members are consistent with project goals and objectives. An effective mechanism to provide performance feedback to team members and their functional organization should be established.
OPEN DISCUSSIONS WITH NO SECRETS
Each member brings to the team unique expertise that needs to be recognized by all. Because of that expertise, each person’s views are important in developing a successful program, and these views need to be heard. Teams must have full and open discussions with no secrets. Cooperation is essential!
The operating principles for a successful WIPT are:
Chartering – PM authority defining WIPT who, what, when, and why.
Launch – Initial meeting to begin implementing WIPT effort and plans.
Initiation – Ensure members understand their individual and collective advisory roles and responsibilities.
Goal Alignment – Ensure member understanding of T&E relationship to program plan, system engineering plan, operational concept, integrated testing concept.
Open discussions – WIPT members have to feel they can be candid in their respective comments knowing that an open forum is how best to vet ideas, concepts, requirements, and especially, risks to effective T&E.
Accurate and complete meeting minutes are very important to capture and track actions, results, and decisions. WIPT chairperson must have someone designated to capture, coordinate, and track the minutes and action items.

14. T&E WIPT Charter (Continued)
4/22/2017
T&E WIPT Charter (Continued)
Six operating principles for implementing success-oriented WIPTs (Continued):
EMPOWERED, QUALIFIED TEAM MEMBERS
Empowerment is critical to making and keeping the agreements essential to effective WIPTs. All representatives assigned to WIPTs at all levels must be empowered by their leadership. They must be able to speak for their superiors, the “principals,” in the decision-making process.
DEDICATED/COMMITTED PROACTIVE PARTICIPATION
Consistent team participation by people with institutional knowledge of the functional areas is necessary for success. Therefore, WIPTs should be organized to allow all stakeholders to participate and right sized to the issue. Membership should be limited to the minimum essential to enhance communication and trust. Other organizations may be added as required based on the needs of the program. Contractor participation shall be in accordance with Service/Agency guidance.
ISSUES RAISED AND RESOLVED EARLY
Team members should openly raise and discuss issues at the earliest possible opportunity. The WIPT should try to resolve issues within the WIPT, seeking additional functional expertise when necessary. In the spirit of teaming and cooperation, issues should not be worked “off-line” beyond the purview of the IPT. When an issue cannot be resolved within the WIPT, the PM should raise the issue as quickly as possible to a decision-making level where resolution can be achieved.
The final three principals for a successful T&E WIPT are:
Empowered members – must be able to speak for his/her organization. Granted, there may be instances where organizational resources may be involved and the representative must obtain his/her superior concurrence.
Must be dedicated and actively engaged in the WIPT tasks and products – Must be at every WIPT meeting and participate in the discussions and decisions on recommendations to the PM
Be able to candidly raise issues and establish a resolution path – Establishing a T&E effort which will assess the product/system performance requires foresight and discussion on the verification and validity of the performance objectives, the metrics to be used, data collection process and tools, and the evaluation process. There maybe be risks to the attainment of the goals to each of these areas. The risks must be clearly identified, assessed as to impact, and eliminated or at least mitigated.

15. What are the T&E WIPT Organizational Relationships
4/22/2017 PM
T&E WIPT organizational relations are both internally to the PMO, and externally to all the stakeholders regarding the T&E effort, plans, resources, and results reporting
External relationships include organizations involved in product/system certifications, ranges and laboratories, special functional requirements such as environmental, personnel and system safety standards and requirements, and the user community
SE WIPT
T&E WIPT
Other WIPTs
Other USD
Other USD
SW T&E Sub-WIPT
IA T&E Sub-WIPT
Other Sub-WIPTs
Other USD
Other USD
The organizational relationships cut across all the PMO other WIPTs.
As a program progresses Sub-WIPTs may be established to address specific T&E issues such as identifying the T&E necessary, metrics, and timing for progressive SW T&E.
Sub-WIPTs usually disestablished once their tasks are accomplished.
Individual technical SMEs may be brought in to address a specific issue/topic.
External organizations may be needed to address issues relative to the other organization, such as the FAA or Homeland Security.
SME s
(as needed)
External Organizations (as needed)
as directed or needed 15

16. What are the T&E WIPT Products
4/22/2017
What are the T&E WIPT Products
There are three primary T&E planning documents – TES, TEMP, and the Integrated T&E Schedule
Test and Evaluation Strategy (TES) at
Milestone A:
For programs on the OSD T&E Oversight List, the TES shall be submitted to the OUSD(AT&L)DDR&E/DDT&E and the DOT&E for approval
TES describes the overall approach for evaluating system capabilities and addresses test resource planning
TES addresses Technology Development Phase evaluations, and the evaluation of system design concepts against the preliminary mission requirements resulting from the Analysis of Alternatives
TES includes the T&E aspects of competitive prototyping, and demonstration of technologies in relevant environments.
The Milestone A TES relies on the Initial Capabilities Document (ICD) as the basis for the evaluation strategy
The T&E WIPT is responsible for overseeing the development, coordination, approval process and implementation of the three basic T&E documents – the Test and Evaluation Strategy (TES), the Test and Evaluation Master Plan (TEMP), and the integrated test schedule.
All three are living documents in that the content changes as the program plan and acquisition strategy and engineering plan matures.
A main area of concern during TES development is the technical maturity of components comprising the respective product, sub-system, and or system. Demonstrated performance, and not marketing data, is a key requirement. 16

17. What are the T&E WIPT Products
4/22/2017
What are the T&E WIPT Products
Test and Evaluation Master Plan (TEMP) at Milestone B, C and Full-Rate Production:
MDAPs, MAIS Acquisition Programs, and programs on the OSD T&E Oversight List shall submit the initial TEMP to the OUSD(AT&L)DDR&E/DDT&E and the DOT&E for approval at MS B
TEMP describes the overall structure, major elements, and objectives for the T&E program
TEMP provides an integrated test schedule; and the resource requirements to accomplish the planned testing
TEMP documents the IOT&E entrance criteria, to be used to determine IOT&E readiness
The TES transitions into the TEMP prior to MS B
The TEMP is the foundation document for the T&E effort for the remainder of the acquisition process and is updated as needed to ensure the T&E events, activities, schedule, and funding remain in sync with the program plan and any approved changes in cost, schedule, and or product/system performance. 17 17

18. What are the T&E WIPT Products
4/22/2017
What are the T&E WIPT Products
Sample Integrated Test Schedule
Funding
RDTE
Production
Total
FOC
IOC
Engineering and Manufacturing Development
CDD
Technology Development
MSD/
Log Demo
Core Capability
ILA
TEMP
EMD
LRIP Lot 2
LRIP Lot 3
FRP Lot 4
LRIP Lot 1 /
IOTE support
Lot 2
x 9
Lot 3
x 14
Lot 5
x 24
L/Lead
Lot 7
Lot 6
FRP
LRIP
Lot 4
x 21 OA
IOT&E
AOTR / OTRR
BLRIP Report
IT2
IT3
LFTE Report
IOCSR x
PCR
SRR
SFR
PDR
CDR
FRR
PRR
HITL
SIL
Production & Deployment
LRIP / IOTE
Lot 9
x 10
Lot 8
IT1
FOTE
Flight Sim
Maint. Trainers
TDFA
OT Training
Initial Trng (T&E)
FRP MYP
ATO (Type Accreditation)
Phase II Verification and Certification Testing
Phase I Definition
Phase IV Post Accreditation
Phase III Validation / Cert Tests
Fiscal Year 94
Quarter 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12
MS B
Post –CDR A
MS C
FRP DR
CPD
IATO
LFTE (Components)
LFTE (Systems)
Lot 1
x 6
Virtual M&S
Prototype
Operations & Support
Prototype Testing
Constructive M&S
IBR
ISTF
IT4
TES
IT5
IATT
Acquisition Milestones
Logistics Events
Major Contract Events
Test & Evaluation
Systems Engineering
Training Systems
DIACAP
Requirements
The integrated schedule is the visual pulse of the T&E effort.
It should display the integrated time sequencing of the major test and evaluation phases and events, related activities, and planned cumulative funding expenditures by appropriation.
It should Include event dates such as major decision points as defined in DoD Instruction , e.g., operational assessments, preliminary and critical design reviews, test article availability; software version releases; appropriate phases of developmental test and evaluation; live fire test and evaluation, JITC interoperability testing and certification date to support FRP Decision Review, and operational test and evaluation; low rate initial production deliveries; Initial Operational Capability; Full Operational Capability; and statutorily required reports, such as the Live-Fire T&E Report and Beyond-LRIP Report.
It should provide a single schedule for multi-DoD Component or Joint events showing all DoD Component system event dates.
Provide the FY quarter when the decision to proceed beyond low-rate initial production is planned.
Dynamic schedule from TD through fielding
Must be consistent with AS, program plan, and System Engineering Plan (SEP)
Must be accurate in dates, events, and critical path sequencing
Should be reviewed at every WIPT meeting 18 18

19. References Service/Agency: Other Guides:
4/22/2017
References
Service/Agency:
Army: AR 73-1; DA Pam 73-1; DA T&E Working-level IPT SMART BOOK
Navy: SECNAV D, paragraph
Air Force: AFI 99-13; AF T&E Guidebook
DISA: DISA INSTRUCTION
Other Guides:
DAG - Chapter 9
AT&L & NII - Rules of the Road: A Guide for Leading Successful Integrated Product Teams
DAU - T&E Management Guide
Each Service/Agency should have published direction and or guidance regarding the establishment, mission, etc., of T&E WIPTs (or ITTs, CTTs, etc.) for the acquisition programs it sponsors.
The best overall generic guidance regarding T&E WIPTs may be found in Chapter 9 of the DAG, and the “Rules of the Road” guide. 19

20. Ms. Darlene-Mosser Kerner, 703-697-3406, darlene.mosser-kerner@osd.mil
4/22/2017
Point of Contact
Ms. Darlene-Mosser Kerner, ,
DT&E Website:
Remember
The T&E WIPT is an Integral partner on the PM’s Team
Any questions and recommendations regarding a T&E WIPT are welcomed
Question, recommendations are welcomed.
Contact Ms. Darlene Mosser-Kerner If you want someone from OUSD(AT&L)DDR&E/DDT&E to make this presentation. 20 20