1. NR-KPP Interoperability Testing & Certification
Joint Interoperability Test Command
9 March 2009

2. Agenda DoD / Joint Staff Directives and Instructions
JITC Testing and Certification Process
NR-KPP Elements
Testing Overview
Test Environment
NR-KPP Testing Timeline
Test & Certification Products

3. Directives and Instructions
This is the primary guidance we use to conduct our evaluations and certifications. We are not going to go over them in detail. We just want you to be aware of what they are and to know that policy does change from time to time.
NSS: National Security System
IT: Information Technology
JCIDS: Joint Capabilities Integration and Development System 3 3 3

4. NR-KPP T&E Process DATA Risk Analysis
Joint Staff J-6
Interoperability &
Supportability
Certification
Documents:
JCD, ICD, CDD, CPD
ISP, ISP Annex, TISP,
and TEMP
Note:
JCD, ICD, & TEMP
are not J-6 certified
Test & Evaluation
Risk
Note:
Interoperability
changes require reentering this process
at the appropriate point:
Document updates
J-6 I&S Certification
Test & Evaluation
Developmental and Operational
Test & Evaluation
Analysis
This slide depicts the overall JS J6 I&S certification process. Detailed info on the entire process can be found in CJCSI In general, it consists of three processes. Program sponsors will submit the requirements/capabilities documents and J-6 will certify them, as appropriate. Then, JITC will perform test and evaluation of the systems based on those certified requirements and issue the appropriate Interoperability Test Certification.
The amount of testing required to make an interoperability certification decision is based on several factors.  These include the number and complexity of the interfaces, the interoperability requirements, the need for an operationally realistic test environment, the criticality of the information exchanged, and the risks involved with the technology being used.  In each case, the test team must work out a cost-effective test approach that will lead to a definitive interoperability determination. The PM/sponsor and JITC will work closely to establish a strategy for evaluating interoperability requirements in the most efficient and effective manner, in an operationally realistic environment.
ICD – Initial Capabilities Document
JCD – Joint Capabilities Document
TEMP – Test and Evaluation Master Plan
I&S - Interoperability and Supportability
DATA
Joint Interoperability Test Certification
Expires after 4 years, or until there are changes affecting interoperability (system or environment)

5. D/E NR-KPP Compliance IOP Category 6212.01D NR-KPP 6212.01E NR-KPP
Impact / Interoperability Evaluation
Net-Centricity
Compliance with the Net-Centric Operations and Warfare (NCOW) Reference Model (RM) enterprise services
Compliance with net-centric data and services strategies, DoD IEA
Strengthen NC Data/Service Strategy Compliance
- Evaluate Data/Service visibility, accessibility, understandability
- Evaluate compliance with business rules and principles of DoD IEA
Interfaces / Information Exchanges
Operationally effective
information exchanges
Compliant solution architecture products including specified
operationally effective information
exchanges
Use of DoDAF, JCSFL, Enterprise Architecture, etc.
- Verify E2E info exchange IAW the solution architecture
Standards Compliance
Compliance with:
Global Information Grid (GIG) Key Interface Profiles (KIPs)
GIG IT standards and profiles
identified in the TV-1
Compliance with GIG Technical Guidance (GTG), including TV-1, GESPs (KIPs until GESPs available)
KIPs transition to GESPs and combined with DISR compliance into GTG element
- Evaluate applicable GESPs/KIPs via SC/IOP testing
- Evaluate standards compliance
IA Compliance
Compliance with DoD information assurance requirements
Compliance with DoD IA requirements
Verification that IA requirements met – no change
Verify C&A status
IOP testing is conducted using the approved IA configuration
JITC IA testing where applicable
Supportability
Compliance with Spectrum Supportability and E3, Joint Tactical Radio System (JTRS), Selective Availability Anti Spoofing Module (SAASM) [Part of J-6 System Validation and not NR-KPP]
Compliance with Spectrum Certification and E3, SAASM, and JTRS requirements
Added Supportability to NR-KPP compliance
Verify current DD 1494 (spectrum cert) and E3 compliance
Verify GPS SAASM compliance or waiver
- Verify JTRS compliance or waiver

6. NR-KPP Elements
JITC certifies a system meets interoperability requirements contained in the J-6 certified NR-KPP (e.g., CPD, ISP/TISP, ISP Annex, NR-KPP Package)
NR-KPP Statement
Includes descriptions of the system’s operation, traceable between the views – defines end-to-end operationally effective information exchange
Solution Architectures
Describes the activities, standards, and data strategy required to establish, use, operate, maintain, and manage an enterprise-level (EIE MA, COIs) net-centric information environment
Net-Centric Data and
Services Strategies
Set of managed technical interface standards (DISR/GESPs), bringing visibility and stability to critical interfaces
Technical Standards/Interfaces
There are five major elements in the NR-KPP.
The take away point for this slide is that JITC certifies a system meets its interoperability requirements contained in the J-6 certified requirements. We try not to do more or less than what is required.
COI - Communities of Interest
DIACAP - Defense Information Assurance Certification and Accreditation Process
DISR - DOD Information Technology Standards Registry
EIEMA – Enterprise Information Environment Mission Area
GESP – GIG Enterprise Service Profile
GIG – Global Information Grid
GPS – Global Positioning System
JTRS – Joint Tactical Radio System
KIP – Key Interface Profile
SAASM – Selective Availability Anti-Spoofing Module
Information Assurance
DIACAP, including availability, integrity, authentication, confidentiality, and non-repudiation
Supportability
Verify completion of spectrum availability, electromagnetic compatibility, JTRS usage, and GPS SAASM requirements
Inspection
Lab Test
Operational Test

7. Solution Architectures
Testing Overview
Solution Architectures
Integrated Architecture products define end-to-end information exchange and use for a given capability.
Focus is on joint critical exchanges (NR-KPP threshold) and includes all types: automated data and voice.
MOPs with attributes found in SV-6, e.g., timeliness, throughput, periodicity, size, etc. for assessment of both critical and non-critical exchanges.
MOEs for determining operational effectiveness and completeness of information exchanges. Use OV-5/6c and SV-5 data to determine if system performance meets operational requirements. Applies to assessment of both critical and non-critical exchanges.
Interoperability MOPs / MOEs incorporated into TEMP
Leverage output from PM, OTA, and other COCOM / Service / Agency events
Architecture Products themselves are assessed during document reviews. For interoperability, the focus is on the information exchanges. During our evaluation, we will verify to ensure that the system can effectively exchange information to accomplish its mission.
MOP – Measure of Performance
MOE – Measure of Effectiveness
NR-KPP – Net-Ready Key Performance Parameter
OTA – Operational Test Agency
OV – Operational View
PM – Program Manager
SV – System View
TEMP – Test & Evaluation Master Plan

8. Net-Centric Data and Services Strategies
Testing Overview
Net-Centric Data and Services Strategies
Identify net-centric services and shared enterprise-level data
Verify services and data are properly registered, visible, and accessible
Conformance testing for data / schemata (e.g., proper XML format)
During interoperability testing, verify correct provision and use of services / data and any related performance parameters (QoS, timeliness, etc.)
Net-centric services / data analyzed by:
COI (may be multiple COIs, overlapping COIs)
Design test scenarios to verify effective information exchanges within and among declared COIs
EIE MA (Enterprise Info Environment Mission Area – i.e., core services – NCES)
Design test scenarios to verify the implementation and use of core services as they become available
Core net-centric standards
IPv6 is a net-centric enabling technology that must be considered
XML, WSDL, SOAP, UDDI, etc… used to implement SOA via web services
These are the aspects of Net-Centricity that most concern Joint Interoperability Test & Certification. We focus on the services and the data. If a system claims that it implements the net-centric services and XML tagging of data, then we will verify that the system does what is claimed in the certified requirements documents.
One thing we have noticed is that several programs have claimed that they tag all data and their systems are net-centric when they are not. Just a word of caution, only put down what your system currently implements and not all the buzz words in Otherwise, we will hold your system to the certified requirements and if claimed items (such as GESPs/KIPs) are not implemented, then the status will be “not met.”
NCES – Net-Centric Enterprise Services
QoS – Quality of Service
SOA – Service-Oriented Architecture
SOAP – Simple Object Access Protocol
UDDI – Universal Description, Discovery and Integration
WSDL – Web Services Description Language
XML – eXtensible Markup language
Web Services Description Language -- an XML-formatted language used to describe a Web service's capabilities as collections of communication endpoints capable of exchanging messages. WSDL is a critical component of UDDI, an XML-based service discovery registry. WSDL is the language that UDDI uses.
Simple Object Access Protocol -- a lightweight XML-based messaging protocol used to encode the information in Web service request and response messages before sending them over a network.
Universal Description, Discovery, and Integration-- It is a Web-based distributed directory that enables businesses to list themselves on the Internet and discover each other, similar to a traditional phone book's yellow and white pages.
SOA – Service-Oriented Architecture (6212 defines as using web services)

9. Technical Standards/Interfaces
Testing Overview
Technical Standards/Interfaces
GESP Declaration (or KIP declaration, e.g., for pre E documents) contained in certified CPD, ISP, ISP Annex, TISP, or NR-KPP package, TV-1 from DISROnline.
List of applicable GESPs/KIPs, GESP/KIP profiles, and implementation options, and TV-1 standards
JITC reviews applicable standards called out by GESP/KIP/TV-1.
Identify standards/profiles that require testing vs. other evaluation methods.
Collect data to support standards conformance certification.
Analyze and roll up the results.
Information exchanges across key interfaces verified IAW performance parameters during interoperability testing.
Available GESP/KIP/standards information:
DISRonline to select DISR GESP/KIP profiles, TV-1 standards
CPD – Capability Production Document
DISR – DOD Information Technology Standards Registry
GESP – GIG Enterprise Service Profile
GIG – Global Information Grid
ISP – Information Support Plan
KIP – Key Interface Profile
NR-KPP – Net-Ready Key Performance Parameter
SAASM – Selective Availability Anti-Spoofing Module
TISP – Tailored ISP
TV – Technical View

10. Information Assurance
Testing Overview
Information Assurance
During / after DT, JITC will:
Verify the DIACAP process has been accomplished.
Review the System Identification Profile (SIP), Certification Report, Plan of Action and Milestones (POA&M) and the IATO, as applicable.
During / after OT, JITC will:
Review the Comprehensive DIACAP Package.
Work closely with PMs and OT testers to evaluate IA aspects of the systems.
Review OT report and verify IA compliance, as appropriate.
Test configuration must mirror approved operational configuration (i.e., testing in a realistic IA environment).
Results of any IA accreditation, waivers, etc. collected for reporting in certification.
JITC has capabilities to support IA assessments; however, usually we report on what has been done during DT/OT and DIACAP processes to satisfy this element of the NR-KPP. We try not to duplicate any effort.
If your system has gone through the DIACAP process, then you need to provide us with that info to include the IATO or ATO, as applicable, to include in our certification letter. Also, during interoperability testing, we will design our test configuration to mirror the approved IA configuration as documented in the IATO or ATO, to ensure that the test reflects the real operational environment as much as possible.
IA – Information Assurance
(I)ATO – (Interim) Approval to Operate
Note: The Interim DoD Certification and Accreditation (C&A) Process Guidance was published on July 6, The guidance stated that the DIACAP had superseded the DITSCAP for the certification and accreditation process.

11. Testing Overview Supportability Spectrum Supportability:
Spectrum Certification:
JITC reports on the stage of the DD 1494.
JITC reports on the J/F 12, if the process is complete.
E3 - JITC reports on E3 control which may include HERO, HERF, HERP, MIL-STD-461 and MIL-STD-464A testing.
JTRS - If the system uses radio-based communications, and the radio is in the 2 MHz to 2 GHz spectrum, and the radio is not JTRS, then JITC will report on the ASD-NII authorization to use a non-JTRS solution.
GPS SAASM - JITC will verify that each JCIDS document, or ISP, TISP, or EISP contains a statement that the program is in compliance with CJCSI E, and that analysis of capability technical documentation and test data confirm compliance.
There are three parts of the NR-KPP Supportability element.
Spectrum supportability: Ensures there are RF Spectrum resources available for the program in its intended environment. Also, ensures the program does not cause electromagnetic interference to other systems in its intended operational environment.
JTRS: Joint Tactical Radio System. All radio-based communications requirements must be fulfilled with JTRS, unless specifically authorized by ASD/NII.
GPS SAASM: Selective Availability Anti-Spoofing Module. All GPS receivers are required to implement the SAASM.
All systems that use radio communications in the 2 MHz to 2 GHz range are required to use JTRS to meet those requirements. The only exception is the program must have an authorization from ASD/NII to use any other radios. JITC will report on the ASD/NII authorization, if applicable.
All GPS receivers must implement the SAASM and have a statement in their capabilities documents, ISP, TISP, or EISP to that effect. JITC will report on the statement and if the program implements a non-SAASM compliant GPS

12. NR-KPP Testing Timeline
Milestone A
Draft CDD
Milestone B
Certified CDD, Initial ISP
Milestone C
Certified CPD, Final ISP
Full-Rate Production DR
Engineering and Manufacturing
Development
Technology
Development
Production and
Deployment
Mission Effective
E2E Info Exchange
Technology Development
Mission Effectiveness E2E Info Exchange
Net-Centric Data/Svcs, GTG/GESP, and IA Integration
Mission Effectiveness E2E Info Exchange
Net-Centric Data/Svcs, GTG/GESP, and IA Compliance
Mission Effectiveness E2E Info Exchange
Certification
Emphasis
Net
Enabled
Interoperability
Certified
Integrated
Components
Constructive
& Virtual
Simulations
Components
NSS / IT
NSS/IT
Composition
NSS / IT
Solutions
Architecture
Operational, System, Technical View Integration
Operational, System, Technical View Demonstration
Net-Centric
Data/Svcs
COI and Enterprise
Data / Service Integration
COI and Enterprise
Data / Service Compliance
Information
Technology
While not every program/system follows the same development path, this chart provides a rough idea of when various elements of the NR-KPP come into play.
NR-KPP evaluation spans the entire life-cycle with approach developed before MS B and focus on all components across five elements as the program progresses to MS C and FRPD
Supports Critical
Operational Issues
& Measures
of Effectiveness
GTGs/GESPs
GTG/GESP Integration
GTG/GESP Compliance
DIACAP Activity 2
Compliance
DIACAP Activity 3
Compliance IA
Supportability Rqmts
Verification
Spectrum Supportability, JTRS and GPS SAASM compliance
Supportability
Assess
Technology
Demonstrate System
Integration and Interoperability
Achieve Operational
Capability
NR-KPP evaluation approach developed before MS B

13. Interoperability Test and Certification Products
Description
System can be fielded (Y/N)?
Standards Conformance Certification
System is certified for conformance to a standard/standards profile No
Joint Interoperability Test Certification
Full system certification. System meets at least all critical interoperability requirements
Yes
Limited Joint Interoperability Test Certification
System meets subset of critical interoperability requirements
Yes, with ICTO
Interim Joint Interoperability Test Certification
A capability module has adequately demonstrated interoperability for at least all critical threshold requirements identified for the increment
Special Interoperability Test Certification
Based on other J-6 approved requirements (i.e., no NR-KPP) -- E.g., use of UCR for voice switches
Non-Certification
Critical operational impacts expected
Provides a warning to the warfighter
Interoperability Assessment
PM would like to determine the interoperability status
System may lack J-6 certified requirements

14. Contact Information
Hotline
24/7 C4I Technical Support
JITC (5482)
Joint Interoperability Tool (JIT)
Lessons Learned reports
NATO Interface Guide
System Tracking Program (STP)
Test events
Test plans and reports
Certification results
General Testing Support
(301) or

15. Questions?

17. DAU NR-KPP Training
3-hour Program Manager-focused NR-KPP Computer Based Learning module (Net-Ready Key Performance Parameter Course - CLM 029) available online at
Do not need to be a registered DAU student to take module
Lesson Objectives
Explain the joint interoperability test and certification process
Explain program managers’ responsibilities in joint interoperability testing
Describe the operational and test and evaluation implications of the NR-KPP
Understand the 4-year interoperability re-certification requirement and process
While the DAU course was developed for PMs, there is a relatively large portion dedicated to testing. This course can help one understand the NR-KPP from the PM’s perspective, which can only help when coordinating testing requirements.
DAU – Defense Acquisition University