1. Michael Morris Syzygy Technologies, Inc.
Effectiveness and Efficiency of SOVT as a Means to Demonstrate Interoperability
Michael Morris
Syzygy Technologies, Inc.
Good Morning Ladies and Gentlemen.
I’m Ray Smith from Syzygy Technologies Inc San Diego, California.
Whenever I mention San Diego, all thoughts usually go to the weather. No exception today.
I checked the weather forecast for San Diego this morning. Temperatures along the coast are predicted to reach about 65 degrees. It’s winter!
Syzygy Technologies is a small business. We do Systems Engineering and Software QA for primarily US Navy customers in San Diego. I’m Director of Operations for our effort in support of SPAWAR Systems Center San Diego.
San Diego, CA

2. Outline Current Process Concerns Concept Install & Testing Process
Recommendations
Process
Summary

3. The Current SOVT Process
Test event is final step for new installation
Purpose and scope is to install and demonstrate system capabilities
1 shot 1 kill approach (single installation expected)
All future testing evolutions are dependent on SOVT acceptance
The challenge: Fitting Spiral Development into More Effective OT&E
Spiral development provides other advantages besides User Buy-In. It is a risk-managed process – engaging multiple interest groups - using a cyclic approach to incrementally growing both definition and implementation.
Spiral development presents a different challenge to Software Quality Assurance (SQA) and Operational Testing and Evaluation. This paper will develop a comparison of Development Testing /Operational Testing (DT/OT) for traditional development with DT/OT for functionality acquired through a spiral development.
The major difference will be seen to be in timing and scope rather than diligence.

4. Issues & Concerns
The lack of an intermediate level of interoperability testing
Individual systems SOVT does not assess integration/interoperability
High rate of system deficiencies identified
System unavailable during training & operational events
Command resource conflict
Warfighter pays in loss of system capabilities
Sponsor cost impacted
The challenge: Fitting Spiral Development into More Effective OT&E
Spiral development provides other advantages besides User Buy-In. It is a risk-managed process – engaging multiple interest groups - using a cyclic approach to incrementally growing both definition and implementation.
Spiral development presents a different challenge to Software Quality Assurance (SQA) and Operational Testing and Evaluation. This paper will develop a comparison of Development Testing /Operational Testing (DT/OT) for traditional development with DT/OT for functionality acquired through a spiral development.
The major difference will be seen to be in timing and scope rather than diligence.

5. Concept End to End (E2E) SOVT
End to End SOVT includes high level testing of legacy system
More granular testing of newly installed systems
Test team consist of SMEs (Subject Matter Experts), Systems Sponsor, Maintenance & Training Agencies, and Warfighter
Interoperability testing
Systems familiarity & training
Scalable to fit Command requirements/configurations
Success is based on interaction between all agencies previously mentioned above
Spiral Development is a repetitive system development process for identifying, designing, testing and implementing a set of defined functional capabilities within one increment. SD is iterative (repetitive) for increased refinement of both requirements and developed capabilities and can also be incremental (successive, usually equal portions of the entirety). It is a risk-managed process. Firstly, it is a risk-managed process in defining (refining) the requirements as they become better understood or sometimes actually change (evolve or emerge). Secondly, it is a risk managed process in implementing the developed capabilities. Risks are carefully defined, mitigated, and tracked throughout the spiral. Spiral Development involves users, testers, and developers throughout the process. Feedback is solicited from all Interest Groups (Stake-holders) with the goal of incrementally growing both system definition and deployment. Like traditional development milestones are used to gage progress

6. Install and Testing Process
Correct installation issues from testing.
Document result to Command and SYSCOM.
Review Metrics for future installations. Analysis of SOVT Metrics.
C4ISR Systems installation.
SYSCOM
SYSCOM Confirms System Installation Consolidated E2E SOVT teams engaged.
Teams conduct testing. Find, fix issues and train personnel.
Report issues not corrected.
SYSCOM Consolidates issues for Program office & Maintenance Agency
SYSCOM
Update Metrics
Program
Office
Maintenance
Agency
Program Office and Maintenance Agency revise installation plans, procedures and SOVT documentation.
Maintenance Agency correct legacy issues from testing.
Program Office provide up to date system testing documentation.
Update Metrics
Maintenance
Agency
CASREP, HELP DESK DATA

7. Recommendations Institute interoperability testing during SOVT process
Systems metrics to be collected
Provide metrics for analysis of future SOVT test
Use data collected to better ascertain installation concerns
Data provided for the sponsor’s IV&V process prior to field testing
Finally, data will help support future R&D efforts on C4I capability
Operational Test & Evaluation - Objectives - Characteristics
Requirements Traceability – Significant Requirements must be traced from definition through design, development, and testing as a single thread. Regardless whether requirement definers, designers, developers, and testers interact as a group, the actual requirements must be traceable for both DT & OT.
Risk Aversion Process – OT&E has a goal to avert risk. The goal is to deploy a useful system.
Single Interest Group – OT&E focuses on the needs of the Users
Measures Suitability & Performance – OT&E works from pre-established measures of performance, reliability, suitability, maintainability and other measures.
Full Definition – Full Implementation is the objective of OT&E. There are some exceptions, but full is the goal.
Mile Stones are used in the planning for OT&E

8. Summary
Implementing a SOVT process that conducts testing of the entire C4I suite will:
Ensure new system installation has not effected legacy systems
Prove all systems in the C4I suite are operational
Decrease the number of STRs (Software Trouble Reports) generated and system down time
Provide system operators and administrators time to become familiar with system