Fitting Spiral Development to more effective OT&E Ray Smith SYZYGY Technologies, Inc. Good Morning Ladies and Gentlemen. I’m Ray Smith from Syzygy Technologies Inc San Diego, California. Whenever I mention San Diego, all thoughts usually go to the weather. No exception today. I checked the weather forecast for San Diego this morning. Temperatures along the coast are predicted to reach about 65 degrees. It’s winter! Syzygy Technologies is a small business. We do Systems Engineering and Software QA for primarily US Navy customers in San Diego. I’m Director of Operations for our effort in support of SPAWAR Systems Center San Diego. San Diego, CA

Introduction The Concept The Challenge Types of Development Spiral Development Characteristics OT&E Objectives A Fitting Navy Readiness Example Summary Here are some of the items I would like to discuss in my brief The Challenge. Fitting Spiral Development into More Effective OT&E Spiral Development Characteristics. Spiral Development has some traditional development characterics, but also some unique characteristics OT&E Characteristics. I’ll focus on some of the objectives of OT&E The Good Fits. I’ll cover some Spiral Development characteristics that fit well OT&E Objectives The Not so Good Fits. Then I’ll cover the Spiral Development characteristics that do not fit well OT&E Objectives Summary. In summary, I’ll recap what might work in “Fitting Spiral Development into More Effective OT&E”

The Concept Evolutionary Acquisition (EA) “strategies” rely on a Spiral Development (SD) “process” for an earlier delivery to the warfighter A Trade-off…. May be less than full capability (…and less than fully tested), but… Affordable, risk-reduced, agile, and earlier delivery of a product Best Means of getting advanced technologies to the user The challenge: Fitting Spiral Development into More Effective OT&E Spiral development provides other advantages besides User Buy-In. It is a risk-managed process – engaging multiple interest groups - using a cyclic approach to incrementally growing both definition and implementation. Spiral development presents a different challenge to Software Quality Assurance (SQA) and Operational Testing and Evaluation. This paper will develop a comparison of Development Testing /Operational Testing (DT/OT) for traditional development with DT/OT for functionality acquired through a spiral development. The major difference will be seen to be in timing and scope rather than diligence.

The Challenge Fitting Spiral Development into More Effective OT&E DOD programs are increasingly going to spiral development to: Get early buy-in from users on enhanced capabilities. Spiral development presents a different challenge to: Software Quality Assurance (SQA) Operational Testing and Evaluation. The challenge: Fitting Spiral Development into More Effective OT&E Spiral development provides other advantages besides User Buy-In. It is a risk-managed process – engaging multiple interest groups - using a cyclic approach to incrementally growing both definition and implementation. Spiral development presents a different challenge to Software Quality Assurance (SQA) and Operational Testing and Evaluation. This paper will develop a comparison of Development Testing /Operational Testing (DT/OT) for traditional development with DT/OT for functionality acquired through a spiral development. The major difference will be seen to be in timing and scope rather than diligence.

Development Processes Traditional - One extended process Full system capability, but after a long time Evolutionary - Multiple sequential extended processes Full system capability per evolution – new technology as it comes Incremental - Multiple sequential processes Full system capability achieved by increments Spiral - Multiple maturing cycles Gradually maturing capability – early use of new technology Spiral Development is a repetitive system development process for identifying, designing, testing and implementing a set of defined functional capabilities within one increment. SD is iterative (repetitive) for increased refinement of both requirements and developed capabilities and can also be incremental (successive, usually equal portions of the entirety). It is a risk-managed process. Firstly, it is a risk-managed process in defining (refining) the requirements as they become better understood or sometimes actually change (evolve or emerge). Secondly, it is a risk managed process in implementing the developed capabilities. Risks are carefully defined, mitigated, and tracked throughout the spiral. Spiral Development involves users, testers, and developers throughout the process. Feedback is solicited from all Interest Groups (Stake-holders) with the goal of incrementally growing both system definition and deployment. Like traditional development milestones are used to gage progress

Spiral Development (SD) Characteristics Risk-managed process Stakeholder Involvement Cyclic Approach Incrementally grow definition & Implementation Milestone Anchored Spiral Development is a repetitive system development process for identifying, designing, testing and implementing a set of defined functional capabilities within one increment. SD is iterative (repetitive) for increased refinement of both requirements and developed capabilities and can also be incremental (successive, usually equal portions of the entirety). It is a risk-managed process. Firstly, it is a risk-managed process in defining (refining) the requirements as they become better understood or sometimes actually change (evolve or emerge). Secondly, it is a risk managed process in implementing the developed capabilities. Risks are carefully defined, mitigated, and tracked throughout the spiral. Spiral Development involves users, testers, and developers throughout the process. Feedback is solicited from all Interest Groups (Stake-holders) with the goal of incrementally growing both system definition and deployment. Like traditional development milestones are used to gage progress

Operational Test & Evaluation Objectives - Characteristics Operational Effectiveness Operational Suitability Reliability Mean Time Between Operational Mission Failures (MTBOMF) Maintainability Mean Corrective Maintenance Time for Operational Mission Failures (MCMTOMF) Availability Operational Availability (Ao) = UPTIME / UPTIME+DOWNTIME Survivability Operational Test & Evaluation - Objectives - Characteristics Requirements Traceability – Significant Requirements must be traced from definition through design, development, and testing as a single thread. Regardless whether requirement definers, designers, developers, and testers interact as a group, the actual requirements must be traceable for both DT & OT. Risk Aversion Process – OT&E has a goal to avert risk. The goal is to deploy a useful system. Single Interest Group – OT&E focuses on the needs of the Users Measures Suitability & Performance – OT&E works from pre-established measures of performance, reliability, suitability, maintainability and other measures. Full Definition – Full Implementation is the objective of OT&E. There are some exceptions, but full is the goal. Mile Stones are used in the planning for OT&E

Spiral development Develop Design Test Define Refine Conceptualize Spiral n… Test Define Refine Conceptualize Feedback

Spiral Development - OT&E Fit Conceptualize / Feedback Phases Planning is critical – Feedback is vital Define /Refine Phases Requirements traceability – Risk Management Design Phases Development Phases Test (Development Test / Operational Test - DT/OT) Simulation is one solution

Navy Readiness Example BEFORE Spiral Development Reporting Units TYCOMS CNO &FLTCOMS JCS Units TRMS GCCS-M JCS AFTER Spiral Development Reporting Units TYCOMS CNO &FLTCOMS JCS 1 SPIRAL Units TRMS GCCS-M JCS 2 SPIRAL 3 SPIRAL

Summary Plan, Plan, Plan Requirements traceability Risk Management Simulation