1. Design and Development Risk Management Dr. Dick Hayes, Evidence Based Research, Inc. CDR. Steve Palmquist, USCG George Thurmond, Georgia Tech John Birkler, RAND Bill Lese, Logicon John Smith, Decisive Analytics Cyrus Staniec, Logicon Debora Donnellon, NIMA

2. Design and Development Risk Management Dimensions Product Service Market Risks –Cost, Schedule, and Quantity Tradeoffs –Expectations and Value Technical Risks –Maturity of Technology –Technical Advance Required –Interoperability Management Processes –Process Indicators –Personnel Indicators –Productivity Indicators

3. Management Processes Checklist Identify Warning Signs, Enable Timely Interventions Process Indicators –Changing goals - “lowering the bar” –Slipping milestones “stretch out” –ECP’s that raise costs –Parallel development efforts without strong oversight –Failure to establish and maintain top down, integrated vision –Progress report vagueness, lack of item resolution –Low levels of technical interaction –Weak Configuration Management Productivity Indicators During Development –“Healthy” Debugging Rate –Ease of Integration –Improving Test Results over Time

4. Management Processes Checklist (2 of 2) Personnel Indicators –Changes in key personnel –Leadership inexperienced in similar projects –Breadth of senior personnel knowledge –Corporate culture barriers

5. Metrics for Assessing Products and Services Explicit Metrics, Consistent over time, are essential Key Dimensions Include Reliability Ease of Maintenance Ease of Use Interoperability Design Characteristics Measures of Systems Performance Measures of C4ISR Effectiveness Measures of Force Effectiveness Measures of Policy Effectiveness Ignoring any key dimension is likely to lead to problems

6. Design and Development Risk Management: Longer Term Creation of Specialized Training for C4ISR Acquisition Managers (IT acquisition is different from systems acquisition) Creation of a Community of C4ISR Acquisition Managers –Web site –Workshops –Annual Symposium –Common Literature and References Implement Virtual Collaboration Tools Using IT to Support C4ISR Acquisitions –Faster –More Inclusive –Clearer Consensus –Cheaper but limited IT capability in community, work process change required, leadership needed.

7. Life Cycle Tracking of Relevant Data –Program Data and Information –Simulations, Experiments, Tests Improved Contractual Flexibility –Internet Based –Pre-qualified –Specialized Services –Examples: GSA Schedules Design and Development Risk Management: Longer Term (2 of 2)

8. Short Term Recommendations Increase Visibility & Quality of Application of the C4ISR Architecture Framework (V 2.0) – formally applies to “all on-going & planned C4ISR Architectures” – But lacks standards, quality control, and formal approval process Emplace a process to link across individual Architectures

9. Other Good Ideas Information Dissemination is Critical –CROP vs. Drinking from A Firehose –Subscription, Limited Push, Rich Pull desirable –Experimentation is essential to determine best mix Co-evolution is essential –Market risk element in JWIDs, ACTD’s, etc. –Systems need CONOPS, Doctrine, Training, Maintenance, etc. –Fools gold vs. golden nuggets

10. Existing Tools and Sources of Help Software Program Managers Network www.spmn.com Information Technology Resources Board www.itrb.gov Software Development CMM Software Acquisition CMM People CMM Architecture Tradeoff Analysis Method (ATAM) www.sei.cmu.edu C4ISR Architecture Framework V2.0 Joint C4ISR Analysis/Planning System (JCAPS) C4ISR Decision Support Center (DSC)