1. Improving Productivity on an SEI Level IV Project presented by Kelly Ohlhausen

2. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 2 Agenda Project Description - Where we were… Why is it important? How did we find the answers? What did we find? Where we are now… Details of what we did Summary

3. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 3 Where we were... One of four projects evaluated during an SEI CBA-IPI Level IV assessment. (June 2001) Productivity identified as a risk from the beginning (productivity = ELOC/hour) Software bid was aggressive 10% management challenge 12% customer challenge Real-time embedded system to be completed in two deliveries to the customer

4. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 4 Where we were...

5. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 5 Why aren’t we meeting the goal? Was the productivity goal set unrealistically high? Process - Are these the right processes? Are there too many? Too few? Was the mix of engineering skills appropriate? Was there enough training? Software projects can’t ever reach goals so why bother setting them?

6. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 6 Why is this important? Productivity estimates impact customer and management satisfaction, profits, and potentially future business. Our customers need to be able to trust the numbers we use in a bid. Profit margins disappear when projects are bid with unrealistically high productivity levels.

7. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 7 Finding the Answers Initiated a Raytheon Six Sigma project to determine what, if any, actions could be taken to improve productivity during the implementation stage Raytheon Six Sigma is an iterative knowledge- based process for improvement

8. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 8 Analysis Factors Attrition Learning curve (training new people) Experience level Hardware availability Overlapping stages System engineering Customer directed slowdown

9. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 9 Information Sources Metrics: size, cost, schedule, delivery, staffing, defect containment, requirements stability Earned value progress metrics, audits, action items Lessons Learned from prior stages Two other projects using the same processes R6Sigma Defect Containment Metric Analysis project results

10. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 10 Information Sources Meeting minutes (e.g. weekly progress, metrics analysis) Inputs from engineers and managers

11. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 11 Information Sources The search for information took place during R2 object design stage. Software DevelopmentR1 R2 - Requirements Analysis - System Design - Object Design X - Implementation - Software Integration & TestX - System Integration & Test - Acceptance Test

12. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 12 Requirements for Application Changes were to be put into place immediately. The R2 implementation stage schedule had to be shortened by four weeks without removing scope to compensate for a schedule slip. The software integration and test stage had to start on time. Changes could not have a negative impact on quality.

13. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 13 What we found... Moving toward SEI Level V (Optimizing) with a number of smaller actions involving defect containment and process change management could significantly impact productivity.

14. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 14 Where we are now... * as of 2/1/02 34.3% increase

15. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 15 Where we are now... Earned Value Metrics * as of 2/1/02

16. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 16 What was changed... Escaping defects –What should be covered by a unit test was addressed in detail during the implementation stage training (JIT) –Stress was placed on pushing boundaries and not relying just on nominal values. More special case unit tests were used.

17. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 17 What was changed... Escaping defects (cont.) –Run the unit tests on the target processors during implementation. Don’t wait until integration to discover a platform issue. –A step was added to the stage process for extra testing to identify rogue pointers before unit testing The full impact of these changes will not be known until integration and testing stage.

18. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 18 What was changed... Developed the code within the configuration management tool from the beginning. Gave people a chance to learn the tool in small steps instead of all at once. All tools and hardware identified and checked out BEFORE the stage began (matching versions across test lab, compiler updates in place, instructions available)

19. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 19 What was changed... We did better defining the earned value progress metrics –Made sure there was enough detail (stage process steps were defined at the correct granularity) –Used realistic weighting Because… An incorrect plan yields misleading information. Time is wasted investigating non-issues or not identifying a problem in time.

20. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 20 What was changed... Did better at following the standards and processes. –Invited the right people for the review. –Assigned roles to focus the review, esp. facilitator –The implementation stage Just-In-Time (JIT) training was used to remind the team of these issues. Because… –Products that do not meet standards have an effect on time during reviews as well as rework.

21. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 21 Summary There was no conclusive evidence that the productivity goal was unrealistic. Relatively small changes made an impact. The research uncovered improvements for every stage. If we had applied these improvements from the beginning of the project, we can only imagine what the overall productivity would have been.

22. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 22 Conclusion Take time to look at what you are doing. Learn, not just from your own experience but from the experience of others. Continuously try to identify optimization opportunities. Improvement can be made, even on an SEI Level IV program.

23. © 2002, Raytheon Company. All Rights Reserved. 8 February 2002 Page 23 The greatest obstacle to discovering the shape of the earth, the continents and the ocean was not ignorance but the illusion of knowledge. Daniel J. Boostin The Discoverers