Successful Program Execution Center for Systems and Software Engineering (CSSE) Executive Workshop 15 March 2006 Center for Systems and Software Engineering (CSSE) Executive Workshop 15 March 2006 Marilee J. Wheaton General Manager Systems Engineering Division Marilee J. Wheaton General Manager Systems Engineering Division ©2006 by The Aerospace Corporation

2 People and teamwork Controlling risk Processes Accountability Cost and schedule estimating Stability Industrial base Key Attributes of Successful Program Execution

3 People and Teamwork Contractor execution team and PM should be first line of defense for mission success –Second line is contractor functional organization check and balance –Third line is government, FFRDC and SETA team –Government is not staffed to be the first line of defense Space Quality Improvement Council (SQIC)

4 What Is The SQIC? Space Quality Improvement Council –An industry forum with government sponsorship –Identifies and addresses common problems and solutions Executive-level engineering and quality control leadership from the NSS contractor community –Mission assurance processes –Product quality issues that affect all (Example: FPGAs) –Acquisition policy affecting quality Facilitated by The Aerospace Corporation –Carries forward consensus industry recommendations to government sponsors –Not for attribution

5 Space Quality Improvement Council (SQIC) Key Accomplishments Contractor data sharing –New SQIC/GIDEP NSS Advisory Forum, operational Sept –Anomaly/mission failure investigations Standardized MOU developed and ready when needed Selective specs and standards reintroduction –Industry representatives on all standards committees –Industry consensus on specs and standards to put on contract SMC and NRO requiring that these be on new contracts –Common playing field in bidding executable contracts –Key element in assuring mission success Helps government be a “smarter buyer”

6 Smart Buyer 1 Helping System Program Directors (SPDs) and acquisition executives become smarter NSS buyers –Be aware of the demands and expectations placed on your industry counterparts Smart Buyer 2 Now that the buyer understands the contractor environment… The buyer must formulate an executable plan and maintain an executable baseline.

7 How Do We Control Risk? Four variables in project management: cost, schedule, performance, and risk Need to define risk rigorously and cap it at an acceptable level If you cap the other three variables, risk grows continuously –Example: 1990s NASA Mars missions, which failed Assertion: Capping risk and driving schedule hard leads to best cost performance

8 How Do You Cap Risk? Use standards for engineering and manufacturing –Appropriate test program, robust QA, PMP, etc. Rigorous system engineering and test –Example: JPL Mars Rover “incompressible test list” –Test like you fly, fly like you test Manage risk at a sufficiently senior level on a program –Allowing lower levels to trade mission success for cost and schedule leads to unbounded risk

9 How Do You Cap Risk? (Cont.) Government/industry team manages risk incrementally –Robust mission assurance tailored to risk assessment –Contractor uses “buildup” process in design and test to define and manage risk –Establish confidence at each major step—not just at the end of the program –Find and fix defects early Establish environment that encourages problem reporting –Example: Weekly Watchlist (“No Ambushes”)

10 Processes Use validated processes that ensure predictable, repeatable results –Processes represent lessons learned and best practices from past experiences –Help prevent repeating mistakes Rigorously manage the baseline implementation Fund and manage disciplined, validated technology insertion Require credible independent assessments prior to program initiation and at key milestones

11 Processes (Cont.) Use standards and measure performance against those standards –Put core specs and standards on contract –Examples: PM&P, SE, test –Government program office and FFRDCs ensure that “best practices” are being used Independent flightworthiness and mission assurance certification –Rigorous EELV mission assurance process has 2000 verification tasks prioritized against risk assessment –Underpins “thumbs up” in Flight Readiness Review (FRR)

12 Accountability Nearly every failure in space systems resulted from a breakdown in accountability Need clear definition of accountability on government/contractor team –At every organization level and flows down to individuals executing program –Key program people remain in the job long enough to ensure they are accountable for their decisions that affect mission success

13 Accountability (Cont.) Key roles of government –Define requirements –Set risk level and assure responsible risk management –Ensure that proper practices are used –Incentivize contractor team and assess performance Contractor balances holistic enterprise perspective and tailored program needs –Line organizations are accountable for the product –Functional organizations are accountable for the processes –Prime takes accountability for the quality of output by subcontractors and vendors Streamline government-industry decision-making process: In-plant representation

14 Cost and Schedule Estimating Recognizes that best value is not necessarily lowest cost bid Government must place value on non-deliverables essential to mission success (Examples: SE, MA, QA,…) –Then industry will also value them –Exclude “name-that-tune-in-three-notes” bids Has a well-established Independent Cost Estimating (ICE) and program control function Budget program to 80% confidence, including a management reserve sized by risk –Expend reserves to execute unforeseen elements of baseline program—not new requirements

15 Stability Stable, manageable baselines—requirements, budget, and schedule –Manage necessary but unplanned changes –Rigorous systems engineering process for assessing impact of new requirements –New requirements must come with new funding Allows trade spaces vs. “cast-in-concrete” requirements –Capabilities, cost, and schedule Architectures that allow right-sized programs (can be executed in about 5 years) –Regulates appetite of user community

16 Industrial Base Compete when in the best interests of the government –Government should recognize the need for stability, longevity, and performance of industrial base Government aligns contract and fee structure to balance risk between government and contractor –Example: Titan IV contract A world-class supplier base –Multiple sources, particularly at sub-vendor level Government should require and enforce subcontract management plan –Put it on contract (How do we evaluate and enforce?)

17 Knowing When We Are Turning the Corner Effective programs that meet user demands with 100% mission success Efficient execution—meeting cost, schedule, and performance baselines Retirement of program risk as a function of time is measurable and visible Government and contractor teams meet their commitments Attributes of World-Class Industry/Government Acquisition Team Promises Made = Promises Kept