1. University of Southern California Center for Systems and Software Engineering Enablers and Inhibitors for Expediting Systems and Software Development Sue Koolmanojwong and Jo Ann Lane COCOMO Forum October 16, 2012

2. University of Southern California Center for Systems and Software Engineering Overview Characterize “expediting” Overview of current research Enablers and Inhibitors for “expediting” –Single system –Systems that participate in one or more systems of systems (SoS) –SoS capabilities Observations 2

3. University of Southern California Center for Systems and Software Engineering What Does it Mean to “Expedite Systems Engineering”? Expedite “systems engineering” or “system development”? –Most are interested in “system development”: Capability development schedule from concept to delivery –Some will include enhancement, maintenance, retirement 3 Early decisions can affect ability to expedite later….

4. University of Southern California Center for Systems and Software Engineering General Ways to “Expedite” Minimal engineering/quick solutions Minimal features Commercial-off-the-Shelf (COTS) solutions Lean approach –Eliminate non-value adding activities –Reduce wait times Pacing –Go slow to establish good Foundation Architecture Interfaces Relatively low complexity –Then go fast 4

5. University of Southern California Center for Systems and Software Engineering Balancing “Expedited Engineering” Trades to consider when “expediting” –Long term affordability –Flexibility/adaptability for meeting future needs –Desired level of performance/speed/throughput –Maintainability –Securability –… and others Trades may –Reduce future flexibility –Result in Degradation of existing capabilities System limitations Later rework 5 Depending on the situation/need, it may be OK to incur technical debt….

6. University of Southern California Center for Systems and Software Engineering Tradespace Example: System Flexibility Goal of “flexibility” is to go beyond quick solution to build in flexibility that will allow system to –Easily evolve in the future to meet future (often unknown) needs –Interoperate with future systems (e.g., in one or more SoS environments) Must balance “flexibility” with “complexity” –Performance issues may result if system tries to be “everything for everyone” Ways to evaluate flexibility –Total ownership costs –Option analyses using Monte Carlo techniques 6

7. University of Southern California Center for Systems and Software Engineering Expediting Development and Increasing Value through Flexibility* Flexibility in design –Routinely improves expected value by 25% or more –Enables system to Avoid future downside risks Take advantage of new opportunities –Often reduces initial capital expenditures Greater expected value at less cost Enables manager to better control the risks Substantial increases on the return on investment “Sweet-spot” found through Monte Carlo analysis of business options –Identifies how much engineering/system performance/system capacity is enough –Allows future decisions/investments to be made when more is known about the future Types of flexibility to explore depend on the context 7 * Richard de Neufville and S. Scholtes, Flexibility in Engineering Design, The MIT Press, Cambridge MA, 2010.

8. University of Southern California Center for Systems and Software Engineering Factors in Expedited Systems Engineering 2012 PSM Workshop –Representatives from commercial, military/ defense, and academic sectors –List the enablers and inhibitors A Single System An Existing Single System A System of Systems –Rate High, Medium, Low for the impact level 8

9. University of Southern California Center for Systems and Software Engineering Enablers and Inhibitors 9 A Single SystemAn Existing Single SystemA System of Systems A clean sheet of paper Greenfield Considerable freedom in development NDI Architecture Process choices Enhancement, Maintenance, Perfection Brownfield or continuation of previously Greenfield Major of minor modification after first delivery Multidisciplinary Constituent-systems are operationally independent Diseconomy of scale

10. University of Southern California Center for Systems and Software Engineering Top 10 Inhibitors – Similarities 10 A New Single SystemAn Existing Single SystemA System of Systems Requirements Volatility Lack of Interoperability UnprecedentednessHigh numbers of external interfaces Lack of / incompatible standard & protocol Delayed authority to proceed/start with fixed milestone UnprecedentednessRequirements Volatility Infeasible schedule/staffing profileVague RequirementsUnprecedentedness Lack of Domain ExperienceEmbedded poor quality softwareHigh numbers of external interfaces Technology VolatilityConflicting StakeholdersInfeasible schedule/staffing profile High numbers of external interfaces Delayed authority to proceed/start with fixed milestone Inability to test across systems Vague RequirementsInfeasible schedule/staffing profile Delayed authority to proceed/start with fixed milestone Under average people / Personnel Capability Technical debtLack of Domain Experience Technology ImmaturityInteroperability / compatibilityTechnology Volatility

11. University of Southern California Center for Systems and Software Engineering Top 10 Inhibitors – Differences 11 A New Single SystemAn Existing Single SystemA System of Systems Requirements Volatility Lack of Interoperability UnprecedentednessHigh numbers of external interfaces Lack of / incompatible standard & protocol Delayed authority to proceed/start with fixed milestone UnprecedentednessRequirements Volatility Infeasible schedule/staffing profileVague RequirementsUnprecedentedness Lack of Domain ExperienceEmbedded poor quality softwareHigh numbers of external interfaces Technology VolatilityConflicting StakeholdersInfeasible schedule/staffing profile High numbers of external interfaces Delayed authority to proceed/start with fixed milestone Inability to test across systems Vague RequirementsInfeasible schedule/staffing profile Delayed authority to proceed/start with fixed milestone Under average people / Personnel Capability Technical debtLack of Domain Experience Technology ImmaturityInteroperability / compatibilityTechnology Volatility

12. University of Southern California Center for Systems and Software Engineering Top 10 Enablers - Similarities 12 A New Single SystemAn Existing Single SystemA System of Systems Rapid Prototyping Target hardware lab / test like you fly & simulation Customer /tech requirements flexibility Target hardware lab / test like you fly & simulation Incremental test and feedbackRapid Prototyping Customer /tech requirements flexibilityIncremental Delivery & feedback Target hardware lab / test like you fly & simulation Incremental test and feedbackFlexible / Tailorable rulesIncremental test and feedback Incremental Delivery & feedbackAgile/lean approachCommon standard and protocol Decision making authorityRapid PrototypingReusing assets Best people / personnel capabilityCommon standard, interfaceTools and automation Agile/lean approachCustomer /tech requirements flexibilityCommon standard, interface Less context switching when doing multiple projects Domain knowledgeBest people / Personnel Capability Tools and automation Understanding of the existing system and interfaces Team cohesion

13. University of Southern California Center for Systems and Software Engineering Top 10 Enablers - Differences 13 A New Single SystemAn Existing Single SystemA System of Systems Rapid Prototyping Target hardware lab / test like you fly & simulation Customer /tech requirements flexibility Target hardware lab / test like you fly & simulation Incremental test and feedbackRapid Prototyping Customer /tech requirements flexibilityIncremental Delivery & feedback Target hardware lab / test like you fly & simulation Incremental test and feedbackFlexible / Tailorable rulesIncremental test and feedback Incremental Delivery & feedbackAgile/lean approachCommon standard and protocol Decision making authorityRapid PrototypingReusing assets Best people / personnel capabilityCommon standard, interfaceTools and automation Agile/lean approachCustomer /tech requirements flexibilityCommon standard, interface Less context switching when doing multiple projects Domain knowledgeBest people / Personnel Capability Tools and automation Understanding of the existing system and interfaces Team cohesion

14. University of Southern California Center for Systems and Software Engineering Observations Flip side of a coin 14 EnablersInhibitors Best peopleUnder average people Decision making authorityLack of decision making authority Team CohesionConflicting stakeholders Incremental Test and FeedbackInability to test across systems

15. University of Southern California Center for Systems and Software Engineering Observations Grey Area Common standard Flexible rules Requirements Volatility Requirements Flexibility Agile/lean approach Constituent Documentation 15

16. University of Southern California Center for Systems and Software Engineering Observations Overlap & Similar but not the same Lack of Domain Experience Unprecedentedness Technology Immaturity Technology Volatility Lack of decision making at lower levels Lack of decision making authority 16

17. University of Southern California Center for Systems and Software Engineering Next Steps Consolidate list of enablers and inhibitors –Considerable overlap Continue –Identification of enablers and inhibitors –Solicit additional evaluations of enablers and inhibitors Evaluate related technical debt issues Map enablers and inhibitors to cost model parameters Refine cost models using –Enablers/inhibitors –Technical debt information

18. University of Southern California Center for Systems and Software Engineering Backup Charts

19. University of Southern California Center for Systems and Software Engineering Conclusions, Recommendations, and Results Single System New Number Identified Estimated Impact Levels (5 assessments) HighMediumLow Enablers26375535 Inhibitors29 7042 Single System Existing Number Identified Estimated Impact Levels (5 assessments) HighMediumLow Enablers28305448 Inhibitors35328946 SoS Number Identified Estimated Impact Levels (5 assessments) HighMediumLow Enablers30386143 Inhibitors35498437 Scope of enablers and inhibitors: People, process, tools, product

20. University of Southern California Center for Systems and Software Engineering A New Single System 20 RankEnablersInhibitors 1Rapid PrototypingRequirements Volatility 2Target hardware lab (experimentation / test lab) / test like you fly & simulation Unprecedentedness 3Customer /tech requirements flexibilityDelayed authority to proceed/start with fixed milestone 4Incremental test and feedbackInfeasible schedule/staffing profile 5Incremental Delivery & feedbackLack of Domain Experience 6Decision making authorityTechnology Volatility 7Best people / personnel capabilityHigh numbers of external interfaces 8Agile/lean approachVague Requirements 9Less context switching when doing multiple projects Under average people / Personnel Capability 10Tools and automationTechnology Immaturity 11Common standard, interfaceConflicting Stakeholders 12Flexible / tailorable rulesLack of decision making authority 13Model-based engineeringlarge number of subcontractors / stakeholders 14Building the common architecture/foundationLack of development infrastructure 15Reusing assetsRules and Regulations 16Risk ManagementSequential Development 17Team cohesionClassification / sensitivity 18Business process reengineering / process streamlining Fear to protest the contract award resulting in poor requirements 19COTSPersonnel Turnover 20Overnight buildBad RFP

21. University of Southern California Center for Systems and Software Engineering An Existing Single System 21 RankEnablersInhibitors 1Target hardware lab (experimentation / test lab)Requirements Volatility 2Incremental test and feedbackHigh numbers of external interfaces 3Incremental Delivery & feedbackUnprecedentedness 4Flexible / tailorable rulesVague Requirements 5Agile/lean approachEmbedded poor quality software 6Rapid PrototypingConflicting Stakeholders 7Common standard, interfaceDelayed authority to proceed/start with fixed milestone 8Customer /tech requirements flexibilityInfeasible schedule/staffing profile 9Domain knowledgeTechnical debt 10Understanding of the existing system and interfaces Interoperability / compatibility 11Best people / personnel capabilitylarge number of subcontractors / stakeholders 12Less context switching when doing multiple projects Backpropagation 13Tools and automationLack of understanding of the existing system and interfaces 14Reusing assetsUnder average people / Personnel Capability 15Team cohesionLack of Domain Experience 16Feasibility Evidence, Milestone reviewTechnology Volatility 17Model-based engineeringTechnology Immaturity 18Colocation of hw & sw engineersClassification / sensitivity 19Development process tailoring/adjustmentMultiple operational sites with different configuration / platform / OS 20Risk ManagementOutdated / stovepipe technology

22. University of Southern California Center for Systems and Software Engineering A System of Systems 22 RankEnablersInhibitors 1Customer /tech requirements flexibilityLack of Interoperability 2Rapid PrototypingLack of / incompatible standard & protocol 3Target hardware lab (experimentation / test lab) / test like you fly & simulation Requirements Volatility 4Incremental test and feedbackUnprecedentedness 5Common standard and protocolHigh numbers of external interfaces 6Reusing assetsInfeasible schedule/staffing profile 7Tools and automationInability to test across systems 8Common standard, interfaceDelayed authority to proceed/start with fixed milestone 9Best people / Personnel CapabilityLack of Domain Experience 10Team cohesionTechnology Volatility 11Synchronization and StabilizationTechnology Immaturity 12flexible / tailorable rulesVague Requirements 13Model-based engineeringLarge number of subcontractors / stakeholders 14Building the common architecture/foundationLack of development infrastructure 15Agile/lean approachLack of communication between teams 16Incremental Delivery & feedbackClassification / sensitivity 17COTSPoor / unknown heritage/pedigree 18Constituent DocumentationBad RFP 19Decision making authorityPersonnel Turnover 20Development process tailoring/adjustmentUnder average people / Personnel Capability

23. University of Southern California Center for Systems and Software Engineering Recent Research Overview 23 Key Approaches for Expedited Engineering Commercial-Off-the-Shelf (COTS) Products Investment in product-line architectures Reuse of existing systems/components Repurposing existing systems/components Value-stream focus (lean) Going fast in general (crisis response) Single purpose architecture Using the right people Key Approaches for Incorporating Flexibility Employ open architectures Design for reuse Develop/use product lines Standard interfaces, protocols, services, data Options-based design Incremental commitment Common Causes of Technical Debt Pressures to compress schedule Lack of requirements understanding Lack of system understanding Inflexible architectures/software Overly complex design/implementation Delayed defect resolution Inadequate testing Lack of current documentation Parallel development in isolation Delayed refactoring Can be extended to incorporate other “-ilities”… Capability Options New system or system of system(s) New procedures for using existing systems Changes to existing system or SoS - Some robust, well integrated - Others very fragile, close to end of life - Which to invest in/which to retire Existing vs. new technologies How much, how fast, how accurate, etc. is enough

24. University of Southern California Center for Systems and Software Engineering Single System Development Perspective Choices driven by potential –Market share –Future opportunities –Technical debt –Cost of failure to provide needed capability 24