ICSM Patterns and Common Cases

ICSM Patterns and Common Cases
For v2: Replace “LCO” by “VCR”, and “LCA” by “FCR”. Remove LCO entry from list of acronyms (slide 51). -- JPA Barry Boehm CS 510, Fall 2017

The Incremental Commitment Life Cycle Process: Overview
Stage I: Definition Stage II: Development and Operations Anchor Point Milestones The Incremental Commitment Life Cycle Process: Overview This slide shows how the ICM spans the life cycle process from concept exploration to operations. Each phase culminates with an anchor point milestone review. At each anchor point, there are 4 options, based on the assessed risk of the proposed system. Some options involve go-backs. These options result in many possible process paths. The life cycle is divided into two stages: Stage I of the ICM (Definition) has 3 decision nodes with 4 options/node, culminating with incremental development in Stage II (Development and Operations). Stage II has an additional 2 decision nodes, again with 4 options/node. One can use ICM risk patterns to generate frequently-used processes with confidence that they fit the situation. Initial risk patterns can generally be determined in the Exploration phase. One then proceeds with development as a proposed plan with risk-based evidence at the VCR milestone, adjusting in later phases as necessary. For complex systems, a result of the Exploration phase would be the Prototyping and Competition Plan discussed above. Risks associated with the system drive the life cycle process. Information about the risk(s) (feasibility assessments) supports the decision to proceed, adjust scope or priorities, or cancel the program. Synchronize, stabilize concurrency via FEDs Risk patterns determine life cycle process 03/19/2008 Fall 2017 ©USC-CSSE ©USC-CSSE 2

The ICM as Risk-Driven Process Generator
Stage I of the ICM has 3 decision nodes with 4 options/node Culminating with incremental development in Stage II Some options involve go-backs Results in many possible process paths Can use ICM risk patterns to generate frequently-used processes With confidence that they fit the situation Can generally determine this in the Exploration phase Develop as proposed plan with risk-based evidence at VCR milestone Adjustable in later phases Fall 2017 ©USC-CSSE

Different Risk Patterns Yield Different Processes
As illustrated in the four example paths through the Incremental Commitment Model in this slide, the ICM is not a single monolithic one-size-fits-all process model. As with the spiral model, it is a risk-driven process model generator, but the ICM makes it easier to visualize how different risks create different processes. In Example A, a simple business application based on an appropriately-selected Enterprise Resource Planning (ERP) package, there is no need for a Valuation or Foundations activity if there is no risk that the ERP package and its architecture will not cost-effectively support the application. Thus, one could go directly into the Development phase, using an agile method such as a Scrum/Extreme Programming combination would be a good fit. There is no need for Big Design Up Front (BDUF) activities or artifacts because an appropriate architecture is already present in the ERP package. Nor is there a need for heavyweight waterfall or V-model specifications and document reviews. The fact that the risk at the end of the Exploration phase is negligible implies that sufficient risk resolution of the ERP package’s human interface has been done. Example B involves the upgrade of several incompatible legacy applications into a service-oriented web-based system. Here, one could use a sequential waterfall or V-model if the upgrade requirements were stable, and its risks were low. However, if for example the legacy applications’ user interfaces were incompatible with each other and with web-based operations, a concurrent risk-driven spiral, waterfall, or V-model that develops and exercise extensive user interface prototypes and generates a Feasibility Evidence Description (described on chart 12) would be preferable. In Example C, the stakeholders may have found during the Valuation phase that their original assumptions about the stakeholders having a clear, shared vision and compatible goals with respect the proposed new system’s concept of operation and its operational roles and responsibilities were optimistic. In such a case, it is better to go back and assure stakeholder value proposition compatibility and feasibility before proceeding, as indicated by the arrow back into the valuation phase. In Example D, it is discovered before entering the Development phase that a superior product has already entered the marketplace, leaving the current product with an infeasible business case. Here, unless a viable business case can be made by adjusting the project’s scope, it is best to discontinue it. It is worth pointing out that it is not necessary to proceed to the next major milestone before terminating a clearly non-viable project, although stakeholder concurrence in termination is essential. Fall 2017 03/19/2008 ©USC-CSSE ©USC-CSSE 4 4

The ICM Process Decision Table: Key Decision Inputs
Product and project size and complexity Requirements volatility Mission criticality Nature of Non-Developmental Item (NDI)* support Commercial, open-source, reused components Organizational and Personnel Capability * NDI Definition [DFARS]: a) any product that is available in the commercial marketplace; b) any previously developed product in use by a U.S. agency (federal, state, or local) or a foreign government that has a mutual defense agreement with the U.S.; c) any product described in the first two points above that requires only modifications to meet requirements; d) any product that is being produced, but not yet in the commercial marketplace, that satisfies the above criteria. Fall 2017 ©USC-CSSE

The ICM Process Decision Table: Key Decision Outputs
Key Stage I activities: incremental definition Key Stage II activities: incremental development and operations Suggested calendar time per build, per deliverable increment Fall 2017 ©USC-CSSE

Common Risk-Driven Special Cases of the ICM
Example Size, Complexity Change Rate % /Month Criticality NDI Support Org, Personnel Capability Key Stage I Activities : Incremental Definition Key Stage II Activities: Incremental Development, Operations Time per Build; per Increment 1. Use NDI Small Accounting Complete Acquire NDI Use NDI 2. Agile E-services Low 1 – 30 Low-Med Good; in place Agile-ready Med-high Skip Valuation , Architecting phases Scrum plus agile methods of choice <= 1 day; 2-6 weeks 3. Architected Agile Business data processing Med 1 – 10 Med-High most in place Combine Valuation, Architecting phases. Complete NDI preparation Architecture-based Scrum of Scrums 2-4 weeks; 2-6 months 4. HW component with embedded SW Multisensor control device 0.3 – 1 Med-Very High In place Experienced; med-high Concurrent HW/SW engineering. CDR-level ICM DCR IOC Development, LRIP, FRP. Concurrent Version N+1 engineering SW: 1-5 days; Market-driven 5. Indivisible IOC Complete vehicle platform Med – High High-Very High Some in place Determine minimum-IOC likely, conservative cost. Add deferrable SW features as risk reserve Drop deferrable features to meet conservative cost. Strong award fee for features not dropped SW: 2-6 weeks; Platform: 6-18 months 6. NDI- Intensive Supply Chain Management 0.3 – 3 Med- Very High NDI-driven architecture NDI-experienced; Thorough NDI-suite life cycle cost-benefit analysis, selection, concurrent requirements/ architecture definition Pro-active NDI evolution influencing, NDI upgrade synchronization SW: 1-4 weeks; System: 6-18 months 7. Hybrid agile / plan-driven system C4ISR Med – Very High Mixed parts: Mixed parts; Med-Very High Mixed parts Full ICM; encapsulated agile in high change, low-medium criticality parts (Often HMI, external interfaces) Full ICM ,three-team incremental development, concurrent V&V, next-increment rebaselining 1-2 months; 9-18 months 8. Multi-owner system of systems Net-centric military operations Very High Many NDIs; some in place Related experience, med-high Full ICM; extensive multi-owner team building, negotiation Full ICM; large ongoing system/software engineering effort 2-4 months; months 9. Family of systems Medical Device Product Line 1 – 3 Related experience, med – high Full ICM; Full stakeholder participation in product line scoping. Strong business case Full ICM. Extra resources for first system, version control, multi-stakeholder support 1-2 months; months C4ISR: Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance. CDR: Critical Design Review. DCR: Development Commitment Review. FRP: Full-Rate Production. HMI: Human-Machine Interface. HW: Hard ware. IOC: Initial Operational Capability. LRIP: Low-Rate Initial Production. NDI: Non-Development Item. SW: Software Fall 2017 ©USC-CSSE

Case 1: Use NDI Exploration phase identifies NDI opportunities
NDI risk/opportunity analysis indicates risks acceptable Product growth envelope fits within NDI capability Compatible NDI and product evolution paths Acceptable NDI volatility, some open-source components highly volatile Acceptable usability, dependability, interoperability NDI available or affordable Example: Small accounting system Size/complexity: Low Anticipated change rate (% per month): Low Criticality: Low NDI support: Complete Organization and personnel capability: NDI-experienced Key Stage I activities: Acquire NDI Key Stage II activities: Use NDI Time/build: Driven by time to initialize/tailor NDI Time/increment: Driven by NDI upgrades Case 1: Suppose you have an application for which an appropriate NDI (COTS, open source, reuse library, customer-furnished package) solution is available, and other options of developing perhaps a better version yourself or outsourcing such a development. Even if you produce a better solution (frequently not the case), you will generally incur more expense and take longer to begin to capitalize on its benefits. And you will often find that the NDI package has features that you hadn’t realized you would need, but are there when you need them. On the other hand, there are risks that may disqualify some NDIs. They may be overly complex for your needs, incompatible with your other applications, or highly volatile. See the discussion in Section 33.1 of Software Engineering Economics (Boehm, 1981) for more about the pros and cons of NDI solutions. Fall 2017 ©USC-CSSE

Case 2: Pure Agile Methods
Exploration phase determines Low product and project size and complexity Fixing increment defects in next increment acceptable Existing hardware and NDI support of growth envelope Sufficient agile-capable personnel Need to accommodate rapid change, emergent requirements, early user capability Example: E-services Size/complexity: Low Anticipated change rate (% per month): 1-30% Criticality: Low to medium NDI support: Good; in place Organization and personnel capability: Agile-ready, medium to high capability Key Stage I activities: Skip Valuation and Architecting phases Key Stage II activities: Scrum plus agile methods of choice Time/build: Daily Time/increment: 2-6 weeks Case 2: If your project is small (less than 10 people) and its criticality involves the loss of discretionary vs. essential funds, a pure agile method such as Extreme Programming (Beck, 1999), Scrum (Schwaber, 2002), or Crystal (Cockburn, 2002) is generally best if you have relatively high-capability, agile-ready personnel. The risks of using a more formal, plan-driven approach are less adaptability to change and belated feedback on the product’s capabilities. The biggest risk is to try to develop the application all at once for several months, and then find that it is a mismatch to the users’ needs. Agile developers have found that the best way to address this risk is to organize the project into short (2-6 week) delivery increments that may be incomplete, but provide early useful capabilities. Any flaws can then be detected early and fixed in the next increment (for very high criticality applications, this would not be acceptable). On a small project it is also easy to set up a daily build and regression test structure that identifies integration problems early when they are easier to fix. Some risks of using the agile approach is that it is harder to write a contract for what is being developed; that it may sub optimize for early success by using unscalable capabilities (fourth-generation languages, running all in main memory); or high personnel turnover. See (Boehm and Turner, 2004), pp for an example risk analysis of an agent-based event planning system, ending with a decision to go agile. Fall 2017 ©USC-CSSE

Case 3: Architected Agile
Exploration phase determines Need to accommodate fairly rapid change, emergent requirements, early user capability Low risk of scalability up to 100 people NDI support of growth envelope Nucleus of highly agile-capable personnel Moderate to high loss due to increment defects Example: Business data processing Size/complexity: Medium Anticipated change rate (% per month): 1-10% Criticality: Medium to high NDI support: Good, most in place Organization and personnel capability: Agile-ready, med-high capability Key Stage I activities: Combined Valuation and Architecting phase, complete NDI preparation Key Stage II activities: Architecture-based scrum of scrums Time/build: 2-4 weeks Time/increment: 2-6 months Case 3: For medium-size (20-80 people), medium complexity (reasonably mature and scalable technology; largely compatible shareholders), agile methods can be scaled using an Architected Agile approach with early investment in a largely change-prescient architecture and user/developer/customer team building. For relatively stable projects (0.3-1% change/month), plan-driven methods can be used with low risk. But for higher rates of changes (1-10%/month), a more agile approach is less risky. A risk analysis of a 50-person, medium sized architecture-based agile supply chain management project is provided on pages of (Boehm and Turner, 2004). A number of organizations in such areas as corporate infrastructure, medical, aerospace, and ERP applications have reported significant gains in adaptability and quality of the Architected Agile approach over plan-driven methods for such projects. However, others that had less capable and agile-ready people, less management and customer commitment, and less up-front architecture investment have not. (Boehm, 2007) Fall 2017 ©USC-CSSE

Medical Case Study -- USA
1400 software people; 7M SLOC; 7 sites 4 in Europe, 2 in India 500 medical applications; 500 financial; others Survivability-critical software problems Reliability, productivity, performance, interoperability Sarbanes-Oxley requirements Management receptive to radical change Some limited experimental use of agile methods Led by top software technologist/manager Committed to total change around Scrum and XP Fall 2017 ©USC-CSSE

Medical-USA Adoption Profile
July July 2005 Recruit top people from all sites into core team(s) Get external expert help Develop architecture Early Scrum successes with infrastructure Revise policies and practices Train, reculture everyone Manage expectations July 2005 – July 2006 Begin full-scale development Core teams as mentors Fall 2017 ©USC-CSSE

Key Practices – USA Medical
Include customers and marketers New roles; do’s/don’ts/opportunities; CRACK personnel; full collaboration and teamwork; expectations management Scrum; most XP practices; added company practices 6-12 person teams with team rooms, dedicated servers Hourly smoke test; nightly build and regression test Just-in-time analysis; story-point estimates; fail fast; detailed short-term plans; company architecture compliance Embrace change in applications and practices Global teams: wikis, daily virtual meetings, act as if next-door Release management 2-12 week architecting Sprint Zero; month Sprints; Release Sprint; 1-6 month beta test Next Sprint Zero concurrent with Release Sprint Initiative manager and team Define practices; evolve infrastructure; provide training; guide implementation; evaluate compliance/usage; continuous improvement Fall 2017 ©USC-CSSE

Case 4: Formal Methods Biggest risks: Software/hardware does not accurately implement required algorithm precision, security, safety mechanisms, or critical timing Example: Security kernel or safety-critical LSI chip Size/complexity: Low Anticipated change rate (% per month): 0.3% Criticality: Extra high NDI support: None Organization and personnel capability: Strong formal methods experience Key Stage I activities: Precise formal specification Key Stage II activities: Formally-based programming language; formal verification Time/build: 1-5 days Time/increment: 1-4 weeks Case 4: Formal methods involve the development and verification of a precise mathematical specification of the behavior of a hardware and/of software systems; an implementation of the system in formal-semantics-based languages; and a mathematical proof that the implementation is exactly equivalent to the specification (no more; no less). Such methods are expensive relative to standard commercial practice and require scarce high-capability personnel: commercial organizations charge roughly $ per line of code to formally verify software products. However, they are generally inexpensive relative to a massive product recall, expensive lawsuits, or a massive loss of valuable and confidential information. Current formal methods generally require repeating the expensive mathematical proof process whenever the specification or implementation is changed, making the approach less viable for systems with highly volatile requirements. In general, non-developmental items are not precisely specified and verified enough to be safely used in such systems. Also, formal methods have limited scalability; almost all fully-verified software systems have less than 10,000 lines of code. However, some progress is being made toward modularizing such systems so that implementations and proofs can be built up incrementally via lower-level lemmas and theorems. Fall 2017 ©USC-CSSE

Case 5: Hardware Component with Embedded Software
Biggest risks: Device recall, lawsuits, production line rework, hardware-software integration DCR carried to Critical Design Review level Concurrent hardware-software design Criticality makes Agile too risky Continuous hardware-software integration Initially with simulated hardware Low risk of overrun Low complexity, stable requirements and NDI Little need for risk reserve Likely single-supplier software Case 5: The application classes above have been mostly software-intensive. The differences in economic and risk patterns for hardware-intensive projects (Boehm and Lane, 2007) will create different risk-based special cases of the ICM. Once a project commits to a particular manufacturing approach and infrastructure, the hardware cost of change will be much higher. And the primary costs at risk are in development and manufacturing, particularly if the component is cheaper to replace than to fix or if fixes can be accomplished by software workarounds. (Introduce example). Thus, the ICM Stage I activities of producing and validating detailed specifications and plans are much more cost-effective than for the agile cases above. They will often go beyond the usual level of detail (Critical Design Review versus evidence-based Preliminary Design Review) for an ICM Development Commitment Review (DCR), since so many of the details are likely to be major sources of rework expenditures and delay if wrong. And after Initial Operational Capability (IOC) development, the rework risks generally dictate an incremental low rate initial production phase before a full-rate production phase in Stage II. In case the component is planned to evolve to subsequent hardware-software reconfigurations, the ICM approach of having a concurrent systems engineering team developing specifications and plans for the “N+1st “ increment could be adopted. The length of these later increments will be driven by the product’s marketplace or competitive situation. Fall 2017 ©USC-CSSE

Case 5: Hardware Component with Embedded Software (continued)
Example: Multi-sensor control device Size/complexity: Low Anticipated change rate (% per month): % Criticality: Medium to very high NDI support: Good, in place Organization and personnel capability: Experienced; medium to high capability Key Stage I activities: Concurrent hardware and software engineering; CDR-level ICM DCR Key Stage II activities: IOC Development, LRIP,FRP, concurrent version N+1 engineering Time/build: 1-5 days (software) Time/increment: Market-driven Fall 2017 ©USC-CSSE

Case 6: Indivisible IOC Biggest risk: Complexity, NDI uncertainties cause cost-schedule overrun Similar strategies to case 5 for criticality (CDR, concurrent HW-SW design, continuous integration) Add deferrable software features as risk reserve Adopt conservative (90% sure) cost and schedule Drop software features to meet cost and schedule Strong award fee for features not dropped Likely multiple-supplier software makes longer (multi-weekly) builds more necessary Case 6: More complex hardware intensive systems, such as aircraft, may have a great deal of software that can be incrementally developed and tested, but may have an indivisible hardware IOC that must be developed as a unit before it can be safely tested (e.g., an automobile or aircraft’s braking system or full set of safety-critical vehicle controls). Relative to the cost and duration of the software increments, the indivisible hardware IOC has two significant sources of risk: It cannot fit into smaller software increments It cannot drop required hardware features to meet IOC schedule/cost/quality as independent variable. The first can be addressed by synchronizing the hardware IOC with the Nth software increment (e.g., (Rechtin and Maier, 1997) osculating orbits for hardware and software). If some combination of schedule/cost/quality is truly the project IOC’s independent variable, then one does not want to commit to a most-likely combination of schedule/cost/quality, as there will be a roughly 50% chance of an overrun or quality shortfall in completing the indivisible IOC. Rather, it is better to determine a conservative IOC cost and schedule for meeting the quality objective, and use the difference between the conservative cost and schedule and the most-likely cost and schedule as a risk reserve. The best way to do this is to use the conservative cost and schedule as the IOC target, and to determine a set of desired but non-essential, easy to drop software features that could be developed within the risk reserve. Then, if the most-likely indivisible IOC capability begins to overrun, some desired software features can be dropped without missing the scheduled delivery time and cost. There should also be a strong award-fee incentive for the developers to minimize the number of features that need to be dropped. Fall 2017 ©USC-CSSE

Case 6: Indivisible IOC (continued)
Example: Complete vehicle platform Size/complexity: Medium to high Anticipated change rate (% per month): % Criticality: High to very high NDI support: Some in place Organization and personnel capability: Experienced, medium to high capability Key Stage I activities: Determine minimum-IOC likely, conservative cost; Add deferrable software features as risk reserve Key Stage II activities: Drop deferrable features to meet conservative cost; Strong award fee for features not dropped Time/build: 2-6 weeks (software) Time/increment: months (platform) Fall 2017 ©USC-CSSE

Case 7: NDI-Intensive Biggest risks: incompatible NDI; rapid change, business/mission criticality; low NDI assessment and integration experience; supply chain stakeholder incompatibilities Example: Supply chain management Size/complexity: Medium to high Anticipated change rate (% per month): % Criticality: Medium to very high NDI support: NDI-driven architecture Organization and personnel capability: NDI-experienced; medium to high capability Key Stage I activities: Thorough NDI-suite life cycle cost-benefit analysis, selection, concurrent requirements and architecture definition Key Stage II activities: Pro-active NDI evolution influencing, NDI upgrade synchronization Time/build: 1-4 weeks (software) Time/increment: months (systems) Case 7: Our experiences in developing USC web-service applications between 1996 and 2004 was that they went from 28% of the application’s functionality being delivered by NDI components to 80% (Yang et al, 2005). A similar trend was identified by the 2001 Standish Report, which reported that 53% of the functionality of commercial software applications was being delivered by NDI components in 2000 (Standish, 2001). The economics of NDI-intensive systems dictates a bottom-up versus a top-down approach to system development, in which the capability envelope of the NDI determines the affordable requirements, rather than a top-down requirements-to-capability approach. A large supply-chain management system may need to choose among several NDI candidates each for such functions as inventory control, trend analysis, supplier/customer relations management, transportation planning, manufacturing control, and financial transactions; and evaluate not only the candidates’ cost/performance aspects, but also their interoperability with each other and with the corporation’s legacy infrastructure. Besides NDI assessment, other significant sources of effort can be NDI tailoring, NDI integration, and effect of NDI version volatility and obsolescence; see (Yang et al, 2005). A particular challenge in Stage II is the effect of NDI volatility and obsolescence. Surveys have indicated that commercial NDI products have a new release about every 10 months, and that old releases are supported by the vendor for about 3 releases. Some large systems have had about 120 NDI components, indicating that about 12 components will have new releases each month, and that not upgrading will leave each component unsupported in about 30 months. In such cases, a great deal of attention needs to be paid to upgrade synchronization, and to pro-active NDI evolution influencing. Some large organizations synchronize their NDI upgrades to their major re-training cycles of about months. For additional NDI best practices, see (Wallnau et al, 2002). Fall 2017 ©USC-CSSE

Case 8: Hybrid Agile/Plan-Driven System
Biggest risks: large scale, high complexity, rapid change, mixed high/low criticality, partial NDI support, mixed personnel capability Example: C4ISR system Size/complexity: Medium to very high Anticipated change rate (% per month): Mixed parts; 1-10% Criticality: Mixed parts; medium to very high NDI support: Mixed parts Organization and personnel capability: Mixed parts Key Stage I activities: Full ICM; encapsulated agile in high changed; low-medium criticality parts (often HMI, external interfaces) Key Stage II activities: Full ICM, three-team incremental development, concurrent V&V, next-increment rebaselining Time/build: 1-2 months Time/increment: months Case 8: Some large, user-intensive systems such as C4ISR systems, air traffic control systems, and network control systems have a mix of relatively stable, high-criticality elements (sensors and communications; key business logic) and more volatile, more moderately critical elements such as GUI displays, electronic warfare countermeasures, and interfaces with externally evolving systems. As many acquisitions of this nature have shown, it is highly risky to apply one-size-fits-all processes and incentive structures to these differing classes of elements. Instead, in Stage I, it is important to determine which system elements belong in which class along with the other functions of understanding needs, envisioning opportunities, NDI assessment, scoping the system, and determining feasible requirements and increments performed for complex systems in Stage I; to architect the system to encapsulate the volatile parts for development by agile teams; and to organize to use plan-driven development for the other elements of the overall system. In Stage II, the cycles for the agile teams are likely to be significantly shorter than those for the plan-driven teams. Fall 2017 ©USC-CSSE

Case 9: Multi-Owner System of Systems
Biggest risks: all those of Case 8 plus Need to synchronize, integrate separately-managed, independently-evolving systems Extremely large-scale; deep supplier hierarchies Rapid adaptation to change extremely difficult Example: Net-centric military operations or global supply chain management Size/complexity: Very high Anticipated change rate (% per month): Mixed parts; 1-10% Criticality: Very high NDI support: Many NDIs; some in place Organization and personnel capability: Related experience, medium to high Key Stage I activities: Full ICM; extensive multi-owner teambuilding, negotiation Key Stage II activities: Full ICM; large ongoing system/software engineering effort Time/build: 2-4 months Time/increment:18-24 months Case 9: In this situation, your goal is to integrate a set of existing systems (or guide and evolve the integration of a set of existing systems). These systems are primarily developed, owned, and maintained by an organization other than the one that is attempting to manage and guide the set of systems as a system of systems. Because of the independence of these constituent systems, the SoS organization has little or no formal control over the processes used to maintain and evolve the constituent systems. The SoS may be an enterprise wide business SoS, with the constituents being primarily COTS products along with some legacy applications. Or it may be Department of Defense (DoD) warfighting SoS, where the constituent legacy systems are integrated to increase capabilities on the battle field. Traditional systems engineering (SE) activities are typically tailored for the SoS case to define an SoS architecture, better coordinate the activities of multiple systems in migrating to the SoS architecture, and provide synchronization points for the SoS. In (OUSD AT&L, 2008), pilot studies have shown that many DoD SoS have re-organized the traditional SE activities into a set of seven core elements: 1) translating capability objectives, 2) understanding systems and relationships, 3) assessing performance to capability objectives, 4) developing, evolving, and maintaining SoS design, 5) monitoring and assessing changes, 6) addressing new requirements and options, and 7) orchestrating updates to SoS. Further analysis shows, that these elements map fairly well to the hybid agile/plan-driven case (Case 8) at the SoS level. What makes this case different from Case 8, is that each of the constituent systems is using their own processes, which could be any of the above cases, depending on the scope, complexity, and characteristics of the constituent system. What is key is that the Case 9 extends Case 8 to include information or participation of the constituent systems in the agile planning activities and lets the “battle rhythm” of the constituent system increments guide the SoS plan-driven and V&V activities. Fall 2017 ©USC-CSSE

Case 10: Family of Systems
Biggest risks: all those of Case 8 plus Need to synchronize, integrate separately-managed, independently-evolving systems Extremely large-scale; deep supplier hierarchies Rapid adaptation to change extremely difficult Example: Medical device product line Size/complexity: Medium to very high Anticipated change rate (% per month): 1-3% Criticality: Medium to very high NDI support: Some in place Organization and personnel capability: Related experience, medium to high capability Key Stage I activities: Full ICM; full stakeholder participation in product line scoping; strong business case Key Stage II activities: Full ICM; extra resources for first system, version control, multi-stakeholder support Time/build: 1-2 months Time/increment: months Case 10: Families of systems are typically a set of systems that belong to a product line and can be easily used to customize a solution for a given need. This might be a suite of medical devices or a suite of applications to customer support. This is often the set of systems developed by a vendor that become the NDI components for CASE 7 above. Again, the rigor required for the SoS case is present here. However, in this situation, the family of systems is typically owned and evolved by a single organization/vendor and presents a case where the owning organization has much more control over the evolution of the components of the family of systems, thus possibly reducing some risks and allowing the ICM process to be a little more streamlined. Fall 2017 ©USC-CSSE

ICM and Brownfield Development
Many process models are Greenfield-oriented Requirements→Design→Develop→Test→Operate Failed Greenfield project example Corporate central financial system To replace spaghetti-code collection of COBOL programs Improved ICM Brownfield approach Concurrent new-system definition and legacy system re-engineering Fall 2017 ©USC-CSSE

Failed Greenfield Corporate Financial System
Used waterfall approach Gathered requirements Chose best-fit ERP system Provided remaining enhancements Needed to ensure continuity of service Planned incremental phase-in of new services Failed due to inability to selectively phase out legacy services Dropped after 2 failed tries at cost of $40M Fall 2017 ©USC-CSSE

Legacy Business Services
Legacy Systems Patched, Highly Coupled Financial and Non-Financial Services Legacy Business Services Contract Services Project Services Budgeting Staffing Deliverables Management Billing Earned Value Management Subcontracting Change Tracking Progress Tracking Work Breakdown Structure Scheduling Reqs, Configuration Management Fall 2017 ©USC-CSSE

ICM Approach to Brownfield Engineering
Understanding needs Analysis of legacy system difficulties Envisioning opportunities Concurrently decouple legacy financial and non-financial services, explore new system phase-in and architecture options System scoping and architecting Extract legacy financial, non-financial services Prioritize, plan for incremental financial services phase-in/out Feasibility evidence development Successful examples of representative service extractions Evidence of cost, schedule, performance feasibility Fall 2017 ©USC-CSSE

Result of Legacy Re-engineering
Legacy Business Services Contract Services Project Services Contract Financial Services Billing Subcontract payments ... Contract Non-Financial Services Deliverables mgmt. Terms compliance ... General Financial Services Accounting Budgeting Earned value Payroll ... General Non-Financial Services Progress tracking Change tracking ... Project Financial Services WBS Expenditure categories ... Project Non-Financial Services Scheduling Staffing Reqs CM ... Fall 2017 ©USC-CSSE

Frequently Asked Question
Q: Having all that ICM generality and then using the decision table to come back to a simple model seems like an overkill. If my risk patterns are stable, can’t I just use the special case indicated by the decision table? A: Yes, you can and should – as long as your risk patterns stay stable. But as you encounter change, the ICM helps you adapt to it. And it helps you collaborate with other organizations that may use different special cases. Fall 2017 ©USC-CSSE

Using Risk to Balance Assurance and Agility - Overview
Fall 2017 ©USC-CSSE

Risks Used in Risk Comparison
Environmental risks Technology uncertainties Many stakeholders Complex system-of-systems Legacy compatibility Agility risks Scalability Use of simple design Personnel turnover Too-frequent releases Not enough agile-capable people Plan-driven risks Rapid change Need for rapid results Emergent requirements Not enough discipline-capable people Fall 2017 ©USC-CSSE

Four Examples Three agent-based systems Medical case
Medium – SupplyChain.com application Small – Event Managers application Large – National Information System for Crisis Management (NISCM) application Medical case Each example results in a development strategy based on risk analyses Fall 2017 ©USC-CSSE

SupplyChain.com Profile
Turnkey agent-based supply chain management systems Distributed, multi-organization teams of about 50 people Parts of applications are relatively stable, while others are highly volatile Architectures are driven by a few key COTS packages that are also continually evolving Fall 2017 ©USC-CSSE

SupplyChain.com Home Ground Chart

SupplyChain.com Risks Fall 2017 ©USC-CSSE

SupplyChain.com Mitigations

SupplyChain.com Strategy
VCR FCR Fall 2017 ©USC-CSSE

Event Managers Project
Small startup company Diverse set of smaller agent-based planning applications This project: support for managing the infrastructure and operations of conferences and conventions Widening variety of options and interdependencies, makes an agent-based approach highly attractive Fall 2017 ©USC-CSSE

Event Managers Home Ground Chart

Event Managers Mitigations

NISCM Profile Broad coalition of government agencies and critical private-sector organizations Support cross-agency and public-private sector coordination of crisis management activities Adaptive mobile network, virtual collaboration, information assurance, and information integration technology Private-sector system-of-systems integration contractor Fall 2017 ©USC-CSSE

NISCM Home Ground Chart

Risk Profiles Across Agent-Based Examples

References - I Beck, K., Extreme Programming Explained, Addison Wesley, 1999. Boehm, B., “Some Future Trends and Implications for Systems and Software Engineering Processes”, Systems Engineering 9(1), pp. 1-19, 2006. Boehm, B., Brown, W., Basili, V., and Turner, R., “Spiral Acquisition of Software-Intensive Systems of Systems, CrossTalk, Vol. 17, No. 5, pp. 4-9, 2004. Boehm, B. and Lane J., "21st Century Processes for Acquiring 21st Century Software-Intensive Systems of Systems." CrossTalk: Vol. 19, No. 5, pp.4-9, 2006. Boehm, B., and Lane, J., “Using the ICM to Integrate System Acquisition, Systems Engineering, and Software Engineering,” CrossTalk, October 2007, pp. 4-9. Boehm, B., and Lane, J., “A Process Decision Table for Integrated Systems and Software Engineering,” Proceedings, CSER 2008, April 2008. Boehm, B., “Future Challenges and Rewards for Software Engineers,” DoD Software Tech News, October 2007, pp Boehm, B. et al., Software Cost Estimation with COCOMO II, Prentice Hall, 2000. Boehm, B., Software Engineering Economics, Prentice Hall, 2000. Carlock, P. and Fenton, R., "System of Systems (SoS) Enterprise Systems for Information-Intensive Organizations," Systems Engineering, Vol. 4, No. 4, pp , 2001. Carlock, P., and J. Lane, “System of Systems Enterprise Systems Engineering, the Enterprise Architecture Management Framework, and System of Systems Cost Estimation”, 21st International Forum on COCOMO and Systems/Software Cost Modeling, 2006. Checkland, P., Systems Thinking, Systems Practice, Wiley, 1980 (2nd ed., 1999). Department of Defense (DoD), Defense Acquisition Guidebook, version 1.6, Department of Defense (DoD), Instruction , Operation of the Defense Acquisition System, May 2003. Department of Defense (DoD), Systems Engineering Plan Preparation Guide, USD(AT&L), 2004. Electronic Industries Alliance (1999); EIA Standard 632: Processes for Engineering a System Galorath, D., and Evans, M., Software Sizing, Estimation, and Risk Management, Auerbach, 2006. Hall, E.T., Beyond Culture, Anchor Books/Doubleday, 1976. Fall 2017 ©USC-CSSE ©USC-CSSE 48

References -II Highsmith, J., Adaptive Software Development, Dorset House, 2000. International Standards Organization, Information Technology Software Life Cycle Processes, ISO/IEC 12207, 1995 ISO, Systems Engineering – System Life Cycle Processes, ISO/IEC 15288, Jensen, R. “An Improved Macrolevel Software Development Resource Estimation Model,” Proceedings, ISPA 5, April 1983, pp Krygiel, A., Behind the Wizard’s Curtain; CCRP Publication Series, July, 1999, p. 33 Lane, J. and Boehm, B., "System of Systems Cost Estimation: Analysis of Lead System Integrator Engineering Activities", Information Resources Management Journal, Vol. 20, No. 2, pp , 2007. Lane, J. and Valerdi, R., “Synthesizing SoS Concepts for Use in Cost Estimation”, Proceedings of IEEE Systems, Man, and Cybernetics Conference, 2005. Lientz, B., and Swanson, E.B., Software Maintenance Management, Addison Wesley, 1980. Madachy, R., Boehm, B., Lane, J., "Assessing Hybrid Incremental Processes for SISOS Development", USC CSSE Technical Report USC-CSSE , 2006. Maier, M., “Architecting Principles for Systems-of-Systems”; Systems Engineering, Vol. 1, No. 4 (pp ). Maier, M., “System and Software Architecture Reconciliation,” Systems Engineering 9 (2), 2006, pp Northrop, L., et al., Ultra-Large-Scale Systems: The Software Challenge of the Future, Software Engineering Institute, 2006. Pew, R. W., and Mavor, A. S., Human-System Integration in the System Development Process: A New Look, National Academy Press, 2007. Putnam, L., “A General Empirical Solution to the Macro Software Sizing and Estimating Problem,” IEEE Trans SW Engr., July 1978, pp Rechtin, E. Systems Architecting, Prentice Hall, 1991. Schroeder, T., “Integrating Systems and Software Engineering: Observations in Practice,” OSD/USC Integrating Systems and Software Engineering Workshop, October 2007. Fall 2017 ©USC-CSSE ©USC-CSSE 49

List of Acronyms B/L Baselined
C4ISR Command, Control, Computing, Communications, Intelligence, Surveillance, Reconnaissance CD Concept Development CDR Critical Design Review COTS Commercial Off-the-Shelf DCR Development Commitment Review DI Development Increment DoD Department of Defense ECR Exploration Commitment Review EVMS Earned Value Management System FCR Foundations Commitment Review FED Feasibility Evidence Description FMEA Failure Modes and Effects Analysis FRP Full-Rate Production GAO Government Accountability Office GUI Graphical User Interface Several further acronyms are TBD Fall 2017 ©USC-CSSE ©USC-CSSE 50

List of Acronyms (continued)
HMI Human-Machine Interface HSI Human-System Interface HW Hardware ICM Incremental Commitment Model IOC Initial Operational Capability IRR Inception Readiness Review IS&SE Integrating Systems and Software Engineering LRIP Low-Rate Initial Production MBASE Model-Based Architecting and Software Engineering NDI Non-Developmental Item NRC National Research Council OC Operational Capability OCR Operations Commitment Review OO&D Observe, Orient and Decide OODA Observe, Orient, Decide, Act O&M Operations and Maintenance Fall 2017 ©USC-CSSE ©USC-CSSE 51

List of Acronyms (continued)
PDR Preliminary Design Review PM Program Manager PR Public Relations PRR Product Release Review RUP Rational Unified Process SoS System of Systems SoSE System of Systems Engineering SSE Systems and Software Engineering SW Software SwE Software Engineering SysE Systems Engineering Sys Engr Systems Engineer S&SE Systems and Software Engineering USD (AT&L) Under Secretary of Defense for Acquisition, Technology, and Logistics VCR Validation Commitment Review V&V Verification and Validation WBS Work Breakdown Structure WMI Warfighter-Machine Interface Fall 2017 ©USC-CSSE ©USC-CSSE 52

ICSM Patterns and Common Cases

Similar presentations

Presentation on theme: "ICSM Patterns and Common Cases"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

ICSM Patterns and Common Cases

Similar presentations

Presentation on theme: "ICSM Patterns and Common Cases"— Presentation transcript:

Similar presentations

About project

Feedback