Software and System Acquisition

Software and System Acquisition
CS 577b Software Engineering II -- Introduction 18 September 2018 Software and System Acquisition CS 577b Software Engineering II Supannika Koolmanojwong April 11, 2011 © USC Center for Software Engineering

Outline Definition of Software Acquisition
Life Cycle of Software Acquisition Net-Centric Systems of Systems (NCSOS) Top 10 risks of Software-Intensive SOS (SISOS) 04/11/2011 © 2011 USC-CSSE

Software Acquisition US firms spend $250 annually acquiring software (1996) Software investment results in productivity gains and strategic advantages Consideration Determine the software requirements Identify potential suppliers Prepare contract requirements Evaluate and select supplier Manage supplier performance Accept the software 04/11/2011 © 2011 USC-CSSE

Software Acquisition Acquisition Approach Acquisition Source Custom
Package Acquisition Source Internal Resource External Provider 04/11/2011 © 2011 USC-CSSE

Software Acquisition Decision
Paul Nelson , William Richmond , Abraham Seidmann, Two dimensions of software acquisition, Communications of the ACM, v.39 n.7, p.29-35, July 1996 04/11/2011 © 2011 USC-CSSE

Deciding to Develop or Acquire: Options
Criteria Develop Outsource OTS Development cost, speed ** Life cycle cost * • - ** Scarce staff Non- core competency function Core competency function Unique needs Control of evolution 04/11/2011 © 2011 USC-CSSE

Process Pattern Decision Driver
Decision Criteria Importance Architected Agile Use Single NDI NDI-Intensive Services-Intensive Alternatives More than 30% of features available in NDI/NCS 0 – 1 2 – 3 3 – 4 Has a single NDI/NCS that satisfies a complete solution 4 Very unique/ inflexible business process 2 – 4 0 – 1 Life Cycle Need control over upgrade / maintenance Rapid Deployment; Faster time to market 2- 4 Architecture Critical on compatibility 1 – 3 Internet Connection Independence 0 – 4 Need high level of services / performance 0 – 3 0 – 2 Need high security 0 - 4 Asynchronous Communication Access Data anywhere Resources Critical mass schedule constraints Lack of Personnel Capability 0 – 2 Little to no upfront costs (hardware and software) Low total cost of ownership Not-so-powerful local machines 1 – 4 Here is my proposed decision driver that will help the development team to select the appropriate process pattern. The decision criteria is divided into 4 main categories. For example the alternatives - If you have possible NDI / NCS The importance level with the scale of 1-3, low- medium-high, represent how critical this criteria is for your project. The last 4 columns represent the maximum-minimum boundary for each decision category For Note: Decision importance scale varies from project to project Decision Criteria Rating Scale; 0:Very Low; 1:Low; 2: Medium; 3:High; 4:Very High Importance Rating Scale: 1:Low; 2: Medium; 3:High 09/03/2010 @USC CSSE

ICSM Common Cases Special Case Example Size, Complexity
Change Rate (%/Month) Criticality NDI Support Org, Personnel Capability 1. Use NDI Small Accounting Complete 2. Agile Simple business application Low 1 – 30 Low-High Good; in place Agile-ready High 3. Architected Agile Competitive cellphone feature Med 1 – 10 Med-High most in place 4. Formal Methods Security kernel; Safety-critical LSI chip Extra High None Strong formal methods experience 5 SW embedded HW component Multisensor control device 0.3 – 1 Med-Very High In place Experienced; med-high 6. Indivisible IOC Complete vehicle platform Med – High High-Very High Some in place 7. NDI- Intensive Supply Chain Management 0.3 – 3 Med- Very High NDI-driven architecture NDI-experienced; Med-high 8. Hybrid agile / plan-driven system Power grid control Med – Very High Mixed parts: Mixed parts; Med-Very High Mixed parts 9. Multi-owner system of systems Crisis management Very High Many NDIs; some in place Related experience, med-high 10. Family of systems Medical Device Product Line 1 – 3 Related experience, med – high 11. Brownfield Major Upgrade Incremental legacy phase-out NDI as legacy replacement Legacy re-engineering 12. Maintenance On-going maintenance/ small hardware and/or software upgrades to legacy system Changes: Low System: Low –Very High Low-medium average over time Changes: Low-Very High Depends on legacy system Maintenance with some overall knowledge of legacy system 04/11/2011 © 2011 USC-CSSE

Comparison of process areas
CMMI-DEV CMMI - SVC CMMI - ACQ Causal Analysis and Resolution (CAR) Configuration Management (CM) Decision Analysis and Resolution (DAR) Measurement and Analysis (MA) Organizational Process Definition (OPD) Organizational Process Focus (OPF) Project Planning (PP) Work Planning (WP) Project Monitoring and Control Work Monitoring and Control (WMC) Project Monitoring and Control (PMC) Integrated Project Management Integrated Work Management (IWM) Integrated Project Management (IPM) Quantitative Project Management Quantitative Work Management (QWM) Quantitative Project Management (QPM) Supplier Agreement Management (SAM) Product Integration (PI) Requirements Development (RD) Technical Solution (TS) Validation (VAL) Verification (VER) Capacity and Availability Management (CAM) Incident Resolution and Prevention (IRP) Service Continuity (SCON) Service Delivery (SD) Service System Development (SSD) Service System Transition (SST) Strategic Service Management (STSM) Agreement Management (AM) Acquisition Requirements Development (ARD) Acquisition Technical Management (ATM) Acquisition Validation (AVAL) Acquisition Verification (AVER) Solicitation and Supplier Agreement Development (SSAD) Organizational Performance Management (OPM) Organizational Process Performance (OPP) Organizational Training (OT) Process and Product Quality Assurance (PPQA) Requirements Management (REQM) Risk Management (RSKM) 03/02/2011 © 2011 USC-CSSE

Software acquisition phase milestones
Phase initiation milestone Phase completion milestone Steps in software acquisition process Planning Idea is developed Release the request for proposal (RFP) Planning organizational strategy Implementing organizational process Determining the software requirements Contracting RFP is released Sign the contract Identifying potential suppliers Preparing contract requirements Evaluating and selecting the supplier Product implementation Contract is signed Receive the software product Managing supplier performance Product acceptance Software product is received Accept the product Accepting the software Follow-on Software product is accepted Product is no longer in use Using the software 04/11/2011 © 2011 USC-CSSE

Planning phase Process steps -- key inputs and outputs
Planning organizational strategy Acquirer's objectives Strategic areas Quality characteristics of software Organizational strategy for acquiring software General practices Implementing organizational process Steps 3-9 of the process Organizational strategy Contracting practices Organization's policies Establish a software acquisition process for organization Supplier qualification and selection process Determining the software requirements Software definition Supplier evaluation criteria Acquirer and supplier obligations Quality plan and maintenance plan content Software being acquired defined Quality and maintenance plans defined Proposal evaluation standards Contingency plan Request for proposal (RFP) 04/11/2011 © 2011 USC-CSSE

Contracting phase Process steps -- key inputs and outputs
Identifying potential suppliers Supplier performance data from prior contacts Supplier evaluation criteria Definition of software Results being acquired User survey questionnaire Information of software MOTS software/suppliers Candidate list User survey Preparing contract requirements Supplier and acquirer responsibilities Supplier performance standards Acquirer's terms and conditions Quality assurance clauses Payment provisions Acceptance criteria Supplier performance criteria Evaluation and test criteria Tie payments to deliverables Prepared contract Legal counsel review Evaluating and selecting the supplier Supplier proposals Proposal evaluation standards Supplier qualification and selection process Visit supplier facilities User survey results Evaluation of proposals Evaluation of suppliers Qualified suppliers list Supplier selection Negotiated contract 04/11/2011 © 2011 USC-CSSE

Product Implementation – Acceptance – Follow-on phases
Step Inputs* Outputs Product implementation Managing supplier performance Negotiated contract Contract milestones Acquirer's deliverables provided to supplier Monitor supplier progress Supplier performance criteria Work segments approved Completed milestones Software deliverables Reliability and quality measurements Feedback to supplier Product acceptance Accepting the software Acceptance criteria Evaluation criteria Test criteria Maintenance plan Establish acceptance process Acceptance process Acceptable software Usable documentation Follow-on Using the software Documentation Support available Quality plan Contracting practices evaluated Practices to change Practices to retain User satisfaction assessment Supplier performance data 04/11/2011 © 2011 USC-CSSE

Checklists for software acquisition
Checklist 1: Organizational strategy Establish the rights and responsibilities of supplier and acquirer based on the capabilities of each organization Checklist 2: Define the software Quality characteristics Included deliverables Support Checklist 3: Supplier evaluation Financial soundness Experience and capabilities Development and control processes Technical assistance Quality practices Maintenance service Product usage Produce warranty Costs Contracts 04/11/2011 © 2011 USC-CSSE

Checklist 4: Supplier and acquirer obligations Determine interm rights and responsibilities for each milestone Checklist 5: Quality and maintenance plans What the quality plan should contain What the maintenance plan should contain Checklist 6: User survey Operation Reliability Maintenance service Performance Checklist 7: Supplier performance standards Performance criteria Evaluation and test Correction of discrepancies Acceptance criteria 04/11/2011 © 2011 USC-CSSE

Checklist 8: Contract payments Payment provisions for maximum chance for success and reward for the supplier achieving satisfactory progress Checklist 9: Monitor supplier progress Actions that ensure adequate visibility of supplier progress Checklist 10: Software evaluation Functionality Performance Reliability Availability Ease of modification Serviceability Ease of installation Ease of use Adequacy of documentation Cost to acquire and use Checklist 11: Software test Actions needed to maintain adequate control over the software test Checklist 12: Software acceptance Actions for certification process Remedies needed in case the supplier fails to perform 04/11/2011 © 2011 USC-CSSE

Context: “The World is Flat” -Friedman, 2005
Information processing functions can be performed almost anywhere in the world Low-cost global fiber-optic communications Overnight global delivery services Significant advantages in outsourcing to low-cost suppliers But significant risks also Competitive success involves pro-actively pursuing advantages While keeping risks manageable 04/11/2011 © 2011 USC-CSSE

Example Success Patterns - e.g. Walmart, Dell
Identify corporate focus and strategy Focus on core competitive discriminators Develop value chain linking inputs to outputs and values Eliminate non-value-adding activities Identify which can be outsourced Select, engage outsourcing partners Monitor progress vs. plans Adapt to changes in marketplace, technology, environment 04/11/2011 © 2011 USC-CSSE

Preparing for Outsourcing (I) -Gartner, 2004
Self-Awareness Strengths, weaknesses, opportunities, threats (SWOT) Resource Management Future vs. current allocation of resources, accountability Competences Competitive core competencies; outsourceable competencies Organization Multimodal: skills, operations, cross-cutting improvements 04/11/2011 © 2011 USC-CSSE

Preparing for Outsourcing (II) -Gartner, 2004
Roles Less on skills; more on business processes and objectives Change Readiness Not just more adaptive, but more pro-active Communication Less hierarchical, open 04/11/2011 © 2011 USC-CSSE

Core Competencies Include Outsourcing
Sourcing strategy and framework Business strategy and Op Con, technical architecture, parts outsourced Source selection Request for proposals; evaluation criteria, evaluation processes, selection processes Contact negotiation Statement of work, deliverables, award fees, change adaptiveness, principled negotiation Outsourcing management Teambuilding, expectations management, change management, progress monitoring, corrective action Success-critical stakeholder win-win 04/11/2011 © 2011 USC-CSSE

Life Cycle Evolution Risks: -Outsourcing or OTS
Divergence of objectives Accommodation of special needs Erosion of core competencies Continuing-change costs COTS products: new release every 8-10 months; unsupported after 3 releases Renegotiating supplier agreements Lack of technology scalability, evolvability 04/11/2011 © 2011 USC-CSSE

21st Century Acquisition Challenges
Transformational net-centric systems of systems (NCSOS) With broad and deep supplier chains Increasingly rapid change, need for agility Technology, competition, environment changes Requirements emergent rather than pre-specifiable Increasing need for discipline and high assurance Security, safety, performance, interoperability, … Increasing dependence on COTS, reused, and legacy components 04/11/2011 © 2011 USC-CSSE

Complexity of NCSOS Solution Spaces
Size: M lines of software code Number of external interfaces: Number of “Coopetitive” suppliers: Even more separate work locations Depth of supplier hierarchy: 6-12 levels Number of coordination groups: Reviews, changes, risks, requirements, architecture, standards, procedures, technologies, -ilities, integration, test, deployment, personnel, infrastructure, COTS,… Key personnel spend 60 hours/week in meetings Unprecedentedness Emergence Rapid change 04/11/2011 © 2011 USC-CSSE

Rapid Change Trends Global connectivity and competition accelerate change More ripple effects of technology, marketplace changes Increased need for agility, continuous learning Need to balance agility and plan-driven dependability Decline of THWADI (That’s how we’ve always done it) Avoid technical agility, administrative THWADI Hybrid agile/plan-driven processes needed for larger systems Need for incremental processes, methods, tools, skills Need for pro-active technology, marketplace monitoring Education: Need to learn how to learn 04/11/2011 © 2011 USC-CSSE

Failure Mode I: Build-to-Spec Deliverables - Purchasing agent metaphor
Rapid change: heavy spec change traffic, slow contract changes Plus deep supplier chain: slowdowns multiply, changes interact Plus emergent rqts: many initial specs wrong; more changes Plus build-to-spec award fee: supplier inertia Bottom line: late rework, overruns, mission shortfalls 04/11/2011 © 2011 USC-CSSE

Failure Mode II: Sequential Document-Driven Milestones - Waterfall, V-model, MIL-STD-1521B
Rqts. Emergence, COTS: freeze rqts. too early Plus document-completion progress metrics: hasty point solutions, undiscovered risks Plus rapid change: problems with Failure Mode I Bottom line: more late rework, overruns, mission shortfalls 04/11/2011 © 2011 USC-CSSE

Failure Mode III: Hasty Best-of-Breed Source Selection - Overambitious startup schedule
Complexity, emergence: incomplete, unvalidated system architecture, solicitation SOWs Deep, wide supplier chains: incompatible legacy solutions, COTS; critical-path modeling & simulation needs Rapid change: rapid COTS evaluation, version obsolescence GSAW: 8-11 months/release; 3 supported releases Bottom line: serious integration problems, overruns, mission shortfalls 04/11/2011 © 2011 USC-CSSE

Failure Mode IV: Incremental Document-Driven Development - Risk-insensitive “spirals”; ambitious increment schedules High assurance of –ilities: deferral to later increments Complex NCSOS: early-increment architecture inadequate for later-increment –ilities. Rapid change: destabilization of ambitious increment schedules; increment completion delays; next increment destabilized Bottom line: serious security, safety, scalability problems; destabilized development; more rework, overruns, shortfalls 04/11/2011 © 2011 USC-CSSE

Conclusions So Far 20th century contracting practices have many failure modes for 21st century NCSOS High likelihood of serious overruns, mission shortfalls Still good for 20th century type acquisitions Purchasing agent metaphors a poor match to NCSOS acquisition Acquirers need to operate in OODA loops Observe, Orient, Decide, Act New acquisition policies, processes, and practices needed Empower acquisition corps to deal with NCSOS challenges Being worked out in various programs Example provided next 04/11/2011 © 2011 USC-CSSE

Emerging Scalable Spiral Process - For 21st century systems engineering and acquisition
The C4ISR Metaphor for NCSOS Acquisition Role of OODA loops Example: Internet Spiral Example: FCS Win Win Spiral Model Risk-Driven Scalable Spiral Model Increment view Life-cycle view Role of anchor point milestones Acquisition management implications People management implications 04/11/2011 © 2011 USC-CSSE

Acquisition C4ISR Via Spiral OODA Loops – Example: ARPANet/Internet Spiral
Observe new/updated objectives, constraints, alternatives Usage monitoring Competition, technology, marketplace ISR Orient with respect to stakeholders priorities, feasibility, risks Risk/Opportunity analysis Business case/mission analysis Prototypes, models, simulations Operate as current system Accept new system Act on plans, specifications Keep development stabilized Change impact analysis, preparation for next cycle (mini-OODA loop) Decide on next-cycle capabilities, architecture upgrades, plans Stable specifications, COTS upgrades Development, integration, V&V, risk management plans Feasibility rationale Life Cycle Architecture Milestone for Cycle 04/11/2011 © 2011 USC-CSSE

Acquisition Management Implications - I
20th century build-to-spec contracting practices usable in part Good fit for stabilized-increments team But not for rebaselining, V&V teams Time & materials or equivalent Award fee based on cost/effectiveness These apply all the way down the supplier chain Need top-level award fee for cost-effective team balancing No stable distribution of effort 04/11/2011 © 2011 USC-CSSE

Acquisition Management Implications - II
Don’t skimp on system definition phases But avoid analysis-paralysis Use Feasibility evidence generation as progress metric Use more evidence-based source-selection processes Competitive exercise as proof of capability Preceded by multistage downselect Use Schedule/Cost as Independent Variable processes Prioritized features as dependent variable Top priority: transformational empowerment of acquisition corps Education, mentoring, tools, techniques 04/11/2011 © 2011 USC-CSSE

Top 10 SISOS Risks and Spiral Mitigation Strategies
Risk 1:Acquisition Management and Staffing Over-commitment, especially requirements from legacy systems predetermined and allocated to hardware, software, and humans before architecting the SISOS and contracting Mitigation: risk-driven concurrent engineering and evolutionary development may need several spiral cycles of risk resolution to get the right combination of requirements, architecture, system elements, and life-cycle plans lack of rapid response to change; software expertise and decision authority are scattered at low management levels across various project elements a project needs an integrated software and information processing leader reporting directly to the project manager Key staff shortages and burnout; evolutionary acquisition project can go on for years Career path development, mentoring junior staff to provide replacements for key personnel, incremental completion bonuses 04/11/2011 © 2011 USC-CSSE

Risk 2: Requirements/Architecture Feasibility
Committing to a set of requirements or architecture without validating feasibility E.g. one-second-response time requirement a custom $100 million system to meet the one-second requirement when a prototype belatedly showed that a $30 million COTS-based system with a four-second-response time would be sufficient Mitigation: Win-Win Spiral Model’s anchor-point milestone pass/fail criteria and feasibility rationale explicitly address this risk 04/11/2011 © 2011 USC-CSSE

Risk 3:Achievable Software Schedules
More complex, longer schedule, higher risk Mitigation: Schedule feasibility, software cost and schedule estimation Explicit development and critical-path analysis of project activity networks software reuse, COTS, reducing rework, top personnel, and better tools SAIV 04/11/2011 © 2011 USC-CSSE

Risk 4: Supplier Integration
Integration team cohesion Mitigation: making them first-class stakeholders in negotiating establishing early training and team-building activities for selected suppliers; proactively identifying needs for supplier collaboration and networking 04/11/2011 © 2011 USC-CSSE

Risk 5:Adaptation to Rapid Change
Continuous adaptation to change across dozens of suppliers, IPTs, and external interoperators can be completely destabilizing Mitigation: balancing change and stability is incremental development Feasibility Evidence, milestone stabilization, synchronization COTS-watch, and interoperability-watch activities; cross-supplier and cross IPT networking; change-anticipatory architectures; and agile change control and version control capabilities 04/11/2011 © 2011 USC-CSSE

Risk 6: Software Quality Factor Achievability
the most difficult sources of SISOS requirements/architecture feasibility risk Scenario-dependent and complex Mitigation: establish a quality factor trade space by replacing single value quality factor requirements with a range between acceptable and desired values Quality Trade-Off Analysis 04/11/2011 © 2011 USC-CSSE

Risk 7: Product Integration and Electronic Upgrade
SISOS - integrated across supplier hierarchies, IPT domains, computing platforms, vehicle platforms, critical scenarios, operational profiles, system modes, and friendly-to adversarial environments Integration can cause significant delays and rework Potential mistakes: wrong version’s upgrades, update mismatches, out-of-synchronization data Mitigation: up-front involvement of software-oriented integration, test, supportability, and maintenance stakeholders in win-win negotiations architecting the software to accommodate continuous operation and synchronized upgrades 04/11/2011 © 2011 USC-CSSE

Risk 8: Software COTS and Reuse Feasibility
synchronizing COTS upgrades A) Update COTS during implementation B) continue to use unsupported version of COTS Mitigation: establishing key COTS vendors as strategic partners and success-critical stakeholders; proactive COTS-watch experimentation and participation in user groups 04/11/2011 © 2011 USC-CSSE

Risk 9: External Interoperability
Large SISOS - more than 100 independently evolving external systems Mitigation: establishing proactive stakeholder win-win relationships with critical interoperability systems, including memoranda of agreement on interoperability preservation Joint Capabilities Integration and Development System 04/11/2011 © 2011 USC-CSSE

Risk 10:Technology Readiness
Compatible technology maturity Mobile device, software middleware services Mitigation: satisfying a feasibility rationale for the key advanced technologies, including the exercise of models, simulations, prototypes Identifying fallback technology capabilities in case key new technologies prove inadequate for usage 04/11/2011 © 2011 USC-CSSE

Conclusions 20th century acquisition practices inadequate to address 21st century system acquisition challenges Net-centric systems of systems, emergent requirements, rapid change, high assurance Risk-driven Scalable Spiral Process provides framework for coping with 21st century challenges Stabilized increments using 20th century practices New practices for agile rebaselining, thorough V&V Transformational empowerment of acquisition corps a top priority Education, mentoring, tools, techniques 04/11/2011 © 2011 USC-CSSE

Software and System Acquisition

Similar presentations

Presentation on theme: "Software and System Acquisition"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Software and System Acquisition

Similar presentations

Presentation on theme: "Software and System Acquisition"— Presentation transcript:

Similar presentations

About project

Feedback