SE 470 Software Development Processes James Nowotarski 19 May 2003

Course Map Overview. Introduction. History Content. Rational Unified Process. Extreme Programming Implementation. Tools, Training, Roles. CMM, Metrics. Selection & Evaluation Briefings (Term Papers) Assignments Quizzes Week Memorial Day

Recap the Capability Maturity Model (CMM) Understand key issues of methodology selection and evaluation Discuss term papers (last chance) Today’s Objectives

Topic Duration CMM Recap15 minutes CMM Jeopardy45 minutes CMM Quiz35 minutes *** Break15 minutes Methodology Selection: Organization50 minutes Methodology Selection: Project20 minutes Project Methodology Adequacy15 minutes Today’s agenda

Topic Duration CMM Recap15 minutes CMM Jeopardy45 minutes CMM Quiz35 minutes *** Break15 minutes Methodology Selection: Organization50 minutes Methodology Selection: Project20 minutes Project Methodology Adequacy15 minutes Today’s agenda

CMM Levels Repeatable (2) Repeatable (2) Defined (3) Defined (3) Managed (4) Managed (4) Optimized (5) Optimized (5) Initial (1)

Key process areas (KPAs) Maturity levels Process capability Indicate Key process areas Contain Key practices Contain Goals Achieve

CMM Appraisal Method Team Selection Response Analysis On-site visit Interviews & document reviews Findings based on the CMM 1 Maturity Questionnaire KPA Profile 6

Appraisal Methods Software Process Assessments (SPA) –Performed in open, collaborative environment –Focuses on improving the organization’s software process –Now called CMM-Based Appraisal for Internal Process Improvement (CBA-IPI) Software Capability Evaluations (SCE) –Performed in a more audit-oriented environment –Focuses on identifying risks associated with a contractor –Team’s recommendation will help select contractors or set fees

CMM Issues in the Real-World “Level envy” Areas not addressed –Business strategy and linkage to IT –Operations, help desk, support –Management of the IT human resource –Application portfolio –Tools –Risk Many question whether it is worth the effort to pursue levels 4 and 5

CMM Maturity Profile 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% Initial 19.3% Repeatable 43.2% Defined 23.4% Managed 7.3% Optimized 6.8% % of Organizations August 2002 Based on assessments from of 1124 organizations

USA and Offshore Profiles 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0% Initial 23.3% RepeatableDefinedManagedOptimized % of Organizations August 2002 Based on 645 U.S. organizations and 479 offshore organizations 14.0% 47.4% 37.6% 22.0% 25.3% 5.3% 10.0% 2.0% 13.2% USA Offshore

CMM Marketing tool to win clients, who are based predominantly in US and Europe Clients using Indian service providers should have certain key processes in place: –service level agreements –identifying business requirements –scoping requirements –managing changes Approximately 50 of the 70 or so publicly-acknowledged Level 5 CMM-certified organizations are in India

Time to Move Up to to to to 5 Largest observed value that is not an outlier 75th percentile Median (50th percentile) 25th percentile Smallest observed value that is not an outlier Recommended time between appraisals (18-30 mos) Number of months to move to next maturity level

CMM-based Software Process Improvement (SPI) Time and cost often exceed expectations –18-24 months to advance 1 level –Can cost $2K per software engineer per year –1-2% full-time resources (e.g., 5-10 in a 500-person organization) –2-4% of rest of organization’s time Difficult KPAs –Planning and tracking Key success factors –Senior management is engaged –Participation and buy-in at all levels, including middle management and technical staff –Clearly stated, well understood SPI goals –Clear assignment of responsibility –SEPG staffed by highly respected people

Software Process Improvement Models ISO ISO TickIT IT specific models A number of models enable software development organizations to compare their practices to a set of “best practices” Total Quality Management (TQM) Six Sigma General models

Software Process Improvement Models International collaborative effort (including SEI) Sparked by an investigative study sponsored by the U.K. Ministry of Defense (MOD) Objective: To develop a standard in the area of software process assessment –establish a common framework for expressing the process capability ratings resulting from a conformant assessment –provide a migration path for existing assessment models and methods wishing to become conformant ISO 15504

Software Process Improvement Models Over time, proliferation of CMMs: –Capability Maturity Model for Software (SW-CMM ® ) –Systems Engineering Capability Model (SECM) (may or may not include software) –Integrated Product Development Capability Maturity Model (IPD-CMM) –Software acquisition Many organizations wish to integrate improvement efforts across disciplines Differences among these multiple models made integration difficult SEI developed common framework to support integration of current and future discipline-specific maturity models The common framework is called the Integrated CMM (CMMI) –“Each CMMI model is designed to be used in concert with other CMMI models, making it easier for organizations to pursue enterprise-wide process improvement at their own pace” The Integrated CMM (CMMI) has emerged as the latest thinking from SEI

CMMI Incorporates and extends: –Capability Maturity Model for Software (SW-CMM ® ) –Systems Engineering Capability Model (SECM) –Integrated Product Development Capability Maturity Model (IPD-CMM) –Supplier sourcing CMMI-SW model released August 2002 SW-CMM ® to be sunsetted by end of 2003 (stay tuned) What’s different about CMMI-SW: –Stronger linkage to business objectives and customer needs –Greater alignment with relevant ISO standards –Standard CMMI Appraisal Method for Process Improvement (SCAMPI SM ) V1.1 as a replacement for CMM-Based Appraisal for Internal Process Improvement (CBA IPI) and Software Capability Evaluation (SCE SM ) Objective: To develop a standard in the area of software process assessment –establish a common framework for expressing the process capability ratings resulting from a conformant assessment –provide a migration path for existing assessment models and methods wishing to become conformant CMMI integrates process improvement models for product and service development and maintenance

CMMI What’s different about CMMI-SW: –Stronger linkage to business objectives and customer needs –Greater alignment with relevant ISO standards –Standard CMMI Appraisal Method for Process Improvement (SCAMPI SM ) V1.1 as a replacement for CMM-Based Appraisal for Internal Process Improvement (CBA IPI) and Software Capability Evaluation (SCE SM ) –New process areas –New names for maturity levels Still outside scope of CMMI: –People CMM (P-CMM) –Software Acquisition CMM (SA-CMM)

CMMI New names for maturity levels: LevelOldNew 1Initial 2RepeatableManaged 3Defined 4ManagedQuantitatively Managed 5Optimizing

CMMI ts/02.reports/02tr029.html (600+ page pdf) For more information

Topic Duration CMM Recap15 minutes CMM Jeopardy45 minutes CMM Quiz35 minutes *** Break15 minutes Methodology Selection: Organization50 minutes Methodology Selection: Project20 minutes Project Methodology Adequacy15 minutes Today’s agenda

Topic Duration CMM Recap15 minutes CMM Jeopardy45 minutes CMM Quiz35 minutes *** Break15 minutes Methodology Selection: Organization50 minutes Methodology Selection: Project20 minutes Project Methodology Adequacy15 minutes Today’s agenda

Topic Duration CMM Recap15 minutes CMM Jeopardy45 minutes CMM Quiz35 minutes *** Break15 minutes Methodology Selection: Organization50 minutes Methodology Selection: Project20 minutes Project Methodology Adequacy15 minutes Today’s agenda

Methodology Selection Process 1. Pre-RFP 2. RFP3. Proposal Submissions 4. Proposal Evaluations 5. Vendor Selection 6. Procurement Method 7. ROI Analysis8. Negotiate Contract Objective: Identify best solution to meet stated business need while minimizing cost and risk

1. Pre-RFP Also known as Requirements Definition Preliminary analysis for management (not given to vendor) Serves as basis for Request for Proposal (RFP) and evaluation criteria May be a simple presentation (small firm) or a formal report Most important step in the system procurement process

1. Pre-RFP Factors to consider –Organization and project sizes –Locations/Degree of distribution –Diversity of application developers –IT culture –Cost/Budget Request for Information (RFI) –Vendors are called/consultants consulted/research conducted –Breadth of alternatives is identified –Vendors identified to participate in future stages

1. Pre-RFP Sections of the Pre-RFP Report Problem statement –Current state –Gaps –Risks Alternative solutions Ratings (of each alternative) Range of costs and benefits Recommended alternative and rationale

1. Pre-RFP Categories of Methodology Providers Developed and delivered by consultants Software tool vendors Industry consortia or other groups

2. RFP Blueprint Confirms in detail the exact requirements stated in both business and technical terms Limited distribution (e.g., 3-5 vendors) –Protect confidentiality –Keep selection process manageable

2. RFP Business need/Requirements Statement of Work to be done –Methodology characteristics –Implementation plan –Training strategy –Maintenance and support –Cost/Budget Procedural details –Form and structure of proposal –Schedule (meetings, demos, selection) –Key contacts Selection criteria Sample Contents of an RFP

2. RFP Multiple solutions available that will fit the need Multiple vendors can provide the same solution Corporate policy requires it When should an RFP be used?

2. RFP RFP team develops better understanding of their needs from both a technical and business perspective Compels vendors to create competitive solutions Does not favor one vendor over another (in theory) “Everybody singing from same hymn book” –Vendors working from same set of rules and requirements Facilitates evaluation of competitive solutions –Provides a foundation on which to base a more rigorous evaluation of a vendor Advantages of Using an RFP

3. Proposal Submissions Forums to answer vendor questions (written, oral) –Vendor conferences before proposal submission Response content and format Sometimes requires "proof" statements, such as "This feature was implemented 12 months ago and is currently installed at 10 sites. Names and addresses are provided in the reference section."

4. Proposal Evaluations Business and Technical Solutions –Rating scale: (0=unresponsive,..., 5=exceptional) Vendor qualifications (site visits, reference checks) Preliminary cost, value, and risk analysis Cost proposal may be a separate document from technical proposal Vendor demo Personnel assignment May be a two-stage process, with only a “short list” of 2-3 vendors doing demos and making “best and final” offers Question: Who are the key stakeholders in this process?

4. Proposal Evaluations Ability and track record of vendor to meet commitments? Satisfaction levels of vendor’s current customers, particularly long-term customers? Vendor’s track record for providing support? Any litigation pending against vendor? Is the vendor financially stable? In selecting a vendor, there are major management and technical considerations Management considerations

4. Proposal Evaluations Ability and track record of vendor to meet technical challenges of project? Level of vendor’s expertise in your industry (e.g., Financial Services)? With your type of IT organization? Quality of vendor’s past work? Are metrics available? In selecting a vendor, there are major management and technical considerations (cont.) Technical considerations

Content –building blocks (processes, deliverables, etc.) –pre-defined routes Delivery vehicle (e.g., browser) Tools for authoring and publishing content Tools for applying the methodology to a specific project –project planning and estimating tools –process management tools –project management tools Deliverable templates tightly coupled with a development and/or execution platform Training and support Maintenance What does a methodology product consist of?

Content Methodology tools Fit with development tools Deployment and Support Vendor Cost Small groups Identify criteria to be considered in each of the following categories when considering methodology products for your organization

Small groups Examples: Content - Which life cycle models are supported? - Does the methodology support component-based development?

5. Vendor Selection Vendor site visits Weighted score method Final cost, value, and risk analysis –Costs -- one-time vs. recurring -- fixed vs. variable –Benefits -- tangible vs. intangible

6. Procurement Methods Purchase –not that popular because of fear of obsolescence –longest-term commitment of these 3 methods Subscription –usually months in duration –often done with an option to buy

7. ROI Analysis Metrics to justify investment in methodology remains a weak link Vendors supply very little help for customers developing ROI models for methodology Managers must commit to developing custom cost, benefit, and risk models to justify their methodology investments

8. Contract Negotiation Do’s –Include vendor responses to RFP in the contract –Consider intellectual property issues –Leverage outside expertise in negotiations –Provide incentives/penalties Don’ts –Buy vaporware instead of proven solutions –Purchase low bid unless the value is there –Settle on final offer prematurely

Topic Duration CMM Recap15 minutes CMM Jeopardy45 minutes CMM Quiz35 minutes *** Break15 minutes Methodology Selection: Organization50 minutes Methodology Selection: Project20 minutes Project Methodology Adequacy15 minutes Today’s agenda

Context Content Course of Action Methodology Selection: Project The Course of Action (the selection of the system development process) is driven by: –Content factors: the result to be achieved –Context factors: internal and external influences

Decision Tree Buy vs. Build Packaged Agile Trad. Asset vs. Non- Asset Technology Generation Asset-Based Legacy See Table 1 Context 1 See Table 2 Context 2 See Table 3 Context 3 See Table 4 Context 4 Buy Build Agile vs. Traditional Asset Non-Asset Netcentric Legacy

Table 1 - Buy versus Build Content Factors

Context 1 - Buy versus Build Context Factors –Is a suitable package available? –Availability of tools (development tools, plug- ins, etc.) –Previous success with system development process –Management understanding of and commitment to one of the approaches

Table 2 - Asset versus Non-Asset Content Factors

Context 2 - Asset versus Non-Asset Context Factors –Is suitable asset available? –Availability of tools (development tools, plug- ins, etc.) –Availability of asset-experienced resources –Previous success with system development process –Management understanding of and commitment to one of the approaches

Table 3 - Technology Generation Content Factors

Context 3 - Technology Generation Context Factors –Availability of tools (development tools, plug- ins, etc.) –Previous success with system development process –Need for the client to be considered leading edge –Management understanding of and commitment to one of the approaches

Table 4 - Agile vs. Traditional Content Factors

Context 4 - Agile vs. Traditional Context Factors –Availability of tools (development tools, plug-ins, etc.) –Previous success with system development process –Experience with object technology –Need for the client to be considered leading edge –Result will be used as an asset –Management understanding of and commitment to one of the approaches

Words of Caution A system development process is not all-exclusive. In all probability the project will use concepts, tasks, deliverables from more than one system development process One project, however, will follow primarily one system development process For specific projects some factors may not apply, or additional factors can have an influence on the decisions.

Topic Duration CMM Recap15 minutes CMM Jeopardy45 minutes CMM Quiz35 minutes *** Break15 minutes Methodology Selection: Organization50 minutes Methodology Selection: Project20 minutes Project Methodology Adequacy15 minutes Today’s agenda

Project Methodology Adequacy Content Was there an adequate set of systems development methodology and standards to guide project activities? –What existed? –Was it adequate? Key Questions Deployment Was the set of systems development methodology and standards rolled out to developers so that they had awareness and understanding of how to use them? Usage Did project personnel adhere to the standards and methodology?

First half of term papers to be presented Next Class: June 2

Extra Slides