Software Quality Engineering CS410 Class 2 Software Process Models

SW Process Models A process model is: A process model is not: Methodology - 1) A body of methods, rules, and postulates (accepted theory or statement) employed by a discipline. 2) A particular procedure or set of procedures. (Webster’s 9th Collegiate Dictionary 1988) Life Cycle - synonym for methodology A process model is not: Method

SW Process Models (cont.) Method - A step-by-step technical approach for performing one or more of the major activities identified in an overall methodology for software development. For example: Structured analysis and Object-Oriented analysis are both methods for carrying out the analysis phase of a software development project.

SW Process Models (cont.) Common Process Models Waterfall Prototyping Rapid Throwaway Evolutionary Spiral Iterative Object-Oriented Cleanroom

Waterfall First attempt at controlling SW development. Encourages requirements gathering, ordered stages, and documentation Divide and conquer approach Clearly defined stages Requirements Design Code Test Maintenance

Waterfall (cont.) Deliverables at each stage Entry/Exit criteria for each stage Feedback between stages Emphasis on documents Waterfall model Figure 1.2 p. 6 Figure 2.1 p. 15

Waterfall (cont.) Requirements Gathering/Analysis Stage Meetings with customers Feedback forms Trade shows Conferences Internal requirements (ISO, etc.) Requirements document is deliverable

Waterfall (cont.) Architectural Design Stage Ensure operational consistency with product line(s), and standards Architecture: The structure of the components of a program/system, their relationships, and principles and guidelines governing their design and evolution over time. Gross organization and control structure Protocols for communication, synchronization, and data access

Waterfall Sample Source: MS Project

High-Level Design (HLD) Develop external functions and interfaces User interfaces Application Programming Interface (API) System Interfaces Inter-component Interfaces Data structures Design control structure Ensure functional requirements are met

High-Level Design (HLD) Ensure components fit into system/product structure (feedback to Architecture stage) Ensure component design is complete Ensure external functions can be accomplished (feedback to Requirements stage)

Low Level Design (LLD) Identify parts, modules, macros, include files, etc. that will be written or changed Finalize detail level of design Verify changes in HLD (feedback to HLD) Verify correctness/completeness of HLD (feedback to HLD) Create component test plans (from requirements and design specs)

Code Stage Transform LLD into coded parts Code modules, macros, includes, messages etc. Create component test cases Verify design (feedback to LLD and HLD)

Unit Test Stage Verify code against LLD/HLD Execute all new and/or changed code Branch coverage Statement coverage Logic coverage Verify error messages, error handling, return codes, input parameters May require stubs and drivers

Component Test Test the combined software parts that make up a component after the parts have been integrated into a library (Configuration Management - CM) Objective is to test: External user interfaces (do they meet requirements) Inter-component interfaces APIs

Component Test Functionality Intra-component (module) interfaces Error recovery Messages Concurrency (multi-tasking) Shared Resources Existing functionality (regression)

System-Level Test Four portions System Test System Test, Regression Test, Performance, Usability System Test Test concurrency Stress test Test overall system stability and completeness

System-Level Test Regression Test Performance Test Usability Test Test original (unchanged) functions Performance Test Validate performance of system/product against requirements Record performance metrics Establish baselines (I.e. benchmark) Usability Test Test for usability requirements

Goal of Testing Verification Validation Verify that the system/product we built meets all of the user requirements for usability, performance, reliability, etc. Verify that the intrinsic quality is high and that standards have been met. Validation Validate that the requirements that drove the process were correct.

Waterfall (cont.) Advantages: Disadvantages: Better than an adhoc approach Process, requirements, entry/exit criteria, designs are all documented Disadvantages: Assumption that requirements will not change Heavy emphasis on documents Performance problems detected late in cycle Rework is costly Feedback is not formalized

Prototyping Model Designed to deal with changing or unknown customer requirements. A prototype is a partial implementation of the product expressed either logically or physically with all external interfaces presented. Prototype is ‘used’ by customer to help develop requirements

Prototyping Model Prototyping steps: Figure 2.2 p. 20 Requirements gathering and analysis Quick design Build prototype Customer evaluation Refinement of design and prototype (and requirements) Decision: Iterate or accept Figure 2.2 p. 20

Prototyping Model Key to success: Quick turnaround and inexpensive Throwaway Prototyping Good for: High-risk projects Complex problems Performance concerns “quick and dirty” approach Once customer satisfied, then development of “real thing” begins

Prototyping Model Evolutionary Prototyping Some requirements are known Each iteration evolves (refines) prototype May or may not become final product

Prototyping Model Advantages Disadvantages Good for interface design Good when requirements/problem not understood Problems detected/corrected early Requirements refined and validated Disadvantages Hard to know when to stop Could be costly Tempting to use prototype as product (in throwaway approach)

Spiral Model Refinement of the Waterfall Model Focus is on Risk Management and prototyping Idea: A sequence of steps (cycle) is executed for each level of elaboration. A risk analysis is performed in each cycle. Prototyping is applied to the high-risk areas Figure 2.3 p. 23

Spiral Model Advantages Risk Driven Incorporates prototypes May encourage reuse Eliminates bad alternatives early Incorporates maintenance Allows for quality objectives to be incorporated

Spiral Model Disadvantages Immature process Looser checkpoints (compared to Waterfall with the documents and entry/exit criteria) Requires good understanding of Risk-Analysis and good risk driven decisions

Iterative Model Idea: Start with subset of requirements and develop a subset of the product to meet these requirements. After analysis of product, iteration is done and process is repeated with new requirement subset. Goal: Provide a system/product to user that meets evolving customer needs with improved design based on feedback and learning.

Iterative Model Combination of Waterfall and Prototyping Non-iterative portion like Waterfall Model Iterative portion like Prototyping Model Similar to Spiral model Key elements Test suite developed at each iteration Each iteration is verified and validated Figure 2.4 p. 26

Iterative Model Advantages Disadvantages Complexity broken down Problems detected early Allows evolution of requirements Smaller development teams Disadvantages Risk Analysis not formalized Less parallelism

Object-Oriented Model Paradigm shift away from data and control, to objects which incorporate both data and methods. Eight step process (Branson and Herness) 1. Model the essential system User view Essential activities Identify essential model

Object-Oriented Model 2. Derive candidate essential classes Class and method selection based on the essential model identified in step 1 3. Constrain the essential model Model must be constrained to work within the limits of the target implementation environment 4. Derive additional classes Derive classes/methods specific to implementation environment (I.e. environmentals)

Object-Oriented Model 5. Synthesize classes Organize classes into a class hierarchy 6. Define interfaces Interfaces and class definitions are written based on the class hierarchy 7. Complete design Complete design to include logic, system interactions, method invocations 8. Implement solution

Object-Oriented Model Phases Analysis: steps 1-2 Design: steps 3-6 (can be iterated) Implementation: steps 7-8 Reviews are performed to enhance control of the project Prototypes can be used for the essential model

Object-Oriented Model Advantages OO may promote higher reuse levels Promising new technology Works well on small projects Disadvantages No commonly recognized OO model Training Paradigm shifts Process needs to mature

Cleanroom Model Statistical (mathematical) based Based on correctness verification and incremental development Developers must ‘prove’ designs/code rather than test designs/code Statistical testing replaces unit testing Figure 2.5 p. 33

Cleanroom Model Advantages Disadvantages Developers are motivated to “get it right the first time” - no unit test safety net Promises significant quality improvements Disadvantages All (statistical) testing is based on expected use Learning curve, training requirements Limited use in industry Questions regarding scalability

Pop Quiz In 27 words or less: Define: CMM Defect Rate ISO 9000 Spiral Model DPP What are the advantages of doing a Malcolm Baldrige Quality assessment?

Review CMM Waterfall Model Prototyping Spiral Model High Level vs Low Level Unit vs Component Testing Prototyping Spiral Model

Software Quality Engineering CS410 Class 3 DPP, Process Maturity, Quality Standards

Defect Prevention Process A process for continually improving a software development process It is not a software methodology or model Three main steps Analyze existing defects or errors to trace the root cause Suggest preventative actions to eliminate the defect root cause Implement the preventative actions

Defect Prevention Process Four key elements: Causal analysis meeting - analyze defects for each stage of the development process, identify root cause, identify prevention actions, look for trends Action team - prioritize and implement action items Stage kickoff meeting - level setting, emphasis on quality improvement through action items and process improvements, pitfalls identified and discussed

Defect Prevention Process Action tracking and data collecting - record all problems, root causes, and actions (and action status) DPP should be done at every stage (unlike postmortem which is at end of entire project) DPP addresses ISO 9000 corrective action DPP can be used with any development model, but may be more effective with some (I.e. iterative)

Process Maturity Frameworks and Quality Standards Attempt to measure implementation maturity of a software development process (I.e. how well is it executed), and analyze the quality management systems in place Capability Maturity Model (CMM) Software Productivity Research (SPR) assessment Malcolm Baldrige process/assessment ISO 9000

Capability Maturity Model Developed by Software Engineering Institute (SEI) at Carnegie-Mellon Univ. A (process) maturity assessment framework Based on 85 item questionnaire Five levels of process maturity (p. 40-41) 1. Initial 2. Repeatable 3. Defined 4. Managed 5. Optimizing

SPR Assessment (p. 43) Developed by Software Productivity Research (SPR) Inc. Based on 400 question questionnaire Questions are rated 1-5, and overall process rating is expressed on 1-5 scale Questions focus on Strategic corporate issues Tactical project issues Quality, Productivity, Customer satisfaction

Malcolm Baldrige Assessment Malcolm Baldrige National Quality Award (MBNQA) - “most prestigious quality award in the United States” Seven categories of examination criteria Leadership Information and analysis Strategic quality planning Human resource utilization Quality assurance of products and service Quality results Customer Satisfaction

Malcolm Baldrige Assessment 28 examination items 1000 possible points awarded Scoring is based on three evaluation dimensions Approach - methods used to address the item Deployment - extent to which the approach is applied Results - outcomes and effects Winners need to score 70% or better to be considered Evaluation feedback/suggestions are provided regardless of score

ISO 9000 International Organization for Standards A set of standards for quality assurance Heavy influence in Europe Focus Process quality Process Implementation

ISO 9000 Goal - to achieve certification/registration Formal Audit performed Trial-runs can be performed by independent consulting firms - goal is to get feedback and suggestions First time failures 60% - 70% 20 elements to guideline (p. 46) Heavy emphasis on documentation (p. 47)

ISO 9000 ISO Focus on metrics (statistical techniques) Product Metrics Goals Collect data and report values on regular basis Identify current metric performance level Take action if performance level is unacceptable Establish improvement goals Process Metrics Goals Determine if quality objectives are being met Track adherence to process model and methods Track defect prevention effectiveness