Tom Gilchrist, CQA, CSQE Quality Assurance in ISD and Maintenance Projects How do you do QA when the time it takes is longer that the time to do the work?

10/13/00.1 Disclaimers The views and opinions expressed in this presentation are not necessarily those of my employer. No financial connection with vendors mentioned. Your results may vary.

10/13/00.1 Improvement PROCESS Major determinants of product cost, schedule, and quality performance.

10/13/00.1 Improvement PROCESSTECHNOLOGY Major determinants of product cost, schedule, and quality performance.

10/13/00.1 Improvement PEOPLE PROCESSTECHNOLOGY Major determinants of product cost, schedule, and quality performance.

10/13/00.1 Goals and Objectives Understand…. How ISD and maintenance SDLC work. How to Do SQA in ISD Projects Deployment Sharpen Your SQA Skills

10/13/00.1 Some Level Setting What is Quality? What is SQA? What are Peer Reviews? What is a “Customer”? What are the goals of SQA?

10/13/00.1 Low Quality in Software Unneeded Complexity Rework -Wrong -Unclear -Missing -Extra

10/13/00.1 Software Quality Assurance Check software products and processes to verify that they comply with the applicable procedures and standards. (Process Checks/Audits) Review and measure the quality of software products and processes throughout development. (Peer Reviews) Provide software project management (and other appropriate parties) with the results of reviews and process checks. Work with the software project during early stages to establish plans, standards, and procedures to keep errors from occurring in the first place.

10/13/00.1 Peer Review Methods Walkthroughs Minimal overhead Developer training Quick turnaround MethodsTypical GoalsTypical Attributes Little/no preparation Informal process Meetings No measurement Not a Formal Review! Formal Reviews Detect and report all defects efficiently and effectively. Formal process Known Coverage Rate Moderator Checklists Customer Reviewers Measurement Desk Checks Minimal overhead Quick turnaround Little/no preparation Informal process No measurement No Meetings Not a Formal Review!

10/13/00.1 SQA Plan…Peer Reviews (exit criteria)

10/13/00.1 Customer vs. End User End User: -The person who uses the software product in the value stream Customer: -Anyone who is affected by the end product or intermediate software engineering deliverables being produced (reviewed).

10/13/00.1 Software Development Model ENTRYENTRY EXITEXIT Input Output ENTRYENTRY EXITEXIT Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists Time

10/13/00.1 Exit Criteria Cost Schedule Quality ENTRYENTRY EXITEXIT Input Output ENTRYENTRY EXITEXIT Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists

10/13/00.1 How SDLC’s Work Waterfall SDLC Waterfall SQA Techniques Maintenance & Sustaining SDLC ISD SDLC

10/13/00.1 Waterfall SDLC Feasibility Analysis Requirements Definition Alternatives Analysis Functional Specification Preliminary Design Detailed Design & Construction Verification Installation Software Development Lifecycle

10/13/00.1 Waterfall SDLC Feasibility Analysis Requirements Definition Alternatives Analysis Functional Specification Preliminary Design Detailed Design & Construction Verification Installation Software Development Lifecycle All planning is oriented toward a single delivery date. All analysis and design are done before coding and testing. Model or document driven.

10/13/00.1 The Project Requirements Finished Product SpecsDesignCodeTest Time Project Plan

10/13/00.1 When Waterfall Works Solution is well understood. Developers are well practiced. End-user process being automated is stable and well understood. Mature project management.

10/13/00.1 Idealized Waterfall Sequence Time Requirements Design Code Requirements Design Code Managed by intermediate software engineering deliverable

10/13/00.1 Development Models The new revolution BANG! The old system exists New system being developed Adopted from Tom Gilb, Principles of Software Engineering Management

10/13/00.1 Development Models The new revolution BANG! The old system exists New system being developed Old system New System Adopted from Tom Gilb, Principles of Software Engineering Management

10/13/00.1 The “Plan” Usable Results POC

10/13/00.1 The “Result” Usable Results POC Inc 1

10/13/00.1 The “Final Results” Inc 1 Inc 2 Inc 3 Inc n Inc n+1 POC

10/13/00.1 Phased Development Feasibility Analysis Requirements Definition Alternatives Analysis Functional Specification Preliminary Design Detailed Design & Construction Verification Installation Software Released Incrementally Detailed Design & Construction Verification Installation Detailed Design & Construction Verification Installation Construction Release 1 Construction Release 2 Construction Release 3 This is NOT ISD

10/13/00.1 ISD Phases Approval Proof of Concept Increment

10/13/00.1 ISD SDLC Software Development Lifecycle ApprovalPlan Dev Initial Product Propose Review Grow Product Plan APRINCPOCINC Review Propose Grow Product Plan Review Propose Proof-of-ConceptIncrement 2Increment 1 Plan DoCheck Act POC INC 2 INC 1

10/13/00.1 Idealized ISD/Sustaining Sequence INC Time RequirementsDesignCode INC Incremental Development or Sustaining Original Waterfall Development Triggered by CR’s and/or PR’s

10/13/00.1 INCPOCINC POC Real-World ISD/Sustaining How can one spend the flow time necessary to do Peer Reviews when the flow time to do them is sometimes longer than the increment?

10/13/00.1 SQA in ISD Projects Tools of SQA Strategies & Goals Process vs. Product Creative SQA Opportunities

10/13/00.1 Peer Review Methods Walkthroughs Minimal overhead Developer training Quick turnaround MethodsTypical GoalsTypical Attributes Little/no preparation Informal process Meetings No measurement Not a Formal Review! Formal Reviews Detect and report all defects efficiently and effectively. Formal process Known Coverage Rate Moderator Checklists Customer Reviewers Measurement Desk Checks Minimal overhead Quick turnaround Little/no preparation Informal process No measurement No Meetings Not a Formal Review!

10/13/00.1 Peer Reviews ENTRYENTRY EXITEXIT Output... ENTRYENTRY EXITEXIT Output Input Standards, templates, rules and checklists Standards, templates, rules and checklists Walkthrough Desk Check Walkthrough Desk Check Program Element Creation Program Element Creation Formal Review Formal Walkthrough Formal Desk Check Formal Review Formal Walkthrough Formal Desk Check Formal Review Formal Walkthrough Formal Desk Check Formal Review Formal Walkthrough Formal Desk Check Walkthrough Desk Check Walkthrough Desk Check Kickoff Readiness Kickoff Readiness Kickoff Readiness Kickoff Readiness

10/13/00.1 SQA Interventions Output ENTRYENTRY EXITEXIT Input Program Element Creation StartDuringEnd Kickoff Meeting Readiness Review

10/13/00.1 SQA Interventions Output ENTRYENTRY EXITEXIT Input Program Element Creation StartDuringEnd Walkthrough Desk Check Kickoff Meeting Readiness Review

10/13/00.1 SQA Interventions Output ENTRYENTRY EXITEXIT Input Program Element Creation StartDuringEnd Kickoff Meeting Readiness Review Walkthrough Desk Check Formal Review Formal Walkthrough Formal Desk Check

10/13/00.1 Software Development Model ENTRYENTRY EXITEXIT Input Output ENTRYENTRY EXITEXIT Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists Time

10/13/00.1 Fan-In vs Fan-out

10/13/00.1 Fan-In vs Fan-out

10/13/00.1 Fan-In vs Fan-out

10/13/00.1 Readiness Review Identify people/organizations producing product for the process Identify the people/organization who will do the work Identify and document potential major errors (build/modify checklists) Confirm task/process entry conditions Identify resources Forum for learning Reduce the amount of time needed by the author(s) to produce a deliverable by making sure things are ready to do the job.

10/13/00.1 Generic Document Worklist Before creating the deliverable... -Hold Kickoff Meeting Invite the right participants Plan for a successful document Build/Refine Checklists While working on the deliverable… -Use the checklists -Use Walkthroughs and Desk Checks as needed to reduce rework (major errors). When finished with the deliverable… -As per SQA plan, perform the appropriate Peer review -Send peer review data summary to SQA focal

10/13/00.1 CR or PR Maintenance CR or PR

10/13/00.1 Approved CR or PR Approved CR or PR Maintenance Approved CR or PR

10/13/00.1 Approved CR or PR Targeted to a Release Approved CR or PR Approved CR or PR Maintenance Approved CR or PR Approved CR or PR Targeted to a Release Approved CR or PR Targeted to a Release

10/13/00.1 Maintenance Change Approved CR or PR Targeted to a Release Approved CR or PR Targeted to a Release Integrated Test

10/13/00.1 SQA Sustaining/ISD Strategies Ignore On the Schedule -No Change -Demotion -Creative Techniques -Sampling Off the Schedule -Sampling with Process Improvement

10/13/00.1 Demotion Trigger for new/first creation Peer Review Define less rigorous Peer Review for Updates to deliverables Tracking/Reporting System

10/13/00.1 SQA Plan…Peer Reviews (Exit Criteria)

10/13/00.1 Creative Techniques Demo Peer Reviews Risk Assessment Others...

10/13/00.1 Deliverable Maintenance Output Standards, templates, Rules and Checklists Program Element Modification ENTRYENTRY EXITEXIT Program Element Documentation Input

10/13/00.1 Maintenance Approved CR or PR

10/13/00.1 Maintenance Requirements Approved CR or PR

10/13/00.1 Maintenance RequirementsDesign Approved CR or PR

10/13/00.1 Maintenance RequirementsDesignCode Approved CR or PR

10/13/00.1 Maintenance RequirementsDesignCode Approved CR or PR Integrated Tests

10/13/00.1 RequirementsDesignCode Approved CR or PR Integrated Tests Demo Walkthrough Maintenance

10/13/00.1 “Demo” Peer Reviews Attendees: -Authors with changed documents -Customer requesting change -Downstream Customers (like testing, data conversion, implementation, etc) Process Used: Walkthrough or Formal Review Trigger: Ready for Test

10/13/00.1 Sampling On Schedule Randomness Timing of Decision Type of Peer Review Risk

10/13/00.1 Sample With Improvement INC Design Doc & Checklist INC Design Doc & Checklist Design Doc & Checklist

10/13/00.1 Sample With Improvement INC Design Doc & Checklist INC Formal Review Design Doc & Checklist Design Doc & Checklist

10/13/00.1 Sample With Improvement INC Design Doc & Checklist INC Address Errors Found? Design Doc & Checklist Design Doc & Checklist Formal Review

10/13/00.1 Sample With Improvement INC Design Doc & Checklist INC Process Improvement (Defect Prevention Process) Address Errors Found? Design Doc & Checklist Design Doc & Checklist Formal Review

10/13/00.1 Sample With Improvement INC Design Doc & Checklist INC Process Improvement (Defect Prevention Process) Address Errors Found? Design Doc & Checklist Design Doc & Checklist Formal Review

10/13/00.1 Benefits... Can be “off schedule” Reduce reliance on 100% inspection Manageable Measure Results -Development capability -Improvement capability

10/13/00.1 Possible Triggers Number or Major Errors found. Testing Results Calendar Customer Feedback

10/13/00.1 Beware of “Open Secrets" - these indicate goal “stated” vs “actual” misalignments. Doing things right is punished Nobody takes notice Doing it wrong is rewarded How to Fail...Impede Performance

10/13/00.1 Goals and Objectives Understand…. How ISD and maintenance SDLC work. How to Do SQA in ISD Projects Deployment Sharpen Your SQA Skills

10/13/00.1