1. Thomas L. Gilchrist tomg@tomgtomg.com Testing Basics Slide Set 2: Software Processes By Tom Gilchrist 2009-v2

2. Thomas L. Gilchrist tomg@tomgtomg.com Tonight... Process Quality Assurance & Quality Control Life Cycle Terminology Checklists

3. Thomas L. Gilchrist tomg@tomgtomg.com Process

4. Thomas L. Gilchrist tomg@tomgtomg.com Individual Professionalism InOut ?

5. Thomas L. Gilchrist tomg@tomgtomg.com Black Box EXITEXIT ENTRYENTRY InOut

6. Thomas L. Gilchrist tomg@tomgtomg.com EXITEXIT ENTRYENTRY InOut Glass Box

7. Thomas L. Gilchrist tomg@tomgtomg.com Work Process Definition Process Input Process Output What must be provided? Who provides it? What are the acceptance Standards? ("Clear-Cut" i.e. Unambiguous and measurable) What must be delivered? Who uses it? What are the deliverable Standards? ("Clear-Cut" i.e. Unambiguous and measurable) Objectives: What specific products/services are created or tansformed? Trigger: When does it start? ENTRYCRITERIAEXITCRITERIAWORKPROCESS Supplier, Input, Process, Output, Customer SIPOC – Black Box

8. Thomas L. Gilchrist tomg@tomgtomg.com Expanded Black Box Worksheet SupplierInputConditionProcessCustomerOutputCondition

9. Thomas L. Gilchrist tomg@tomgtomg.com Software Development Model ENTRYENTRY EXITEXIT Input Output ENTRYENTRY EXITEXIT Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists Time

10. Thomas L. Gilchrist tomg@tomgtomg.com Exit Criteria Cost Schedule Quality ENTRYENTRY EXITEXIT Input Output ENTRYENTRY EXITEXIT Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists

11. Thomas L. Gilchrist tomg@tomgtomg.com ENTRYENTRY EXITEXIT Input Output... ENTRYENTRY EXITEXIT Output Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists Generic Work Process Flow Intermediate Program Element

12. Thomas L. Gilchrist tomg@tomgtomg.com Understanding Rework WORK TO BE DONE WORK REALLY DONE Work Being Done People Productivity *From IEEE Engineering Management Review, Winter 1994

13. Thomas L. Gilchrist tomg@tomgtomg.com Understanding Rework WORK TO BE DONE WORK REALLY DONE UNKNOWN REWORK Work Being Done People Productivity Quality *From IEEE Engineering Management Review, Winter 1994

14. Thomas L. Gilchrist tomg@tomgtomg.com Understanding Rework WORK TO BE DONE WORK REALLY DONE UNKNOWN REWORK Work Being Done People Productivity Quality Rework Discovery KNOWN REWORK *From IEEE Engineering Management Review, Winter 1994

15. Thomas L. Gilchrist tomg@tomgtomg.com Understanding Rework WORK TO BE DONE WORK REALLY DONE UNKNOWN REWORK Work Being Done People Productivity Quality Rework Discovery KNOWN REWORK *From IEEE Engineering Management Review, Winter 1994

16. Thomas L. Gilchrist tomg@tomgtomg.com Understanding Rework WORK TO BE DONE WORK REALLY DONE UNKNOWN REWORK Work Being Done People Productivity Quality Rework Discovery KNOWN REWORK *From IEEE Engineering Management Review, Winter 1994 Why is rework bad?

17. Thomas L. Gilchrist tomg@tomgtomg.com QUALITY COST SCHEDULE Scope Manpower Build-up Productivity –Process –Technology –People Management

18. Thomas L. Gilchrist tomg@tomgtomg.com Organizational Entropy Ron Nelson "Left to their own, organizations tend to migrate toward individual practice". Experiment: Put a hot brick on a table. Come back tomorrow. Do you expect it to still be hot? Organizations behave similarly.

19. Thomas L. Gilchrist tomg@tomgtomg.com Professional Practices How are Professional Practices Communicated? They have no benefit if they are not used. –Penetration –Compliance –Outcome of Use They will be abandoned if not monitored and improved.

20. Thomas L. Gilchrist tomg@tomgtomg.com Some Organizational Objectives A Quality Goal Metrics Initiatives; Measure Quality SEI Maturity Goals Better Schedule Predictions Lower/Control Costs Improve Efficiency Shorter Cycle Times Measure and Reduce Variation

21. Thomas L. Gilchrist tomg@tomgtomg.com Some Organizational Objectives A Quality Goal Metrics Initiatives; Measure Quality SEI Maturity Goals Better Schedule Predictions Lower/Control Costs Improve Efficiency Shorter Cycle Times Measure and Reduce Variation What Are Yours?

22. Thomas L. Gilchrist tomg@tomgtomg.com Quality Assurance & Quality Control

23. Thomas L. Gilchrist tomg@tomgtomg.com Quality Control Processes and methods used to check work and observe whether or not standards/requirements are met. (QAI) –(Focus on inspections, testing, and removal of defects before shipment of products.)

24. Thomas L. Gilchrist tomg@tomgtomg.com Quality Assurance The Set of support activities needed to provide adequate confidence that processes are established and continuously improved in order to produce products that meet specifications and are fit for use. (QAI) –(Management of processes not management of results.)

25. Thomas L. Gilchrist tomg@tomgtomg.com Software Quality Assurance Check software products and processes to verify that they comply with the applicable procedures and standards. (Process Checks/Audits) Review and measure the quality of software products and processes throughout development. (Static & Dynamic Testing) Provide software project management (and other appropriate parties) with the results of the testing. Work with the software project during early stages to establish plans, standards, and procedures to keep errors from occurring in the first place.

26. Thomas L. Gilchrist tomg@tomgtomg.com Correct Detect Prevent Cost of Quality Low

27. Thomas L. Gilchrist tomg@tomgtomg.com Correct Detect Prevent Correct Detect Prevent Reduction through an emphasis on prevention. Reduce Rework Cost of Quality Low

28. Thomas L. Gilchrist tomg@tomgtomg.com Cost and Impact of Errors/Defects Errors are introduced in all activities. The longer errors go undetected and unresolved, the more they cost to fix. Rework is inconsistent with quality objectives.

29. Thomas L. Gilchrist tomg@tomgtomg.com Error/Defect Classification Number of Errors MinorMajor"Show Stoppers" Find and remove as many Major and Show Stopper errors as early as possible. Error Severity

30. Thomas L. Gilchrist tomg@tomgtomg.com Severity Major Error An issue or defect which will probably have significantly increased costs to find and fix later in the development process or operational stages. Could conceivably make the product being produced unusable for it's intended purpose. Minor Error An issue or defect that is not major. The costs of fixing later is not significantly higher than fixing now.

31. Thomas L. Gilchrist tomg@tomgtomg.com Validation and Verification Validation –Are we building the right product? Is the work product suitable for its intended use in its intended environment? Verification –Are we building the product right? Does the work product satisfy the conditions and constraints imposed on it by other work products and work processes?

32. Thomas L. Gilchrist tomg@tomgtomg.com Life Cycle Terminology

33. Thomas L. Gilchrist tomg@tomgtomg.com Process A set of activities that represent the way work is performed. –They communicate how job tasks are to be done. –Assure some level of predictability. –Documented. –Provide a basis for measurement and improvement.

34. Thomas L. Gilchrist tomg@tomgtomg.com Terminology Method/Task: A sequence of steps for carrying out a process. Metric: A measurable value of some characteristic.

35. Thomas L. Gilchrist tomg@tomgtomg.com Big Bang Model Unit System

36. Thomas L. Gilchrist tomg@tomgtomg.com ENTRYENTRY EXITEXIT Input Output... ENTRYENTRY EXITEXIT Output Input Program Element Creation Program Element Creation Standards, templates, rules and checklists Standards, templates, rules and checklists Generic Work Process Flow Intermediate Program Element

37. Thomas L. Gilchrist tomg@tomgtomg.com Idealized Waterfall Sequence Time Requirements Design Code Requirements Design Code Managed by intermediate software engineering deliverable Test

38. Thomas L. Gilchrist tomg@tomgtomg.com Workflow for IT Type Systems Integrated Software & Systems Engineering Curriculum (iSSEc) project ver 0.5

39. Thomas L. Gilchrist tomg@tomgtomg.com Waterfall SDLC Feasibility Analysis Requirements Definition Alternatives Analysis Functional Specification Preliminary Design Detailed Design & Construction Verification Installation Software Development Lifecycle

40. Thomas L. Gilchrist tomg@tomgtomg.com The Project Requirements Finished Product SpecsDesignCodeTest Time Project Plan

41. Thomas L. Gilchrist tomg@tomgtomg.com Where is rework introduced? Requirements Finished Product SpecsDesignCodeTest Time Project Plan

42. Thomas L. Gilchrist tomg@tomgtomg.com Where is rework found? Requirements Finished Product SpecsDesignCodeTest Time Project Plan

43. Thomas L. Gilchrist tomg@tomgtomg.com Where is rework fixed? Requirements Finished Product SpecsDesignCodeTest Time Project Plan

44. Thomas L. Gilchrist tomg@tomgtomg.com The Quality Lever Requirements Design Coding Development Testing Acceptance Testing Improve Product Production 40-100X 30-70X 15-40X 10X 3-6X 1X Data from Boehm: Software Engineering Economics, 1981 High Visibility/Reward Low Visibility/Reward

45. Thomas L. Gilchrist tomg@tomgtomg.com Evolutionary Development Build 1 (Life Cycle Inside) Build 2 (Life Cycle Inside) Build 3 (Life Cycle Inside) Time

46. Thomas L. Gilchrist tomg@tomgtomg.com Idealized ISD/Sustaining Sequence INC Time RequirementsDesignCode INC Incremental Development or Sustaining Original Waterfall Development Triggered by CR’s and/or PR’s

47. Thomas L. Gilchrist tomg@tomgtomg.com INCPOCINC POC Real-World ISD/Sustaining How can one spend the flow time necessary to do testing when the flow time needed is sometimes longer than the increment?

48. Thomas L. Gilchrist tomg@tomgtomg.com Spiral Model Requirements Plan, Life Cycle Plan Risk Analysis Proto- type 1 Simulations, models, benchmarks Concept of operation Risk Analysis Proto- type 2 Software Requirements validation Development Plan Integration & Test Plan Plan next phases Proto- type32 Operational Prototype Software product design Design validation and verification Determine objectives, alternatives, constraints Commitment partition Risk Analysis Evaluate alternatives, identify, resolve risks Detailed Design Code Unit Test Integration Test Acceptance Test Implementation

49. Thomas L. Gilchrist tomg@tomgtomg.com Rapid Prototyping Model Requirement Gathering Quick Design Build Prototype Evaluate & Refine Requirements Engineer Product

50. Thomas L. Gilchrist tomg@tomgtomg.com PDCA Cycle Plan Do Check Act

51. Thomas L. Gilchrist tomg@tomgtomg.com PDCA Cycle Plan Do Check Act Determine Goals & Targets Determine Methods for Reaching Goals Educate and Train in Methods Implement Methods Examine Results Review Materials Maintain Correct Improve

52. Thomas L. Gilchrist tomg@tomgtomg.com The “V” Model Requirements Design Code Integration Test System Test Unit Test Relationship between Software and Testware

53. Thomas L. Gilchrist tomg@tomgtomg.com Test Life Cycle Model Operational Business Need Define Requirements Design System Build System Acceptance Test System Test Integration Test Unit Test Validate Verification Integration Component Testing happens throughout the software life cycle. Adopt a Testing Life Cycle mentality. Static Testing Dynamic Testing

54. Thomas L. Gilchrist tomg@tomgtomg.com Checklist

55. Thomas L. Gilchrist tomg@tomgtomg.com Process Checklists Entry Criteria Checklist Exit Criteria Checklist Defines and Describes

56. Thomas L. Gilchrist tomg@tomgtomg.com Black Box EXITEXIT ENTRYENTRY InOut

57. Thomas L. Gilchrist tomg@tomgtomg.com Initial Level Informal Testing Do Work Down- stream Customers Input Source Materials Errors Found Process Product Entry Criteria Loosely Defined Exit Criteria For Quality Not Defined Work Process ENTRYENTRY EXITEXIT

58. Thomas L. Gilchrist tomg@tomgtomg.com Initial Level Informal Testing Do Work Down- stream Customers Re-Work Input Source Materials Errors Found Process Product Entry Criteria Loosely Defined Exit Criteria For Quality Not Defined Work Process ENTRYENTRY EXITEXIT

59. Thomas L. Gilchrist tomg@tomgtomg.com ENTRYENTRY EXITEXIT Input Output... ENTRYENTRY EXITEXIT Output Input Before the Class During the Class Text Book, Rules and Standards Work Process Flow Intermediate Program Element StudentsInstructor Text Book, Rules and Standards

60. Thomas L. Gilchrist tomg@tomgtomg.com Fan-in, Fan-out