Copyright (c) 2003 Howard E. Dow1 Results from Inspecting Test Automation Scripts Howie Dow
Copyright (c) 2003 Howard E. Dow2 Agenda Audience - Project - Terminology What the team did Results and data Benefits, pitfalls and how to Conclusions
Copyright (c) 2003 Howard E. Dow3 Car Cah Parked Pahked Rs ???? Idea Idear
Copyright (c) 2003 Howard E. Dow4 Target Audience Engineers Project leaders Ideally Trained on inspections Some inspection experience
Copyright (c) 2003 Howard E. Dow5 Project Call center system Functional tests Goals Defect free test scripts On time Increase skill level
Copyright (c) 2003 Howard E. Dow6 Skill levels
Copyright (c) 2003 Howard E. Dow7 Terminology Inspection = Software Inspection = Formal Inspection = “Fagan Style” Inspection Reviews Walk-Throughs Any technique
Copyright (c) 2003 Howard E. Dow8 Terminology Inspection characteristics Goals Rules, Roles, Responsibilities Process
Copyright (c) 2003 Howard E. Dow9 Goals of Inspections Find defects Collect data Archive data Use data Learn
Copyright (c) 2003 Howard E. Dow10 Etiquette, Customs and Rules Focus on the product Be prepared Open mind Language
Copyright (c) 2003 Howard E. Dow11 Roles and Responsibilities ModeratorRecorder Reader Author Management Inspector
Copyright (c) 2003 Howard E. Dow12 Planning Product Overview Preparation Inspection Rework Data analysis and storage Re-inspect (optional) Inspection Process
Copyright (c) 2003 Howard E. Dow13 Audience - Project - Terminology What the team did Results and data Benefits, pitfalls and how to Conclusions Agenda
Copyright (c) 2003 Howard E. Dow14 ModeratorRecorder ReaderAuthor Inspector Manager Roles and Responsibilities
Copyright (c) 2003 Howard E. Dow15 Product Overview Planning Preparation Inspection Rework Data analysis and storage Re-inspect (optional) Allowed suggestions Time not collected Inspection Process Used
Copyright (c) 2003 Howard E. Dow16 Summary of key differences Omit product overview Involved management Suggesting fixes allowed Strong bias to re-examine after rework by another team member Omission - collecting preparation time
Copyright (c) 2003 Howard E. Dow17 Ready to go 17 Scripts 3-6 inspectors
Copyright (c) 2003 Howard E. Dow18 Data Collected Script identifier Author Date completed Date inspected Size Defect: number, location, description Inspection time Inspectors
Copyright (c) 2003 Howard E. Dow19 Audience - Project - Terminology What the team did Results and data Benefits, pitfalls and how to Conclusions Agenda
Copyright (c) 2003 Howard E. Dow20 Results No operational defects found in inspected test scripts On time delivery Increased skill level Defects
Copyright (c) 2003 Howard E. Dow21 Summary Data - 1 Totals Scripts inspected: 17 LOC inspected: 12,949 Defects found: 544 Time spent: 24.2 hours 75 hours
Copyright (c) 2003 Howard E. Dow22 Summary Data - 2
Copyright (c) 2003 Howard E. Dow23 Summary Data - 3 Calculated Defect density: 42 Defects/KLOC Inspection rate: 535 LOC/hour 173 LOC/hour Defect find rate: 22.5 Defects/hour 7 Defects/hr
Copyright (c) 2003 Howard E. Dow24 Summary Data - 4 Severity typeQuantityPercent of total High152.8% Medium % Low % Total544100%
Copyright (c) 2003 Howard E. Dow25
Copyright (c) 2003 Howard E. Dow26
Copyright (c) 2003 Howard E. Dow27
Copyright (c) 2003 Howard E. Dow28
Copyright (c) 2003 Howard E. Dow29
Copyright (c) 2003 Howard E. Dow30
Copyright (c) 2003 Howard E. Dow31 Observation
Copyright (c) 2003 Howard E. Dow32 Observation
Copyright (c) 2003 Howard E. Dow33 Observation ???
Copyright (c) 2003 Howard E. Dow34 Observation
Copyright (c) 2003 Howard E. Dow35 Excellent! Observation
Copyright (c) 2003 Howard E. Dow36 Audience - Project - Terminology What the team did Results and data Benefits, pitfalls and how to Conclusions Agenda
Copyright (c) 2003 Howard E. Dow37 Benefits No operational defects reported On time delivery Value of process shown Some increase in expertise
Copyright (c) 2003 Howard E. Dow38 Pitfalls Must be “Sloppy” data collection Need for more training
Copyright (c) 2003 Howard E. Dow39 How to Find a “critical” project Suggest doing something different Keep it simple - be flexible Propose “Let’s put more eyes on the code.” “Let’s inspect the code.”
Copyright (c) 2003 Howard E. Dow40 Audience - Project - Terminology What the team did Results and data Benefits, pitfalls and how to Conclusions Agenda
Copyright (c) 2003 Howard E. Dow41 Conclusions Test scripts can be inspected Share your results Other testers Developers Useful as a learning tool Avoid unrealistic expectations
Copyright (c) 2003 Howard E. Dow42 References The original article on software inspections. “Design and Code Inspections to Reduce Errors in Program Development”, Michael Fagan, IBM Systems Journal, Volume 15, No. 3, This may be available as reprint at: Handbook of Walkthroughs, Inspections, and Technical Reviews, Daniel P. Freedman and Gerald M. Weinberg, 3 rd Edition, Dorset House, 1990 Software Inspection, Tom Gilb and Dorothy Graham, Addison-Wesley, 1990
Copyright (c) 2003 Howard E. Dow43 Howie Dow Thank you for your time. Questions please