Role Based Peer Reviews A. Winsor Brown Oct. 2, 2009 CS577a Fall 2009 Role Based Peer Reviews A. Winsor Brown Oct. 2, 2009 (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Goals of Presentation Put things in perspective: Quality Models and Metrics Process: Role Based (team) Peer Reviews as practiced in CS577 (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Agenda Reviews per IEEE J-STD-016-1995: Quality Models and Metrics  Peer Reviews as practiced in CS577 (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Quality Model Types All four types represented: Process, Product, Property, Success (P3S) Product What’s a defect? Problem reports Property Defects [over time]: Removal and residual injection rates Defect Density Success Defect removal rate Problem/Trouble Reports Open over time Process Macro: Defect injection and removal; workflow Micro: ETVX, defect removal techniques, etc. (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Product Models Related to Quality What’s a defect? An instance of non-conformance with the initiating requirements, standards, or exit criteria or other “checklists” Can exist in the accuracy/completeness of requirements, standards, and associated interface/reference documents Determined ONLY by the responsible Author of an artifact Typically start out as concerns in informal or agile reviews What’s an “issue” Concerns that can NOT be fixed by the author of the artifact under review In developments with large number of people or cycles, issues are usually tracked to closure. (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Defect Categories Severity Major Minor A Condition that causes an operational failure, malfunction, or prevents attainment of an expected or specified result Information that would lead to an incorrect response or misinterpretation of the information by the user An instance of non-conformance that would lead to a discrepancy report if implemented as is Minor A violation of standards, guidelines, or rules, but would not lead to a discrepancy report Information that is undesirable but would not cause a malfunction or unexpected results (bad workmanship) Information that, if left uncorrected, may decrease maintainability (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Defect Categories (continued) Class Missing Information that is specified in the requirements or standard, but is not present in the document Wrong Information that is specified in the requirements or standards and is present in the document, but the information is incorrect Extra Information that is not specified in the requirements or standards but is present in the document (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Defect Categories (continued) Type Unavoidable Unavoidable defects (AKA changes) arise because of the methods, techniques or approaches being followed necessitate changes. Examples include changes arising because of the dynamics of learning, exploration in IKIWISI situations, code or screen contents reorganizations taken on as an "afterthought", replacement of stubs or place-holders in code, etc. Such situations are often "planned for" and expected to occur. Avoidable Changes in analysis, design, code or documentation arising from human error, and which could be avoided through better analysis, design, training, etc. Examples include stub replacement that violates win conditions or requirements such as execution time, memory space: for instance the replacement of a "stub" which breaks a critical timing constraint. (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Defect Categories (continued) Severity Class Type Major Missing Avoidable Wrong Unavoidable Minor Extra (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Agenda Quality Models and Metrics Peer Reviews as practiced in CS577  (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Role Based Agile Internal/Informal Review The main activities are following Planning Overview (optional) Preparation Review Meeting Rework Roles (four) & Participants Review Leader (recommended: quality focal point for cs577) but must have been able to produce the produce; also plays "Tester" if only three participants “Reader” (next type of person in the artifact’s chain) “Tester” (reviews external interfaces; describes tests) Author (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Agile Internal/Informal RBP Review Planning Overview Preparation Concern Log Problem List Review Rework Review Result Summary (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

Agile Internal/Informal RBP Review 1. Planning - Check that the document is ready to be reviewed - Set up for success; identify people-role pairs 2. Overview - Explain purpose and intent of document 3. Preparation - Get ready to fulfill role in meeting: mark up artifact - Completely understand artifact from role's perspective 4. Meeting - Identify, classify and record defects - Leader records concerns on concern log - Initial categorization of concerns as D/I/x 5. Rework - Correct artifact defects 6. Follow-Up - Ensure all defects identified are corrected - Ensure no new defects are introduced - Enter all "issues" in Bugzilla (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009

CS577 Defect Reporting Concepts Range of Defect identification and reporting mechanisms One at a time: Problem report system Multiple issues/problems found by a single reviewer: Agile Artifact Review (AAR) – only two types of forms (Issues/Concern Log and Defect List) Agile Internal/Informal RBP Review (AIR): Two types of forms; Performed in a Role-Based fashion Agile Formal Review: Three different types of forms Internal/Informal Review: Four different types of bigger forms Formal Review: Four different types of bigger forms Fagan’s Inspection: Five different types of forms (c) 2003..2009 AWBrown & CSSE Oct. 2, 2009