1. Role Based Peer Reviews A. Winsor Brown Oct. 2, 2009
CS577a Fall 2009
Role Based Peer Reviews
A. Winsor Brown Oct. 2, 2009
(c) AWBrown & CSSE
Oct. 2, 2009

2. Goals of Presentation
Put things in perspective: Quality Models and Metrics
Process: Role Based (team) Peer Reviews as practiced in CS577
(c) AWBrown & CSSE
Oct. 2, 2009

3. Agenda Reviews per IEEE J-STD-016-1995: Quality Models and Metrics 
Peer Reviews as practiced in CS577
(c) AWBrown & CSSE
Oct. 2, 2009

4. Quality Model Types
All four types represented: Process, Product, Property, Success (P3S)
Product
What’s a defect?
Problem reports
Property
Defects [over time]: Removal and residual injection rates
Defect Density
Success
Defect removal rate
Problem/Trouble Reports Open over time
Process
Macro: Defect injection and removal; workflow
Micro: ETVX, defect removal techniques, etc.
(c) AWBrown & CSSE
Oct. 2, 2009

5. Product Models Related to Quality
What’s a defect?
An instance of non-conformance with the initiating requirements, standards, or exit criteria or other “checklists”
Can exist in the accuracy/completeness of requirements, standards, and associated interface/reference documents
Determined ONLY by the responsible Author of an artifact
Typically start out as concerns in informal or agile reviews
What’s an “issue”
Concerns that can NOT be fixed by the author of the artifact under review
In developments with large number of people or cycles, issues are usually tracked to closure.
(c) AWBrown & CSSE
Oct. 2, 2009

6. Defect Categories Severity Major Minor
A Condition that causes an operational failure, malfunction, or prevents attainment of an expected or specified result
Information that would lead to an incorrect response or misinterpretation of the information by the user
An instance of non-conformance that would lead to a discrepancy report if implemented as is
Minor
A violation of standards, guidelines, or rules, but would not lead to a discrepancy report
Information that is undesirable but would not cause a malfunction or unexpected results (bad workmanship)
Information that, if left uncorrected, may decrease maintainability
(c) AWBrown & CSSE
Oct. 2, 2009

7. Defect Categories (continued)
Class
Missing
Information that is specified in the requirements or standard, but is not present in the document
Wrong
Information that is specified in the requirements or standards and is present in the document, but the information is incorrect
Extra
Information that is not specified in the requirements or standards but is present in the document
(c) AWBrown & CSSE
Oct. 2, 2009

8. Defect Categories (continued)
Type
Unavoidable
Unavoidable defects (AKA changes) arise because of the methods, techniques or approaches being followed necessitate changes. Examples include changes arising because of the dynamics of learning, exploration in IKIWISI situations, code or screen contents reorganizations taken on as an "afterthought", replacement of stubs or place-holders in code, etc. Such situations are often "planned for" and expected to occur.
Avoidable
Changes in analysis, design, code or documentation arising from human error, and which could be avoided through better analysis, design, training, etc. Examples include stub replacement that violates win conditions or requirements such as execution time, memory space: for instance the replacement of a "stub" which breaks a critical timing constraint.
(c) AWBrown & CSSE
Oct. 2, 2009

9. Defect Categories (continued)
Severity
Class
Type
Major
Missing
Avoidable
Wrong
Unavoidable
Minor
Extra
(c) AWBrown & CSSE
Oct. 2, 2009

10. Agenda Quality Models and Metrics Peer Reviews as practiced in CS577 
(c) AWBrown & CSSE
Oct. 2, 2009

11. Role Based Agile Internal/Informal Review
The main activities are following
Planning
Overview (optional)
Preparation
Review Meeting
Rework
Roles (four) & Participants
Review Leader (recommended: quality focal point for cs577) but must have been able to produce the produce; also plays "Tester" if only three participants
“Reader” (next type of person in the artifact’s chain)
“Tester” (reviews external interfaces; describes tests)
Author
(c) AWBrown & CSSE
Oct. 2, 2009

12. Agile Internal/Informal RBP Review
Planning
Overview
Preparation
Concern Log
Problem List
Review
Rework
Review Result Summary
(c) AWBrown & CSSE
Oct. 2, 2009

13. Agile Internal/Informal RBP Review
1. Planning - Check that the document is ready to be reviewed
- Set up for success; identify people-role pairs
2. Overview - Explain purpose and intent of document
3. Preparation - Get ready to fulfill role in meeting: mark up artifact
- Completely understand artifact from role's perspective
4. Meeting - Identify, classify and record defects
- Leader records concerns on concern log
- Initial categorization of concerns as D/I/x
5. Rework - Correct artifact defects
6. Follow-Up - Ensure all defects identified are corrected
- Ensure no new defects are introduced
- Enter all "issues" in Bugzilla
(c) AWBrown & CSSE
Oct. 2, 2009

14. CS577 Defect Reporting Concepts
Range of Defect identification and reporting mechanisms
One at a time: Problem report system
Multiple issues/problems found by a single reviewer: Agile Artifact Review (AAR) – only two types of forms (Issues/Concern Log and Defect List)
Agile Internal/Informal RBP Review (AIR): Two types of forms; Performed in a Role-Based fashion
Agile Formal Review: Three different types of forms
Internal/Informal Review: Four different types of bigger forms
Formal Review: Four different types of bigger forms
Fagan’s Inspection: Five different types of forms
(c) AWBrown & CSSE
Oct. 2, 2009