1. CMPT 371 Software Management
Peer Reviews
Nathaniel Osgood
Department of Computer Science
University of Saskatchewan

2. Recall: Key Best Practices
Conscious, proactive risk scanning & management
Accountable Positions
Peer reviews
Risk-driven incremental delivery
Binary mini-milestones
Continuous integration & smoke test
Regular status updates & brief meetings
Assertions
(Team-oriented) version control practices
Defect estimation
Rigorous risk-driven testing (especially test creation before coding)
Focused prototypes
Time Estimation/boxing
Scheduling
Process improvement
Change control mechanisms
Pair programming as examples peer reviews

3. Key Best Practices Conscious, proactive risk scanning & management
Accountable Positions
Peer reviews
Risk-driven incremental delivery
Binary mini-milestones
Continuous integration & smoke test
Regular status updates & brief meetings
(Team-oriented) version control practices
Defect estimation
Rigorous risk-driven testing (especially test creation before coding)
Focused prototypes
Time Estimation/boxing
Scheduling
Process improvement
Change control mechanisms
Pair programming as examples peer reviews

4. Reviews: Why? Can find larger fraction of bugs than testing
More cost-effective than testing
IBM found 3.5 hours/error for inspection removal vs hours/error for testing
Easily pay for themselves (“Quality is Free”)
More flexible than testing
Need not wait for executable code
Can perform at all stages of software engineering process
Can be done early in the development of a component
Can assess communications issues (clarity, style, commenting, etc.)

5. Other Benefits of Peer Reviews To…
Person reviewing the artifact (Clarify understanding, learn coding tricks, stylistic ideas)
Person whose artifact is being reviewed
Improving technique, learn
Broader culture
Spread of knowledge about code base
Spread of knowledge of standards, coding styles
Code written with other people in mind

6. Importance of Early Reviews
Requirements
Early artifacts have disproportionate impact on development process
UI design
Design
Unit implementation
Unit testing
Marketing documents
Help System/Documentation plans

7. Good Points for Peer Reviews

8. Guidelines for reviews
Keep impersonal: Focus on artifact, not people
Keep review team small (3-7 participants)
Try to identify -- but not solve -- problems during review
Limit meeting to no more than 2 hours
Require advanced preparation for formal reviews
Be sensitive to cultural and human components
Prioritize focus for more major issues
Just 1-2 minutes spent discussing each sol’n

9. Spectrum of Review Formality 1
One classification
Different terms used in different places
Important thing is functional distinctions

10. Characteristics in Weiger’s Spectrum

11. Characteristics of More Formal Review Type…

12. A Few Reviews Peer deskcheck, pass around Walkthrough
This involves preparation time for people (because review not done live) but no meeting
Walkthrough
No preparation time -- just see for the first time when being presented
Team review (“structured walkthrough”)
Like inspection, but combined roles, author often leads meeting, inspector request to discuss sections of interest; may have more discussion on sol’ns

13. Inspection: Best Practices (Wiegers)
Inspect upstream documents first
Begin inspect documents early in their lives
Focus on major defects
Emphasize defect prevention and process improvement
Plan inspections to address project & inspection objectives
Check against source and related documents
Prepare & inspect at your organization's optimum rates
Measure your benefits from inspections
Use serious, quantitative entry and exit conditions

14. Good People to Include in a Review

15. Size Tradeoffs More inspectors …
Catch more bugs (greater “synergy”)
Despite pre-meeting preparation, some studies suggest large fraction of bugs found in meeting
Make meeting harder to schedule
Slow meeting progress
Can stall programmer’s work
Sometimes do several parallel inspections rather than big shared inspection

16. A Generic Inspection Process
LANNING
Ori
entation
Meeting
Preparation
Review
Rework
another
review
needed
Work unit
Close -
out No
FOR THE M
EETING
Yes
A typical inspection process undergoes a life cycle parallel to the project’s. This life cycle begins with the initial planning, performed by senior technical personnel or the project manager. Planning consists of choosing the appropriate review teams for every inspection, determining the material needed and specifying the dates (or milestones) at which inspections should be performed.
Planning the inspection is followed by an initial (opening or orientation) meeting. The purpose of this meeting is to establish a clear view of the work material and the goals of the review, and to make sure that the inspectors feel qualified and committed to participating in the process.
The third phase of the inspection, the preparation for the meeting, is to be carried out by the reviewers independently. The objective is to allow them the time to prepare for a rigorous and fast-paced review meeting. If the work package is design documents, it can be studied in detail by the individual reviewers, who should identify as many significant concerns as possible. If the subject of inspection is a real, constructed product, individual review cannot have the same functionality. Instead, the reviewers should become acquainted with the design and construction documents and note the objects of potential risk and quality concerns according to their knowledge and experience. In both cases, this information is filed on a separate reviewer data form. Minor issues should rather be noted on the work documentation. A review inspection goes through all issues noted on the reviewer data forms, and can become unreasonably lengthy if all concerns, no matter how trivial, are noted there. These data sheets can be used to statistically confirm the initial concerns and review checklists.
The collaborative peak of the inspection process is the review meeting, attended by the moderator, the inspectors, the project team and a secretary. The secretary is responsible for recording and timing the proceedings. The moderator raises the issues reported by the inspectors sequentially. The project team “defends” their work in a constructive dialog “against” the reviewers. It is essential that it is the work under defense, and not the project team. The moderator is responsible for keeping the meeting focused, to the point, constructive and calm. The objective of the inspection review is not to assess the performance of the project team, but to produce a consolidated report of the non-minor issues, to provide group synergy, share knowledge among the review and project teams, to improve the reviewing skills of the reviewers, and provide statistical information concerning the development process. This information is recorded by the secretary during the meeting, and complements the reviewers’ reports generated during the preparation phase. The actual meetings should not last too long – after 90 minutes or so the participants begin to get tired and efficiency drops.
The end of the review meeting initiates the rework cycle for the work unit. The project team resolves the issues raised at the meeting and in the inspectors’ reports, and categorizes them accordingly (e.g. related to specifications, requirements and design). Ideally, they should record the time allocated to correcting each one of them. This report along with the corrected work is handed back to the moderator for verification. Having reviewed the work and that relevant data sheets, the moderator makes his/her recommendation as to the need for another review. If one is not needed, he/she is responsible for signing off the procedure with the inspectors, generating a summary statistics report, proposals for the improvement of the inspection process and entering the review data into the appropriate quality databases. If the work unit does not “pass”, the inspection loop is repeated under the supervision of the same moderator, as shown in the flowchart.
The inspection life cycle is used in design and construction for the release of major work units, like the architectural design, structural details, material management, laying of the steel re-bar, the assembly of formwork, or the pouring and finishing of concrete. Inspections are formal and documented, mainly because of their legal significance and corresponding contractual obligations. However, they are seldom focused on improving the review process itself or on creating lessons learned through statistical use of defect data and errors – construction inspections are usually only as good as the participating individuals’ experiences and knowledge.
Peňa-Mora ,et. al., 2003

17. Stages: Planning Participants review material on own before meeting
Moderator assigned at this point
Author contributes objectives for inspection
Based on historic data moderator estimates # of meetings required to do reviews of desired scope
Moderator
Invites participants
Helps author prepare package of materials for inspections
Distributes package to participants several days ahead of time

18. Stages: Overview Often a separate meeting
Author more informally describes perspective on product
Sometimes the inspection package is distributed during this meeting
Sometimes skip if
Participants already familiar with product
Overview can be described in package

19. Stages: Preparation Most preparation centers around inspection package
The deliverable to be inspected
Standards/Requirements/Specifications
Typo list/individual issue log
Work aids to help identify defects
(e.g. Common defects for this sort of deliverable)
Test documentation to verify this deliverable

20. Stages: Meeting 1 Deliverables
"inspection summary report“ (moderator)
Work product appraisal
Information to communicate to mgmt, etc.
"issues log“
Indication of what changes are needed to complete inspection process
May stop inspection if identified errors are too serious to make it worth it to continue

21. Meeting Participant Roles
Author: shares perspective
Moderator: leads process
Reader: presents pieces of code (and perspectives on) to inspectors
Can help cataylze shared understanding by inspectors
Inspectors: (any participant, including those assigned to other roles)
Can critique code
Can identify possible issues where errors
Recorder: Documents issues
Typically 3-4 participants

22. Stages: Rework Author addresses most items in issues log
Sometimes issue log items get assigned to others
Sometimes just log defects in defect control system to be followed up later
Result
Updated work product
Annotated issue log indicating resolution for each item

23. Stages: Followup
Often with moderator as "verifier" (moderator decides when process is over)
Verifier confirms that changes have been successfully made
Baselining of changed deliverable into SCCS

24. Stages: Causal analysis
This basically uses inspection process to improve
The development process
The inspection process
Focus on process improvement and not on people
Try to identify root cause of defects
E.g. Ambiguous explanations in requirements, design specs, inconsistent naming conventions

25. What is Expected of You
Some peer reviews of your artifacts throughout term
Pair programming
Pair Testing
Deskcheck
1 inspection of an artifact for the semester
Signoff by all persons
Reporting on this as part of the milestone deliverables