CSE403 Software Engineering Autumn 2000 A bit more performance and then Technical Reviews Gary Kimura Lecture #21 November 13, 2000.

Slides:



Advertisements
Similar presentations
Tom Gilchrist, CQA, CSQE Quality Assurance in ISD and Maintenance Projects How do you do QA when the time it takes is longer that the.
Advertisements

Damian Gordon.  Static Testing is the testing of a component or system at a specification or implementation level without execution of the software.
Systematic Software Reviews Software reviews are a “quality improvement processes for written material”.
Software Project Management Lecture # 11. Outline Quality Management ( chapter 26 - Pressman )  Software reviews  Formal Inspections & Technical Reviews.
Technical Reviews A Presentation By Manuel Richey for EECS 814 November 17 th, 2005.
Chapter 10 Schedule Your Schedule. Copyright 2004 by Pearson Education, Inc. Identifying And Scheduling Tasks The schedule from the Software Development.
Copyright © 1994 Carnegie Mellon University Disciplined Software Engineering - Lecture 1 1 Disciplined Software Engineering Lecture #7 Software Engineering.
CLEANROOM SOFTWARE ENGINEERING
Software Quality Assurance (SQA). Recap SQA goal, attributes and metrics SQA plan Formal Technical Review (FTR) Statistical SQA – Six Sigma – Identifying.
Testing Without Executing the Code Pavlina Koleva Junior QA Engineer WinCore Telerik QA Academy Telerik QA Academy.
Overview Lesson 10,11 - Software Quality Assurance
Week 7: Requirements validation Structured walkthroughs Why have walkthroughs When to have walkthroughs Who participates What procedures are helpful Thoughtless.
Copyright © 1998 Philip M. Johnson (1)‏ Introduction to Formal Technical Reviews Philip Johnson Associate Professor Department of Information and Comp.
Software Measurement and Process Improvement
IS 421 Information Systems Management James Nowotarski 16 September 2002.
SE 555 Software Requirements & Specification Requirements Validation.
CSC 395 – Software Engineering Lecture 9: Testing -or- How I Stopped Worrying and Learned to Love the Bug.
 QUALITY ASSURANCE:  QA is defined as a procedure or set of procedures intended to ensure that a product or service under development (before work is.
Software Quality Assurance
Problem Management Overview
1 Software Inspections and Walkthroughs Author: A. Frank Ackerman Presented by Cynthia Johnson EEL6883.
Design Reviews Peer Reviews. Agenda Peer Reviews Participants of Peer Review Preparation for a Peer Review Session The Peer Review Session Post-peer Review.
S/W Project Management
Software Inspections and Walkthroughs By. Adnan khan.
CLEANROOM SOFTWARE ENGINEERING.
SoITSSpecifications of IT systems? 1 Specifications of IT systems checking and validating Jens Bennedsen and Peter Gorm Larsen
Software Quality Assurance Lecture #4 By: Faraz Ahmed.
OSF/ISD Project Portfolio Management Framework January 17, 2011.
S oftware Q uality A ssurance Part One Reviews and Inspections.
Software Quality Assurance Activities
Software Inspections. Defect Removal Efficiency The number of defects found prior to releasing a product divided by The number of defects found prior.
CSE403 Software Engineering Autumn 2001 Technical Reviews Gary Kimura Lecture #19 November 14, 2001.
Copyright © 1998 Philip M. Johnson (1) Introduction to Formal Technical Reviews Philip Johnson Associate Professor Department of Information and Comp.
Formal and Informal Peer Reviews
Unit 8 Syllabus Quality Management : Quality concepts, Software quality assurance, Software Reviews, Formal technical reviews, Statistical Software quality.
Phil Cronin Anne Hill Allen Schones CIS841 Summer on Campus 1998 IN-PROCESS INSPECTIONS FOR OBJECT ORIENTED DESIGNS.
CEN rd Lecture CEN 4021 Software Engineering II Instructor: Masoud Sadjadi Phases of Software.
PAPER PRESENTATION: EMPIRICAL ASSESSMENT OF MDE IN INDUSTRY Erik Wang CAS 703.
IT Requirements Management Balancing Needs and Expectations.
Disciplined Software Engineering Lecture #7 Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Sponsored by the U.S. Department.
Formal Technical Reviews Matt Graham 18 November 2004 EECS 814 University of Kansas.
... there is no particular reason why your friend and colleague cannot also be your sternest critic. --Jerry Weinberg --Jerry Weinberg.
From Quality Control to Quality Assurance…and Beyond Alan Page Microsoft.
Jump to first page (C) 1998, Arun Lakhotia 1 Quality Assurance: Reviews and Walkthroughs Arun Lakhotia University of Southwestern Louisiana Po Box
CHAPTER 9 INSPECTIONS AS AN UP-FRONT QUALITY TECHNIQUE
Copyright © 1994 Carnegie Mellon University Disciplined Software Engineering - Lecture 7 1 Design and Code Reviews - Overview What are design and code.
Software Testing and Maintenance 1 Code Review  Introduction  How to Conduct Code Review  Practical Tips  Tool Support  Summary.
(1) Introduction to Software Review Philip Johnson Collaborative Software Development Laboratory Information and Computer Sciences University of Hawaii.
Reviews and Inspections. Types of Evaluations Formal Design Reviews conducted by senior personnel or outside experts uncover potential problems Inspections.
Page 1 TEST in the large RELEASE REWORK ASSESS packaged application documentation models and source code management documents requirement alloc. matrix.
1 Phase Implementation. Janice Regan, Overview of Implementation phase Create Class Skeletons Define Implementation Plan (+ determine subphases)
Unit III Static Testing. static testing Testing of a component or system at requirements or implementation level without execution of any software, e.g.,
1 These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 5/e and are provided with permission by.
Thomas L. Gilchrist Testing Basics Set 5: CM & Peer Reviews By Thomas L. Gilchrist CSQE,CSQA 2009.
© Michael Crosby and Charles Sacker, 2001 Systematic Software Reviews Software reviews are a “quality improvement process for written material”.
1 Lecture 12: Chapter 16 Software Quality Assurance Slide Set to accompany Software Engineering: A Practitioner’s Approach, 7/e by Roger S. Pressman Slides.
These slides are designed to accompany Software Engineering: A Practitioner’s Approach, 7/e (McGraw-Hill 2009). Slides copyright 2009 by Roger Pressman.1.
Mahindra Satyam Confidential Quality Management System Software Defect Prevention.
Peer Review Presented by : Basker George. Peer ( 同等的人 ) Review( 回顾 ) During the development of software, defects are inevitably ( 不可避免 ) injected. Defect.
More SQA Reviews and Inspections. Types of Evaluations  Verification Unit Test, Integration Test, Usability Test, etc  Formal Reviews  aka "formal.
Software Project Management Lecture # 12. Outline Quality Management ( chapter 26 - Pressman )  SQA  Who does it?  SQA Activities  Software reviews.
Review Techniques SEII-Lecture 16
CIS 375 Bruce R. Maxim UM-Dearborn
Systematic Software Reviews
CSE403 Software Engineering Autumn 2000 Performance Issues
Applied Software Project Management
QA Reviews Lecture # 6.
WALKTHROUGH and INSPECTION
Software Reviews.
3. Software Quality Management
Presentation transcript:

CSE403 Software Engineering Autumn 2000 A bit more performance and then Technical Reviews Gary Kimura Lecture #21 November 13, 2000

Today Moved Quiz #4 to Wednesday December 6 th Please read Parnas: Software Aging this week Finish up with performance Formal technical reviews Material from Philip Johnson Associate Professor Department of Information and Computer Sciences University of Hawaii More practical reviews

A sideways look at performance Percent of CPU usage and idle times are sometimes a measure of system efficiency. This metric is pretty much ignores speed and is more concerned with why the processor is idling. Resolving page faults and lock contention are two of the big bug-a-boos here.

Heap performance enhancements What are groups doing? Have you looked at lock contention? Have you looked at how often you’re going to need to call down to the OS? Have you looked at memory footprint and data structure layout?

Cross platform development By nature, does cross platform development go against achieving performance goals?

What is a Formal Technical Review? A method involving a structured encounter in which a group of technical personnel analyzes or improves the quality of the original work product as well as the quality of the method.

Reviews improve schedule performance Reviews reduce rework. – Rework accounts for 44% of dev. cost! – Reqs (1%), Design (12%), Coding (12%), Testing (19%) Reviews are pro-active tests. – Find errors not possible through testing. Reviews are training. – Domain, corporate standards, group. Why review? We test! ReqDesignCodeTest RRR

Why review? Who benefits? Formal technical review provides: – Defect information to the author. – Information on work product and development to peers. – Fault likelihood data to testers. – Product status to management.

True FTR is well-defined Well-defined process – Phases (orientation, etc.) – Procedures (checklists, etc.) Well-defined roles – Moderator, Reviewer, Scribe, Author, etc. Well-defined objectives – Defect removal, requirements elicitation, etc. Well-defined measurements – Forms, consistent data collection, etc.

FTR is effective quality improvement Reviews can find % of all defects. Reviews are technical, not management. Review data can assess/improve quality of: – work product – software development process – review process Reviews reduce total project cost, but have non-trivial cost (~15%) Upstream defect removal is times cheaper. Reviews disseminate domain knowledge, development skills, and corporate culture.

Industry Experience with FTR Aetna Insurance Company: – FTR found 82% of errors, 25% cost reduction. Bell-Northern Research: – Inspection cost: 1 hour per defect. – Testing cost: 2-4 hours per defect. – Post-release cost: 33 hours per defect. Hewlett-Packard – Est. inspection savings (1993): $21,454,000 IBM (using Cleanroom) – C system software – No errors from time of first compile.

Who, What, and When Who decides what should be reviewed? – Senior technical personnel, project leader What should be reviewed? – Work products with high impact upon project risks. – Work products directly related to quality objectives. – “Upstream” work products have higher impact. When should review be planned? – Specify review method and target work products in software development plan/quality plan.

The range of review practice TekInspect Development Method Non-CleanroomCleanroom FTRinFTR Code Inspection (Fagan76) Inspection (Gilb93) 2-Person Inspection (Bisant89) N-Fold Inspection (Martin90) Walkthrough (Yourdon89) Verification- based Inspection (Dyer92) Active Design Reviews (Parnas85) FTArm (Johnson94) ICICLE (Brothers90) Scrutiny (Gintell93) CAIS (Mashayekhi94) ManualTool-Based Code Reading (McConnell93) Software Review (Humphrey90) Phased Insp. (Knight93)

Families of Review Methods Walkthroughs Minimal overhead Developer training Quick turnaround Requirements elicitation Ambiguity resolution Training Method FamilyTypical GoalsTypical Attributes Little/no preparation Informal process No measurement Not FTR! Technical Reviews Formal process Author presentation Wide range of discussion Inspections Detect and remove all defects efficiently and effectively. Formal process Checklists Measurements Verify phase

Example Planning Data

Example Orientation Data

Preparation Objectives – Find maximum number of non-minor issues. Procedure for reviewers: – Allocate recommended time to preparation. – Perform individual review of work product. – Use checklists and references to focus attention. – Note critical, severe, and moderate issues on Reviewer Data Form. – Note minor issues and author questions on work product. PlanningOrientationPreparationReview Mt.ReworkVerify

Example Issue Classification Critical – Defects that may cause the system to hang, crash, produce incorrect results or behavior, or corrupt user data. No known work-arounds. Severe – Defects that cause incorrect results or behavior with known work-arounds. Large and/or important areas of the system is affected. Moderate – Defects that affect limited areas of functionality that can either be worked around or ignored. Minor – Defects that can be overlooked with no loss of functionality.

Example checklist

Example Preparation Data

Why not write on the work product? Advantages of Reviewer Data Sheet: – Minor issues are “pre-filtered” from review meeting, saving meeting time. – Reviewers articulate issues clearly during preparation, saving meeting time. – Preparation statistics gathering simplified. – Preparation effectiveness (% true defects, % redundancy) and checklist effectiveness is measurable. – Issues can be presented in order of importance. – Data sheet indicates effectiveness of checklists. Disadvantages of Reviewer Data Sheet: – Requires extra time (15 minutes?) – Discourages last minute preparation. – Makes quality of preparation more visible.

Review Meeting Objectives – Create consolidated, comprehensive listing of non-minor issues. – Provide opportunity for group synergy. – Improve reviewing skill by observing others. – Create shared knowledge of work product. Procedure – Moderator requests issues sequentially. – Reviewers raise issues. – Scribe notes issues on Scribe Data Sheet. – Scribe Data Sheet is visible to everyone. PlanningOrientationPreparationReview Mt.ReworkVerify

Example Review Meeting Data

Example Rework Data

Verify Objectives – Assess the (reworked) work product quality. – Assess the inspection process. – Pass or fail the work product. Procedure for moderator: – Obtain reworked product and Author Data Sheet. – Review work product/data sheet for problems. – Provide recommendation for work product. – Perform sign-off with reviewers. – Compute summary statistics for inspection. – Generate any process improvement proposals. – Enter review data into quality database. PlanningOrientationPreparationReview Mt.ReworkVerify

Example Verify Data

More Practical Code Reviews One problem with any code review is not putting anyone on the defensive too much –Guard against ganging up on the poor developer –Balanced with the need to highlight issues in their code Small peer group Less formal setting Usually takes a few meetings –One to have the developer give an overview if needed Give everyone some time to review the code –Meet to get reviewers feedback Quite often this can be done informally Can be done as a group meeting, or informally one-on-one, or via and other groupware products In reality the testers and performance engineers probably do the best code review