Quality Management, Peer Review, & Architecture Review Board ©USC-CSSE

Outline Quality Management Peer Review Architecture Review Board

Objectives of QM To ensure the high quality process in order to deliver high quality products (c) USC-CSSE

Quality Management in 577ab IIV&V Configuration Management Defect Reporting and Tracking Testing Buddy Review Architecture Review Board Core Capability Drive through Design Code Review Document template Sample artifacts (c) USC-CSSE

Quality Guidelines Design Guidelines Coding Guidelines Describe design guidelines on how to improve or maintain modularity, reuse and maintenance How the design will map to the implementation Coding Guidelines Describe how to document the code in such as way that it could easily be communicated to others (c) USC-CSSE

Coding Guidelines C: http://www.gnu.org/prep/standards/standards.html Java: http://geosoft.no/development/javastyle.html Visual Basic: http://msdn.microsoft.com/en-us/library/h63fsef3.aspx (c) USC-CSSE

Quality Guidelines Version Control and History Chronological log of the changes introduced to this unit Implementation Considerations Detailed design and implementation for as-built considerations Unit Verification Unit / integration test Code walkthrough / review / inspection (c) USC-CSSE

Quality Assessment Methods Methods, tools, techniques, processes that can identify the problems Detect and report the problem Measure the quality of the software system Three methods of early defect identification peer review, IIV&V, Automated Analysis (c) USC-CSSE

Outline Quality Management Peer Review Architecture Review Board

Peer Review Reviews performed by peers in the development team Can be from Fagan’s inspections to simple buddy checks Peer Review Items Participants / Roles Schedule (c) USC-CSSE

Defect Classification Severity : Major/ Minor Type: Missing / Wrong / Extra Category Logic , Syntax, Clarity. Performance, Interface, … (c) USC-CSSE

Defect terminologies Activity: This is the actual activity that was being performed at the time the defect was discovered. Trigger: The environment or condition that had to exist for the defect to surface. What is needed to reproduce the defect? Impact: For in-process defects, select the impact which you judge the defect would have had upon the customer if it had escaped to the field.  (c) USC-CSSE

Orthogonal Defect Classification Target Defect Type Triggers Impact Severity Qualifier Operational Concept / Requirements Correctness Completeness Consistency Clarity / Testability Traceability Design Conformance Logic/ Flow Backward Compatibility Lateral Compatibility Concurrency Internal Document Language Dependency Side Effect Rare Situations Simple Path Complex Path Coverage Variation Sequencing Interaction Workload/Stress Recovery/Exception Startup/Restart Hardware Configuration Software Configuration Blocked Test (previously Normal Mode) Installability Serviceability Standards Integrity/Security Migration Reliability Performance Documentation Requirements Maintenance Usability Accessibility Capability Major Minor Missing Wrong / Incorrect Extra Design / Code Assign/Init Checking Alg/Method Func/Class/Object Timing/Serial Interface/O-O Messages Relationship http://www.research.ibm.com/softeng/ODC/DETODC.HTM (c) USC-CSSE

(These attributes are usually available when the defect is opened.) Opener Section (These attributes are usually available when the defect is opened.) Defect Removal Activities Triggers Impact Design Rev, Code Inspection, Unit test, Function Test, System Test. Design Conformance Logic/ Flow Backward Compatibility Lateral Compatibility Concurrency Internal Document Language Dependency Side Effect Rare Situations Simple Path Complex Path Coverage Variation Sequencing Interaction Workload/Stress Recovery/Exception Startup/Restart Hardware Configuration Software Configuration Blocked Test (previously Normal Mode) Installability Serviceability Standards Integrity/Security Migration Reliability Performance Documentation Requirements Maintenance Usability Accessibility Capability   (c) USC-CSSE

(These attributes are usually available when the defect is fixed.) Closer Section (These attributes are usually available when the defect is fixed.) Target Defect Type Qualifer Age Source Design/Code Assign/Init Checking Alg/Method Func/Class/Object Timing/Serial Interface/O-O Messages Relationship Missing Incorrect Extraneous Base New Rewritten ReFixed Developed In-House Reused FromLibrary Outsourced Ported (c) USC-CSSE

Requirement Defect Type (1/2) A requirements defect is an error in the definition of system functionality Correctness: Wrongly stated requirements Examples: An incorrect equation, parameter value or unit specification A requirement not feasible with respect to cost, schedule and technology Completeness: Necessary information is missing Missing attributes, assumptions, and constraints of the software system. No priority assigned for requirements and constraints. Requirements are not stated for each iteration or delivery (c) USC-CSSE

Requirement Defect Type (2/2) Consistency: A requirements that is inconsistent or mismatched with other requirements Examples: requirements conflict with each other Requirements are not consistent with the actual operation environment (eg. Test, demonstration, analysis, or inspection) have not been stated. Traceability: A requirement that is not traceable to or mismatched with the user needs, project goals, organization goals (c) USC-CSSE

Unavoidable defects Changes arising because of The dynamic of learning Exploration in IKIWIKI situations Code or screen content reorganization taken on as an “afterthought” Replacement of stubs of placeholders in code (c) USC-CSSE

Avoidable defects Changes in analysis, design, code or documentation arising from human error Can be avoided though better analysis, design, training (c) USC-CSSE

Product Assurance Requirements Verification IIV&V Automated Analysis (c) USC-CSSE

Configuration Management Configuration items and rationale Identification systems Storage of configuration items Configuration Controls Status and accounting Baselining events (c) USC-CSSE

Defect and Change Management Processes Change Control Board (c) USC-CSSE

COQUALMO  Cost, Schedule and Quality (c) USC-CSSE

Current COQUALMO System COCOMO II Software development effort, cost and schedule estimate COQUALMO Software Size Estimate Defect Introduction Model Software platform, Project, product and personnel attributes Number of residual defects Defect density per unit of size Defect Removal Model Defect removal profile levels Automation, Reviews, Testing (c) USC-CSSE

Coding defects introduction range (c) USC-CSSE

Defect Removal Profiles (c) USC-CSSE

Partion of COQUALMO Rating Scale COCOMO II p.263 Very Low Low Nominal High Very High Extra High Automated Analysis Simple compiler syntax checking Basic compiler capabilities Compiler extension Basic req. and design consistency Intermediate-level module Simple req./design More elaborate req./design Basic dist-processing Formalized specification, verification. Advanced dist-processing Peer Reviews No peer review Ad-hoc informal walk-through Well-defined preparation, review, minimal follow-up Formal review roles and Well-trained people and basic checklist Root cause analysis, formal follow Using historical data Extensive review checklist Statistical control Execution Testing and Tools No testing Ad-hoc test and debug Basic test Test criteria based on checklist Well-defined test seq. and basic test coverage tool system More advance test tools, preparation. Dist-monitoring Highly advanced tools, model-based test (c) USC-CSSE

V&V and Quality Focal Point in 577 Verify whether the work product correctly satisfies the specifications found in any predecessor work product, such as requirements or design documents Identify any deviation from standards Suggest improvement opportunities to the author Promote the exchange of techniques and education of the participants

577 Products to V&V All interim and final development work products are candidates for review, including: goals, operational concepts and requirements specifications user interface specifications, designs, and prototypes architecture, high-level design, and detailed designs and models source code test plans, designs, cases, and procedures software development plans, including project management plan, configuration management plan, and quality assurance plan

Outline Quality Management Peer Review Architecture Review Board

[CSCI 577a FCR] [CSCI 577a DCR and 577b RDCR]