Chapter 4-Static Testing Techniques Types of Reviews Review Process Static analysis using tools

Static Testing It involves testing software work products manually, or with a set of tools, but they are not executed. Can be used to ‘test’ any form of document including source code, design documents and models, requirement specifications, test plan, and user manual. The primary objective of static testing is to improve the quality of software products by assisting engineers to recognize and fix their own defects early in the software development process. It starts early in the Life cycle

Static Testing Types of the defects that are easier to find during the static testing are: Deviation from standards Missing requirements Design defects Non-maintainable code Inconsistent interface specifications

Static Testing Benefits Early feedback on quality is established Rework cost is low Development Productivity improvement Improves product quality Creates common understanding by exchanging information Learn from defects found and prevent the occurrence of similar defects

Static Testing Techniques Done as a review process or automated with static analysis tools A review process can be informal or formal Informal reviews A two person team conduct an informal review. The goal is improve the quality of the work. Informal reviews are not documented. Formal review Formal reviews follow a formal process During project planning, project manager should allow time for formal reviews

Types of Reviews Informal (peer review) Walkthrough Technical review Inspection Formal Reviews

Formal Reviews Characteristics Identify problems, defects and improvments Follow rules amount of code/ pages to be reviewed, how much time will be spent. Prepare. Each participant is expected to prepare for and contribute to the review Most of the problems found through the review process are found during preparation, not at the actual review Write a report The review group must produce a written report summarizing the results of the review and make that report available to the rest of the product development team.

Roles of Review Meeting Moderator (review leader): Guide the meeting Author: person who has written the code Presenter: someone other than the author who presents the author’s code Record keeper (Scribe): documents the problems found and follow-up actions suggested Reviewers: experts in the subject area Observers: passive The manager : decides on the execution of reviews & schedules them

Formal Review Process PlanningKick-Off Preparati on Review meeting Rework Follow- up change request (CR) and report A formal review process consists of six main steps:

Formal Review Process Step 1: Planning A request for review by the author to the moderator Perform entry check(done by author or moderator) Completeness: Minimal Functionality: (for unit testing, code has been compiled) The documents should not reveal a large number of major defects Readability : (documents to be reviewed should be with line numbers) Complexity: (for unit testing) References needed for the review is stable and available (e.g. Requirements and Design Documents) A moderator has to take care of the scheduling like date, time, place and Select review members Determine the parts to be reviewed

Formal Review Process Step 2: Kick- off (optional) Preparation before the review meeting to explain the objectives of the review, the review process, the roles of participants and distribute the documents. Inform the group about the review meeting schedule two or three days before the meeting Rules of the meeting 1–2 hours long About 125 lines of code reviewed, pages of document Group size is between 3 and 7. Step 3: Preparation Each reviewer carefully reviews the work package and understand it (prepare questions and suggestions)

Formal Review Process Step 4: Review meeting: The author gives an overview of the document. The presenter reads the document line by line. The reviewers raise questions if the document is seen to have defects. They may suggest ways to fix it, but defects are not fixed in the meeting The scribe documents the change requests (CR) and the suggestions Decide if another meeting is necessary If the number of defects found per page exceeds a certain limit, the document must be reviewed again after rework. (exit criteria)

Formal Review Process Step 4: Review meeting: Change request (CR) contains: Give a brief description of the issue or action item. Assign a severity level (critical, major or minor) to a CR. Assign a person to follow up the issue. Since a CR documents a potential problem, there is a need for interaction between the author of the code and one of the reviewers, possibly the reviewer who made the CR Set a deadline for addressing a CR

Formal Review Process Step 5: Rework At the end of the meeting, the record keeper produces a summary of the meeting and the CR The author makes an attempt to address the issues within the agreed-upon time (deadline) Not every defect that is found leads to rework

Formal Review Process Step 6: Follow-up The moderator validate that the document was modified as documented in the CRs and ensure that the suggested improvements have been implemented correctly. Summarizing the review process (collect measurements) Check the exit criteria

Walkthrough A review where the author leads the team through a manual or simulated execution of the product using predefined scenarios The content of the document is explained step-by-step by the author Audience have different viewpoints Useful in high-level document (e.g. requirement specification or architecture design) The goals maybe: Evaluate the content of the document Discuss the validity of a proposed solution Establish common understanding of the document

Technical Review Technical reviews vary from quite informal to very formal Defects are found by the experts (such as architects, designers, key users) who focus on the content of the document The goals of the technical review are: To ensure that an early stage the technical concepts are used correctly To assess the value of technical concepts and alternatives in the product To inform participants about the technical content of the document Usually not concerned with comparing the document with its references

Inspection It is the most formal review type The presenter or reader, isn't the original programmer This forces someone else to learn and understand the material being presented It is led by the trained moderators Before inspection the documents are prepared and checked thoroughly by the reviewers Compare the work with its sources and other referenced documents and using check lists The defects found are documented in a logging list or issue log A formal follow-up is carried out by the moderator applying exit criteria

Coding Standards and Guidelines Standards are the established, fixed, have-to-follow-them rules—the do's and don'ts Guidelines are the suggested best practices, the recommendations Examples Corporate Terminology and Conventions. Industry Requirements. (e.g. medical, pharmaceutical ) Government Standards. Graphical User Interface (GUI) Security Standards Testers should be aware of these standards and test against them

Testing the Specification Specification Attributes Checklist Complete Accurate Precise, Unambiguous, and Clear Consistent Relevant Feasible Code-free Testable

Examples Complete: Is anything missing or forgotten? Sales needs to be able to see which contracts will be expiring within the upcoming 90 days. Who or what are "Sales"? What does "to see" mean? Do they need the physical contracts or just a list? What constitutes a contract? What makes a contract "expire" and why do they care? Upcoming 90 days? Starting from when? Does this view change day- by-day or weekly or monthly or hourly or what? What constitutes a day in this context, 24 hours (a day in a single location) or the global day (47 hours?)

Examples Precise, Unambiguous, and Clear: Is the description exact and not vague? Accurate: Is the proposed solution correct? Does it properly define the goal? The product shall communicate all roads predicted to freeze The product shall show the weather for the next 24 hours. The system shall be completely reliable The system shall be fast

Examples Code-free: Does the specification stick with defining the product and not the underlying software design, architecture, and code? Correction The product shall display pictures of goods for the customer to click on. The product shall enable the customer to select the goods he wishes to order.

Examples Testable: Is enough information provided that a tester could create tests to verify its operation? Consistent: Is the description of the feature written so that it doesn't conflict with itself Relevant: Is it extra information that should be left out? Feasible: Can the feature be implemented with the available resources? The network shall handle all unexpected errors without crashing.

List of Problem Words Always, Every, All, None, Never: make sure that it is, indeed, certain. Think of cases that violate Certainly, Therefore, Clearly, Obviously, Evidently accepting something as a given Some, Sometimes, Often, Usually, Ordinarily, Customarily, Most, Mostly too vague Good, Fast, Cheap, Efficient, Small, Stable. unquantifiable terms. They aren't testable If…Then…(but missing Else). what will happen if the “if” doesn't happen

Quiz Explain what a tester should worry about with this line from a spec: “The software will allow up to 100 million simultaneous connections, although no more than 1 million will normally be used” Explain what's wrong with this specification statement: “When the user selects the Compact Memory option, the program will compress the mailing list data as small as possible using a Huffman-sparse-matrix approach”

Code Review A systematic approach to examining source code in detail It is time consuming, so they use divide and conquer Review the code, not to evaluate the author of the code. It should be planned and managed in a professional manner

Code Review Checklist Data Declaration Errors bugs are caused by improperly declaring or using variables or constants Computation Errors Data Reference Errors bugs caused by using a variable, constant, array, string, or record that Comparison Errors Less than, greater than, equal, not equal, true, false Control Flow Errors control constructs in the language not behaving as expected Subroutine Parameter Errors incorrect passing of data to and from software subroutines Input / Output Errors reading from a file, accepting input from a keyboard or mouse, and writing to an output device such as a printer or screen

Code Review Checklist

Static Analysis Analysis of software development artifacts, e.g. requirements or code, carried out without execution of these software development artifacts, usually carried out by means of a supporting tool Static analyzer: A tool that carries out static analysis. Static analyzer can be used by: Programmers before code review Programmers before and during component integration Designers during modeling Analyst to check requirements

Types of Static Analysis Tools Check code standards Calculate code metrics Depth of nesting Cyclomatic complexity Frequency of comments Show the code structure (could be graphical ) Control flow: sequence of instruction execution Data flow: follow a data item as it is accessed and modified Compilers

Static Analysis Tools Types of errors that can be found by a static analysis tool. Reference a variable with uninitialized value Unreachable code Improper declaration of variables or declaring unused variables Security violations Standard violations Inconsistent interface with other components Syntax violation

Quiz 1.What key element makes formal reviews work? 2.Besides being more formal, what's the big difference between inspections and other types of reviews? 3.Name several advantages to performing static testing techniques.