1. Advances In Software Inspection

2. Content Software Development Process Informal Review
Software Inspection Process
Performed Software Inspections
Results
Conclusions and Future

3. SW Development Process
Doris Burckhart
4/27/2017
SW Development Process
Collect Requirements
identify needs & common issues, define priorities and work-plan, define components
Pre-design Investigations
perform pre-design investigations -> identify candidate technologies/techniques
High-level Design
develop high-level design
Detailed Design Implementation
refine design, implement code
By applying a Software Development Process the development has been divided into a number of sequential phases intended to help pace and organise the work. Each phase has been defined to produce an obvious deliverable, i.e. document and/or code.
The phases are: collect requirements; identify and evaluate candidate technologies and techniques capable of addressing the common issues identified from the requirements; produce a design for each component covering the most important aspects; refine the design to add more detail; implement and unit test according to the design; integrate with other components.
The in-house written code is comprised of about lines of C++ including external software.
Testing & Integration
unit test components
integrate with other subsystems
Software Development Environment

4. Review - Inspection Review: Inspection:
Doris Burckhart
4/27/2017
Review - Inspection
Review:
Presentation of each SW Component to the Group in each Development Phase
Discussion and Coordination with other components
Goal: Clarification and Accept/Reject Decision
Inspection:
Quality Improvement Process to the software project
Goal: Defect Detection & Defect Prevention
Reviews take the form of presentations followed by discussions during open meetings with all developers involved in the project. Each software component is presented to the group in each of the phases. The goal is clarification, enhancement and an accept or possibly reject decision.
Formal Inspection goes through documents and code thoroughly and in a well planned manner. The goal is defect detection and ultimately defect prevention.

5. Comparison of Inspections and Reviews
Issue
Technical Review
Inspection
Objective
Determine the suitablity of a work product for it's intended use.
Determine the suitability of a work product for it's intended use, but beyond that search for anomalies by examination through educated inspectors.
Roles
A minimum of two persons is required. Each one of them can assume multiple roles. Since the scope is different additional persons e.g. management or customer representative can participate, but this is not regarded as a role in the process.
Additionally required roles are the Author and the Reader. The roles are explicitly separated and can not be assumed by one person.
Input
The inputs are very similar than the ones of the inspection. It has to be observed that checklists are not mentioned i.e. not required.
Additional inputs are: Inspection reporting forms, Inspection checklists, Hardware product specifications, Hardware performance data. Some of these inputs are optional.

6. Issue
Technical Review
Inspection
Output
The only output is an Action Item List and a record (meeting minutes) of the technical review.
The outputs are a formal inspection report, a formal defect summary and a defect list with classified defects. The emphasis is on providing a standard output of found errors which also would allow statistical evaluations.
Entry Criteria for the Meeting
No special criteria mentioned.
It is explicitly stated that the meeting has to be re-scheduled if the inspection leader finds that the participants are not well prepared.
Meeting
Keep the described rules of a review meeting.
The defined roles have to be explicitly kept. The reader and not the author will present the material. The other roles also have to be formally kept.
Outcome of the Meeting
Generate of the defined review report.
At the end of the inspection the decision has to be made to accept the work product, close the inspection, but require a rework, reject the work product and require a re-inspection after the rework is done.

7. Informal Review From the start of the project Results: Drawback:
Doris Burckhart
4/27/2017
Informal Review
From the start of the project
document preparation and monthly open meetings
present status, results, proposals
inform colleagues - receive feedback
suggestions -> enhancements -> acceptance
Results:
Coherent set of end-product components
Increased communication
Drawback:
Lack of time of reviewers
No code review
Reviews have been performed since the beginning of the project. The developer of a component prepares the document which is read by his colleagues. Reviews are organised during the monthly Back-end meetings where a developer presents the status and results from a development phase or a list of items to be investigated for the next phase. The aim is to inform other project members and also to receive feedback. During the subsequent discussion open items are clarified, the relation to other project components is discussed and the component in its phase is accepted or suggestions for modifications and enhancements are made. Decisions for further development are taken. Each component of the Back-end project has been reviewed before progressing to the next phase and co-ordinated with the development of other components. Code has generally not been reviewed. Regular review has lead to a coherent set of end-product components. Reviews have created the basic culture for individual developers located in geographically distinct institutes to form a team and to work successfully in a common project. A drawback was that the reviews on the documentation and code were not performed thoroughly and often team members did not find the time to read the documents before the meeting.

8. Inspection Process Map
Doris Burckhart
4/27/2017
Inspection Process Map
Based on Tom Gilb’s Inspection method
Sources
Inspection Plan
Rules
Planning & Entry
Checklists
Inspection Team: Inspection Leader Authors Inspectors
Product
Kick-off Meeting
Issue log tables
Checking
Logging Meeting
Action Lists
Brainstorming
The Back-end inspection process applied in the Atlas Back-end software subsystem is based on the inspection method developed by Tom Gilb. Inspection is based on a well defined and organised procedure following a sequence of actions including meetings and work done by individuals. It is important to provide this basic framework in order to avoid inspection team participants wasting valuable time. Deviations from the proposed structure are allowed whenever it increases overall efficiency.
. Documents are checked for cleanness and consistency against rules which have been adopted by the project. Regularly updated concise checklists are provided for the inspection team members to help in their efficient participation. The development of in-house standards which are available for requirements, design and code allows to separate commonly agreed rules from personal taste.
Each Inspection is organised by an inspection leader. He assembles the Inspection team, which consists of the document author, three to five inspectors and the inspection leader
Change Requests
Edit
Follow-up
Data Summary
Exit
Exit Product

9. List of performed Inspections
Doris Burckhart
4/27/2017
List of performed Inspections
Requirement Inspection
Design Inspection
Code Inspection
180 pages of documents
8000 lines of code
Formal inspections were introduced while development was already well under way. Therefore only a limited number of deliverables which were produced could be inspected, only two of them in more than one phase. Three inspections on requirements, two on design and four on implementation documents were performed.
The evaluation was performed during the phase when software inspection was introduced to the project and its team. Initially a steep learning curve was observed, because inspection was started basically from an initial level of zero. A high number of issues of varying severity was found. With the implicit education in the team the number of issues found became lower while concentrating more around major defects. Authors who have participated in software inspection previously as an inspector generally provide a cleaner deliverable from the start.

10. Results: Issue log table
Doris Burckhart
4/27/2017
Results: Issue log table
Each issue is logged, discussed, checked
Per Inspection : 10 to 200 issues found Number of recorded issue logs depend on:
Type of Inspection
Phase of Project
Entry conditions
Experience of Inspectors
Counting system
-> Improved Code
-> Improved Documentation
Each inspector filled one set of issue log tables while inspecting and before the logging meeting. Non-trivial issues were discussed during the meeting. Subsequently the tables were edited by the author and cross-checked by the peer. The number of issue logs found depended on the type of inspection (requirements, design, code), the project phase, the entry conditions (availability of a clean set of rules and guidelines), the experience of the inspectors and on the counting system. Variations between 10 and 200 issue logs per inspection were found, minor issues included. Issue counting in the future will be restricted to major issues.
It is important to keep the criticism constructive when logging issues and during the discussion in the logging meeting.

11. Inspection Process Results
Doris Burckhart
4/27/2017
Inspection Process Results
Change Requests
To Rules for Requirements, Design or Coding use of ’shall’ ’should’ ’may’ for Requirements adopt standard command line parameters use of coding conventions program exit status convention
Action Lists
Actions to be performed outside the Inspection Process
Questions to be clarified, i.e. beyond the scope of Inspection
Change requests and action lists were discussed and recorded during the meeting. The need for changes of checklists or rules came up. Certain rules turned out to be too restrictive or not sufficiently concise, checklist had to be updated according to the evolution of the project and improvements concerning the inspection process and the development process itself were suggested. For example, for the requirements inspection a recommendation for the use of the words ‘shall’, ‘should’ and ‘may’ was found to be necessary; for coding the need for standard command line parameters was discovered and program exit status conventions were suggested. Action items came up for issues which were beyond the scope of the particular inspection and had to be handled outside, for example feasibility tests.

12. Observations Requirements Design Code Doris Burckhart 4/27/2017
Requirements inspection were found to be the most important. They are the first in the chain and a defect may propagate to the end product even with perfect code. They are also the least time consuming because no or few mother documents must be read and they are generally a few pages long only. According to general experience it takes about one hour to find a major defect by inspection at an early stage and about nine hours when testing. They turned out to be useful to re-visit and clarify strategy and goals. In one case we experienced the need for a redefinition of the component composition - a component was split into two distinct parts.
Design inspection are the hardest to perform. It was found to be difficult to define a good set of guidelines which is not trivial but also not too restrictive. The use of design patterns will be investigated.
Code inspection are the most time consuming ones. The number of documents involved is high: code must be checked for internal consistency and against coding rules, the users guide and the implementation documentation must be inspected and compared against the design and requirements documents. Automatic checking tools were employed where possible. Sampling has been performed in some cases and will be employed regularly in the future while concentrating on critical areas.

13. Adaptation to Environment
Doris Burckhart
4/27/2017
Adaptation to Environment
Everything is allowed
which helps improving
process
product
communication
cooperation, education
integration, coherence
while keeping Consistency
and
improving Efficiency
In the HEP environment a number of factors must be considered being different from the industrial world and their working methods and therefore software inspection has been adapted according to our needs. Participants in a project in the HEP DAQ environment are each expected to fulfil all the roles required for the development of a product in a sequential or parallel order whilst industry provides specialised departments for each task. In order to reach the same level of expertise project members with varying professional background would have to be trained formally in each of the areas and have working experience for a significant amount of time.
The group is comprised of a few Cern resident people and a number of collaborators from outside institutes with additional duties. Inspection could not be imposed by a project management as this is common in industry. Participants were convinced of the benefits by receiving personally the appropriate explanations and by making positive experience. We rely on participation by conviction.
A guiding principle for the introduction and for the use of the software inspection method is that everything is allowed which helps improving the product, the overall production process including communication amongst project participants or the inspection process itself while keeping consistency and improving efficiency.

14. Efficiency - Flexibility
Doris Burckhart
4/27/2017
Efficiency - Flexibility
Efficiency
Inspection is time consuming
- don’t waste time and effort of inspectors
Careful planning and Clear Instructions
Solid Process Framework
Inspect Samples
Motivation of peers
Flexibility
Build in change Management
Well defined procedure
- but each inspection to be handled individually
Because of the time-consuming nature of inspection efficiency (effectiveness within time) is important to avoid boredom and wasting inspectors time. A solid process framework with clear instructions, entry condition checking, focussed meetings, project specific checklists and rules help save time.Samples can be inspected and the choice of peers according to their own working area increases motivation.
Efficiency must be balanced with the flexibility of handling each inspection individually while recording deviations from the standard. Build-in change management concerning rules and the process itself help in keeping inspection lively and up-to-date.

15. Conclusions Team Success Reviews prepare the ground and stabilize SDP
Doris Burckhart
4/27/2017
Conclusions
Reviews prepare the ground and stabilize SDP
Adaptation of the inspection method for the Environment
Gain in quality and experience Appreciated by authors and peers
Help for team building in a distributed environment
Reviews have proven to help stabilise the Software Development Process and are an integral part. People get the chance to present their work, receive feedback and integrate it better into the common project.
During the evaluation of software inspection the method has been successfully introduced . Project members became familiar with the method, the procedure was adapted to the project and the working environment and sets of rules and checklist were established. Authors appreciated the gain in quality of their work.
Through Review and Inspection it becomes obvious already during the development that the success of the project relies on the entire team.
Team
Success

16. Future Quality Good understanding for the next phase:
Doris Burckhart
4/27/2017
Future
Good understanding for the next phase:
stabilize inspection process and keep style
provide a helpful framework based on experience
use it through entire development cycle
‘lighter’ inspection - faster turnaround time
use sampling techniques
keep real logging meetings where possible
stay flexible and efficient
A second more stable phase can be envisaged now. The aim is to apply inspection to all the deliverables in the project in the three basic development phases of requirements, design and implementation in particular at the start of the life cycle, which fits well to the start of the development of the final Atlas DAQ software.
Light’ inspection with faster turnaround time with the same checking criteria but through sampling and by concentrating on critical areas will be used in more cases, and metrics will be provided. Open logging meetings as opposed to electronically conducted inspections will be kept where possible. Dynamism and flexibility for the process must be kept alive. The quality level can be raised smoothly by raising the level of standards and rules -> with the aim of continuously improving quality.
Quality