Certification of Reusable Software Artifacts

Many (failed) reuse programs assume that software components that have been tested and used are candidates to be reused. However, testing is not enough to guarantee successful reuse of a component. Frequently additional testing, evaluation, and documentation are necessary for systematic, efficient reuse. The additional testing, evaluation, and documentation of software artifacts is called "certification."

The Need for Certification Suppose that a preliminary set of requirements has been developed for a new software system. At first glance, domain analysis and library or repository assessment (whether locally- developed or not) seem sufficient for successful software reuse.

Domain analysis lets the software engineer or manager determine the appropriate reusable assets that an ideal system would need. Assessment of available repositories allows the software engineer to determine if the desired reusable asset was available. If it were available, then the component could be included in the new system.

Typical software testing practice: • Many organizations require that each delivered source code module be tested according to organizational standards. • Perhaps each decision path within the module is to be executed by some test data during the testing phase. • If no errors occur, the module is assumed to be correct. • Correct modules are tested and then used.

Unfortunately, much more is necessary if the system is of any size or complexity. • Domain analysis is an abstraction and must allow for inexact, approximate matches between different terms. • Library search algorithms are imperfect and must rely on inexact approximations. • An artifact in a reuse library may undergo some degree of modification before it is placed into service in a new system.

Suppose the same module is reused in an application other than the one for which it was ordinarily created. • If the new application system has a real-time requirement, then it is not immediately clear that the module can be used in the new system, because of the new, real-time constraint.

If the documentation for the module included an estimate for an upper bound of the running time of the module, in terms of the size of the data the number of arithmetic operations the actual time in a particular environment then the module is a much more attractive candidate for reuse than an equivalent module without it.

The degree to which the artifact is reused should be reflected in the cost estimate for the new system. A source code module that is 90 percent unchanged from the original component in the reuse library should require less testing than a new source code component of the same size and complexity. The amount of testing would be different for 75% reuse and for 50 % reuse.

Certification is to be done in addition to testing Certification is to be done in addition to testing. Consider a source code module that is a candidate for inclusion in a reuse library. Good software engineering practice requires that a description of the module's interface number and type of arguments, return type, purpose, and algorithm be included in the documentation of the module. The documentation of the system is adequate for the initial application. There may be other problems of interface, standards, etc., for which the documentation is not adequate.

Suggestions for Certification of Source Code: • domain analysis • metrics At least: • Size of the interconnection to other modules. • Number of logical predicates in the module. • Stability of the module • Coding problems, such as memory allocation or pointer access

C code should be run through the lint utility if available. It should also be run through a C++ compiler to obtain more information about inconsistencies in: • number and type of arguments to a function • a function's return type.

Atomic operations Any source code that makes use of a shared system resource must be atomic. Atomic operations, once started, are either: • uninterruptable or else • essential data must be rolled back to a previous state if the CPU is interrupted.

Operating system calls can usually be assumed to be atomic Operating system calls can usually be assumed to be atomic. However, using the return values of user-defined functions that contain system calls within their code, instead of using system calls directly, may leave essential data in an inconsistent state if there is a context switch during that function's execution.

Standards Source code should be evaluated to make sure it meets: interface standards the organization's coding standards.

test plan A test plan and the set of test results should be available and linked to the module when both are placed into the reuse library.

untested logic Any untested logic in the module should be indicated in the module's documentation so that the probability of an untested logical error in the module when used in a new environment can be estimated.

documentation Documentation should indicate performance of the module in a real-time environment. Documentation should also indicate system resources needed for the module. Documentation should be read by a person not familiar with the module in its original setting.

Requirements Certification • domain analysis • Each entry in the requirements traceability matrix should be included in the domain analysis. • metrics function point metrics or similar • Cost estimates for previous systems built using requirements should be compared to the actual costs and deviations should be noted and explained.

Design Certification • domain analysis • metrics .. interconnection between modules. • cost estimates for the previous systems are compared to the actual costs and any unusual deviations should be noted and explained. • documentation The documentation should be read by a person not familiar with the module in its original setting.

Software reliability and quality The idea is to measure the number of errors and determine the pattern over time. This approach can help: • Determine when to stop testing. • Identify high quality systems. • Identify low quality systems (which are poor candidates for reuse).

The certify on demand approach ••• • Place all potentially-reusable components in a reuse library • Identify those components that are not certified with a special symbol or flag. • Domain analysis process and library search algorithms are the same as before.

Certify on Demand (continued) • If the component is certified, the component is used as desired, either as is, or with necessary modifications. This is exactly the same procedure as when all reuse library components are certified. • If the reuse library component is not certified, it should be given additional testing because it may not have been tested previously in the new environment for which it was intended.

Certify on Demand (continued) The idea of deferring an action is common in computer science: • deferred evaluation of operands in expressions • late binding in Lisp programs • lazy look-ahead in Pascal.

To delay the certification process until the software artifact is actually to be reused avoids some up front costs but can slow down the software development process at a critical stage. Two general rules of Biggerstaff (1987) are useful for considering this method: "Don't attempt to reuse something unless you intend to use it at least three times." "Don't attempt to reuse something unless you haven't already used it at least three times."