Autumn 1996 Week 4 SEM Testing Large Systems. u Needs to be more systematic, since a team is involved and there may be no single person familar with the whole system. u “Top-down” or “bottom-up” testing. u Top-down testing starts with the main program (top-level) and tests this using program stubs, then inserts the lower modules in order. u Bottom-up testing starts with the lowest level modules and tests these using driver programs, then works up to the top-level. u Many systems use a mixture of these approaches with different people testing different modules at the same time.

Autumn 1996 Week 4 SEM Top-down Testing. u Ideally the modules are tested one at a time. u Sometimes it is necessary to test a small number simultaneously. e.g. 3 procedures, one to open a file, one to write to it, and a third to close it. u The lower level modules are replaced by “stubs” u A stub has the same parameters as the real module. u It writes a message to say that control has passed correctly to this module. u If results are returned, then known values are set and supplied to the calling module.

Autumn 1996 Week 4 SEM Top-down Testing. u 1. The main module is tested using stubs for all other modules. u 2. Subordinate stubs are replaced by real modules one at a time. u 3. Tests are carried out as each real module is inserted. u 4. On completion of a set of tests, another stub is replaced with a real module. u 5. Regression testing (repeating some or all of the previous tests) may also be carried out to ensure that no new errors have been introduced.

Autumn 1996 Week 4 SEM Bottom -up Testing. u Drivers must be constructed for the lower level modules. The complete program cannot be tested or shown until all the modules have been tested. u 1. Low-level modules are combined into clusters (may have a cluster of one). u 2. A driver is written to coordinate input and output. u 3. The cluster is tested. u 4. Drivers are removed and clusters are combined, moving up the program structure.

Autumn 1996 Week 4 SEM Comparison. u In practivce, a combination of these methods is used. u Disadvantages of top-down testing u Stubs may be difficult to produce. u Output may be difficult to interpret. u These are the advantages of bottom-up testing.

Autumn 1996 Week 4 SEM Error prevention at the Design Stage. u “Error detection” means discovery of an error within a system. u “Passive error detection” - found during execution of system (i.e. it crashes). u “Active error detection” - found as result of a test in the software. u Defensive Designing - each module is written assuming that the other modules contain errors. Hence all data supplied to the module must be checked before it is executed (as far as possible) Can check type and range of values, but not actual value.

Autumn 1996 Week 4 SEM Error Correction. u 1. Build redundancy into the system - where possible add extra features to check the values generated so far. Extreme case, two functionally- equivalent systems produced by two different teams and run in parallel. u Error Correction - what to do if an error is found. Usualy either u the system is halted and information printed. u for some errors, the system merely records the error and proceeds. u Error Tolerance - isolation of damage caused by an error. Applies when system needs to carry on after an error has been detected.

Autumn 1996 Week 4 SEM Examining a Design. u 1. Review - someone else reads the specification and comments. u 2. Walkthrough - the designer leads a group of colleagues through a section of design. The others ask questions, comment on possible errors, aspects not covered etc. u 3. Inspection - formal evaluation of a written document by a group other than the author. It is examined in detail for faults or violation of standards or requirements.

Autumn 1996 Week 4 SEM Types of Testing Requirements Analysis System Design Implementation Installation Maintenance SYSTEM TESTING Tested as a whole VALIDATION TESTING Results of requirements analysis compared with software constructed. INTEGRATION TESTING construction and verification of program and data structures. MODULE TESTING concentrates on each module in turn. ACCEPTANCE TESTING tests the system with real data. Both verification and Validation.

Autumn 1996 Week 4 SEM Testing Large Systems. u There are several different stages of testing, namely module testing, subsystem testing, integegration testing and acceptance testing. u “Alpha testing” - system developer tests sytem with real data in presence of the customer. This continues until the system is considered satisfactory. u “Beta testing” - system is delivered to a number of customers, who use it and report back on problems.

Autumn 1996 Week 4 SEM Inherent Problems in Software System Design u 1. Building large systems is a very complex problem. u 2. Developers must understand not only the tools and techniques of software engineering but also details of the application domain. u 3. Developers must also deal with an enormous variety of user requirements. u 4. As computer systems change (often very rapidly), much has to be re-learnt in order to keep up-to-date.

Autumn 1996 Week 4 SEM C.A.S.E. u Computer Aided Software Engineering. u Term was coined in early 1980s to denote the use of tools which help in the software development cycle. CASE focusses on requirements analysis and design specification. u “Workbenches” are systems designed to support analysis and design stages. They contain tools for diagram editing system, facilities for design anlysis and checking, data dictionaries, query languages, report generation, forms generation tools and skeleton code generators. u They make it easy to try out prototypes as part of the design of the final system.

Autumn 1996 Week 4 SEM Z and Formal Methods. u When you are part of a large team, it is very important to have all the modules and interfaces rigorously defined. u The use of a formal language for such definitions reduces the opportunity for misuderstandings. u Z describes what a modules does and not how it does it. u To do this, it provides a rigorous definition of the input data and the output results. If the modules have been correctly defined, with no global variables (and hence no strange side-effects) then this is all that the other modules need to know.

Autumn 1996 Week 4 SEM Example of Z Specification. in?out ! in? > = 0 out ! = 3 in? Power in?, out ! : Z in? >= 0 out ! = 3 in? u Consider a very simple example. A module which receives a non-negative integer and calculates the value of 3 raised to that power. A diagrammatic version would be: u This may also appear as a schema.

Autumn 1996 Week 4 SEM References. u Software Engineering - Sommerville. chapter 9 pp u Software Engineering - Pressman. chapter 9 pp (less detail). u Z An Introduction to Formal Methods - Diller. especially part I.

Autumn 1996 Week 4 SEM Z Terminology u In the declaration part of the schema for Power, we had in?, out : Z u This is equivalent to the two statements: in? belongs to Z, the set of all integers. out belongs to Z, the set of all integers. u In the predicate part, we had the two formulas in? >= 0 out = 3 in?

Autumn 1996 Week 4 SEM More Terminology u N = = { n:Z | n>=0. n} u N is defined as a set of objects where { n belongs to the set of all integers | n is greater than or equal to zero. object is n} u i.e. N is the set of all positive integers. u I suggest you use either the full mathematical terminology or the full English version.

Autumn 1996 Week 4 SEM Describing Sets. u x is a member of the set X u x belongs to X  x  X u These all have the same meaning. You can use any of them, although the last is the usual mathematical terminology for a Z schema.

Autumn 1996 Week 4 SEM Operations on Sets. u Union. X U Y is the union of set X and set Y. It contains all elements which are members of either set X or set Y. Duplicates are only included once. u Intersection. X Y is the intersection of the sets X and Y. It contains all elements which occur in both X and Y. u Cartesian Product X x Y. This is the set of all pairs whose first element belongs to X and whose second element belongs to Y.

Autumn 1996 Week 4 SEM Example of Cartesian Product. u Suppose X is the set of letters {a,b,c,d} and Y is the set of numbers {3.. 6} u Then XxY is the set of all pairs with a letter from X and a number from Y. e.g. (a,3), (a,4), (a,5), (a,6), (d,6), (d,5), (d,3), (d,4), (c,3), (b,6), (b,3), (c,6),..... u Can you fill in the remaining ones?

Autumn 1996 Week 4 SEM Consider a telephone system. u Let Person be the set of all people eligible to have use of a telephone. u Let Phone be the set of all possible telephone numbers. u Then we have the Cartesian product ( person, Phone) u The actual pairs is the set telephones, which is part of this Cartesian product. u i.e. telephones is contained in Person x Phone

Autumn 1996 Week 4 SEM Example of a Schema  The mastermind program has a module to generate a code. Codegen Colour : {red, blue, green, yellow} Code4 : Colour x Colour x Colour x Colour out : Code4 out  Code4

Autumn 1996 Week 4 SEM Descriptions of Modules. u Pseudo-code, flowcharts, etc describe how a module achieves its results. u They can be converted into code in a particular language. u There is usually only one correct piece of code for one flow-chart. u Z-specification describes the input, the output and the relation between them. u It makes no attempt to describe how this is achieved. u There are usually several equally correct implementations of a Z-specification.

Autumn 1996 Week 4 SEM Use of Z specification. u Once a Z specification has been agreed, ANY code which carries out this function is acceptable. u Some may be more efficient than others, and these will be preferred. u Other considerations may apply - e.g if the rest of the program is written in one language, it may be desirable to have this module in the same language. u It is often useful for the module to check that the input data is in the correct format and provide some sort of error indication if it is not.