Slide 14.1 © The McGraw-Hill Companies, 2002 Object-Oriented and Classical Software Engineering Fifth Edition, WCB/McGraw-Hill, 2002 Stephen R. Schach

Slide 14.2 © The McGraw-Hill Companies, 2002 CHAPTER 14 IMPLEMENTATION PHASE

Slide 14.3 © The McGraw-Hill Companies, 2002 Overview l Choice of programming language l Fourth generation languages l Good programming practice l Coding standards l Module reuse l Module test case selection l Black-box module-testing techniques l Glass-box module-testing techniques

Slide 14.4 © The McGraw-Hill Companies, 2002 Overview (contd) l Code walkthroughs and inspections l Comparison of module-testing techniques l Cleanroom l Potential problems when testing objects l Management aspects of module testing l When to rewrite rather than debug a module l CASE tools for the implementation phase l Air Gourmet Case Study: Black-box test cases l Challenges of the implementation phase

Slide 14.5 © The McGraw-Hill Companies, 2002 Implementation Phase l Programming-in-the-many l Choice of Programming Language –Language is usually specified in contract l But what if the contract specifies –The product is to be implemented in the “most suitable” programming language l What language should be chosen?

Slide 14.6 © The McGraw-Hill Companies, 2002 Choice of Programming Language (contd) l Example –QQQ Corporation has been writing COBOL programs for over 25 years –Over 200 software staff, all with COBOL expertise –What is “most suitable” programming language? l Obviously COBOL

Slide 14.7 © The McGraw-Hill Companies, 2002 Choice of Programming Language (contd) l What happens when new language (C++, say) is introduced –New hires –Retrain existing professionals –Future products in C++ –Maintain existing COBOL products –Two classes of programmers »COBOL maintainers (despised) »C++ developers (paid more) –Need expensive software, and hardware to run it –100s of person-years of expertise with COBOL wasted

Slide 14.8 © The McGraw-Hill Companies, 2002 Choice of Programming Language (contd) l Only possible conclusion –COBOL is the “most suitable” programming language l And yet, the “most suitable” language for the latest project may be C++ –COBOL is suitable for only DP applications l How to choose a programming language –Cost-benefit analysis –Compute costs, benefits of all relevant languages

Slide 14.9 © The McGraw-Hill Companies, 2002 Choice of Programming Language (contd) l Which is the most appropriate object-oriented language? –C++ is (unfortunately) C-like –Java enforces the object-oriented paradigm –Training in the object-oriented paradigm is essential before adopting any object-oriented language l What about choosing a fourth generation language (4GL)?

Slide © The McGraw-Hill Companies, 2002 Fourth Generation Languages l First generation languages –Machine languages l Second generation languages –Assemblers l Third generation languages –High-level languages (COBOL, FORTRAN, C++) l Fourth generation languages (4GLs) –One 3GL statement is equivalent to 5–10 assembler statements –Each 4GL statement intended to be equivalent to 30 or even 50 assembler statements

Slide © The McGraw-Hill Companies, 2002 Fourth Generation Languages (contd) l It was hoped that 4GLs would –Speed up application-building –Applications easy, quick to change »Reducing maintenance costs –Simplify debugging –Make languages user friendly »Leading to end-user programming l Achievable if 4GL is a user friendly, very high-level language

Slide © The McGraw-Hill Companies, 2002 Fourth Generation Languages (contd) l Example –(“Just in Case You Wanted to Know” box, page 438) l The power of a nonprocedural language, and the price

Slide © The McGraw-Hill Companies, 2002 Productivity Increases with a 4GL? l The picture is not uniformly rosy l Problems with –Poor management techniques –Poor design methods l James Martin suggests use of –Prototyping –Iterative design –Computerized data management –Computer-aided structuring l Is he right? Does he (or anyone else) know?

Slide © The McGraw-Hill Companies, 2002 Actual Experiences with 4GLs l Playtex used ADF, obtained an 80 to 1 productivity increase over COBOL –However, Playtex then used COBOL for later applications l 4GL productivity increases of 10 to 1 over COBOL have been reported –However, there are plenty of reports of bad experiences

Slide © The McGraw-Hill Companies, 2002 Actual Experiences with 4GLs (contd) l Attitudes of 43 Organizations to 4GLs –Use of 4GL reduced users’ frustrations –Quicker response from DP department –4GLs slow and inefficient, on average –Overall, 28 organizations using 4GL for over 3 years felt that the benefits outweighed the costs

Slide © The McGraw-Hill Companies, 2002 Fourth Generation Languages (contd) l Market share –No one 4GL dominates the software market –There are literally hundreds of 4GLs –Dozens with sizable user groups l Reason –No one 4GL has all the necessary features l Conclusion –Care has to be taken in selecting the appropriate 4GL

Slide © The McGraw-Hill Companies, 2002 Key Factors When Using a 4GL l Large sums for training l Management techniques for 4GL, not for COBOL l Design methods must be appropriate, especially computer-aided design l Interactive prototyping l 4GLs and complex products

Slide © The McGraw-Hill Companies, 2002 Key Factors When Using a 4GL (contd) l Dangers of a 4GL –Deceptive simplicity –End-user programming

Slide © The McGraw-Hill Companies, 2002 Good Programming Practice l Use of “consistent” and “meaningful” variable names –“Meaningful” to future maintenance programmer –“Consistent” to aid maintenance pyrogrammer

Slide © The McGraw-Hill Companies, 2002 Good Programming Practice Example Module contains variables freqAverage, frequencyMaximum, minFr, frqncyTotl l Maintenance programmer has to know if freq, frequency, fr, frqncy all refer to the same thing –If so, use identical word, preferably frequency, perhaps freq or frqncy, not fr –If not, use different word (e.g., rate ) for different quantity l Can use frequencyAverage, frequencyMyaximum, frequencyMinimum, frequencyTotal l Can also use averageFrequency, maximumFrequency, minimumFrequency, totalFrequency l All four names must come from the same set

Slide © The McGraw-Hill Companies, 2002 Good Programming Practice (contd) l Issue of self-documenting code –Exceedingly rare l Key issue: Can module be understood easily and unambiguously by –SQA team –Maintenance programmers –All others who have to read the code

Slide © The McGraw-Hill Companies, 2002 Good Programming Practice (contd) l Example –Variable xCoordinateOfPositionOfRobotArm –Abbreviated to xCoord –Entire module deals with the movement of the robot arm –But does the maintenance programmer know this?

Slide © The McGraw-Hill Companies, 2002 Prologue Comments l Mandatory at top of every single module –Minimum information »Module name »Brief description of what the module does »Programmer’s name »Date module was coded »Date module was approved, and by whom »Module parameters »Variable names, in alphabetical order, and uses »Files accessed by this module »Files updated by this module »Module i/o »Error handling capabilities »Name of file of test data (for regression testing) »List of modifications made, when, approved by whom »Known faults, if any

Slide © The McGraw-Hill Companies, 2002 Other Comments l Suggestion –Comments are essential whenever code is written in a non-obvious way, or makes use of some subtle aspect of the language l Nonsense! –Recode in a clearer way –We must never promote/excuse poor programming –However, comments can assist maintenance programmers l Code layout for increased readability –Use indentation –Better, use a pretty-printer –Use blank lines

Slide © The McGraw-Hill Companies, 2002 Nested if Statements l Example –Map consists of two squares. Write code to determine whether a point on the Earth’s surface lies in map square 1 or map square 2, or is not on the map

Slide © The McGraw-Hill Companies, 2002 Nested if Statements (contd) l Solution 1. Badly formatted

Slide © The McGraw-Hill Companies, 2002 Nested if Statements (contd) l Solution 2. Well-formatted, badly constructed

Slide © The McGraw-Hill Companies, 2002 Nested if Statements (contd) l Solution 3. Acceptably nested

Slide © The McGraw-Hill Companies, 2002 Nested if Statements (contd) l Combination of if-if and if-else-if statements is usually difficult to read l Simplify: The if-if combination if is frequently equivalent to the single condition if && l Rule of thumb –if statements nested to a depth of greater than three should be avoided as poor programming practice

Slide © The McGraw-Hill Companies, 2002 Programming Standards l Can be both a blessing and a curse l Modules of coincidental cohesion arise from rules like –“Every module will consist of between 35 and 50 executable statements” l Better –“Programmers should consult their managers before constructing a module with fewer than 35 or more than 50 executable statements”

Slide © The McGraw-Hill Companies, 2002 Remarks on Programming Standards l No standard can ever be universally applicable l Standards imposed from above will be ignored l Standard must be checkable by machine

Slide © The McGraw-Hill Companies, 2002 Remarks on Programming Standards (contd) l Examples of good programming standards –“Nesting of if statements should not exceed a depth of 3, except with prior approval from the team leader” –“Modules should consist of between 35 and 50 statements, except with prior approval from the team leader” –“Use of goto s should be avoided. However, with prior approval from the team leader, a forward goto may be used for error handling”

Slide © The McGraw-Hill Companies, 2002 Remarks on Programming Standards (contd) l Aim of standards is to make maintenance easier –If it makes development difficult, then must be modified –Overly restrictive standards are counterproductive –Quality of software suffers

Slide © The McGraw-Hill Companies, 2002 Software Quality Control l After preliminary testing by the programmer, the module is handed over to the SQA group

Slide © The McGraw-Hill Companies, 2002 Module Reuse l The most common form of reuse

Slide © The McGraw-Hill Companies, 2002 Module Test Case Selection l Worst way—random testing l Need systematic way to construct test cases

Slide © The McGraw-Hill Companies, 2002 Module Test Case Selection (contd) l Two extremes to testing l 1.Test to specifications (also called black- box, data-driven, functional, or input/output driven testing) –Ignore code. Use specifications to select test cases l 2.Test to code (also called glass-box, logic- driven, structured, or path-oriented testing) –Ignore specifications. Use code to select test cases

Slide © The McGraw-Hill Companies, 2002 Feasibility of Testing to Specifications l Example –Specifications for data processing product include 5 types of commission and 7 types of discount –35 test cases l Cannot say that commission and discount are computed in two entirely separate modules—the structure is irrelevant

Slide © The McGraw-Hill Companies, 2002 Feasibility of Testing to Specifications l Suppose specs include 20 factors, each taking on 4 values –4 20 or 1.1  test cases –If each takes 30 seconds to run, running all test cases takes > 1 million years l Combinatorial explosion makes testing to specifications impossible

Slide © The McGraw-Hill Companies, 2002 Feasibility of Testing to Code l Each path through module must be executed at least once –Combinatorial explosion

Slide © The McGraw-Hill Companies, 2002 Feasibility of Testing to Code (contd) l Flowchart has over different paths

Slide © The McGraw-Hill Companies, 2002 Feasibility of Testing to Code (contd) l Can exercise every path without detecting every fault

Slide © The McGraw-Hill Companies, 2002 Feasibility of Testing to Code (contd) l Path can be tested only if it is present l Weaker Criteria –Exercise both branches of all conditional statements –Execute every statement

Slide © The McGraw-Hill Companies, 2002 Feasibility of Testing to Code (contd) l Can exercise every path without detecting every fault l Path can be tested only if it is present l Weaker Criteria –Exercise both branches of all conditional statements –Execute every statement

Slide © The McGraw-Hill Companies, 2002 Coping with the Combinatorial Explosion l Neither testing to specifications nor testing to code is feasible l The art of testing: l Select a small, manageable set of test cases to –Maximize chances of detecting fault, while –Minimizing chances of wasting test case l Every test case must detect a previously undetected fault

Slide © The McGraw-Hill Companies, 2002 Coping with the Combinatorial Explosion l We need a method that will highlight as many faults as possible –First black-box test cases (testing to specifications) –Then glass-box methods (testing to code)

Slide © The McGraw-Hill Companies, 2002 Black-Box Module Testing Methods l Equivalence Testing l Example –Specifications for DBMS state that product must handle any number of records between 1 and 16,383 (2 14 –1) –If system can handle 34 records and 14,870 records, then probably will work fine for 8,252 records l If system works for any one test case in range (1..16,383), then it will probably work for any other test case in range –Range (1..16,383) constitutes an equivalence class l Any one member is as good a test case as any other member of the class

Slide © The McGraw-Hill Companies, 2002 Equivalence Testing (contd) l Range (1..16,383) defines three different equivalence classes: –Equivalence Class 1: Fewer than 1 record –Equivalence Class 2: Between 1 and 16,383 records –Equivalence Class 3: More than 16,383 records

Slide © The McGraw-Hill Companies, 2002 Boundary Value Analysis l Select test cases on or just to one side of the boundary of equivalence classes –This greatly increases the probability of detecting fault

Slide © The McGraw-Hill Companies, 2002 Database Example l Test case 1: 0 recordsMember of equivalence class 1 (and adjacent to boundary value) l Test case 2: 1 recordBoundary value l Test case 3:2 recordsAdjacent to boundary value l Test case 4:723 recordsMember of equivalence class 2

Slide © The McGraw-Hill Companies, 2002 Boundary Value Analysis of Output Specs l Example: In 2001, the minimum Social Security (OASDI) deduction from any one paycheck was $0.00, and the maximum was $4, –Test cases must include input data which should result in deductions of exactly $0.00 and exactly $4, –Also, test data that might result in deductions of less than $0.00 or more than $4,984.80

Slide © The McGraw-Hill Companies, 2002 Overall Strategy l Equivalence classes together with boundary value analysis to test both input specifications and output specifications –Small set of test data with potential of uncovering large number of faults

Slide © The McGraw-Hill Companies, 2002 Glass-Box Module Testing Methods l Structural testing –Statement coverage –Branch coverage –Linear code sequences –All-definition-use path coverage

Slide © The McGraw-Hill Companies, 2002 Structural Testing: Statement Coverage l Statement coverage: –Series of test cases to check every statement –CASE tool needed to keep track l Weakness –Branch statements l Both statements can be executed without the fault showing up

Slide © The McGraw-Hill Companies, 2002 Structural Testing: Branch Coverage l Series of tests to check all branches (solves above problem) –Again, a CASE tool is needed l Structural testing: path coverage

Slide © The McGraw-Hill Companies, 2002 Linear Code Sequences l In a product with a loop, the number of paths is very large, and can be infinite –We want a weaker condition than all paths but that shows up more faults than branch coverage l Linear code sequences –Identify the set of points L from which control flow may jump, plus entry and exit points –Restrict test cases to paths that begin and end with elements of L –This uncovers many faults without testing every path

Slide © The McGraw-Hill Companies, 2002 All-definition-use-path Coverage l Each occurrence of variable, zz say, is labeled either as –The definition of a variable zz = 1 or read (zz) –or the use of variable y = zz + 3 or if (zz < 9) errorB () l Identify all paths from the definition of a variable to the use of that definition –This can be done by an automatic tool l A test case is set up for each such path

Slide © The McGraw-Hill Companies, 2002 All-definition-use-path Coverage (contd) l Disadvantage: –Upper bound on number of paths is 2 d, where d is the number of branches l In practice –The actual number of paths is proportional to d in real cases l This is therefore a practical test case selection technique

Slide © The McGraw-Hill Companies, 2002 Infeasible Code l It may not be possible to test a specific statement –May have an infeasible path (“dead code”) in the module l Frequently this is evidence of a fault

Slide © The McGraw-Hill Companies, 2002 Measures of Complexity l Quality assurance approach to glass-box testing –Module m1 is more “complex” than module m2 l Metric of software complexity –Highlights modules mostly likely to have faults l If complexity is unreasonably high, then redesign, reimplement –Cheaper and faster

Slide © The McGraw-Hill Companies, 2002 Lines of Code l Simplest measure of complexity l Underlying assumption: –Constant probability p that a line of code contains a fault l Example –Tester believes line of code has 2% chance of containing a fault. –If module under test is 100 lines long, then it is expected to contain 2 faults l Number of faults is indeed related to the size of the product as a whole

Slide © The McGraw-Hill Companies, 2002 Other Measures of Complexity l Cyclomatic complexity M (McCabe) –Essentially the number of decisions (branches) in the module –Easy to compute –A surprisingly good measure of faults (but see later) l Modules with M > 10 have statistically more errors (Walsh)

Slide © The McGraw-Hill Companies, 2002 Software Science Metrics l [Halstead] Used for fault prediction –Basic elements are the number of operators and operands in the module l Widely challenged l Example

Slide © The McGraw-Hill Companies, 2002 Problem with These Metrics l Both Software Science, cyclomatic complexity: –Strong theoretical challenges –Strong experimental challenges –High correlation with LOC l Thus we are measuring LOC, not complexity l Apparent contradiction –LOC is a poor metric for predicting productivity l No contradiction — LOC is used here to predict fault rates, not productivity

Slide © The McGraw-Hill Companies, 2002 Code Walkthroughs and Inspections l Rapid and thorough fault detection –Up to 95% reduction in maintenance costs [Crossman, 1982]

Slide © The McGraw-Hill Companies, 2002 Comparison: Module Testing Techniques l Experiments comparing –Black-box testing –Glass-box testing –Reviews l (Myers, 1978) 59 highly experienced programmers –All three methods equally effective in finding faults –Code inspections less cost-effective l (Hwang, 1981) –All three methods equally effective

Slide © The McGraw-Hill Companies, 2002 Comparison: Module Testing Techniques (contd) l Tests of 32 professional programmers, 42 advanced students in two groups (Basili and Selby, 1987) l Professional programmers –Code reading detected more faults –Code reading had a faster fault detection rate l Advanced students, group 1 –No significant difference between the three methods l Advanced students, group 2 –Code reading and black-box testing were equally good –Both outperformed glass-box testing

Slide © The McGraw-Hill Companies, 2002 Comparison: Module Testing Techniques (contd) l Conclusion –Code inspection is at least as successful at detecting faults as glass-box and black-box testing

Slide © The McGraw-Hill Companies, 2002 Cleanroom l Different approach to software development l Incorporates –Incremental process model –Formal techniques –Reviews

Slide © The McGraw-Hill Companies, 2002 Cleanroom (contd) l Case study l 1820 lines of FoxBASE (U.S. Naval Underwater Systems Center, 1992) –18 faults detected by “functional verification” –Informal proofs –19 faults detected in walkthroughs before compilation –NO compilation errors –NO execution-time failures

Slide © The McGraw-Hill Companies, 2002 Cleanroom (contd) l Fault counting procedures differ: l Usual paradigms –Count faults after informal testing (once SQA starts) l Cleanroom –Count faults after inspections (once compilation starts)

Slide © The McGraw-Hill Companies, 2002 Cleanroom (contd) l Report on 17 Cleanroom products [Linger, 1994] –350,000 line product, team of 70, 18 months –1.0 faults per KLOC –Total of 1 million lines of code –Weighted average: 2.3 faults per KLOC –“[R]emarkable quality achievement”

Slide © The McGraw-Hill Companies, 2002 Testing Objects l We must inspect classes, objects l We can run test cases on objects l Classical module –About 50 executable statements –Give input arguments, check output arguments l Object –About 30 methods, some with 2, 3 statements –Do not return value to caller—change state –It may not be possible to check state—information hiding –Method determine balance —need to know accountBalance before, after

Slide © The McGraw-Hill Companies, 2002 Testing Objects (contd) l Need additional methods to return values of all state variables –Part of test plan –Conditional compilation l Inherited method may still have to be tested

Slide © The McGraw-Hill Companies, 2002 Testing Objects (contd) l Java implementation of tree hierarchy

Slide © The McGraw-Hill Companies, 2002 Testing Objects (contd) l Top half l When displayNodeContents is invoked in BinaryTree, it uses RootedTree.printRoutine

Slide © The McGraw-Hill Companies, 2002 Testing Objects (contd) l Bottom half l When displayNodeContents is invoked in method BalancedBinaryTree, it uses BalancedBinaryTree.printRoutine

Slide © The McGraw-Hill Companies, 2002 Testing Objects (contd) l Bad news –BinaryTree.displayNodeContents must be retested from scratch when reused in method BalancedBinaryTree –Invokes totally new printRoutine l Worse news –For theoretical reasons, we need to test using totally different test cases

Slide © The McGraw-Hill Companies, 2002 Testing Objects (contd) l Two testing problems: l Making state variables visible –Minor issue l Retesting before reuse –Arises only when methods interact –We can determine when this retesting is needed [Harrold, McGregor, and Fitzpatrick, 1992] l Not reasons to abandon the paradigm

Slide © The McGraw-Hill Companies, 2002 Module Testing: Management Implications l We need to know when to stop testing –Cost–benefit analysis –Risk analysis –Statistical techniques

Slide © The McGraw-Hill Companies, 2002 When to Rewrite Rather Than Debug l When a module has too many faults –It is cheaper to redesign, recode l Risk, cost of further faults

Slide © The McGraw-Hill Companies, 2002 Fault Distribution In Modules Is Not Uniform l [Myers, 1979] –47% of the faults in OS/370 were in only 4% of the modules l [Endres, 1975] –512 faults in 202 modules of DOS/VS (Release 28) –112 of the modules had only one fault –There were modules with 14, 15, 19 and 28 faults, respectively –The latter three were the largest modules in the product, with over 3000 lines of DOS macro assembler language –The module with 14 faults was relatively small, and very unstable –A prime candidate for discarding, recoding

Slide © The McGraw-Hill Companies, 2002 Fault Distribution In Modules Not Uniform (contd) l For every module, management must predetermine maximum allowed number of faults during testing l If this number is reached –Discard –Redesign –Recode l Maximum number of faults allowed after delivery is ZERO

Slide © The McGraw-Hill Companies, 2002 Air Gourmet Case Study: Black-Box Test Cases l Sample black-box test cases l Appendix J contains complete set

Slide © The McGraw-Hill Companies, 2002 Challenges of the Implementation Phase l Module reuse needs to be built into the product from the very beginning l Reuse must be a client requirement l Software project management plan must incorporate reuse