A Controlled Experiment in Maintenance Comparing Design Patterns to Simpler Solutions By Prechelt, Unger, Tichy, Brössler, and Votta Presentation by Chris DeCelles

What are Design Patterns? Software design patterns package proven solutions to recurring design problems in a form that simplifies reuse Define terminology to improve communication from designers to maintainers

Purpose of Experiment To determine whether or not the use of design patterns in development of programs is beneficial to maintainers of those programs

Experiment Design 2 different versions of each program PAT: version employing design patterns ALT: “alternative” version employing simpler design than PAT Amount of pattern knowledge PRE: before taking pattern course POST: after taking pattern course Language: C++ All groups had similar experience in C++

Observer: Stock Ticker (ST) Stock Ticker program directs continuous stream of stock trades from stock market to one or more displays that are also part of program

Stock Ticker PAT vs. ALT ST PAT Consists of seven classes 343 lines Contains OBSERVER pattern in which 4 of 7 classes participate ST ALT Consists of seven classes 279 lines Includes one class that contains instance variable for each display; no dynamic registration of observers

ST Work Task 1 “In the given program listing only one of the two concrete display types is used. Enhance the program such that a second display [of the yet unused display type] is shown.”

ST Task 1 Solutions PAT Invoke pattern method subscribe with a new instance of the display ALT Introduce a new display instance variable and invoke the displaying of new data on each data update Must comprehend structure of program, in particular how displays receive data

ST Task 1 Expectations E1: Before taking patterns course, PAT subjects should require more time than ALT subjects E2: After taking patterns course, PAT group may understand program structure more quickly than ALT group

ST Task 1 Results E1 is supported: Before patterns course, PAT subjects require more than twice as much time than ALT subjects E2 is refuted: After patterns course, PAT subjects still require more time than ALT subjects

ST Task 1 Conclusion For Stock Ticker program and this type of maintenance task, use of OBSERVER pattern may be harmful

ST Work Task 2 “Change the program so that new displays can be added dynamically at runtime.”

ST Task 2 Solutions PAT Realize that nothing needed to be done ALT Add functionality of an OBSERVER (at least two lines had to be changed, one line had to be deleted, and one method had to be added)

ST Task 2 Expectations E3: ALT version at disadvantage

ST Task 2 Results E3 is confirmed: Task is “completed” on average 29% faster on PAT version; insignificant whether or not patterns course was taken

Boolean Formulas (BO) Boolean Formulas contain a library for representing Boolean formulas (AND, OR, XOR, NOR, and variables) and for printing formulas in two different styles. Contains a small main program that generates formula and invokes both printing routines

Boolean Form. PAT vs. ALT BO PAT 11 classes 470 lines COMPOSITE pattern represents boolean formulas Printing routines implemented as VISITOR patterns BO ALT 8 classes 374 lines VISITOR pattern is completely missing Printing routines located directly in COMPOSITE classes

BO Work Task 1 “Enhance the program to evaluate the Boolean formulas, i.e., to determine the result for a given formula represented by a COMPOSITE and values of the variables.”

BO Task 1 Solutions PAT Create a new VISITOR class ALT Add new methods to each concrete class of COMPOSITE

BO Task 1 Expectations E1: More time for PAT groups to understand application of VISITOR pattern than for ALT groups to find where to add methods E2: Pattern course helps PAT and ALT since both use COMPOSITE E3: PAT group should benefit more from pattern course

BO Task 1 Results E1 ????: PAT group faster than ALT group before pattern course, but slower than ALT after pattern course E2 confirmed: Both groups benefit from pattern course E3 is wrong: ALT group benefits more from course than PAT group

BO Task 1 Conclusion Inconclusive Results VISITOR may not be detrimental to understanding system despite its complexity

Decorator: Communication Channels (CO) Communication Channels is a wrapper library Provides a FACADE pattern to a system library Application of FACADE is irrelevant to experiment

CO PAT vs. ALT PAT 365 lines 6 classes DECORATOR pattern; classes for logging, data compression, and encryption ALT 318 lines 1 class Structured, not OO design Flags and if- sequences used to turn functionality on or off

CO Work Task 1 “Enhance the functionality of the program such that error-correcting encoding (bit redundancy) can be added to communication channels.”

CO Task 1 Solutions PAT Add a new DECORATOR class ALT Make additions and changes at various points in the existing program

CO Task 1 Expectations E1: Before the pattern course, PAT group will be faster because pattern simplifies required work in this task E2: Pattern course will be more beneficial to PAT group

CO Task 1 Results E1 is confirmed: PAT groups are 38% faster than ALT groups E2 is rejected: Pattern course does not help PAT more than ALT

CO Task 1 Conclusions In this case, pattern use is independent of pattern knowledge PAT solutions were error-free, before and after pattern course ALT solutions contained errors in 7 out of 8 subjects before pattern course, and in 6 out of 7 after pattern course

CO Work Task 2 “Determine under which conditions a reset() call will return the ‘impossible’ result.” Result codes: OK Failure impossible

CO Task 2 Solutions Both groups Find the spots where the states were changed PAT Spots are spread over different decorator classes ALT 1 class

CO Task 2 Expectations E3: ALT program easier to understand because spots more localized than PAT; ALT group should be faster E4: ALT program should have fewer errors than PAT

CO Task 2 Results E3 is supported: ALT groups complete task more quickly E4 is rejected: Both PAT and ALT groups have similar error rates

CO Work Task 3 “Create a channel object that performs compression and encryption.”

CO Task 3 Solutions ALT Create single object with parameters for functionality flags PAT Determine proper nesting of decorators to get required functionality in requested order Note: Like reversal of CO Task 1

CO Task 3 Expectations E5: PAT groups will take longer E6: PAT groups will make more errors

CO Task 3 Results E3 is supported: ALT group 53% faster E4 is supported: PAT had 6 wrong solutions out of 14; no errors observed for ALT

Graphics Library (GR) Library for creating, manipulating, and drawing simple types of graphical objects on different types of graphical output devices.

GR PAT vs. ALT PAT 682 lines 13 classes ABSTRACT FACTORY pattern for generator classes and COMPOSITE for hierarchical object grouping ALT 663 lines 11 classes Uses quasi- COMPOSITE No hierarchical group nesting

GR Work Task 1 “Add a third type of output device (plotter).”

GR Task 1 Solutions PAT Introduce new concrete factory class, extend factory selector method, and add two concrete product classes ALT Enhance switch statements in all methods of generator class, and add appropriate graphical objects classes

GR Task 1 Expectations E1: ALT program easier to understand E2: Both groups benefit from pattern course, PAT group benefits more

GR Task 1 Results E1 supported: ALT group faster E2: Both groups benefit from pattern course, but ALT group benefits more

GR Work Task 2 Determine whether or not a certain sequence of operations would result in an x-shaped figure

GR Task 2 Expectations E3: No significant differences between PAT and ALT, because task deals with COMPOSITE structure found in both E4: Both groups benefit from pattern course since both deal with COMPOSITE pattern

GR Task 2 Results E3 confirmed: Both groups similar in speed E4 rejected: Performance does not depend on pattern knowledge

Lessons Learned It is usually, but not always, useful to use a design pattern (even) if there are simpler alternatives Use software engineering common sense to find the exceptions where a simpler solution should be preferred, even if a design pattern could be easily applied

Lessons Learned If in doubt, use design patterns Design patterns provide flexibility for unexpected new requirements Knowledge of design patterns beneficial, even for simple programs

Authors’ closing comments Software engineers should study a variety of alternative approaches to software design including but not restricted to design patterns Different approaches have different trade-offs