A Controlled Experiment in Maintenance Comparing Design Patterns to Simpler Solutions Fadi (Fadhi) Wedyan CS614 Spring 2007

Presentation Outlines Introduction Objective Experiment setup Experiment Conduction Validity Threads Conclusions and Comments

Introduction Design patterns aim to simplify maintenance and increase usability. Design patterns increase flexibility and reduce coupling. Design patterns increases program complexity in the case where the flexibility offered by the pattern is unnecessary.

Objective Investigate, with respect to maintenance time, whether it is useful to design programs with design patterns even if the actual problem is simpler than that solved by the design pattern. – Is the complexity introduced by the design pattern always justified?

Experiment Setup Four different programs, well-documented, written for the purpose of the experiment. Two versions of each program are developed: Pattern version (PAT) and Alternative version (ALT). Implementation in C++. Patterns implemented : ABSTRACT, FACTORY, COMPSITE, DECORATOR, FAÇADE, OBSERVER, and VISITOR.

Experiment Setup, Cont’d In each program, at least one of flexibility functionality offered by the design patterns is not needed by the maintenance tasks. 29 subjects are used, all professional SE. - Some of them have theoretical or practical pattern knowledge. - Subjects are distributed into four balanced groups.

Experiment Setup, Cont’d Three Independent variables: – Programs and maintenance tasks. – Program version. – Amount of pattern knowledge. Two dependent variables: – Time. – Correctness.

Experiment Setup, Cont’d SequenceGroup AGroup BGroup CGroup D PRE1 st Prop.ST PATGR PATCO ALTBO ALT PRE2 nd Prob.GR ALTST ALTBO PATCO PAT Pattern Course POST3 rd Prob.CO ALTBO ALTST PATGR PAT POST4 th Prob.BO ALTCO PATGR ALTST ALT Table 1: Order of Programs Per Group

Experiment Conduction First Problem: Stock Ticker (ST) – Directs stocks to one or more displays. – Patterns: OBSERVER Task 1: activate the second display unit. Expectations: – E1: PRE-PAT Subjects require more time (supported) – E2: POST-PAT Subjects require less time. (Not Supported). Conclusion: OBSERVER may be harmful!

First Problem: Stock Ticker (ST), Cont’d Task 2: Allow program to add new displays dynamically. Expectations: – E3: PAT Subjects require less time (supported) Conclusion: Nothing, unfair task? Post and Pre are close for both programs

Second Problem: Communication Channels (CO) Pattern: DECORATOR for adding functionality. Task 1: Enhancing functionality. Expectations: – E1: PAT subjects are faster. (Supported) – E2: Pattern Knowledge enhance the ALT solution (Not supported) Conclusions: – Patterns are preferable (time + correctness). – Positive effect of pattern use is independent of pattern knowledge.

Second Problem: Communication Channels (CO), Cont’d Task 2: Checking channel status Expectations: – E1: ALT faster (Supported) – E2: ALT less errors (Not supported) Conclusions: – Subjects were tired in the afternoon!

Second Problem: Communication Channels (CO), Cont’d Task 3: Create a channel with different functionality. Expectations: – E1: ALT faster (Supported) – E2: ALT less errors (Not supported) Conclusions: Harmful use of patterns? Or bad experiment?

Third Problem: Graphics Library (GR) Patterns: ABSRACT, FACTORY, COMPOSITE. Task 1: Adding an output device type. Expectations: – E1: ALT little faster (Supported) – E2: POST faster (supported) Conclusions: ?

Third Problem: Graphics Library (GR) Task 2: Operation Sequence. Expectations: – E3: No significant different (Supported) – E4: POST faster (not supported) Conclusions: ?

Forth Problem: Boolean Formulas (BO) Patterns: VISITOR, COMPOSITE. Task 1: Evaluating the formula Expectation: – E1: PAT Faster (supported) – E2: POST Knowledge helps (supported)

Validity Threads Internal Threads – Group similarity – Time stamps. External Threads – Original designers and implementers may be the ones who perform maintenance. – Real program are less documented. – Real maintainers implement and test their solutions. – Effect of other design patterns and/or variations.

Conclusions The use of design patterns is usually justified. Common sense works! Some times justified use of design pattern was harmful Some times unjustified use of design patterns was useful