Experimental Evaluation of Pair Programming Copyright, 2001 © Jerzy R. Nawrocki European Software Control & Metrics ESCOM’01 ESCOM’01 Poznan University of Technology Poznan, Poland Jerzy Nawrocki, Adam Wojciechowski Poznan University of Technology Poznan, Poland

J. Nawrocki, Experimental Evaluation.. Plan of the lecture Introduction Personal Software Process eXtremme Programming Description of the experiment ResultsConclusions

J. Nawrocki, Experimental Evaluation.. IntroductionIntroduction Pair programming Creator Quality assurer if (x=y) z=0; Must be x==y

J. Nawrocki, Experimental Evaluation.. IntroductionIntroduction After some time.. Creator Quality assurer x-=y; else How to test it?

J. Nawrocki, Experimental Evaluation.. IntroductionIntroduction Nosek’s experiment Write a script that performs a database consistency check.

J. Nawrocki, Experimental Evaluation.. IntroductionIntroduction Nosek’s experiment 30 minutes 42 minutes Completion time (aver.)

J. Nawrocki, Experimental Evaluation.. IntroductionIntroduction Nosek’s experiment 71 % 100 % Completion time Effort 143 % 100 %

J. Nawrocki, Experimental Evaluation.. IntroductionIntroduction Main weakness ‘If several tasks each take an hour, combine them to form a larger task.’ Kent Beck Extreme Programming Explained ‘If several tasks each take an hour, combine them to form a larger task.’ Kent Beck Extreme Programming Explained Only one short assignment (45’).

J. Nawrocki, Experimental Evaluation.. IntroductionIntroduction The Utah experiment programming assignments 6 weeks 4 programming assignments 6 weeks 50% - 60% 100% Completion time

J. Nawrocki, Experimental Evaluation.. IntroductionIntroduction Weak points What were the assignments about? What was the process? What were the sizes? What was the time in hours? What was the deviation in time and size? What were the assignments about? What was the process? What were the sizes? What was the time in hours? What was the deviation in time and size?

J. Nawrocki, Experimental Evaluation.. IntroductionIntroduction About our experiment Aim: experimental evaluation of pair programming. Aim: experimental evaluation of pair programming. XP-like process PSP-like process Fall semester 1999/2000; Poznan University of Technology, Poznan, Poland Fall semester 1999/2000; Poznan University of Technology, Poznan, Poland

J. Nawrocki, Experimental Evaluation.. Plan of the lecture Introduction Personal Software Process eXtremme Programming Description of the experiment ResultsConclusions

J. Nawrocki, Experimental Evaluation.. Personal Software Process Time and defects are recorded. Defect type standard. 0 Coding standard. Size measurement. Process Improvement Proposal. 0.1 Software size estimation. Test reports. 1 Task planning. Schedule planning Design templates. Code & design reviews.2.1 Incremental approach3

J. Nawrocki, Experimental Evaluation.. Personal Software Process Planning Designing Coding Compiling Testing Postmortem Requirements Product + data

J. Nawrocki, Experimental Evaluation.. Plan of the lecture Introduction Personal Software Process eXtremme Programming Description of the experiment ResultsConclusions

J. Nawrocki, Experimental Evaluation.. eXtreme Programming (XP) Pair programming. Test-centred quality assurance. Simple solution. Spike solutions. Keep moving. Pair programming. Test-centred quality assurance. Simple solution. Spike solutions. Keep moving. We applied: User stories. An on-site customer representative. Planning game. CRC cards. Continuous integration User stories. An on-site customer representative. Planning game. CRC cards. Continuous integration We didn’t apply:

J. Nawrocki, Experimental Evaluation.. Plan of the lecture Introduction Personal Software Process eXtremme Programming Description of the experiment ResultsConclusions

J. Nawrocki, Experimental Evaluation.. Description of the experiment XP-like pair programming XP2 PSP 0.1 (time, defect & size measurement) PSP Test-centred QA, Spike solutions XP1

J. Nawrocki, Experimental Evaluation.. Description of the experiment Programming assignments 1. Estimate the mean and standard deviation of a sample of n real numbers. 2. Calculate the linear regression parameters. 3. Count the logical lines in a program, omitting comments and blank lines. 4. Count the total program LOC, the total LOC in each object the program contains, and the number of methods in each object. C/C++

J. Nawrocki, Experimental Evaluation.. Plan of the lecture Introduction Personal Software Process eXtremme Programming Description of the experiment ResultsConclusions

J. Nawrocki, Experimental Evaluation.. ResultsResults There is almost no difference between XP2 and XP1. Prog 1Prog 2Prog 3 Prog 4

J. Nawrocki, Experimental Evaluation.. ResultsResults Prog 1Prog 2Prog 3 Prog 4 Pair programming is more predictable than individual one.

J. Nawrocki, Experimental Evaluation.. ResultsResults Prog 1Prog 2Prog 3 Prog 4

J. Nawrocki, Experimental Evaluation.. ResultsResults Prog 1Prog 2Prog 3 Prog 4 Pair programming leads to more stable solutions.

J. Nawrocki, Experimental Evaluation.. ConclusionsConclusions XP-like pair programming appears less efficient than it is reported by J.T. Nosek and L. Williams et al.XP-like pair programming appears less efficient than it is reported by J.T. Nosek and L. Williams et al. Pair programming is more predictable one than individual one both in completion time and program size.Pair programming is more predictable one than individual one both in completion time and program size. The experiment was restricted to relatively small programs ( LOC).The experiment was restricted to relatively small programs ( LOC).