Effects of developer experience on learning and applying Unit Test-Driven Development Roberto Latorre

Guideline Introduction Formalization of UTDD Process Study Definition Analyzing

Introduction Test-Driven Development(TDD) A(cceptance) TDD U(nit) TDD Business level U(nit) TDD Unit tests

UTDD UTDD is not a testing technique, but a programming/design practice UTDD sometimes appears as one of the most efficient software development techniques to write clean and flexible code on time Conflicting results exist

UTDD One of the main problems with UTDD is the difficulty in isolating its effects from other context variables. e.g.: characteristics of experts’ and novices’ UTDD development process

UTDD Process UTDD is based on a minute-by-minute iteration between writing unit tests and writing functional code.

Detailed Process 1) Write a unit test. 2) Run all tests and see the new one fails. 3) Implement just enough code to pass the new test. 4) Run the new test and all other unit test cases written for the code. Repeat the process from step3 until all tests are passed. 5) Refactor the code. Refactoring includes modifications in the code to reduce its complexity and/or to improve its understandability and maintainability. 6) If refactoring, re-run all tests and correct the possible errors. 7) Go to the next functionality and repeat the process from step 1. New functionalities are not considered as proper implementations unless the new unit test cases and all the prior tests are passed.

UTDD Process programming tasks in development process (UTDD mantra): Red tasks, related to write a test that fails (steps 1 and 2). Green tasks, to make the test work quickly (steps 3 and 4). Refactor tasks, that eliminate all duplications from both test code and application code (steps 5 and 6). Beck

Conformance Evaluation The larger the value of the metric, the higher the conformance to the UTDD rules. • UTDD changes, the changes satisfying that unit tests were written before related pieces of the applica tion code. We include in UTDD changesthose cases where the subject does not validate that the test fails after a test code change (weak UTDD change). To identify UTDD changes we apply the same rules proposed by M¨ uller and H¨ ofer [20]. • Refactorings, the changes in the structure of the code that do not alter its observable behavior.

Research Question How does the developer’s experience affect the effort required to learn and properly use UTDD?

Study Subjects Participants: industrial developers without having a previous experience with TDD, but with skills with Java and Junit(To avoid the influence from designing of test problems and technical difficulties) UTDD approach difficulties, designing of test problems and technical difficulties.

Study Subjects 30 programmers: Juniors Intermediates Seniors UTDD approach difficulties, designing of test problems and technical difficulties.

Study Material Back-end requirement in Java application Junit Easy requirements Moderate requirements Difficult requirements

Previous Training Brief course about UTDD Initial resistance: due to inexperience when no support for UTDD is available, inexperienced UTDD developers tend to return to more traditional techniques interesting

Study Design requirements were organized in eight groups of three requirements (G1-G8). Each group comprised an easy requirement (R1), a moderate one (R2) and a difficult one (R3)

Study Design

Study Variables and Data Collection Task correctness and completeness The application code generated during experiments Process conformance Programming performance

Study Variables and Data Collection Task correctness and completeness Process conformance Section 2 Programming performance

Study Variables and Data Collection Task correctness and completeness Process conformance Programming performance Effective development time Active(typing\producing code) Passive(reading code\looking for bug\self-training)

Questionnaires 1) How difficult has it been for you to understand the requirement? (1-5) 2) How many changes/corrections in the structure of the existing code have you needed to successfully implement the requirement?

Questionnaires Q1. How difficult has it been for you to learn UTDD? (1-5) Being 1 = Extremely easy; 3 = Neither easy nor difficult; and 5 = Extremely difficult Q2. How useful do you feel UTDD is? (1-5) Being 1 = Not at all useful; 2 = Slightly useful;3 = Moderately useful; 4 = Very useful; and 5 =Extremely useful.

Tracking tools Eclipse plugins SVN BaseCamp Track Integrate code Keep track of time

Analyzing the Study Results Task correctness and completeness Process conformance Programming performance

Task correctness and completeness Using Concordion acceptance tests If fails, the subject had to solve the problem in a new UTDD cycle

Learning process Q1:How difficult has it been for you to learn UTDD? Q2:How useful do you feel UTDD is?

More quantitative analysis Average self-training time

Assurance of faith to UTDD Classify changes UTDD changes Refactoring Other changes

Assurance of faith to UTDD Observations Ratios of changes tends to be constant as study goes UTDD changes grow, while other changes drop Number of UTDD changes for intermediate and senior is larger than junior in average Number of refactoring is nearly constant; juniors do more refactoring

Conformance

Learning curves

Study the real effect K-means cluster C-cluster None Fast Slow Outlier High UTDD conformance from the beginning; able to use UTDD properly Fast Good enough to handle UTDD in 1 or 2 iterations Slow Started with a low level and required 4 to 5 iterations Outlier Worst performance and the only one to say it is difficult in questionnaire

Learning curves using k-means

Result The learning curves with UTDD depended on a trade-off between experience and discipline not all the changes are made following the UTDD rules

Programming efficiency Estimation equation(baseline) Where Gi identifies sets of requirements G1 and G2; DGi, JGi, JUGi, TGi are time for designing, programming/refactoring Java code, programming Junit code, testing the requirements Bls is the baseline development ability of subject s

Programming efficiency Normalize the development times of the sets of requirements implemented with UTDD (G3-G8) using the corresponding value of bls E.g. , 1.25 means 25% worse than baseline

Programming efficiency(no comprehension time)

Result of questionnaire on comprehension

Time comparison Efficiency grows as experiment acvances Juniors are always worst in efficiency

Conclusion of efficiency unlike process conformance, efficiency mainly correlates with expertise P-cluster Yes subjects thatwere able to continuously apply UTDD in an efficient manner when the experiment concluded

Correlation between UTDD and efficiency As the development advanced and the knowledge in UTDD became better, some subjects reached a trade-off between the use of UTDD and efficiency

Retention Ability

Retention Ability Though they showed a little decline, the three subjects were able to continue using UTDD effectively and efficiently