1. Effects of developer experience on learning and applying Unit Test-Driven Development
Roberto Latorre

2. Guideline Introduction Formalization of UTDD Process Study Definition
Analyzing

3. Introduction Test-Driven Development(TDD) A(cceptance) TDD U(nit) TDD
Business level
U(nit) TDD
Unit tests

4. UTDD
UTDD is not a testing technique, but a programming/design practice
UTDD sometimes appears as one of the most efficient software development techniques to write clean and flexible code on time
Conflicting results exist

5. UTDD
One of the main problems with UTDD is the difficulty in isolating its effects from other context variables.
e.g.: characteristics of experts’ and novices’ UTDD development process

6. UTDD Process
UTDD is based on a minute-by-minute iteration between writing unit tests and writing functional code.

7. Detailed Process 1) Write a unit test.
2) Run all tests and see the new one fails.
3) Implement just enough code to pass the new test.
4) Run the new test and all other unit test cases written for the code. Repeat the process from step3 until all tests are passed.
5) Refactor the code. Refactoring includes modifications in the code to reduce its complexity and/or to improve its understandability and maintainability.
6) If refactoring, re-run all tests and correct the possible errors.
7) Go to the next functionality and repeat the process from step 1. New functionalities are not considered as proper implementations unless the new unit test cases and all the prior tests are passed.

8. UTDD Process programming tasks in development process (UTDD mantra):
Red tasks, related to write a test that fails (steps 1 and 2).
Green tasks, to make the test work quickly (steps 3 and 4).
Refactor tasks, that eliminate all duplications from both test code and application code (steps 5 and 6).
Beck

9. Conformance Evaluation
The larger the value of the metric, the higher the conformance to the UTDD rules.
• UTDD changes, the changes satisfying that unit tests
were written before related pieces of the applica
tion code. We include in UTDD changesthose cases
where the subject does not validate that the test fails
after a test code change (weak UTDD change). To
identify UTDD changes we apply the same rules
proposed by M¨ uller and H¨ ofer [20].
• Refactorings, the changes in the structure of the code
that do not alter its observable behavior.

10. Research Question
How does the developer’s experience affect the effort required to learn and properly use UTDD?

11. Study Subjects
Participants: industrial developers without having a previous experience with TDD, but with skills with Java and Junit(To avoid the influence from designing of test problems and technical difficulties)
UTDD approach difficulties, designing of test problems
and technical difficulties.

12. Study Subjects 30 programmers: Juniors Intermediates Seniors
UTDD approach difficulties, designing of test problems
and technical difficulties.

13. Study Material Back-end requirement in Java application Junit
Easy requirements
Moderate requirements
Difficult requirements

14. Previous Training Brief course about UTDD
Initial resistance: due to inexperience
when no support for UTDD is available, inexperienced UTDD developers tend to return to more traditional techniques
interesting

15. Study Design
requirements were organized in eight groups of three requirements (G1-G8).
Each group comprised an easy requirement (R1), a moderate one (R2) and a difficult one (R3)

16. Study Design

17. Study Variables and Data Collection
Task correctness and completeness
The application code generated during experiments
Process conformance
Programming performance

18. Study Variables and Data Collection
Task correctness and completeness
Process conformance
Section 2
Programming performance

19. Study Variables and Data Collection
Task correctness and completeness
Process conformance
Programming performance
Effective development time
Active(typing\producing code)
Passive(reading code\looking for bug\self-training)

20. Questionnaires
1) How difficult has it been for you to understand the requirement? (1-5)
2) How many changes/corrections in the structure of the existing code have you needed to successfully implement the requirement?

21. Questionnaires Q1. How difficult has it been for you to learn
UTDD? (1-5) Being 1 = Extremely easy; 3 = Neither easy nor difficult; and 5 = Extremely difficult
Q2. How useful do you feel UTDD is? (1-5)
Being 1 = Not at all useful; 2 = Slightly useful;3 = Moderately useful; 4 = Very useful; and 5 =Extremely useful.

22. Tracking tools Eclipse plugins SVN BaseCamp Track Integrate code
Keep track of time

23. Analyzing the Study Results
Task correctness and completeness
Process conformance
Programming performance

24. Task correctness and completeness
Using Concordion acceptance tests
If fails, the subject had to solve the problem in a new UTDD cycle

25. Learning process
Q1:How difficult has it been for you to learn UTDD? Q2:How useful do you feel UTDD is?

26. More quantitative analysis
Average self-training time

27. Assurance of faith to UTDD
Classify changes
UTDD changes
Refactoring
Other changes

28. Assurance of faith to UTDD
Observations
Ratios of changes tends to be constant as study goes
UTDD changes grow, while other changes drop
Number of UTDD changes for intermediate and senior is larger than junior in average
Number of refactoring is nearly constant; juniors do more refactoring

29. Conformance

30. Learning curves

31. Study the real effect K-means cluster C-cluster None Fast Slow Outlier
High UTDD conformance from the beginning; able to use UTDD properly
Fast
Good enough to handle UTDD in 1 or 2 iterations
Slow
Started with a low level and required 4 to 5 iterations
Outlier
Worst performance and the only one to say it is difficult in questionnaire

32. Learning curves using k-means

33. Result
The learning curves with UTDD depended on a trade-off between experience and discipline
not all the changes are made following the UTDD rules

34. Programming efficiency
Estimation equation(baseline)
Where Gi identifies sets of requirements G1 and G2; DGi, JGi, JUGi, TGi are time for designing, programming/refactoring Java code, programming Junit code, testing the requirements
Bls is the baseline development ability of subject s

35. Programming efficiency
Normalize the development times of the sets of requirements implemented with UTDD (G3-G8) using the corresponding value of bls
E.g. , 1.25 means 25% worse than baseline

36. Programming efficiency(no comprehension time)

37. Result of questionnaire on comprehension

38. Time comparison Efficiency grows as experiment acvances
Juniors are always worst in efficiency

39. Conclusion of efficiency
unlike process conformance, efficiency mainly correlates with expertise
P-cluster
Yes
subjects thatwere able to continuously apply UTDD in an efficient manner when the experiment concluded

40. Correlation between UTDD and efficiency
As the development advanced and the knowledge in UTDD became better, some subjects reached a trade-off between the use of UTDD and efficiency

41. Retention Ability

42. Retention Ability
Though they showed a little decline, the three subjects were able to continue using UTDD effectively and efficiently