The Effectiveness of Test-first Programming By Shawn Morgan Greet yourself, and say that you’ll be presenting on Test-first programming.

Test-driven Development Test-first Programming Test-last Programming TDD Hierarchy Test-driven Development (TDD) Test-first Programming (TFP) We all know about test-driven development (TDD), read an article on it and lecture. As the hierarchy shows, TFP goes head-to-head with TLP. We’ll be comparing the two over the course of the presentation. Test-last Programming (TLP)

What’s TFP? Purpose Process Test-before-implementation Create test based on some functionality Implement code to pass test Run test and verify the code TFP is when you write up a functional test based on some set requirements, implement the code, and then run the test to verify the code works as expected.

Background Previous studies The authors of the article this presentation is based on show previous studies that took place before the article was written, with the article’s study being in the last row.

Background Previous studies Three professors conducted a case study using test-driven development, and found that writing tests improved code quality without any cost to productivity. Notice how the focus was not test-first.

Background Previous studies Later on, a more controlled study took place where the same approach was taken, but the authors found that productivity had declined as a result. Notice again how they went with the test-last approach.

Background Previous studies The article’s study, on the other hand, has the programmers writing the tests first. This approach produced the best results, and you can see the difference here.

Article’s Experiment Two groups: Compare test-first vs test-last Control Group – test-last programming Experiment Group – test-first programming Compare test-first vs test-last Hypothesis Gather data Theory So in the article’s experiment two groups participated to represent each approach. The control group would create their code first, while the experiment group would create their tests first. We’ll see the hypothesis made, the data that was gathered, and the resulting theory.

2-Stage Null Hypothesis The hypothesis created was divided into two stages.

Stage 1 In the first stage, they think that test-driven development alone will improve quality and productivity.

Stage 2 In stage 2, they think that, for each test that is produced…

Stage 2 …quality and productivity will be increased even more, since testing enhances task focus, ergo enhancing both of the factors. Since test-first programming forces test creation rather than test-last, we should expect better results than in TLP.

2-Stage Hypothesis

Story Comparison TLP TFP For each typical--what they call– story.. (Left-side, right-side, only talk about the process and not the siginificance).

Story Comparison TLP TFP

Story Comparison TLP TFP

Worst-case Time Comparison TLP TFP (Left-side, right-side, only walk through how it was done, and—again—don’t mention the significance of anything).

Worst-case Time Comparison TLP TFP

Worst-case Time Comparison TLP TFP

Worst-case Time Comparison TLP TFP

Graphical Representation of Results - Quality This first graph shows the results on how quality varied between TFP and TLP. Variability in quality increased as more tests were made. Notice how TFP didn’t consistently outperform in this area.

Graphical Representation of Results - Productivity (x100%) For productivity, the authors realized how much better productivity was for TFP.

Alternate Hypothesis (Theory) Modify the hypothesis: Quality improvement not proportional as expected Bummer! Productivity improvement much more noticeable …but why? So now they have to modify their hypothesis. They realize that quality improvement was not as expected. On the other hand, improvement in productivity was much more noticeable. (Put the formulas below for emphasis)

Why was Productivity Better? Productivity was better because of: Better task understanding Better task focus Faster learning Lower rework effort They concluded from their data that the task was more understood since the requirements were made clear up front through creating tests, more effort was spent by each task at a time rather than implementing all of them at once, an increase in productivity caused the programmers to understand the tasks more effectively, and reworking tests one by one was much easier than having to fix multiple tests at once.

Conclusion Article’s experiment shows TFP can… Encourage programmers to be more productive Lessen rework of implemented code Increase quality of implemented code Outperforms test-last programming To sum it all up, TFP… [Note: Do NOT add citations. This goes to the teacher if he wants them]