How Programmers Read Regular Code: A Controlled Experiment Using Eye Tracking ICPC 2015 Ahmad Jbara and Dror G. Feitelson The Hebrew University of Jerusalem, Israel.

Regular Code Repetitions of the same basic pattern

Regular Code Repetitions of the same basic pattern

Regular Code Repetitions of the same basic pattern

Previous Work Regular function can be as easy to comprehend as the non- regular function of the same problem. This despite being  Longer (LOC)  More complex (cyclomatic complexity) Speculation  Leveraging the understanding of the first instances to ease the understanding of repeated ones

This Work We use eye tracking to measure time and effort spent reading and understanding regular code. Questions:  Do developers follow any pattern when they are required to comprehend regular code?  Are their efforts equally divided between regular segments?  Assuming there is such a pattern. Which model might fit this pattern.

The Experiment Two different problems – image processing domain Two implementations for each problem: regular/non-regular. Comprehension task was “understanding functionality“ Each subject performed one comprehension task on one implementation. Each regular function was divided into areas of interest (AOI)  One AOI for each repeated instance Dependent variables (AOIs)  Fixation count  Total fixation time  Average pupil dilation

Regular Versions Diamond program Median program

Results – Correctness and Time These results follow those of our previous work.

Difficulty of Programming Style Asked the subjects to rank difficulty of their function.

Results and Analysis – Heat Map Diamond program Median program We used heatmaps to identify regions with special attention We see that subjects largely fixate on the first instances (red spots) Attention is not reduced with length (focus was renewed. Left map)

Results and Analysis– AOI Same as pervious with all measures and clear separation between AOIs Measures show high values in the earlier instances and low in the other ones.

Results and Analysis– AOI Same but for the diamond program

Verifying the Results – a post-experiment question We asked the subjects about their approach and effort allocation to the different parts in the code. We showed them with their heatmaps and asked if they match their impression. More than 70% stated clearly that they spent more time in the first instances. One subject said: I spent much efforts at the beginning, tried to understand the loops at the beginning because I saw that they repeat themselves. In particular I realized that the differences are vey small so it is easy to infer about other.

Verifying the Results – a post-experiment question

Modelling Effort in Repeated Instances Not all code segments should have equal weight (have same structure) But, many metrics present high measurements and are not sensitive to code context.  MCC : conditions in the 10 th instance are counted just like those in the first. Misleading! Purpose  Build a model that predicts efforts needed to understand a repeated instance based on its ordinal number.

Modeling Effort in Repeated Instances Shows the models found by curve fitting of fixation data as a function of AOI. All models are significant (except one).

Modeling Effort in Repeated Instances

Characteristics of Functions One may claim that the exponential is the best ( considering functions characteristics ) Others are bad because they grow to positive/negative infinity for larger xs. Theoretically that is right but the number of repeated instances does not grow to very large values. So for some thresholds other model could be a good fit.

Conclusions

Thank you!