To Fix or to Learn? How Production Bias Affects Developers’ Information Foraging during Debugging 1 David Piorkowski 1 Scott D. Fleming 2 Christopher Scaffidi 1 Margaret Burnett 1 Irwin Kwan 1 Austin Z. Henley 2 Jamie Macbeth 3 Charles Hill 1 Amber Horvath 1 1 Oregon State Univ. 2 Univ. of Memphis 3 Clemson Univ.

Software Maintenance and Learning Software maintenance requires learning about code Methodological exploration cuts coding time in half Lots of work has been done to help developers with learning – e.g. Binkley and Lawrie, Kevik et al., Moreno et al., Wang et al. all ICSME ’14 2 Robillard et al. TSE ‘04

Problem: Programmers avoid comprehension “Wherever possible, programmers avoid comprehension” Studies confirming that programmers try to code as quickly as possible – e.g. Brandt et al. CHI ‘09, LaToza et al. ICSE ‘06, Maalej et al. TOSEM ‘14 3 Rohem et al. ICSE ‘12

Production bias may explain this Production Bias – Motivational paradox – “Do” instead of learn – Even if learning would be better Goal: Understand how production bias affects information seeking in debugging 4 Carroll and Rosson ‘87

Approach Conduct an observational study of debugging Using Information Foraging Theory (IFT) as a conceptual framework – Explains and predicts how people seek information 5

Cues & ScentTopology & Patches Prey Information Foraging Theory (IFT) Predator

IFT & Production Bias foraging to fix vs. foraging to learn Differences in… RQ1 … foraging goals sought? RQ2 … patches decisions were made? RQ3 … cues attended to? RQ4 … tactics used? 7

Method Participants: 9 student programmers Either: – Fix a defect in jEdit – Learn enough to onboard a new programmer Procedure: – 30 mins fixing or learning – Talk aloud – Retrospective interview 8

IFT & Production Bias foraging to fix vs. foraging to learn Differences in… RQ1 … foraging goals sought? RQ2 … patches decisions were made? RQ3 … cues attended to? RQ4 … tactics used? 9

RQ1: Similar goals P2: “I’m going to search for that delete lines thing in the code to see what it does.” P10: “…so I can search for that text. Delete lines.” 10 P4: “Delete lines. I’m searching for delete lines” P11: “Okay I should be looking for this text (delete lines). How do I find this text?”

IFT & Production Bias foraging to fix vs. foraging to learn Differences in… RQ1 … foraging goals sought? RQ2 … patches decisions were made? RQ3 … cues attended to? RQ4 … tactics used? 11

RQ2: Patch Types 12 Patch Types: Locations where a foraging decision was made

RQ2: Fixers and Learners made decisions in different patches 13

Fixers wanted details “Since [the bug is] something that I'm trying to fix in a hurry, I would prefer to do as little of that really high level architectural stuff.” – Participant 3 14

Learners wanted the bigger picture 15 P5 (learn) was interested in code structure Foraged within patches that exposed hierarchy

16 “I'm going to go try and find some much higher level point [in the code].” – Participant 5 Learners wanted the bigger picture

17 “I'm going to go try and find some much higher level point [in the code].” – Participant 5 Learners wanted the bigger picture

18 “I'm going to go try and find some much higher level point [in the code].” – Participant 5 Learners wanted the bigger picture

19 “I'm going to go try and find some much higher level point [in the code].” – Participant 5 Learners wanted the bigger picture

20 “I'm going to go try and find some much higher level point [in the code].” – Participant 5 Learners wanted the bigger picture

21 “I'm going to go try and find some much higher level point [in the code].” – Participant 5 Learners wanted the bigger picture

RQ2: Implications Tools – Details vs Big Picture – Low-level details may not help learners – High-level details may not help fixers Theory – Cost / benefit differences between fixers and learners 22

IFT & Production Bias foraging to fix vs. foraging to learn Differences in… RQ1 … foraging goals sought? RQ2 … patches decisions were made? RQ3 … cues attended to? RQ4 … tactics used? 23

RQ3: Cue Types Cue types: Based not on the content, but on the source of inspiration 24 Source Code Content Inspired Domain Text Output Inspired

RQ3: Fixers and Learners attended to different cues 25

Fixers and Learners used the same cue types differently Source-code content inspired: – Fixers: Mainly within Editor patch type All but 2 instances – Learners: High-level and low-level patch types High level: Package Explorer and Outline View Low level: Editor and Stack Trace 26

RQ3: Implications Tools: – Change cues accordingly for fixing or learning – Fixers: draw cues from code – Learners: draw cues from output Theory: – Cue types non-reliance on content 27

Conclusions Fix Focus: Low-level details RQ2: Editor and Stack Trace patch types RQ3: Source-code inspired cue types Learn Focus: High-level details RQ2: Search Results patch types RQ3: Output-inspired cue types 28