1. 15/05/2015Dr Andy Brooks1 MSc Software Maintenance MS Viðhald Hugbúnaðar Fyrirlestrar 31 & 32 Maintenance-Oriented IDEs

2. 15/05/2015Dr Andy Brooks2 Case Study Dæmisaga Reference Eliciting Design Requirements for Maintenance-Oriented IDEs: A Detailed Study of Corrective and Perfective Maintenance Tasks, Andrew J. Ko, Htet Htet Aung, and Brad A. Myers, 27th International Conference on Software Engineering (ICSE´05), pp 126-135, 2005 © ACM

3. 1. Introduction Software development tools typically focus on the creation and modifying of code. Yet the greatest cost is incurred in software maintenance which involves reading and navigating code. –understanding code “… an important challenge in software engineering research is to create new and more useful tools for understanding and reshaping software as its requirements change.” The authors decided to study expert Java programmers using the Eclipse IDE to work on five maintenance tasks. 15/05/2015Dr Andy Brooks3

4. 1. Introduction The study reveals the interleaving of three fundamental activities. 1)“collecting a group of task-relevant code fragments, which we call a working set” 2)“navigating these code fragments’ dependencies (such as uses and declares relationships)” 3)“repairing or creating the necessary code” 15/05/2015Dr Andy Brooks4

5. 1. Introduction On average the expert Java programmers spent 35% of their time “simply navigating between dependencies”. On average the expert Java programmers spent 46% of their time inspecting task- irrelevant code. 15/05/2015Dr Andy Brooks5

6. Overview of study method expert programmers in a private lab, working alone think-alouds and full-screen video capture –think-aloud approach inferred from reference [12] of the case study article 70 minutes to complete five maintenance tasks –no fixed order in which to attempt tasks –same code, same tasks to eliminate interpretations based on using different code/tasks work interruptions to reduce artificiality –references [10] and [15] of the case study article suggest interruptions are frequent in the software engineering workplace monetary incentives to complete the tasks –$10 for each completed task 15/05/2015Dr Andy Brooks6 3. The Eclipse Study

7. 3.1 The Participants 10 male programmers –average age 23 (±3) –6 senior and 2 doctoral computer science students –1 MS student in computer engineering –1 MS student in information systems In a pre-test survey, 7 described themselves as Java experts while the remaining 3 described themselves as having above-average Java expertise. All 10 reported using Eclipse or Visual Studio regularly. Hours spent programming each week? –the group reported an average of 24 hours (±20) 15/05/2015Dr Andy Brooks7 3. The Eclipse Study ± standard deviation

8. 3.2 The Paint Program 503 lines Java Swing application –9 Java classes in 9 files Users can draw, erase, clear and undo colored strokes. –a PaintObjectConstructor creates strokes from mouse locations accumulated between mouse up and mouse down events 15/05/2015Dr Andy Brooks8 3. The Eclipse Study

9. 3.3 Tasks Table1 © ACM 15/05/2015Dr Andy Brooks9 3. The Eclipse Study

10. 3.3 Interruptions designed to mimick real interruptions on average around every 3 minutes flashing taskbar with an audible alert –to be clicked on to answer an arithmetic question 2-digit multiplication question with answer box –questions did not contain 1 and 0 digits $2 deduction for each interruption ignored or answered incorrectly –not enforced 15/05/2015Dr Andy Brooks10 3. The Eclipse Study

11. 3.3 Tools PC and 17” 1024x768 LCD –video recording at 12 frames per second in 24-bit colour had no noticeable impact on the PC´s performance Eclipse 2.1.2 IDE (released March 2004) Internet browser –default page Java 1.4 API documentation Participants´ were told to use any resource they desired. 15/05/2015Dr Andy Brooks11 3. The Eclipse Study

12. 4. A qualitative assessment The initial analysis of screen-captured video took about 40 hours. –two of the authors working together The following three activities were revealed: –collecting a group of task-relevant code fragments the working set –navigating dependencies within the working set “What defines this variable´s value?” “What uses this variable´s definition?” –creating/editing code to perform the maintenance task 15/05/2015Dr Andy Brooks12

13. 4. A qualitatitive assessment When starting a task (before building a working set), a programmer typically asked one of the following two questions: –How does X work? –Why did(n´t) X happen? Eclipse sometimes provided hidden or inconsistent feedback. Programmers made mistakes when copy and pasting code. 15/05/2015Dr Andy Brooks13

14. Figure 2. ©ACM The 502 lines of the Paint program. The dark regions portray the working set for one programmer working on the THICKNESS task. The light gray regions portray code that was read. The white regions portray code that was scrolled through but not read. 15/05/2015Dr Andy Brooks14 9 Java classes 9 files

15. Table 3. Time spent on task ©ACM Timing proportions exclude time spent on interruptions. Time spent on handling interruptions was on average 22% (±6) of a programmer´s time. Average times are independent of whether the task was completed. Each programmer completed an average of 3.5 (±0.8) tasks. LINE and SCROLL are the more difficult tasks. 15/05/2015Dr Andy Brooks15 5. An Empirical Assessment ± standard deviation

16. Figure 3. Division of labor ©ACM (excluding interruptions) Paint´s static and dynamic dependencies were determined prior to transcribing the videos. Two of the authors working together took about 3 hours to transcribe the video recording for each programmer. 15/05/2015Dr Andy Brooks16 5. An Empirical Assessment ∑=98%?

17. Inefficiency of searching and hypothesising There were 40 instances where programmers began a task by searching for a task-relevant name e.g. “line” for the LINE task. –In 34 instances, programmers performed searching by scrolling through the code. –In 6 instances, Eclipse´s find and replace was used. –Only half of the searches led to task-relevant code. 88% (±11) of programmers´ hypotheses about the cause of runtime failures were false. 15/05/2015Dr Andy Brooks17 5.2 Forming a Working Set

18. Recovering working sets Five programmers temporarily abandoned the LINE and SCROLL tasks. In Eclipse, working sets are represented by open file tabs and the state of the package explorer. Closing file tabs and package explorer nodes for the temporarily abandoned task means the working set has to be recovered later. Programmers spent an average of 60 seconds (±28) recovering their working sets. 15/05/2015Dr Andy Brooks18 5.2 Forming a Working Set

19. 5.3.2 Navigations of Indirect Dependencies bookmarks unuseful? When navigating within a file, programmers performed lengthy visual searches using the scroll bars or the page up and down keys. To help with navigation, three programmers tried to use Eclipse´s bookmark facility to mark task-relevant code. The three programmers, however, always ended up with more than two bookmarks and they were unable to recall what each bookmark represented. –they had to click on bookmarks to remember “which was no faster than their average scrolling time” 15/05/2015Dr Andy Brooks19

20. Indirect relationships between files When many file tabs were open, programmers could not read the file names in full and had to search through the file tabs. If the package explorer had many nodes expanded, programmers had to scroll. These overheads cost each programmer an average of 5 (±1) minutes. 15/05/2015Dr Andy Brooks20 5.3 Navigating the Working SetFigures 4. and 5. ©ACM

21. Figure 6 marking valid syntax as incorrect ©ACM The invalid syntax caused by a missing delimiter (semi- coma or curly brace) is above the marked valid syntax. Programmers knew the highlighted code was valid and had to spend time searching for the invalid code. 15/05/2015Dr Andy Brooks21 5.4 Hidden and Inconsistent Feedback

22. When is code compiled in incremental compilation? Eclipse´s incremental compilation was often invoked only after a file was saved. Sometimes programmers tried to repair code they had already repaired because the repair remained marked as invalid. Each instance of inconsistently marked code cost an average of 38 seconds before programmers realised the inconsistency. –24 instances in total across the 10 programmers 15/05/2015Dr Andy Brooks22 5.4 Hidden and Inconsistent Feedback

23. Off-screen duplication of gSlider reference Three programmers overlooked the off-screen error an average of 3 times before noticing it. –cost an average of 6 (±1) minutes to these programmers 15/05/2015Dr Andy Brooks23 5.4 Hidden and Inconsistent Feedbackred, green, blue

24. 5.5 Copy and Paste Errors All the programmers used copy and paste for the LINE and THICKNESS tasks. On average, each programmer copied and pasted 4 (±3) times. On a few occasions, programmmers left one or more visually indistinct or off-screen references unchanged. –On average, programmers spent 3 (±1) minutes testing false hypotheses before finding these errors. On a few occasions, programmers copied only part of a code pattern. –On average, programmers spent 4 (±1) minutes testing false hypotheses before finding these errors. 15/05/2015Dr Andy Brooks24

25. Figure 9. © ACM (1) colorChangeListener should be thicknessChangeListener (2) programmer forgets to add the new tPanel to colorPanel. 15/05/2015Dr Andy Brooks25 5.5 Copy and Paste Errors

26. 6. Design Requirements and Ideas for a new type of IDE “... we limit our discussion to requirements that have yet to be satisfied by a modern IDE” –Eclipse´s “open declaration” and other search tools are essential for any maintenance IDE. On way of helping solving the problems identified is to provide a larger screen. –more room for the names in file tabs –fewer off-screen code fragments –less need to scroll in the package explorer More screen space, however, cannot fully alleviate the identification and navigation of direct and indirect dependencies. 15/05/2015Dr Andy Brooks26

27. Table 5 Design requirements ©ACM Requirement 1 Programmers formed working sets of task-relevant methods and statements. Solution Provide a working set interface that supports the quick addition and removal of task-relevant code fragments. Requirement 2 Because programmers had to store their working sets in the interactive state of file tabs and package explorer, when they changed tasks, they lost their working set. Solution Automatically save and recover of working sets of task- relevant code fragments, ensuring that the tools used to navigate working sets are distinct from the tools used to represent working sets. 15/05/2015Dr Andy Brooks27

28. Table 5 Design requirements ©ACM Requirement 3 When programmers found task-relevant code, they tended to glance at its dependencies. Also, more than 60% of navigations of indirect relationships were for the purpose of comparison. All of these incurred significant visual search costs. Solution When programmers add code to a working set interface, automatically add its direct and indirect dependencies. Then, directly or indirectly related code could be placed side-by-side, avoiding the interactive overhead of opening and closing file tabs. 15/05/2015Dr Andy Brooks28

29. Table 5 Design requirements ©ACM Requirement 4 When copying code, programmers often left indistinct or off-screen references unchanged. Because they believed the copied code was correct, it was the last place they checked for errors. Solution Copied code should maintain a dependency with its “original” so that unchanged references can be marked as “suspect” until verified. These markers should be apparent even when off-screen. Requirement 5 When copying a pattern of code that was distributed within or between files, programmers often duplicated only part of the pattern, leading to dead-end data. Solution When programmers copy code, the IDE should check if the programmer is neglecting any dependencies in the copied code and offer to help collect them. 15/05/2015Dr Andy Brooks29

30. Table 5 Design requirements ©ACM Requirement 6 Programmers searched for task-relevant names, but only half of such searches led to task-relevant code. Programmers also used surface features of Paint’s output to deduce the cause of failures, but only a 1/4 of such features correlated with the cause. Solution Let programmers ask about program output of interest and have the IDE gather all of the code that was directly responsible for the output in question. This way, the correct working set can be built automatically by the IDE. 15/05/2015Dr Andy Brooks30

31. Figure 10. Concept sketch for a maintenance- oriented development environment. ©ACM 15/05/2015Dr Andy Brooks31 THICKNESS task example working set

32. Figure 10. Concept sketch for a maintenance- oriented development environment. 15/05/2015Dr Andy Brooks32 (1) Management of working sets for active maintenance tasks. (2) (3) (4) Selection-based dependency highlighting of (5) bslider. (4) Is an incorrect reference and so (4) highlights an error. (6) Dependency highlighting. (7) A copy relationship supports comparison between the original and the copied code. (8) Attention is drawn to unchanged references in copied code. (9) Hide unrelated code (...) to bring the indirectly related code closer. (10) Allow users to aks ‘how’ and why’ questions => the WhyLine tool

33. 6.5 Context In-Place A meta key could be provided. When the meta key is pressed, code surrounding the code fragments is shown. –i.e. there would be no need to navigate to see the surrounding code 15/05/2015Dr Andy Brooks33

34. External validity The 10 programmers are likely not to be representative of programmmers in industry. The Paint program is small and is not representative of the large, heavily maintained programs in industry. 15/05/2015Dr Andy Brooks34 7. Conclusions

35. Critical commentary by Andy Resources permitting, programmers in industry use screens larger than 17” and use two screens to manage their work. –this further weakens external validity Navigating dependencies is stated to represent 35% of programmers´ time in the conclusion, but Figure 3. states that the percentage is 16%. –This throws some doubt on the accuracy of the quantitative aspects reported in the case study article. 15/05/2015Dr Andy Brooks35

36. Critical commentary by Andy Even though the programmers had made prior use of Eclipse, programmers were observed using visual searches rather than making use of Eclipse´s find and replace. Would the results of the study be different had the participants undergone extensive training in the use of Eclipse? –human-computer interaction problems can be solved by training The case study article has a literature reference to the underlying methodology which makes use of the think- aloud technique. Strangely, there is no mention of think- alouds in the case study article. –the case study article should have been clear on this issue 15/05/2015Dr Andy Brooks36