1. Program Comprehension Program Understanding Behavior During Debugging Of Large Scale Software Anneliese von Mayrhauser (Andrews) and A. Marie Vans Rizal M Nor (Joey) rnor@cs.kent.edu

2. About the Author ● http://www.cs.colostate.edu/~aaa http://www.cs.colostate.edu/~aaa ● A. Marie Vans, Anneliese von Mayrhauser, Gabriel Somlo: Program understanding behavior during corrective maintenance of large-scale software. Int. J. Hum.-Comput. Stud. (IJMMS) 51(1):31-70 (1999) ● Anneliese von Mayrhauser, A. Marie Vans: Program Understanding During Software Adaptation Tasks. ICSM 1998:316-325 ● Anneliese von Mayrhauser, A. Marie Vans: Program Understanding Behavior During Adaptation of Large Scale Software. IWPC 1998:164- ● Anneliese von Mayrhauser, A. Marie Vans: Program understanding needs during corrective maintenance of large scale software. COMPSAC 1997:630-637 ● Anneliese von Mayrhauser, A. Marie Vans: On Increasing Our Knowledge of Large-Scale Software Comprehension. Empirical Software Engineering (ESE) 2(2):159-163 (1997) ● Anneliese von Mayrhauser, A. Marie Vans: Hypothesis-Driven Understanding Processes During Corrective Maintenance of Large Scale Software. ICSM 1997:12-20 ● Anneliese von Mayrhauser, A. Marie Vans: Identification of Dynamic Comprehension Processes During Large Scale Maintenance. IEEE Trans. Software Eng. (TSE) 22(6):424-437 (1996) ● Anneliese von Mayrhauser, A. Marie Vans: On the Role of Hypotheses during Opportunistic Understanding While Porting Large Scale Code. WPC 1996:68-77 ● Anneliese von Mayrhauser, A. Marie Vans: Program Understanding: Models and Experiments. Advances in Computers (AC) 40:1-38 (1995) ● Anneliese von Mayrhauser, A. Marie Vans: Program Comprehension During Software Maintenance and Evolution. IEEE Computer (COMPUTER) 28(8):44-55 (1995)EEAnneliese von Mayrhauser, A. Marie Vans: Comprehension Processes During Large Scale Maintenance. ICSE 1994:39-48 ● Anneliese von Mayrhauser, A. Marie Vans: From Code Comprehension Model to Tool Capabilities. ICCI 1993:469-473

3. Introduction 1.What kinds of actions do programmers perform when debugging code? 2.Do programmers follow the Integrated Comprehension model of (von Mayrhauser & Vans, 1995a) Do they switch between its three model components? 3.Is it possible to identify a specific comprehension process that is common to the subjects and thus indicative of debugging tasks? 4.Are there certain types of information programmers tend to look for during corrective maintenance?

4. Introduction ● Corrective Maintenance: – Understand System – Generate/Evaluate hypotheses concerning problem – Repair Code (Without breaking anything) – Regression Test ● Maintainers are not always “experts in application area”* and even when they are, they “may not be experts in the implementation language”*. ● The papers tries to observe both *.

5. Recap on Mayrhauser 1995 Maintenance Tasks Cognitive Models

6. Integrated Comprehension Model ● For a large scale system, it is assumed that maintenance engineers unfamiliar with the domain starts by building a program modal. ● However, to assume that a full program model is built before abstracting to the situation model/domain level would create cognitive overload. ● So it is expected that the engineer has to do some abstraction at the domain or situation model.

7. Integrated Comprehension Model / Integrated Metamodel

8. Design of the Study ● Observing Professional Maintenance Engineers – Consisted at least 40,000 lines of code – Ask Engineers to think aloud during corrective maintenance (for video and tape records) – Classified participants to degree of prior experience to the code. – Protocol Analysis ● Think aloud reports of subjects working on tasks are transcribed and classifed using categories decided on prior to the actual analysis. ● Each statement in the transcript is encoded as one of the a priori categories.

9. Classification of Subjects

10. Protocol Analysis ● Actions – enumeration of action types as they relate to the cognition model – Action types classify programmers activities, both implicit and explicit – Example: ● Generating hypotheses about program behavior ● Mental simulation of program statement execution

11. Protocol Analysis ● Segmentation and Information Needs – Segmentation classifies action types into those involving domain(top-down), situation, or program models – Information Needs are information and knowledge items that supports successful completion of maintenance tasks.

12. Protocol Analysis ● Process Analysis – Determine the nature of actions over time graphically. – And Frequency of swithces. – Each action has already been classified by level of abstraction (Program,Situation and Domain level).

13. Results of Study: Programmers Action

14. Action Type: Top-Down Model

15. Action Type: Situation Model

16. Action Type: Program Model

17. Results of Study: Processes C1: Language Expert, Some Accumulated Knowledge

18. C2: Domain Expert, No Accumulated Knowledge

19. C3: Domain Expert, Little Accumulated Knowledge

20. C4: Domain Expert, Significant Accumulated Knowledge

21. Results: Information Needs less significant results not included

22. Conclusion ● Actions – Use of knowledge and generating hypotheses are important programmer actions when working at all levels of abstraction. – At the lower levels, chunking and storing acquired informationis also common.

23. Conclusion ● Process – Little experience in the domain means comprehension will occur at lower levels of abstraction until enough domain experience allows connections to be made from the code to higher levels of abstraction. – Having domain experience but little or no accumulated knowledge about the software will cause comprehension to occur at lower levels of abstraction, but will also allow more efficient use of the existing domain knowledge to make direct connections into the domain model. The situation model can more easily be a bridge between the program and top-down models. – Domain expertise and significant experience with the software allows connection between all three levels of abstraction to be easily made.

24. Conclusion ● Information Needs – Domain concepts and connected program, situation and top down model information is important during corrective maintenance. – Reason because information above the program model level has to be search for and connection made manually. No technology can help doing that.

25. My Thoughts ● Initially doubtful, but the more I read her papers, the more I convince I get. However, only 4 test subject-1 novice, 2 intermediate and 1 expert. Not big enough sample size to really make a win. ● What about comparisons with other maintenance task, adaptive, perfective(enhancement), code reuse, code leveraging? ● The paper assumed that programmers behave in a certain way due to prior knowledge/information needs available, but would the model still be correct if all programmers were given information needs prior to the corrective maintenance task, or would it differ?

26. Extended thoughts ● Can we find out the skills required to understand large code bases from this model? ● Are patterns (like patterns in Java) helps programming, by providing the programmer easier understanding through better abstraction? I think yea, if patterns are carefully created, otherwise, it could be more misleading than helps. ● If we know the users current state, maybe through observing behavior, can we have a program that will give clues to help the user with his search, let it be situation model (suggestions), program model (fish eye view), domain model(top components view)?