1. CS4723 Software Validation and Quality Assurance Lecture 9 Bug Report Management

2. 2 Issue Tracking System  After software testing  Testers find a lot of bugs  They need to tell the developers about the bugs  They usually put them into a so-called issue tracking system  Compared with Mailing list  Hard to manage, come with all other mails  Not categorized by topic  No features for describing specific aspects in SE (versions, components, commits, etc)

3. 3 Issue Tracking System  A platform for developers to communicate with each other  Like a forum  People can register and raise a issue  The one who raise the issue will describe the issue in details  Others can comment

4. 4 Issue Tracking System  What is special  More structured for describing issues  Component, assignees, schedules, status, resolution  Customizable  Change the contents while progress is made (status, resolution)  Sometimes allow anonymous issue raising  Users of software are involved  They use the software and raise bugs

5. 5 Issue: an example

6. 6 Issues – Life Cycle Submission Triaging Inspecting Closed Fixing Verifying Reporter Triager Developer Duplicate/Invalid Not reproducible Retriage Refix Assigned Fixed Re-open

7. 7 Issues  Type of issues  Process of issues  Resolution of issues

8. 8 Type of issues  Bug reports  Feature Requests  Patches

9. 9 Type of Issues  Bug report (e.g., system shows wrong message)  About a bug of the software  Raised by a user/developer  Should include:  Step to reproduce  Expected behavior  Actual behavior  Stack trace or error message if any

10. 10 Example of a bug report https://bugzilla.mozilla.org/show_bug.cgi?id=876495

11. 11 Type of Issues  New feature (e.g., add sorting to results)  About add a new feature, e.g., add sorting by modify time  Raised by a user/developer

12. 12 Example of a feature request https://bugzilla.mozilla.org/show_bug.cgi?id=382491

13. 13 Type of Issues  Patch (e.g., add checking for input validity)  A fix to the software  By a professional user or a developer on a important fix  Should include:  Version to patch  Patch code: basically a code diff, Diff (buggy, correct)

14. 14 Example of a patch https://bugzilla.mozilla.org/show_bug.cgi?id=657763

15. 15 Process of an issue  Open/New  The issue is raised  Nobody in the project has looked at it yet  Assigned  A person called triager assign issue to a developer  Bug report: the developer will first reproduce the bug, and then try to fix  Feature request: discuss with colleagues on whether to accommodate the request, and implement the feature  Patch: Validate the patch  Milestone: may be assigned to a sub-group

16. 16 Process of an issue  Closed  When the decision on the issue is made in any way  Fixed, usually accompany with code commits  Reject  Later  Reopened  After the issues is closed, something happens and the issue become active again  Incomplete fix  Start to implement a postponed feature  Or revoke any wrong decision before

17. 17 Resolution of an issue  Fixed  A bug is fixed  A feature is added  A patch is applied  Invalid  Bug cannot be reproduce  Feature does not make sense (request is not understandable)  Patch is not correct

18. 18 Example of invalid bug reports https://bugzilla.mozilla.org/show_bug.cgi?id=198057

19. 19 Resolution of an issue  Duplicate  The issue is a duplicate of another existing issue  Often happens for user raised issues  Usually bug report / feature request  Some issue tracking system allows merge of duplicate issues  Won’t fix  The developers decide to not fix the bug or accommodate the new feature  Limited human resource, new version is about to released

20. 20 Example of duplicate bug reports https://bugzilla.mozilla.org/show_bug.cgi?id=137325

21. 21 Example of won’t fix bug reports https://bugzilla.mozilla.org/show_bug.cgi?id=143329 https://bugzilla.mozilla.org/show_bug.cgi?id=290131

22. 22 Existing Issue Tracking Systems  Many project hosting websites provide issue tracking systems also  Google Code: so you will also have a issue tracking system  GIT Hub  Source Forge  Issue tracking service provider:  Bugzilla  Companies often has their own issue tracking system  Users can submit issues, but the tracking system is not public

23. 23 Tips on submitting an issue report  Search for existing reports  Specify: product, platform/OS, version  Describe  Input / Steps to reproduce  Expected results / actual results  Bug: Always reproducible or happen randomly (maybe related to concurrency, system resource, environ, etc.)  Provide  Snapshot / bug: error message  Bug: Stack trace

24. 24 Submit a bug report  Demo on sourceforge, bugzilla, and github https://bugzilla.mozilla.org/enter_bug.cgi https://sourceforge.net/p/sweethome3d/bugs/new/ https://github.com

25. 25 What are the most important part in a bug report?  Step to reproduce?  Test cases?  Screenshots?  Error Messages?  Stack Traces?  Version?  …

26. 26 A survey did by a number of German Scientists  With 130 developers and 215 reporters  Ask about what are the information they provided / used and expect to be useful  Information types include:  steps to reproduce, stack traces, test cases, observed behavior, screenshots, code examples, summary, version,…

27. 27 Answer of the developers  Step to reproduce: 83%  Stack trace: 57%  Test cases: 51%  Observed Behavior: 33%  Screenshots: 26%  Expected behavior: 22%  Code examples: 14%  Summary: 13%  Version: 12%  Error report: 12%

28. 28 Mismatch of understanding

29. 29 Tips on triaging and handling issue reports  Triaging:  Search for existing reports  Talk with the developers who are familiar with the area  Decide the severity and who should handle the report (according to the expertise and workload)  Handling an issues report (e.g., fix, add feature)  Reproduce  For features: communicate on the issue tracking system about whether and how to support it  If reproducible  Always report progress as comments in the issue tracking system: others may help or change their work accordingly

30. 30 Review of issue tracking system  Where the bugs go to after you find them  Type of issues  Resolution of issues  How to write a good bug report?  How bug reports are handled?