1. TWO CASE STUDIES OF OPEN SOURCE SOFTWARE DEVELOPMENT: APACHE AND MOZILLA
HAKAN TERZIOGLU
2/24/2019
EEL 5881

2. OSS development– characteristics?
Differences between OSS development and usual industrial style of dev.
Number of participants
Assignment of work
No explicit system-level design
No project plan, schedule, or list of deliverables
2/24/2019
EEL 5881

3. OSS development-contd.
Lack of coordination mechanisms?
What are the possible reasons for the success of OSS development?
Linus’s Law: ‘Many eyeballs’ looking for the problems
Developers have real fashion
2/24/2019
EEL 5881

4. OSS development
Linux
Apache – the most widely deployed Web server (> 7 million Web sites)
We need empirical evidence
2/24/2019
EEL 5881

5. Article structure Study of the Apache server (Study I)
Hypotheses formed according to the results of Study I
Study of the Mozilla project
The data will be checked if they support the hypotheses formed
2/24/2019
EEL 5881

6. Research questions
Two key sets of properties will be focused by the questions
Basic parameters of the process by which Apache and Mozilla came to exist
The outcomes of these processes
2/24/2019
EEL 5881

7. Research questions-contd.
First set of questions:
Q1:What were the processes used to develop Apache and Mozilla?
Q2:How many people wrote code for new functionality? How many people reported problems? How many people repaired defects?
2/24/2019
EEL 5881

8. 10/1/2002 First set of questions- contd.
Q3:Were these functions carried out by distinct groups of people, that is, did people primarily assume a single role? Did large numbers of people participate somewhat equally in these activities, or did a small number of people do most of the work?
Q4:Where did the code contributors work in the code? Was strict code ownership enforced on a file or module level?
2/24/2019
EEL 5881

9. Research questions-contd.
Second set of questions:
Q5:What is the defect density of Apache and Mozilla code?
Q6:How long did it take to resolve problems? Were the high priority problems resolved faster than low priority problems? Has resolution interval decreased over time?
2/24/2019
EEL 5881

10. Data Sources Apache Data Sources Mozilla Data Sources
Developer List( )
Concurrent Version Control Archive(CVS)
Problem Reporting Database (BUGDB)
Mozilla Data Sources
CVS Archive
Bugzilla problem tracking system
2/24/2019
EEL 5881

11. Data Sources-contd. Commercial Projects Data Sources
Extended Change Management System (ECMS)-for initiating and tracking
Source Code Control system (SCCS)-for managing different versions of the files
2/24/2019
EEL 5881

12. Commercial Development Process
Projects chosen for the comparison
Why they were chosen?
Size of them
Familiarity of the authors with these projects
2/24/2019
EEL 5881

13. Study I: The Apache Project: The Apache Development Process
Q1:What was the process used to develop Apache?
Roles and Responsibilities
Identifying Work to be Done
Assigning and Performing Development Work
Prerelease Testing
Inspections
Managing Releases
2/24/2019
EEL 5881

14. The Apache Development Process
Quantitative Results
The size of the Apache Development Community? (Q2)
Participation is quiet wide:400 individuals contributing the code
182 people contributed to 695 fixes
249 people contributed to 6092 code submissions
3060 people submitted 3975 PRs-whereas 458 individuals submitted 591 reports that caused a change in the code
2/24/2019
EEL 5881

15. The Apache Development Process
Quantitative Results-contd.
How was work distributed within the Development Community?
The top 15 developers contributed more than 83% of the MRs and deltas, 88% of added lines, and 91% of deleted line
The core of 15 developers produced only 66% of the fixes
The participation rate was 26 developers per 100 fixes and 4 developers per 100 code submissions
2/24/2019
EEL 5881

16. The Apache Development Process
2/24/2019
EEL 5881

17. The Apache Development Process
Quantitative Results-contd.
How was work distributed within the Development Community?-contd.
Comparison of code productivity of Top Apache Developers and Top Developers in Several Commercial Projects
2/24/2019
EEL 5881

18. The Apache Development Process
Quantitative Results-contd.
Who reports problems?
The BUGDB had 3975 distinct PRs.
2600 developers submitted one report, 306 submitted two, 85 submitted three, and the maximum number of PRs submitted by one person was 32.
Of the top 15 problem reporters only three are also core developers.
2/24/2019
EEL 5881

19. The Apache Development Process
Code ownership
Defects
Measure of post-release defects—disadvantages?
KLOCA (defects per thousand lines of code added)
2/24/2019
EEL 5881

20. The Apache Development Process
Defect density results
The user-perceived defect density of the Apache product is inferior to that of the commercial products.
Defect density of the code before system test is much lower.
2/24/2019
EEL 5881

21. The Apache Development Process
Time to Resolve Problem Reports
50% within a day, 75% within 42 days, 90% within 140 days
2/24/2019
EEL 5881

22. The Apache Development Process
Priority
In Apache BUGDB, the priority field is entered by the person reporting the problem
Categorizing the PRS which reflect the number of users affected
2/24/2019
EEL 5881

23. The Apache Development Process
Hypotheses
#1:Open source developments will have a core of developers who control the code base. This core will be no larger than 10 to 15 people, and will create approximately 80% or more of the new functionality.
#2:For projects that are so large that 10 to 15 developers cannot write 80% of the code in a reasonable time frame, a strict code ownership policy will have to be adopted to separate the work of additional groups, creating in effect, several related OSS projects.
2/24/2019
EEL 5881

24. The Apache Development Process
Hypotheses-contd.
#3:In successful open source developments, a group larger by an order of magnitude than the core will repair defects, and a yet larger group will report problems.
#4:Open source developments that have a strong core of developers but never achieve large numbers of contributors beyond that core will be able to create new functionality but will fail because of a lack of resources devoted to finding and repairing defects.
2/24/2019
EEL 5881

25. The Apache Development Process
Hypotheses-contd.
#5:Defect density in open source releases will generally be lower than commercial code that has only been feature-tested, that is, received a comparable level of testing.
#6:In successful open source developments, the developers will also be users of the software.
#7:OSS developments exhibit very rapid responses to customer problems.
2/24/2019
EEL 5881

26. STUDY II:THE MOZILLA PROJECT The Mozilla Development Process
Roles and Responsibilities- mozilla.org staff
Identifying work to be done
Assigning and performing development work
Pre-lease testing
Inspections
Managing releases
2/24/2019
EEL 5881

27. The Mozilla Development Process
Quantitative Results
The size of the Mozilla Development Community
486 people contributed the code and 412 people contributed code to PR fixes-CVS
6837 people reported about PRs and 1403 people reported PRs that make a change in the code
2/24/2019
EEL 5881

28. The Mozilla Development Process
External participation
2/24/2019
EEL 5881

29. The Mozilla Development Process
2/24/2019
EEL 5881

30. The Mozilla Development Process
Code ownership
The module owner is responsible for fielding bug reports, enhancement requests, patch submissions, etc.
So code ownership is enforced in Mozilla Project.
2/24/2019
EEL 5881

31. The Mozilla Development Process
Defects
2/24/2019
EEL 5881

32. The Mozilla Development Process
Time to resolve Problem Reports
The mandatory inspection of changes in Mozilla almost doubles the PR resolution interval
Significant relationship between interval and priority
Substantial variation in the PR resolution interval by module
2/24/2019
EEL 5881

33. The Mozilla Development Process
2/24/2019
EEL 5881

34. HYPOTHESES REVISITED
Open source developments will have a core of developers who
control the code base, and will create approximately 80% or more of the new
functionality. If this core group uses only informal ad hoc means of coordinating their work, the group will be no larger than 10 to 15 people.
If a project is so large that more than 10 to 15 people are required to complete 80% of the code in the desired time frame, then other mechanisms,rather than just informal ad hoc arrangements, will be required in order to coordinate the work. These mechanisms may include one or more of the following:
explicit development processes, individual or group code ownership, and
required inspections.
2/24/2019
EEL 5881

35. HYPOTHESES REVISITED
In successful open source developments, a group larger by an
order of magnitude than the core will repair defects, and a yet larger group (by
another order of magnitude) will report problems.
Open source developments that have a strong core of developers but never achieve large numbers of contributors beyond that core will be able to create new functionality but will fail because of a lack of resources devoted to finding and repairing defects.-NOT ABLE TO TEST
Defect density in open source releases will generally be lower than commercial code that has only been feature-tested, that is, received a comparable level of testing.-TENTATIVELY SUPPORTED
In successful open source developments, the developers will also be users of the software.
OSS developments exhibit very rapid responses to customer problems.
2/24/2019
EEL 5881