TWO CASE STUDIES OF OPEN SOURCE SOFTWARE DEVELOPMENT: APACHE AND MOZILLA HAKAN TERZIOGLU 2/24/2019 EEL 5881

OSS development– characteristics? Differences between OSS development and usual industrial style of dev. Number of participants Assignment of work No explicit system-level design No project plan, schedule, or list of deliverables 2/24/2019 EEL 5881

OSS development-contd. Lack of coordination mechanisms? What are the possible reasons for the success of OSS development? Linus’s Law: ‘Many eyeballs’ looking for the problems Developers have real fashion 2/24/2019 EEL 5881

OSS development Linux Apache – the most widely deployed Web server (> 7 million Web sites) We need empirical evidence 2/24/2019 EEL 5881

Article structure Study of the Apache server (Study I) Hypotheses formed according to the results of Study I Study of the Mozilla project The data will be checked if they support the hypotheses formed 2/24/2019 EEL 5881

Research questions Two key sets of properties will be focused by the questions Basic parameters of the process by which Apache and Mozilla came to exist The outcomes of these processes 2/24/2019 EEL 5881

Research questions-contd. First set of questions: Q1:What were the processes used to develop Apache and Mozilla? Q2:How many people wrote code for new functionality? How many people reported problems? How many people repaired defects? 2/24/2019 EEL 5881

10/1/2002 First set of questions- contd. Q3:Were these functions carried out by distinct groups of people, that is, did people primarily assume a single role? Did large numbers of people participate somewhat equally in these activities, or did a small number of people do most of the work? Q4:Where did the code contributors work in the code? Was strict code ownership enforced on a file or module level? 2/24/2019 EEL 5881

Research questions-contd. Second set of questions: Q5:What is the defect density of Apache and Mozilla code? Q6:How long did it take to resolve problems? Were the high priority problems resolved faster than low priority problems? Has resolution interval decreased over time? 2/24/2019 EEL 5881

Data Sources Apache Data Sources Mozilla Data Sources Developer Email List(EMAIL) Concurrent Version Control Archive(CVS) Problem Reporting Database (BUGDB) Mozilla Data Sources CVS Archive Bugzilla problem tracking system 2/24/2019 EEL 5881

Data Sources-contd. Commercial Projects Data Sources Extended Change Management System (ECMS)-for initiating and tracking Source Code Control system (SCCS)-for managing different versions of the files 2/24/2019 EEL 5881

Commercial Development Process Projects chosen for the comparison Why they were chosen? Size of them Familiarity of the authors with these projects 2/24/2019 EEL 5881

Study I: The Apache Project: The Apache Development Process Q1:What was the process used to develop Apache? Roles and Responsibilities Identifying Work to be Done Assigning and Performing Development Work Prerelease Testing Inspections Managing Releases 2/24/2019 EEL 5881

The Apache Development Process Quantitative Results The size of the Apache Development Community? (Q2) Participation is quiet wide:400 individuals contributing the code 182 people contributed to 695 fixes 249 people contributed to 6092 code submissions 3060 people submitted 3975 PRs-whereas 458 individuals submitted 591 reports that caused a change in the code 2/24/2019 EEL 5881

The Apache Development Process Quantitative Results-contd. How was work distributed within the Development Community? The top 15 developers contributed more than 83% of the MRs and deltas, 88% of added lines, and 91% of deleted line The core of 15 developers produced only 66% of the fixes The participation rate was 26 developers per 100 fixes and 4 developers per 100 code submissions 2/24/2019 EEL 5881

The Apache Development Process 2/24/2019 EEL 5881

The Apache Development Process Quantitative Results-contd. How was work distributed within the Development Community?-contd. Comparison of code productivity of Top Apache Developers and Top Developers in Several Commercial Projects 2/24/2019 EEL 5881

The Apache Development Process Quantitative Results-contd. Who reports problems? The BUGDB had 3975 distinct PRs. 2600 developers submitted one report, 306 submitted two, 85 submitted three, and the maximum number of PRs submitted by one person was 32. Of the top 15 problem reporters only three are also core developers. 2/24/2019 EEL 5881

The Apache Development Process Code ownership Defects Measure of post-release defects—disadvantages? KLOCA (defects per thousand lines of code added) 2/24/2019 EEL 5881

The Apache Development Process Defect density results The user-perceived defect density of the Apache product is inferior to that of the commercial products. Defect density of the code before system test is much lower. 2/24/2019 EEL 5881

The Apache Development Process Time to Resolve Problem Reports 50% within a day, 75% within 42 days, 90% within 140 days 2/24/2019 EEL 5881

The Apache Development Process Priority In Apache BUGDB, the priority field is entered by the person reporting the problem Categorizing the PRS which reflect the number of users affected 2/24/2019 EEL 5881

The Apache Development Process Hypotheses #1:Open source developments will have a core of developers who control the code base. This core will be no larger than 10 to 15 people, and will create approximately 80% or more of the new functionality. #2:For projects that are so large that 10 to 15 developers cannot write 80% of the code in a reasonable time frame, a strict code ownership policy will have to be adopted to separate the work of additional groups, creating in effect, several related OSS projects. 2/24/2019 EEL 5881

The Apache Development Process Hypotheses-contd. #3:In successful open source developments, a group larger by an order of magnitude than the core will repair defects, and a yet larger group will report problems. #4:Open source developments that have a strong core of developers but never achieve large numbers of contributors beyond that core will be able to create new functionality but will fail because of a lack of resources devoted to finding and repairing defects. 2/24/2019 EEL 5881

The Apache Development Process Hypotheses-contd. #5:Defect density in open source releases will generally be lower than commercial code that has only been feature-tested, that is, received a comparable level of testing. #6:In successful open source developments, the developers will also be users of the software. #7:OSS developments exhibit very rapid responses to customer problems. 2/24/2019 EEL 5881

STUDY II:THE MOZILLA PROJECT The Mozilla Development Process Roles and Responsibilities- mozilla.org staff Identifying work to be done Assigning and performing development work Pre-lease testing Inspections Managing releases 2/24/2019 EEL 5881

The Mozilla Development Process Quantitative Results The size of the Mozilla Development Community 486 people contributed the code and 412 people contributed code to PR fixes-CVS 6837 people reported about 58000 PRs and 1403 people reported 11616 PRs that make a change in the code 2/24/2019 EEL 5881

The Mozilla Development Process External participation 2/24/2019 EEL 5881

The Mozilla Development Process 2/24/2019 EEL 5881

The Mozilla Development Process Code ownership The module owner is responsible for fielding bug reports, enhancement requests, patch submissions, etc. So code ownership is enforced in Mozilla Project. 2/24/2019 EEL 5881

The Mozilla Development Process Defects 2/24/2019 EEL 5881

The Mozilla Development Process Time to resolve Problem Reports The mandatory inspection of changes in Mozilla almost doubles the PR resolution interval Significant relationship between interval and priority Substantial variation in the PR resolution interval by module 2/24/2019 EEL 5881

The Mozilla Development Process 2/24/2019 EEL 5881

HYPOTHESES REVISITED Open source developments will have a core of developers who control the code base, and will create approximately 80% or more of the new functionality. If this core group uses only informal ad hoc means of coordinating their work, the group will be no larger than 10 to 15 people. If a project is so large that more than 10 to 15 people are required to complete 80% of the code in the desired time frame, then other mechanisms,rather than just informal ad hoc arrangements, will be required in order to coordinate the work. These mechanisms may include one or more of the following: explicit development processes, individual or group code ownership, and required inspections. 2/24/2019 EEL 5881

HYPOTHESES REVISITED In successful open source developments, a group larger by an order of magnitude than the core will repair defects, and a yet larger group (by another order of magnitude) will report problems. Open source developments that have a strong core of developers but never achieve large numbers of contributors beyond that core will be able to create new functionality but will fail because of a lack of resources devoted to finding and repairing defects.-NOT ABLE TO TEST Defect density in open source releases will generally be lower than commercial code that has only been feature-tested, that is, received a comparable level of testing.-TENTATIVELY SUPPORTED In successful open source developments, the developers will also be users of the software. OSS developments exhibit very rapid responses to customer problems. 2/24/2019 EEL 5881