1. Revealing Semantic Quality Characteristics of Free and Open Source Software Stergios Draganidis stdraganidis@gmail.com Kerstin V. Siakas siaka@it.teithe.gr Department of Informatics Technological Educational Institute of Thessaloniki, Greece

2. Outline This paper describes the initial stage of a research project at the department of Informatics of Alexander Technological Education Institute of Thessaloniki in Greece 1 The research project investigates the quality of FLOSS 2 considering underlying human factors that affect the six high level quality characteristics according to ISO-9126 1 This work is funded by the Greek Ministry of Education (25%) and European Union (75%) under the EPEAEK II program “Archimedes ΙΙ“ 2 Free/Libre Open Source Software

3. Objectives to introduce a scientific, rational and structured instrument for FLOSS product assessment to correlate the quality characteristics described in ISO-9126 with measurable aspects and underlying factors of FLOSS collaborative development to use the results from the initial stage for improvement of the survey instrument to be used on a larger scale

4. State-of-the-art During the last decades a large number of FLOSS products have been developed The development of software through collaborative informal networks is proposed to produce high quality products Several products have become competitive in a variety of applications Gnome – desktop environments Linux – operating systems Mozilla – web browsers Apache – web servers

5. FLOSS Free/Libre and Open Source Software Combination of the terms “Free Software” – Free Software Foundation “Open Source Software” – Open Source Initiative Emphases the ‘libre’ meaning of the word free Refers to software (available source code) that gives the freedom to the user to: run the program for any purpose to study the code to modify the code to distribute the original or modified program without any restriction or any obligation

6. Free Software – Open Source Software two ideological movements that target the same goal: To evolve knowledge and develop better products The motivations of each movement differ and many discussions have been made to disambiguate their nature the terms “free software” and “open source software” are usually used interchangeably to describe the same thing

7. FLOSS Development Collaboration and co-creation of the code by developers (professional & amateur programmers) and users of varying levels of programming experience Typical FLOSS developers: work: independently of any organisation for sake of the community projects isolated in their own pace, space and time are: geographically dispersed simultaneously users of the software

8. Typical Developer Contribution Start from existing or partial code Take part in the development of a new tool or a utility Post the evolving code to a newsgroup whose members will try out the program and provide feedback (programming solutions, improvements) The enhanced code is then posted to the group, who accepts or discards it and redistribute the final product freely available to the wider community

9. One person = developer + user The developers are themselves users of the software  better understanding and solution of the problems or bugs In Proprietary Software: the developer has difficulties to understand the often ambiguous user requirements Inadequate communication between the developer and the user creates inflexible to future changes products and products that are difficult to maintain  One person acting as developer and user simultaneously in the development of FLOSS products seems to be a prominent feature affecting the quality characteristics of the software product

10. FLOSS Quality The first FLOSS products were expert developers using empirical procedures and metrics that ensure high quality products delivery Today, the quality of FLOSS projects is not based on a specific method or metrics The evaluation of quality is based on arbitrary criteria that usually are not sufficient or adequate to represent the quality of the product Projects hosts provide limited metrics eg. sourceforge.net provides: projects size number of developers activity rank weekly download

11. * http://www.infonomics.nl/FLOSS/report/ Free/Libre and Open Software: Survey and Study International Institute of Infonomics University of Maastricht, The Netherlands in 2002 First attempt for quantification and measurement of underlying factors of quality The Acronym FLOSS was used for first time Delivered a large number of measurements and primary data related to FLOSS usage and development The aims: To provide indicators that rationally explain the functioning of the collaborative networks that do not have primary dependence on money To investigate possible impact of non monetary collaborative development on computer industry, business and economy

12. ISO/IEC 9126 Quality Model A framework for the evaluation of software product quality Applicable to every type of software Defines six high level product quality characteristics: Portability Efficiency Reliability Functionality Usability Maintainability Product QualityAttributesSub-characteristics Measurable entity Not defined in standard

13. The ISO/IEC 9126 Standard

14. ISO 9126 Divisions The standard is divided into four parts which address: the quality model - ISO/IEC 9126-1 (2001) external metrics - ISO/IEC 9126-2 (2003) internal metrics - ISO/IEC 9126-3 (2003) quality in use metrics - ISO/IEC 9126-4 (2004)

15. Research methodology : Survey on-line questionnaire http://sdragan.phpnet.us/questionnaire.htm Department of Informatics Technological Educational Institute of Thessaloniki, Greece

16. Target group Active persons of FLOSS projects, such as: Project managers Programmers users The research will be carried out from April 2007 until August 2008

17. Questionnaire design Php and MySql was used to implement the online questionnaire 21 questions relating to significant attributes describing human features and processes that have impact on quality of FLOSS according to ISO-9126 Emphasis on: Simplicity, user friendliness and wide acceptance unbiased understandable questions Demographic questions related to the responders for clustering of the results investigation of possible impacts to the quality characteristics

18. Sample Issue Concerns Minimum tolerated sample size for smallest sub-group of importance? How to ensure a representative sample? Our target group in the initial stage: The Greek Floss community Results may not be representative for the wider FLOSS community

19. Pilot Study: Questionnaire moderation & promotion The initial questionnaire was developed in the Greek language distributed to many individuals in charge of Greek FLOSS communities aiming to evaluate the questionnaire regarding the validity and outline to secure future commitment from the FLOSS community 9 responses were received with useful comments The feedback was used to improve the questionnaire

20. Questionnaire moderation & Promotion The moderated questionnaire was translated to English in order to reach a greater population outside Greece Available on-line to enable easy completion and analysis of the results The on-line questionnaire has been promoted to many FLOSS communities in order to ensure statistically sufficient responses and produce valid results

21. Survey process Acquisition of primary data that describe attributes and processes constituting FLOSS development Data Storage in a database system in order to simplify appropriate transformation of the data before the analysis Data analysis Statistical analysis (SPSS-15) Data mining methods will be applied to predict the behaviour of the project to classify projects according to their quality measurements to point out quality related attributes of the FLOSS development

22. Data Analysis Considerable effort was made to enable the extraction of valid results rapidly from the primary data A system that collects the questionnaire responses and processes them accordingly has been developed

23. Expected Outcome to reveal factors that may have impact on the quality of FLOSS products to increase the comprehension of FLOSS quality attributes to raise awareness of the dynamics that influence the development process to contribute to a more scientific, rational and structured assessment of FLOSS products & processes

24. Conclusions The initial stage of a research project aiming to determine quality level of FLOSS products was described Our approach evolved the research approaches that concentrate on internal characteristics of ISO-9126 to include external characteristics and quality in use The correlation of quality characteristics of ISO- 9126 with measurable aspects and underlying factors of FLOSS collaborative development was expected to provide a more scientific, rational and structured tool for FLOSS product assessment

25. Further Work Stage 1: Extending the research group Collaboration welcome! Searching for EU funding Preparing proposal Improving the questionnaire Further promotion of the questionnaire

26. Further Work Stage 2: Collecting data Analysis of the collected data Statistical analysis of the data by SPSS Application of data mining methods Stage 3: Point out different perspectives and summarize them to useful information Dissemination

27. Thank you for your attention Any questions..