1. A Survey-based Study of Maintainability Metrics Luiz Paulo Coelho Ferreira luizpcf@gmail.com

2. Introduction All software development environment faces the same problems: Code will continuously change. If the code of a system grow its maintainability cost grows with it. A well written code can decrease the maintainability cost. 2

3. CheckIO On line teaching platform. Students can solve and publish problems and their solutions are evaluated by other students. Python based; 131 problems collected; 6k+ solutions; 3

4. Survey Settings Survey should last no more than 10 minutes 4 problems to be analyzed; Each question should contain 2 equivalent solutions (ES); A problem should not contain more than 6 ES; The responder should explain its reasons for their choices. 4

5. Pilot Survey Pilot Survey has applied in a group of persons with high knowledge in software engineering and maintainability; Question about the reasons should be an open question; 6 persons responded to the pilot. 5

6. Survey Options Based on results from pilots we choose the options from a multiple choice question of reasons; Documentation Code Complexity Indentation/Code Style Variables Nomenclatures Number of Lines of Code Modularity Expressions size 6

7. Results (Partial) 137 responses (and growing) 7

8. Q1 – Color Maping Solutions 1 and 2 had worse results compared to 3, 4, 5. Solution 4 has 3 votes on CheckIO 8

9. Q2 – Dark Labyrinth Solutions 3 had the best result compared to all Solution 3 has 7 votes on CheckIO 9

10. Q3 – Mind Switcher Solutions 2 had the best result compared to all Solution 5 has 7 votes and 6 has 3 on CheckIO 10

11. Q4 – Lantern River Solutions 6 had the best result compared to all Solution 6 has 3 votes, 4 and 2 have 2 and 5 has 1 11

12. Results(To be done) What is the option that impact positively in good solutions and negatively in bad solutions? Are there any impact on people who have low or none knowledge in python in the research? And graduate x undergraduate? 12

13. Threats to Validity All responders are Brazilians Code in English; Python only; Simple problems and small solutions; Only 4 problems analyzed. 13

14. Conclusions (so far) In most cases CheckIO users have the same opinion as ours programmers But it still have some improvements to be done; Indentation and Style is the most common reasons to chose a good or bad code and there is lot of tools to solve this problem; Documentation, Code Complexity and Variables Nomenclature are others high voted reasons; Number of Lines of Code, except by Q3, did not received high number of votes. 14

15. Questions ?????????? 15