1. Nov 2013- 1 Quality Process for Research Software - Timothy C. Lethbridge Research Software Doesn't Have to be Buggy: A Model-Driven, Test-Driven and Agile Process Applied in a University Research Team November 2013 Timothy C. Lethbridge

2. Nov 2013- 2 Quality Process for Research Software - Timothy C. Lethbridge Motivations for This Talk I was explaining to researchers at a conference how I have been able to maintain Umple –They were interested in the process I followed It has been a very positive experience to experience ‘real’ software development –And not get bogged down it –While maintaining quality –And getting good feedback from students

3. Nov 2013- 3 Quality Process for Research Software - Timothy C. Lethbridge Research Software: Often Short Life or ‘Not Production Quality’ Software developed by computer science researchers often only lasts as long as one student’s PhD –Perhaps a little longer It usually gets messy –The next generation of students often wants to rewrite it The principal investigator wants to move on to other things –The software becomes a drag –Lack of resources for maintenance –Grants are for research, not development

4. Nov 2013- 4 Quality Process for Research Software - Timothy C. Lethbridge Why would we want research software to last for the long haul? To really ‘get it right’ To act as infrastructure for countless experiments and idea exploration To allow the professor and students to get really good at real software engineering

5. Nov 2013- 5 Quality Process for Research Software - Timothy C. Lethbridge My Vision Prodution quality software that persists for 20 or more years Continual improvement of the process Top-notch training, education, and research platform

6. Nov 2013- 6 Quality Process for Research Software - Timothy C. Lethbridge My Approach Adopt software engineering best practices in the software used in the research Blend and mode-switch research approaches –Design based research –Action research –Grounded theory –Empirical experimentation

7. Nov 2013- 7 Quality Process for Research Software - Timothy C. Lethbridge The Platform: Umple Model-oriented compiler –Adds UML and patterns to Java, C++ etc. –Textual UML –High quality code generator for class and state diagrams –Web-based + command line + Eclipse Main site: http://www.umple.orghttp://www.umple.org Web app: http://try.umple.orghttp://try.umple.org Manual: http://manual.umple.orghttp://manual.umple.org Open source: –Google: http://code.umple.orghttp://code.umple.org –Mirrors on Github and SourceForgeGithub SourceForge

8. Nov 2013- 8 Quality Process for Research Software - Timothy C. Lethbridge Robustness maintained because of / despite contributions by >30 HQP 2 completed PhDs (+ 7 underway) 4 completed Masters (+ 1 underway) 22 undergraduate capstone projects (+ 6 underway) 3 postdocs UCOSPUCOSP: Students from UBC, Simon Fraser, UAlberta, USask, URegina, Laurentian, Waterloo, Wilfred Laurier, Guelph, Bishops, Sherbrooke, UNB, Dalhousie International contributions from Brazil. Countless students taught in classrooms –Umple helps students learn UML (CSEET 2011)CSEET 2011

9. Nov 2013- 9 Quality Process for Research Software - Timothy C. Lethbridge Essential elements of the process Red ones are key Open source Multi-level test driven development Model driven development Carefully categorized issues list Continuous integration Lean live documentation for users and developers Low management overhead Marketing and publication

10. Nov 2013- 10 Quality Process for Research Software - Timothy C. Lethbridge Open Source Open to the world as early as possible If there are spinoffs, let them be based on service, not software

11. Nov 2013- 11 Quality Process for Research Software - Timothy C. Lethbridge Multi-level test driven development Use test cases as specifications Multiple levels in the layered architecture –For Umple: Parsing, metamodel instance, generation, execution In Umple, the compiler itself is a big test Note: We have not applied this yet to the user interface http://qa.umple.org

12. Nov 2013- 12 Quality Process for Research Software - Timothy C. Lethbridge Model driven development Always start with the model and generate code from it –It is possible –Requires indoctrination, and students sometimes have to be asked to start again! Simplifies code, reduces volume, enforces abstraction That is what Umple is about, but we have also developed other systems in Umple –Key Umple benefits compared to other MDD tools Model is code; traditional code embedded in mode No need to edit generated code Bidirectional traceability

13. Nov 2013- 13 Quality Process for Research Software - Timothy C. Lethbridge Carefully managed issues list Very-easy, easy, moderate, hard, very-hard Defect, enhancement, undergraduate project, graduate project Priority: Critical, Vhigh, High, Med, Low, Vlow Ownership of issue, but not code http://bugs.umple.org

14. Nov 2013- 14 Quality Process for Research Software - Timothy C. Lethbridge Continuous integration Single master trunk for all work Server builds then posts results –http://cc.umple.orghttp://cc.umple.org Test result browser –http://qa.umple.orghttp://qa.umple.org –100% test pass rate maintained 95% of the time –100% test pass rate within an hour of a failure 99.9% of the time –Only two ‘backouts’ in the last year (failed tests that could not be quickly fixed)

15. Nov 2013- 15 Quality Process for Research Software - Timothy C. Lethbridge Lean Live Documentation for Developers and End Users Code commenting is key: –But comments transfer into documentation –Auto-generated diagrams are a result of MDD –http://metamodel.umple.orghttp://metamodel.umple.org –http://grammar.umple.orghttp://grammar.umple.org User manual is generated: http://manual.umple.orghttp://manual.umple.org If someone doesn’t understand something they update the wiki –http://wiki.umple.orghttp://wiki.umple.org –Exploration of requirements done in issues and wikis –Developers log experiences and progress for all to see

16. Nov 2013- 16 Quality Process for Research Software - Timothy C. Lethbridge Low Management Overhead As a professor, no time for much project management –About 2 hours a week direct interaction with students actively working on the software –I manage issues, set up policies, refine documents –Periodically I work on an issue Code sprint (20 hours) once a semester Code review of patches and commits –Focus on checking the tests –Delegation of inspection to others –Some refactoring

17. Nov 2013- 17 Quality Process for Research Software - Timothy C. Lethbridge Marketing and Publication Not just journals and conferences We need users and feedback from users of our tools Live mirrors: Google Code, Github, FacebookFacebook, Google+, Twitter, Email lists, blogsGoogle+

18. Nov 2013- 18 Quality Process for Research Software - Timothy C. Lethbridge Use of our software in teaching Several professors have taught software engineering concepts using our work –Introduction to software engineering –Advanced software design –Software engineering graduate course –Score project at ICSE –Others

19. Nov 2013- 19 Quality Process for Research Software - Timothy C. Lethbridge Introducing students to Umple 1. Train in the use of the software 2. Development environment setupDevelopment environment setup 3. Coaching on process 4. Code sprint 5. Have them create several patches for easy to moderate problems in several architecture areas 6. Make them a committer (happens for most students)

20. Nov 2013- 20 Quality Process for Research Software - Timothy C. Lethbridge Conclusions Make your academic software high quality by using –TDD –MDD –Continual intregration –Careful agile process It works! Benefits –You and your students get better at software engineering –You can do better research because your infrastructure is better

21. Nov 2013- 21 Quality Process for Research Software - Timothy C. Lethbridge Questions? This talk is available at: http://bit.ly/1fMkhzZ Or http://www.eecs.uottawa.ca/~tcl/presentations/CSER- Nov2013-Talk-RshSWNotBuggy.ppt