1. www.canarie.ca 1 This Changes Everything: Accelerating Scientific Discovery through High Performance Digital Infrastructure CANARIE’s Research Software Program Scott Henwood | HPCS – Halifax, NS | June 26, 2014

2. www.canarie.ca CANARIE More than just the National Research and Education Network 2

3. www.canarie.ca 3

4. What are we mandated to do? Network Operations Technology Innovation Private Sector Innovation 4

5. www.canarie.ca Research Software Program Formerly known as “Research Middleware” 5

6. www.canarie.ca When the national network was formed … 6 Image source: Flickr user pchow98 (Creative Commons)

7. www.canarie.ca Network & Network Services 7

8. www.canarie.ca Challenge 1 Making use of digital infrastructure 8

9. www.canarie.ca The Missing Piece: Software for Researchers 9 Enables access to the digital infrastructure Keeps researcher focus on research, not technology Accelerates time to discovery

10. www.canarie.ca Research Platform Development (2007-2012) Two rounds of research platform development took place during CANARIE’s 2007-2012 mandate 21 research platforms funded Resulting research platforms cover a wide variety of disciplines, mostly in “big data” 10 Image: Expresspony (Creative Commons)

11. www.canarie.ca Participants include: HEPnet Canada CYBERSKA 11

12. www.canarie.ca Platform Architecture (2007-2012) 12 Platform 1 Service 1 Service 2 Service 3 Service 4 Platform 2 Service 1 Service 2 Platform 3 Service 1 Service 2 Service 3 Platform 4 Service 1 Service 2 Service 3 Service 4 Service 5

13. www.canarie.ca Valuable functionality locked inside Platforms 13 Source: John Gress/Reuters

14. www.canarie.ca Design differences would have made sharing difficult 14

15. www.canarie.ca Some work repeated over and over 15

16. www.canarie.ca Challenge 2 Addressing opportunities for improvement encountered in Research Platform development 16

17. www.canarie.ca Research Software Services (2012-2013) Developed in consultation with Platform developers from the previous mandate Services centric versus Platform-centric Short term goal was sharing services between existing platforms: Done Mid term goal is to provide the beginnings of a library of services that can be used during future platform development: Underway Ultimately, have a library of services of value to all Canadian researchers. 17

18. www.canarie.ca Platform Architecture after initial service round (2012-2013) 18 Platform 1 Service 1 Service 2 Platform 2 Platform 3 Platform 4 Service 1 Service 2 Service 1 Service 2 Service 1 Service 2 Service 3 Service 4 Service A Service B Service C

19. www.canarie.ca Services for digital infrastructure access 19 Babelcloud Scheduling of jobs across multiple cloud resources using a single API Cloud Computing Batch Service (by HEP Research Computing) Prepare batch jobs by selecting VM images, applications, data and parameters and submit job for processing Shared Software Repository (by UVic HEP Research Computing) A read-only networked filesystem service (CVMFS) for applications used by multiple VM images (Compute Canada trialing with GenAp)

20. www.canarie.ca Services for digital infrastructure access 20 Visualization Server Pool Management (by the Institute for Space Imaging Science) Centralized session management of virtual machines running visualization software CICSTART A set of platform support services including: File transfer and caching User Authentication A virtual file system A data catalogue A scripting language CYBERSKA

21. www.canarie.ca Efficient reuse of software Services from Platforms (2014-2015) 21 Software Platforms Previous Mandate: 21 Platforms funded 21 12 15 45 Software Services New Services created …leveraged by new Platforms …spinoff additional Services.

22. www.canarie.ca 1 st 15 software Services as reused by 12 new software Platforms Imagine the collaboration possible with 60 software Services! 22 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 11 12

23. www.canarie.ca Platform Architecture (2014-2015) Reduction in the time between funding grant and actual start of research More opportunities for collaboration 23 Platform 1 Service A Service 2 Platform 2 Service 1 Platform 3 Service 1 Service 2 Platform 4 Service 1 Service 2 Service 1 Service B Service C Service D Service E Service F Service G Service H Service I Service J

24. www.canarie.ca Challenge 3 There’s more than one way to make an effective Service 24

25. www.canarie.ca Managed Services Platform Service Owner’s Cloud Resources Production Managed Service 25

26. www.canarie.ca Self-deployed Service Platform Platform Owner’s Cloud Resources Production Service Owner’s Cloud Resources Evaluation & Download 26 Self-deployed Service: Demo Instance Self-deployed Service: Production Instance

27. www.canarie.ca Component Service Service Owner’s Cloud Resources Download Only Platform 27 Component Service

28. www.canarie.ca Challenge 4 There isn’t a single online resource for research software in Canada 28

29. www.canarie.ca Introducing the Canadian Research Software Portal science.canarie.ca 29

30. www.canarie.ca Software Service Registry 30

31. www.canarie.ca Service information 31

32. www.canarie.ca Service reliability 32

33. www.canarie.ca Service monitoring: Failure Notification Email notification Web service Nagios plugin 33

34. www.canarie.ca Service usage 34

35. www.canarie.ca Platform info 35

36. www.canarie.ca Moving Forward Research software program direction 36

37. www.canarie.ca So where are we headed? 37 Research, collaboration, and innovation. Anytime. Image source: www.michaelnichols.org

38. www.canarie.ca And what about the Research Software Program? 38 Continue to lead the development of research software Work closely with digital infrastructure ecosystem partners to develop tools and applications

39. www.canarie.ca Challenge 5 Advancing the ecosystem 39

40. www.canarie.ca Service evolution Expanding and sustaining the catalogue of Services –through digital infrastructure providers …and through other funding bodies …internationally. 40 More value through increased use of Services –Promotion by CANARIE …and through digital infrastructure providers …and through other funding bodies …internationally.

41. www.canarie.ca New researchers, new challenges Research groups without experienced software teams backing them up have increasing need for research platform functionality Opportunity to create a generalized platform for some types of research? 41 Needs some more discussion with the community Image source: leadingeffectively.com

42. www.canarie.ca Research Software Portal - roadmap Today: A software service and platform registry Tomorrow: Other types of research resources as well Ultimately: A destination for research software in Canada 42

43. www.canarie.ca Research Software long-term goal VM Images Scripts Data Sets Research Portal 43 Platform 25 Service 1 Service 2 Platform 32 Service 1 Service 2 Platform 33 Service 1 Service 2 Platform 42 Service 1 Service 2 Service A Service B Service C Service D Service E Service F Service G Service H Service I Service J Service K Service L Service M Service N Service O Service P Service Q Service R Service S Service T

44. www.canarie.ca Further Info 44

45. www.canarie.ca Software Technical Advisory Committee (STAC) 45 A group of volunteers from the community with extensive experience in research software development Looking for 1 or two more members Qualifications: –Working senior developer, architect or hands-on manager –Knowledge of all aspects of research software development –Available for 1 hour/month (max) for video conference –Ideally from central or eastern Canada

46. www.canarie.ca Technical Update – Newsletter 3 editions out so far One edition approximately every two months, maybe a little less during the summer Contributions from the community are most welcome To subscribe, email support@science.canarie.ca 46 Image: Lewis Hine, 1910

47. www.canarie.ca Scott Henwood scott.henwood@canarie.ca 613-943-5434 47