Software Infrastructure for Sustained Innovation (SI 2 ) PI meeting Arlington, VA January 17-18, 2013 Ewa Deelman, University of Southern California Miron Livny, University of Wisconsin Madison

2 SI 2 PI Meeting Goals  Bring together Principle Investigators of the leading software cyberinfrastructure projects and discuss issues relevant to the community as we move into the future  Need for NSF software to be used effectively by scientists and engineers  SI2 projects:  Scientific Software Elements (SSE)  Scientific Software Integration (SSI)  SI2 Institute Conceptualizing

3 SI2 PIs  60 PIs  9 invited participants  Provide different types of software: HPC codes and libraries, middleware, domain-specific software and tools  Workshop Materials on the website –1-pagers –Project websites –Questionnaire, question contributors: James Howison, Nancy Wilkins-Diehr 62 Projects: 12 Institute Conceptualization, 20 SSIs, 30 SSEs

4 Invited participants  Jim Herbsleb, Carnegie Mellon University  Jason Priem, University of North Carolina, Chapel Hill  James Howison- University of Texas Austin  Jim Jagielski, Director Apache Software Foundation  Neil Chue Hong, Director UK Software Sustainability Institute  James Taylor, Emory University  Michael McLennan, Purdue University  Jarek Nabrzyski, Notre Dame  Von Welch, Indiana University

5 Meeting Goals  Increase the awareness of SI2 projects within the program and more broadly within NSF  Foster collaborations  Discuss common concerns/issues  Learn from each others’ experiences  Contribute to the discussion on the shape of Software Institutes

6 Meeting Outcomes  A website with materials from all the projects  The beginning of a dialogue amongst projects  Analysis of the questionnaire  A report on the discussions (volunteer writers needed)

7 Initial Survey Results, 59 responses as of 1/7/13 (42 finished survey) 1) Novel Science (20) 2) Quality Software (13) 3) Community Growth (7) 4) Sustainability (3) Others:  “reproducible science” x2  “impact to field”, “integrative collaborative science that is novel”  “support improved science code development processes” x2  “change and improve the software and research culture of the science community”  “transfer of skills and best practice”, “community education” “Rank the Goal of the project” 1—most important

8 “What metrics will your project collect?” CategoryMetrics41 responses Communities and Users Number and diversity of contributors/users12 Number of different application domains2 User feedback and experiences/surveys10 Number of end user issues/tickets, avg resolve time 3 Number of software elements integrated1 Performance statistics against benchmarks5 UsageSoftware usage (CPU hours)10 Amount of quality datasets accessible2 Number of software elements used in integrated ways 2 Number of visitors/visits2 Number of members on mailing list2 Exemplar: CyberGIS Software Integration for Sustained Geospatial Innovation, Shaowen Wang, PI

9 Metrics Continued CategoryMetrics SoftwareNumber and severity of bugs reported and resolved 3 Interactions with the open source community/current open source mechanisms employed/what is contributed back to open source community 2 New standards defined1 Number of downloads17 ScienceNumber of publications and their impact measures 8 Number of citations11

10 “What project infrastructure are you using?” TypeNameUsing Version controlSVN24 Git19 Mercurial3 Software hostingGithub5 (4 more moving) Bitbucket3 Google code2 Sourceforge1 Mailing listsMailman and others19 WebsitesCustom13 (probably and undercount) Wordpress4 41 responses

11 TypeNameUsing Bundled InfrastructureHubZero3 Apache1 Eclipse1 Project managementJira3 Build and testJenkins3 “Do you support outside contributions?”: 28 yes, 5 no Plus a bunch of other tools

12 Approaches to sustainability “no software is an island”  Integration with other projects  Being part of larger projects/gateways  Open access/ Open to contributions  Technology transfer  Diversification of users/communities / Outreach  Establishing foundations/be part of consortiums  Building ties with industry  Be written into others grants  Standardization  Software reuse  Automated software generation

13 “What services could a software institute offer that you would be most likely to use?  Workforce development –Training and cross training of CS and domain scientists –“Access to qualified and motivated human resources”  Mentoring –Fostering collaborations between SI2s and outside –Expert advice/Project mentoring –Financial support  Community Building –Meetings to discuss shared challenges and solutions  Sustainability –Sustaining codes –Software repository –Ranking of institute content

14  Best Practices/Standardization –Establishing protocols for interoperability –Tool recommendation –Technology forecasts  Services –Provide auditing services (security, privacy, automated testing) –Code reviews –Integration of toolkits –GUI/website development –Organizing data

15 Projects you look to as exemplars Apache.orgR projectUK Software Institute Astronomy/HEP projectsiRODSPETSc CondorProtein Data BankHubZero DropboxAmazonGoogle docs “Easy to use”“Community trusts it”“Impressive technology” “Great user services”“Active development group” “international consortium model” “Integrative services”“Sound underlying infrastructure” “diverse community” “Serves both large users and long tail users” “Great Community”“successful models for supporting sustainability” “devotion and responsiveness to users” “Broad appeal among govt, commercial & academia” “simplicity” “open API”“good documentation”“active vibrant community”

16 Projects you look to as exemplars Apache.orgR projectUK Software Institute Astronomy/HEP projectsiRODSPETSc HTCondorProtein Data BankHubZero DropboxAmazonGoogle docs “Easy to use”“Community trusts it”“Impressive technology” “Great user services”“Active development group” “international consortium model” “Integrative services”“Sound underlying infrastructure” “diverse community” “Serves both large users and long tail users” “Great Community”“successful models for supporting sustainability” “devotion and responsiveness to users” “Broad appeal among govt, commercial & academia” “simplicity” “open API”“good documentation”“active vibrant community”

17 Agenda Thursday January 17 th, :15-9:45 amNSF OCI PerspectiveAlan Blatecky 9:45-10:30 amKeynote 1: Accessible, transparent, reproducible analysis with Galaxy James Taylor 10:30-11:00amBreak 11:00am-12:30pmPanel 1: How to measure the impact of software? James Howison 12:30-1:30pmLunch 1:30pm-2:00pmCISE and Big DataSuzi Iacono 2:00-3:30pmPanel 2: What does it mean to Conceptualize?Nancy Wilkins-Diehr 3:30-4:00pmBreak 4:00-4:45pmTalk and discussion on Software SustainabilityNeil Chue Hong 4:45-5:15pmSI2 Program Current and FutureDan Katz 6:00pm-8:00pmReception and Poster sessionWestin Hotel, Fitzgerald C

18 Agenda Friday January 18 th, :00-8:30 amBreakfast 8:30-9:15 amKeynote 2: Software Ecosystems and ScienceJim Herbsleb 9:15-10:30amPanel 3: Managing a software project—the dos and don’ts Phil Papadopoulos 10:30-11:00amBreak 11:00-12:15pmGroup Discussion: Getting the scientists onboard! How do you make your software useful? Miron Livny 12:15-12:30pmConcluding remarksMiron Livny 12:30pmBox Lunch Administrative support: Larry Godinez