Process changes: Internal processes of CASA, external contributions, release schedule Mark G. Rawlings, CASA Build & Test Lead NRAO, Charlottesville Acknowledgements: Ville Suoranta, Darrell Schiebel & Jeff Kern

CASA Main Topics Dealing With Change - Version Control Systems – Centralized vs. Distributed: Subversion (SVN) vs. Git – Proposed Implementation What’s In The Box? - Review of CASA package types – Test, Stable, Monthly – Pre-Release, Release How And When? - Continuous Integration vs. Continuous Delivery – Questions & Implementation Challenges 2015 CASA Users Committee Face-to-Face2

CASA “Change is inevitable - except from a vending machine.” -- Robert C. Gallagher 2015 CASA Users Committee Face-to-Face3

CASA Dealing With Change: Version Control (a.k.a. Revision Control, Source Control) Systems System that enables multiple people to simultaneously work on single project – Each contributor edits own copy, chooses when to share changes – Enables one person to use multiple computers to work on project – Integrates work done simultaneously by different contributors. In cases when two contributors make conflicting edits to same part of file, human assistance for resolution usually required – Provides access to historical versions of project Allows specific changes to be undone, version roll-back, reproduction & understanding of bug reports on prior versions Logs allow determination of when, why, by whom each change was made 2015 CASA Users Committee Face-to-Face4

CASA Dealing With Change: Version Control (a.k.a. Revision Control, Source Control) Systems Can be centralized (e.g. Subversion, CVS) or distributed (e.g. Git, Mercurial): 2015 CASA Users Committee Face-to-Face5

CASA CASA SVN, Branches & Version Numbering 2015 CASA Users Committee Face-to-Face6 CASA currently uses Subversion (SVN)

CASA Version Control: Branching & Merging Branching: Duplication of object under version control (e.g. source code file or directory tree) so modifications can happen in parallel along both branches – CASA has development (“trunk”) & (pre-)release branches Merging: Reconciliation of multiple changes made to version- controlled collection of files. When two branches merge, one set of files is produced containing both sets of changes – Centralized VCS (e.g. SVN) can make this process painful for large numbers of contributors/changes! – Currently minimized for CASA by requiring developers to commit to both branches (when in pre-release phase) 2015 CASA Users Committee Face-to-Face7

CASA Pros & Cons of DVCS for CASA Pros – Makes developers more directly accountable for own commits – Allows more flexibility for collaboration & greater code maturity before general distribution – Avoids large merges (although we mostly avoid those now) – Allows CASA Development Team and GitHub Casacore developers to keep to unified casacore codebase – Distributed approach relies on distribution of multiple complete copies of codebase (multiple “backups”) – Makes adoption of external astronomical community contributions simpler Cons – More complicated conceptually – Versatile: More rope with which to hang ourselves CASA Users Committee Face-to-Face8

CASA CASA Package Production Branch: Trunk – Test: Nightly builds that start Stable: Any Test package that passes all automated unit & regression tests (~200; run nightly on all supported platforms) – Monthly: Snapshot of most recent Stable package at start of each month (also Pipeline snapshots) Branch: Release (Re-created ~6 weeks prior to public release) – Pre-release: Usually produced several times per week Release: (Manually) promoted Pre-release package All now routinely produced for RHEL5 & 6; two most recent OS X versions 2015 CASA Users Committee Face-to-Face9

CASA 2015 CASA Users Committee Face-to-Face10 Proposed Git-based Architecture

CASA Proposed Git Architecture: Key Features NRAO host Bitbucket Server-based organization-wide Git repository in-house – CASA developers act as “Gatekeepers” who approve/reject pull requests; can also push – Small subset with greater admin permissions (rebase, enable force push for limited periods, etc.) External ATNF ASAP SVN repository dependency to be deprecated – NAOJ libsakura replaces it, initially as 3 rd party package External GitHub casacore component to be incorporated as Git submodule External possible contributions from the external CASA science user community will be as pull requests to NRAO CASA Git repository – Requires external expert users to have (more limited) accounts 2015 CASA Users Committee Face-to-Face11

CASA Continuous Integration vs. Continuous Delivery Not the same thing! Continuous integration (CI): “The practice, in software engineering, of merging all developer working copies to a shared mainline several times a day.” – Handled for CASA by CI server. Currently Jenkins, but will probably move to Atlassian Bamboo Checks for changes to codebase every ~3 mins, attempts builds, reports results to committing developers Continuous Delivery: “A software engineering approach in which teams keep producing valuable software in short cycles and ensure that the software can be reliably released at any time.” – Not yet done by CASA, although stable/pre-release packages approach this in microcosm somewhat… 2015 CASA Users Committee Face-to-Face12

CASA “To CD or not CD: That is the question.” 2015 CASA Users Committee Face-to-Face13

CASA “To CD or not CD: That is the question.” Takeaway question: How desirable is Continuous Delivery to the CASA user base (community end users & supported observatories)? Implications: – New features would be available to end users more quickly – An end to current (nominal) six-monthly public release cycle – Many more (incremental) releases – Would require some re-engineering of package production workflow In extremis: each bug fix / new feature would need to be science user tested individually prior to integration. Would probably need to either: – Have B&T Group selectively integrate all changes for each incremental release; or… – Have separate testing branch for every bug fix / new feature 2015 CASA Users Committee Face-to-Face14

CASA Key Points (For End Users) CASA Group is planning to move to Git (with an enterprise Git server – probably self-hosting with Atlassian Bitbucket Server): – External users will be able to submit pull requests to CASA to contribute to the codebase – Ideally, only validated software gets delivered to users(?) – Could transition to continuous delivery model (or something closer to it), if desired(?) 2015 CASA Users Committee Face-to-Face15