Virtual Lab Overview 5/21/2015 xxxxxxxxxx NWS/MDL/CIRA

Virtual Lab (VLab) Defined o Goals o Components Development in the VLab AWIPS Development in the VLab o Process for potential AWIPS II project o Accessing the latest AWIPS II baseline o Pushing code to the baseline Agenda

The VLab provides an environment where collaboration and innovation among geographically diverse scientists and developers can thrive. Virtual Lab

Enable the NWS and Partners to: Reduce the time and cost of transitions of NWS field innovations to enterprise operations, Minimize redundancy and leverage complementary, yet physically separated, skill sets, Forge scientific and technical solutions based on a broad, diverse consensus, and Promote a culture based on collaboration and trust VLab Goals

The VLab is comprised of two main components: 1.Virtual Lab Collaboration Services (VLCS) 2.Virtual Lab Development Services (VLDS) VLab Components

Virtual Lab Collaboration Services (VLCS) Redmine (Project Mgmt/Issue tracking) Git Virtual Lab High Level View Pre-existing projects (e.g.Local Application) Subversion Trac - Requests for VLab development projects are made within the VLCS - VLCS supports communities (group of users with their own space to collaborate via a private wiki, forum, document library, blog, and calendar) - VLCS supports science sharing and developer training - VLCS will contain a searchable/sortable listing of all projects - VLCS will allow developers to register their areas of expertise and interests to provide a searchable DB of developer resources Project entries within the VLCS can point to pre-existing projects located at external web sites. Project entries may point to a Redmine project web site. Jenkins Continuous Integration Gerrit (Code Review) Use VM's for build environments Virtual Lab Development Services (VLDS) SVN

Provides services and a framework supporting development to NWS developers and partners. Services include o Project management o Issue tracking o Revision control (git and subversion) o Code review o Continuous integration VLab Development Services

Easy to use framework Offers different levels of service on a project by project basis o Basic - project management, issue tracking, and revision control o Moderate - Basic with code review o Advanced - Moderate with continuous integration Accessible o Access from any subnet via https (no VPN required) VLab Development Services

Development code resides in the VLab in Git or subversion repositories Development issues are tracked in the VLab Development code is checked in through the VLab Development code is related to issues through the VLab VLab Development High Level Process

Development code is reviewed (only Git repositories) through the VLab Development code is verified/built in the VLab Integration testing is completed outside of the VLab VLab Development High Level Process (cont.)

Project management and issue tracking - Redmine Revision Control – Git and Subversion Code Review - Gerrit Continuous Integration - Jenkins VLab Services Details

VLab Development Services Architecture Git code repository/ versioning and integration Jenkins Continuous Integration Redmine Change tracking Continuous Integration builds Queries and Reports to change database Gerrit Gatekeeper & review Commit Developer (Currently must have NOAA LDAP account) Based on RTS slide Clone/fetch repos via smart https using Redmine access control Push changes for review Clone/fetch repo via Gerrit ssh using Gerrit access control

Web site that supports project management o Issue tracking o WIKI o Browse project repository User logs in with their NOAA LDAP credentials Administrator or project manager assigns members to projects with appropriate roles Development in VLab Redmine

Basic projects o Access/Commit to project's Git repository via Redmine  Smart https o Roles assigned in Redmine control "Commit Access" to Git Development in VLab Revision Control

Moderate/Advanced projects o Read access to Git repo via Redmine o Read/Write access via Gerrit Code review and verif ication rules set up on a project by project basis  Control is defined down to the branch level Help is available through VLab WIKI Development in VLab Revision Control

Code changes are pushed to Gerrit for review If configured, reviewers are notified of review request Reviewers review code in Gerrit Approved and reviewed code is merged into the specified branch in the project’s VLab repository Development in VLab Gerrit

Provides a web based continuous integration tool o Jenkins can be used to automate the verification and testing of checked in code:  Does code build?  Do unit tests execute properly?  Do custom checks verify (e.g., is there an associated Redmine ticket assigned to the developer?) o Jenkins can also be used to schedule builds based on a cron pattern Development in VLab Jenkins

Project Repository AWIPS2_Core _FOSS Repository Project Repository Virtual Lab Repositories Virtual Lab AWIPS II Development Clones of VLab repos on developer's machine AWIPS2 Repositories Eclipse AWIPS2_FOSS Repository AWIPS2_Core Repository Development Baseline Repository

AWIPS Development Project States: Under development Approved for integration into the operational baseline AWIPS II Development in VLab

Development baseline o A Git repository that contains code currently under development that is slated for the operational baseline as well as release branches Operational baseline o The Raytheon managed O&M code that is running in the field Integration issue o A ticket that gives a project permission to push the project's code to the development baseline AWIPS II Development in VLab Vocabulary

1.Project owner signs into the VLCS. 2.Project owner requests a VLab Development Project via the VLab Development Project Request Form. 3.VLab Support Team (VLST) is notified of the new request via . AWIPS II Development in VLab Process for Potential AWIPS II Project

4.VLST member logs into VLCS and reviews the request via workflow. a.If the request is a clear fit, the VLab development project is created (Redmine, Jenkins, Git, Gerrit), and the request is approved. b.If the request needs clarification or further review, the VLST as a whole reviews the request. AWIPS II Development in VLab Process for Potential AWIPS II Project (cont.)

5.Project owner is notified upon approval automatically via VLab workflow. 6.The project owner adds users to the Redmine project's team -- assigning them to a role (e.g., Developer, Reporter, Manager). 7.Project has its own Redmine project for issue tracking and Git repository for project specific code. 8.Project team uses Git, Redmine, Gerrit, and Jenkins to work on the project. Make sure awips management knows about project and get A2 experts involved early in design and code reviews. 9.Members of project have read access to the latest AWIPS development baseline repositories. AWIPS II Development in VLab Process for Potential AWIPS II Project (cont.)

10.If a project is approved for integration into the AWIPS II baseline a DCS is created in the Raytheon O&M Redmine. Project owner creates an integration issue within their Redmine project referencing the DCS #. 11.The integration issue must be assigned to the project developer or owner, who then pushes the changes to a dev_ branch in their repository via Gerrit for review, noting the integration ticket number within the commit message. 12.Jenkins verifies that the checked in change is associated with a valid integration ticket that is assigned to the submitter. AWIPS II Development in VLab Process for Potential AWIPS II Project (cont.)

13.Developers iterate through proposed changes. 14.AWIPS-Approvers approve or reject the change based on feedback from the reviewers. 15.If approved, code is merged by the Integrator into the dev_ branch. 16.Nightly build of each dev organization’s release branch can be setup. 17.Testing of the nightly build can be done by the development organization on their own test beds. 18.Your dev organization’s build coordinator requests that changes in your branch be pulled by the Raytheon O&M group in Silver Spring. AWIPS II Development in VLab Process for Potential AWIPS II Project (cont.)

19.Raytheon O&M (Silver Spring) will then pull the changes from the Vlab into their integration branch Code will be built and smoke tested Assuming the build and smoke test succeed the changes will be pushed to the master release branch and imported into Dimensions 20.Raytheon O&M will push the master release branch to the Vlab’s AWIPS2_Dev_Baseline repositories master release branch (e.g., master_14.3.1) 21.Development organizations are responsible for keeping their release branches up-to-date. AWIPS II Development in VLab Process for Potential AWIPS II Project (cont.)

VLab provides the tools you need to collaborate and develop projects using best practices from virtually anywhere! Summary

Questions?