1. Connect. Communicate. Collaborate 1 QUATIC 2007 Lisbon New University (Portugal), September 12-14, 2007 Quality Assurance in perfSONAR Release Management Jeff W. Boote, Andreas Hanemann, Loukik Kudarimoti, Panagiotis Louridas, Luís Marta*, Michalis Michael, Nicholas Simar, Ilias Tsompanidis * Foundation for National Scientific Computing (FCCN) Av. Do Brasil 101, Lisbon lmarta@fccn.pt

2. Connect. Communicate. Collaborate 2 Presentation outline Introduction –Challenges of Quality Assurance in SOA projects PerfSONAR system –Requirements of the project PerfSONAR Release Management process –Related work –Micro Releases and Bundle Releases –Testing methods –Visualization tools Conclusion and Future Work

3. Connect. Communicate. Collaborate 3 Introduction: Challenges of SOA projects Development of software within organizations: –Established best practices exist; Development of Service-Oriented Architectures: –New quality assurance challenges arise; –Set of loosely-coupled systems; –Potential involvement of different organizations; –Testing parts individually is no longer sufficient; –Collaboration issues are very important! How did we address these challenges in perfSONAR?

4. Connect. Communicate. Collaborate 4 perfSONAR system: Introduction perfSONAR (Performance focused Service Oriented Network monitoring ARchitecture) system –Is a joint effort of EU-funded IST project GN2-JRA1, Internet2, and ESnet –The solution is deployed and further elaborated in European Research Backbone Géant Connected European National Research and Education Networks (NRENs) Internet2’ s Abilene network ESnet (Energy Sciences network in US) RNP (Brazilian NREN) –Open source development also for other interested networks –Name reflects the choice of Service Oriented Architecture

5. Connect. Communicate. Collaborate 5 perfSONAR system: Measurement framework User Interface Layer perfSONAR Web Services Layer Local Measurement Point and Database Layer User interface 1User interface 2 Domain A - servicesDomain B - servicesDomain C - services Metric 1 Measurement Point Metric 2 Measurement Point Type 1 Metric 2 Measurement Point Type 2 Domain ADomain BDomain C

6. Connect. Communicate. Collaborate 6 perfSONAR system: Objectives Deliver a framework for network performance measurements in research management networks. A set of simple web services representing functionalities of network monitoring: –Active and passive network monitoring; –Monitoring data storage –Alarms –Authentication and Authorization Use of NMWG (v2) protocol for communication: –SOAP XML messages –Open Grid Forum (OGF) Network Measurement Working Group (NMWG)

7. Connect. Communicate. Collaborate 7 perfSONAR system: Requirements General aim of Release Management: –Provide services for installation with an assured quality; Particular aims for perfSONAR Release Management: –Release bundling (service bundles) Services are released together to assure interoperability; –Integration testing Individual testing of services enhanced with collaboration testing;

8. Connect. Communicate. Collaborate 8 perfSONAR system: Requirements Particular aims for perfSONAR Release Management: –Documentation requirements For each service, a set of documents have to be provided –Visualization tools releases Measurements have limited usefulness without visualization tools –MDM pilot requirements A rollout of services is going to be fixed as part of the project contract

9. Connect. Communicate. Collaborate 9 Release Management Process – Related Work Model selected as basis for perfSONAR: –Release Management process for FreeBSD OS Anonymous CVS access to the general public; Approximately 300 with write access (commiters) Direction of the project provided by an elected core- team of very senior developers; Code base split into Current, Stable and Release branches; For each release, 3 time periods of 15 days applied to check the code;

10. Connect. Communicate. Collaborate 10 Micro Releases and Bundle Releases Micro Releases: –Services tested in an individual basis –Service is fully tested – internally and functionally: –Not tested for collaboration with other services; –Aimed at early adopters; Bundle Releases: –Set of requirements to combine individually released services into perfSONAR Bundle Releases: Hand Over Process;

11. Connect. Communicate. Collaborate 11 Micro Releases and Bundle Releases - Summary Hand Over Process: Different services from different development teams; Release Management done per-service; Ensure services meet expected quality levels; Service X Dev. team 1 Service Y Dev. team 2 Service Z Dev. team 3 perfSONAR bundle release Hand Over Process Micro - Release X Micro - Release Y Micro - Release Z

12. Connect. Communicate. Collaborate 12 Micro Releases – Code Development Phases Three code development branches are distinguished: CURRENT branch: –Contains the newest versions of the services being developed; STABLE branch: –Tested versions of the services –Basis for the creation of the service RELEASE branch: –Contains the code which is being ‘handed over’; –Obligatory naming convention for the ‘per Service Release Branch’: - -Release- Example: RRDType-MA-Release-05062006

13. Connect. Communicate. Collaborate 13 Micro Releases – Code Development Phases 45 days before the release date, an email is sent to the development lists announcing 15 days to code freeze. Meanwhile, developers perform MCF Sweeps: –MCF is Merge From Current – merge a tested change from development to STABLE branch. 30 days before release, code repository enters a code slush: –For 15 days, only bug or security fixes may enter the STABLE branch, with approval of the Release Engineering Team.

14. Connect. Communicate. Collaborate 14 Micro Releases - Code Development Phases 15 days before deadline, the RELEASE branch is created and a release candidate is released for widespread testing. Code enters a new code freeze where new changes are much harder to justify. Release Candidates are tested by the community, until the final release is ready. After approval by the Release Team and bumping up the version number, the Micro Release of the service is announced.

15. Connect. Communicate. Collaborate 15 Micro Releases - Code Development Phases Code Development phases in perfSONAR:

16. Connect. Communicate. Collaborate 16 Bundle Releases – Hand Over Process Phases to ensure the collaboration of the web services: –Selecting services that go into the Bundle: Usually, a mix of updated services and new services with Micro Releases ready; Pre-requisite: documentation complete – 5 documents; –Testing services: Test collaboration among services; Test joint installation procedures; Detect problems related to software dependencies among services;

17. Connect. Communicate. Collaborate 17 Bundle Releases – Hand Over Process Documents Specify a list of documents for developers to provide and give them templates and samples: –Functional Specification of Service describes functionalities supported by the service (success and failures factors); –Interface Specification: For each functionality describes possible request and response messages;

18. Connect. Communicate. Collaborate 18 Bundle Releases – Hand Over Process Documents –Installation Actions Specification: Explanation of installation action for all types of installations; –Configuration Files Description: Describes configuration files that are required for running the service; –Metadata Configuration File Description: –Overview of the metadata configuration file, its building blocks;

19. Connect. Communicate. Collaborate 19 Testing Methods White box testing: –JUnit tests are used – written in parallel with code development –Performed by code developers; Black box testing –Implemented XML interactions are tested; –Tests with: Allowed values, border values, not allowed values; Check wrong syntax in XML query generation;

20. Connect. Communicate. Collaborate 20 Release of Visualization Tools Release process runs in parallel with development of services; STABLE branches created for services – visualization tools collaboration development starts; Test collaboration with new features of services; –Manual testing using GUIs of tools; JUnit for testing the internals;

21. Connect. Communicate. Collaborate 21 Conclusions Release management process in perfSONAR: –Project is SOA Additional considerations taken to ensure reliable collaboration of developed services. –We hope our considerations… Micro and Bundle releases; Necessary documentation; Testing methods; –…are valuable input for similar projects.

22. Connect. Communicate. Collaborate 22 Future Work Developers would like to spend less time writing documentation / Testing team wants more detailed documentation; Create Release Specification document –Written before development starts, by Release Team and Development Team Leader; –Specify what is expected from the developers in the next release, according to release strategy; Improve Bug reporting using Bugzilla; A pilot installation of perfSONAR services is taking place;

23. Connect. Communicate. Collaborate 23 Thank you for your attention! Any questions? http://www.perfsonar.net