1. CCSM Software Engineering Coordination Plan Tony Craig SEWG Meeting Feb 14-15, 2002 NCAR

2. Outline Background New (and improved) policies and procedures –Change Review Boards –Repository Access –Testing –Status Accounting –Documents –Planning and Prioritization –Management –Other Issues Roles Summary

3. Acronyms CCSM - Community Climate System Model CGD - An NCAR division SSC - CCSM Scientific Steering Committee SECP - Software Engineering Coordination Plan CSEG - CCSM Software Engineering Group ESMF - Earth System Model Framework, a NASA funded project SCIDAC - Scientific Discovery through Advanced Computing, a DOE funded project CVS - A source code control tool CRB - Change Review Board

4. Background CCSM started as an NCAR centric project with a handful of developers and scientists. Little “management” was required. There has been rapid growth in the number of developers, number of collaborators, and importance of the work. CCSM coordination procedures are behind considering the growth and size of the project. Now CCSM needs to catch up.

5. Overall Goals Improve quality. Improve coordination, communication, and control. Improve productivity. Continue to allow “individual” development. Recognize that CCSM is a community project. Minimize overhead and burden or “process”.

6. New Policies and Procedures Change Review Boards Repository Access Testing Status Accounting Documents Planning and Prioritization Management Other Issues

7. New Policies and Procedures, the picture Developer REPO Testing Change Review Board Releases REPO Tools Status Accounting CVS Testing Scripts BUG Reporting And Tracking Experiment Database GUI Documents, WEB

8. Change Review Boards (CRB) Goals –Improve communication and coordination of both scientific and software tasks. –Better control of production versions. –Better prioritization of development tasks. –Improve quality by implementing reviews. –Formally distinguish tasks that are under development and tasks that are ready for production.

9. Change Review Boards How will they operate –CRB for each component and full CCSM. –Developer would submit change request. –CRB (approx 4 people) is responsible for prioritizing model changes. –CRB asks community for input on change request. –Reviewed for “improvement”, performance, testing status, coding quality, and documentation. –CRB accepts, rejects, defers, or returns request.

10. Change Review Boards Implementation status –Developing procedures for atmosphere CRB, meetings to start soon. –Developing procedures for CCSM CRB, meetings to start soon. –Evaluating need for land, ocean, and ice component CRBs. –Need to choose/develop status accounting tool. Developing requirements. –Have implemented changes in repository policy to create consistency between repository and CRB.

11. Repository Access Goals –Open CCSM repository to more developers. –Formalize use policy and procedures. –Improve coordination and control of development efforts. –Clearly separate development tasks in the repository from production versions of the models. –Have all development occur on branches. –Place some limitations on individual access, and have an ability to enforce limitations.

12. Repository Access How will this work –Continue to use CVS. –Developers formally request access to repository. Access granted on a limited basis, read/write permissions specified. –All development will take place on CVS branches. Gatekeepers manage main trunk. –Developers provide regular progress updates to CRBs or working groups. –CCSM will have tools to monitor usage. Managers will retract access privileges if necessary.

13. Repository Access Implementation Status –Repository access policy exists. –Repository access request web page exists. –Need to review current developers’ status. –Need to clarify the review process for granting access. –Need to improve tools to monitor repository usage. –Need to develop procedure for coordination of development, including testing requirements.

14. Testing Goals –Improve testing capabilities, ease of testing. –Provide developers with model requirements, test requirements, test scripts. –Develop an overall CCSM testing strategy to include nightly build and smoke tests, unit tests, regression testing, port validation capabilities, and error growth validation. –Discuss where scientific validation fits into overall testing strategy.

15. Testing Implementation Status –Atm model has some test scripts, being used by the development community. –CCSM coupled model has some test scripts. –Simple GUI under development, used in-house for configuring and testing the coupled model. –Need to develop model requirements. –Hiring a new person to focus full-time on testing issues (SCIDAC funds). –Need to develop test strategy and hierarchy of scripts for components and fully coupled models.

16. Status Accounting Goals –Track change requests and project priorities. –Improve bug reporting and tracking capability. –Create an experiment/dataset database. –Improve communication of status of model releases and development versions. –Link many aspects of development; CRB, repository, bugs, testing, web, and experiment database.

17. Status Accounting Implementation Status –ccsm-bugs request/tracking tool exists. –Implementing ccsm-logs experiment tracking log. –Need to develop “tool” for tracking change requests for components and CCSM. –Need tools to monitor repository status. –Hiring a new software engineer to help us develop our tools and infrastructure (SCIDAC funds).

18. Documents Goals –Improve communication and coordination. –Reduce the amount of confusion and rework.

19. Documents Implementation Status –Many documents exists, not kept up to date. –Need to develop strategy for sharing, using, and updating documents; version control. –Need to implement/improve documents for production (users guides) development (charters, requirements, design, and reference documents; testing procedures, CVS help) management (policies, procedures, plans)

20. Planning and Prioritization Goals –Improve communication. –Clarify priorities.

21. Planning and Prioritization Implementation Status –Limited number of planning documents exist. –Developing requirements documents and task lists. –Change review boards and associated accounting tools will be used to communicate some short/medium term priorities. –Need to develop long term planning process to regularly review issues like base code rewrites, hardware issues, and the status of infrastructure support.

22. Management Goals –Clarify organizational structure. –Put in place procedures that improve coordination and communication without burdening the project. –Need to strengthen decision capabilities.

23. Management Implementation Status –Some management in place; CCSM manager (Kiehl), CCSM liason (Merilees), and CSEG manager (Craig). Role of CSEG becoming better defined. –Writing charters for components and CCSM. –Need to clarify relationship between working groups, CRBs, CSEG, SSC, and CGD sections.

24. Other Issues Data Management. Code Review. Library/Utility implementation plan. Coordination with SCIDAC and ESMF. –Providing funds for three (3) new Software Engineers in CSEG. Computer scientist in CCSM/CSEG. Training.

25. Roles CSEG to take a leading role in developing the infrastructure tools and coordinating the day-to-day process. Component working groups to help manage repository access, change review, development, planning, and documentation of component models. SEWG will continue to oversee the software engineering process and make recommendations.

26. Summary New policies and procedures –Change Review Boards –Repository Access –Testing –Status Accounting –Documents –Planning and Prioritization –Management –Data, Utilities, Training

27. New Policies and Procedures, the picture Developer REPO Testing Change Review Board Releases REPO Tools Status Accounting CVS Testing Scripts BUG Reporting And Tracking Experiment Database GUI Documents, WEB 50% 10% 40% 10% 80% 0%

28. On The WEB www.ccsm.ucar.edu - CCSM web pagewww.ccsm.ucar.edu www.cgd.ucar.edu/~csm/ - CCSM Developers web page (csm/rio)www.cgd.ucar.edu/~csm/ –SECP page –Repository access policy –Repository access request page –Bug reporting and tracking –More! THE END

29. CSEG Responsibilities Component development, performance.. Component model gatekeepers; may have to help coordinate testing, development, change requests, repository status, etc. Testing; one (1) full time test engineer for CCSM plus everyone helps improve test scripts and automation. Infrastructure support and tools; one (1) full time tools support person for CCSM plus others in CSEG as needed.