e-Science Experiences: Software Engineering Practice and the EU DataGrid Lee Momtahan and Andrew Martin Oxford University Software Engineering Centre

Contents EU-DataGrid Challenges Comparisons Conclusions

EU-DataGrid 9.8m Euro project over 3 years; 21 partners in 15 countries; application in particle physics (and bioinformatics, and earth sciences); PetaBytes of data: datasets to be catalogued, replicated where necessary; seamless delivery of computing resources 200 staff, meeting infrequently (60 FTE)

Project Goals build application frameworks potentially involving huge amounts of data, compute power and distribution provide secure, managed, uniform access to such resource facilitate collaboration, and remote access to data and scientific instruments manage such facilities as a persistent service

Work Package Structure

Our role becoming involved after project started funded to bring computer science/software engineering experience to the project intending to help by modelling aspects of design in order that the system may be better understood, designed, built, documented in passing made the observations documented here interested in the generality of these issues for e-Science

Challenges Requirements Volatility –Novel paradigm; New diversity; Volatile off-the-shelf components Geographical Separation –communication can easily become a limiting factor (Brookes); Physicists are used to collaborating in experiments –but software? System Decomposition –Political concerns; geographic determination

Challenges Project Processes and Authority –there is a quality plan… how do you get people to follow it? is a commercially-based process appropriate? what about traditional academic means of QA?

Challenges Planning and tracking –exit criteria for an iteration seems to be the completion of a document detailing the problems found in testing

Comparisons academic software production vs. commercial software production academic software production vs. other academic activity CMM Level for Software? For Paper/Proposal Writing? open source software vs. open source development open source models vs. publicly-funded research publication in journals vs. publication to a repository

Conclusions Practice of software engineering in an e- Science context is substantially different to industrial practice Industrial models do not seem appropriate Open source models seem to fit better Publication and review are the key to quality and process improvement