Commercialisation at NeSC An operational perspective Dr Rob Baxter Software Development Group Manager NeSC Review, 30/09/2003

Overview NeSC project strategy The GCPs and edikt Some highlights of the year How do we do it? Where are we going?

NeSC project strategy The Grid must be built on standards cf. the Internet, the World Wide Web Core middleware is fundamental cf. TCP sockets libraries, http servers Our strategy has been identify e-Science application needs develop standards and core middleware build applications, leveraging middleware This has proved very successful for e-Science and provides a solid foundation for e-Business

Software projects at NeSC NeSC has two flavours of project development - the GCP projects research and development - the edikt programme GCP projects funded by £3.6m EPSRC/DTI (+ equal in-kind) allow focused development on key e-Science problems demand direct engagement with business edikt funded by £2.3m SHEFC RDG allows exploration of new ideas, some good, some bad

NeSC project portfolio GCP: SunDCG - m/w OGSA-DAI - m/w GridWeaver - fabric MS.NETGrid - m/w FirstDIG - app PGPGrid - app BRIDGES - app ODD-Genes - demo edikt: BinX - m/w Eldas - m/w AstroBinX - app Osage - app [chronological starts]

Collaborators roll-call In the last two years we’ve worked with IBM, Sun Microsystems, HP Oracle, Microsoft First plc, Pepper’s Ghost Productions ltd, Raytheon Company inc and AstroGrid, myGrid ESNW, NEReSC, LeSC, GSC Scottish Centre for Genomic Technology & Informatics, MRC Human Genetics Unit

Some highlights OGSA-DAI £1.8m, NeSC, EPCC, ESNW, NEReSC, IBM, Oracle core data access middleware based on GGF standards  OGSA-DAI team helping write the specifications in use in e-Science projects  AstroGrid, myGrid, FirstDIG, BioSimGrid, BioGrid (Jp) 1000 downloads reached last week follow-on project planned (DAIT, £1.5m) gained EPCC/NeSC entry to the Globus Alliance  only UK site alongside Argonne, ISI and PDC

Some highlights cont. MS.NETGrid demonstration and training of OGSI on.NET full house at AHM 2003!  great interest at the NeSC booth too delivered course & demos eSI invited by MS to lecture on Grid/.NET, Budapest, Oct FirstDIG EPCC, FirstBus South Yorkshire application of OGSA-DAI and related data mining  to draw together real, disparate, dirty data sources to add real value “the results of this exercise will revolutionise the way we do things in the bus industry”  Darren Unwin, Divisional Computing Manager, FirstBus South Yorkshire

Some highlights cont. ODD-Genes EPCC, GTI, MRC HGU a demonstration of OGSA-DAI and SunDCG in action the highlight of the NeSC booth at AHM 2003! “This project has demonstrated how Grid technologies can be used to enable true e-Science - discoveries that would not otherwise have been achieved without this infrastructure in place”  Professor Peter Ghazal, Director, GTI

How do we do it? The NeSC/EPCC SDG thinks like a company we do fixed-priced contracts to deliver software X against customer requirements must plan and execute properly or we get cost overrun but we also operate on the technology bleeding edge…  …so we multiply all our task estimates by  2.3 Software Development Group: 30 staff SDG Manager 3 Project Managers, 2 Architects 5 Principal Consultants/Team Leaders 19 software developers Well-defined project lifecycle selection, planning, execution, delivery and review

Project selection Two flavours, two methods GCP Projects Commercial Group prospect based around CommStrat proposals developed with clients and SDG project staff “project licence” agrees handover from CG to SDG edikt edikt management define proposals based on work with our e-Science stakeholders project proposals reviewed by Advisory Board

Project planning Older projects used standard development plan Newer projects use more comprehensive project management plan a quality plan template based on NASA SEL and other sources (CMU SEI, Microsoft, Oracle) covers all aspects of the project  development plan  risk and issue monitoring  project tracking metrics  documentation and coding standards  QC and QA

Project planning cont. Leading edge development needs flexibility We use phased development models staged delivery  good where scope is ill-defined and we expect requirements to change/evolve, e.g. OGSA-DAI, SunDCG  define scope in quarterly blocks (say) design to schedule  good where scope is well-understood and initial requirements are well defined, e.g. MS.NETGrid, FirstDIG  work against prioritised requirements until effort runs out evolutionary prototyping  good for demos or exploring requirements, e.g. ODD-Genes

Project execution All project teams have project leader developers technical reviewers Bigger projects will have an architect… e.g. OGSA-DAI, edikt projects …and/or a Review Board e.g. OGSA-DAI, SunDCG, edikt Weekly team meetings are expected action lists, task breakdowns, risk & issue lists

Project execution cont. Based on standard document templates requirements, component design, test… Use standard tools e.g. Rational Rose, TogetherSoft, JUnit, ant Test early, test often aim for high unit test coverage have developed automated test setups for distributed middleware over course of GCP projects Monitor overruns, apply triage review plans regularly, revise if required

Delivery and review All projects postmortemed what was good? what was bad? what can we learn? For customer-led projects, delivery is easy site visit, installation, training limited period of free support longer term support contracts always offered Middleware is a bit different monitor web downloads set up web-based bug reporting disseminate, disseminate, disseminate!

Where is this taking us? Solutions for e-Science today… …are solutions for e-Business tomorrow universal data access and integration universal data format description Web and Grid services  WSDL, SOAP, HTTP, Java,.NET distributed resource management distributed, heterogeneous systems integration Our projects reflect our vision for the commercial Grid

In summary NeSC’s software development programme has been - and continues to be - a great success We leverage off existing strengths at EPCC the institute that thinks it’s a company Professional planning and execution is key backed by an unrivalled CS research base International recognition from a standing start to the Globus Alliance in 2 years SunDCG software already distributed by Sun Real business impact “this will revolutionise the way we do things” Darren Unwin, FirstBus South Yorkshire