A DΙgital Library Infrastructure on Grid EΝabled Technology Release BETA Release BETA Build Plan Geneva - 1 March 2007 Paolo Fabriani

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Summary  Release Beta Numbers  Analysis of component dependencies  Integration configurations: a proposal  Expected times for build cycles  Build scheduling and execution  ETICS usage & infrastructure needs  Naming conventions update

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Release BETA Numbers  From ETICS  20 Subsystems  More than 200 Components  Services, Stubs, API, Service Archives  From Savane  158 Components  75 Services  49 Libraries  34 Portlets

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Release BETA Numbers CategoryServicesLibrariesPortletsTotal DIS44-8 DVOS45211 Keeper42-6 BMM13-4 VDL3-811 CMS56112 MMS3126 Content Security1--1 Annotation11711 Search CSDS/DF31-4 Index Personalisation3-11 Feature Extraction-617 PDS2518 PRS21-3 PES3-14 POS24-6 Application Layer21811 General-1-1 Total

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March DVOS DIS BMMKeeperVDL Dependency Analysis CM(2) MM(3) Annotation Cont Sec Feat Ext PRes (2) POpt(3) PExec(3) PDes (4)gLiteDIR(4)Search (4) Index (4) Personalisation (5)Application Layer (5)

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March DVOS DIS BMMKeeperVDL Dependency Analysis CM(2) MM(3) Annotation Cont Sec Feat Ext DIR(4)Search (4) Index (4) PRes (2) POpt(3) PExec(3) PDes (4)gLite Personalisation (5)Application Layer (5)

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March DVOS DIS BMMKeeperVDL Dependency Analysis CM(2) MM(3) Annotation Cont Sec Feat Ext DIR(4)Search (4) Index (4) PRes (2) POpt(3) PExec(3) PDes (4)gLite Personalisation (5)Application Layer (5)

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Dependency Analysis  20 Subsystems  1 without declared dependencies (independent)  i.e. gLite  Can be potentially built at any time  8 without dependants (non-critical)  i.e. POpt, MM, PDesign, Personalisation, Appl Layer, Annotation, Feature-extraction, Content-security  Have deps but no comp relies on them (can be post-poned if unavail.)  11 with dependants (critical)  5 from CL (DVOS, VDL, DIS, Keeper, BMM)  Process Resources  Process Execution  Content Management  3 from I&SM (Search, Index, DIR)  Need to be built in the above order

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Dependency Analysis Collective Layer (VDL + DIS + DVOS + Keeper + BMM) Proc ResCM MM Proc Opt Proc Exec Proc DesignSearch + Index + DIR PersonalisationApplication Layer Annot Cont Sec Feat Ext gLite

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Integration Configurations A Proposal Integration Configuration Critical ComponentsNon-Critical Components enabled by the previous build #1Collective LayergLIte #2Process ResourcesFeature Extraction, Content Security, Annotation #3Content ManagementProcess Optimisation #4Process ExecutionMetadata Management #5Search + Index + DIRProcess Design #6-Personalisation, Application Layer

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Expected Time for Build Executions  Build execution times:  Whole DILIGENT project: approximately 4 hours  Individual subsystems: 10 mins to 1 hour  Most time spent in checkout (Build Server: CPU 2 x Xeon 2.8GHz, 1GB ram)  Build errors require:  Action from developers: bug fixing and possibly local build  Experience with Alpha (and Alex) tells:  Average fixing time for build bugs (e.g. bad commands and/or dependencies in ETICS, missing libraries and classes, etc.) in a build execution was half day  Difficult bugs (e.g. vcsPrefix mis-usage) took nearly two weeks to solve

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Release Candidate Beta Exptected Times for Build Cycles AlphaRC Beta From 1st integration build to last deployTest 8 weeks6 weeks (19/3 to 31/4) Number of components89158 Number of integration configurations 46 Average nr of new components per integration configuration 2226 Average days from Build to DeployTest 4 days Number of Build-DeployTest cycles 9 (6 days/cycle) 6 to ? (depends on bugs found during DeployTest)

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Considerations  Consistent risk of missing a complete RC Beta for 31/4  Some components scheduled for  15 April (4 comps: VDL & CMS)  31 April (3 comps: CMS & PES)  31 May (1 comp: Search)  Unlikely to have 100% successful “build+deployTest” in one week for each integration build  Risk that some developments won’t complete by 31/3 even if scheduled  April TCOM  Possible solutions  Postpone deadline for a complete RC Beta  Remove components from Beta release

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Release Beta Exptected Times for Build Cycles  Release Beta process is from 1/5 to 30/6 (2 months)  How long to fix bugs discovered in testing and produce a new integration build? Depends on:  Number and Severity of bugs (more difficult than build errors  )  Reactivity of developers (good, so far )  Development still on-going (most likely to happen  )  Outcome of deployment testing (not expected to degrade )  Not less than 10 days = No more than 6 build/test cycles  But we must also consider time for testing  Possible solution  Start testing earlier than 1/5 on available components

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March ETICS Usage & Infrastructure Needs  Foreseen usage of ETICS:  Dynamic/Static dependencies  “Module renaming” issue not yet solved in current version. We’ll keep using static dependencies  Dependency scoping adoption (build/runtime)  Saves time when building and testing  Infrastructure Needs:  No further infrastructure requirements

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Build Scheduling and Execution  Alpha  All 0_1_X integration builds have been stopped  HEAD  Project and Subsystem builds are scheduled nightly  Beta  Subsystem integration builds will be scheduled for build as soon as available  Project integration builds will be triggered manually

Digital Libraries Powered by the Grid Release BETA Meeting - Geneva, 1-2 March Configuration Naming  Alpha Release  Project-level configurations renamed to  org_diligentproject_0_1_1 ...  org_diligentproject_0_1_4  Beta Release  Module configurations will be  >_0_2_X  Project-level configurations will be  org_diligentproject_0_2_1  org_diligentproject_0_2_2 ...