1 ALMA CPM cominf (archive) Alisdair Manning 12/10/2015 1
The Plan AQ incremental release every release Easy performance improvements for RH DataPacker NGAS Pipeline Ingestion and meta-data extraction Relational Asdm (no delivery) DataPacker: re-assemble RelAsdm delivery Harvester APDM XML (no delivery) 2015 SourceCatalogue Adapt 12/10/20152
Priorities and Resources 3 meChristopheCADCHeiko 1.Schemas 2.DataTracker 3.ALMA schema 4.Harvester 5.SourceCat 6.ASDM Updates 7.NGAS 1.Pipeline Int. 2.RH - DtD 3.RH - OODT 4.Harvester 5.DataPacker 6.NGAS 7.Download Manager 1.AQ1.Monitoring 1.ALMA schema 2.NGAS 3.DB Docs 4.APDM 5.SourceCat 1.AQ1.Monitoring1.Pipeline Int. 2.RH - DtD 3.RH - OODT 4.Harvester 5.Download Manager 12/10/20153
Priorities Update I 1.ASA – close loop on deliveries with DataTracker accepted CYCLE2-OFF SCO ARC DT 1, 8: data transfer 2: poll 3: PI from ARC? 4 :all there? 5: 6: start PP 7: write DB 12/10/20154
Priorities Update II 5 2.Pipeline Integration (including OODT into the RH) stalled – missing XML, RH tasks. To resume ASAP. 3.Monitoring Database end of ASA incremental updates Harvester extensions fixes, +spatial resolution, -gal, -abstract, -bib AQ features auto-complete, freqSupport, export, API, state RH features cycle2, GOUS, hierarchical display Performance -aq, +rh, -rh 12/10/20155
Priorities Update III 6 5.SourceCat schema restructuring CYCLE2-OFF deployed 6.SourceCat refactoring / VO re-architecture 7.NGAS maintenance / performance / merge with ESO codebase meeting with Andreas at ESO 8.AsdmUpdate 612/10/2015
Relational ALMA 7 DelayHarvester and Sourcecat work (2015) ASDM required for AQUA / ERMA create insert / retrieve performance tests Control’s schema in Garching dev env. Cost / benefit. Suitable? Performant? Iterate. 16/03 Review by DBAs Modify module build order ARCHIVE / ICD31/03 Generate hibernate mappings from model18/05 Migrate online code, harvester, data packer20/07 APDM for ComInf tools (or use Oracle 11gr2 features?) Feasible? Performance tests30/09 Hibernate mappings from model.09/11 Migrate DP, Harvester, online code.31/ /10/2015
Plan from CPM Harvester corrections, galactic longitude / latitude, spatial resolution DataTracker 1 st release at ARCs Detect ingestion of pipeline product from JAO Does products PI belong to ARC? Have all pieces made it to the ARC? to PI Another one month before end of proprietary period Pipeline ingestion into Archive SourceCatalogue bugs, refactoring, data model changes, history AsdmUpdate DataPacker expansion to re-assemble pipeline products for delivery 12/10/20158
Lost Time CYCLE2-OFF cost a lot of effort Need to verify against base configuration – better instructions. Scripting. DBA work: importing data for debugging / testing Scripted Firefighting – performance of RH, NGAS issues in production Broken unit tests in ONLINE system in particular – needs refactoring VLT RH issues needed investigation – unanticipated usage 12/10/20159