Critical Design Review, April 2003 <overview talk> Critical Design Review, April 2003

Critical Design Review, April 2003 <intro> Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive goals process prototypes plan Critical Design Review, April 2003

Critical Design Review, April 2003 </intro> Critical Design Review, April 2003

Critical Design Review, April 2003 <goals> Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive what WSA does ingest to DB, index serve to users supply analysis tools survey-wide processing recalibration pairing, upper limits proper motions etc survey pixmaps and catalogues Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive what is a "science archive" ? not just a repository expect most science will be done through the archive VO-ready from the outset ==> calibrated/documented/engineered ==> tools and services Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive levels of archive service L1 = area subset pixel maps and catalogues L2 = complex queries on catalogues L2a = joint queries with other archives L3 = database manipulations meet these aspirations in slices Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive V1 functionality Jan 2004 browse frame DB, download generate cut-outs and catalogues on demand generate mosaics to 0.8 degrees box/cone search on UKIDSS surveys joint searches with 2MASS, SSS, SDSS SQL queries on any parameter combinations Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive V2 functionality in Jan 2005 provide simple server-based tools histograms and xy plots from returned parameters SED plot generate large mosaics/stacks on demand user-tuneable re-extraction and stacking deliver all services as web services for AstroGrid Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive V3 goals advanced server based visualisation tools pannable large area imaging multi-D plotting and rotation server based data analysis tools correlation function parameter cluster analysis principal component analysis ability to upload user algorithms Critical Design Review, April 2003

Critical Design Review, April 2003 </goals> Critical Design Review, April 2003

Critical Design Review, April 2003 <process> Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive design process three phase s requirements / analysis and design / build general principles staged approach to functionality learn from SDSS experience prototype programme tests functionality informs design decisions Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive requirements capture top level requirements by JAC example science useages from UKIDSS analysis ==> functional requirements SRAD agreed with all stakeholders Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive analysis and design status hardware : separate pixel/catalogue solns DBMS V1 : SQL Server DBMS V2 : TBD (scaleability concerns) relational modelling : done curation tasks : designed but not written pixel processing V1 : all prototyped pixel processing V2 : under design user interface : designed and prototyped Critical Design Review, April 2003

Critical Design Review, April 2003 </process> Critical Design Review, April 2003

Critical Design Review, April 2003 <prototypes> Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive experiments/prototypes data volume / scaleability DBMS deployment web technology user interface hardware network Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive data volume SuperCOSMOS 10 GB/day moved to RAID array 2000 WSA 100 GB/day storage/ingest not hard real challenge is trawl rate move to parallellised modular system VSA 600 GB/day believe WSA design will scale Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive web technology and user interface SuperCOS web design, cgi forms 6dFGRS SQL interface Tomcat, servlets SSA ditto large volume WSA V1 as above WSA V2 web services, VO publish WSA V3 grid services Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive DBMS deployment SuperCOS flat files SDSS-EDR Objectivity 6dFGRS SQL Server on small DB SSA SQL Server on large DB WSA V1 SQL Server SuperCOS DB2 test (not public) WSA V2 ?? Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive hardware and network variety of loaned equipment SCSI vs IDE vs fibre linked key issue : aggregate query rate vs hardware type, RAID level, number of discs current : 80 MB/s target 300 MB/s CASU-WFAU transfer tests : 1 MB/s continued tests TCP tuning / firewall upgrades / GridFTP Critical Design Review, April 2003

Critical Design Review, April 2003 </prototypes> Critical Design Review, April 2003

Critical Design Review, April 2003 <plan> Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive management structure VDFS planning PI Emerson coIs McMahon Lawrence Work Leader Hambly Local manager Williams Task Leadership as required Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive stakeholders UKIRT Adamson UKIDSS Warren, Lawrence VISTA VPB, VDFS ManTeam ESO Quinn Grid GSC, AGOC Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive management tools Deliverables V1,2,3 functionalities Schedule Gantt charts Effort matrix Task lists Risk Plan Weekly meetings Quarterly Reports Periodic Reviews Critical Design Review, April 2003

Critical Design Review, April 2003 WFCAM Science Archive key dates WFCAM CDR Sep 2002 SRAD agreement Dec 2002 WSA CDR Apr 2003 SSA prototype Jun 2003 V1 operational Jan 2004 V2 CDR Apr 2004 V2 operational Jan 2005 VSA Phase B start Feb 2005 VSA operational Sep 2006 Critical Design Review, April 2003

Critical Design Review, April 2003 </plan> Critical Design Review, April 2003

Critical Design Review, April 2003 </overview talk> Critical Design Review, April 2003