Status of Computing at PDR B.E. Glendenning (NRAO), G. Raffi (ESO)

ALMA Project ASAC 2-3 April 2003 Computing IPT2 Phase 2 Scope Scope: Observing preparation and support through archival access to automatically produced (pipeline) images

ALMA Project ASAC 2-3 April 2003 Computing IPT3 Management Planning Computing Plan (framework) and 15 subsystem agreements prepared –Computing Plan submitted to document approval process (comments from JAO to be accounted for) –Subsystem agreements approved by Computing IPT Management model: –Agreement “contracts” followed by subsystem scientists (scope) and software management (schedule, cost) –Initial PDR, yearly CDRs and release planning –Synchronized 6-month releases across all subsystems –SSR requirements mapped to releases for progress tracking

ALMA Project ASAC 2-3 April 2003 Computing IPT4 Management Planning (2) –Readiness reviews, acceptance tests, and delivery to interim operations in 2007 –“Data flow” items get a second development period during interim operations review requirements after first experience with operations Not subject to current agreements – institutional responsibility could shift PackageFeature nameShort descriptionSSR requirement Numbers To be completed at Release (R ) Status at Milestone T0 (N,P,C) ObsProjectCheckObsDatabaseCheck observation database for conflict 3.0R10R3.0N ObsProjectLocal saveSave/recover programs to/from local disk in human readable form 3.0-R11, 3.1R3, 3.1R12 R1.0N

ALMA Project ASAC 2-3 April 2003 Computing IPT5 AIPS++ Evaluation AIPS++ (along with the ESO Next Generation Archive System) is a major package proposed for use by ALMA –Both to ensure at least one data reduction package is available to users and in implementing ALMA systems (notably the Pipeline) AIPS++ is a very controversial package (long development period, has not yet received wide acceptance) ALMA Computing jointly with SSR committee, has arranged several evaluations (next presentation by R.Lucas) –Audit of capabilities based on documentation –AIPS++/IRAM tests to test suitability for millimeter data –Benchmarking tests NRAO/AIPS++ Consortium have set up a technical review

ALMA Project ASAC 2-3 April 2003 Computing IPT6 AIPS++ technical review Held at the National Radio Astronomy Observatory in Socorro on March 5-6, 2003 Review panel: Roger Brissenden (CfA), Hilton Lewis (Keck), Chair, Andrew Lumsdaine (U. Indiana), Dave McConnell (ATNF), Steve Scott (OVRO), David Silva (ESO), Doug Tody (NRAO) and Rick White (STSCI). Report submitted to NRAO Director and circulated to AIPS++ Executive Committee.

ALMA Project ASAC 2-3 April 2003 Computing IPT7 AIPS++ review summary - major recommendations (1/2) Note: All here given recommendations are a short summary of the full text Change the focus of AIPS++ – from a system aiming to be a general-purpose radio astronomy package to a system that is developed to meet the strategic goals of the Consortium. Modify the Development Process –The AIPS++ master schedule should be consistent with the major project schedules (e.g., GBT, ALMA, EVLA). Strengthen the Project Management and the Project Team Further Strengthen the Project Scientist Role

ALMA Project ASAC 2-3 April 2003 Computing IPT8 AIPS++ review summary - major recommendations (2/2) Short-Term Priorities (next 12 months) –Significant improvements in reliability and stability –Performance improvements approaching the performance of AIPS –Synthesis imaging capability covering many common use cases of the VLA –Reduced emphasis on graphical user interface Proceed With the Proposed Technology Changes –The Panel finds that the case for changing the technology to support future development of AIPS++ is compelling. The proposed architectural change to the framework has technical merit and a proof of concept should be pursued at this time, in parallel with current development. However, it is critically important that this not be allowed to distract the AIPS++ team from completing the current development.

ALMA Project ASAC 2-3 April 2003 Computing IPT9 Current AIPS++ Architecture

ALMA Project ASAC 2-3 April 2003 Computing IPT10 Proposed AIPS++ Architecture

ALMA Project ASAC 2-3 April 2003 Computing IPT11 ALMA Proof of Concept AIPS++ proposes to use ALMA Common Software (ACS) technology and interoperability designs (exact technical relation being defined) –CORBA, Python, Container/Component, Java –To be confirmed by AIPS++ Executive Committee. This should allow AIPS++ to interoperate much more readily with other software packages and (particularly) ALMA subsystems –In particular it becomes easier in principle for the Pipeline to adopt engines from other packages –provided some data model interoperability exists

ALMA Project ASAC 2-3 April 2003 Computing IPT12 Preliminary Design Review March 18-20, Tucson –Panel R. Doxsey (CHAIR) (STScI, Head HST Mission Office) P. Quinn (ESO, Head of Data Management and Operations Division) N. Radziwill (NRAO, Head of GBT Software Development) J. Richer (Cambridge, ALMA/UK Project Scientist, Science Advisory Committee) D. Sramek (NRAO, ALMA Systems Engineering IPT Lead) S. Wampler (NSO, Software Architect) A. Wootten (NRAO, ALMA/North America Project Scientist)

ALMA Project ASAC 2-3 April 2003 Computing IPT13 Hot Topics presented at PDR (1/2) Missing operational requirements Science committee is being set up to answer operational questions (Lucas and Schwarz are Computing representatives there) Conflict of priority at ATF If the issue of operators is solved, we do not see here a conflict in the long run, although in 2003 we have quite some pressure. Splitting of Archive and Pipeline operation between OSF and SOC Long term baseline concept (not for Interim Science) : –Archive will be in two copies: OSF and SOC. –Calibration and Quick look running at the OSF (due to their timing) while Science Pipeline will run at the SOC.

ALMA Project ASAC 2-3 April 2003 Computing IPT14 Hot Topics presented at PDR (2/2) Only OSF in 5 years plan Santiago (SOC) and RSCs commissioning is not included. Santiago Center (SOC) commissioned in 2008 possibly with a limited bandwidth (e.g. 10 Mb/s -> 100 Mb/s) to check operations model and reliability asap. Santiago link in 2007 Good to monitor OSF and gain experience with link. Could have limited bandwidth. RSC installation (once or twice?) Baseline plan: prototype system to run the array, delivering the final (pipeline and archive) systems as late in the construction period as possible: one in Chile and one to each RSC.

ALMA Project ASAC 2-3 April 2003 Computing IPT15 Possible issues Staffing (especially at NRAO) not yet at plan, although full recruitment now in progress Data rate increase sought by Science Software Requirements group, based on assumption that Base Correlator change will be approved. –Correlator enhancements need corresponding increase in data rates to be of most importance –Cost implications need to be determined

ALMA Project ASAC 2-3 April 2003 Computing IPT16 PDR Recommendations (1/2) Unofficial top-level results, based on exit debriefing by panel to IPT Management and M. Rafal –Official report and IPT response by mid-April Well prepared PDR; at or ahead of where similar projects have been at this stage The ALMA Project needs to develop an understanding of operations, and the Computing IPT needs to fold this in to their planning and priorities. This might result in reprioritization of SSR requirements ALMA project needs to define clearly the steps necessary for getting to, and implementing, Interim Operations and the consequent requirements for computing support

ALMA Project ASAC 2-3 April 2003 Computing IPT17 PDR Recommendations (2/2) The interaction with AIPS++ requires careful management. Operational (Pipeline) requirements on AIPS++ need further development. Testing effort seems satisfactory; management will need to follow-up to ensure subsystem unit tests are in fact carried out

ALMA Project ASAC 2-3 April 2003 Computing IPT18 Computing Goals for Support prototype antenna tests 2.Use AIPS++ for ALMA Computing work (Pipeline, Off-line, DRUI) while completing feasibility phase: - AIPS++ benchmarking - Support use of ACS in AIPS++ Proof of Concept  Review progress on AIPS++ in 1 year (CDR2) Produce 1st integrated ALMA software Release Waiting for ALMA Operations Plan (software aspects)