The ALMA Software System Joseph Schwarz (ESO)

ACS Course, 15 January 2007The ALMA Software System ALMA is… A radio aperture-synthesis telescope covering ν GHz (λ ~ mm), and consisting of m antennas and 12 7m antennas that will make detailed images of astronomical objects. They can be positioned as needed with baselines from 0.15 to 14 kilometers so as to give the array a zoom-lens capability, with angular resolution reaching 10 milliarcseconds. A leap of over two orders of magnitude in both spatial resolution and sensitivity over existing instruments ALMA's great sensitivity and resolution make it ideal for medium scale deep investigations of the structure of the submillimeter sky. A joint project of the North American, European and Japanese astronomical communities. Courtesy of Al Wootten, ALMA/US Project Scientist Location: the Llano de Chajnantor, Chile, at an altitude of about 5000m.

ACS Course, 15 January 2007The ALMA Software System Complete Frequency Access

ACS Course, 15 January 2007The ALMA Software System Where can such transparent skies be found? Living Earth ALMA...or here Here...

ACS Course, 15 January 2007The ALMA Software System Performance requirements Baseline correlator H/W produces ~ 1 Gbyte/s Must reduce to average/peak data rates of 6/60 Mbyte/s (baseline) Raw (uv) data ~ 95%, image data ~ 5% of the total Implies ~ 180 Tbyte/y to archive Archive access rates could be ~5 higher (cf. HST) Feedback from calibration to operations ~ 0.5 s from observation to result (pointing, focus, phase noise) Science data processing must keep pace (on average) with data acquisition

ACS Course, 15 January 2007The ALMA Software System Software Scope From the cradle… Proposal Preparation Proposal Review Program Preparation Dynamic Scheduling of Programs Observation Calibration & Imaging Data Delivery & Archiving Afterlife: Archival Research & VO Compliance

ACS Course, 15 January 2007The ALMA Software System And it has to look easy… From the Scientific Software Requirements: “The ALMA software shall offer an easy to use interface to any user and should not assume detailed knowledge of millimeter astronomy and of the ALMA hardware. “The general user shall be offered fully supported, standard observing modes to achieve the project goals, expressed in terms of science parameters rather than technical quantities...” The expert must still be able to exercise full control But what is simple for the user will therefore be complex for the software developer. Architecture should relieve developer of unnecessary complexity Separation of functional from technical concerns

ACS Course, 15 January 2007The ALMA Software System System data flow 6-60 Mbyte/s ~0.5 s feedback time

ACS Course, 15 January 2007The ALMA Software System Distributed development ALMA OSF & AOS DRAO Penticton ALMA ATF Arcetri Observatory Jodrell Bank Brera Observatory IRAM Grenoble ATC Edinburgh NAOJ ESO NRAO Obs de Paris Univ. Calgary c c Santiago SCO MPI Bonn DAMIR/IEM Madrid

ACS Course, 15 January 2007The ALMA Software System 10 Distributed operations ~30 km ~15 km This facility must operate 24 hours/day, 365 days/year ~30 km ~15 km

ACS Course, 15 January 2007The ALMA Software System 11

ACS Course, 15 January 2007The ALMA Software System 12 Distributed runtime the AOS (5000m) 66 Antennas Every antenna in ALMA has its own computer (ABM) and its own container on that computer. 40 devices (LRUs) per antenna Each hardware device on the antenna is represented as a component in that antenna’s container. Correlator 1000 LRUs in the ARTM Operations, Archive, Data reduction/analysis At the OSF (2900m, 30 km from the AOS) ACS will have to manage > 3500 components

ACS Course, 15 January 2007The ALMA Software System Run-time Challenges & Responses Changing observing conditions High data rates Diverse user community (novice to expert) Distributed hardware & personnel AOS: antennas scattered km from correlator AOS-OSF: operators are ~30 km from array OSF-SOC-ARCs: PIs, staff, separated from OSF by 1000s of km, often by many hours in time zone Dynamic Scheduler Integrated scalable Archive Flexible observing tool, GUIs High-speed networks Distributed architecture –CORBA & CORBA services –Container/Component model –XML serialization

ACS Course, 15 January 2007The ALMA Software System Development-time challenges & responses Evolving requirements Changing data rates New observing modes New hardware (ACA) IT advances Distributed development Different s/w cultures Iterative development Modular, flexible design Unified architecture (HLA) Functional subdivisions aligned to existing project organization Implemented via ACS –Don’t do it twice –If you must do the same thing, do it the same way everywhere E-Collaboration tools

ACS Course, 15 January 2007The ALMA Software System "What the deuce is it to me?" You say that we go round the sun. If we went round the moon it would not make a pennyworth of difference to me or to my work." Separation of concerns Functional: Physics, algorithms, hardware… PIs can concentrate on their research specialties Software encapsulates aperture synthesis expertise Technical: Communications, databases, etc. Subsystem teams should concentrate on function Technical architecture should provide simple and standard ways to: Access remote resources Store and retrieve data Manage security needs Communicate asynchronously between subsystems, components

ACS Course, 15 January 2007The ALMA Software System ALMA Common Software (ACS) Main vehicle for handling technical concerns Framework for distributed object architecture Used all the way down to device level in control system Built on CORBA, but hides its complexity Wraps selected CORBA services Multi-language support: Java, C++, Python Vendor-independent High-quality open-source ORBs available (e.g., TAO) System evolving to meet developers’ needs Initial developer resistance Developers now asking for more Dedicated team of systems-oriented developers

ACS Course, 15 January 2007The ALMA Software System Container/Component Interfaces container Comp CORBA ORBs Services lifecycle interface: init() run() restart() Comp functional interface: observe() container service interface other ACS services Manager deployment configurations My container starts and stops me and offers its services, some of which I don’t notice I only care about the Lifecycle IF of my components

ACS Course, 15 January 2007The ALMA Software System Data modeling & code generation Define data structure and content abstractly UML: Project (APDM) & Science (ASDM) data models Spreadsheets: Hardware device characteristics Open-source code generation framework Wrapped & supplied by ACS Generate automatically: XML schemas, Java, C++, SQL, docs from UML Binding classes from schemas Type-safe native language access to data Automatic validation possible Exchange of XML documents between subsystems In a language-independent way Via the Archive or via the network

ACS Course, 15 January 2007The ALMA Software System Avoiding nasty surprises Iterative development Releases every 6-months (4 major, 3 minor to date) 6 major ACS releases to date Function-based teams develop & deliver integrated s/w on shorter time scales (typically, 3-4 FBTs per release cycle) Periodic “CDR”s: Focus on plans for next FBTs, next release Emphasize integrated system Simulate interferometry while waiting for hardware We’ll be ready for first fringes at the ATF in Q2 Two integrated user tests at the Antenna Test Facility (ATF) so far Optical pointing: feedback led to... Holography: greatly improved robustness, user satisfaction

ACS Course, 15 January 2007The ALMA Software System 20 ATF: Where the action is VLA site, Plains of San Augustin, New Mexico, USA

ACS Course, 15 January 2007The ALMA Software System ALMA Schedule 2007 Q2: First fringes on prototype antennas at ATF (VLA site) First production antenna on site at OSF in Chile 2008 Q1: Second production antenna at OSF 2008 Q3: Two-antenna interferometer at OSF 2008 Q4: “Array software” complete 2009 Q1: Three-antenna interferometer at AOS 5000m) Start of commissioning 2010 Q4: Early science operations 2012 Q3: Hardware & “Observatory s/w” complete Full science operations (full antenna complement)