ICALEPCS WE2.4-6I ALMA Common Software Status and Development G.Chiozzi a, A.Caproni a e, R.Cirami e,P.Di Marcantonio e,D.W.Fugate d, S.Harrington b, B.Jeram a, M.Pesko c, M.Sekoranja c, H.Sommer a, K.Zagar c a ESO b NRAO c Cosylab d U.Calgary e INAF-AOT

2 ICALEPCS WE2.4-6I ALMA Common Software Status Contents What is ACS? Where are we? Main features Platforms Highlights of the last two years Projects and community Conclusion Component/Container model Event Handling and Notification Channel Threading support Real time Bulk Data Simulation Alarm System Task and parameters GUIs Benchmarking

3 ICALEPCS WE2.4-6I ALMA Common Software Status What is ACS? ACS is a software infrastructure for the development of distributed systems based on the Component/Container paradigm common application framework and design patterns, not just libraries well tested software that avoids duplication make upgrades and maintenance reasonable common development environment and tools

4 ICALEPCS WE2.4-6I ALMA Common Software Status Where are we? Developed for ALMA and used by several other projects. ACS is based on a kernel of software contributed by cosylab and developed for ANKA. Our collaboration started in Trieste at ICALEPCS ½ of allocated development effort spent until now Total allocated ~25 man years + additional external contribution (~10). 9 th release Extensively used in the field: –ALMA Test Interferometer and labs –ALMA software integrations –Other projects

5 ICALEPCS WE2.4-6I ALMA Common Software Status Main Features ACS provides the basic services needed for object oriented distributed computing. Among these: –Transparent remote object invocation –Object deployment and location based on a container/component model –Distributed error and alarm handling –Distributed logging –Distributed events The ACS framework is based on CORBA and built on top of free CORBA implementations.

6 ICALEPCS WE2.4-6I ALMA Common Software Status Supported Platforms Operating system: Linux, RH-E + other flavours Real-time: RTAI and VxWorks Languages: C++, JAVA, Python CORBA middleware: TAO (& ACE) (C++), JacORB (Java), Omniorb (Python), CORBA services. Embedded ACS Container: PC104, Debian, 300Mhz Geode, 256MB RAM, 256 MB flash (CosyLAB microIOC)

7 ICALEPCS WE2.4-6I ALMA Common Software Status Policy and License: LGPL The strategy to provide common features to our users is: –Use as much as possible open-source tools, instead of implementing things. Do not reinvent the wheel Do not pay for licenses –Identify the best way to perform a certain task among the many possibilities –Wrap with convenience and unify APIs ACS is distributed under LGPL license

8 ICALEPCS WE2.4-6I ALMA Common Software Status Highlights of the Last Two Years New in ACS for: developers to use in their code: libraries, convenience classes, utilities to improve the quality of the code test/integration/administrators, transparently from application code (i.e. in principle transparent to Component developers)

9 ICALEPCS WE2.4-6I ALMA Common Software Status Component Container Evolution/Cleanup Container Services –Full separation between Container and Container Services –Cleaner interfaces –Easier to replace Container implementation –The most important services provided now by the ContainerServices are… Component life cycle –Plain instantiation of Components not sufficient –Standard lifecycle state machine introduced for the Container to manage Components lifecycle interface: init() cleanUp() container Comp functional interface: observe() container service interface getComponent(“CompB”); Logger getLogger();

10 ICALEPCS WE2.4-6I ALMA Common Software Status Master Component ALMA subsystems interact with the Executive. Executive treat all in the same way.  Lifecycle for subsystems, not only components Fits smoothly into acs concept: -each subsystem needs a mastercomponent -it is a component with a specific interface -ACS defines the underlying state machine Implementation: -a generator (using open-architectureware) maps UML to state machines -generator creates convenience base classes -state machine has been refined in a couple of design iterations The introduction of the Master Component has been very effective. Cost of prototype generator not higher than cost of developing the MasterComponent in code

11 ICALEPCS WE2.4-6I ALMA Common Software Status Event Handling and Notification Channel Events are widely used in ALMA for synchronization and asynchronous, *-to-* communication. Decoupling of Consumers and Suppliers Very easy interface: –Supplier classes –Consumer classes Contract based on IDL data structures. Strong naming conventions and checking tools Administrative interface Quality of service

12 ICALEPCS WE2.4-6I ALMA Common Software Status Threading Support Many Components have a multi-threaded structure Management of threads was a source of problems Developed easy-to-use threading classes: –Override a run() method –Use the thread manager Based on ACE Threads in C++, concurrent library in Java

13 ICALEPCS WE2.4-6I ALMA Common Software Status A change in paradigm! Real-time Support VxWorks → TICS Entire LCU in real-time OS ACS provides complete Container/Component in real- time environment Support will probably remain for other projects (VLT) RTAI → ALMA RT Kernel inside Linux Component not real-time ↔ small time-critical functions in Kernel. Less code in real-time, but more complex to debug ACS provides easy: –Communication with kernel modules –Logging from kernel modules –Kernel module management

14 ICALEPCS WE2.4-6I ALMA Common Software Status Bulk Data Transfer Requirements from the correlator: –64 MB (megabyte)/sec Based on CORBA A/V streaming service TAO C++ implementation Very easy interface, based on our use cases No CORBA A/V visible See poster: PO Achieved Performance Gigabit P2P Ethernet BD throughput around 800 Mbits/s (~100 MB/s) requirements fulfilled CORBA throughput around then 500 Mbit/s (~ 55 MB/s) Estimated gain in the throughput around 30%

15 ICALEPCS WE2.4-6I ALMA Common Software Status Simulation Why simulation? –Distributed development –Features or entire subsystems not yet available –Test a subsystem in isolation Simulation of Components from IDL interface spec. Dumb default or “intelligent” simulation See presentation: WE4A.2-5O

ICALEPCS WE2.4-6I ALMA Common Software Status CERN Laser ACS Alarm System: Laser Feasibility prototype to re-use CERN Laser Alarm System (TH2.3-7O) Keep the same API Reuse the Laser Alarm Console ACS Component/Container replaces J2EE acsjms implements jms for ACS on top of Notification Channel IDL interfaces replace EJB interfaces Reimplementatio n of Laser interfaces The challenge: reuse a complete subsystem/service in a very different software infrastructure

17 ICALEPCS WE2.4-6I ALMA Common Software Status Tasks and Parameters ACS is used in ALMA also as data reduction infrastructure framework Requirement: data reduction to be started as a stand-alone process. A program which starts-up, performs processing and shuts down. Implementation: –Stand alone executable –Static container –Works with and without ACS suite Input parameters are complex data sets: –Parameter set definition (xml) –Parameter set instantiation (xml) –Validation –Parsing

18 ICALEPCS WE2.4-6I ALMA Common Software Status GUIs A LabView prototype has been implemented ACS Supports ABeans development with an Eclipse plug-in Some projects are using Qt Different projects and different subsystems have different requirements!

19 ICALEPCS WE2.4-6I ALMA Common Software Status Performance and Benchmarking ACS has performance requirements to satisfy Changes to code and upgrade of external libraries can affect performance Created performance measurements and reporting framework Performance of Component to Component communication, notification channel, logging system…

20 ICALEPCS WE2.4-6I ALMA Common Software Status ACS installations and projects

21 ICALEPCS WE2.4-6I ALMA Common Software Status The ACS community HPT Hexapod Telescope (Germany → Chile) Sardinian Radio Telescope (Italy) OAN 30m (Spain) ALMA (Chile) ANKA (Germany) APEX (Chile)

22 ICALEPCS WE2.4-6I ALMA Common Software Status ICALEPCS - 2 nd ACS Workshop

23 ICALEPCS WE2.4-6I ALMA Common Software Status Conclusion Core concepts very stable In use in ALMA and other projects: we have been getting a lot of useful feedback A lot of work to do: –Most packages available, but features incomplete –In particular scalability and performance issues. We know what to do. –Make easier the life of developers: abstract concepts, code generation (ACS code generation developed by community). Having a user’s base in addition to our main project has provided important feedback, cross-fertilization of concepts and ideas and contributed to software quality

24 ICALEPCS WE2.4-6I ALMA Common Software Status Questions (& Answers) MO2.2-1IThe ALMA Computing Project – Update and Management Approach WE2.4-6IThe ALMA Common Software ACS Status and Developments WE3A.3-6OThe ALMA Telescope Control System WE4A.2-5OA generic software interface simulator for ALMA common software PO Transmitting huge amounts of data design implementation and performance of the bulk data transfer mechanism in ALMA ACS PO Migration from ACS 1.1 to ACS 4 at ANKA PO ALMA Correlator Real-Time Data Processor TH2.3-7OFirst Operational experience with Laser (K.Sigerud, CERN) ICALEPCS Papers

25 ICALEPCS WE2.4-6I ALMA Common Software Status Reserve slides

26 ICALEPCS WE2.4-6I ALMA Common Software Status Performance Average C++ throughput: 1500 event/s (100 bytes) Average C++ throughput: 3500 event/s (100 bytes)