AtacamaLargeMillimeterArray ACS Training Notification Channels (last updated by Heiko Sommer, Nov. 2009)

UTFSM Valparaiso, Nov. 2009ACS Training2 What is a Notification Channel? (1) Abstract A data distribution service that allows many processes to deliver or receive data without “knowing” each other directly. This data is usually small, used to notify all interested parties of something that happened (“event”, “message”). Also works for asynchronous communication inside a process. “Publish-Subscribe”, “Message based system” Channel-1 A B Channel-2 Msg-1 C Msg-2

Container Client “C” Corba (TAO) Notify-Srv UTFSM Valparaiso, Nov. 2009ACS Training3 What is a Notification Channel? (2) In ACS Service defined by Corba, free implementation from TAO. The “channel” abstraction is implemented as a remote object in a dedicated “Notify Service” process. The NC object receives (sync) data from and resends (async) data to registered ACS components or clients. The “event” data type must be an IDL-defined struct. Comp. A Msg-1 Msg-2 NC-1 NC-2 Comp. B ACS NC

UTFSM Valparaiso, Nov. 2009ACS Training4 Dis-/ Advantages of using NCs NCs are an alternative to direct “Request/Reply” calls. –Without using a NC, the most similar design is for the consumer to register a callback directly in the data-supplying process. NCs decouple the communicating partners –Reduces build time dependencies –Makes partners independent of each others lifetimes (e.g. subscribe to events before the sender has been created) NCs can protect the sender from slow receivers (within the limits of buffering and loss policies) NC may offer the flexibility of attaching one NC’s output to another NC’s input, like Unix pipes. Intermediate filtering and data interception possible. This is not yet used in ACS projects. Use of NCs makes debugging the system more difficult. Don’t use NCs for massive data transfers! (-> bulk data) Don’t use NCs for high-frequency and highly structured data. Each packet will contain complete self-description of the data (Corba Any).

UTFSM Valparaiso, Nov. 2009ACS Training5

UTFSM Valparaiso, Nov. 2009ACS Training6 Using NCs in ACS - Overview ACS provides a simplified API to access and use NCs. –A Supplier class that takes events for a given NC. –A Consumer class that delivers events from a given NC to a user’s registered callback method. –These classes are available for Java, C++, Python –ACS NC classes work with components and for client applications. –Nearly all CORBA complexity is hidden from developers. ACS only supports the push-push model, out of the many Corba choices. Configuration of NCs via the CDB Conventions: –the IDL structs used as event data types should have names ending in “ Event ”. –Channels are named using const string s defined in IDL, to ensure same channel names used in C++, Java, Py –Send / receive only one type of event per NC.

UTFSM Valparaiso, Nov. 2009ACS Training7 The ACS NC Supplier class An ACS class designed to publish events Constructed for a particular NC. Creates the NC if necessary. Immediately connects to the NC. Method “ publishEvent ” takes IDL-struct-defined objects. Can disconnect from the NC, releasing remote resources. May destroy the NC. Normally not used, because other publishers may be affected (requires sync’ing).

UTFSM Valparaiso, Nov. 2009ACS Training8 The ACS NC Consumer class (1) An ACS class designed to process structured events. The developer must –create a subclass of Consumer for each notification channel and override the processEvent method –or provide a “handler function” to a Consumer object (delegation). Whether that particular object can process all types of events on a given channel is up to the developer to decide. Corba calls the push_structured_event method when an event comes in. The ACS class then pre-processes the data and forwards it to the user code. The developer has no control over when the event data gets injected. Just needs to react (and return control quickly!).

UTFSM Valparaiso, Nov. 2009ACS Training9 The ACS NC Consumer class (2) Subscribe to and unsubscribe from all types of events. Create an NC to subscribe to, if it’s not yet created. Specify when it is ready to start receiving events. Suspend and resume the connection to the channel at any time. Disconnect from the NC when done (Important!) Filter out events it does not want to process. Get notified when a Supplier begins publishing a new type of event and dynamically subscribe to it. The same holds true when subscriptions are no longer offered.

UTFSM Valparaiso, Nov. 2009ACS Training10 Event Filtering Filtering is provided on three different levels: A consumer of structured events from “Channel-A” will never see structured events published by “Channel-B”. Consumers will only process structured events for the type_name’s they are subscribed to (which can also be dynamically unsubscribed from). Rarely used: The ability to “ignore” entire events based on various values in the structured event. A consumer provides the type_name and a filter string based on extended trader constraint language. This string can be as simple as “$Temperature<=100” for simple CORBA types (implying there is a CORBA integer in the structured event and it is named “Temperature”). This becomes non-trivial for filtering user-defined IDL structs though…

UTFSM Valparaiso, Nov. 2009ACS Training11 NC configuration NCs can be configured in the CDB. The ACS NC libs read and apply this configuration. To do this, provide an XML in the ACS CDB: $ACS_CDB/CDB/MACI/Channels/yourChannel/yourChannel.xml which must follow the schema: $ACSROOT/config/CDB/schemas/EventChannel.xsd (online link)online link There are a host of debug, quality of service, and administrative properties that can be configured. This is not used much in practice, therefore we don’t present the details here. See the EventChannel schema for the detailed description.

New Features (1) Since ACS 8.1 the Notification Service can be restarted safely after a crash –automatically by ACS daemons –or manually. The consumer objects will then be reconnected automatically. The supplier objects can continue to use the old NC reference. However, while the Notify Service (and thus the NC) are unavailable, the supplier will get exceptions when publishing events. The Supplier class can store failed events in a circular buffer and resend them after reconnection to the NC, if the user registers an EventProcessingCallback object with callback methods for the different good and bad cases of event delivery: –Event Sent: The event was (eventually) sent successfully to the NC –Event Stored in queue: The event could not be sent and was stored –Event Dropped: The event was removed from the temporal queue and will be discarded. UTFSM Valparaiso, Nov. 2009ACS Training12

New Features (2) ACS allows users to start many Notify Service processes, and to decide on a per-NC basis in which process the NC is hosted. –The default Notify Service is started along with all ACS services –Additional Notify Service instances can be started with acsNotifyService -s -w -n SpecialNotifyService or through the ACS services daemon. CDB config file CDB/MACI/Channels/Channels.xml : UTFSM Valparaiso, Nov. 2009ACS Training13 Default Channel “RISKY_NC_BADDATA” will run inside “SpecialNotifyService”

Current Development ACS is in the process of integrating the Supplier and Consumer classes with the ContainerServices, instead of offering them as a stand-alone library. –ACS can clean up better to avoid remote memory leaks We are working on offering a common interface for CORBA- based NCs and the DDS standard –See –DDS also publish-subscribe, w/o central service –Performance and jitter often better than Corba NC, especially when network multicast gets used. –Less self-descriptive than NCs, makes generic clients hard to implement. Construction of a new Event GUI, see next slide. UTFSM Valparaiso, Nov. 2009ACS Training14

UTFSM Valparaiso, Nov. 2009ACS Training15 Event GUI (new dev.)

UTFSM Valparaiso, Nov. 2009ACS Training16 Questions???

UTFSM Valparaiso, Nov. 2009ACS Training17 Quality of Service Properties EventReliability - Handles the kind of guarantee that can be given for a specific event getting delivered to a Consumer. Represented by BestEffort or Persistent. ConnectionReliability - Handles the kind of guarantee that can be given for the connections between the Notification Channel and its clients. Can have the same values as EventReliability. StopTime - Specifies an absolute expiry date. Timeout - Specifies a relative expiry date. StartTime - Specifies an absolute time after which the event will be discarded. PriorityOrder - Used to control the order of the events delivered to the consumers. Default value is 0 but ranges from –32767 to

UTFSM Valparaiso, Nov. 2009ACS Training18 Quality of Service Properties (continued) OrderPolicy - Used by a proxy to arrange events in the dispatch queue. Can be the following values: AnyOrder, FifoOrder, PriorityOrder, and DeadlineOrder. DiscardPolicy - Defines the order in which events are discarded when overflow of internal buffers occur. Holds the same values as OrderPolicy plus an addition value: LifoOrder. MaximumEventsPerConsumer - Defines a bound for the maximum number of events the channel will queue for a given consumer. The default value is 0. MaximumBatchSize - Not relevant to the ACS API! PacingInterval - Not relevant to the ACS API!

UTFSM Valparaiso, Nov. 2009ACS Training19 Administrative Properties MaxQueueLength - Specifies the maximum number of events the channel will queue internally before starting to discard events. MaxConsumers - Defines the maximum number of consumers that can be connected to a particular channel. MaxSuppliers - Defines the maximum number of suppliers that can be connected to a particular channel. RejectNewEvents - Defines how to handle events when the number of events exceeds MaxQueueLength value. Set this Boolean value to true for IMPL_LIMIT exceptions to be thrown. A false value implies events will be discarded silently.

UTFSM Valparaiso, Nov. 2009ACS Training20 What is the TAO Notification Service? CORBA-based Notification Service is a service which extends the existing OMG Event Service, adding to it the following new capabilities: The ability to transmit events in the form of a well-defined data structure, in addition to Anys and Typed-events as supported by the existing Event Service. The ability for clients to specify exactly which events they are interested in receiving, by attaching filters to each proxy in a channel (not yet implemented in acsnc). The ability for the event types required by all consumers of a channel to be discovered by suppliers of that channel, so that suppliers can produce events on demand, or avoid transmitting events in which no consumers have interest.

UTFSM Valparaiso, Nov. 2009ACS Training21 What is the TAO Notification Service? The ability for the event types offered by suppliers to an event channel to be discovered by consumers of that channel so that consumers may subscribe to new event types as they become available. The ability to configure various quality of service properties on a per-channel, perproxy, or per-event basis. An optional event type repository which, if present, facilitates the formation of filter constraints by end-users, by making information about the structure of events which will flow through the channel readily available.

UTFSM Valparaiso, Nov. 2009ACS Training22 What is a Structured Event? A structured event is a data structure defined in IDL that is sent across the Notification Service by suppliers to consumers. There is only one field that must be “filled-out”: type_name. Consumers subscribe events based on this field. The optional fields: –event_name is basically an arbitrary string defined by the developer. –Of particular interest to the developer: filterable_data[]. This is a sequence of CORBA properties (string/CORBA Any pairs) that can be filtered using the extended trader constraint language discussed above.

UTFSM Valparaiso, Nov. 2009ACS Training23 Architectural Perspective

UTFSM Valparaiso, Nov. 2009ACS Training24 Structured Events

UTFSM Valparaiso, Nov. 2009ACS Training25

UTFSM Valparaiso, Nov. 2009ACS Training26 IDL for other NC Courses #pragma prefix "alma" module ACSCOURSE_MOUNT { Mount5 * Nearly identical to Mount1 except for the fact that this example publishes/consumes events from an event channel and that exception handling has been ommitted. */ interface Mount5 : ACS::ACSComponent { /** (Pre)sets a new non-moving position for the antenna. The position coordinates are given in azimuth and elevation. az position azimuth (degree) elev position elevation (degree) * */ void objfix (in double az, in double elev); };

UTFSM Valparaiso, Nov. 2009ACS Training27 IDL for other NC Courses /** The name of the event channel our interface implementation will send events to. It is specified in the IDL instead of the implementation language to make it harder for developers * of Consumer applications to confuse names (i.e., using "MountChannel" instead of "mountchannel". */ const string MOUNT_CHANNEL = "mountchannel"; /** * This IDL structure defines an "event" type that will sent across the network to any consumers subscribed to it */ struct MountEventData { double Azimuth; double Elevation; };

UTFSM Valparaiso, Nov. 2009ACS Training28 ACSEventAdmin The ACSEventAdmin graphical user interface was created and implemented in the Python programming language to administer and monitor all ALMA event channels. It supports the following operations: Creating new notification channels Reconfiguring a channel’s properties at run-time Destroying notification channels Obtaining a list of all active channels Obtaining the total number of suppliers and consumers connected to a channel Obtaining the total number of events that have been published on a channel Obtaining a list of all types of ALMA events that have been published on a channel Monitoring all events using the callback design pattern

UTFSM Valparaiso, Nov. 2009ACS Training29 Event Browser (superseded by Event GUI)