1. J. Gutleber, L. Orsini, 2005 March 15
OLSWG Meeting
J. Gutleber, L. Orsini, 2005 March 15
XDAQ
Error Handling

2. Topics Exceptions in XDAQ Local Exceptions System WideError Handling
Error Handling Contexts
Error Handling Use Cases
Uniform Error Message Format
Programming Approaches
Dealing with Errors in XDAQ
Distributed Error Handling
An Example
XDAQ III Topical Meeting

3. Exceptions in XDAQ An error/exception is characterized by
an identifier (What has happened)
an originator (Who and where)
a time (When)
the context ( Who is in charge of handling)
All exceptions are treated as software objects
All exceptions types are defined and hierarchically organized
e.g C++ class hierarchy
Exceptions can be detected in a particular portion of code
either the exception is handled and recovered locally
or the exception must be notified to a external entity for handling
finally an exception that cannot be handled is reported to the user
Tools must be available for
error detection
error notification
error reporting
XDAQ III Topical Meeting

4. Local Exceptions reported exceptions method exceptions
layer 3
reported exceptions
layer 2
method
exceptions
layer 1
layer 0
handled exceptions
XDAQ III Topical Meeting

5. System Wide Error Handling
System Sub-System layers W E D F A C B
Applications and Error Handlers
Report Lines
XDAQ III Topical Meeting

6. Errors Handling Contexts
Context A
e.g. Incomplete event
Error handling intelligence
e.g. re-start event building
Context B
e.g. Could not send event fragment
Context C
e.g Network send failure
One or more error handlers for each context
An error can be treated by a handler or is forwarded to a super-context
A program always reports into a single context
A program can switch the reporting contexts at run-time
Contexts are assigned by static or run-time configuration
XDAQ III Topical Meeting

7. Error Handling Use Cases
Single thread
(synchronous)
Different threads
(asynchronous)
Different processes
(synchronous)
(asynchronous)
XDAQ III Topical Meeting

8. Uniform Error Message Format
An error schema was defined together with Michele, Alex
Schema can be mapped to SOAP and other data formats
initial implementation for SOAP
will follow implementation binary I2O (performance)
combined use of different transports
This is the proposed format to be used for exchange error messages among CMS subsystems
supports integration of Magnet Test
XDAQ III Topical Meeting

9. Content Definition General error information Compulsory information
Identifier (URI detailed format TBD)
Notifier (originator of the error, detailed format TBD)
Date/Time
Context
Optional Information
Severity
Message (open format)
Recursive definition for nested errors
Multiple error collections
Support for qualified error (user defined schema)
Possibility to plug a user defined errors. E.g Tracker, ECAL, RC, XDAQ etc.
Capability to use other industry standard formats e.g CBE by IBM
XML format definition
other formats possible e.g. binary I2O format
XDAQ III Topical Meeting

10. Notification Schema
XDAQ III Topical Meeting

11. Programming Approaches
Single Thread, Synchronous
try { … } catch() { … } clauses
Multi-Thread, Asynchronous
Error processor and callback pattern
Multi-Process, Synchronous
SOAP call with Fault reply
Multi-Process (distributed), Asynchronous
Error notification message
XDAQ III Topical Meeting

12. Dealing with Errors in XDAQ
Error are defined as C++ classes in a hierarchy
All errors inherit from xcept::Exception
E.g. class MyException: public xcept::Exception {}
Well defined interface
Stack history
Each package provides a error repository
packagename/exceptions
E.g. toolbox/exception/OutOfMemory.h
Have properties
standard properties (line, module, message, id etc.)
user definable properties ( system, time etc.)
XDAQ III Topical Meeting

13. Distributed Error Handling
Poor man’s solution
SOAP according defined schema
Protocol dependent
XDAQ exceptions with SOAP serializer/deserializer for defined schema
Extensible to other technologies
Sentinel
Pluggable XDAQ extension (application)
Homogeneous use of XDAQ C++ exceptions
Protocol independent
SOAP/XML
Binary I2O ( with network configurability)
Static or Dynamic configuration
use of XML extensibility
use with auto discovery
XDAQ III Topical Meeting

14. A Fault Tolerant EVB Example(I)
Readout Units
Builder Units
XDAQ III Topical Meeting

15. A Fault tolerant EVB Example(II)
Sentinel …
/* notify error to interested handler */
sentinel->notify(ru::exception::OutOfSynch)
“Event Builder Context”
Error are reported to the specified context …
/* call back with message containing error */
BU::onException(/* exception …*/) {
if ( exception == “ru::exception::OutOfSynch”)
// disable RUi
// log error }
XDAQ III Topical Meeting

16. Fault Tolerant EVB Example(III)
Faulty Readout Unit is masked
Event building process can continues
with uncompleted events
XDAQ III Topical Meeting

17. Supplemental information

18. Latest Releases FEDStreamer V1.0 (FED I2O Data Streamer)
XDAQ interface to GIII based on E. Cano drivers (required bigphys)
FRL headers support
New data format interface (EVB compatible)
SOAP and WEB interfaces
Pheaps V1.0 (Physical Memory Heaps)
new device drivers
multiple memory chunks
multiple processes
memory allocation protection
ready for general purpose physical allocation on 2.6 (no bigphys on 2.6)
XDAQ full packaged V3.0.1
support release for FEDStreamer V1.0 and Pheaps V1.0
XDAQ III Topical Meeting