KID - KLOE Integrated Dataflow Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor KLOE Data Flow Online disk Memory Tape library DB Detector Offline disk Staging disk Beijing, CHEP 2001 Presented by Sfiligoi Igor

KLOE Software Environment Online monitors Reconstruction processes Analysis programs Beijing, CHEP 2001 Presented by Sfiligoi Igor

KLOE Software Environment Online monitors Memory buffers Reconstruction processes Files on online disks Analysis programs Files on staging disks For debugging purposes Local files Copied on demand Beijing, CHEP 2001 Presented by Sfiligoi Igor

KID (KLOE Integrated Dataflow) Makes applications data-source independent Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor KID package C library No kernel changes needed Only adapted applications can use it Mimics standard stream operations open/get/close functions Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor KID package (cont.) Also provided FORTRAN interface A_C input module Most analysis programs are based on A_C Totally transparent to final user Beijing, CHEP 2001 Presented by Sfiligoi Igor

merge:ybos:file1,ybos:file2,ybos:file3 Data Source Section URI – based All KLOE data sources are accessible New can be added if needed ybos:myfile spy:/tmp/B_1_1 merge:ybos:file1,ybos:file2,ybos:file3 Beijing, CHEP 2001 Presented by Sfiligoi Igor

URI advantages serial:port1 remote:ybos:myfile@mynode Very flexible Defining a new protocol section, a new data source can be accessed without modifi- cations to the application serial:port1 Different data sources can be combined inside the same URI remote:ybos:myfile@mynode Very flexible Easily extensible Fairly easy to learn and remember Simple text string Human readable dbdatarec:run_nr=16932 and stream_code=‘rad’ Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor KID Internals C library Modular Each protocol(data source) implemented in a different module Function pointers used to execute the right code, once the protocol has been parsed C++ or other OO language not used in order to be more general Multi-threading concurrency managed in the core module Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor KID modules Local resource modules Remote resource module Data handling interfaced module Support modules Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor KID modules Local resource modules A file ybos:filename?options A memory buffer spy:buffername?options Remote resource module Data handling interfaced module Support modules Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor KID modules Local resource modules Remote resource module Any remote resource remote:URI@node?options Uses a KID-specific multithreaded daemon to export resources from the nodes Data handling interfaced module Support modules Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor KID modules Local resource modules Remote resource module Data handling interfaced module SQL query-based db:SQL-query?options Files obtained from the query are requested to the data handling system Frequently used shortcuts dbTYPE:where-part-of-a-SQL-query?options Support modules Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor KID modules Local resource modules Remote resource module Data handling interfaced module Support modules Merge several sources in one merge:(URI1,…,URIn)?options Find first available source try:(URI1,…,URIn)?options Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor Creating new modules Free text URI parameter gives plenty of flexibility Several text parsing functions are part of the KID library Modular structure Minimal source code overhead Developer can work on real problems Beijing, CHEP 2001 Presented by Sfiligoi Igor

Presented by Sfiligoi Igor Conclusions KID is now used in all the production and most of the analysis applications Allows access to ALL KLOE data sources After a short initial introduction period, now used profitably by almost all KLOE members Beijing, CHEP 2001 Presented by Sfiligoi Igor