1. KID - KLOE Integrated Dataflow
Beijing, CHEP 2001
Presented by Sfiligoi Igor

2. Presented by Sfiligoi Igor
KLOE Data Flow
Online disk
Memory
Tape library DB
Detector
Offline disk
Staging disk
Beijing, CHEP 2001
Presented by Sfiligoi Igor

3. KLOE Software Environment
Online monitors
Reconstruction processes
Analysis programs
Beijing, CHEP 2001
Presented by Sfiligoi Igor

4. KLOE Software Environment
Online monitors
Memory buffers
Reconstruction processes
Files on online disks
Analysis programs
Files on staging disks
For debugging purposes
Local files
Copied on demand
Beijing, CHEP 2001
Presented by Sfiligoi Igor

5. KID (KLOE Integrated Dataflow)
Makes applications data-source independent
Beijing, CHEP 2001
Presented by Sfiligoi Igor

6. Presented by Sfiligoi Igor
KID package
C library
No kernel changes needed
Only adapted applications can use it
Mimics standard stream operations
open/get/close functions
Beijing, CHEP 2001
Presented by Sfiligoi Igor

7. Presented by Sfiligoi Igor
KID package (cont.)
Also provided
FORTRAN interface
A_C input module
Most analysis programs are based on A_C
Totally transparent to final user
Beijing, CHEP 2001
Presented by Sfiligoi Igor

8. merge:ybos:file1,ybos:file2,ybos:file3
Data Source Section
URI – based
All KLOE data sources are accessible
New can be added if needed
ybos:myfile
spy:/tmp/B_1_1
merge:ybos:file1,ybos:file2,ybos:file3
Beijing, CHEP 2001
Presented by Sfiligoi Igor

9. URI advantages serial:port1 remote:ybos:myfile@mynode Very flexible
Defining a new protocol section, a new data source can be accessed without modifi-
cations to the application
serial:port1
Different data sources
can be combined inside the same URI
Very flexible
Easily extensible
Fairly easy to learn and remember
Simple text string Human readable
dbdatarec:run_nr=16932 and stream_code=‘rad’
Beijing, CHEP 2001
Presented by Sfiligoi Igor

10. Presented by Sfiligoi Igor
KID Internals
C library
Modular
Each protocol(data source) implemented in a different module
Function pointers used to execute the right code, once the protocol has been parsed
C++ or other OO language not used in order to be more general
Multi-threading concurrency managed in the core module
Beijing, CHEP 2001
Presented by Sfiligoi Igor

11. Presented by Sfiligoi Igor
KID modules
Local resource modules
Remote resource module
Data handling interfaced module
Support modules
Beijing, CHEP 2001
Presented by Sfiligoi Igor

12. Presented by Sfiligoi Igor
KID modules
Local resource modules
A file
ybos:filename?options
A memory buffer
spy:buffername?options
Remote resource module
Data handling interfaced module
Support modules
Beijing, CHEP 2001
Presented by Sfiligoi Igor

13. Presented by Sfiligoi Igor
KID modules
Local resource modules
Remote resource module
Any remote resource
Uses a KID-specific multithreaded daemon to export resources from the nodes
Data handling interfaced module
Support modules
Beijing, CHEP 2001
Presented by Sfiligoi Igor

14. Presented by Sfiligoi Igor
KID modules
Local resource modules
Remote resource module
Data handling interfaced module
SQL query-based
db:SQL-query?options
Files obtained from the query are requested to the data handling system
Frequently used shortcuts
dbTYPE:where-part-of-a-SQL-query?options
Support modules
Beijing, CHEP 2001
Presented by Sfiligoi Igor

15. Presented by Sfiligoi Igor
KID modules
Local resource modules
Remote resource module
Data handling interfaced module
Support modules
Merge several sources in one
merge:(URI1,…,URIn)?options
Find first available source
try:(URI1,…,URIn)?options
Beijing, CHEP 2001
Presented by Sfiligoi Igor

16. Presented by Sfiligoi Igor
Creating new modules
Free text URI parameter gives plenty of flexibility
Several text parsing functions are part of the KID library
Modular structure
Minimal source code overhead
Developer can work on real problems
Beijing, CHEP 2001
Presented by Sfiligoi Igor

17. Presented by Sfiligoi Igor
Conclusions
KID is now used in all the production and most of the analysis applications
Allows access to ALL KLOE data sources
After a short initial introduction period, now used profitably by almost all KLOE members
Beijing, CHEP 2001
Presented by Sfiligoi Igor