Streaming Metadata H. Schellman July 20, 1999

Stream Primitives Descriptions for classes of filters Events will be classified by an OR of StreamPrimitives Represented by both bits and names Add new primitives - do not reassign bits!

L3TriggerFileFamily Physical Stream Type EventFileTape

StreamTypes StreamTypes exist at 3 levels Each L3 filter has a StreamType assigned when filters are parsed Each L3 Trigger has a StreamType assigned when the trigger list is parsed. Either by OR of filter StreamTypes or override. Each Event which passes L3 has a StreamType generated by L3 scriptrunner by OR’ing the StreamTypes of the passed L3Triggers.

Physical Streams A general Physical Stream consists of a set of disk files written by the data logger We expect < 30 such streams. Such Disk files can be grouped onto tape via File Families We expect < 10 File Families There are 2*N possible StreamTypes We must map these to Physical Streams for data logging L3 logic which generates StreamType from passed L3 Triggers will also generate Physical Stream tags for the logger. Wyatt wants at least 10 Wyatt wants at least 10

Where Streaming Information is Generated StreamPrimitives - configuration file/DB - add only Filter/StreamType Map - filter.rcp parsing Trigger/StreamType - Trigger list parsing Event/StreamType - L3 scriptrunner after L3 accept StreamType/Physical Stream - configuration file/DB Physical Stream/File Family - configuration file/DB StreamType to Physical Stream map requires knowledge of rates may be possible to semi-automate it.

Where Streaming Information is Used StreamPrimitives - scriptrunner/offline L3Trigger/StreamType - scriptrunner/offline Event/StreamType - Data Logger/offline StreamType/Physical Stream - Data Logger/ offline Physical Stream/File Family - RIP/SAM? Offline request for a given trigger requires Trigger->Physical Stream map.

Where streaming information is stored L3Chunk contains L3 Triggers, StreamType, Physical Stream, File Family for a given event Event Database contains L3 Triggers, StreamType?, file informationfor a given event File Database contains Physical Stream, StreamTypes?, File Family for a given file Trigger Database contains –StreamPrimitives List –L3Trigger StreamType Map for each trigger list instance Some Database contains –StreamType Physical Stream Map –Physical Stream File Family map Xiao Ling Fan working on this Xiao Ling Fan working on this Schellman?

Tables we need

Luminosity Metadata When doing data analysis, we need to add up the luminosity corresponding to all of the luminosity blocks in the data sample. If a file is lost from a data sample we need to know which data blocks were lost. So there needs to be a file -> lumblocks mapping. If it is possible for a file to contain partial lumblocks, we also need the L0 time stamps for the begin/end of the file so we can prorate fractional lumblocks. Decided didn’t need L0 tag Decided didn’t need L0 tag

Luminosity Block contents Run # Lum Block # L0 scaler tag for begin and end of Lum Block Scalers for that lumblock (may not be needed) Integrated lum for that lumblock - lotsa words Optional stuff? # of events in each physical stream for that lum block. Names of any missing physical streams for that lum block Decided didn’t need L0 tag Decided didn’t need L0 tag

File information about Luminosity Lum Blocks associated with file L0 beam scaler tag for start of file L0 beam scaler tag for end of file Can use the scaler tags to prorate partial lum blocks. Decided didn’t need L0 tag Decided didn’t need L0 tag

Event Information about Luminosity Lum block # L0 scaler tag Decided didn’t need L0 tag Decided didn’t need L0 tag