1. STOREGLTE A Data Model Toolkit for the Atlas Software Architecture
Paolo Calafiura, C. Leggett,
H. Ma, D. Quarrie, S. Rajagopalan

2. StoreGate, a Data Model Toolkit…
Event Data Model
Hit
Reconstruction
Tracking
Analysis D H T
“The Event”
Ask to your night shift colleague in the counting room (assuming you remember how a counting room looks like) to explain how Track reconstruction works and he may come up with a diagram like this: the event raw-data digits are read in and decoded, wire hits are reconstructed and fed into some tracking algorithm, which creates the event tracks ready to be analyzed
Now show this diagram to a software architect and she will say: “Ah! A blackboard architecture: you must be dealing with some pattern recognition problem!”
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

3. The Blackboard Pattern
Knowledge
Source
Knowledge
Source
Knowledge
Source
operates on
execute
Data Store
read
write
wipe
Control
Framework
activates
Your architect would then go to the blackboard and draw the following object diagram.
The blackboard is the central data store. It provides an interface to read and write to it. Elements of the solution (tracks) and intermediate results (hits) are posted on the store by the relevant Knowledge sources (Tracking, Hit Reconstruction etc).
Knowledge sources are subsystems that solve specific aspects of the problem. They are activated in turn by the Control Framework, they examine the contents of the data store(s) and determine if they can contribute to the solution.
The Control Framework runs a loop that monitors the changes to the blackboard and decides what action to take next.
loop
nextSource
adapted from the ”Siemens” Patterns book
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

4. Data Store Pros and Cons
Event
Store
Track
Collection
Tracking
Analysis
Write
TrackColl
Read
Data Flow
Control Flow
Separates stable data from evolving algorithms
Algorithms can be independently added, updated or removed from a running system
Algorithms need to locate data in the store
Control Framework must be configured correctly to reach the desired solution
Separates stable data from evolving algorithms
algorithmic code is known to be the least stable part of “immature” software systems
it also the most “proprietary” one: domain specialists tend to have exclusive control of this code, while data objects which appears on the store have often multiple clients and multiple providers
Algorithms need to locate data in the blackboard
Once you separate the data from the algorithms producing them you need to identify each data object with a “well known name” that allows the producers and the consumers to locate it
Algorithms can be independently added, updated or removed from a running system
this allows to explore in parallel the solution space in the (never-ending) development stages with minimal impact (at least until run-time)
Control Framework must be configured correctly to reach the desired solution
if you remove the hit reco, the tracking and the analysis will still run ok technically speaking but won’t produce the desired results
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

5. StoreGate, a Data Model Toolkit…
What is StoreGate?
A type-centric Data Store
StoreGateSvc: Data Store client API (Gaudi service)
Data Object virtual proxy with lazy instantiation
A set of Data Object design tools
Handle for “automatic update” of time-varying data
Persistable inter-object references
Polymorphic STL-like Collections with defined element ownership
Memory pool for optimized memory allocation
Type-centric here means type-safe (static checking) and that the data registry key contains the object type
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

6. StoreGateSvc: Type-centric Store
retrieve(type, key)
record(type, key)
Type
StoreMap
Templated
read/write
Key
ProxyMap
All types
in store
DataProxy
All instances of
a given type
Virtual proxy
Data Object
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

7. Identifying Data Objects
Data object type
User defined key (well-known name)
Default data object
Use Data Object History (provenance)
Reduce user-defined keys
Not yet implemented My
Tracking
provenance
Write
“MyTracks”
“MyTracks”
TrackCollection
Event
Store
“YourTracks”
TrackCollection
Read
“MyTracks”
Read
“YourTracks” My
Analysis
Your
Analysis
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

8. StoreGate, a Data Model Toolkit…
StoreGateSvc API
To record (write) a data object
StatusCode sc =
p_sgSvc  record(pTrkColl, TrkCollKey);
To retrieve (read) a data object
p_sgSvc  retrieve(pTrkColl[,TrkCollKey]); p_sgSvc  retrieve(beginTC, endTC);
where :
pTrkColl is a (const) pointer to a TrackCollection
TrkCollKey is an identifier for a data object. It can be a simple string like: “MyTrackCollection”
beginTC, endTC are iterators over all TrkColl instances in the store
No detail just mention that All these methods are templated in TrackCollection.
Perhaps also the “retrieveAll”
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

9. Control Object Access and Creation
IConverter
createObj
createRep
DataProxy
Virtual Proxy that collaborates with Gaudi (one day, POOL) transient-persistent conversion mechanism
In the future may use history info to reconstruct data object on-demand
Lazy data object instantiation (on client access)
Support/enforce const-correctness
StoreGateSvc allows to “freeze” a data object ready to be used or written out
DataProxy
accessData
creates
DataObject
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

10. DataHandle for Time-Varying Data
Purpose:
Hide store dynamics from client code
Handle is bound to DataProxy during initialization
Automatically updated (again using lazy instantiation) when Data Object enters a new validity range
Use cases:
alignment data
calibration objects
detector description information
anything which has an associated validity range
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

11. Asynchronous update of Calibration Data
Alg
Handle
Storegate
Proxy
PPSvc
IOVSvc
IOVDbSvc
PersSvc
CondDB
NovaCnv
regHandle(handle,key)
bind(handle,key)
initialize
bindHandle(handle)
regProxy(proxy)
regFcn(this,&Fcn,handle,key) 
accessData()
updateAddress(TA)
updateAddress(TA)
getData(key) T*
DataObj*
IOA
IOA
IOV,string
setRange(clID,key,IOV)
buildIOA(string)
buildIOA(string)
IOA
partial IOA
execute
setAddress(IOA)
Shock and awe, don’t discuss
createObj(IOA) 
DataObj* T*
reset()
reset()
async
callback(dbKey)
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

12. StoreGate, a Data Model Toolkit…
Work with User Types
What is a Data Object? An object recorded into StoreGate (SG)
Only constraint is to be an “Assignable”
No base class for SG data objects
SG uses generic programming to work with user types adapting to the interface they provide (“export”)
Same strategy used for SG collections and inter-object relationships
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

13. Inter-Object Relationships
Need Persistable Ptr that
is technology-independent
does not require Hit to derive from any base class
supports deferred access, creating the pointed-at object upon dereferencing
allows to point at
SG primary objects (Data Objects)
SG secondary objs (elements of a Data Object)
StoreGateSvc
registry
<<DataObject>>
HitCollection
unknown
DataLink is not enough
Links to secondary objects
Most often relationship is to an element of a data object (which is usually a container)
Locating one of these elements requires container-specific indexing info
ElementLink
Loosely speaking a DataLink to the container with a container-specific Indexing Policy allowing to locate the element
There are pre-defined Indexing Policies for all STL-like containers and a few Atlas containers
<<Element>>
Hit
Track
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

14. DataLink and ElementLink
StoragePolicy
getDataObj
IndexingPolicy
locateElement
Policy-based design (parameterized inheritance)
ElementLink is a composition of policies
Default Indexing Policy generated according to host container type
IndexingPolicy
StoragePolicy
Host
ElementLink
getElement
StoragePolicy
Host
DataLink
getDataObj
Host
GenerateIndexing
Policy
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

15. StoreGate, a Data Model Toolkit…
The Need for Speed
Atlas L2 and EF are using Athena & SG
share Algorithms and Services with offline
Performance Studies
raw-data object access
Speed-up > x10
Optimized containers
Key hashing
A lot of work
random access
perfect hash
record
Fixed number of digits/event (1M)
Yellow cleanup
Green IDC sequential
Black SG sequential
write ~ 5 ms (1GHz)
read ~ 20 ms
optim read ~500 ns
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

16. Back to the Drawing Board
Missed reqs and implementation surprises
Writing concrete and reading abstract
“Symlinks” (Concrete DObj)  (ABC)
Proper solution when reflection available
Templates and physical design
Use PIMPL idiom to hide template implementation
Keep libraries and executables small
Templates and compilers
Undetected compilation errors
unit tests also improve compilation coverage
Unreadable compiler msgs (100+ lines)
boost concept-checking and compiler assertions
This is the blackboard in Hong’s BNL with the original SG API
Poor physical design came from heavily templated code
coupling via “loaded” header files
most implementation changes in SG required massive recompilations
Unreadable compiler error messages
users of STL know what I am talking about
still a nuisance
Undetected compilation errors
template compilation is JIT, SG errors appeared later in the integration stage
unit tests now exercise (and force compilation of) every single method
occasional problems with the odd combination of arguments
Writing concrete and reading abstract
the maker of say a cell collection knows which concrete collection it is making (say LArCellCollection)
downstream analysis algorithms may want to work in terms of abstract CellCollection (say to combine results from multiple calorimeters)
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

17. StoreGate, a Data Model Toolkit…
Status and Outlook
SG blackboard is a mature product
Used extensively by Atlas developers offline and online
Performance and memory footprint meet requirements
Data Object tools (links, containers, DataHandle) complete but newer: will need some fine tuning
Need tools to exploit Athena History Objects
to Identify Data Objects
(perhaps) to steer Reconstruction-on-demand
Integration with LCG
New data link storage policy based on POOL Refs
Use SEAL Dictionary for Inheritance info
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

18. Volume ON

19. StoreGateSvc in Athena/Gaudi
Application
Manager
Converter
Sequencer
Converter
Converter
Event
LoopMgr
Event Store D H T
Data
Files
Message
Service
Persistency
Service
StoreGateSvc
JobOptions
Service
Algorithm
Algorithm
Particle Prop.
Service
StoreGateSvc
Data
Files
Persistency
Service
Athena-Gaudi Object Diagram
The main components of the architecture can be seen in the slide.
The green boxes play the role of the Control component in the blackboard diagram
The Algorithms are the Knowledge sources
The StoreGateSvc provides an API to access Data Store (the blackboard)
The conversion mechanism (in blue) act as bridge between the framework and the specific technology of persistency mechanism.
Other algorithm services are in grey
Detector
Store
Other
Services
Auditors
Histogram
Service
Scripting
Service
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…

20. StoreGate, a Data Model Toolkit…
IOV Class Example
class myIOVClass {
public: virtual StatusCode CallBackFcn(list<string> &dbKeys); private: const DataHandle<CalAlign> m_align1; const DataHandle<CalAlign> m_align2;
myIOVAlg2* that; };
myIOVClass::initialize() { p_IOVSvc->regHandle(m_align1, dBkey1); p_IOVSvc->regFcn(&myIOVClass::CallBackFcn, this, m_align1, dBkey1); p_IOVSvc->regFcn(&myIOVClass::CallBackFcn, this,
m_align2, dBkey2);
p_IOVSvc->regFcn(&myIOVClass::CallBackFcn, this,
&myIOVClass2::OtherCallBackFcn, that); }
Note callbacks
CHEP 2003, March 24
// session category 7
StoreGate, a Data Model Toolkit…