1. LCG/Blueprint RTAG http://iguana.cern.chAugust 5-6, 2002 Lassi A. Tuura, Northeastern University CMS Components Generic Infrastructure Bits Lassi A. Tuura Northeastern University, Boston

2. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 2 Frameworks: Structure Federationwizards Detector/EventDisplay Data Browser Analysis job wizards Generic analysis Tools ORCA FAMOS Objytools GRID OSCAR COBRA Distributed Data Store & Computing Infrastructure CMStools Consistent User Interface Coherent basic tools and mechanisms Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

3. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 3 ODBMS GEANT 3 / 4 CLHEP PAW Replacement C++ Standard Library + Extension Toolkits C++ Standard Library + Extension Toolkits Frameworks: Layering Calibration Objects Calibration Objects Generic Application Framework Physics modules Grid-Uploadable BasicServices Adapters and Extensions Configuration Objects Configuration Objects Event Objects Event Objects (Grid-aware) Data-Products SpecificFrameworks Event Filter Reconstruction Algorithms Physics Analysis Data Monitoring Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

4. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 4 GEANT 3 / 4 Qt Classlib Open Inventor Open Inventor C++ Standard Library + Extension Toolkits C++ Standard Library + Extension Toolkits Generic Components Calibration Objects Calibration Objects Generic Application Framework Physics modules Grid-Uploadable BasicServices Adapters and Extensions Configuration Objects Configuration Objects Event Objects Event Objects (Grid-aware) Data-Products SpecificFrameworks Event Filter Reconstruction Algorithms Physics Analysis Data Monitoring Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

5. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 5 Quick IGUANA Overview v A generic toolkit for user interfaces and visualisation r Builds on existing high-quality libraries (Qt, OpenInventor, Anaphe, …) r Used to implement specific visualisation applications in other projects r Used in CMS and D0 (and L3—pretty static) r Portable: UNIX and Windows actively used – Few limits to porting to new platforms v Main technical focus: provide a platform that makes it easy to integrate GUIs as a coherent whole, to provide application services and to visualise any application object r Many categories / layers: GUI gadgets & support, application environment, data visualisers, data representation methods, control panels, … r Designed to integrate with and into other applications r Apart from a tiny kernel, virtually everything else is in plug-ins – Plug-ins and dynamic linking are 100% orthogonal Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

6. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 6 Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components Quick IGUANA Overview Integration Action Visualise Results, Modify Objects, Further Interaction

7. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 7 Building Blocks v Basic services layer r Portability + utilitiesClasslib r GUI and 2D graphicsQt r 3D graphicsOpenInventor, OpenGL v Internal utility services r Plug-in manager – Generic base – Concrete implementation – (More details later on) r Internal state management and service components – (Components, abstract interfaces and concrete classes) Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

8. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 8 Graphics Toolkits v GUI and 2D Graphics: Qt r Easy to use r Very rich functionality, lots of community experience r Very portable, uses platform GUI conventions: looks “natural” r Excellent documentation r Free for our purposes v 3D Graphics: OpenInventor r Same as above—portability from OpenGL; both commercial and free exist r An incredibly rich toolkit! v Qt and OIV limited strictly to these parts of IGUANA r Not using Qt or OIV classes (containers and many other utilities) outside their respective “natural” graphics domains: other parts use other libraries r Reasons explained when discussing our component approach Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

9. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 9 Debugging + Robustness v DebugAssertions; Validation; Stack tracing; Programmatic core dump v LogRich debugging-oriented logging facility v SignalPOSIX; Standard C/C++; Win32 SEH; Fatal error handling and recovery v LowMemoryPoolGraceful low-memory protection v ErrorException chaining for increased error abstraction as errors are propagated through the application (much used in implementation) v SaveErrno Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

10. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 10 Major Tools v xtype_info r Extensible std::type_info replacement r Minimal meta-class information v MultiMethod r Multi-method dispatcher r Works with arbitrary argument lists – “Arbitrary”: Any polymorphic class can be used as “virtual” argument; any number of “virtual” and non-virtual arguments can be dispatched – Macro sugar for up to 3 virtual arguments and 2 non-virtual args r Optimised to be storage and performance efficient even if hundreds of classes and methods exist v Registry r Sort of like Windows registry but concept modified to actually work Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

11. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 11 System Abstraction v SharedLibraryShared library loading, unloading; list of currently loaded libraries v Time, TimeSpan  s-resolution calendar time, conversions, formatting, calculations v TimeInfoHigh-resolution (ns) monotonic time, system and process times v HostInfoHost itself, DNS lookups v ProcessInfoPID etc v ResourceInfoProgram + system resource info, limits v SystemInfoSystem information: memory, os, … v UserInfoUsers: home dir, login, real name, … Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

12. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 12 Bits & Numerics v BitIteratorIterating over data as bits v BitOpsCollection of optimised bit magic operations v BitPatternConstruction of bit patterns v BitTraitsBit type properties of integral types v IntBitsType selection by required number of bits: {exact, fast} x {signed, unsigned} v IntTraitsPoor man's std::numeric_limits Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

13. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 13Miscellaneous v Environment, ShellEnvironment v Callback, Hook, HookList v MapIterator v PODVector v Range v Sequence v SearchPath v Unwind v UUID v Utility Macros Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

14. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 14 I/O: Basics v IOChannelAbstraction of a non-seekable I/O sources and targets v StorageAbstraction of a seekable (file-like) storage v IOBuffer, IOSize,Basic I/O insulation: system IOOffsetcharacteristics (e.g. LFS) v DirIteratorIterator for reading directory contents like readdir() Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

15. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 15 I/O: Files, File Systems v File, FileAcl, FileMapping r File operations r File and anonymous memory mapping r High-throughput I/O channel transfers (e.g. file to socket) v Filename r Filename abstraction r Existence, type, permissions checking r Core support for file backup, versioning, autosaving, locking r Filename expansion, substitution, abbreviation, completion, globbing r File rename, move, copy r Secure temporary file handling r Current working directory r Description of system file name conventions Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

16. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 16 I/O: IPC v IOChannelPairPair of I/O channels (e.g. pipe or socket pair) v SocketAll socket stuff (TCP, UDP, UNIX, …) r InetSocket, InetServerSocket, InetAddress r LocalSocket, LocalServerSocket, LocalAddress r SocketPair v NamedPipeFile system FIFOs v PipeAnonymous FIFOs v PipeCmd“popen()” v IOSelectorI/O switch a la “select” (implementations available: select, poll, /dev/poll, real-time signals, sigio; Win32 select simulation, overlapped I/O (completion ports?)) Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

17. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 17 I/O: High-Level Streams v Input/Output Streams r Base interface r Buffered r Filtering r Linked r Pushback (input only) r C++ Standard (std::i/ostream) r C Standard (FILE *) r Storage (cf. I/O Basics) r IOChannel (cf. I/O Basics) v Storage implementations r Memory r C++ Standard r C Standard Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

18. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 18 Compression + Archives v Data checksums r Interface, Adler32, CRC32 (MD4, MD5?) v Compressor and decompressor engines r Interface, BZIP, ZLIB r Generic interactive engine v Input & output streams r Compressing and decompressing base interface r BZIP, ZLIB, GZIP, ZIP, TAR r Checksum v Archives files and their members r TAR, ZIP Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

19. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 19Modules v What’s a module? r An (extension) component to a framework (often dynamically loaded) r Framework knows only that a module extends a particular API (usually abstract interfaces), not how and what it needs to do so r Examples abound: Netscape plug-ins, Apache modules, Python and PERL modules, Illustrator and PhotoShop extensions, … r Already used in many ways in CMS projects: package builders, detector units, hit/digi readers and writers, and many algorithms are essentially extensions of APIs provided by COBRA, and “subframeworks” in ORCA v Why modules? r Eases management—framework doesn’t need to know lots of details about the modules, but can simply load them and gain additional capabilities r Helps to give symbolic names to capabilities and allow framework to reason about them: “here’s all you will need to read data product X” r Allows better services to external parties (Python, GRID, IGUANA, …) without having to put that knowledge into many applications Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

20. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 20 The Plug-In Manager v Maintains a list of directories with plug-in definitions v Plug-in = shared library with well-defined entry points r Self-describing through a query entry point r But really orthogonal to dynamic loading – Some or all plug-ins can be built into the program v Automatically detects new, changed or removed plug-ins r For new or changed ones loads the plug-in, queries for its capabilities, and unloads the plug-in r Maintains a cache of plug-in properties r Updates the cache if the directory is writable v When requesting information from the database, uses the cache instead of loading all the plug-ins all the time v A separate generic package r IGUANA architecture defines a specific implementation Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

21. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 21 Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

22. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 22 Abstract or Concrete? v Abstract or concrete implementations? r We use both: a lot of abstract interfaces and decoupling at the bottom level, more concrete implementation usage on the top r If you need a specific service, you use it directly – Access through shared state object which records references to “application state extensions”, but knows nothing about them – Clients ask specific services to look themselves up in the state – Services vary in “abstractness”: current selection state does not need to know anything about GUI, object context menu is defined in terms of Qt – Only if the service has multiple alternative implementations the service has an abstract interface; otherwise clients use a concrete implementation  Services can and do have mini-plug-in architectures of their own  Proxies and envelopes sometimes better than abstract interfaces r Catch-all IgExtension for those mini-plug-in systems but it’s not an IInterface – Concrete plug-in manager defines the interfaces specific to its mission r Trying to keep things simple and straightforward! Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components

23. August 5-6, 2002 Lassi A. Tuura, Northeastern University http://iguana.cern.ch 23 External Components v Qt and OIV: use concrete implementations only r Not trying to encapsulate these in any way – … but restrict them to their “native domains” r Reason: build modularity firewalls in the system – No point in trying to abstract so rich toolkits – Major benefits in going for full integration – Changing package known to incur limited pain  We know exactly what we are using and from where  We know we can get rid of the original package  Plan: expend N months to swap with equally rich or better product Overview Graphics Classlib Plug-in Manager Components Overview Graphics Classlib Plug-in Manager Components