SEAL Developers 3, 2003 Lassi A. Tuura, Northeastern University CMS Components Plug-in Manager Lassi A. Tuura Northeastern University, Boston

March 3, 2003 Lassi A. Tuura, Northeastern University 2 CMS Frameworks: Structure Federationwizards Detector/EventDisplay Data Browser Analysis job wizards Generic analysis Tools ORCA FAMOS Objytools GRID OSCAR COBRA Distributed Data Store & Computing Infrastructure CMStools Consistent User Interface Coherent basic tools and mechanisms

March 3, 2003 Lassi A. Tuura, Northeastern University 3 ODBMS GEANT 3 / 4 CLHEP PAW Replacement C++ Standard Library + Extension Toolkits C++ Standard Library + Extension Toolkits Frameworks: Layering Calibration Objects Calibration Objects Generic Application Framework Physics modules Grid-Uploadable BasicServices Adapters and Extensions Configuration Objects Configuration Objects Event Objects Event Objects (Grid-aware) Data-Products SpecificFrameworks Event Filter Reconstruction Algorithms Physics Analysis Data Monitoring

March 3, 2003 Lassi A. Tuura, Northeastern University 4Modules v What’s a module? r An (extension) component to a framework (often dynamically loaded) r Framework knows only that a module extends a particular API (usually abstract interfaces), not how and what it needs to do so r Examples abound: Netscape plug-ins, Apache modules, Python and PERL modules, Illustrator and PhotoShop extensions, … r Used in many ways in CMS projects: package builders, detector units, hit/digi readers and writers, and many algorithms are essentially extensions of APIs provided by COBRA, and “subframeworks” in ORCA v Why modules? r Eases management—framework doesn’t need to know lots of details about the modules, but can simply load them and gain additional capabilities r Helps to give symbolic names to capabilities and allow framework to reason about them: “here’s all you will need to read data product X” r Allows better services to external parties (Python, GRID, IGUANA, …) without having to put that knowledge into many applications

March 3, 2003 Lassi A. Tuura, Northeastern University 5 Plug-ins in CMS: Background v Much of CMS software has “done” plug-ins for a while r IGUANA: a few years – The origin of the model described here – Apart from a tiny kernel, virtually everything else is in plug-ins – Plug-ins and dynamic linking are 100% orthogonal r COBRA: probably longer than I have been in CMS – Essentially the same model as IGUANA, technically not so refined but functionally much the same – Planned to use IGUANA system but then SEAL came along r We have a few years of experience with this model and like it v Basic principles r Framework outside r Simplicity r Lightweight

March 3, 2003 Lassi A. Tuura, Northeastern University 6

March 3, 2003 Lassi A. Tuura, Northeastern University 7 Implementation Features v Maintains a list of directories with plug-in module definitions v Plug-in module = shared library with well-defined entry points r Self-describing through a query entry point r But really orthogonal to dynamic loading – Some or all plug-ins can be built into the program v Automatically detects new, changed or removed modules r For new or changed ones loads it, queries its capabilities, and unloads it r Maintains a cache of module properties r Updates the cache if the directory is writable v When requesting information from the database, uses the cache instead of loading all the modules all the time v Two packages: generic base plus specific implementation r The model described here is the specific one, the generic one has no model

March 3, 2003 Lassi A. Tuura, Northeastern University 8 Technical Basics v Any C++ type can be created through plug-in mechanism v Any module/package can define a new plug-in category r Also other plug-in modules (the only way we use it, our kernel is empty) r Different plug-in categories in same or different projects coexist peacefully v Separate dynamic loading and factory interfaces r Clients see only a (powerful) factory r Modules see a relatively simple factory provider interface r Physical decomposition choices have no impact on interfaces r Dynamic loading is optional, changing packaging does not affect code v Automated caching r Program can reason about run-time environment without loading it all r Optional and transparent, failure to cache only shows up as (a big) slowdown v No external meta data: every plug-in module is self-contained r Build system automatically provides a list of available plug-in modules

March 3, 2003 Lassi A. Tuura, Northeastern University 9 Implementation Structure v Plug-in manager r (Green means user-visible class) r Practical layers: concrete object factories – IgBrowserDB, IgSiteDB, … r Concrete implementation – IgPluginDB, IgPluginDirectory, IgPlugin, IgPluginDef – IgPluginDBViewBase, IgPluginDBView, IgPluginDBItem r Generic caching and module-loading base – IgPluginCache, IgPluginBase – Knows how to manage plug-in modules (shared libraries) – Maintains a plug-in information cache per directory – The cache is an automatically updated text file (provided it is writable)

March 3, 2003 Lassi A. Tuura, Northeastern University 10 Plug-In Cache Object Factory Object Factory Implementation Structure… v Each module has three entry points: attach, detach, and query r Query just caches information: the module capabilities are recorded in the cache and the in-memory database that merges all caches r Attach and detach happen on module load/unload: module installs factories for various named plug-in objects it can create v Application state management and service components r Object location service: how you find all those plug-ins you’ve created r Won’t get into this in this presentation, but related and already involved Plug-in Database Plug-In Cache Module Object Factory Attached Unattached

March 3, 2003 Lassi A. Tuura, Northeastern University 11 System API Base Manager Module Providers Module Providers Factory Clients Factory Clients Cache View Item Structure

March 3, 2003 Lassi A. Tuura, Northeastern University 12 Classes #1 v Basic machinery r IgPluginCache: Maintains raw cache data per directory – Automatic update mechanism – Means to access per-module raw cache data (= “descriptors”) r IgPluginBase: Base class for a loadable module – Ability to load, unload, query, check if attached, check entry points, … – A derived class must implement actual behaviour, no meaning to given to the above, just services that typical plug-in managers will find useful r SharedLibrary (from classlib): Dynamic loading support

March 3, 2003 Lassi A. Tuura, Northeastern University 13 Classes #2 v Syntactic sugar: our plug-in manager model r IgPlugin, IgPluginDef: Module wrapper – Module providers derive IgPluginDef – Unit for managing the cached data r IgPluginDirectory: A directory of our modules – Derives IgPluginCache and instantiates IgPlugin – Mechanism to write to and reconstruct from cache r IgPluginDB: Collection of plug-in data – Collection of IgPluginDirectories – Collection of plug-in views (categories) r IgPluginDBViewBase, IgPluginDBView – An actual view of the plug-in database r IgPluginDBItem: Cached item – Interacts with the IgPluginDBView through IgPlugin and IgPluginDB – Also reconstructed from the cache (managed by IgPluginDB and view)

March 3, 2003 Lassi A. Tuura, Northeastern University 14 Classes #3 v Practical layer r IgBrowserDB, IgBrowserInfo – Browser factory, details about what is cached about browsers (= name) – Note! This is the only level at which the actual object type, factory argument lists, constructor arguments etc. are defined and used! r IgSiteDB, IgSiteInfo – etc. r …

March 3, 2003 Lassi A. Tuura, Northeastern University 15 Module Provider Interface class MyPluginDef : public IgPluginDef { public: virtual voidquery (void); virtual voidattach (void); }; DEFINE_IGUANA_PLUGIN (MyPluginDef); void MyPluginDef::query (void) { IgBrowserDB::Def ::declare (this); } void MyPluginDef::attach (void) { IgBrowserDB::Def ::installFactory (this); }

March 3, 2003 Lassi A. Tuura, Northeastern University 16 Module Provider Interface // COBRA way of doing the same thing static PKBuilder me;

March 3, 2003 Lassi A. Tuura, Northeastern University 17 Factory Client Interface void SomeFoo (void) { IgState*state = …; IgSite*site = …; IgBrowser*browser = IgBrowserDB::get () ->instantiate (“MyFoo”, state, site); }

March 3, 2003 Lassi A. Tuura, Northeastern University 18 Category Definition class IgBrowserInfo : public IgPluginDBItem { public: typedef IgBrowser Object; struct Factory { virtual ~Factory (void); virtual Object *create (IgState *state, IgSite *parent) = 0; }; template struct AutoFactory : Factory { virtual Object *create (IgState *state, IgSite *parent) { return new T (state, parent); } }; // Some methods (ctor, create(), attach(), factory()) } class IgBrowserDB : public IgPluginDBView { public: virtual IgBrowser *instantiate (const std::string &name, IgState *state, IgSite *parent); };