7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland1 Object Reconstruction for CMS Analysis The Design Implementation and Deployment of a Functional Prototype OO Reconstruction Software for CMS CHEP February 2000 David Stickland,Princeton University

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland2 The ORCA Project Project Started in Fall 1998 l Primary Goals è Supply the collaboration with high-quality tools for detector optimization, trigger and physics studies è In-depth prototyping of SW architectures and tools l Secondary Goals è Accomplish a transition to Object Oriented style (people and code) è Increase base of skilled developers è Allow development of a user base è Test-bed for deployment of other OO projects

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland3 Transition Strategy l Top Down design of entire project impractical è Gain experience by deploying the project in phases l ORCA 1 (Dec 1998) è Basic reconstruction in all sub detectors, tolerate wrapped Fortran l ORCA 2 (Spring 1999) è OO Implementations of all major components l ORCA 3 (Oct 1999) è Persistency implemented è Used for Trigger Studies è In “Production” l ORCA 4 (Feb 2000) è Satisfy “Functional Prototype” Milestone. è A “complete” reconstruction package More than a set of reconstruction algorithms! We are Here

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland4 Training l Developers: è Established a finite set of understandable rules è Summer 99: many developers followed CMS organized OO Analysis/Design and UML courses. To date more than 50 physicists and SW engineers have code in ORCA l Users: è Many users need to be able to use ORCA without wanting to know much about OO design è Very successful web-based and interactive tutorial given an CERN in December ‘ New users followed this introductory course

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland5 Worldwide Collaboration l Worldwide collaboration and accessibility is at the crux both now and when LHC runs. SW is written by a few people at each of many institutes SW is used by a few (often different) users at each of many other institutes è Project Management weekly meetings held as VRVS conference (late PM CERN time, early AM Ca. time) p Typical ~5 remote sites connect p US users also organized their own SW/ORCA meetings è SCRAM developed in CMS. Very effective tool for code distribution and build management. p ORCA installed at steadily growing number of remote sites: Caltech, Fermilab, Moscow, Ecole Polytechnique, INFN (Bari)...

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland6 Persistency l Currently we use GEANT3 simulation via CMSIM package. Migration to GEANT4 during 2000 l Digitization is very expensive in CPU (200 min-bias events for each signal event at high-luminosity) l Objectivity Database è Use of Database is transparent to user program. No user code needs to be changed. è Objectivity handles etc, do not appear in user code è All Objectivity access is via CARF Framework l ORCA 3: Calorimeter and Muon digits persistent

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland7 ORCA and HLT l Trigger (HLT)/Physics studies have been first “user” for ORCA. l In October 99 we ran ORCA3 as a production job reading CMSIM signal and minimum bias files and filling two large databases with persistent ORCA results. è Used by Physicists studying HLT p 19 CPU’s running production (Linux and Sun) p Two production federations p Multiple AMS to reduce interference / timeouts p Automatic tape backup to HPSS as jobs complete p Automatic copying to two user federations p 30k events/day processed (>700GB in Obj/Db) è Ran essentially faultlessly

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland8 First HLT studies with ORCA Pythia Zebra files with HITS ORCA Digitization Objectivity Database HEPEVT ntuples CMSIM ORCA user Analysis ntuples PAW User Analysis ORCA ntuple production MC Prod. DB pop. User Analysis

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland9 Turn-over curves of DT Trigger Very preliminary!

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland10 March 2000 HLT Production Plans l 2M events ORCA reconstructed with high-luminosity pile-up l 2-4 Tera-Bytes in Objectivity/Db l 400 CPU-weeks l ~6 Production-Units è import existing catalogs, è schedule and monitor jobs è ooTidy and copy to hpss è export new catalog and system files l ~1-2 Production Units off CERN site l Copy of all data at CERN in hpss, use of IT/ASD AMS- backend to stage data to ~1TB of disk pools l Mirroring of Data to a few off-site centers, including trans- Atlantic Users want (need!) now what they were promised for

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland11 ORCA Production 2000 Signal Zebra files with HITS ORCA Digitization (merge signal and MB) Objectivity Database HEPEVT ntuples CMSIM HLT Algorithms New Reconstructed Objects MC Prod. ORCA Prod. HLT Grp Databases ORCA ooHit Formatter Objectivity Database MB Objectivity Database Catalog import Objectivity Database Objectivity Database ytivitcejbOesabataD Mirrored Db’s (US, Russia, Italy..)

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland12 CARF Experiences l CARF Architecture: On-demand reconstruction (see V.Innocente talk -this session) è Multi-layered applications p read CMSIM files, fill ooHit Db’s p merge min-bias and signal ooHit’s, create “crossing”, digitize p read ooDigi’s, create new RecObj p filter on meta-data, create new event-collections è All possible within one program but separate applications simplify production è Runtime loading p simplifies users responsibilities, application is built as actions are demanded p package initializers know the dependencies p accessing a new Db object triggers library loading

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland13 On-Demand & Production l On demand processing makes user code very simple è “give me an iterator over jets of type 1 constructed from Calorimeter recHits”: p jet algo 1 requests iterator over Calo recHits p Calo hit reconstructor requests iterator over CaloDigi p Calo Digitizer requests iterator over caloHits p Calo Hit Maker processes CMSIM files to extract GEANT3 hits è process branches to Db at any level that has been previously completed and made persistent è user needs to know nothing about deeper levels l Production process also profits from multi-layered approach.

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland14 Stability and Robustness l We have made very rapid progress due to (meeting) an aggressive set of milestones. è Cost in stability and robustness l We feel compelled to embark on a phase of consolidation, design and process improvement è Stability è Quality testing and quality improvement è Coherence p to ease use rather than to enforce monolithic style è Formal requirements, design and implementation documentation p to disseminate understanding rather than to accumulate paper Professional Engineering Crucial

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland15 After ORCA 4 l Close the macro- life-cycle: analysis  design  implementation  testing  analysis… è Re-analyze requirements and assess current design and implementations è More emphasis on data handling and its impact on design l Prepare for Pre-production Software.

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland16 Software & Architecture l When project started, most people were worried about ways to bring on the physicists, develop the sub-detector software etc. è Important, major emphasis of the last year, but actually less critical in the long term l Engineering of the architecture, and crucially the data- handling issues, are really the critical items è Tracking algorithms can, and will, be rewritten many times. But having an architecture that allows and keeps track of plug-and-play is vital. è Even now we face very large datasets (multi TB). Production, automation, mirroring, evolution are (some of) the hard issues. Reconstruction is much more than the reconstruction code

7 Feb 2000CHEP '00. CMS/ORCA. Abstract A108. D.Stickland17 Conclusions l In one year ORCA has gone from plan to reality è We have demonstrated it is possible to achieve an OO transition in a reasonably ordered way è We are supplying useful code for CMS design/optimization è The ORCA team now numbers about 50 developers è Persistency is the bedrock, not an addition è CARF Architecture proving suitable and flexible è We have made the transition from development code to production environment Now entering a critical phase of supporting deployed code and simultaneously undergoing full reanalysis and design...