1. John Marshall, 1 John Marshall, University of Cambridge LCD-WG2, May 6 2010

2. John Marshall, 2 Pandora Redesign Whilst Pandora works very well, old code has reached a point where it is extremely difficult to extend. It is not flexible enough to try out new ideas and improvements... ILD Letter of Intent version of Pandora has been frozen and a new version is being written from scratch. This is much more than just a re-implementation, it is a major redesign; Pandora is now a framework for running decoupled particle flow algorithms: Increased flexibility, designed to make it easy to try out new ideas Independent of any specific software framework and any specific detector details Properly designed code, taking findings from previous PFAs into account, makes it easier to maintain Easier for other people to get involved; simple for users to create and run their own algorithms Pandora framework helps separate physics in particle flow algorithms from C++ memory management The new Pandora is a separate library, with no dependencies. A user application, in any framework, accesses the library via a simple C++ API (application programming interface). Will now give a very brief overview of new Pandora framework and summarise the current status (for more detail, please see slides presented at LCWS10)...LCWS10

3. John Marshall, 3 New Pandora Structure Specify Geometry Create Calo Hits Create Tracks Create MC Particles Register User Algorithms Clustering Algorithm Topological Association Algorithms Statistical Reclustering Algorithm Photon ID Algorithm Fragment Removal Algorithms Track-cluster Association Algorithms PFO Construction Algorithm Pandora Algorithm Manager Calo Hit Manager Cluster Manager MC Manager Geometry Helper Pandora Settings Track Manager Particle Flow Object Manager Get Particle Flow Objects User Application:Pandora Framework, can treat as “black box”: Pandora Algorithms: Pandora API Pandora Content API

4. John Marshall, 4 Pandora API To run Pandora, a user needs to write a small application in their chosen software framework. This application uses the PandoraAPI to supply Pandora with details of the detector geometry and of the calo hits and tracks in each event. Pandora then builds its own simple objects. Construction of these objects is simple; the user makes a Parameters class, fills the member variables and then calls the API Create function. Example member variables for a track: d0, z0, track state at start, track state at ECal, etc. All member variables must be specified, or an exception will be thrown when Create is called. The user can provide this information in any order, then call the API ProcessEvent function. Finally, user calls the API GetParticleFlowObjects function. Pandora API Pandora Calo Hit Manager Cluster Manager Track Manager MC Manager Particle Flow Object Manager User Application, e.g. ILDPandora PandoraApi::Track::Parameters parameters; parameters.m_d0 =...;... PandoraApi::Track::Create(pandora, parameters);

5. John Marshall, 5 Pandora Objects Calo Hit Position + normal vectors Calorimeter cell size Absorber material in front of cell Time of first energy deposition Calibrated energy (mip equivalent, EM, Had) Layer + pseudolayer Hit type + detector region Density weight Surrounding energy IsDigital, IsIsolated + IsPossibleMip flags Associated MC particle Associated user object Track 2D impact parameters Momentum at d.c.a Particle mass Charge sign Start track state End track state ECal track state ReachesECal flag List of track state projections to calorimeter surfaces Associated cluster Associated MC particle Associated user object PFO formation flag “Clusterless” PFO formation flag Cluster List of constituent calo hits, ordered by pseudolayer Mip fraction EM energy measure Had energy measure Corrected energy estimates Initial direction Current direction Result of linear fit to all hits in cluster Energy-weighted centroid ShowerStart layer Shower profile properties List of associated tracks Particle Flow Object PDG Code Charge Mass Energy Momentum List of tracks List of clusters Mixture of properties specified by user and value-added properties, but all simple and well defined physics quantities for use in particle flow algorithms.

6. John Marshall, 6 Pandora Managers Pandora Calo Hit Manager Cluster Manager Track Manager MC Manager Particle Flow Object Manager Pandora Managers are designed to store named lists of their respective objects. These objects can be accessed by the Pandora Algorithms, which perform the reconstruction. The algorithms interact with the Managers in a controlled way, via PandoraContentAPI, and the Managers perform the memory management. At any instant each Manager has a “current” list, which can be accessed by an algorithm. Parent algorithms can manipulate the current list in order to control scope and behaviour of daughter algorithms. The Managers store information about currently running algorithms so they can keep track of lists. Algorithms can use the PandoraContentAPI to modify lists and/or save new lists. Algorithms can use the API without worrying about how the managers work – separation of physics and C++ memory management! Pandora Content API e.g. Clustering Algorithm

7. John Marshall, 7 Pandora Algorithms In the new Pandora framework, the algorithms contain almost exclusively physics-driven code, alongside the following typical usages of the PandoraContentAPI: Create new clusters and particle flow objects Modify clusters, by adding hits, merging or deleting Access the current lists of Pandora objects Save new lists of clusters, calo hits or tracks Run a daughter algorithm, etc... Static helper functions are provided to perform tasks that are useful to multiple algorithms, and the Pandora algorithms are configured via xml and can be swapped in/out without recompiling. Pandora will provide a comprehensive library of built-in algorithms. However, we want other people to get involved: Easy for a user to create their own algorithm and we provide a template. User then calls API functions and implements their own algorithm logic. Once the algorithm settings are added to the xml file (an XmlHelper makes this easy), the algorithm will then run in the Pandora framework, without needing to rebuild the Pandora library. At LCWS10, presented details of re-implementation of Pandora algorithms, together with first low energy (91GeV) results. Will now quickly summarise these results, before discussing the most recent changes...LCWS10

8. John Marshall, 8 Completed Algorithms Looping tracks Cone associations Back-scattered tracks 9 GeV 6 GeV Layers in close contact 9 GeV 6 GeV 3 GeV Fraction of energy in cone Neutral hadron Charged hadron Photon Clustering Algorithm Topological Association Algorithms Photon ID Algorithms Fragment Removal Algorithms Track-cluster Association Algorithms PFO Construction Algorithm Cluster first layer position Projected track position Cone-based forward projective method

9. John Marshall, 9 Results at 91GeV First tests were performed for the ILD detector concept, using MC samples of approximately 10,000 Z  uds generated with the Z decaying at rest with E z = 91.2GeV. At this energy, all newly implemented code and framework is exercised, without (a strong) need for statistical reclustering and/or leakage corrections. For fair comparison, reclustering, leakage corrections and all use of track-relationship information was turned off in old Pandora (at the time, was still work in progress for new Pandora). Excellent agreement observed, by construction.  E / E (rms 90 ) = 3.79% All Events, 91GeV PFO energy sum, old vs. new

10. John Marshall, 10 Reclustering Algorithms that “steer” the reclustering have now been fully implemented and validated. Have deviated from old Pandora here, tidying the algorithms and more clearly defining the role of each. Introduces some small differences between new and old Pandora output. However, extensive validation shows that results remain statistically identical to old Pandora. 12 GeV32 GeV 18 GeV 30 GeV Track 38 GeV At high jet energies, performance degrades due to increasing overlap between hadronic showers from different particles: Pandora addresses this problem with statistical reclustering: Clusters that have been incorrectly merged together are identified via consistency of cluster energy and associated track momentum. Attempts are made to redistribute the hits by using different clustering parameters or entirely different clustering algorithms.

11. John Marshall, 11 Reclustering As expected, reclustering algorithms are very important at higher energies. All Events, 200GeV All Events, 360GeV All Events, 500GeV

12. John Marshall, 12 Track Relationships Pass/fail track pfo selection cuts Kink (or Split) Prong VV VV Cluster associated with track E = E daughter P = P daughter q= q daughter PID = PID parent E =  good daughters P =  P good daughters q= q parent PID = PID parent E = E good daughter P = P good daughter q=  q both daughters PID: examine track PIDs E =  E daughters P =  P daughters q=  q daughters PID: examine track PIDs New Pandora code allows track parent, daughter and sibling relationships to be specified. Uses this information when determining which tracks can be associated to clusters and when building charged PFOs. For ILD, extract track relationships from v  and kink finders in MarlinReco. Charged PFOs are built using illustrated rules. Require at least one track with a cluster association, or a track which is deemed to be low p T

13. John Marshall, 13 Photon Recovery Photon-like cluster Mip-like cluster Have recently implemented a cluster fragmentation mechanism. This can be considered as a “local reclustering”: A cluster can be broken up into different combinations of smaller fragments. The original cluster and all fragment combinations can still be accessed and their properties queried. When the best fragment combination has been identified, call EndFragmentation API. ClusterLists will be tidied accordingly, any temporary clusters created will be deleted. This functionality has been used in a newly implemented PhotonRecovery algorithm, which splits clusters up into photon-like fragments and mip-like fragments. Further photon id helper functions, based on shower-profiles, have also been implemented.

14. John Marshall, 14 Remaining Work 1. Leakage correction For high energy jets, non-containment of showers is significant. To address this issue, have successfully reproduced old Pandora identification of clusters exiting the detector. Currently use this information when e.g. considering track-cluster consistency in reclustering. Now need to reproduce old Pandora use of muon chamber information to estimate leakage and energy deposition in the coil. However, this treatment is rather crude and will eventually be replaced. 2. Photon clustering 3. Consolidation, profiling and cpu/memory usage improvements Peter has been working on an algorithm that runs the main clustering algorithm, applies a shower-profile based selection and saves identified photon clusters as a separate named cluster list. The removal of these clusters allows for improved identification of hadronic showers when the clustering algorithm is called again. Can simply “plug-in” this algorithm to provide additional photon id, without any other changes. Would like to include this in first official release of PandoraPFANew, but it still requires thorough validation.

15. John Marshall, 15 Plans for next few months Pandora provides some simple cluster energy corrections, which remove effects of calo hits with anomalously high energies. The corrected cluster energy is used in comparisons with track momenta and when creating neutral PFOs. Energy correction functionality is designed to be modular. If a function asks for corrected energy, cluster will redirect to a helper function to provide the correction. Corrected energy is then cached in the cluster until cluster changes. We will provide a simple API so that a user can change a function pointer in the helper class, allowing them to use their own Energy Correction function. Work is already under way to investigate a software-compensation based correction (Katja and Frank). Energy Corrections Particle identification Just as a user can change the energy correction function, the modular structure of PandoraPFANew will allow a user to register different particle identification functions. As the functions may be compiled as part of the user application, these can be detector specific functions. We encourage people with existing particle id functions to get involved; we are interested in both “fast” (an indication) and “full” (accurate) versions of these functions. Modular PandoraPFANew allows the user application to change the energy corrections and particle id functions without even needing to rebuild the Pandora library.

16. John Marshall, 16 Summary The new Pandora framework is complete and has been rigorously tested. It is stable and has been unchanged for over six months now. Most of the Pandora algorithms have been fully implemented and validated, reproducing the old Pandora results. To complete the reconstruction, need to address leakage. Then want to consolidate this first complete version of code, which renders old Pandora code obsolete. Current applications that use the Pandora library: ILD Pandora application completed (our default test application), Norman Graf and Jeremy McCormick working on SlicPandora application, Every Pandora algorithm is automatically and instantly available for both ILD and SiD concepts. Have many ideas for moving forwards and improving Pandora within the new framework: New clustering algorithms, Long list of topological association improvements, Improved treatment of leakage, Particle identification and energy correction helper functions, Easy for other people to get involved and write new algorithms or contribute ideas...