1. 1Malcolm Ellis - CM16 - 11th October 2006 MICE Software  Status report: u Data Challenge s Design Iteration s MICEEvent/MICERun s DataCards s MiceModules s Applications s Testing s Documentation u Recent releases  Demonstration u Simulation of MICE Stage 6 u Simulation of KEK test beam u Visualisation of simulated and real data  Data storage/online report (for Paul Kyberd)

2. 2Malcolm Ellis - CM16 - 11th October 2006 Data Challenge – SW perspective  Purpose – to push the development of and test the use of the software tools needed for MICE by performing a complete cycle of simulation, digitisation, reconstruction and analysis of the MICE experiment.  Requires: u Working software tools u Documentation u Analysis techniques u Computing resources (e.g. GRID)  Should help produce: u Improved software tools (debugging defects discovered in the process of performing the challenge). u Improved documentation. u Increase the number of MICE who are able to use the MICE software (maybe?)  Originally stated goal was to be ready to begin by CM16 (i.e. now).  We are not quite there yet, but are hopefully only a week or two late.  Scope has been extended to include analysis of real data (KEK test beam).

3. 3Malcolm Ellis - CM16 - 11th October 2006 Software Tools  G4MICE is divided up into packages (domains) that provide: u Libraries dedicated to particular tasks (e.g. Interface, Detector modeling, Reconstruction, Visualisation, etc...) u “Applications”, that is, compiled executables that are linked with one or more of the libraries and which are dedicated to performing one or more “Use Cases” (e.g. Simulation of the beam passing through a stage of MICE).  Data flow inside G4MICE: u C++ classes represent physics (e.g. SciFiHit, MCParticle, TofSpacePoint, etc...) u Persistency scheme allows these classes to be written to and read from disk. u Real data converters read raw data files (e.g. KEK test beam) and produce C++ classes.

4. 4 G4MICE Domains Recon Algorithms SciFiSpacePoint TofSpacePoint SciFiKalTrack etc... MICEEvent MICERun SciFiHit CkovDigit etc... Programs that you can run! Simulation Histograms etc...

5. 5Malcolm Ellis - CM16 - 11th October 2006 MICEEvent, MICERun, etc...  Currently store C++ classes that contain physics information in two containers: u MICEEvent – classes that have a “life” corresponding to one event (particle trigger), e.g.: s MCParticle s SciFiHit s TofSpacePoint s SciFiKalTrack s ZustandVektor u MICERun – classes that have a “life” over a data taking run, e.g.: s MiceModule (modeling a configuration of MICE) s VlpcCalib (calibration information for the SciFi tracker) s TrajectoryManager (Kalman filter package manager class)  Will eventually add a third category: u MICESpill – collection of events that are read out in a single burst (1 ms). This will be implemented when it is required to simulate/study pile- up effects and make a more accurate model of the DAQ setup.

6. 6Malcolm Ellis - CM16 - 11th October 2006 Configurations  Between test beams, 6 stages of MICE, multiple running modes per setting (e.g. beam momentum, emittance, cooling channel parameters, etc) and systematic studies (e.g. misalignments, optimising component layout, etc) there is the need to define and control a vast number of configurations of the MICE hardware.  MiceModule solution was presented at CM15 and has been extended since then.  A configuration is implemented as a tree of instances of this class that hold all the information (geometry, material, detector, magnetic field, etc) required to define that configuration.  First use of this for a systematic study will be the tracker station spacing optimisation.

7. 7Malcolm Ellis - CM16 - 11th October 2006 Materials  “MiceMaterials” class has been extended to keep track of the physical properties that a given material has in GEANT4.  This gives us two important benefits: 1. We can now keep track of all MICE material descriptions and see if any of the properties change from one version of GEANT4 to another. A test will be implemented soon to do this. 2. The material properties are now available to all of G4MICE (no longer just those that can link with GEANT4) so that (for example) the Reconstruction can ask for the radiation length of a volume to calculate multiple scattering covariance matrix or the event display can add information about the material to the visualisation of the geometry.

8. 8Malcolm Ellis - CM16 - 11th October 2006 Applications  For a given task (e.g. Simulation) an application is defined in G4MICE.  This is a separate directory in CVS in which an executable is created by linking one or more of the G4MICE libraries.  Special purpose code in the main() routine can be defined by anyone to use any of the G4MICE features.  Current list of applications includes: Analysis, Digitization, EventDisplay, FileSplitter, KekOct2005, Optics, PawHistograms, PawNtuples, PidStudies, Reconstruction, RootEvent, RootHistograms, RootTrees and Simulation  Anyone who discovers a new need that cannot be fulfilled by the existing applications can create a new one for the special purpose (e.g. KekOct2005 is specifically for the analysis of the KEK test beam in October 2005).  A LOT of analyses can be conducted using the existing applications (see next slide).

9. 9Malcolm Ellis - CM16 - 11th October 2006 Example Analyses  Determine the emittance of a beam: 1. Simulate the beam (Simulation Application), Monte Carlo hits are written to an output file. 2. Calculate the emittance of the beam (Analysis application), Monte Carlo hits are read back in from the file and the analysis routines are used to determine the emittance.  Determine the distribution of particles passing through TOF2: 1. Simulate the beam (Simulation Application), Monte Carlo hits are written to an output file. 2. Run an application to produce a histogram or ntuple (in PAW or ROOT format) reading in the ouput file and making histograms or an ntuple of the desired values.  Determine the momentum resolution of the tracker: 1. Simulate the beam (Simulation Application). 2. Digitise the hits (Digitization Application). 3. Reconstruct the digitised hits (Reconstruction Application). 4. Read in the results of the reconstruction and create histograms or ntuples of the reconstructed and Monte Carlo truth momenta and plot the difference...

10. 10Malcolm Ellis - CM16 - 11th October 2006 Testing  The software tools are of course under constant development (and, on average, improvement).  Even with the very small number of developers currently working on G4MICE it is easy to make changes that improve one area of performance and simultaneously degrade others.  In addition, crucial components (internal AND external) need to be tested to ensure that the physics has not changed (e.g. what if GEANT4 changes the multiple scattering model? Would we notice?!)  The answer to these (and more) problems is a comprehensive set of tests.  This was started by Yagmur some time ago, and has now been moved to a separate CVS area (MICETESTS) and is currently growing rapidly.  By mice-2-0 we will have reasonable coverage of the MICE code in Unit Tests as well as a few System and Integration tests to ensure some level of control over the physics performance of the software.

11. 11Malcolm Ellis - CM16 - 11th October 2006 e.g. MuScat vs GEANT4  Comparison of MuScat analysis with GEANT4 shows large difference with versions prior to 4.8.1.  G4MICE has migrated to 4.8.1.  System test will be developed to monitor the multiple scattering simulation so that we can keep track of any changes with future upgrades to newer versions.

12. 12 Testing Output (mice-1-9-2) Unit Tests of the Interface Package Testing the CkovHit class... Passed Testing the EmCalHit class... Passed Testing the MCParticle class... Passed Testing the MCVertex class... Passed Testing the SciFiHit class... Passed Testing the SpecialHit class... Passed Testing the TofHit class... Passed Testing the VirtualHit class... Passed Testing the VlpcEventTime class... Passed Testing the VlpcHit class... Passed Testing the VmeAdcHit class... Passed Testing the VmeTdcHit class... Passed Testing the ZustandVektor class... Passed Testing the dataCards class... Passed Testing the TofDigit class... Passed Testing the CkovDigit class... Passed Testing the EmCalDigit class... Passed Testing the SciFiDigit class... Passed Unit Tests of the Config Package Testing the MiceModule class... Passed Unit Tests of the Persist Package Testing the Persistency of the CkovHit class... Passed Testing the Persistency of the EmCalHit class... Passed Testing the Persistency of the MCParticle class... Passed Testing the Persistency of the MCVertex class... Passed Testing the Persistency of the SciFiHit class... Passed Testing the Persistency of the SpecialHit class... Passed Testing the Persistency of the TofHit class... Passed Testing the Persistency of the VirtualHit class... Passed Testing the Persistency of the VlpcEventTime class... Passed Testing the Persistency of the VlpcHit class... Passed Testing the Persistency of the VmeAdcHit class... Passed Testing the Persistency of the VmeTdcHit class... Passed Testing the Persistency of the ZustandVektor class... Passed Testing the Persistency of the CkovDigit class... Passed Testing the Persistency of the EmCalDigit class... Passed Testing the Persistency of the SciFiDigit class... Passed Testing the Persistency of the TofDigit class... Passed Unit Tests of the Simulation Package Generating some Simulated data Testing the MCParticle class... Passed Testing the MCVertex class... Passed Unit Tests of the DetModel Package Testing the CkovHit class... Passed Testing the EmCalHit class... Passed Testing the SciFiHit class... Passed Testing the SpecialHit class... Passed Testing the TofHit class... Passed Testing the ZustandVektor class... Passed Unit Tests of the DetResp Package Generating Digits from the Simulated data Testing the CkovDigit class... Passed Testing the EmCalDigit class... Passed Testing the SciFiDigit class... Passed Testing the TofDigit class... Passed

13. 13Malcolm Ellis - CM16 - 11th October 2006 GRID  Serious analyses are going to require a lot of CPU and disk space (in particular anything that involves the simulation of RF background!).  We now have a MICE VO (Virtual Organisation)!  A GRID portal has been developed at Imperial College.  Unfortunately, I’ve not had any luck using it since getting a Fermilab GRID certificate and joining the MICE VO (successful use to date was with an Imperial GRID certificate and membership of the dteam VO).  Hoping someone at Imperial can help put me in touch with the experts there and sort out what is most likely a trivial mistake or omission on my part...

14. 14Malcolm Ellis - CM16 - 11th October 2006 Documentation  All of these tools are no use if you don’t know how to use them!  Even worse is documentation that is inaccurate!  As part of the current preparation for the data challenge I am in the process of updating the documentation on the MICE website.  It is still very limited, but can be found here: http://www.mice.iit.edu/software/DataChallenge/Documentation/Documentation.html  The main areas that are covered are: u Installation guide u Documentation of the crucial G4MICE components u Task assignments & progress

15. 15Malcolm Ellis - CM16 - 11th October 2006 Installation Guide  G4MICE is now capable of operating under three operating systems: u Linux (always the default, serious productions will be run on Linux) u Macintosh OS X u Cygwin on Windows (using g++ not VC++)  The MAC and Windows versions are suitable for users to learn how to run small G4MICE jobs as well as for developers to work on code.  A page explaining how to install G4MICE from scratch is now on the web here: u http://www.mice.iit.edu/software/DataChallenge/Documentation/ProfessorsGuide.html http://www.mice.iit.edu/software/DataChallenge/Documentation/ProfessorsGuide.html  It is hoped that we will eventually be able to provide a single script that will perform all of these steps for the user.  The procedure has so far been tested successfully by a number of students and post-docs, we aim to eventually make it Professor-friendly...

16. 16Malcolm Ellis - CM16 - 11th October 2006 G4MICE Documentation  The existing documentation on G4MICE includes: u Doxygen documentation, generated automatically from the source code every day. u Access to the CVS repository for browsing the code. u The architectural diagram (for identifying dependencies) and list of package managers (so you know who to contact for help). u Bug database (bugzilla) for searching open or past bugs, or entering a new one. u Datacards – a page describing each of the datacards that the user can set to control the operation of G4MICE. u MiceModules – a page describing how the MICE configurations are specified. u Units – a page describing the use of physical units in G4MICE u Materials – a page describing the specification of materials in G4MICE u Applications – a page describing how to use each of the existing applications.

17. 17Malcolm Ellis - CM16 - 11th October 2006 Task Assignments & Progress  On the web from the Data Challenge website: u http://www.mice.iit.edu/software/DataChallenge/Progress/progress.html http://www.mice.iit.edu/software/DataChallenge/Progress/progress.html  Has not been updated by me for quite some time (sorry!).  I aim to update it before the end of this week and do so more regularly between now and the release of mice-2-0 for the start of production.

18. 18Malcolm Ellis - CM16 - 11th October 2006 Recent Releases  Two recent releases have made progress towards mice-2-0: u mice-1-9-1: s Major work on Simulation, including simulated hit classes. s First version of new testing scheme, including 36 tests. s Extension of Monte Carlo output to include information needed for later Analysis in the new format (ZustandVektor). u mice-1-9-2: s Major work on Digitisation, including the Digitised hit classes. s CKOV geometry prepared by Hideyuki now in CVS. s Basic CkovHit, CkovDigit classes written and tests prepared. s Dummy CKOV digitisation code written (as a place holder) – requires someone to do the software work to liase with CKOV experts to ensure correct physics is implemented. s Additional tests (of Digitisation, Digits classes and Persistency)  Ongoing work (during CM16): u Preparation of tests for Reconstruction (classes and algorithms). u First working version of HepRep event display for simulated and reconstructed hits.

19. 19Malcolm Ellis - CM16 - 11th October 2006 Demonstration  I will now try to demonstrate the use of G4MICE for a couple of quick applications  Time permitting, I will show examples using two different configurations (MICE Stage VI and KEK test beam) as well as looking at Simulated and Real Data. 1. Logging in and running the setup script 2. Where to find the G4MICE Applications 3. What a data card file looks like 4. What a configuration is 5. How to run the simulation 6. How to run the digitisation 7. How to run the reconstruction 8. How to use the new visualisation system.

20. 20 Ingredients Geometry (e.g. Boxes, Tubes) Materials (e.g.Pb, Al, Be, BC600) Sensitive Detectors (create hits) Properties (station number, fibre pitch, etc) Positions & Angle (e.g. alignment, staging) Properties (e.g. density, state temp, press, X0) DataCards Configuration (e.g. KEK test beam) Run Control (Number of events, beam position, size, source, etc...) Sim/Digi/Rec/Analysis (RF background Digitisation parameters etc...) I/O (Input/Output files Statistics output, etc...) BeamTools (time varying EM fields)

21. 21Malcolm Ellis - CM16 - 11th October 2006 G4MICE Demonstration using Cygwin/g++ installation

22. 22Malcolm Ellis - CM16 - 11th October 2006 Data Storage/Online Report  At the DAQ workshop recently held in Daresbury Paul Kyberd presented a look at the time scale for installing and commissioning online hardware in the MICE hall, based on the following assumptions: u Month is 20 working days u Installation is 8 hours per day u Conflicts are direct, meaning that two jobs cannot be done simultaneously. u A lot of the numbers were guesses, and he asked for people to look over them and let him know if they look correct or not. u Some of the items were not strictly DAQ or computing but have an impact.  Full talk available on the web: u http://www.mice.iit.edu/daq/daq3/daq3_kyberd_datataking.ppt http://www.mice.iit.edu/daq/daq3/daq3_kyberd_datataking.ppt

23. 23Malcolm Ellis - CM16 - 11th October 2006 Phases of Installation LCR Installation Jan 07 Feb 07 Mar 07 Apr 07 May 07 Jun 07 Jul 07 Aug 07 Sep 07 Tuning Data Phases Hall installation MICE work

24. 24Malcolm Ellis - CM16 - 11th October 2006 LCR Installation Jan 07 Feb 07 Mar 07 Apr 07 May 07 Jun 07 Jul 07 Aug 07 Sep 07 Tasks Install cabinets and racks Install Networking and test transfers Purchase DAQ computer and install Purchase Local storage and install Commission DAQ computer and storage Install sub-detector electronics Install and commission slow controls Transfer simulated data to tape at Atlas Analyse simulated data at tier1 and tier2 Fast analysis of beam tuning data at tier1 Inputs LCR complete. Including power and A/C. LCR layout – rack space. Internal cable runs. Cable runs from hall. Installation timings. MICE DAQ system V1.0 LCR Installation

25. 25Malcolm Ellis - CM16 - 11th October 2006 Hall Installation Jan 07 Feb 07 Mar 07 Apr 07 May 07 Jun 07 Jul 07 Aug 07 Sep 07 Hall installation Tasks Install sub-detectors (including cable runs to the LCR). Install beam detectors Conflicts Work on hall infrastructure Outputs Command and control communications with the LCR Working detectors and beam monitors Inputs

26. 26Malcolm Ellis - CM16 - 11th October 2006 Tuning Jan 07 Feb 07 Mar 07 Apr 07 May 07 Jun 07 Jul 07 Aug 07 Sep 07 Tuning Tasks Set up counters with beam Read out counters Tune beam at Initial setting (using fast grid) other settings as possible Integrate detectors into the DAQ TOF, Cerenkov, Calorimeter, Scintillating fibre tracker Detector pedestal (and calibration) Inputs LCR set up. All power, racks and A/C DAQ computer and local storage. Network connection to tier1 and 2. Data tapes written at Atlas Working DAQ Beam Counters in position and surveyed

27. 27Malcolm Ellis - CM16 - 11th October 2006 Paul’s Conclusions  My guess is there is time to do what we need.  Problem is likely to be that many detectors arrive towards the end of the period.  My worry is that I have forgotten something  I believe the Hardware/infrastructure for the DAQ is on track (see above)  I do not understand the Software/Controls infrastructure well enough to be confident everything will come together  Please look over his full talk and let him know what is correct, incorrect or missing!

28. 28Malcolm Ellis - CM16 - 11th October 2006 Data Volume in Phase 1  At the second DAQ workshop in February 2006 Paul and I made a first estimate of the data storage requirements. At that time, we estimated that MICE would produce 10 TB per month.  This was perceived to be too much, and we have since put in more accurate numbers to attempt to make a more accurate estimate.  In particular this estimate is just for MICE Phase 1, to assist with funding requests and planning for the coming year.  Assumptions: u MICE Phase 1 = Stages 1, 2 & 3 (i.e. 0, 1 and 2 trackers). u Each stage will last for three months. u Each month is 30 days with 75% efficiency (i.e. data taking). u One spill of 1 ms every second (during data taking) in which 600 particle triggers (i.e. muons) will be recorded and need to be stored. u This gives 3.5x10 9 events per stage. u Does not include calibration, etc, runs. Currently assume that this is of order 5% of physics data taking and so can be neglected for now. u Also assume that Monte Carlo requirements are of order 10M events, and thus also negligible size compared with real data.

29. 29Malcolm Ellis - CM16 - 11th October 2006 Event Size  Data size comes from several sources: u Tracker (firmware dependent) u PID detectors (depends on choice of front end electronics) u Controls and Monitoring – great progress made at recent DAQ workshop, we have a reasonable estimate for this now.  Tracker – assume zero suppression firmware works and is appropriate for all Phase 1 data taking (quite likely), then each event from a zero suppressed tracker will occupy of order 2100 bytes per event.  PID detectors – baseline for CKOV and EmCal is currently flash ADCs! Total event size per detector is thus: u TOF: 68 bytes per event u CKOV: 1600 bytes per event u EmCal: 48000 bytes per event!  Controls and monitoring. Based on extrapolations from D0 and DIAMOND, we think that the C&M data would take up about 2 MB per day.

30. 30Malcolm Ellis - CM16 - 11th October 2006 Total Data Size  Adding these different sources together, and using the assumptions for number of events, gives: u 0 trackers: 158 TB per stage (i.e. month) u 1 tracker: 165 TB per stage (i.e. month) u 2 trackers: 171 TB per stage (i.e. month)  As you can see, the previous assumption that the tracker is the dominant source of data is no longer true.  Remember that this does not include the 5% factor for calibration runs, nor the (negligible) Monte Carlo requirements.  In total, the estimate for Phase 1 comes to a bit over 500 TB!  Is this too much?  If so, how are we going to help get the PID data size down?  How can we (the collaboration) help Jean-Sebastien with this?  Breaking news – zero suppression may be possible in flash ADCs, reduce PID data size to 10~20% of value on previous slide.  This may bring us back to 50% more than the first estimate (i.e.15 TB per month, but as 10 was considered to be too much...)

31. 31Malcolm Ellis - CM16 - 11th October 2006 Next Steps  We have made a lot of progress, with limited (and decreasing!) mouse-power, however there are still a number of crucial things to complete: u Finish work on SciFi decoding. u Debug SciFi seed and validate Kalman track fit. u Add reconstruction code to create ZustandVektor from reconstructed tracks and output DST output containing just ZustandVektors for analysis. u Update Analysis code to use ZustandVektor for emittance calculation. u Complete first version of HepRep visualisation. u Prepare data cards and configurations needed by Analysis group for production. u Run test productions on GRID to validate scripts. u Finish tests and documentation. u Release mice-2-0.

32. 32Malcolm Ellis - CM16 - 11th October 2006 Warning  In my view there should be two different sets of people using the software: u Developers – the people who know a bit more about C++ and who write the code, produce the tests documentation, etc. u Users – the people who have a specific task that they want to achieve (e.g. determine the size of a detector)  To date the “Users” have actually been some set of the “Developers” who have been using the software they’ve developed (often for a third party...).  We are already losing a VERY large fraction of our FTE for developers (i.e. Yagmur, Chris, Rikard) and will soon lose others whose names have been mentioned recently as taking up the slack (i.e. students who will soon be writing up).  It is also a problem that these same people are the core of the Analysis team, so we are being hit hard in two areas that already suffer from a lack of resources.

33. 33Malcolm Ellis - CM16 - 11th October 2006 A Potential Solution  The people who are left can continue to work on the software as fast as we are able to.  However, we will be able to work faster if we are not also being asked to run simulations and analyses for people.  The Analysis group needs help.  The software group has produced (and is always improving) the tools that the Analysis group needs.  Join the Analysis group and learn how to use the tools!  This will benefit: u The Analysis group s More people working on the problems s More resources to perform studies (instead of having 10 tasks and one student...) u The Software group s Frees up resources to work on building and testing software. s Using the software will discover problems, which the software group can then fix (and add a test to ensure that it doesn’t come back in the future).