BES III Computing at The University of Minnesota Dr. Alexander Scott

6/13/2008BES III software at MN2 Acknowledgements  Cheng Ping Shen (Hawaii) –Traveled to Minnesota to help set up BESIII environment  Alexey Zhemchugov (Dubna) –Created pacman installation of BOSS, advised on setup  Pete Zweber, Nick Howell (Minnesota) –Extensive testing of local installation

6/13/2008BES III software at MN3 Talk Outline  Release installation and testing  Monte Carlo farm implementation  Comparison of IHEP and MN Monte Carlo  Analysis support  Future issues

6/13/2008BES III software at MN4 Installing Releases  We use Alexey’s pacman packaging of the BES environmentpacman –Instructions for install located at Alexey’s siteAlexey’s site –We installed both and successfully  release had some version drift  had to manually update packages –Download and installation takes ~5 hours –Directory structure fully built by package  Execute shell scripts to set environment –Release ready for use

6/13/2008BES III software at MN5 Testing Releases  We verified that Alexey install alone sufficient to generate MC –Started over with install from scratch on a workstation isolated from network –From workstation, successfully:  installed release  generated Monte Carlo  ran sample analysis package  We are developing documentation on doing remote installs

6/13/2008BES III software at MN6 Monte Carlo Example (6.3.1)  Generated 1M generic J/ψ decays  Used getacAlg analysis package to reconstruct J/ψ→  (η c )  Selected candidates based on good tracks, showers  Compared variable distributions between IHEP and MN Thanks to Cheng Ping

6/13/2008BES III software at MN7 Example  Generated 1M J/ψ decays  Used ρ  analysis package to reconstruct J/ψ→ ρ   Superimposed MN variable distributions on IHEP  Great agreement Thanks to Nick Howell

6/13/2008BES III software at MN8 Monte Carlo Farm  Used the CLEO-c farm implementation for BES III generationCLEO-c farm implementation  Only cosmetic changes required so far –Both have a “physics” and “reconstruction” step –Easily modifiable job structure –Generic distributed job infrastructure

6/13/2008BES III software at MN9 Physics Generation File #include "$ENV{KKMCROOT}/share/jobOptions_KKMC.txt" KKMC.CMSEnergy = 3.770; KKMC.CMSEnergySpread=0.0014; KKMC.InitializedSeed={400081,1,0}; KKMC.NumberOfEventPrinted=5333; KKMC.GeneratePsi3770=true; EvtDecay.userDecayTableName = “psipp.dec"; #include "$ENV{BESEVTGENROOT}/share/BesEvtGen.txt" BesRndmGenSvc.RndmSeed = ; #include "$ENV{BESSIMROOT}/share/G4Svc_BesSim.txt" G4Svc.RunID = ; #include "$ENV{ROOTIOROOT}/share/jobOptions_Digi2Root.txt" RootCnvSvc.digiRootOutputFile = Test_633_BES3_PsiPP_1_217804_0.rtraw"; MessageSvc.OutputLevel = 5; ApplicationMgr.EvtMax = 5333;

6/13/2008BES III software at MN10 Farm Infrastructure  3 servers + 40 worker nodes –hyper-threaded dual Xeon 2.66 GHz processors –handle 4 jobs apiece  2 dual quad-core machines –2.33 GHz processors –3x as powerful as other nodes –handle 8 jobs apiece  Gigabit connections to all machines  10 TB of storage space  Condor handles job distribution

6/13/2008BES III software at MN11 Monte Carlo Samples  MN farm samples generated using releases and –Statistics  ψ’: 3.5 s, 50 kB (5M/day)  ψ’’: 3.5 s, 50 kB (5M/day)  J/ψ: 2.7 s, 30 kB (6.5M/day)  Monte Carlo generated –Generic samples:  2M J/ψ, 4.5M ψ’, 1.5M ψ’’ –IHEP comparison samples  1M each of J/ψ, ψ’, and ψ’’

6/13/2008BES III software at MN12 MN vs. IHEP Comparison  Attempt to generate identical Monte Carlo to IHEP sample –Use the same scripts, random seeds, and parameter files as IHEP –Generate 1M each of J/ψ, ψ’, ψ’’ –Comparisons made at the track/shower level –Distributions match extremely wellDistributions  IHEP believes discrepancies due to multiple random number engines

6/13/2008BES III software at MN13 Sample Distributions

6/13/2008BES III software at MN14 Analysis Support  Documentation on analysis support for users Documentation  Users can create analysis areas and compile packages –Set up environment and own workarea –Use cmt to create packages within workarea –Check out TestRelease package to run your own packages  Developing our own MC testing package

6/13/2008BES III software at MN15 Issues for the Future

6/13/2008BES III software at MN16 Network Transfers  Transfer speeds differ between IHEP→MN and MN→IHEP –30 vs. 300 Mbps using iperf w/ 5 MB window  Require TCP/IP windows to be opened or transfer limited by latency –Investigating dynamic window sizing for scp operationsdynamic window sizing –Would like better transfer method

6/13/2008BES III software at MN17 Actual Run Simulations  Some questions remain before simulations of data –Based on CLEO-c, some components missing  Random triggers? Constants? Run by run info? –Triggers and runinfo can be handled by existing infrastructure –Manual installation requires constants database setup  Not a part of pacman installation  Already present or not yet integrated?