Software Overview Akiya Miyamoto KEK JSPS Tokusui Workshop 2012 21-December-2012 Topics MC production Computing reousces GRID Future events Topics MC production.

Slides:



Advertisements
Similar presentations
31/03/00 CMS(UK)Glenn Patrick What is the CMS(UK) Data Model? Assume that CMS software is available at every UK institute connected by some infrastructure.
Advertisements

Keynote Workshop Charge Akiya Miyamoto, KEK 16-April-2009 ILD Software Workshop.
Taikan Suehara et al, ILC Tokusui Workshop, 17 Dec page 1 Taikan Suehara (Kyushu) Tomohiko Tanabe (Tokyo) LCFIPlus status and plan.
The DataGrid Project NIKHEF, Wetenschappelijke Jaarvergadering, 19 December 2002
Belle computing upgrade Ichiro Adachi 22 April 2005 Super B workshop in Hawaii.
IFIN-HH LHCB GRID Activities Eduard Pauna Radu Stoica.
Large scale data flow in local and GRID environment V.Kolosov, I.Korolko, S.Makarychev ITEP Moscow.
1 Deployment of an LCG Infrastructure in Australia How-To Setup the LCG Grid Middleware – A beginner's perspective Marco La Rosa
08/11/908 WP2 e-NMR Grid deployment and operations Technical Review in Brussels, 8 th of December 2008 Marco Verlato.
Computing plans for the post DBD phase Akiya Miyamoto KEK ILD Session ECFA May 2013.
SiD Software Status. Framework Overview Pythia, WHIZARD, etc. Pythia, WHIZARD, etc. SLIC Tracking, PFA, etc. LCSim StdHep LCIO Events JAS3 (or any AIDA.
Status of ILC Computing Model and Cost Study Akiya Miyamoto Norman Graf Frank Gaede Andre Sailer Marcel Stanitzki Jan Strube 1.
Software Common Task Group Report Akiya Miyamoto KEK ALCPG09 30 September 2009.
Computing for ILC experiment Computing Research Center, KEK Hiroyuki Matsunaga.
Computing and LHCb Raja Nandakumar. The LHCb experiment  Universe is made of matter  Still not clear why  Andrei Sakharov’s theory of cp-violation.
SiD Software and Computing: Status / Plans Jan Strube Tohoku University Thanks to Norman Graf (SLAC) for material.
ALCPG software: status and future plans LCWS UT Arlington October 23, 2012 Norman Graf (for the sim/reco group)
Cosener’s House – 30 th Jan’031 LHCb Progress & Plans Nick Brook University of Bristol News & User Plans Technical Progress Review of deliverables.
Overview of day-to-day operations Suzanne Poulat.
Workshop KEK - CC-IN2P3 KEK new Grid system 27 – 29 Oct. CC-IN2P3, Lyon, France Day2 14: :55 (40min) Koichi Murakami, KEK/CRC.
Computing for ILC experiments Akiya Miyamoto KEK 14 May 2014 AWLC14 Any comments are welcomed.
Status of the LHCb MC production system Andrei Tsaregorodtsev, CPPM, Marseille DataGRID France workshop, Marseille, 24 September 2002.
1 PRAGUE site report. 2 Overview Supported HEP experiments and staff Hardware on Prague farms Statistics about running LHC experiment’s DC Experience.
Sim/Recon DBD Editors Report Norman Graf (SLAC) Jan Strube (CERN) SiD Workshop SLAC, August 22, 2012.
Computing Resources for ILD Akiya Miyamoto, KEK with a help by Vincent, Mark, Junping, Frank 9 September 2014 ILD Oshu City a report on work.
RD’s Report on Detector Activity General Overview Project Advisory Sakue Yamada December 14, 2012 Sakue Yamada.
Norman Graf (SLAC) Software Software KILC12, Daegu, Korea.
ILC DBD Common simulation and software tools Akiya Miyamoto KEK ILC PAC 14 December 2012 at KEK.
LCG Generator Meeting, December 11 th 2003 Introduction to the LCG Generator Monthly Meeting.
SiD performance for the DBD Jan Strube CERN. Overview Software Preparation (CERN, SLAC) Machine Environment (CERN, SLAC) Tracking Performance (C. Grefe)
T3 analysis Facility V. Bucard, F.Furano, A.Maier, R.Santana, R. Santinelli T3 Analysis Facility The LHCb Computing Model divides collaboration affiliated.
Software Common Task Group Meeting Akiya Miyamoto 6-June-2011.
Status of tth analysis 1. √s=500 GeV 2. √s=1 TeV 3. MC requests R. Yonamine, T. Tanabe, K. Fujii T. Price, H. Tabassam, N. Watson, V. Martin ILD Workshop,
Software tools and Computing Akiya Miyamoto KEK 29-September-2006 At FJPPL meeting.
KISTI & Belle experiment Eunil Won Korea University On behalf of the Belle Collaboration.
Lcsim software: status and future plans ECFA-LC DESY May 28, 2013 Norman Graf (for the sim/reco group)
Summary of Software and tracking activities Tokusui Workshop December 2015 Akiya Miyamoto KEK.
Software Common Task Group Report Akiya Miyamoto ILC PAC POSTECH 3 November 2009.
Next Step Akiya Miyamoto, KEK 17-Sep-2008 ILD Asia Meeting.
Benchmarking Production Jan Strube (CERN) for the SiD Production team.
Computing Resources for ILD Akiya Miyamoto, KEK with a help by Vincent, Mark, Junping, Frank 9 September 2014 ILD Oshu City a report on work.
Large scale data flow in local and GRID environment Viktor Kolosov (ITEP Moscow) Ivan Korolko (ITEP Moscow)
LHCb report to LHCC and C-RSG Philippe Charpentier CERN on behalf of LHCb.
DD4hep-Based Simulation Nikiforos Nikiforou CERN/PH-LCD ILD Meeting 2014 Oshu City, September 9 th, 2014.
KEK GRID for ILC Experiments Akiya Miyamoto, Go Iwai, Katsumasa Ikematsu KEK LCWS March 2010.
Report from SB2009 Working Group Akiya Miyamoto 8-July-2010 DESY ILD Software and Integration Workshop.
ATLAS Distributed Analysis Dietrich Liko IT/GD. Overview  Some problems trying to analyze Rome data on the grid Basics Metadata Data  Activities AMI.
Linear Colliders in the HSF Jan Strube (PNNL) 1. Introduction Large Data rates (comparable to Belle-II) ~ 18 PB / year raw data at nominal running at.
Lcsim software: status and future plans ECFA-LC DESY May 30, 2013 Norman Graf (for the sim/reco group)
Breaking the frontiers of the Grid R. Graciani EGI TF 2012.
ALICE Physics Data Challenge ’05 and LCG Service Challenge 3 Latchezar Betev / ALICE Geneva, 6 April 2005 LCG Storage Management Workshop.
BES III Software: Beta Release Plan Weidong Li 19 th October 2005.
GDB Meeting CERN 09/11/05 EGEE is a project funded by the European Union under contract IST A new LCG VO for GEANT4 Patricia Méndez Lorenzo.
Generator status Akiya Miyamoto KEK 23 February 2016.
GRID & Parallel Processing Koichi Murakami11 th Geant4 Collaboration Workshop / LIP - Lisboa (10-14/Oct./2006) 1 GRID-related activity in Japan Go Iwai,
Using the Grid for the ILC Mokka and Marlin on the Grid ILC Software Meeting, Cambridge 2006.
LHCb Computing 2015 Q3 Report Stefan Roiser LHCC Referees Meeting 1 December 2015.
Computing requirements for the experiments Akiya Miyamoto, KEK 31 August 2015 Mini-Workshop on ILC Infrastructure and CFS for Physics and KEK.
ILD MCProduction with ILCDirac
ILD Soft & Analysis meeting
ILD Soft & Analysis meeting
Akiya Miyamoto KEK 1 June 2016
Grid Computing for the ILC
Brief summary of discussion at LCWS2010 and draft short term plan
LHCb Computing Model and Data Handling Angelo Carbone 5° workshop italiano sulla fisica p-p ad LHC 31st January 2008.
Simulation use cases for T2 in ALICE
Computing in ILC Contents: ILC Project Software for ILC
Bernd Panzer-Steindel CERN/IT
ILD Ichinoseki Meeting
The LHCb Computing Data Challenge DC06
Presentation transcript:

Software Overview Akiya Miyamoto KEK JSPS Tokusui Workshop December-2012 Topics MC production Computing reousces GRID Future events Topics MC production Computing reousces GRID Future events

MC Production for DBD Simulator model: ilcsoft-v  Frozen in summer.  Initial samples with v were superseded after simulating almost all 1 TeV samples due to error in lorentz boost for crossing angle.  500 GeV samples: ~ September due to a delay of producing 6f samples  No IP Z smearing except aa_lowpt samples. aa_lowpt: smeared with  z =225um (1TeV),  z =300um (500GeV) gausian.  Mostly by ILD_o1_v05 model. Reconstruction: ilcsoft-v01-16 and ILDConfig/v01-16-p0[3-5]*  With aa_lowpt background overlaid. Production procedure  Processes and nb of events : base on requests from analysis group  Job submission : from DESY using a DB tool and a human resource there.  sim, rec, dst, dst-merged files: replicated to KEK SE, in LCG catalog  Little contributions of KEK CPU resources /12/21 JSPS Workshop 2012: Software Overview

After DBD production Generator samples  250 & 350 GeV Higgs, 2f, 4f, 350 samples have been generated Planed to generated 1f, 3f, 5f, 6f, aa_2f, aa_4f, aa_lowpt samples Simulation/Reconstruction  Depends on demands/requests. Who does what analysis ? ILD official production after DBD has not been discussed yet.  Small scale productions for physics study or detector optimization could be made by each user.  Large scale production, which requires significant amount of CPU and storage, has to be discussed among group and planned properly in advance /12/21 JSPS Workshop 2012: Software Overview

Computing resources New KEKCC system since April  Batch server : ~3000 nodes shared wiith Belle, T2K ILC batch queue users : 5~10% of total CPU usage  HSM system: Shared with GRID: 200TB ( ~30% usage, mainly DBD samples ) non-GRID : 140TB ( ~ 50% usage )  Troubles: ILC users crashed batch servers many time reason: too many small files/write many small file. jlclogin2/jlccpuXX/jlccpuss0X  In operation as it is with SL4.7 ( very very old )  jlccpuss02: RAID crashed. Replacement of disk&RAID Card does not help jlclogin3 (purchased last year)  Main user is FPCCD study. Large memory size ( 96GB) /12/21 JSPS Workshop 2012: Software Overview

GRID KEKCC provides  UI, SE, CE/WN for ILC VO.  Very useful for users to access GRID resources  Little usage of KEKCC GRID WNs. ILCDirac  SiD and CLIC use ILCDirac  ILD may shift to use ILCDirac for MCProduction  Dr. S.Poss – ILCDirac tutorial on 10 Oct.  Easy to use GRID resources outside Japan ( DESY, CERN, …) /12/21 JSPS Workshop 2012: Software Overview

ILCDirac ILCDirac tutorial at KEKCC by Stephane Poss (CERN), 10 Oct /12/21 JSPS Workshop 2012: Software Overview

GRID - Future Human resources in DESY to support ILC VO is disappearing.  ILC VO management : DESY  Sharing among institutes ? Contact AM when you apply ILC VO member ship.  DESY resource for MC production may not be secured.  find new person ?  ILCDirac ? Needs a support team. CERN is willing to continue ILCDirac support  How KEK ( or Japan ) should contribute to ILC VO GRID. KEKCC resources are mainly for Belle/Belle2 and JPARC. Technical Issues  LCG file catalog vs DIRAC file catalog Keep consistency between them is not automatic. Consistent file deleting is a challenge.  Switch to DIRAC catalog only ?  Some times, worker node setting is not what DIRAC assumes. As a result, infinite number of jobs are directed to that WNs /12/21 JSPS Workshop 2012: Software Overview

LC Software expert meeting 31 Jan – 1-Feb., 2013 at CERN Preliminary agenda interface to geant4 simulation DD4Hep, LCDD (gdml) files vs. in memory conversion from TGeo to G4- Shapessensitive detectors interface to reconstruction : GEAR-like API ; org.lcsim ; Reco-API - other ? common tracking toolkit (tightly linked - but not restricted to AIDA WP2) common PFA and Flavor Tag tools - future plans GRID infrastructure for LC - common use of ILCDIRAC, management of ILC VO across the LC community, share of responsibilities between labs and groups common software tools: - documentation - Wikis (Confluence) - portal - nightly builds, test infratructure common software releases, installations, /12/21 JSPS Workshop 2012: Software Overview

Lecture on ILCSoft /12/21 JSPS Workshop 2012: Software Overview 07 June 2012 Was this useful ? ILCSoft is relatively stable now. May be good time to learn. Do we need more lectures ? When ? On what topics ?

Summary ILD DBD production scheduled was delayed due to - delay of ILC TDR parameter  delay of generator samples - delay of fixing tracking code (Silicon trackers) But the production has completed successfully. Now its time to develop a plan for next step  For participating Snowmass activity  International discussion is about to start  Further contributions ( more than LCFIPlus, KalTest ) to ILC software for the international community would be requested /12/21 JSPS Workshop 2012: Software Overview