1. August 1-4, 2012 APCTP Mini-Workshop on Nuclear Physics and Nuclear Astrophysics Dogye Campus, Kangwon National University 0

2. Kihyeon Cho Contents Introduction  Theory-experiment-simulation Simulation Project  Geant4 Supercomputing porting Summary 1

3. 2

4. 3

5. Kihyeon Cho e-Science paradigm Theory Simu- lation Experi- ment Theory-Simulation PGS LQCD Geant4 To probe the Standard Model and search for New Physics CDF Belle/Belle II e-Science paradigm e-Science paradigm Theory-Experiment-Simulation K.Cho and H.W.Kim, JKPS (2009) Theory-Exp. FKPPL B physics e-HEP env. - Remote Cont. - DP, DH - EVO

6. Kihyeon Cho International Collaborations 5 Belle*/Belle II* @ KEK, Japan CDF*@FNAL, USA e-Science @KISTI IN2P3@France France-Korea Particle Physics Laboratory * Official Collaborations The Leader of FKPPL - CDF Project (Kihyeon Cho) The Chair of Belle II Data Handling working Group (Kihyeon Cho )

7. Kihyeon Cho 6 ⇒ Feed-back between experiments (Belle & CDF) and theories inside KISTI 1. Theory-Experiment 1. Theory-Experiment

8. Kihyeon Cho 7

9. 8

10. 9

11. 1. Data production CDF Remote Control Room @KISTI 2. Data processing Pacific CAF(CDF Analysis Farm) ⇒ North America CAF @KISTI 3. Data Analysis Collaboration EVO servers @KISTI 4. Belle II Data Handling System Working Group Chair (K. Cho) To study high energy physics anytime anywhere even if we are not on-site (accelerator laboratory) 2. Experiment-Computing Simulation

12. Kihyeon Cho CDF Operation Center KISTI Remote Control Room We take shifts at KISTI even if we are not at Fermilab.

13. Kihyeon Cho 12

14. Kihyeon Cho Theory model – Simulations  MadGraph-pythia-PGS  LQCD  Geant4 .. 13 Ex) A study of Higgs model using cyberinfrastructur\ @KISTI 3. Theory-Simulation 3. Theory-Simulation

15. Kihyeon Cho 14 - To study high energy physics anytime and anywhere - Fusion research of experiment-computing–theory PublicationPublication comment Collider physics using e-Science paradigm of experiment-computing- theory B physics Leading Belle II Data Handling Working Group which consists of more than 30 persons from 12 countries e-Science paradigm e-Science paradigm Theory-Experiment-Simulation

16. Kihyeon Cho e-Science paradigm Theory (Nuclear physics) Simu- lation (Geant4,e t) Experi- ment (RISP Accelerat or) For Nuclear Physics and Nuclear Astrophysics - r-p process - Shell models… To apply domestic accelerator experiment To apply domestic accelerator experiment Theory-Experiment-Simulation Astro-Nuclear Simulation by RISP accelerator’s user group

17. 16

18. 17 Geant4 동아일보 (2012.7.5) 2012 년 7 월 4 일 오후 4 시 ( 한국시각 ) 21C 물리학계 최고의 사건 ! – Higgs 입자 발견

19. Kihyeon Cho Project Outline Title: Development of Geant4 for accelerator and medical physics PI: Kihyeon Cho Sponsor: Creative project by KISTI Budget: ~ half million dollar Period: Jan. 1, 2012~ Dec. 31, 2012 (Possibly extended) (Possibly extended)Goal:  Porting on Supercomputing  Simulation for HEP and Medical physics using Geant4  Profiling and Large-scale test, evaluation and improvement  Geant4 multi-proecssing R&D Web page  https://hep.kisti.re.kr/geant4 https://hep.kisti.re.kr/geant4 18

20. Kihyeon Cho Man Power KISTI, Korea  Kihyeon Cho, Soonwook Hwang, Taegil Bae, Youngjin Kim, Sangsu Ryu Yonsei University, Korea  Kyungho Kim, Youngjoon Kwon National Cancer Center  Se Byeong Lee, Sungkoo Cho, Jaeik Shin Fermilab, USA  Soon Yung Jun, Daniel Elvira, James Kowalkowski, Philippe Canal, Krzysztof Genser CMU, USA  Dongwook Jang UCSF, USA  Jungwook Shin, Bruce Faddegon 19

21. Kihyeon Cho Global Community (FKPPL) HEP - Accelerator R&D - Detector, … Medical Physics -Proton Accelerator,.. Geant4 R&D Grid FarmSupercomputer Geant4 Collaboration - code development UC San Francisco -Proton Accelerator GEANT4 Application - TOPAS R&D Infra- structure HEP Medical Physics Fermilab - Next generation simulation package development Appli- cation CERNIN2P3KEK InternationalCollaboration 20

22. Kihyeon Cho High Energy Physics vs. Medical Physics High Energy PhysicsMedical Physics Input dataN/A50MB (ex, CT data) FarmGrid Farm/ SupercomputerSupercomputer PortingGrid FarmSupercomputer Application PackageHEP ApplicationTOPAS, GATE CommunityHEP Application (Accelerator and Detector) Proton Accelerator Center International CollaborationFermilab, USAUCSF/SLAC/MGH, USA Contents of international Collaboration Detector simulation package using Multi- thread/parallel computing Proton Accelerator using Geant4 application 21

23. Kihyeon Cho Work on Progress Geant4 R&D  Supercomputing Porting  SUN Cluster Porting - Geant4-ST, MPI(ParGeant4), Geant4-MT  IBM AIX OS – Geant4-MT => Performance measurement and analysis  Parallel Computing Using Many core  Collaboration with Fermilab Accelerator physics application  Accelerator and Detector R&D, etc.  Nuclear Astrophysics  Theory models (MadGraph,..) Medical Application  Proton accelerator, therapy, etc. 22

24. Kihyeon Cho Simulation on secondary beam Object  Geant4 Simulation for secondary beams and dE/dxMethod  To analyze secondary beam depending on incident beam energy, target material, interaction model, …Output  Secondary particles  Energy dissipation… 23

25. Kihyeon Cho Preliminary results 24

26. 25 1.Geant4 및 HEP 소프트웨어 설치 2. 슈퍼컴퓨팅 초보사용자 신청서 접수 3. 전략과제 신청서 접수

27. Kihyeon Cho 26

28. Kihyeon Cho 27

29. Kihyeon Cho 28

30. Kihyeon Cho 29

31. Kihyeon Cho 30

32. Kihyeon Cho 31

33. Kihyeon Cho 32

34. Kihyeon Cho KISTI Supercomputing Porting Sun Linux Cluster  Geant4 ST  Geant4 MPI(ParGeant4)  Geant4 MT IBM AIX OS  Geant4-MT  Medical Application 33 Tachyon(SUN) Phase 2 ManufactureSUN Blade 6048 ArchitectureCluster Process modelIntel (Nehalem) # of Nodes3,200 nodes # of CPU cores 25,600 (8 per node) Rpeak (Tflops) 300 324 Total Memory76TB Disk Storage1.2PB Tape Storage2PB Interconnection NetworkIB 4X QDR

35. Kihyeon Cho Supercomputing Porting Done  HEP software  Cernlib, CLHEP, ROOT  Belle library  Geant4  Geant4 ST (Single Thread)  Geant4 MPI Interface  ParGeant4 To do list  Geant4 MT (Multi Thread)  Nuclear Astro-physics Channel  Theory models.. 34

36. Kihyeon Cho |-- ROOT | |-- root -> root_5.22.00. patches/ | |-- root_5.22.00.patches | |-- root_v5.14.00 | `-- root_v5.14.00.tar.gz |-- TOP-C | `-- 2.5.2 |-- belle | |-- belle | |-- belle.install.tar.gz | |-- cern ->../cern/ | |-- i386 | `-- local |-- cern | |-- 2001 | |-- 2005 | |-- 2006 | |-- 2007 | `-- root ->../ROOT/root |-- etc | |-- bashrc_general | `-- cshrc_general |-- marshalgen | `-- 1.0 `-- public |-- bin /applic/applications/hep |-- CLHEP | `-- 2.1.2.2 |-- Geant4 | |-- 9.4.p03 | |-- 9.5 | `-- 9.5.p01 |-- Geant4-data | |-- G4ABLA3.0 | |-- G4EMLOW6.19 | |-- G4EMLOW6.23 | |-- G4NDL0.2 | |-- G4NDL3.14 | |-- G4NDL4.0 | |-- G4NEUTRONXS1.0 | |-- G4NEUTRONXS1.1 | |-- G4PII1.2 | |-- G4PII1.3 | |-- PhotonEvaporation2.1 | |-- PhotonEvaporation2.2 | |-- RadioactiveDecay3.3 | |-- RadioactiveDecay3.4 | |-- RealSurface1.0 | |-- geant4-data-conf.csh | `-- geant4-data-conf.sh KISTI 슈퍼컴 Geant4/ HEP Directory 구조

37. Kihyeon Cho [## Geant4 환경변수 설정 @tachyon2c ~]$ source /applic/applications/hep/Geant4/9.5.p01/bin/geant4.(c)sh [@tachyon2c ~]$ source /applic/applications/hep/Geant4/9.5.p01/share/Geant4-9.5.1/geant4make/geant4make.(c)sh ## 테스트 디렉토리 생성 및 예제 파일 복사 @tachyon2c ~]$ mkdir test [@tachyon2c ~]$ cp -r /applic/applications/hep/Geant4/9.5.p01/share/Geant4-9.5.1/examples/extended/analysis/A01 test [@tachyon2c ~] cd test [## 컴파일 및 실행 @tachyon2c ~]$ make [@tachyon2c ~]$ $G4WORK_DIR/bin/Linux-g++/A01app...................................... HadCalLayerPhysical[-1] HadCalLayerLogical 1 Lead HadCalScintiPhysical[0] HadCalScintiLogical 0 Scintillator /HadCalorimeter Idle> /vis/open OGL # for using OpenGL Idle> /vis/drawVolume # example geometry drawn Idle>/vis/scene/add/trajectories # to make visualization, add trajectories and hits Idle>/vis/scene/add/hits Idle>/run/beamOn 1 # run an event Login node 실행 예제

38. Kihyeon Cho Nuclear Physics and nuclear Astrophysics Nuclear physics and Astrophysics theory needs 10,000 cores and 1PB storage.  Shell model  r-p process  Etc. To buy cluster To use KISTI supercomputer constantly  Processing on MoU between RISP of IBS and KISTI supercomputing Center => It take times 37

39. Kihyeon Cho 초보사용자 http://helpdesk.ksc.re.kr 38

40. Kihyeon Cho 전략과제 http://www.ksc.re.kr 39

41. Kihyeon Cho 40

42. Kihyeon Cho 1. Geant4 User Forum Website (https://hep.kisti.re.kr/geant4)https://hep.kisti.re.kr/geant4 2. Geant4 Indico Website (https://hep.kisti.re.kr/indico)) 3. User support mail 3. User support mail account (hep@kisti.re.kr) Geant4 사용자 지원 Geant4 사용자 지원

43. Kihyeon Cho Summary As one of e-Science paradigm of experiment- Theory-Simulation, Simulation (so-called Geant4) project has been launched this year at KISTI. Simulation project supports Nuclear Physics and Astro-nuclear Physics. User support for supercomputer  hep@kisti.re.kr hep@kisti.re.kr  https://hep.kisti.re.kr/geant4 42

44. HEP@KISTI