Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul,

Slides:



Advertisements
Similar presentations
GridPP July 2003Stefan StonjekSlide 1 SAM middleware components Stefan Stonjek University of Oxford 7 th GridPP Meeting 02 nd July 2003 Oxford.
Advertisements

Heavy Flavor Physics in LHC era 조기현, 남수현, 김정현, 김영진, 배태길 High Energy Physics Team KISTI (Korea Institute of Science and Technology Information) October.
Report on the Belle II Data Handling Group Kihyeon Cho (As a chair, on behalf of the Belle II DH Group) High Energy Physics Team KISTI (Korea Institute.
E-Science paradigm for hadron collider physics Kihyeon Cho High Energy physics Team KISTI APCTP LHC-Physics Workshop Konkuk University, Seoul, Korea August.
Heavy Flavor Physics as e-Science
1 Kihyeon Cho & Soonwook Hwang (KISTI) Super Belle with FKPPL VO & AMGA Data Handling.
Belle II Data Handling System Kihyeon Cho * (on behalf of the Belle II Computing Group) *Presenter High Energy Physics Team KISTI (Korea Institute of Science.
고에너지물리를 위한 협업환경 도구와 KISTI EVO 서버 김현우, 조기현 * 한국과학기술정보연구원 (KISTI) * 교신저자 2007 년 11 월 일 KOSTI2007/ 한국콘텐츠학회, 대전 배재대.
12/11/01 Matt Bridges Advisor: Ralph Morelli. What is Web Analytics? In traditional commerce, store owners can observe their customers habits: What time.
F Fermilab Database Experience in Run II Fermilab Run II Database Requirements Online databases are maintained at each experiment and are critical for.
LIA-FKPPL Workshop, Sungkyunkwan U, Seoul, March 15-16, KIhyeon Cho (KISTI) and Aurore Savoy-Navarro (LPNHE) CDF in FKPPL.
Two e-Science Components for HEP: EVO and Remote Control Room Hyunwoo KIM, Minho Jeung Kihyeon CHO* Sep , Daejeon (* Corresponding Author)
KISTI Grid CA Status Report KISTI Supercomputing Center Sangwan Kim APGridPMA Meeting Mar 8, 2010 Academia Sinica, Taipei, Taiwan.
ORACLE
11 ALICE Computing Activities in Korea Beob Kyun Kim e-Science Division, KISTI
A Design for KCAF for CDF Experiment Kihyeon Cho (CHEP, Kyungpook National University) and Jysoo Lee (KISTI, Supercomputing Center) The International Workshop.
Belle MC Production on Grid 2 nd Open Meeting of the SuperKEKB Collaboration Soft/Comp session 17 March, 2009 Hideyuki Nakazawa National Central University.
CDF Grid at KISTI 정민호, 조기현 *, 김현우, 김동희 1, 양유철 1, 서준석 1, 공대정 1, 김지은 1, 장성현 1, 칸 아딜 1, 김수봉 2, 이재승 2, 이영장 2, 문창성 2, 정지은 2, 유인태 3, 임 규빈 3, 주경광 4, 김현수 5, 오영도.
Belle II Data Management System Junghyun Kim, Sunil Ahn and Kihyeon Cho * (on behalf of the Belle II Computing Group) *Presenter High Energy Physics Team.
YongPyong 2012 Winter Conference on Particle Physics February 20~23,
Bookkeeping Tutorial. Bookkeeping & Monitoring Tutorial2 Bookkeeping content  Contains records of all “jobs” and all “files” that are created by production.
International Workshop on HEP Data Grid Nov 9, 2002, KNU Data Storage, Network, Handling, and Clustering in CDF Korea group Intae Yu*, Junghyun Kim, Ilsung.
Datasets on the GRID David Adams PPDG All Hands Meeting Catalogs and Datasets session June 11, 2003 BNL.
Collider Physics based on e-Science Paradigm of Experiment-Computing-Theory Kihyeon Cho, Junghyun Kim and Soo-hyeon Nam High Energy Physics Team KISTI.
Two Types of Remote CDF Shifts (Remote Control Room) for CDF Experiment 김현우 CDF Korea Workshop, Daejeon APRIL
The Advanced Data Searching System The Advanced Data Searching System with 24 February APCTP 2010 J.H Kim & S. I Ahn & K. Cho on behalf of the Belle-II.
Common Use Cases for a HEP Common Architecture Layer J. Templon, NIKHEF/WP8.
Report on Belle II Data Handling Group Kihyeon Cho High Energy Physics Team KISTI (Korea Institute of Science and Technology Information) June 16~18, 2010.
KISTI-GSDC SITE REPORT Sang-Un Ahn, Jin Kim On the behalf of KISTI GSDC 24 March 2015 HEPiX Spring 2015 Workshop Oxford University, Oxford, UK.
SAM - Sequential Data Access via Metadata Schema Metadata Functionality Workshop Glasgow University April 26-28,2004.
Heavy Flavor Physics in the LHC era 조기현, 남수현, 김정현, 김영진, 배태길 고에너지물리연구팀 KISTI (Korea Institute of Science and Technology Information) April 13-15, 2011 The.
DØ Data Handling & Access The DØ Meta-Data Browser Pushpa Bhat Fermilab June 4, 2001.
Overview of grid activities in France in relation to FKPPL FKPPL Workshop Thursday February 26th, 2009 Dominique Boutigny.
GridPP11 Liverpool Sept04 SAMGrid GridPP11 Liverpool Sept 2004 Gavin Davies Imperial College London.
National HEP Data Grid Project in Korea Kihyeon Cho Center for High Energy Physics (CHEP) Kyungpook National University CDF CAF & Grid Meeting July 12,
The CDF-LPNHE group Presentation to the CDF Meeting at KISTI, Daejon
E-Science paradigm for CDF experiment Kihyeon Cho High Energy Physics Team KISTI (Korea Institute of Science and Technology Information) October 8,2010.
Integration of the ATLAS Tag Database with Data Management and Analysis Components Caitriana Nicholson University of Glasgow 3 rd September 2007 CHEP,
May Donatella Lucchesi 1 CDF Status of Computing Donatella Lucchesi INFN and University of Padova.
DCAF (DeCentralized Analysis Farm) Korea CHEP Fermilab (CDF) KorCAF (DCAF in Korea) Kihyeon Cho (CHEP, KNU) (On the behalf of HEP Data Grid Working Group)
Bookkeeping Tutorial. 2 Bookkeeping content  Contains records of all “jobs” and all “files” that are produced by production jobs  Job:  In fact technically.
DCAF(DeCentralized Analysis Farm) for CDF experiments HAN DaeHee*, KWON Kihwan, OH Youngdo, CHO Kihyeon, KONG Dae Jung, KIM Minsuk, KIM Jieun, MIAN shabeer,
International Workshop on HEP Data Grid Aug 23, 2003, KNU Status of Data Storage, Network, Clustering in SKKU CDF group Intae Yu*, Joong Seok Chae Department.
Display Page (HTML/CSS)
March 1, CDF Grid Daejung Kong and Kihyeon Cho* (High Energy Physics Team, KISTI) * Corresponding author June 25, 2009 Korea CDF Meeting, SNU, Korea.
San Diego Supercomputer Center National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center National Partnership for Advanced.
Belle II Computing Fabrizio Bianchi INFN and University of Torino Meeting Belle2 Italia 17/12/2014.
Geant4 International Workshop June 13, Kihyeon Cho Contents Introduction  High Energy Physics at KISTI Geant4 Project Simulation on secondary.
Tutorial Instructions Kihyeon CHO (KISTI). Structure of Lecture Tutorial –Monday ~ Wednesday 9:30-17:30 Lunch break 12:30~13:30 –Lunch is up to you. –You.
Enabling Grids for E-sciencE EGEE-II INFSO-RI Status of SRB/SRM interface development Fu-Ming Tsai Academia Sinica Grid Computing.
Collider Physics based on e-Science Paradigm Kihyeon Cho (On behalf of High Energy Physics KISTI) APCTP YongPyong 2010 YongPyong Resort, Korea,
Evolving Architecture for Beyond the Standard Model Kihyeon CHO (KISTI) Yongpyong-High Joint Winter Conference on Particle Physics, String and Cosmology.
ATLAS Computing: Experience from first data processing and analysis Workshop TYL’10.
ATLAS Distributed Computing Tutorial Tags: What, Why, When, Where and How? Mike Kenyon University of Glasgow.
HEP Group Activity Kihyeon Cho (KISTI) KISTI-Belle II Open Meeting, KISTI, Daejeon, Korea Monday, February 23, 2014.
Evolving Architecture for Beyond the Standard Model Kihyeon CHO (KISTI) Yonsei Nuclear and Particle Workshop Yonsei University, Seoul, Korea April 29,
Design of AMGA Web Application Based on Belle II User Requirements Taesang Huh, Geunchul Park, Jae-Hyuck Kwak, Seungwoo Rho, Soonwook Hwang, JungHyun Kim,
IT 210 Week 7 DQ 2 To purchase this material link 210-Week-7-DQ-2 For more courses visit our website
PI: Kihyeon Cho & Soonwook Hwang Sponsor: Creative project by KISTI
Kihyeon Cho Supercomputing Center KISTI
Particle Physics at KISTI
e-Science for High Energy Physics
 YongPyong-High Jan We appreciate that you give an opportunity to have this talk. Our Belle II computing group would like to report on.
Current Grid System in Belle
Data Processing for CDF Computing
Collider Physics based on e-Science Paradigm
Belle II experiment Requirement of data handling system Belle II Metadata service system Data Cache system at Belle II experiment Summary.
Heavy Flavor Physics Experiments in the LHC era
EXP file structure.
Kihyeon Cho* (High Energy Physics Team, KISTI)
Presentation transcript:

Belle-II Data Handling working group status Junghyun Kim, Daejung Kong Kihyeon Cho* (High Energy Physics Team, KISTI) 1 Korea Belle II meeting SNU, Seoul, Korea * Corresponding Author

Contents KISTI Supercomputing center Activities of Belle II DH working group Problems of Belle Data Handling Status of Belle II Data Handling Plan 2

3 응용지원실 융합자원실 응용지워팀 선도연구팀선도연구팀 슈퍼컴 자원팀 연구망 자원팀 사이버인프라 응용연구자 KISTI 슈퍼컴퓨팅 본부 SC-SP SMB 팀 과학기술정보보호팀 차세대연구환경개발 실 기반기술개발실 HEP 연구 팀 응용환경개발팀 응용서비스팀 서비스기반기술팀 e-Cell 팀 NSDC 구축팀 과학기술융합망팀 GEOSS 팀 일반 사용자 전략분야 사용자 서비스기술 사용자 협업가시화팀 e-Science 그 리드기술개발팀 Ref. 이지수본부장 ( )

4 Who is who? 고에너지물리연구팀 (Physicists) – 조기현 ( 팀장 ) – 김정현, 공대정, 남수현 e-Science 그리드 기술개발팀 (Developers) – 황순욱 ( 팀장 ) – 유상수, 김유상, 김남규, 안선일 등 NSDC 구축팀 (Computing Technicians) – 장행진 ( 팀장 ) – 김법균, 유진승, 윤희준, 권석면, Christopher 등 Other teams – 김현우, 이상동, 김정한, 홍태용 등

Belle II DH Working Group Group Leader: K. Cho Participants: J.H. Kim, D.J.Kong, Thomas Kuhr, Takanori Hara, Tom Fifield, Nishida Shohei, Martin Sevier, Henryk Palka KISTI AMGA group: Sunil Ahn, Soonwook Hwang 5

6

7

Group Activities Data Handling Working Group Meeting –May 27, :00-17:00 –Contents Define meta data Extraction of meta data Interaction between AMGA and Tom’s activities Discussion – Attributes of meta data –Existing Belle database –SAM metadata structure (CDF/D0) –Different levels of metadata 8

Group Activities Belle II Computing Meeting Report: –Feb. 17, 09 - Introduction of AMGA (Dr. Hwang) –April 17, 09 - Plan (Dr. J.H. Kim) –May 20, 09 - DH Scenario (Dr. J.H. Kim) –June 16, 09 - The architecture in AMGA (Dr. J.H.Kim) 9

10

Problems of Belle Data Handling DH based on PHP, HTTP and postgreSQL No Grid Durability Scalability  SAM, SRB (No Grid)  AMGA (Grid)  Working with KISTI AMGA group 11

12

Belle II Data Handling Scenario Attribute of Meta Data Architecture 13

14

15

Extraction of Attributes Done Doing 16

Total used data  Exp#7, Run , uds MC Extraction of information 17 ? Stream

-rw-r--r-- 1 cho hep 125 May 25 12:57 mcprod-on_resonance-evtgen-uds-00-all-e000007r b _1600.list -rw-r--r-- 1 cho hep May 25 12:57 mcprod-on_resonance-evtgen-uds-00-all-e000007r b _1600.log -rw-r--r-- 1 cho hep 373 May 25 12:57 mcprod-on_resonance-evtgen-uds-00-all-e000007r b _1600.lum File from MDST 1)***. list => Logical file location 2)***.log => Properties of events 3)***.lum => File (Run) information The information 18

on_resonance]$ more mcprod-on_resonance-evtgen-uds-00-all-e000007r b _1600.list bfss08:/bdata/mcprod/dat/e000007/evtgen/uds/00/all/0807/on_resonance/28/evtgen-uds-00-all-e000007r b _1600.mdst on_resonance]$ more mcprod-on_resonance-evtgen-uds-00-all-e000007r b _1600.log 0: Parameters ; 7 ; 2865 ; 1 ; 3 ; 3 ; 0 ; 1 0: Parameters ; 7 ; 2865 ; 5 ; 4 ; 4 ; 1 ; 0 0: Parameters ; 7 ; 2865 ; 6 ; 5 ; 4 ; 3 ; 0 1: Parameters ; 7 ; 2865 ; 2 ; 5 ; 4 ; 2 ; 1 0: Parameters ; 7 ; 2865 ; 8 ; 3 ; 4 ; 1 ; 0 0: Parameters ; 7 ; 2865 ; 9 ; 0 ; 2 ; 0 ; 0 1: Parameters ; 7 ; 2865 ; 7 ; 6 ; 5 ; 3 ; 1 1: Parameters ; 7 ; 2865 ; 12 ; 2 ; 2 ; 1 ; 4 0: Parameters ; 7 ; 2865 ; 10 ; 6 ; 6 ; 1 ; 0 2: Parameters ; 7 ; 2865 ; 4 ; 4 ; 4 ; 1 ; 4 1: Parameters ; 7 ; 2865 ; 11 ; 1 ; 1 ; 0 ; 0 0: Parameters ; 7 ; 2865 ; 13 ; 5 ; 5 ; 4 ; 0 3: Parameters ; 7 ; 2865 ; 3 ; 5 ; 5 ; 3 ; 1 1: Parameters ; 7 ; 2865 ; 15 ; 6 ; 5 ; 2 ; 0 2: Parameters ; 7 ; 2865 ; 14 ; 4 ; 3 ; 1 ; 0 0: Parameters ; 7 ; 2865 ; 16 ; 8 ; 8 ; 4 ; 1 1: Parameters ; 7 ; 2865 ; 19 ; 4 ; 4 ; 2 ; 1 3: Parameters ; 7 ; 2865 ; 17 ; 3 ; 7 ; 3 ; 0 0: Parameters ; 7 ; 2865 ; 20 ; 3 ; 4 ; 0 ; 0 2: Parameters ; 7 ; 2865 ; 18 ; 5 ; 4 ; 5 ; 2 ……….. => Dummy; Dummy; Exp.#; Run#; Evt.#; Total #of positive charged track; Total # of negative charged track; Total # of Ks; Total #of KL => Logical file location 1) ***.list 2) ***.log 19

on_resonance]$ more mcprod-on_resonance-evtgen-uds-00-all-e000007r b _1600.lum Exp Run #evts Lum #evts/lum 4S Continuum 3S above 4S other Total On4S lum. = 3.15 Continuum Lum. = 0 3S Lum. = 0 Above4S Lum. = 0 Other Lum. = 0 Exp.#, Run#, Total # of events, Total Lum./file, #evts/lum, Luminosity type => File information 3) ***.lum 20 => Any suggestions about attributes?

Architecture of AMGA 21

22

23

24

25

Plan MC & Real data test with AMGA –Using Belle flat form –Writing and Reading Long Term –Prototype Oct Birger visit KISTI for two weeks (7/8/09) –Hand down AMGA to KISTI AMGA group –Questions on AMGA 26

Questions on AMGA Scalability –8 million events * 50 times Durability –How many users? Performance –How fast? 27

Backup 28

29