Data Processing for CDF Computing

Slides:



Advertisements
Similar presentations
GridPP July 2003Stefan StonjekSlide 1 SAM middleware components Stefan Stonjek University of Oxford 7 th GridPP Meeting 02 nd July 2003 Oxford.
Advertisements

Physics with SAM-Grid Stefan Stonjek University of Oxford 6 th GridPP Meeting 30 th January 2003 Coseners House.
Fermilab, the Grid & Cloud Computing Department and the KISTI Collaboration GSDC - KISTI Workshop Jangsan-ri, Nov 4, 2011 Gabriele Garzoglio Grid & Cloud.
Report on the Belle II Data Handling Group Kihyeon Cho (As a chair, on behalf of the Belle II DH Group) High Energy Physics Team KISTI (Korea Institute.
Les Les Robertson WLCG Project Leader WLCG – Worldwide LHC Computing Grid Where we are now & the Challenges of Real Data CHEP 2007 Victoria BC 3 September.
Makrand Siddhabhatti Tata Institute of Fundamental Research Mumbai 17 Aug
SCD FIFE Workshop - GlideinWMS Overview GlideinWMS Overview FIFE Workshop (June 04, 2013) - Parag Mhashilkar Why GlideinWMS? GlideinWMS Architecture Summary.
Status Report on Tier-1 in Korea Gungwon Kang, Sang-Un Ahn and Hangjin Jang (KISTI GSDC) April 28, 2014 at 15th CERN-Korea Committee, Geneva Korea Institute.
LIA-FKPPL Workshop, Sungkyunkwan U, Seoul, March 15-16, KIhyeon Cho (KISTI) and Aurore Savoy-Navarro (LPNHE) CDF in FKPPL.
CDF perspectives and Computing needs KISTI, Dec 5 th 2008 DongHee Kim.
YuChul Yang kr October 21, 2005The Korean Physical Society The Current Status of CDF Grid 양유철 *, 한대희, 공대정, 김지은, 서준석, 장성현, 조기현, 오영도,
LcgCAF:CDF submission portal to LCG Federica Fanzago for CDF-Italian Computing Group Gabriele Compostella, Francesco Delli Paoli, Donatella Lucchesi, Daniel.
Two e-Science Components for HEP: EVO and Remote Control Room Hyunwoo KIM, Minho Jeung Kihyeon CHO* Sep , Daejeon (* Corresponding Author)
YuChul Yang Oct KPS 2006 가을 EXCO, 대구 The Current Status of KorCAF and CDF Grid 양유철, 장성현, 미안 사비르 아메드, 칸 아딜, 모하메드 아즈말, 공대정, 김지은,
Building a distributed software environment for CDF within the ESLEA framework V. Bartsch, M. Lancaster University College London.
11 ALICE Computing Activities in Korea Beob Kyun Kim e-Science Division, KISTI
3rd June 2004 CDF Grid SAM:Metadata and Middleware Components Mòrag Burgon-Lyon University of Glasgow.
A short introduction to the Worldwide LHC Computing Grid Maarten Litmaath (CERN)
CHEP 2003Stefan Stonjek1 Physics with SAM-Grid Stefan Stonjek University of Oxford CHEP th March 2003 San Diego.
November 16, 2012 Seo-Young Noh Haengjin Jang {rsyoung, Status Updates on STAR Computing at KISTI.
A Design for KCAF for CDF Experiment Kihyeon Cho (CHEP, Kyungpook National University) and Jysoo Lee (KISTI, Supercomputing Center) The International Workshop.
Interactive Job Monitor: CafMon kill CafMon tail CafMon dir CafMon log CafMon top CafMon ps LcgCAF: CDF submission portal to LCG resources Francesco Delli.
SAMGrid as a Stakeholder of FermiGrid Valeria Bartsch Computing Division Fermilab.
CDF Grid at KISTI 정민호, 조기현 *, 김현우, 김동희 1, 양유철 1, 서준석 1, 공대정 1, 김지은 1, 장성현 1, 칸 아딜 1, 김수봉 2, 이재승 2, 이영장 2, 문창성 2, 정지은 2, 유인태 3, 임 규빈 3, 주경광 4, 김현수 5, 오영도.
1 High Energy Physics (HEP) Computing HyangKyu Park Kyungpook National University Daegu, Korea 2008 Supercomputing & KREONET Workshop Ramada Hotel, JeJu,
A Plan for HEP Data Grid Project in Korea Kihyeon Cho Center for High Energy Physics (CHEP) Kyungpook National University CDF/D0 Grid Meeting August 5,
International Workshop on HEP Data Grid Nov 9, 2002, KNU Data Storage, Network, Handling, and Clustering in CDF Korea group Intae Yu*, Junghyun Kim, Ilsung.
Sejong STATUS Chang Yeong CHOI CERN, ALICE LHC Computing Grid Tier-2 Workshop in Asia, 1 th December 2006.
KISTI-eScience meeting 1 SGC upgrade and future: for CMS-HI program Inkyu Park (Univ. of Seoul) for Korean CMS-HI team.
November SC06 Tampa F.Fanzago CRAB a user-friendly tool for CMS distributed analysis Federica Fanzago INFN-PADOVA for CRAB team.
LIA-FKPPL Workshop, CCIN2P3 in Lyon, France February 26, KIhyeon Cho (KISTI) and Aurore Savoy-Navarro (LPNHE) CDF in FKPPL.
Two Types of Remote CDF Shifts (Remote Control Room) for CDF Experiment 김현우 CDF Korea Workshop, Daejeon APRIL
Site Report BEIJING-LCG2 Wenjing Wu (IHEP) 2010/11/21.
Remote Control Room and SAM DH Shifts at KISTI for CDF Experiment 김현우, 조기현, 정민호 (KISTI), 김동희, 양유철, 서준석, 공대정, 김지은, 장성현, 칸 아딜 ( 경북대 ), 김수봉, 이재승, 이영장, 문창성,
CLRC and the European DataGrid Middleware Information and Monitoring Services The current information service is built on the hierarchical database OpenLDAP.
Application of the EDG Testbed Bockjoo Kim*, Soo-Bong Kim Seoul National University (SNU) Kihyeon Cho, Youngdo Oh, Dongchul Son Center for High Energy.
Evolution of a High Performance Computing and Monitoring system onto the GRID for High Energy Experiments T.L. Hsieh, S. Hou, P.K. Teng Academia Sinica,
US LHC OSG Technology Roadmap May 4-5th, 2005 Welcome. Thank you to Deirdre for the arrangements.
National HEP Data Grid Project in Korea Kihyeon Cho Center for High Energy Physics (CHEP) Kyungpook National University CDF CAF & Grid Meeting July 12,
KISTI & Belle experiment Eunil Won Korea University On behalf of the Belle Collaboration.
May Donatella Lucchesi 1 CDF Status of Computing Donatella Lucchesi INFN and University of Padova.
DCAF (DeCentralized Analysis Farm) Korea CHEP Fermilab (CDF) KorCAF (DCAF in Korea) Kihyeon Cho (CHEP, KNU) (On the behalf of HEP Data Grid Working Group)
DCAF(DeCentralized Analysis Farm) for CDF experiments HAN DaeHee*, KWON Kihwan, OH Youngdo, CHO Kihyeon, KONG Dae Jung, KIM Minsuk, KIM Jieun, MIAN shabeer,
International Workshop on HEP Data Grid Aug 23, 2003, KNU Status of Data Storage, Network, Clustering in SKKU CDF group Intae Yu*, Joong Seok Chae Department.
Eileen Berman. Condor in the Fermilab Grid FacilitiesApril 30, 2008  Fermi National Accelerator Laboratory is a high energy physics laboratory outside.
Accounting in LCG/EGEE Can We Gauge Grid Usage via RBs? Dave Kant CCLRC, e-Science Centre.
March 1, CDF Grid Daejung Kong and Kihyeon Cho* (High Energy Physics Team, KISTI) * Corresponding author June 25, 2009 Korea CDF Meeting, SNU, Korea.
LHC Computing in Korea Seyong Kim For the 1 st Korea-CERN meeting.
Integration of Physics Computing on GRID S. Hou, T.L. Hsieh, P.K. Teng Academia Sinica 04 March,
Global Science experimental Data hub Center April 25, 2016 Seo-Young Noh Status Report on KISTI’s Computing Activities.
Collider Physics based on e-Science Paradigm Kihyeon Cho (On behalf of High Energy Physics KISTI) APCTP YongPyong 2010 YongPyong Resort, Korea,
2 CMS 6 PB raw/run Phobos 50 TB/run E917 5 TB/run.
3 Compute Elements are manageable By hand 2 ? We need middleware – specifically a Workload Management System (and more specifically, “glideinWMS”) 3.
CDF Monte Carlo Production on LCG GRID via LcgCAF Authors: Gabriele Compostella Donatella Lucchesi Simone Pagan Griso Igor SFiligoi 3 rd IEEE International.
European Condor Week CDF experience with Condor glide-ins and GCB - Igor Sfiligoi1 European Condor Week 2006 Using Condor Glide-Ins and GCB to run.
Condor Week 2006, University of Wisconsin 1 Matthew Norman Using Condor Glide-ins and GCB to run in a grid environment Elliot Lipeles, Matthew Norman,
IFAE Apr CDF Computing Experience - Gabriele Compostella1 IFAE Apr CDF Computing Experience Gabriele Compostella, University.
PI: Kihyeon Cho & Soonwook Hwang Sponsor: Creative project by KISTI
Bob Jones EGEE Technical Director
LcgCAF:CDF submission portal to LCG
2nd Asia Tier Centre Forum Summary report 4th April 2017 edoardo
Sergio Fantinel, INFN LNL/PD
Short update on the latest gLite status
KISTI Daejeon, 23rd September 2015
Kihyeon Cho Supercomputing Center KISTI
Particle Physics at KISTI
LHC Data Analysis using a worldwide computing grid
e-Science for High Energy Physics
Collider Physics based on e-Science Paradigm
Kihyeon Cho* (High Energy Physics Team, KISTI)
Presentation transcript:

Data Processing for CDF Computing Feb. 23~28, 2009 APCTP-2009 YongPyong, Korea Data Processing for CDF Computing Minho Jeung, Hyunwoo Kim and Kihyeon Cho* (http://hep.kisti.re.kr) * Corresponding author April 25, 2019

The CDF Collaboration Europe  21 institutions North America France group, LPNHE/ CNRS-IN2P3&UPMC and ParisVII Universities North America  34 institutions Asia  8 institutions Korea group, KISTI, Chonnam, KNU, SNU, SKKU The CDF Collaboration  15 Countries  63 institutions  635 authors 2

Accelerator & CDF performance Tevatron is the world highest energy accelerator at S=1.96 TeV. CDF will be able to take data until to 2010. Data processing and analysis will be done five more years after that. Data taken 4.1 fb-1 (Nov. 2008) 5~6 fb-1 till by 2009

Network between Korea-Fermilab GLORIAD ESNet Mathis Equation BW: Upper bound of TCP throughput MSS: Max Segment Size RTT: Round Trip Time P: Packet Loss Nov. 2005 vs. Jan. 2009

CHEP(Korea) – Fermilab Nov. 10~24, 2005 (2 weeks) Packet loss is low. RTT is unstable. 2019년 4월 25일

Networks between KISTI and abroad (Jan. 2009) KEK GLORIAD-KR/10Gbps Hong Kong CSTNet-APAN-JP/1Gbps Fermilab Seattle ESnet/10Gbps CERN/ IN2P3 GLORIAD/1Gbps Chicago StarLight/10Gbps KISTI KISTI-Fermilab directly via Seattle (not via Chicago) 2019년 4월 25일

KISTI-Fermilab Jan. 6~20, 2009 (2 weeks) Packet loss is zero. RTT is stable. We avoid Chicago area congestions by direct network from Seattle. 2019년 4월 25일

KISTI-CERN Jan. 6~20, 2009 (2 weeks) Packet loss is zero. RTT is stable. 2019년 4월 25일

KISTI-IN2P3 Jan. 6~20, 2009 (2 weeks) Packet loss is zero. RTT is stable. 2019년 4월 25일

CDF Farms around world CAF (2001) DCAF (2003) Grid CAF (2006) CGCC (2008) UCSD CAF Toronto CAF Rut CAF Lyon CAF Taiwan AS BCN CAF CNAF CAF Korean CAF : KISTI, KNU Japan CAF European CAF Pacific North American Tsukuba U ASGC, Taiwan KISTI KNU 11 11

Progress of CDF Farms 12 Name Starting Date Grid middleware Job scheduling Content Site CAF 2001 - Condor Cluster farm inside Fermilab USA (Fermilab) DCAF 2003 Cluster farm outside Fermilab Korea, Japan, Taiwan, Spain (Varcelona, Cantabria), USA (Rutgers, San Diego), Canada, France Grid CAF 2006 LCG/OSG Resource Broker, Grid farm North America CAF European CAF Pacific CAF CGCC (CDF Grid Computing Center) 2008 LCG Big Grid farm Korea (KISTI) France (IN2p3) Italy (CNAF) 12 12

Grid CAF Context Diagram of Grid CAF Grid Farm (European CAF) Grid Farm (North America CAF) User Interface Grid Farm (Pacific CAF) CAF GUI Grid Site (IPAS_OSG) Grid Site (Taiwan-LCG2 ) Grid Site (KR-KISTI-HEP) Grid Site (JP-TSUKUBA) Grid Site (IPAS_CS) Legend Service Component Grid CAF Boundary Grid Site Boundary Job Transport Grid Pool 13 13

The comparison of Grid CAF Head node Work node Grid Middleware Method VO (Virtual Organization) North America CAF Fermilab (USA) USSD (USA) etc OSG Condor over Globus CDF VO European CAF CNAF (Italia) IN2P3 (France) Etc LCG WMS (Workload Management System) Pacific CAF AS (Taiwan) KISTI (Korea) LCG, OSG 14 14

Grid Site (KR-KISTI-HEP) Grid Farm (Pacific CAF) Pacific CAF at KISTI Service Component Legend Repository Grid CAF Boundary Grid Site Boundary Grid Pool Job Data Transportation Control Data Grid Site (KR-KISTI-HEP) Worker Node (hep015) … (hep004) Work Node (hep003) dCache Pool (se-hep) Grid Farm (Pacific CAF) VOMS Authentication Glideins dCache User Interface CafGui CafSubmit CafMon CAF Head Node Submitter Monitor Collector Negotiator Schedd Globus .... Job Submission Web monitoring Job Output Condor CE (hep001) 25 core processors 30 T Byte 15 15

Monitoring of Pacific CAF 16 16

Monitoring of Pacific CAF 17 17

HEP Computing farms in Korea

Jobs / CPU time per VO Jobs / VO Jan.-Nov. 2008 CPU time / VO

CDF Resource Usage - Total Ref. Dr. Donatella Lucchesi

Offsite Usage – USA & Asia => CGCC (CDF Grid Computing Center) Donatella clearly mentioned about KISTI in International Finance Committee Meeting on November ’08. Ref. Prof. DongHee Kim

CDF Grid Computing Center (CGCC) A hardware allocation plan for CGCC Pacific CAF at KISTI CGCC at KISTI Head node paccaf.phys.sinica.edu.tw New Head node VOMS server voms.cnaf.infn.it New VOMS server Work nodes 25 core processes 400 core processes Storage 30 T byte 300 T byte 22 22

Summary Data processing for CDF has been developed from CAF (Central Analysis Farm) to Grid. KISTI is one of leading institutes of CDF data processing.

Thank you. (http://hep.kisti.re.kr)