Junwei Cao March 23 2009 LIGO Document ID: LIGO-G0900271 Computing Infrastructure for Gravitational Wave Data Analysis Junwei Cao.

Slides:

Advertisements

Similar presentations

11/12/2003LIGO Document G Z1 Data reduction for S3 I Leonor (UOregon), P Charlton (CIT), S Anderson (CIT), K Bayer (MIT), M Foster (PSU), S Grunewald.

Advertisements

Condor-G: A Computation Management Agent for Multi-Institutional Grids James Frey, Todd Tannenbaum, Miron Livny, Ian Foster, Steven Tuecke Reporter: Fu-Jiun.

A Computation Management Agent for Multi-Institutional Grids

| October 12, 2011 A Multiple Signal Classification Method for Directional Gravitational-wave Burst Search Junwei.

1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.

Workload Management Workpackage Massimo Sgaravatto INFN Padova.

Slides for Grid Computing: Techniques and Applications by Barry Wilkinson, Chapman & Hall/CRC press, © Chapter 1, pp For educational use only.

Office of Science U.S. Department of Energy Grids and Portals at NERSC Presented by Steve Chan.

Milos Kobliha Alejandro Cimadevilla Luis de Alba Parallel Computing Seminar GROUP 12.

Grid Services at NERSC Shreyas Cholia Open Software and Programming Group, NERSC NERSC User Group Meeting September 17, 2007.

Workload Management Massimo Sgaravatto INFN Padova.

LIGO- GXXXXXX-XX-X Advanced LIGO Data & Computing Material for the breakout session NSF review of the Advanced LIGO proposal Albert Lazzarini Caltech,

Web-based Portal for Discovery, Retrieval and Visualization of Earth Science Datasets in Grid Environment Zhenping (Jane) Liu.

Design considerations for the Indigo Data Analysis Centre. Anand Sengupta, University of Delhi Thanks to -Maria Alessandra Papa (AEI) -Stuart Anderson.

SUN HPC Consortium, Heidelberg 2004 Grid(Lab) Resource Management System (GRMS) and GridLab Services Krzysztof Kurowski Poznan Supercomputing and Networking.

QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.

Grid Toolkits Globus, Condor, BOINC, Xgrid Young Suk Moon.

Vladimir Litvin, Harvey Newman Caltech CMS Scott Koranda, Bruce Loftis, John Towns NCSA Miron Livny, Peter Couvares, Todd Tannenbaum, Jamie Frey Wisconsin.

LIGO-G E ITR 2003 DMT Sub-Project John G. Zweizig LIGO/Caltech Argonne, May 10, 2004.

OSG End User Tools Overview OSG Grid school – March 19, 2009 Marco Mambelli - University of Chicago A brief summary about the system.

Grid Computing. What is a Grid? Many definitions exist in the literature Early definitions: Foster and Kesselman, 1998 –“A computational grid is a hardware.

April 19, 2005Scott Koranda1 The LIGO Scientific Collaboration Data Grid Scott Koranda UW-Milwaukee On behalf of the LIGO Scientific Collaboration.

Open Science Grid For CI-Days Internet2: Fall Member Meeting, 2007 John McGee – OSG Engagement Manager Renaissance Computing Institute.

LSC Segment Database Duncan Brown Caltech LIGO-G Z.

LIGO-G9900XX-00-M LIGO and GriPhyN Carl Kesselman Ewa Deelman Sachin Chouksey Roy Williams Peter Shawhan Albert Lazzarini Kent Blackburn CaltechUSC/ISI.

National Center for Supercomputing Applications The Computational Chemistry Grid: Production Cyberinfrastructure for Computational Chemistry PI: John Connolly.

10/20/05 LIGO Scientific Collaboration 1 LIGO Data Grid: Making it Go Scott Koranda University of Wisconsin-Milwaukee.

LIGO- G E LIGO Grid Applications Development Within GriPhyN Albert Lazzarini LIGO Laboratory, Caltech GriPhyN All Hands.

Patrick R Brady University of Wisconsin-Milwaukee

ARGONNE  CHICAGO Ian Foster Discussion Points l Maintaining the right balance between research and development l Maintaining focus vs. accepting broader.

G Z LIGO Scientific Collaboration Grid Patrick Brady University of Wisconsin-Milwaukee LIGO Scientific Collaboration.

1 Introduction to Grid Computing. 2 What is a Grid? Many definitions exist in the literature Early definitions: Foster and Kesselman, 1998 “A computational.

GT Components. Globus Toolkit A “toolkit” of services and packages for creating the basic grid computing infrastructure Higher level tools added to this.

INFSO-RI Enabling Grids for E-sciencE The US Federation Miron Livny Computer Sciences Department University of Wisconsin – Madison.

QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.

3-2.1 Topics Grid Computing Meta-schedulers –Condor-G –Gridway Distributed Resource Management Application (DRMAA) © 2010 B. Wilkinson/Clayton Ferner.

Grid Technologies  Slide text. What is Grid?  The World Wide Web provides seamless access to information that is stored in many millions of different.

LIGO-G9900XX-00-M ITR 2003 DMT Sub-Project John G. Zweizig LIGO/Caltech.

10/24/2015OSG at CANS1 Open Science Grid Ruth Pordes Fermilab

Virtual Data Grid Architecture Ewa Deelman, Ian Foster, Carl Kesselman, Miron Livny.

Resource Brokering in the PROGRESS Project Juliusz Pukacki Grid Resource Management Workshop, October 2003.

PROGRESS: ICCS'2003 GRID SERVICE PROVIDER: How to improve flexibility of grid user interfaces? Michał Kosiedowski.

Part Four: The LSC DataGrid Part Four: LSC DataGrid A: Data Replication B: What is the LSC DataGrid? C: The LSCDataFind tool.

GRID ARCHITECTURE Chintan O.Patel. CS 551 Fall 2002 Workshop 1 Software Architectures 2 What is Grid ? "...a flexible, secure, coordinated resource- sharing.

The Earth System Grid (ESG) Computer Science and Technologies DOE SciDAC ESG Project Review Argonne National Laboratory, Illinois May 8-9, 2003.

Some Grid Science California Institute of Technology Roy Williams Paul Messina Grids and Virtual Observatory Grids and and LIGO.

The LIGO Scientific Collaboration Data Grid Client/Server Environment Junwei Cao MIT LIGO Laboratory For the LIGO Scientific Collaboration.

LIGO-G E Generic LabelLIGO Laboratory at Caltech 1 Distributed Computing for LIGO Data Analysis The Aspen Winter Conference on Gravitational Waves,

GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.

Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.

LIGO-G E LIGO Scientific Collaboration Data Grid Status Albert Lazzarini Caltech LIGO Laboratory Trillium Steering Committee Meeting 20 May 2004.

LIGO-G Z LIGO at the start of continuous observation Prospects and Challenges Albert Lazzarini LIGO Scientific Collaboration Presentation at NSF.

Introduction to Grids By: Fetahi Z. Wuhib [CSD2004-Team19]

Contact: Junwei Cao SC2005, Seattle, WA, November 12-18, 2005 The authors gratefully acknowledge the support of the United States National.

State of LSC Data Analysis and Software LSC Meeting LIGO Hanford Observatory November 11 th, 2003 Kent Blackburn, Stuart Anderson, Albert Lazzarini LIGO.

Scott Koranda, UWM & NCSA 14 January 2016www.griphyn.org Lightweight Data Replicator Scott Koranda University of Wisconsin-Milwaukee & National Center.

11/12/2003LIGO-G Z1 Data reduction for S3 P Charlton (CIT), I Leonor (UOregon), S Anderson (CIT), K Bayer (MIT), M Foster (PSU), S Grunewald (AEI),

GridChem Architecture Overview Rion Dooley. Presentation Outline Computational Chemistry Grid (CCG) Current Architectural Overview CCG Future Architectural.

LIGO-G W Use of Condor by the LIGO Scientific Collaboration Gregory Mendell, LIGO Hanford Observatory On behalf of the LIGO Scientific Collaboration.

PROGRESS: GEW'2003 Using Resources of Multiple Grids with the Grid Service Provider Michał Kosiedowski.

LIGO-G Z1 Using Condor for Large Scale Data Analysis within the LIGO Scientific Collaboration Duncan Brown California Institute of Technology.

PX444 – The Distant Universe Lecture 13 The History of Star Formation.

Distributed Physics Analysis Past, Present, and Future Kaushik De University of Texas at Arlington (ATLAS & D0 Collaborations) ICHEP’06, Moscow July 29,

Collaborative Tools for the Grid V.N Alexandrov S. Mehmood Hasan.

PARALLEL AND DISTRIBUTED PROGRAMMING MODELS U. Jhashuva 1 Asst. Prof Dept. of CSE om.

1 Open Science Grid: Project Statement & Vision Transform compute and data intensive science through a cross- domain self-managed national distributed.

Gravitational Wave Data Analysis  GW detectors  Signal processing: preparation  Noise spectral density  Matched filtering  Probability and statistics.

Scott Koranda, UWM & NCSA 20 November 2016www.griphyn.org Lightweight Replication of Heavyweight Data Scott Koranda University of Wisconsin-Milwaukee &

Software Implementation

Presentation transcript:

Junwei Cao March LIGO Document ID: LIGO-G Computing Infrastructure for Gravitational Wave Data Analysis Junwei Cao for the LIGO Scientific Collaboration Tsinghua University / MIT LIGO Laboratory The 3 rd China-U.S. Roundtable on Scientific Data Cooperation Qingdao City, China March

Junwei Cao March LIGO Document ID: LIGO-G Outline Background introduction »Gravitational wave detection »LIGO and LIGO Scientific Collaboration LIGO data computing infrastructure »The LIGO data archiving »The Data monitoring system (online & offline) »The LIGO data grid »The LIGO data replication »Grid enabling data monitoring »The Open Science Grid Cyberinfrastructure perspective

Junwei Cao March LIGO Document ID: LIGO-G Gravitational Waves Gravitational waves are prediction of Einstein’s General Theory of Relativity. No direct detection of GWs is successful for nearly 100 years. A new window to observe the universe

Junwei Cao March LIGO Document ID: LIGO-G LIGO & LSC LIGO: Laser Interferometer Gravitational wave Observatory ( 激光干涉引力波天文台 ) U.S. NSF flagship project with largest single investment; a joint effort of Caltech and MIT LSC: the LIGO Scientific Collaboration with over 500 scientists from over 60 institutes worldwide LIGO Hanford Observatory (H1, H2)LIGO Livingston Observatory (L1)

Junwei Cao March LIGO Document ID: LIGO-G International Network of Detectors LIGO Simultaneously detect signals GEO Virgo TAMA AIGO

Junwei Cao March LIGO Document ID: LIGO-G LIGO Data Archiving Data are comprised of: »Gravitational Wave channel (AS_Q) »Physical Environment Monitors »Internal Engineering Monitors Multiple data products beyond raw data »RDS_R_L1, including both GW and environmental channels »RDS_R_L3, including only AS_Q channel L-RDS_R_L gwf Site (H,L) Data Type GPS Start Time Duration

Junwei Cao March LIGO Document ID: LIGO-G Data Monitoring Toolkit (DMT) SMP DMT Online Use Scenario – control-room type DMT Offline Use Scenario – standalone or grid enabled DMT Monitors Single data stream Name Server /data/node10/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node11/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node12/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node13/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node14/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node15/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node16/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node10/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node11/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node12/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node13/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node14/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node15/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node16/frame/S3/L3/LLO/L-RDS_R_L gwf Stdout Trigger files Alarm files Trend files …… Multiple data streams client lsmp lmsg Server gui base container sigp xml html xsilezcalib dmtenvevent trig …… MonServer frameio …… DMT Libraries

Junwei Cao March LIGO Document ID: LIGO-G DMT Offline (DOL) /data/node10/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node11/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node12/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node13/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node14/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node15/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node16/frame/S3/L3/LHO/H-RDS_R_L gwf /data/node10/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node11/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node12/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node13/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node14/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node15/frame/S3/L3/LLO/L-RDS_R_L gwf /data/node16/frame/S3/L3/LLO/L-RDS_R_L gwf rmon filelist1.txt filelist2.txt multilist.txt stride 16.0 channel_1 H1:LSC-AS_Q channel_2 L1:LSC-AS_Q opt filelist1.txt filelist2.txt rmon]$ export LD_LIBRARY_PATH=/opt/lscsoft/dol/lib rmon]$./rmon -opt opt -inlists multilist.txt Processing multi list file: multilist.txt Number of lists added: 2 Total data streams: 2 Processing frame list file: /home/jcao/rmon/filelist1.txt Number of files added: 1188 Total frame files: 1188 Processing frame list file: /home/jcao/rmon/filelist2.txt Number of files added: 1188 Total frame files: 1188 channel[1]=H1:LSC-AS_Q channel[2]=L1:LSC-AS_Q startgps= stride=16 r-statistic= startgps= stride=16 r-statistic= startgps= stride=16 r-statistic= …… standalone run of rmon DMT offline monitor

Junwei Cao March LIGO Document ID: LIGO-G The LSC Data Grid (LDG) AEI/Golm  Cardiff Birmingham

Junwei Cao March LIGO Document ID: LIGO-G The LDG Software Stack Applications Infrastructures End users & applications Application enabling Middleware / Services Operating Systems and … FC4GCCAutotoolsPythonMySQL Job scheduling / Condor Condor-G Data transfer / GridFTP Worklfow management / Condor DAGman Grid security / Globus GSI Resource management / Globus GRAM Resource location service / GlobusCatalog service / GlobusMetadata service VDS LSCcertUtils LSC Security management LSCsegFind LSC Data management LDM GlueOnasys LSC Job management LDASLALApps LSC CA VOMS The LSC Data Grid Client/Server Environment Version 4.0 (based on VDT 1.3.9) Matlab LDGreport LDRLSCdataFind DMT

Junwei Cao March LIGO Document ID: LIGO-G LIGO Data Replication (LDR) LHO (Tier 0) LLO (Tier 0) LDRMaster LDRMetadataService LDRSchedule LDRTransfer (gridftp) Caltech (Tier 1) RLS Local Storage Module mysql LDRMaster LDRMetadataService LDRSchedule LDRTransfer (gridftp) MIT (Tier 2) RLS Local Storage Module mysql 1. update metadata lfn 2. get lfn 3. generate lfn 4. gridftp 5. rlsadd lfn

Junwei Cao March LIGO Document ID: LIGO-G Users are interfaced with a LIGO friendly language. LDM: Grid Enabling DMT [job] id = test monitor = rmon args = -opt opt input = opt [data] observatory = type = start = end = ldm.sub ~]$ cd ldm ldm]$ source setup.sh ldm]$ cd../rmon rmon]$ ldm_agent rmon]$ ldm_submit ldm.sub Job test has been submitted. rmon]$ more ldm_test_condor.out Processing multi list file: ldm_test_CIT_multilist.txt Number of lists added: 2 Total data streams: 2 …… startgps= stride=16 r-statistic= …… grid-enabled run of rmon DMT offline monitor using LDM universe = globus globusscheduler = ldas-grid.ligo.caltech.edu/jobmanager-condor log = ldm_test_condor.log output = ldm_test_condor.out error = ldm_test_condor.err should_transfer_files = YES when_to_transfer_output = ON_EXIT transfer_input_files = ldm_test_CIT_multilist.txt, ldm_test_CIT_filelist1.txt, ldm_test_CIT_filelist2.txt, /home/jcao/rmon/opt arguments = -inlists ldm_test_CIT_multilist.txt -opt opt environment = LD_LIBRARY_PATH=/dso-test/jcao/dol/lib executable = /home/jcao/rmon/rmon Queue automatically generated Condor-G submission file Users do not bother with technical details of LSC data grid services. Data are located and file lists are generated automatically

Junwei Cao March LIGO Document ID: LIGO-G Open Science Grid (OSG) Grid of Grids 20 thousand CPUs Petabytes of data storage High energy physics Bioinformatics Nanotechnology ……

Junwei Cao March LIGO Document ID: LIGO-G Cyberinfrastructure Vision Providing high-end services – HPC, video conferencing, visualization, digital libraries, collaboration, … Infrastructuralizing high-end resources – supercomputers, data archives, telescopes, observatories, … More technical challenges: security, interaction, cross-VO, data transfer, fault tolerance, QoS … Cyberinfrastructure – NSF’s vision of 21 st century’s infrastructure Transportation Telecommunication Power grids Internet

Junwei Cao March LIGO Document ID: LIGO-G Technical Challenges Open vs. dedicated resources Applications are various Scientific applications are always evolving Cross-domain resource management and scheduling Cross-VO security (authentication and authorization) and trust management Dynamic VO construction Data availability Hybrid resource management QoS using virtual machines Too many to be exhausted …

Junwei Cao March LIGO Document ID: LIGO-G Summary LIGO already heavily relies on a large-scale computing infrastructure Cyberinfrastructure implementation is still facing major technical challenges References: