LIGO-G080280-00-W Use of Condor by the LIGO Scientific Collaboration Gregory Mendell, LIGO Hanford Observatory On behalf of the LIGO Scientific Collaboration.

Slides:



Advertisements
Similar presentations
A walk through some statistic details of LSC results.
Advertisements

11/12/2003LIGO Document G Z1 Data reduction for S3 I Leonor (UOregon), P Charlton (CIT), S Anderson (CIT), K Bayer (MIT), M Foster (PSU), S Grunewald.
Intermediate Condor: DAGMan Monday, 1:15pm Alain Roy OSG Software Coordinator University of Wisconsin-Madison.
Laser Interferometer Gravitational-Wave Observatory Who? Presented By: Bonnie Wooley.
LIGO-G W Report on LIGO Science Run S4 Fred Raab On behalf of LIGO Scientific Collaboration.
LIGO-G W Gregory Mendell, LIGO Hanford Observatory on behalf of the LIGO Scientific Collaboration StackSlide Summary LSC Meeting, August 2005.
Intermediate HTCondor: Workflows Monday pm Greg Thain Center For High Throughput Computing University of Wisconsin-Madison.
Design considerations for the Indigo Data Analysis Centre. Anand Sengupta, University of Delhi Thanks to -Maria Alessandra Papa (AEI) -Stuart Anderson.
LIGO-G E ITR 2003 DMT Sub-Project John G. Zweizig LIGO/Caltech Argonne, May 10, 2004.
Gravitational Wave Detectors: new eyes for physics and astronomy Gabriela González Department of Physics and Astronomy Louisiana State University.
Systematic effects in gravitational-wave data analysis
G D LSC ITR2003, ANL, LIGO Scientific Collaboration ITR 2003 Advisory Committee Meeting K. Blackburn 1, P. Brady 2, S. Finn 3, E.
April 19, 2005Scott Koranda1 The LIGO Scientific Collaboration Data Grid Scott Koranda UW-Milwaukee On behalf of the LIGO Scientific Collaboration.
LIGO-G W Gregory Mendell, LIGO Hanford Observatory on behalf of the LIGO Scientific Collaboration Stackslide search for continuous gravitational.
LIGO-G9900XX-00-M LIGO and GriPhyN Carl Kesselman Ewa Deelman Sachin Chouksey Roy Williams Peter Shawhan Albert Lazzarini Kent Blackburn CaltechUSC/ISI.
10/20/05 LIGO Scientific Collaboration 1 LIGO Data Grid: Making it Go Scott Koranda University of Wisconsin-Milwaukee.
LIGO Data Analysis (G M) Computing Breakout Session 1 1.LIGO Data Analysis Systems Data Management and Analysis 2.LIGO/LSC Analysis Software:
LIGO- G E LIGO Grid Applications Development Within GriPhyN Albert Lazzarini LIGO Laboratory, Caltech GriPhyN All Hands.
Patrick R Brady University of Wisconsin-Milwaukee
G Z LIGO Scientific Collaboration Grid Patrick Brady University of Wisconsin-Milwaukee LIGO Scientific Collaboration.
Update on the LIGO Data Analysis System LIGO Scientific Collaboration Meeting LIGO Hanford Observatory August 19 th, 2002 Kent Blackburn Albert Lazzarini.
OSG Area Coordinators’ Meeting LIGO Applications (NEW) Kent Blackburn Caltech / LIGO October 29 th,
 The workflow description modified to output a VDS DAX.  The workflow description toolkit developed allows any concrete workflow description to be migrated.
Status of LIGO Data Analysis Gabriela González Department of Physics and Astronomy Louisiana State University for the LIGO Scientific Collaboration Dec.
LIGO-G9900XX-00-M ITR 2003 DMT Sub-Project John G. Zweizig LIGO/Caltech.
LIGO Z LIGO Scientific Collaboration -- UWM 1 LSC Data Analysis Alan G. Wiseman (LSC Software Coordinator) LIGO Scientific Collaboration.
Design considerations for the Indigo Data Analysis Centre. Anand Sengupta, University of Delhi Many thanks to -Maria Alessandra Papa (AEI) -Stuart Anderson.
Status of Standalone Inspiral Code Duncan Brown University of Wisconsin-Milwaukee LIGO Scientific Collaboration Inspiral Working Group LIGO-G Z.
Intermediate Condor: Workflows Rob Quick Open Science Grid Indiana University.
The LIGO Scientific Collaboration Data Grid Client/Server Environment Junwei Cao MIT LIGO Laboratory For the LIGO Scientific Collaboration.
Searching for Gravitational Waves with LIGO Andrés C. Rodríguez Louisiana State University on behalf of the LIGO Scientific Collaboration SACNAS
LIGO-G E LIGO Scientific Collaboration Data Grid Status Albert Lazzarini Caltech LIGO Laboratory Trillium Steering Committee Meeting 20 May 2004.
LIGO-G Z LIGO at the start of continuous observation Prospects and Challenges Albert Lazzarini LIGO Scientific Collaboration Presentation at NSF.
15 Dec 2005GWDAW 10 LIGO-G Z1 Overview of LIGO Scientific Collaboration Inspiral Searches Alexander Dietz Louisiana State University for the LIGO.
Intermediate Condor: Workflows Monday, 1:15pm Alain Roy OSG Software Coordinator University of Wisconsin-Madison.
Searching for Gravitational Waves from Binary Inspirals with LIGO Duncan Brown University of Wisconsin-Milwaukee for the LIGO Scientific Collaboration.
Searches for Compact Binary Coalescences in LIGO and Virgo data Gabriela González For the LIGO Scientific Collaboration and the Virgo Collaboration APS.
Junwei Cao March LIGO Document ID: LIGO-G Computing Infrastructure for Gravitational Wave Data Analysis Junwei Cao.
S5 BNS Inspiral Update Duncan Brown Caltech LIGO-G Z.
LIGO-G Z LIGO Observational Results I Patrick Brady University of Wisconsin-Milwaukee on behalf of LIGO Scientific Collaboration.
1 Status of Search for Compact Binary Coalescences During LIGO’s Fifth Science Run Drew Keppel 1 for the LIGO Scientific Collaboration 1 California Institute.
Jan 12, 2009LIGO-G Z1 DMT and NDS2 John Zweizig LIGO/Caltech Ligo PAC, Caltech, Jan 12, 2009.
Contact: Junwei Cao SC2005, Seattle, WA, November 12-18, 2005 The authors gratefully acknowledge the support of the United States National.
The Search For Gravitation Radiation From Periodic Sources Gregory Mendell LIGO Hanford Observatory : The Laser Interferometer Gravitational-wave Observatory.
LIGO Plans for OSG J. Kent Blackburn LIGO Laboratory California Institute of Technology Open Science Grid Technical Meeting UCSD December 15-17, 2004.
State of LSC Data Analysis and Software LSC Meeting LIGO Hanford Observatory November 11 th, 2003 Kent Blackburn, Stuart Anderson, Albert Lazzarini LIGO.
Scott Koranda, UWM & NCSA 14 January 2016www.griphyn.org Lightweight Data Replicator Scott Koranda University of Wisconsin-Milwaukee & National Center.
11/12/2003LIGO-G Z1 Data reduction for S3 P Charlton (CIT), I Leonor (UOregon), S Anderson (CIT), K Bayer (MIT), M Foster (PSU), S Grunewald (AEI),
LSC Meeting LIGO Scientific Collaboration - University of Wisconsin - Milwaukee 1 Software Coordinator Report Alan Wiseman LIGO-G Z.
LSC Meeting LIGO Scientific Collaboration - University of Wisconsin - Milwaukee 1 LSC Software and Other Things Alan Wiseman University of Wisconsin.
LIGO-G v2 The Search For Continuous Gravitational Waves Gregory Mendell, LIGO Hanford Observatory on behalf of the LIGO Science Collaboration The.
LIGO-G Z1 Using Condor for Large Scale Data Analysis within the LIGO Scientific Collaboration Duncan Brown California Institute of Technology.
LIGO: The Laser Interferometer Gravitational-wave Observatory Sarah Caudill Louisiana State University Physics Graduate Student LIGO Hanford LIGO Livingston.
S5 First Epoch BNS Inspiral Results Drew Keppel 1 representing the Inspiral Group 1 California Institute of Technology Nov LSC Meeting MIT, 4 November.
G Z The LIGO gravitational wave detector consists of two observatories »LIGO Hanford Observatory – 2 interferometers (4 km long arms and 2 km.
LIGO-G M Press Conference Scientific Operation of LIGO Gary H Sanders Caltech (on behalf of a large team) APS April Meeting Philadelphia 6-April-03.
Gravitational Wave Data Analysis  GW detectors  Signal processing: preparation  Noise spectral density  Matched filtering  Probability and statistics.
LSC Meeting, June 3, 2006 LIGO-G Z 1 Status of inspiral search reviews Alan Weinstein (LIGO Laboratory / Caltech) For the LSC Internal review.
LIGO-G Z Results from LIGO Observations Stephen Fairhurst University of Wisconsin - Milwaukee on behalf of the LIGO Scientific Collaboration.
Search for gravitational waves from binary inspirals in S3 and S4 LIGO data. Thomas Cokelaer on behalf of the LIGO Scientific Collaboration.
Running LIGO workflows on the OSG
Scott Koranda, UWM & NCSA 20 November 2016www.griphyn.org Lightweight Replication of Heavyweight Data Scott Koranda University of Wisconsin-Milwaukee &
Thomas Cokelaer for the LIGO Scientific Collaboration Cardiff University, U.K. APS April Meeting, Jacksonville, FL 16 April 2007, LIGO-G Z Search.
ASIS Status Report Bruce Allen LSC Meeting, LHO, August 16, 2001
Coherent wide parameter space searches for gravitational waves from neutron stars using LIGO S2 data Xavier Siemens, for the LIGO Scientific Collaboration.
Background estimation in searches for binary inspiral
University of Wisconsin-Milwaukee
The Laser Interferometer Gravitational-Wave Observatory
Status of LIGO Patrick J. Sutton LIGO-Caltech
Software Implementation
Presentation transcript:

LIGO-G W Use of Condor by the LIGO Scientific Collaboration Gregory Mendell, LIGO Hanford Observatory On behalf of the LIGO Scientific Collaboration The Laser Interferometer Gravitational-Wave Observatory Supported by the United States National Science Foundation

LIGO-G W Sources Of Gravitational Waves Black HolesDense Stars Supernovae Stochastic Background Photos:

LIGO-G W LIGO GEOVirgo TAMA AIGO Worldwide Interferometers

LIGO-G W LDAS: CIT LDAS: LLO LSC: PSU LDAS: LHO LSC: UWM LDAS: MIT LSC: Golm LSC: SYR LSC: Cardiff LSC: Birmingham Worldwide Data Analysis using The LIGO Data Grid (LDG) CPUs with a typical clock speed of 2.6 GHz

LIGO-G W The LIGO Data Grid packageName( 'Server' ) version( '4.5' ) pacmanVersionGE('3.18.5') package( 'Server-Environment' ) package( 'VDT_CACHE:Globus' ) package( 'VDT_CACHE:CA-Certificates' ) package( 'VDT_CACHE:CA-Certificates-Updater' ) package( 'VDT_CACHE:Condor' ) package( 'VDT_CACHE:GSIOpenSSH' ) package( 'VDT_CACHE:KX509' ) package( 'VDT_CACHE:MyProxy' ) package( 'VDT_CACHE:UberFTP' ) package( 'VDT_CACHE:EDG-Make-Gridmap' ) package( 'VDT_CACHE:Globus-RLS') package( 'VDT_CACHE:Globus-Core') package( 'VDT_CACHE:Globus-Condor-Setup' ) package( 'VDT_CACHE:PyGlobus' ) package( 'VDT_CACHE:PyGlobusURLCopy' ) package( 'VDT_CACHE:Pegasus' ) package( 'VDT_CACHE:VOMS-Client' ) package( 'VDT_CACHE:Globus-WS' ) package( 'VDT_CACHE:Tomcat-5.5' ) package( 'VDT_CACHE:TclGlobus' ) package( 'Server-FixSSH' ) package( 'Server-RLS-Python-Client' ) package( 'Server-Cert-Util' ) package( 'Server-LSC-CA' ) LDG Client/Server Distribution

LIGO-G W The LIGO Data Grid Grid middleware  LDG Client/Server  Virtual Data Toolkit (VDT) –Globus Toolkit –GSI and X.509 certificates –pyGlobus, tclGlobus, Pegasus, etc…  In house packages  Glue: LSC Data Location & Pipeline Tools  LDR: LSC Lightweight Data Replication  GridFTP for moving data and files  Replica Location Service (RLS)  Onaysis: LSC Online Analysis System Users  500+ scientist in the LIGO Scientific Collaboration  200+ doing data analysis on the LIGO Data Grid High throughput computing  Condor for most analyses  BOINC for LSC Analysis Software  LAL, Matapps,  DMT, etc..

LIGO-G W Use of Condor by the LIGO Scientific Collaboration Condor handles 10’s of millions of jobs per year running on the LDG, and up to 500k jobs per DAG. Condor standard universe checking pointing widely used, saving us from having to manage this. At Caltech, 30 million jobs processed using 22.8 million CPU hrs. on 1324 CPUs in last 30 months. For example, to search 1 yr. of data for GWs from the inspiral of binary neutron star and black hole systems takes ~2 million jobs, and months to run on several thousand ~2.6 GHz nodes.

LIGO-G W Inspiral Analysis Pipeline Multidetector pipeline  Pipeline topology is same for all inspiral searches  BNS, PBH, BBH, spinning BBH  Different template/filtering code used for different searches  Can be used for LIGO-GEO and LIGO-VIRGO analysis Pipeline Description  Inspiral search run on each IFO  Look for triggers coincident in time and mass between detetors  Follow up with signal-based vetoes  Perform coherent analysis of surviving triggers  Follow up candidate events

LIGO-G W Example of a LIGO Inspiral DAG

LIGO-G W Example DAG Within Dag DAG: finds data and generates Fourier Tranforms used by other DAGs DAG: outputs spectra of power supply data:

LIGO-G W Example Test DAG for Condor Regression Testing Makes fake data for each detector (same code is used in Monte Carlo simulations). Run the fully-coherent multi-detector continuous-wave search code, used to search for GWs from rotating neutron stars. Compares the output with reference data.

LIGO-G W The LIGO/Condor Success Story Condor handles most of our searches and is vital to the success of LIGO. Condor and LIGO have a biweekly telecon to discuss issues & enhancements. In approximately the past year, Condor successfully enhanced scaling to support non-trivial O(1M) node DAGs, implemented option to prioritize nodes, e.g., depth-first traversal of DAGs, added categories to limit on number of resource intensive nodes in a DAG, handling of priorities and staggered start of jobs. Condor is working on a list of enhancements to, e.g., speed up of starting DAGs by O(100x), automate finding of rescue DAGs, e.g., when there are DAGs within DAGs, and merging of sub-DAGs. Add standard universe support on RHEL/CentOS and Debian. Condor is compatible with BOINC and can run backfill jobs on the LDG clusters when there are idle cycles. For the future: Our offline/online high throughput computing needs will continue to grow. Online jobs moving towards low latency; need to think about computing needs for realtime detection when Advanced LIGO comes on line.