Statistical problems in network data analysis: burst searches by narrowband detectors L.Baggio and G.A.Prodi ICRR TokyoUniv.Trento and INFN IGEC time coincidence.

Slides:



Advertisements
Similar presentations
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 9 Inferences Based on Two Samples.
Advertisements

S3/S4 BBH report Thomas Cokelaer LSC Meeting, Boston, 3-4 June 2006.
GWDAW 11 - Potsdam, 19/12/ Coincidence analysis between periodic source candidates in C6 and C7 Virgo data C.Palomba (INFN Roma) for the Virgo Collaboration.
AURIGA-LIGO Activity F. Salemi Italy, INFN and University of Ferrara for the LIGO-AURIGA JWG 2nd ILIAS-GW Meeting, October 24th and 25th, Palma de Mallorca,
Probability and Statistics Basic concepts II (from a physicist point of view) Benoit CLEMENT – Université J. Fourier / LPSC
NASSP Masters 5003F - Computational Astronomy Lecture 5: source detection. Test the null hypothesis (NH). –The NH says: let’s suppose there is no.
The Multiple Regression Model Prepared by Vera Tabakova, East Carolina University.
8. Statistical tests 8.1 Hypotheses K. Desch – Statistical methods of data analysis SS10 Frequent problem: Decision making based on statistical information.
Error Propagation. Uncertainty Uncertainty reflects the knowledge that a measured value is related to the mean. Probable error is the range from the mean.
G.A.Prodi - INFN and Università di Trento, Italy International Gravitational Event Collaboration igec.lnl.infn.it ALLEGRO group:ALLEGRO (LSU)
Differentially expressed genes
G. Cowan Lectures on Statistical Data Analysis Lecture 12 page 1 Statistical Data Analysis: Lecture 12 1Probability, Bayes’ theorem 2Random variables and.
G. Cowan 2011 CERN Summer Student Lectures on Statistics / Lecture 41 Introduction to Statistics − Day 4 Lecture 1 Probability Random variables, probability.
Searching for pulsars using the Hough transform Badri Krishnan AEI, Golm (for the pulsar group) LSC meeting, Hanford November 2003 LIGO-G Z.
Inferences About Process Quality
5-3 Inference on the Means of Two Populations, Variances Unknown
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 8 Tests of Hypotheses Based on a Single Sample.
An Efficient Rigorous Approach for Identifying Statistically Significant Frequent Itemsets.
Multiple testing correction
Lucio Baggio - Lucio Baggio - False discovery rate: setting the probability of false claim of detection 1 False discovery rate: setting the probability.
Review of Statistical Inference Prepared by Vera Tabakova, East Carolina University ECON 4550 Econometrics Memorial University of Newfoundland.
LIGO-G Z Coherent Coincident Analysis of LIGO Burst Candidates Laura Cadonati Massachusetts Institute of Technology LIGO Scientific Collaboration.
Results from TOBAs Results from TOBAs Cross correlation analysis to search for a Stochastic Gravitational Wave Background University of Tokyo Ayaka Shoda.
G. Cowan 2009 CERN Summer Student Lectures on Statistics1 Introduction to Statistics − Day 4 Lecture 1 Probability Random variables, probability densities,
Francesco Salemi - London, October 26th - 27th, VIRGO-bars joint data analysis Francesco Salemi for the VIRGO-bars Collaboration.
STATUS of BAR DETECTORS G.A.Prodi - INFN and University of Trento International Gravitational Event Collaboration - 2 ALLEGRO– AURIGA – ROG (EXPLORER-NAUTILUS)
Paris, July 17, 2009 RECENT RESULTS OF THE IGEC2 COLLABORATION SEARCH FOR GRAVITATIONAL WAVE BURST Massimo Visco on behalf of the IGEC2 Collaboration.
Silvia Poggi - GW burst detection strategy in non-homogeneus networks Detection strategies for bursts in networks of non-homogeneus gravitational waves.
INTERPRETATION of IGEC RESULTS Lucio Baggio, Giovanni Andrea Prodi University of Trento and INFN Italy or unfolding gw source parameters starting point:
First results by IGEC2 6 month of data of AURIGA-EXPLORER-NAUTILUS May 20 - Nov 15, 2005 IGEC2 was the only gw observatory in operation search for transient.
Back to basics – Probability, Conditional Probability and Independence Probability of an outcome in an experiment is the proportion of times that.
EMIS 7300 SYSTEMS ANALYSIS METHODS FALL 2005 Dr. John Lipp Copyright © Dr. John Lipp.
Solution of the Inverse Problem for Gravitational Wave Bursts Massimo Tinto JPL/CIT LIGO Seminar, October 12, 2004 Y. Gursel & M. Tinto, Phys. Rev. D,
ILIAS WP1 – Cascina IGEC – First experience using the data of 5 bar detectors: ALLEGRO, AURIGA, EXPLORER NAUTILUS and NIOBE. – 1460.
Issues concerning the interpretation of statistical significance tests.
Statistical Decision Theory Bayes’ theorem: For discrete events For probability density functions.
BCS547 Neural Decoding.
Dec 16, 2005GWDAW-10, Brownsville Population Study of Gamma Ray Bursts S. D. Mohanty The University of Texas at Brownsville.
Lucio BAGGIO - GWDAW11, Potsdam, Dec 21st First joint search of burst gravitational waves by the A URIGA -E XPLORER -N AUTILUS -V IRGO collaboration.
1 Methods of Experimental Particle Physics Alexei Safonov Lecture #25.
1 An example or real data analysis: the VIRGO-bars search for bursts Andrea Viceré for VIRGO – Auriga - Rog.
© Copyright McGraw-Hill 2004
1 Introduction to Statistics − Day 4 Glen Cowan Lecture 1 Probability Random variables, probability densities, etc. Lecture 2 Brief catalogue of probability.
1 Methods of Experimental Particle Physics Alexei Safonov Lecture #24.
Lecture 8 Source detection NASSP Masters 5003S - Computational Astronomy
1 Introduction to Statistics − Day 3 Glen Cowan Lecture 1 Probability Random variables, probability densities, etc. Brief catalogue of probability densities.
Non-Parametric Distributional Tests for Gravitational Wave Transient Event Detection Yeming Shi 1, Erik Katsavounidis 1, Michele Zanolin 2 1 Massachusetts.
G. Cowan Lectures on Statistical Data Analysis Lecture 4 page 1 Lecture 4 1 Probability (90 min.) Definition, Bayes’ theorem, probability densities and.
LIGO-G v2 The Search For Continuous Gravitational Waves Gregory Mendell, LIGO Hanford Observatory on behalf of the LIGO Science Collaboration The.
2005 Unbinned Point Source Analysis Update Jim Braun IceCube Fall 2006 Collaboration Meeting.
LIGO-G Z GWDAW9 December 17, Search for Gravitational Wave Bursts in LIGO Science Run 2 Data John G. Zweizig LIGO / Caltech for the LIGO.
ENGR 610 Applied Statistics Fall Week 7 Marshall University CITE Jack Smith.
TAUP 2007, Sendai, September 12, 2007 IGEC2 COLLABORATION: A NETWORK OF RESONANT BAR DETECTORS SEARCHING FOR GRAVITATIONAL WAVES Massimo Visco on behalf.
LIGO- G Z August 17, 2005August 2005 LSC Meeting 1 Bayesian Statistics for Burst Search Results LIGO-T Keith Thorne and Sam Finn Penn State.
In Bayesian theory, a test statistics can be defined by taking the ratio of the Bayes factors for the two hypotheses: The ratio measures the probability.
G. Cowan Lectures on Statistical Data Analysis Lecture 12 page 1 Statistical Data Analysis: Lecture 12 1Probability, Bayes’ theorem 2Random variables and.
GWDAW11 – Potsdam Results by the IGEC2 collaboration on 2005 data Gabriele Vedovato for the IGEC2 collaboration.
The first AURIGA-TAMA joint analysis proposal BAGGIO Lucio ICRR, University of Tokyo A Memorandum of Understanding between the AURIGA experiment and the.
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 7 Inferences Concerning Means.
Abstract: We completed the tuning of the analysis procedures of the AURIGA-LIGO joint burst search and we are in the process of verifying our results.
Palma de Mallorca - October 24 th, 2005 IGEC 2 REPORT International Gravitational Events Collaboration ALLEGRO– AURIGA – ROG (EXPLORER-NAUTILUS)
A 2 veto for Continuous Wave Searches
Searching for pulsars using the Hough transform
Chapter 4. Inference about Process Quality
This Week Review of estimation and hypothesis testing
Lecture 4 1 Probability (90 min.)
Maria Principe University of Sannio, Benevento, Italy
Coherent Coincident Analysis of LIGO Burst Candidates
Lecture 4 1 Probability Definition, Bayes’ theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests general.
Lecture 4 1 Probability Definition, Bayes’ theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests general.
Presentation transcript:

Statistical problems in network data analysis: burst searches by narrowband detectors L.Baggio and G.A.Prodi ICRR TokyoUniv.Trento and INFN IGEC time coincidence search is taking advantage of “a priori” information template search: matched filters optimized for short and rare transient gw with flat Fourier transform over the detector frequency band many trials at once: - different detector configurations (9 pairs + 7 triples + 2 four-fold) - many target thresholds on the searched gw amplitude (  30) - directional / non directional searches narrowband detectors & same directional sensitivity Cons: probing a smaller volume of the signal parameter space Pros: simpler problem GravStat 2005

… IGEC cont`d data selection and time coincidence search: - control of false dismissal probability - balance between efficiency of detection and background fluctuations background noise estimation - high statistics: 10 3 time lags for detector pairs 10 4 – 10 5 detector triples - goodness of fit tests with background model (Poisson) blind analysis (“good will”): - tuning of procedures on time shifted data by looking at all the observation time (no playground) … what if evidence for a claim would appear ? “GW candidates will be given special attention …” - IGEC-2 agreed on a blind data exchange (secret time lag) GravStat 2005

Poisson statistics For each couple of detectors and amplitude selection, the resampled statistics allows to test Poisson hypothesis for accidental coincidences.   Example: EX-NA background (one-tail  2 p-level 0.71) As for all two-fold combinations a fairly big number of tests are performed, the overall agreement of the histogram of p-levels with uniform distribution says the last word on the goodness-of-the-fit. verified GravStat 2005

A few basics: confidence belts and coverage experimental data physical unknown GravStat 2005

I  can be chosen arbitrarily within this “horizontal” constraint Feldman & Cousins (1998) and variations (Giunti 1999, Roe & Woodroofe 1999,...) Freedom of choice of confidence belt Fixed frequentistic coverage GravStat 2005

Plot of the likelihood integral vs. minimum (conservative) coverage min  C(  ), with background counts N b = Confidence intervals from likelihood integral I fixed, solve for : Compute the coverage Let Poisson pdf: Likelihood: GravStat 2005

Example: Poisson background N b = % 95% 85% N Likelihood integral

Plot of the likelihood integral vs. minimum (conservative) coverage min  C(  ), with background counts N b = Confidence intervals from likelihood integral I fixed, solve for : Compute the coverage Let Poisson pdf: Likelihood: GravStat 2005

Multiple configurations/selection/grouping within IGEC analysis GravStat 2005

Resampling statistics of accidental claims event time series coverage“claims” (0.555) [1] (0.326) [1] expected found Easy to set up a blind search GravStat 2005

Keep track of the number of trials (and their correlation) ! IGEC-1 final results consist of a few sets of tens of Confidence Intervals with min{C}=95%  the “false positives” would hide true discoveries requiring more than  5 two- sided C.I. to reach 0.1% confidence for rejecting H 0 the procedure was good for Upper Limits, but NOT optimized for discoveries  Need to decrease the “false alarm probability” (type I error) GravStat 2005

Freedom of choice of confidence belt Fine tune of the false alarm probability

Example: confidence belt from likelihood integral Poisson background N b = 7.0 Min{C}=95% 1 - C(N  ) P{false alarm} < 0.1% P{false alarm} < 5 % GravStat 2005

What false alarm threshold should be used to claim evidence for rejecting the null H 0 ? GravStat 2005 control the overall false detection probability: Familywise Error Rate <  requires single C.I. with P{false alarm} <  /m Pro: rare mistakes Con: high detection inefficiency control the mean False Discovery Rate: R = total number of reported discoveries F + = actual number of false positives Benjamini & Hochberg (JRSS-B (1995) 57: ) Miller et. al. (A J 122: Dec 2001;

Typically, the measured values of p are biased toward 0. signal The p-values are uniformly distributed in [0,1] if the assumed hypothesis is true Usually, the alternative hypothesis is not known. However, the presence of a signal would contribute to bias the p-values distribution. p-level 1 background pdf FDR control

Sketch of Benjamini & Hochberg FDR control procedure choose your desired bound q on ; OK if p-values are independent or positively correlated compute p-values {p 1, p 2, … p m } for a set of tests, and sort them in creasing order; p-value m determine the threshold T= p k by finding the index k such that p j <(q/m) j for every j>k; reject H 0 q T counts in case NO signal is present (H 0 is true), the procedure is equivalent to the control of the FamilyWise Error Rate at confidence < q

Open questions  check the fluctuations of the random variable FDR with respect to the mean.  check how the expected uniform distribution of p-values for the null H 0 can be biased (systematics, …)  would the colleagues agree that overcoming the threshold chosen to control FDR means & requires reporting a rejection of the null hypothesis ? To me rejection of the null is a claim for an excess correlation in the observatory at the true time, not taken into account in the measured noise background at different time lags. It could NOT be gws, but a paper reporting the H 0 rejection is worthwhile and due. GravStat 2005