Collaboration Meeting Moscow, 6-10 Jun 2011 Collaboration Meeting Moscow, 6-10 Jun 2011 Agustín Sánchez Losa IFIC (CSIC – Universitat de València)

Slides:



Advertisements
Similar presentations
High Energy Gamma Ray Group
Advertisements

Statistical Techniques I EXST7005 Sample Size Calculation.
The Estimation Problem How would we select parameters in the limiting case where we had ALL the data? k → l  l’ k→ l’ Intuitively, the actual frequencies.
Flare Analysis Summary of the present status Agustín Sánchez Losa27-Jul-2011.
14.5m 100 m ~480 m Junction Box ~70 m String-based detector; Downward-looking (45°) PMTs; 2475 m deep; 12 detection lines 25 storeys / line 3 PMTs / storey.
CHAPTER 21 Inferential Statistical Analysis. Understanding probability The idea of probability is central to inferential statistics. It means the chance.
Maximum Likelihood And Expectation Maximization Lecture Notes for CMPUT 466/551 Nilanjan Ray.
Collaboration Meeting CERN, 6-10 Feb 2012 Collaboration Meeting CERN, 6-10 Feb 2012 Agustín Sánchez Losa IFIC (CSIC – Universitat de València)
Experimental Design, Response Surface Analysis, and Optimization
Dynamic Bayesian Networks (DBNs)
1 Detection and Analysis of Impulse Point Sequences on Correlated Disturbance Phone G. Filaretov, A. Avshalumov Moscow Power Engineering Institute, Moscow.
CMPUT 466/551 Principal Source: CMU
1Andrea Caliandro Search of Optimized Cuts for Pulsar Detection Andrea Caliandro - INFN Bari DC2 CloseOut May Goddard Space Flight Center.
Planning under Uncertainty
Bayesian Analysis of X-ray Luminosity Functions A. Ptak (JHU) Abstract Often only a relatively small number of sources of a given class are detected in.
Statistical Analysis of High-Energy Astronomical Time Series Jeff Scargle NASA Ames – Fermi Gamma Ray Space Telescope Jeffrey D. Scargle Space Science.
UNIVERSITY OF SOUTH CAROLINA Department of Computer Science and Engineering On-line Alert Systems for Production Plants A Conflict Based Approach.
Prénom Nom Document Analysis: Data Analysis and Clustering Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
Radial Basis Function Networks
Lecture 12 Monte Carlo Simulations Useful web sites:
EE513 Audio Signals and Systems Statistical Pattern Classification Kevin D. Donohue Electrical and Computer Engineering University of Kentucky.
Binary Variables (1) Coin flipping: heads=1, tails=0 Bernoulli Distribution.
STA Lecture 261 STA 291 Lecture 26 Two types of errors in testing hypothesis. Connection between testing hypothesis and confidence intervals.
Alignment and classification of time series gene expression in clinical studies Tien-ho Lin, Naftali Kaminski and Ziv Bar-Joseph.
Statistical Decision Theory
START OF DAY 8 Reading: Chap. 14. Midterm Go over questions General issues only Specific issues: visit with me Regrading may make your grade go up OR.
CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 11: Bayesian learning continued Geoffrey Hinton.
Lab 3b: Distribution of the mean
CS 782 – Machine Learning Lecture 4 Linear Models for Classification  Probabilistic generative models  Probabilistic discriminative models.
Agustín Sánchez-Losa IFIC (CSIC-Universitat de València) Transient sources, like AGNs or Gamma Ray Bursters, are among.
ETM 607 – Input Modeling General Idea of Input Modeling Data Collection Identifying Distributions Parameter estimation Goodness of Fit tests Selecting.
Confidence intervals and hypothesis testing Petter Mostad
Detection and estimation of abrupt changes in Gaussian random processes with unknown parameters By Sai Si Thu Min Oleg V. Chernoyarov National Research.
A statistical test for point source searches - Aart Heijboer - AWG - Cern june 2002 A statistical test for point source searches Aart Heijboer contents:
Detecting Group Differences: Mining Contrast Sets Author: Stephen D. Bay Advisor: Dr. Hsu Graduate: Yan-Cheng Lin.
The ANTARES neutrino telescope is located on the bottom of the Mediterranean Sea, 40 km off the French coast. The detector is installed at a depth of 2.5.
Observations of TeV blazars seem to be well explained by leptonic synchrotron inverse-Compton models, but there may be an hadronic mechanism involved as.
The IceCube Neutrino Observatory is a cubic kilometer detector at the geographic South Pole. We give an overview of searches for time-variable neutrino.
Slides for “Data Mining” by I. H. Witten and E. Frank.
Dec 16, 2005GWDAW-10, Brownsville Population Study of Gamma Ray Bursts S. D. Mohanty The University of Texas at Brownsville.
Active learning Haidong Shi, Nanyi Zeng Nov,12,2008.
Analyzing Expression Data: Clustering and Stats Chapter 16.
De novo discovery of mutated driver pathways in cancer Discussion leader: Matthew Bernstein Scribe: Kun-Chieh Wang Computational Network Biology BMI 826/Computer.
1 Introduction to Statistics − Day 4 Glen Cowan Lecture 1 Probability Random variables, probability densities, etc. Lecture 2 Brief catalogue of probability.
1 Introduction to Statistics − Day 3 Glen Cowan Lecture 1 Probability Random variables, probability densities, etc. Brief catalogue of probability densities.
POINT SOURCE ANALYSIS WITH 5-LINE DATA SUMMARY AWG PHONE CALL Juan Pablo Gómez-Gozález 20 th April
SUB-NYQUIST DOPPLER RADAR WITH UNKNOWN NUMBER OF TARGETS A project by: Gil Ilan & Alex Dikopoltsev Guided by: Yonina Eldar & Omer Bar-Ilan Project #: 1489.
Visual Tracking by Cluster Analysis Arthur Pece Department of Computer Science University of Copenhagen
A comparative approach for gene network inference using time-series gene expression data Guillaume Bourque* and David Sankoff *Centre de Recherches Mathématiques,
1 Chapter 8: Model Inference and Averaging Presented by Hui Fang.
2005 Unbinned Point Source Analysis Update Jim Braun IceCube Fall 2006 Collaboration Meeting.
1 Multi-Messenger approach in IceCube E. Resconi (Max-Planck-Institute for Nuclear Physics, Heidelberg) LAUNCH, Heidelberg, March 2007.
G. Cowan Lectures on Statistical Data Analysis Lecture 10 page 1 Statistical Data Analysis: Lecture 10 1Probability, Bayes’ theorem 2Random variables and.
Class Seven Turn In: Chapter 18: 32, 34, 36 Chapter 19: 26, 34, 44 Quiz 3 For Class Eight: Chapter 20: 18, 20, 24 Chapter 22: 34, 36 Read Chapters 23 &
A Study on Speaker Adaptation of Continuous Density HMM Parameters By Chin-Hui Lee, Chih-Heng Lin, and Biing-Hwang Juang Presented by: 陳亮宇 1990 ICASSP/IEEE.
Test of a Population Median. The Population Median (  ) The population median ( , P 50 ) is defined for population T as the value for which the following.
Test of a Population Median. The Population Median (  ) The population median ( , P 50 ) is defined for population T as the value for which the following.
The IceCube Neutrino Observatory is a cubic kilometer detector currently under construction at the geographic South Pole. We will give an overview of searches.
IC40 Flares analysis update Mike Baker PS call, Jan
Hierarchical Segmentation of Polarimetric SAR Images
IceCube: Multiwavelength Approach to
CJT 765: Structural Equation Modeling
Search for point-like source in ANTARES
AMANDA-II Point Source Search Results
Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.
Claudio Bogazzi * - NIKHEF Amsterdam ICRC 2011 – Beijing 13/08/2011
Greg Knowles ECE Fall 2004 Professor Yu Hu Hen
Bootstrap Segmentation Analysis and Expectation Maximization
Time-Dependent Searches for Neutrino Point Sources with IceCube
Presentation transcript:

Collaboration Meeting Moscow, 6-10 Jun 2011 Collaboration Meeting Moscow, 6-10 Jun 2011 Agustín Sánchez Losa IFIC (CSIC – Universitat de València)

Outline Flare Analysis Maximum Likelihood Blocks – Likelihood – Algorithms Brute force 1 Change Point 2 Change Point – Stop criteria Likelihood threshold Prior study Fixed Prior To-Do List 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting2

Flare Analysis Find coincidences of gamma flares with neutrino events in ANTARES Correlation proportional to the intensity of source’s flare light Flare periods and intensities have to be identified by denoising the light curves. Use of light curves measured by satellites (FERMI, SWIFT,...) 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting3

Flare Analysis Different methods already described and used in the literature: – “Studies in astronomical time series analysis V” Scargle J D 1998 ApJ 601, 151 – “IceCube: Multiwavelength search for neutrinos from transient point sources” Resconi E 2007 J. Phys.: Conf. Ser – “On the classification of flaring states of blazars” Resconi E 2009 arXiv: v1 – “Studies in astronomical time series analysis VI” Scargle’s Draft (2006) at his web page: “ – Etcetera. Scargle’s Draft describe a general method for different data types and some clues in the way to stop the algorithm Finally studied method is the one for binned data perturbed by a known Gaussian error 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting4

Maximum Likelihood Blocks General idea: – Data available (light curves): {x n,σ n,t n } → {flux,error,time} – Divide the data in significant blocks of approximately constant rate in light emission, chosen and guided by a proper likelihood function – Flare duration irrelevant: not sensitive to the time value or gaps but the consecutive order of the flux values and their error Algorithm: – Divide the data interval from only one block until N blocks (where N is the number of total data points) – Do it in an order that, every time the number of blocks is increased, try to represent qualitatively the different rate periods as best as possible (chosen likelihood) – Decide when to stop 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting5

Maximum Likelihood Blocks Each cell represent a {x n,σ n,t n } data A group of cells conform a block The first cell of a block define a Change Point (CP) 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting6 Another block Another CP Every time the number of blocks is increased look, with a likelihood, for the cells in which is the most probable changes in the flux rate, considered constant inside the block, i.e. look for the most optimum CPs

Maximum Likelihood Blocks The likelihood for each block k, assuming that data, n, in that block comes from a constant rate λ perturbed by a Gaussian error, is: The constant rate λ that maximize that likelihood is: The total logarithmic likelihood, dropping the constant terms that are going to contribute always the same amount, is: 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting7

Maximum Likelihood Blocks The remaining free parameters are the beginning of each block, or CPs (change points), and the total number of blocks, M, i.e. the total amount of CPs The likelihood maximize with so many blocks as the total amount of data, i.e. a block or a CP for each data point, M = N Once this is done remains the choice of the number of blocks to use, M 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting8

Algorithm Brute force: try all possible CPs in the data, and chose the ones that maximize the likelihood. Too expensive in computing time. One CP per iteration: in every step is maintained all the previously found CPs as the optimum ones and only a new CP is added, the one who maximize the likelihood. The best in time computing possible, and not a bad approximation at all, but some flares are more difficult to be found. Two CPs per iteration: in every step “the 2 best CPs inside each block” are compared and chosen the pair which maximize the likelihood. Shows better capacity to detect evident flares in less number of blocks, M Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting9 1CP 2CP

1CP 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting10 One CP testing... Sample for the FERMI source 3C454.3 Likelihood Number of blocks 1-Log(L)/Log(L max ) 1CP 130 days

1CP 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting11 One CP testing... Likelihood Number of blocks 1-Log(L)/Log(L max ) 1CP Sample for the FERMI source PMNJ “found-a-flare-gap” 50 days 50 days

Two CP testing... 2CP 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting12 Sample for the FERMI source 3C454.3 Likelihood Number of blocks 1-Log(L)/Log(L max ) 2CP 130 days

Two CP testing... 2CP 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting13 Likelihood Number of blocks 1-Log(L)/Log(L max ) 2CP Sample for the FERMI source PMNJ days 50 days

6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting14 Two CP testing... Small sample for the FERMI source 3C CP 300 days 300 days

6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting15 Two CP testing... Small sample for the FERMI source 3C CP 300 days 300 days

Stop criteria The chosen algorithm determine the order in which the CPs are added The choice of when stop have been studied for the “Two CP per iteration” algorithm: – Likelihood threshold or similar: Stop when the likelihood value is a given percentage of the maximum likelihood. Not really useful criterion due to the different evolution of the likelihoods. – Scargle’s Prior Study: Made a study of the most optimum γ for this prior: to add to the logarithm likelihood in order to create a maximum. Pretty complicated and does not work with “flat-flared” light curves. – Fixed Scargle’s Prior: In the Scargle’s Draft is mentioned that γ should yields γ≈N, and that has been observed in the previous stop criterion. With this fixed value all flares are now detected but implies a bigger number of unnecessary blocks for the “easy-flares” light curves Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting16

The chosen algorithm determine the order in which the CPs are added The choice of when stop have been studied for the “Two CP per iteration” algorithm: – Likelihood threshold or similar: Stop when the likelihood value is a given percentage of the maximum likelihood. Not really useful criterion due to the different evolution of the likelihoods. – Scargle’s Prior Study: Made a study of the most optimum γ for this prior: to add to the logarithm likelihood in order to create a maximum. Pretty complicated and does not work with “flat-flared” light curves. – Fixed Scargle’s Prior: In the Scargle’s Draft is mentioned that γ should yields γ≈N, and that has been observed in the previous stop criterion. With this fixed value all flares are now detected but implies a bigger number of unnecessary blocks for the “easy-flares” light curves. Stop criteria C Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting17 Number of blocks 1-Log(L)/Log(L max ) Number of blocks 1-Log(L)/Log(L max )

The chosen algorithm determine the order in which the CPs are added The choice of when stop have been studied for the “Two CP per iteration” algorithm: – Likelihood threshold or similar: Stop when the likelihood value is a given percentage of the maximum likelihood. Not really useful criterion due to the different evolution of the likelihoods. – Scargle’s Prior Study: Made a study of the most optimum γ for this prior: to add to the logarithm likelihood in order to create a maximum. Pretty complicated and does not work with “flat-flared” light curves. – Fixed Scargle’s Prior: In the Scargle’s Draft is mentioned that γ should yields γ≈N, and that has been observed in the previous stop criterion. With this fixed value all flares are now detected but implies a bigger number of unnecessary blocks for the “easy-flares” light curves Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting18 Stop criteria Simulation of the light curve Application of the algorithm With different γ values different optimum blocks for the samples and different error between real light curve and obtained blocks

The chosen algorithm determine the order in which the CPs are added The choice of when stop have been studied for the “Two CP per iteration” algorithm: – Likelihood threshold or similar: Stop when the likelihood value is a given percentage of the maximum likelihood. Not really useful criterion due to the different evolution of the likelihoods. – Scargle’s Prior Study: Made a study of the most optimum γ for this prior: to add to the logarithm likelihood in order to create a maximum. Pretty complicated and does not work with “flat-flared” light curves. – Fixed Scargle’s Prior: In the Scargle’s Draft is mentioned that γ should yields γ≈N, and that has been observed in the previous stop criterion. With this fixed value all flares are now detected but implies a bigger number of unnecessary blocks for the “easy-flares” light curves. Stop criteria 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting19 40 days 40 days 70 days 70 days 100 days 100 days 1000 days 1000 days

To Do List Develop a base line estimator in order to define the flare period and build time PDFs Start real data analyzing with those time PDFs 6-10 Jun 2011Agustín Sánchez Losa – Moscow Collaboration Meeting20