Probability and Statistics

Slides:



Advertisements
Similar presentations
Modeling of Data. Basic Bayes theorem Bayes theorem relates the conditional probabilities of two events A, and B: A might be a hypothesis and B might.
Advertisements

Naïve Bayes. Bayesian Reasoning Bayesian reasoning provides a probabilistic approach to inference. It is based on the assumption that the quantities of.
Week 11 Review: Statistical Model A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution.
Uncertainty Everyday reasoning and decision making is based on uncertain evidence and inferences. Classical logic only allows conclusions to be strictly.
Introduction to Bayesian Statistics Harry R. Erwin, PhD School of Computing and Technology University of Sunderland.
Parameter Estimation: Maximum Likelihood Estimation Chapter 3 (Duda et al.) – Sections CS479/679 Pattern Recognition Dr. George Bebis.
Predictive Automatic Relevance Determination by Expectation Propagation Yuan (Alan) Qi Thomas P. Minka Rosalind W. Picard Zoubin Ghahramani.
Basics of Statistical Estimation. Learning Probabilities: Classical Approach Simplest case: Flipping a thumbtack tails heads True probability  is unknown.
Machine Learning CMPT 726 Simon Fraser University
. PGM: Tirgul 10 Parameter Learning and Priors. 2 Why learning? Knowledge acquisition bottleneck u Knowledge acquisition is an expensive process u Often.
Introduction to Signal Estimation. 94/10/142 Outline 
Matched Filters By: Andy Wang.
CS Bayesian Learning1 Bayesian Learning. CS Bayesian Learning2 States, causes, hypotheses. Observations, effect, data. We need to reconcile.
Introduction To Signal Processing & Data Analysis
Jeff Howbert Introduction to Machine Learning Winter Classification Bayesian Classifiers.
Chapter 4 Probability Copyright © 2014 by The McGraw-Hill Companies, Inc. All rights reserved.McGraw-Hill/Irwin.
Review of Probability.
Statistical Analysis of Systematic Errors and Small Signals Reinhard Schwienhorst University of Minnesota 10/26/99.
EE513 Audio Signals and Systems Statistical Pattern Classification Kevin D. Donohue Electrical and Computer Engineering University of Kentucky.
Binary Variables (1) Coin flipping: heads=1, tails=0 Bernoulli Distribution.
Dept. of EE, NDHU 1 Chapter Three Baseband Demodulation/Detection.
1 Probability and Statistics  What is probability?  What is statistics?
Dr. Gary Blau, Sean HanMonday, Aug 13, 2007 Statistical Design of Experiments SECTION I Probability Theory Review.
1 Foundations of Statistical Natural Language Processing By Christopher Manning & Hinrich Schutze Course Book.
Naive Bayes Classifier
Basic Concepts of Discrete Probability (Theory of Sets: Continuation) 1.
CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 11: Bayesian learning continued Geoffrey Hinton.
CS 782 – Machine Learning Lecture 4 Linear Models for Classification  Probabilistic generative models  Probabilistic discriminative models.
Optimal Bayes Classification
BCS547 Neural Decoding. Population Code Tuning CurvesPattern of activity (r) Direction (deg) Activity
Mathematical Foundations Elementary Probability Theory Essential Information Theory Updated 11/11/2005.
2. Introduction to Probability. What is a Probability?
Education as a Signaling Device and Investment in Human Capital Topic 3 Part I.
ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
Baseband Receiver Receiver Design: Demodulation Matched Filter Correlator Receiver Detection Max. Likelihood Detector Probability of Error.
GWDAW10, UTB, Dec , Search for inspiraling neutron star binaries using TAMA300 data Hideyuki Tagoshi on behalf of the TAMA collaboration.
Introduction to Estimation Theory: A Tutorial
ELEC 303 – Random Signals Lecture 17 – Hypothesis testing 2 Dr. Farinaz Koushanfar ECE Dept., Rice University Nov 2, 2009.
Statistical NLP: Lecture 4 Mathematical Foundations I: Probability Theory (Ch2)
Performance of Digital Communications System
Naïve Bayes Classifier April 25 th, Classification Methods (1) Manual classification Used by Yahoo!, Looksmart, about.com, ODP Very accurate when.
Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Chapter 4 Probability.
Bayesian Brain Probabilistic Approaches to Neural Coding 1.1 A Probability Primer Bayesian Brain Probabilistic Approaches to Neural Coding 1.1 A Probability.
Naive Bayes Classifier. REVIEW: Bayesian Methods Our focus this lecture: – Learning and classification methods based on probability theory. Bayes theorem.
ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
Overview G. Jogesh Babu. R Programming environment Introduction to R programming language R is an integrated suite of software facilities for data manipulation,
24 th Pacific Coast Gravity Meeting, Santa Barbara LIGO DCC Number: G Z 1 Search for gravitational waves from inspiraling high-mass (non-spinning)
Search for gravitational waves from binary inspirals in S3 and S4 LIGO data. Thomas Cokelaer on behalf of the LIGO Scientific Collaboration.
Lecture 1.31 Criteria for optimal reception of radio signals.
Oliver Schulte Machine Learning 726
Chapter 3: Maximum-Likelihood Parameter Estimation
Chapter 4 Probability.
Naive Bayes Classifier
Avi Vajpeyi Rory Smith, Jonah Kanner LIGO SURF 16
Lecture 1.30 Structure of the optimal receiver deterministic signals.
Alan Qi Thomas P. Minka Rosalind W. Picard Zoubin Ghahramani
Bayes Net Learning: Bayesian Approaches
Basic Probability Theory
Special Topics In Scientific Computing
Matched Filtering Junwei Cao (曹军威) and Junwei Li (李俊伟)
Discrete Event Simulation - 4
Signal Processing: Propaedeutics
Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.
Statistical NLP: Lecture 4
10701 / Machine Learning Today: - Cross validation,
Search for gravitational waves from binary black hole mergers:
ELEC6111: Detection and Estimation Theory Course Objective
EE513 Audio Signals and Systems
J. Ellis, F. Jenet, & M. McLaughlin
Pattern Recognition and Machine Learning Chapter 2: Probability Distributions July chonbuk national university.
Presentation transcript:

Probability and Statistics Junwei Cao (曹军威) and Junwei Li (李俊伟) Tsinghua University Gravitational Wave Summer School Kunming, China, July 2009

Drawback of Matched Filtering Two premises of matched filtering Premise 1: A given signal is present in data stream Premise 2: The form of h(t) is known Premises are impractical Repeat matched filtering with many different filters A number of “events” are extracted “events” indicate that in the detector happened something, which deserves further scrutiny

Definition of Probability Consider a set S with subsets A, B… Define probability P as a real function For every A in S, P(A)≥0 For disjoint subsets (i.e. A∩B= ), P(A∪B)=P(A)+P(B) P(S)=1 Further more, conditional probability

Two Approaches of Probability Frequentist (also called classical) A, B, … are the outcome of a repeatable experiment, and P(A) is defined as the frequency of occurrence of A In considering the conditional probability, such as P(data | hypothesis), one is never allowed to think about the probability that the parameters take a given a value, nor of the probability that a hypothesis is correct

Two Approaches of Probability(contd.) Bayesian approach Bayes’ theorem

Prior and Posterior Probability Likelihood function Prior probability

Which One to be Chosen Depend on the type of experiment Elementary particle physics is suited for the classical approach since it is the physicist that controls the parameters of the experiment In astrophysics, the sources can be rare, and each one is very interesting individually, e.g. a single BH-BH binary coalesces

Before Parameter Estimation A number of free parameters A family of possible templates Denoted generically as is a collection of parameters A family of optimal filters Determined by eq. 2.12, Must discretize the θ-space For some template, the SNR exceeds a predefined threshold, indicating a detection

Parameter Estimation How to reconstruct parameters of the source Assume that n(t) is stationary and Gaussian Corresponding Gaussian probability distribution for n(t) This is the probability that n(t) has a given realization n0(t)

Likelihood Function Assumption θt is the true value of the parameters θ Likelihood function for s(t), by plugging n0=s-h(θt) into 3.5 Introduce ht≡h(θt)

Posterior Probability Distribution Introduce a prior probability to eq. 3.7 can be an un-flat prior in θt Estimator: a rule for assigning , the most probable value of θt Consistency The bias Efficiency Robustness

Maximum Likelihood Estimator First, the prior probability is flat Max. posterior  Max. likelihood Denote the maximum likelihood estimator by Simpler to maximize Define

Maximum Posterior Probability Maximize the full posterior probability Takes into account the prior probability distribution Non-trivial prior information Example Two-dimensional parameters space (θ1,θ2) Only interested in θ1 Drawback An ambiguity on the value of the most probable value of θ1 Unable to minimize the error on θ1 determination

Bayes Estimator Most probable value of parameters Errors The “operational” meaning Is the value of , averaged over an ensemble of same outputs Drawback: computational costs

Thank You ! Junwei Cao jcao@tsinghua.edu.cn http://ligo.org.cn