NASSP Masters 5003F - Computational Astronomy - 2009 Lecture 3 First, a bit more python. Then some noise statistics.

Slides:



Advertisements
Similar presentations
Modeling of Data. Basic Bayes theorem Bayes theorem relates the conditional probabilities of two events A, and B: A might be a hypothesis and B might.
Advertisements

NASSP Masters 5003F - Computational Astronomy Lecture 5: source detection. Test the null hypothesis (NH). –The NH says: let’s suppose there is no.
NASSP Masters 5003F - Computational Astronomy Lecture 13 Further with interferometry – Resolution and the field of view; Binning in frequency and.
EVSC 495/EVAT 795 Data Analysis & Climate Change Class hours: TuTh 2:00-3:15 pm Instructor: Michael E. Mann.
Lecture 6 Power spectral density (PSD)
DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY STOCHASTIC SIGNALS AND PROCESSES Lecture 1 WELCOME.
G. Cowan Lectures on Statistical Data Analysis 1 Statistical Data Analysis: Lecture 10 1Probability, Bayes’ theorem, random variables, pdfs 2Functions.
Error Propagation. Uncertainty Uncertainty reflects the knowledge that a measured value is related to the mean. Probable error is the range from the mean.
G. Cowan Lectures on Statistical Data Analysis Lecture 2 page 1 Statistical Data Analysis: Lecture 2 1Probability, Bayes’ theorem 2Random variables and.
Spike-triggering stimulus features stimulus X(t) multidimensional decision function spike output Y(t) x1x1 x2x2 x3x3 f1f1 f2f2 f3f3 Functional models of.
A gentle introduction to Gaussian distribution. Review Random variable Coin flip experiment X = 0X = 1 X: Random variable.
Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.
Environmental Data Analysis with MatLab Lecture 24: Confidence Limits of Spectra; Bootstraps.
G. Cowan Lectures on Statistical Data Analysis 1 Statistical Data Analysis: Lecture 8 1Probability, Bayes’ theorem, random variables, pdfs 2Functions of.
G. Cowan Lectures on Statistical Data Analysis Lecture 14 page 1 Statistical Data Analysis: Lecture 14 1Probability, Bayes’ theorem 2Random variables and.
Review of Probability and Random Processes
Lecture 16 Random Signals and Noise (III) Fall 2008 NCTU EE Tzu-Hsien Sang.
G. Cowan Lectures on Statistical Data Analysis Lecture 10 page 1 Statistical Data Analysis: Lecture 10 1Probability, Bayes’ theorem 2Random variables and.
G. Cowan Lectures on Statistical Data Analysis 1 Statistical Data Analysis: Lecture 7 1Probability, Bayes’ theorem, random variables, pdfs 2Functions of.
(1) A probability model respecting those covariance observations: Gaussian Maximum entropy probability distribution for a given covariance observation.
Normal and Sampling Distributions A normal distribution is uniquely determined by its mean, , and variance,  2 The random variable Z = (X-  /  is.
Review of Probability.
Prof. SankarReview of Random Process1 Probability Sample Space (S) –Collection of all possible outcomes of a random experiment Sample Point –Each outcome.
Probability Theory and Random Processes
NASSP Masters 5003F - Computational Astronomy Lecture 4 Random variables continued. Monte Carlo Uncertainty propagation Correct presentation of.
NASSP Masters 5003F - Computational Astronomy Lecture 7 – chi squared and all that Testing for goodness-of-fit continued. Uncertainties in the fitted.
Overview course in Statistics (usually given in 26h, but now in 2h)  introduction of basic concepts of probability  concepts of parameter estimation.
Noise & Uncertainty ASTR 3010 Lecture 7 Chapter 2.
G. Cowan Lectures on Statistical Data Analysis Lecture 3 page 1 Lecture 3 1 Probability (90 min.) Definition, Bayes’ theorem, probability densities and.
NASSP Masters 5003F - Computational Astronomy UNIX* intro We’ll use the ‘command line’ – windows are for wimps. The unix ‘shell’: –This is what.
Random Processes ECE460 Spring, Power Spectral Density Generalities : Example: 2.
NASSP Masters 5003S - Computational Astronomy Lecture 1 Aim: –Become familiar with astronomy data –Learn a bit of programming No set text – –Web.
Introduction to Digital Signals
NASSP Masters 5003F - Computational Astronomy Lecture 6 Objective functions for model fitting: –Sum of squared residuals (=> the ‘method of least.
NASSP Masters 5003F - Computational Astronomy Lecture 14 Reprise: dirty beam, dirty image. Sensitivity Wide-band imaging Weighting –Uniform vs Natural.
ارتباطات داده (883-40) فرآیندهای تصادفی نیمسال دوّم افشین همّت یار دانشکده مهندسی کامپیوتر 1.
Math 3360: Mathematical Imaging Prof. Ronald Lok Ming Lui Department of Mathematics, The Chinese University of Hong Kong Lecture 11: Types of noises.
IE 300, Fall 2012 Richard Sowers IESE. 8/30/2012 Goals: Rules of Probability Counting Equally likely Some examples.
NASSP Masters 5003F - Computational Astronomy Lecture 12 Complex numbers – an alternate view The Fourier transform Convolution, correlation, filtering.
Chapter 1 Random Process
NASSP Masters 5003F - Computational Astronomy Lecture 16 Further with interferometry – Digital correlation Earth-rotation synthesis and non-planar.
1/18/2016Atomic Scale Simulation1 Definition of Simulation What is a simulation? –It has an internal state “S” In classical mechanics, the state = positions.
G. Cowan Lectures on Statistical Data Analysis Lecture 8 page 1 Statistical Data Analysis: Lecture 8 1Probability, Bayes’ theorem 2Random variables and.
Lecture 8 Source detection NASSP Masters 5003S - Computational Astronomy
Discrete-time Random Signals
Outline Transmitters (Chapters 3 and 4, Source Coding and Modulation) (week 1 and 2) Receivers (Chapter 5) (week 3 and 4) Received Signal Synchronization.
Baseband Receiver Receiver Design: Demodulation Matched Filter Correlator Receiver Detection Max. Likelihood Detector Probability of Error.
Random Processes Gaussian and Gauss-Markov processes Power spectrum of random processes and white processes.
G. Cowan Lectures on Statistical Data Analysis Lecture 9 page 1 Statistical Data Analysis: Lecture 9 1Probability, Bayes’ theorem 2Random variables and.
The Chinese University of Hong Kong
1 Review of Probability and Random Processes. 2 Importance of Random Processes Random variables and processes talk about quantities and signals which.
G. Cowan Lectures on Statistical Data Analysis Lecture 5 page 1 Statistical Data Analysis: Lecture 5 1Probability, Bayes’ theorem 2Random variables and.
G. Cowan Lectures on Statistical Data Analysis Lecture 10 page 1 Statistical Data Analysis: Lecture 10 1Probability, Bayes’ theorem 2Random variables and.
Geology 6600/7600 Signal Analysis 04 Sep 2014 © A.R. Lowry 2015 Last time: Signal Analysis is a set of tools used to extract information from sequences.
Digital Communications I: Modulation and Coding Course Spring Jeffrey N. Denenberg Lecture 3c: Signal Detection in AWGN.
CARMA Models for Stochastic Variability (or how to read a PSD) Jim Barrett University of Birmingham.
Bayesian Brain Probabilistic Approaches to Neural Coding 1.1 A Probability Primer Bayesian Brain Probabilistic Approaches to Neural Coding 1.1 A Probability.
6/11/2016Atomic Scale Simulation1 Definition of Simulation What is a simulation? –It has an internal state “S” In classical mechanics, the state = positions.
Institute for Experimental Mathematics Ellernstrasse Essen - Germany DATA COMMUNICATION introduction A.J. Han Vinck May 10, 2003.
디지털통신 Random Process 임 민 중 동국대학교 정보통신공학과 1.
NASSP Masters 5003F - Computational Astronomy Lecture 4: mostly about model fitting. The model is our estimate of the parent function. Let’s express.
Basic Counting StatisticsBasic Counting StatisticsBasic Counting StatisticsBasic Counting Statistics.
A 2 veto for Continuous Wave Searches
Parameter Estimation and Fitting to Data
Outline Introduction Signal, random variable, random process and spectra Analog modulation Analog to digital conversion Digital transmission through baseband.
The Chinese University of Hong Kong
Introduction to Probability & Statistics The Central Limit Theorem
POINT ESTIMATOR OF PARAMETERS
Computing and Statistical Data Analysis / Stat 8
Uncertainty Propagation
Presentation transcript:

NASSP Masters 5003F - Computational Astronomy Lecture 3 First, a bit more python. Then some noise statistics.

NASSP Masters 5003F - Computational Astronomy Python ins and outs We’re going to mostly read our data from FITS files, using a module called pyfits. – ware/pyfits We’ll crunch up the data using a module called numpy. – For graphical output we’ll use module ppgplot, (2 ps) which is a (fairly crude) wrapper to a package called PGPLOT. – – BREAKING NEWS! pylab is better for graphical output.

NASSP Masters 5003F - Computational Astronomy Manuals and example code You won’t need to download manuals for pyfits, numpy, scipy or pylab. I’ll do so and make them available from my home page. It’ll be much quicker for you to copy (or simply access) them from there. Some of these manuals are huge – I recommend you neither print or read them in entirety, but rather: –Read the bits you need using acroread. –Look at the example code I’ll provide you.

NASSP Masters 5003F - Computational Astronomy Random variables – probability density x p(x) Average μ: Variance σ 2 : Units: probability per unit x.

NASSP Masters 5003F - Computational Astronomy Random variables – probability density x p(x) Estimate of μ: Estimate of σ 2 :

NASSP Masters 5003F - Computational Astronomy Random variables – probability density x p(x)

NASSP Masters 5003F - Computational Astronomy Random variables – probability density x p(x) x0x0

NASSP Masters 5003F - Computational Astronomy p(x) Noise 2 most important distributions: –Gaussian …but not necessarily ‘white’

NASSP Masters 5003F - Computational Astronomy Noise –Poisson (= i) …but note Central Limit theorem.

NASSP Masters 5003F - Computational Astronomy Gauss and Poisson

NASSP Masters 5003F - Computational Astronomy Combinations of random variables Weighted average: Uncertainty propagation: if y = f(a,b), Uncertainty in this: Best SNR when: where:

NASSP Masters 5003F - Computational Astronomy Filtering and correlation Uncorrelated ie ‘white’ noise (with a Gaussian probability distribution.)

NASSP Masters 5003F - Computational Astronomy Filtering and correlation Fourier transformed – looks the same – same power at all f – hence ‘white’.

NASSP Masters 5003F - Computational Astronomy Filtering and correlation Autocorrelation function. Power spectrum: where F is the Fourier transform of signal f, eg: The autocorrelation function is:

NASSP Masters 5003F - Computational Astronomy Filtering and correlation Filtering operation correlates the noise. It is still Gaussian but no longer white.

NASSP Masters 5003F - Computational Astronomy Filtering and correlation FT is dominated in this case by low frequencies.

NASSP Masters 5003F - Computational Astronomy Filtering and correlation Autocorrelation function is broadened.

NASSP Masters 5003F - Computational Astronomy Signals and noise Natural signal + background Instrumental filtering/correlation General: + uncorrelated noise + uncorrelated noise Natural signal + background Instrumental filtering/correlation XMM-Newton: + uncorrelated noise + uncorrelated noise Natural signal + background Instrumental filtering/correlation Interferometry: + uncorrelated noise + uncorrelated noise

NASSP Masters 5003F - Computational Astronomy Signal detection Much harder.An obvious source.

NASSP Masters 5003F - Computational Astronomy Signal detection Parent function; data; model. Probability that the data results from a certain model: Reduced chi 2 : divide by deg free. Recall that P signal = 1-P no signal. Many types of signal, but only 1 no-signal. Hence test model with no source. Called the ‘null hypothesis’.

NASSP Masters 5003F - Computational Astronomy Signal detection Null hypothesis requires: –Perfect knowledge of background B. –A good estimate of σ. Sparse sources among gaussian: ok. But… –what about poisson bins with zero counts? Answer: Maximum Likelihood. –what about crowded fields? Answer: Bayes..? –can we trust the distribution? Answer: Monte Carlo.