1. Astronomy Perspective
Ofer Lahav
University College London

2. SCMA IV Cosmology (I, II) Small-N problems (incl. HEP)
Astronomical surveys
Planetary systems
Periodic variability
Developments in statistics
Cross-disciplinary perspectives

3. Astro-Statistics Data Compression Classification Reconstruction
Feature extraction
Parameter estimation
Model selection

4. Astro-Statistics Books
Babu & Feigelson (1992)
Lupton (1993)
Martinez & Saar (2002)
Wall & Jenkins (2003)
Saha (2003)
Gregory (2005) …

5. “That is the curse of statistics, that it can never prove things, only disprove them!
At best, you can substantiate a hypothesis by ruling out, statistically, a whole long list of competing hypotheses, every one that has ever been proposed.
After a while your adversaries and competitors will give up trying to think of alternative hypotheses, or else they will grow old and die, and then your hypothesis will become accepted.
Sounds crazy, we know, but that’s how science works!“
Press et al., Numerical Recipes

6. Methodology & Approaches
Frequentist
Probability is interpreted as the frequency
of the outcome of a repeatable experiment.
Bayesian
The interpretation of probability is more general
and includes ‘a degree of belief’.
* “The information in the data” vs.
“the information about something”

7. Bayes’ Theorem P(A|B) = P(B|A) P(A) / P(B) P(model | data)=
P(data | model) P (model) / P(data)
↑ ↑ ↑
Likelihood Prior Evidence
exp (-2 /2)
(paper only published in 1764)

8. Ed Jaynes (1984) on Bayesian Methods
“communication problems… a serious disease that has afflicted probability theory for 200 years.
There is a long history of confusion and controversy, leading in some cases to a paralytic inability to communicate…”

9. How to choose a prior? * Theoretical prejudice
(e.g. “according to Inflation the universe must be flat” )
* Previous observations
(e.g. “we know from WMAP the universe
is flat to within 2%” )
* Parameterized ignorance ( e.g. ``a uniform prior,
Jeffrey’s prior, or Entropy prior?” )

10. Recent trends Astro-Statistics is more ‘respectable’.
Bayesian approaches are more common,
in co-existence with frequentist methods
More awareness of model selection methods (e.g. AIC, BIC, …)
Computer intensive methods (e.g. MCMC)
are more popular.
* Free packages

11. The Doppler detection method

12. Gregory 2005

13. P=190 days
Gregory 05

14. P=128 days

15. P= 376 days

17. Photometric redshift Probe strong spectral features (4000 break)
Difference in flux through filters as the galaxy is redshifted.

18. Bayesian Photo-z
likelihood
prior
Redshift z
Benitez (BPZ)

19. ANNz - Artificial Neural Network
Output:
redshift
Input:
magnitudes
Collister & Lahav 2004

20. Example: SDSS data (ugriz; r < 17.77)
ANNz (5:10:10:1)
HYPERZ
Collister & Lahav 2004

21. MegaZ-LRG. Training on ~13,000 2SLAQ
MegaZ-LRG *Training on ~13,000 2SLAQ *Generating with ANNz Photo-z for ~1,000,000 LR over 5,000 sq deg
z = 0.046
Collister, Lahav,
Blake et al.

22. Photo-z for SDSS: comparison

23. Cosmology in 1986
Galaxy redshift surveys of thousands of galaxies (CfA1, IRAS)
CMB fluctuations not detected yet
Peculiar velocities popular (S7)
“Standard Cold Dark Matter”
m = 1, =0
H0 = 50 km/sec/Mpc = 1/(19.6 Gyr)

24. The Concordance Model * Reality or ‘Epicycles’? * Sub-components?
* More components?

25. Centre Daily Times Sunday 11 June 2006
“Scientists near end in search for Dark Matter
substance thought to bond universe”

26. Just Six numbers? Baryons b Matter m Dark Energy 
Hubble parameter H0
Amplitude A
Initial shape of perturbations n ¼ 1
Or More?

27. Variations and extensions…
Isocurvature perturbations
Non-Gaussian initial conditions
Non-power-law initial spectrum
Full ionization history
Hot DM, Warm DM, …
Dark energy EoS w(z)
Modified Friedmann eq
Relativistic MOND
Varying ‘constants’
Cosmic Topology …

28. Probes of Dark Matter and Dark Energy
Cosmic Shear
Evolution of dark matter perturbations
Angular diameter distance
Growth rate of structure
Baryon Wiggles
Standard ruler
Angular diameter distance
Supernovae
Standard candle
Luminosity distance
Cluster counts
Evolution of dark matter perturbations
Angular diameter distance
Growth rate of structure
CMB
Snapshot of Universe at ~400,000 yr
Angular diameter distance to z~1000
Growth rate of structure (from ISW)

29. Cosmic Probes CMB Galaxy clustering Cluster abundance
Cluster baryon fraction
Cepheids
SN Ia
IGM Lyman-
Gravitational lensing
Peculiar velocities
ISW
21cm …

30. Sources of uncertainties
Cosmological (parameters and priors)
Astrophysical (e.g. cluster M-T, biasing)
Instrumental (e.g. PSF)

32. From 2dF+CMB (6 parameter fit): m=0.23 §0.02
Cole et al. 2005

33. The SDSS LRG correlation function
Eisenstein et al
2005

34. WMAP3
m =
8 =
n =
 =

35. Statistical measure of shear pattern, ~1% distortion
Background sources
Dark matter halos
Observer
Cosmic Web – bottom up scenario – clusters then filaments then walls (membranes).
Note that filaments not well traced by galaxies & too ephemeral to emit x-rays, so lensing only way to detect.
Statistical measure of shear pattern, ~1% distortion
Radial distances depend on geometry of Universe
Foreground mass distribution depends on growth of structure
A. Taylor

36. > > > > < | | | | Shapelets
Decompose a galaxy into a set of shapelets:
>
> | |
> |
= a00
+ a01 +…
<
> |
aij =
Refregier 2003
where the basis states are based on orthogonal polynomials (SHO eigenstates).
This can generate useful methods for measuring lensing (eg Bernstein & Jarvis 2002, Refregier & Bacon 2003, Goldberg & Bacon 2005) by forming estimators for shear or flexion from aij.

37. Recent w from the CTIO W=P/ W=-0.894+0.156 -0.208 Einstein told us
Jarvis & Jain, astro-ph/

38. Surveys to measure Dark Energy
2010
2015
2005
Imaging
CFHTLS
SUBARU
DES
LSST
SKA
SDSS
ATLAS
KIDS
VISTA
JDEM/
SNAP
Pan-STARRS
Spectroscopy
ATLAS
SKA
FMOS
KAOS
SDSS
Supernovae
Pan-STARRS
DES
LSST
JDEM/
SNAP
CFHTLS
CSP
Clusters
AMI
SZA
APEX
AMIBA
SPT
ACT
DES
CMB
WMAP 2/3
WMAP 6 yr
Planck
Planck 4yr
2005
2010
2015

39. Dark Energy
Task Force

40. Multi-parameter Estimation
Fisher matrix
Fisher (1935)
Tegmark, Taylor & Heavens(1997)
Rocha et al. (2004)

41. DES Forecasts: Power of Multiple Techniques
Assumptions:
Clusters:
8=0.75, zmax=1.5,
WL mass calibration
(no clustering)
BAO: lmax=300
WL: lmax=1000
(no bispectrum)
Statistical+photo-z
systematic errors only
Spatial curvature, galaxy bias
marginalized
Planck CMB prior
w(z) =w0+wa(1–a) % CL
geometric+
growth
Clusters
if 8=0.9
geometric
Frieman, Ma, Weller, Tang,
Huterer, etal

42. Mock Universes:
Models vs. Epoch

43. Wiener Reconstruction of density and velocity fields
Erdogdu, OL, Huchra et al

44. Wiener reconstruction- Big Supercluster
Erdogdu, OL, Zroubi et al. 2004

45. PCA for data compression (by a factor ~1000)
h = 0.5 pc1 – pc2
Madgwick, OL et al.

46. Gravitational Waves (LIGO, LISA…)

47. Further input much needed from statistics
Model selection methodology
MCMC machinery and extensions
Detection of non-Gaussianity and shape finders
Blind de-convolution (eg. PSF)
Object classification
Comparing simulations with data
Visualisation
VO technology

49. Globalisation and the New Astronomy
One definition of globalisation:
“A decoupling of space and time - emphasising that with instantaneous communications, knowledge and culture can be shared around the world simultaneously.”

50. Globalisation and the New Astronomy
How is the New Astronomy affected by globalisation?
Free information (WWW), big international projects,
numerous conferences, telecons…
Recall the Cold War era:
Hot Dark Matter/top-down (Russia)
vs. Cold Dark Matter/bottom-up (West)
Is the agreement on the `concordance model’ a product of globalisation?

51. Globalisation and the New Astronomy
Independent communities are beneficial,
but eventually they should
talk to each other!

52. Conclusions Fundamental issues in statistics will not go away!
Real Data vs. Mock data: the Virtual Observatories
Great need for interaction of astronomers with experts in other fields

53. Thanks! Co-organisers: Jogesh Babu, Eric Feigelson
SOC: JB, EF, Jim Berger, Kris Gorski, Thomas Laredo, Vicent Martinez, Larry Wasserman, Michael Woodroofe
Grad Students: Hyunsook Lee, Derek Young
Conference Planner: John Farris
Sponsors: SAMSI, NSF, NASA, IMS, PSU