Inference in action: the force law in the Solar System Jo Bovy Center for Cosmology and Particle Physics New York University IMPRS Summer School: Statistical Inferences from Astrophysical Data 2009/08/14

Introduction >>''Bayesians'' vs. ''Frequentists'': Plenty of theory and idealized examples, but no ''real'' example this week >>This talk: - Actual astrophysically (somewhat) relevant application - Frequentists vs. Bayesians in practice: strengths and weaknesses of both - Bayes is a framework for inference, not the nec plus ultra: Inference is not just blindly applying Bayes's theorem: exploration, forward modeling often requires careful thought

The general problem >>Infer the dynamics of the Milky Way (the gravitational potential Φ) given a set of positions xi and velocities vi (think Gaia) >>Why is this hard? - Often large uncertainties (e.g., distances) - Often missing data (e.g., line-of-sight velocities in astrometric missions) - Large numbers of stars (~109 for Gaia): computational problem >>No real problem if we had a forward model: e.g., convolving the model with the data uncertainties deals with large uncertainties and missing data

The forward/generative model >>Do we have a forward model?

The forward/generative model >>Do we have a forward model? >>Newton's second law: x'' = F(x) = -∇Φ(x) --> Φ(x) can be used to forward model x'' = a(x)

The forward/generative model >>Do we have a forward model? >>Newton's second law: x'' = F(x) = -∇Φ(x) --> Φ(x) can be used to forward model x'' = a(x) >>But we do not have a, we have x and v, initial conditions for any Φ(x) >> p({x,v} | Φ) p(Φ) p(Φ| {x,v}) = ------------------------ p({x,v})

The forward/generative model >>Do we have a forward model? >>Newton's second law: x'' = F(x) = -∇Φ(x) --> Φ(x) can be used to forward model x'' = a(x) >>But we do not have a, we have x and v, initial conditions for any Φ(x) >> p({x,v} | Φ) p(Φ) p(Φ| {x,v}) = ------------------------ p({x,v}) ---> Positions and velocities tell you nothing!?

The distribution function >>The way out: the distribution function f(x,v): f(x,v) dx dv = #stars per volume dx dv in 6D phase space ∝ probability to find a star in dx dv >> p(x,v | Φ) ∝ f(x,v) >>By making assumptions about the distribution function (DF) we can learn something about the potential

Possible assumptions >>What kind of assumptions does one make?

Possible assumptions >>What kind of assumptions does one make? >>All tracers are bound: Φcannot be too weak (~ lower bound on the strength of the potential): E.g., MW satellites are bound to the MW -> lower bound on the MW mass

Possible assumptions >>What kind of assumptions does one make? >>All tracers are bound: Φcannot be too weak (~ lower bound on the strength of the potential): E.g., MW satellites are bound to the MW -> lower bound on the MW mass >>Tracers are in a steady state (more on this later): - The basic dynamical assumption used to infer (parts of) the potential of the MW and external galaxies

Possible assumptions >>What kind of assumptions does one make? >>All tracers are bound: Φcannot be too weak (~ lower bound on the strength of the potential): E.g., MW satellites are bound to the MW -> lower bound on the MW mass >>Tracers are in a steady state (more on this later): - The basic dynamical assumption used to infer (parts of) the potential of the MW and external galaxies >>Other informative features in the DF: - E.g., Halo streams trace out orbits -> a (e.g., Koposov, Rix, & Hogg 2009)

Toy problem: the Solar System JB, Hogg, & Murray (2009) >>We asked the question: Can we infer the force law in the Solar System from a snapshot of its kinematics? >>Given: x,v of 8 planets on 2009-Apr-01

Toy problem: the Solar System JB, Hogg, & Murray (2009) >>We asked the question: Can we infer the force law in the Solar System from a snapshot of its kinematics? >>Given: x,v of 8 planets on 2009-Apr-01 >>Find: a(r) = -A (r/1AU)-α >>Focuses on the forward modeling issue, not the large uncertainties, missing data, and computational complexity problems

The data >>We did not measure the positions and velocities, but took them from a Solar System integration service (JPL ephemeris) >>Accurate to ~10-6 – 10-8 fractional uncertainty

Where to start? The virial theorem >>The virial theorem for general power-law potentials says >>For the 8 planets:

Where to start? The virial theorem >>The virial theorem for general power-law potentials says >>For the 8 planets:

Where to start? The virial theorem >>The virial theorem for general power-law potentials says >>For the 8 planets: >>Lines cross near a single point --> System is virialized --> System is in a steady-state

The virial estimate? >>Is there anything we can do with the virial relation?

The virial estimate? >>Is there anything we can do with the virial relation? >>Only one equation for two parameters (here): divide the planets into subsamples? (Inner/outer planets?) + bootstrap the result somehow to get uncertainty estimates on A,α

The virial estimate? >>Is there anything we can do with the virial relation? >>Only one equation for two parameters (here): divide the planets into subsamples? (Inner/outer planets?) + bootstrap the result somehow to get uncertainty estimates on A,α >>Obviously there is some constraint coming from the virial relation, but this is hard to quantiy >>Virial relation is good for exploration: We will from now on assume that the planets are in a steady-state

The steady-state assumption >>Any collisionless system satisfies the collisionless Boltzmann equation:

The steady-state assumption >>Any collisionless system satisfies the collisionless Boltzmann equation: >>In a steady-state the first term is zero:

Jeans theorem >>Jeans's theorem states that in a steady-state the DF is a function of the integrals of the motion alone: f ≣f(I)

Jeans theorem >>Jeans's theorem states that in a steady-state the DF is a function of the integrals of the motion alone: f ≣f(I) >>This way the gravitational potential can sneak in, e.g., E = K + Φ >>Very useful, since we have assumed spherical symmetry: - 4 integrals of the motion: E, L (5 for α = 2) - f ≣f(E,L,[e])

What frequentists do with this >>(Some) Frequentists imagine a world of many realizations of the Solar System and test whether the observed Solar System has the expected properties, and they rule out (with a certain confidence) models that do not have the expected properties >>Expected properties: - distribution of radial angle θ(orbital phase) is uniform - no correlation between E and θ - ...

What frequentists do with this >>(Some) Frequentists imagine a world of many realizations of the Solar System and test whether the observed Solar System has the expected properties, and they rule out (with a certain confidence) models that do not have the expected properties >>Expected properties: - distribution of radial angle θ(orbital phase) is uniform - no correlation between E and θ - ... >>Orbital roulette: Test for a given Φ whether these properties are satisfied and rule out models on this basis (Beloborodov & Levin 2004)

Why orbital roulette works >>Uniform θ distribution: for potentials that are too strong, all planets are at aphelion, for weak potentials all planets are at perihelion

Angles

Angles

Why orbital roulette works >>Uniform θ distribution: for potentials that are too strong, all planets are at aphelion, for weak potentials all planets are at perihelion >>Absence of (E,θ) correlations: e.g., a(r) too strong and to steep: inner planets are at aphelion and strongly bound (large E), outer planets are at perihelion and weakly bound (small E)

Different frequentist tests >>Frequentist procedure: - Choose statistic to test whether the θdistribution is flat (e.g., the mean angle, KS, Kuiper) - Choose statistic to test whether E and θare correlated (e.g., rank test) - Combine these somehow, e.g., Bonferroni correction (conservative) - Rule out models with a certain confidence

Different frequentist tests

Different frequentist tests

So what's wrong with orbital roulette? >>Good: - Conceptually simple, intuitive - Better than virial - Given good data (small uncertainties, no missing data) it does a good job of setting bounds on the potential (e.g., it was used successfully to constrain the black hole mass at the GC, Beloborodov et al. 2006)

So what's wrong with orbital roulette? >>Good: - Conceptually simple, intuitive - Better than virial - Given good data (small uncertainties, no missing data) it does a good job of setting bounds on the potential (e.g., it was used successfully to constrain the black hole mass at the GC, Beloborodov et al. 2006) >>Bad: - Test what? Which test? How to combine different tests? - No way to deal with missing data or large uncertainties - only rules out models, it does not give parameter estimates with uncertainties - It uses a very crude model of the data

''Bayesian'' orbital roulette >>Starting point is the same: f ≣ f(I) >>Assuming spherical symmetry: f ≣f(E,e) >>Bayes's theorem: p(A,α| {x,v}) ∝ p(A,α) Πi J(E,e; θ; x,v) f(Ei,ei| A,α)

But what is the distribution function? >>Not a big problem: we can infer it as well and marginalize over it >>Given only 8 data points we can only hope to infer some crude features of the distribution function, roughly the bounds

But what is the distribution function? >>Not a big problem: we can infer it as well and marginalize over it >>Given only 8 data points we can only hope to infer some crude features of the distribution function, roughly the bounds >>Therefore, we model the distribution function f(ln E, e) as a product of a tophat function in ln E, and a tophat in e: our forward/generative model >>This adds four parameters to the model (two times the edges of the tophat) >> p(A,α| {x,v}) --> p(A,α, ln Emin, ln Emax, emin,emax| {x,v}) >>We can marginalize these DF parameters out, analytically even

But what is the distribution function? >>Not a big problem: we can infer it as well and marginalize over it >>Given only 8 data points we can only hope to infer some crude features of the distribution function, roughly the bounds >>Therefore, we model the distribution function f(ln E, e) as a product of a tophat function in ln E, and a tophat in e: our forward/generative model >>This adds four parameters to the model (two times the edges of the tophat) >> p(A,α| {x,v}) --> p(A,α, ln Emin, ln Emax, emin,emax| {x,v}) >>We can marginalize these DF parameters out, analytically even

Jacobians α=2:

Jacobians α=2:

Results Marginalized over the DF + marginalized over A

Discussion >>Given few data points we get reasonably accurate bounds on α (best possible bounds? e ~ 0.01) >>But, we started from 10-6-10-8 accurate data, and get 10-2 results ---> inferring the DF has cost us a lot of information >>This method can be generalized to missing data problems

Frequentist vs. Bayesian method Frequentist result: ~10 percent measurement

Frequentist vs. Bayesian method Frequentist result: Bayesian result: ~10 percent measurement ~2 percent measurement

Why does the Bayesian method perform better? >>The virial relation holds up to about e2 >>For the frequentist method, the median or mean eccentricity is what matters

Why does the Bayesian method perform better? >>The virial relation holds up to about e2 >>For the frequentist method, the median or mean eccentricity is what matters

Why does the Bayesian method perform better? >>The virial relation holds up to about e2 >>For the frequentist method, the median or mean eccentricity is what matters --> median e =~ 0.05 --> ~10 percent measurement possible

Why does the Bayesian method perform better? >>The virial relation holds up to about e2 >>For the frequentist method, the median or mean eccentricity is what matters --> median e =~ 0.05 --> ~10 percent measurement possible >>But we can do better: planets with the (2) lowest eccentricities have the most constraining power --> smallest e's =~ 0.01 --> ~2 percent measurement possible

Why does the Bayesian method perform better? >>The virial relation holds up to about e2 >>For the frequentist method, the median or mean eccentricity is what matters --> median e =~ 0.05 --> ~10 percent measurement possible >>But we can do better: planets with the (2) lowest eccentricities have the most constraining power --> smallest e's =~ 0.01 --> ~2 percent measurement possible >>The Bayesian method comes close to this bound

Conclusions >>Science is in the forward modeling and making necessary and motivated assumptions, probability theory provides a framework for inference >>Both Bayesian and frequentist methods have interesting things to say in this case >>Bayes outperforms the frequentist methods and is the appropriate framework to use here (Galactic dynamics is a ''mature'' field)

Conclusions >>Science is in the forward modeling and making necessary and motivated assumptions, probability theory provides a framework for inference >>Both Bayesian and frequentist methods have interesting things to say in this case >>Bayes outperforms the frequentist methods and is the appropriate framework to use here (Galactic dynamics is a ''mature'' field) >>And the force law in the Solar System is ∝ r-2