1. Carlos Rodriguez on Geometry and Priors
July 8, 2007
(18)

2. Geometry and Priors

5. Is reality coordinate free ?

6. Is reality coordinate free?
Yes/no
Object/subject
Abstraction/instance
Theory/data

8. Geometrization of Inference
Truth: t(x)
Model: { p(x|)}
Geometrization of Inference

10. Embedding in Hilbert Space
Fisher Information metric automagically induced on the tangent bundle !

11. The Volume Form as Prior
A hypothesis space M is said to be regular when (M,g) is a smooth orientable riemannian manifold. A k-dim regular M has volume form:
In arbitrary (orientation preserving) theta coordinates the volume of (M,g) is:

12. Exaaaaaaaaple !!!!

13. Model
Volume prior
( for c = 0.1 )
This is for c = 0.1
Z =

14. VolPrior for Model: (x1,x2) = (u,v) + (e1,e2) with e1,e2 iid N(0,1) and
u = exp(-50*(t1^2+t2^2) and v=exp(-50( (t1-0.1)^2 + (t2-0.1)^2).
Prior(s,t) = |s-t| exp( -100(s^2+t^2) + 10(s+t))

15. n=100 No learning FLAT VOLUME n=10000 n=1000 n=500 n=100 n=1000
/ /
/ /
VOLUME
/ /
For Volume: the relative errors are 0.8, 1.6 and 0.3, 0.6
BUT for FLAT: the rel. errors are 9.6 and 24 !!!!
n=10000
n=1000

16. Ex: Simple Logistic Regression
Racine’s data 3 5
-0.053
0.727 1
-0.296
-0.863
No. of deaths y
No. of animals n
Dose (log g/ml) x
independent.
log (odds of death) = a + b x
logit(p) = log p/(1-p)
Need: Ignorance Prior on (a,b)

17. Ignorance for Logistic Regression
Racine’s data
MCMC: 250k samples
mean a = sd = 3.7
Mean b = sd = 10.0
corr[a,b] = 0.51
Ignorance for Logistic Regression

19. Volumes of
Bitnets = dags of bits

21. Model Selection Close to the Vacuum
Choose among Complete bitnets:
For given data sequence:

22. Worse < BIC < AIC < CIC < Best
% of correct segmentations v/s N. Based on 100 reps for each N. Params at ramdom each time.

23. .jpg .aiff .txt .gz The Iliad: BOOK I + + 01100…0 + CIC
Sing, O goddess, the anger of Achilles son of Peleus, that brought
countless ills upon the Achaeans. Many a brave soul did it send hurrying
down to Hades, and many a hero did it yield a prey to dogs and vultures,
for so were the counsels of Jove fulfilled from the day on which the son
of Atreus, king of men, and great Achilles, first fell out with one
another………… + + +
01100…0
CIC