1 Information Geometry of Self-organizing maximum likelihood Shinto Eguchi ISM, GUAS This talk is based on joint research with Dr Yutaka Kano, Osaka Univ Bernoulli 2000 Conference at Riken on 27 October, 2000

2 Consider a statistical model: -MLE Maximum Likelihood Estimation (MLE)( Fisher, 1922), Take an increasing function. Consistency, efficiency sufficiency, unbiasedness invariance, information

3 -MLE MLE Normal density -MLE given data

outlier MLE -MLE Normal density

5

6 Examples (1 ) (2 ) (3 ) -divergence KL-divergence

7

8 Pythagorian theorem (1,1) (1,0) (0,1) ( t, s ). (0,0) f g h

9 (Pf)

10 Differential geometry of Riemann metric Affine connection Conjugate affine connection Ciszsar’s divergence

11 -divergence Amari’s -divergence

12 -likelihood function M-estimation ( Huber, 1964, 1983) Kullback-Leibler and maximum likelihood

13 Another definition of  -likelihood Take a positive function  (x,  ) and define  -likelihood equation is a weighted score with integrabity.

14 Consistency of  -MLE

15 Influence function Fisher consistency  -contamination model of Asymptotic efficiency Robustness or Efficiency

16 Generalized linear model Regression model Estimating equation

17 Bernoulli regression Logistic regression

18 Misclassification model MLE

19 Group II Group II from Group I = from Logistic Discrimination Mislabel Group I 5 Group II 35 Group I

20 Misclassification Group I 5 data Group II 35 data

21 Poisson regression -likelihood function Canonical link -contamination model

22 Neural network

23 Input Output

24 -maximum likelihood Maximum likelihood

25 Classical procedure for PCA Self-organizing procedure Let off-line data.

26

27 Classic procedure Self-organizing procedure

28 Independent Component Analysis (Minami & Eguchi, 2000) F F

29 S S F Theorem (Semiparametric consistency) (Pf)

30 -likelihood satisfies the semiparametric consistency

31

32 Usual methodself-organizing method Blue dots Blue & red dots

the exponential power 50

34 Concluding remark Bias potential function ? !  -Regression analysis  -Discriminant analysis  -PCA  -ICA  -sufficiency  -factoriziable  -exponential family  -EM algorithm