WAGOS Conformal Changes of Divergence and Information Geometry Shun-ichi Amari RIKEN Brain Science Institute

Information Geometry Systems TheoryInformation Theory StatisticsNeural Networks Combinatorics Physics Information Sciences Riemannian Manifold Dual Affine Connections Manifold of Probability Distributions Math. AI Vision, Shape optimization

Information Geometry ? Riemannian metric Dual affine connections

Manifold of Probability Distributions

Riemannian Structure Fisher information

Affine Connection covariant derivative straight line

DualityDuality Riemannian geometry: X Y X Y

Dual Affine Connections e-geodesic m-geodesic

Divergence positive-definite Z Y M

Metric and Connections Induced by Divergence (Eguchi) Riemannian metric affine connections

Duality:

Two Types of Divergence Invariant divergence (Chentsov, Csiszar) f-divergence: Fisher- structure Flat divergence (Bregman) KL-divergence belongs to both classes

Invariant divergence (manifold of probability distributions; ) Chentsov Amari -Nagaoka

Csiszar f-divergence Ali-Silvey Morimoto

Invariant geometrical structure alpha-geometry (derived from invariant divergence) - connection : dually coupled Fisher information Levi-civita:

： Dually Flat Structure

Dually flat manifold: Manifold with Convex Function coordinates : convex function negative entropy energy Euclidean mathematical programming, control systems, physics, engineering, economics

Riemannian metric and flatness Bregman divergence : geodesic (notLevi-Civita) Flatness (affine)

Legendre Transformation one-to-one

Two flat coordinate systems : geodesic (e-geodesic) : dual geodesic (m-geodesic) “dually orthogonal”

Geometry Straightness (affine connection)

Pythagorean Theorem (dually flat manifold) Euclidean space: self-dual

Projection Theorem Q = m-geodesic projection of P to M Q’ = e-geodesic projection of P to M

dually flat space convex functions Bregman divergence invariance invariant divergence Flat divergence KL-divergence F-divergence Fisher inf metric Alpha connection : space of probability distributions

Space of positive measures : vectors, matrices, arrays f-divergence α-divergence Bregman divergence

divergence KL-divergence

α-representation (Amari-Nagaoka, Zhang) typical case: u-representation,

Divergence over α-representation

β-divergence (Eguchi)

Tsallis -Entropy-- Shannon entropy Generalized log structure

- exponential family cf Pistone exponential

q-Geometry derived from : dually flat

Dually flat structure of q-escort geodesic: exponential family dual geodesic: q-family

q-escort probability distribution Escort geometry

-escort geometry

Dually flat structure of q-escort geodesic: exponential family dual geodesic: q-family

Projection theorem

Max-entropy theorem

-Cramer Rao theorem

-maximum likelihood estimator

-super-robust estimator (Eguchi)

Conformal change of divergence

- Fisher information conformal transformation

Total Bregman divergence (Vemuri)

Total Bregman Divergence and its Applications to Shape Retrieval Baba C. Vemuri, Meizhu Liu, Shun-ichi Amari, Frank Nielsen IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010

Total Bregman Divergence rotational invariance conformal geometry

TBD examples

Clustering : t-center T-center of E

t-center

t-center is robust

How good is Total Bregman Divergence vision signal processing geometry (conformal)

TBD application-shape retrieval Using MPEG7 database; 70 classes, with 20 shapes each class (Meizhu Liu)

First clustering then retrieval

Advantages Accurate; Easy to access (shape representation); Space and time efficient ( only need to store the closed form t-centers, clustering can be done offline, hierarchical tree storage ).

Shape retrieval framework Shape--> Extract boundary points & align them--> Represent using mixture of Gaussians--> Clustering & use k-tree to store the clustering results; Query on the tree.

MPEG7 database Great intraclass variability, and small interclass dissimilarity.

Shape representation

Experimental results

Other TBD applications Diffusion tensor imaging (DTI) analysis [Vemuri] Interpolation Segmentation Baba C. Vemuri, Meizhu Liu, Shun-ichi Amari and Frank Nielsen, Total Bregman Divergence and its Applications to DTI Analysis, IEEE TMI, to appear