Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu Vasant Honavar Department of Computer Science Iowa State University

Decomposition and Independence Decomposition renders problems more tractable. Apply recursively Decomposition is enabled by “independence” Decomposition and independence are dual notions AB A B AB AB

Conditional Decomposition and Independence Seldom are the two sub-problems disjoint All is not lost Conditional Decomposition / Independence Conditioning on C C a.k.a. separator CAB ABACCB CAB C =

Formalization of the intuitions Problem P = (D, S, sol P ) D = Domain, S = Solutions sol P : D S AB sol P Example: Determinant_Computation (M 2, R, det)

Conditional Independence / Decomposition Formalization (Variable Based) P = (D = A X B X C, S, sol P ) P1 = (A X C, S1, sol P1 ), P2 = (B X C, S2, sol P2 ) sol P (A, B, C) = sol P1 (A, C) sol P2 (B, C)

Probabilities I(A, B|C) iff P(A, B| C) = P(A|C) P(B|C) Equivalently P(A, B, C) = P(A, C) P(B|C) P(A, B, C) = f 1 (A, C) f 2 (B, C) Independencies can be represented by a graph where we do not draw edges between variables that are independent conditioned on the rest of the variables. ACB

The Hammersley-Clifford Theorem: From Pairwise to Holistic Decomposability

Outline Generalized Conditional Independence with respect to a function f and properties Theorems Conclusions and Discussion

Conditional Independence with respect to a function f - I f (A,B|C) sol P (A, B, C) = sol P1 (A, C) sol P2 (B, C) Assumptions: – S = S1 = S2 [= G] –. – A, B, C is a partition of the set of all variables – Saturated independence statements – from now on f(A, B, C) = f 1 (A, C) f 2 (B, C) I f (A,B|C)

Conditional Independence with respect to a function f I f (A,B|C) – cont’d ABCf ………… = ACf1f ……… BCf2f ……… I f (A,B|C) iff f(A, B, C) = f 1 (A, C) f 2 (B, C)

Examples of I f (A,B|C ) Multiplicative (probabilities) Additive (fitness, energy, value functions) Relational (relations)

Properties of I f (A,B|C ) 1.Trivial Independence I f (A, Φ|C) 2. Symmetry I f (A, B|C) => I f (B, A|C) 3. Weak Union I f (A, B U D|C) => I f (A, B|C U D) 4. Intersection I f (A, B|C U D) & I f (A, D|C U B) => I f (A, B U D|C) AC D B

Abelian Groups (G, +, 0, -) is an Abelian Group iff – + is associative and commutative – 0 is a neutral element – - is an inversion operator Examples: – (R, +, 0, - ) - additive (value func.) – ((0, ∞), ·, 1, -¹)- multiplicative (prob.) – ({0, 1}, mod2, 0, id)- relational (relations)

Outline Generalized Conditional Independence with respect to a function f Properties and Theorems Conclusions and Discussion

Markov Properties [Pearl & Paz ‘87] If Axioms 1-4 then the following are equivalent Pairwise – (α,β) G => I f (α, β|V\{α,β}) Local - I f (α, V\(N(α)U{α})| N(α)) Global – If C=V\{A, B} separates A and B in G I f (A, B| C=V\{A, B}) αβ V\{α,β} N(α) α A B C

Factorization – Main Theorem

The Factorization Theorem: From Pairwise to Holistic Decomposability

Particular Cases - Factorization Probabilistic – Hammersley-Clifford Additive Decomposability Relational Decomposability

Graph Separability and Independence [Geiger & Pearl ‘ 93] If Axioms 1-4 hold then Sep G (A, B|C )  I f (A, B|C) for all saturated independence statements

Completeness Axioms 1-4 provide a complete axiomatic characterization of independence statements for functions which take values over Abelian groups

Outline Generalized Conditional Independence with respect to a function f Properties and Theorems Conclusions and Discussion

Conclusions (1) Introduced a very general notion of Conditional Independence / Decomposability. Developed it into a notion of Conditional Independence relative to a function f which takes values into an Abelian Group I f (.,.|.). We proved that I f (.,.|.) satisfies the following important independence properties: – 1. Trivial independence, – 2. Symmetry, – 3. Weak union – 4. Intersection

Conclusions (2) Axioms 1-4 imply the equivalence of the Global, Local and Pairwise Markov Properties for our notion conditional independence relation I f (.,.|.)) based on the result from [Pearl and Paz '87]. We proved a natural generalization of the Hammersley-Clifford which allows us to factorize the function f over the cliques of an associated Markov Network which reflects the Conditional Independencies of subsets of variables with respect to f. Completeness Theorem, Graph Separability Eq. Theorem The theory developed in this paper subsumes: probability distributions, additive decomposable functions and relations, as particular cases of functions over Abelian Groups.

Discussion: Relation to Graphoids (-) Decomposition (-) Contraction (+) Weak Contraction Graphoids – No finite axiomatic charact. [Studeny ’92] Intersection Discussion – noninvertible elms.

Discussion – cont’d Graph Separability  Independence Completeness Seems that – Trivial Independence – Symmetry – Weak Union – Intersection Strong Axiomatic core for Independence

Applications