1. Integer Polytope Sampling and Network Tomography Bertrand Haas Edoardo Airoldi Harvard University 06/21/2010 1Whistler Workshop June 2010

2. Plan Motivation: Network Tomography + 2 more applications The Solution Polytope Finding a first vertex of the polytope Finding all the vertices of the polytope Integer Polytope Sampler 1, 2 and 3 and comparisons From sample point to integer sample point Efficiency 2Whistler Workshop June 2010

3. Network Tomography Computer-Router networks Routes between pairs of end- nodes predetermined Routers record traffic Problem: Infer traffic between pairs of end-nodes Network matrix A (incidence) – Column pair of end-nodes – Row edge – A[i,j] = 1 if edge i incident to route of pair j. A[i,j] = 0 otherwise Network Matrix of full rank -> eliminate some rows (flow conservation at each router) No loop -> number of solutions is finite 3Whistler Workshop June 2010

4. 1-Router Networks: Special Interest m x n network matrix A (here m = 7, n = 16) Problem: Given link count b, solve A x = b Underdetermined problem -> sample integer non- negative solutions with a distribution (uniform,…) R-solution space = Flat (dim. n-m) ∧ non-neg. orthant # integer solutions finite -> convex polytope 4Whistler Workshop June 2010

5. 2 more applications Sampling from contingency tables reduces to Network Tomography on a 1-router network. Sampling (directed) networks with given degree vector reduces to a {0,1}-contingency table problem – Intersection of a hypercube with a Flat (TUM) 5Whistler Workshop June 2010

6. Total UniModularity Definition: Matrix is TUM all its sub- determinants are -1, 0 or 1 Theorem (): Network matrices are TUM Theorem (): Polytopes arising from a TUM matrices are integral (vertices are integral). New Problem: 1.Find vertices of the solution polytope. 2.Sample integer points of the polytope. 6Whistler Workshop June 2010

7. Hermite Normal Decomposition Hermite Normal Form of integer matrix A: B = A Q – Where B is integral, lower triangular, B[i,j] ≤ B[i,k], k < i – Q is unimodular A is TUM -> A = [A 1 |A 2 ], where A 1 unimodular Columns of Q 2 generates null-space 7Whistler Workshop June 2010

8. On previous 1-Router Network A = Q = (with columns reshuffled to a nicer form) Q1Q1 Q2Q2 8Whistler Workshop June 2010

9. The First Vertex of the Solution Polytope Solving A x = b A Q (Q -1 x) = b B x’ = b x’ = (b,w) T where x’ = Q -1 x x = Q x’ x = (b’|0) T + Q 2 w where b’ = A 1 -1 b Vertices of solution polytope are intersections of the solution-flat (dim. n-m) with m-coordinate planes. -> Vertices have at most m non-zero coordinates So if x (1) = Q(b,0) T = Q 1 b ≥ 0 then x (1) = v 0 is a first vertex of the solution polytope. 9Whistler Workshop June 2010

10. From the 1 st Vertex to Neighbor Vertices Lemma: each column of Q 2 has at least one -1 Let K j = {k:Q 2 [k,j] = -1} and i = argmin b’ k over all k in K j Then v j = v 0 + b’ i Q 2 [:,j], for all j = 1,…,n-m At most m non-negative coordinates 10Whistler Workshop June 2010

11. From an integer solution to a non-neg. one By similar operation (“pivoting”) we move from one one integer solution to a less negative one. Permute 11Whistler Workshop June 2010

12. The Geometry of Pivoting Example: n=6, m=2 -> hyperplane of coordinates become lines in the flat. Orthants are polyhedra. 1 st solut. Q 1 b = x (1) -> x (2) -> x (3) -> x (4) = v 0 1 st vertex 12Whistler Workshop June 2010

13. Polytope Sampler 1 Associate a coefficient α i to each of the r vertices of the polytope (of dimension n). Decompose the polytope into simplexes: P = S 1 ∨ S 2 ∨ … ∨ S r-n Associate weights p i to each S i (with Σp i =1) for example p i = Vol(S i )/Vol(P) Draw a S i with probability p i Let α 0, α 1,…, α n be the coefficients associated to S i ’s vertices v 0, v 1,…,v n and let V = [v 1 -v 0,…, v n -v 0 ] Draw a vector Y ~ Dir(α 0, α 1, …, α n ) Scale and translate: X = v 0 + V Y 13Whistler Workshop June 2010

14. Polytope Sampler 2 Theorem (): A polytope P of dim. n with r vertices is the projection of a simplex S of dim. r-1 Associate coefficients α 0, α 1,…, α r to P’s vertices Lift P to a simplex S of dim. r-1. Draw a vector Y ~ Dir(α 0, α 1,…, α r ) Project onto P: X = proj P (Y) 14Whistler Workshop June 2010

15. Polytope Sampler 3 Let d i = Σv i,j, d 0 = max(d i ), and v i,0 = d 0 -d i So we lift P to P’ in the plane Σx i = d 0 in R n+1 Define the monomials m i = x 0 ^(v i,0 ) x 1 ^(v i,1 ) … x n ^(v i,n ) so deg(m i ) = d 0 for all i Here: m 0 = x 0, m 1 = x 1, m 2 = x 2 Associate α i to each vertex v i For example all α i = 1 Set Φ(x) = Σ α i m i v i / Σ α i m i Theorem: Φ isom.: (R >0 ) n  int(P’) Here Φ(x) = (x 0,x 1,x 2 ) T /Σx i Draw X ~ dist(R >0 ) n & set Y = Φ(X) for example X i ~ Gamma(a i ) We get Y ~ Dir(a 1,a 2,a 3 ) Example: P = triangle 15Whistler Workshop June 2010

16. Polytope Samplers Comparisons Sampler 1 (Simplicial Decomposition): (+) Perfect uniform distribution (+) Possible finer decomposition (interior points) (-) Simplicial decomposition not straightforward and not unique Sampler 2 (Projection of a Dirichlet dist.) (+) Easy (-) Lifting to simplex not unique (-) only approximate uniform distribution Sampler 3 (map Φ: (R >0 ) n  int(P’)) (+) Direct method; Generalization of Dirichlet (-) Computations might be costly (with high degree) (-) Distribution difficult to control (Jacobian not simple) 16Whistler Workshop June 2010

17. From a sampled point to an integer point Rounding off the coordinates possibly gets out of the solution flat. Project onto the n-m coordinate plane x 1 = x 2 =…= x m = 0, round off, project back to solution flat. Still possible errors and bias on the boundary. 17Whistler Workshop June 2010

18. Efficiency Compare with stat-CS methods – EM algorithm (J. Cao at al.), Metropolis Hasting- Gibbs (C. Tabaldi, M. West) On examples where Solution Polytope = Simplex: – At least one order of magnitude faster – Much more accurate (no error recorded) – Scalability? 18Whistler Workshop June 2010