Zig-Zag Expanders Seminar in Theory and Algorithmic Research Sashka Davis UCSD, April 2005 “ Entropy Waves, the Zig-Zag Graph Product, and New Constant- Degree Expanders” O. Reingold, S. Vadhan, A. Wigderson

Talk Outline Introduction: notations, definitions, facts. Zig-Zag graph product: 1. Overview 2. Construction 3. Analysis – Intuition 4. Analysis

Expansion, Expanders For undirected graph G=(V,E) Vertex expansion parameter is defined as: ε = min |Γ(S)\S| / |S|. Vertex expansion parameter is defined as: ε = min |Γ(S)\S| / |S|. S | |S| ≤|V|/2 G is a good expander if for any S, s.t. G is a good expander if for any S, s.t. |S| ≤|V|/2, then |Γ(S)|≥(1+ε) |S|.

Family of Expander Graphs A family of expander graphs {G i } is a collection of graphs such that for all i: G i is d-regular. G i is d-regular. |V(G i )| is strictly increasing. |V(G i )| is strictly increasing. ε ≥c, for some constant c. ε ≥c, for some constant c.

Undirected D-regular Graphs Notation: Let G be undirected D-regular, then: the adjacency matrix is A(G). the adjacency matrix is A(G). the normalized adjacency matrix is M= 1/D A(G). the normalized adjacency matrix is M= 1/D A(G). Spectrum σ(A)={λ 0,λ 1,…,λ n-1 }. Spectrum σ(A)={λ 0,λ 1,…,λ n-1 }. λ(G)= λ 1. λ(G)= λ Each row/column adds up to D 2. A(G) is (real) symmetric, therefore A(G) is similar to a diagonal matrix. A(G) is similar to a diagonal matrix. σ(A)={λ 0,λ 1,…,λ n-1 } are real. σ(A)={λ 0,λ 1,…,λ n-1 } are real. Rⁿ has an orthonormal basis consisting of eigenvectors of A(G). Rⁿ has an orthonormal basis consisting of eigenvectors of A(G). (D,1 n ) is an eigenpair. (D,1 n ) is an eigenpair.

Expansion, Convergence, and λ(G) G is a good expander then λ(G) is small G is a good expander then λ(G) is small Cheeger & Buser: (d-λ 1 )/2D ≤ e ≤ 2√(d-λ 1 )/D Random walk on G converges to the uniform distribution rapidly if λ(G) is small. Random walk on G converges to the uniform distribution rapidly if λ(G) is small. Proof: (on board) Proof: (on board) We use Rayleligh-Ritz Theorem λ(G) = max / = max ||Mx||/||x|| x perp. to uniform x perp. to uniform

Talk Outline 1. Introduction: notations, definitions, facts. Zig-Zag Graph product: 1. Overview 2. Construction 3. Analysis – Intuition 4. Analysis

Zig-Zag Graph Product Delivers a constant degree family of expanders. Delivers a constant degree family of expanders. Construction is iterative. Construction is iterative. The analysis is algebraic. The analysis is algebraic. Notation: G is (N,D,μ)-graph meaning V(G)=N, G is D-regular and has λ(G) at most μ. Notation: G is (N,D,μ)-graph meaning V(G)=N, G is D-regular and has λ(G) at most μ.

Standard Operations Squaring G: new edge are paths in G of length 2 Squaring G: new edge are paths in G of length 2 (N,D,λ) 2 = (N,D 2,λ 2 ) Tensoring G (Kronecker product) Tensoring G (Kronecker product) (N,D,λ)  (N,D,λ) = (N 2,D 2,λ)

Expander Construction Using the Zig-Zag Graph product Start with a constant-size expander H. Start with a constant-size expander H. Apply simple operations to H to construct arbitrarily large expanders. Apply simple operations to H to construct arbitrarily large expanders. Main Challenge: prevent the degree from growing. Main Challenge: prevent the degree from growing. New Graph Product: compose large graph w/ small graph to obtain a new graph which (roughly) inherits New Graph Product: compose large graph w/ small graph to obtain a new graph which (roughly) inherits Size of large graph Size of large graph Degree of small graph Degree of small graph Expansion from both Expansion from both

The Zig-Zag Graph Product: Theorem 1 Let G 1 be (N,D,λ 1 )-graph and G 2 be (D,d,λ 2 )-graph, then ( G 1 G 2 ) = (ND, d 2, λ 1 + λ 2 + λ 2 2 ) Proof: Later. (Big portion of remaining 23 slides...) z

Talk Outline 1. Introduction: notations, definitions, facts. Zig-Zag graph product: 1. Overview 2. Construction 3. Analysis – Intuition 4. Analysis (all the gory details..)

The Construction Building block: Let H be (D 4,D,1/5)-graph Building block: Let H be (D 4,D,1/5)-graph Construct a family {G i } of D 2 -regular graphs such that Construct a family {G i } of D 2 -regular graphs such that G 1 =H 2 G 1 =H 2 G i+1 = (G i ) 2 H G i+1 = (G i ) 2 H Theorem 2 For every i, G i is (D 4i, D 2, 2/5)- graph. Proof: By induction (on the board). z

Zig-Zag Graph Product – Construction (by example) Vertices in V(G 1 G 2 ) = V(G 1 )  V(G 2 ) u G1G1 z G2G v

Zig-Zag Graph Product – Construction (by example) Vertices in G 1 G 2 = G 1  G 2 z (v,1) (u,1) (u,3) (u,2) (v,2) (v,3)

Edge of G 1 G 2 = V  E 2  E 2 (v,1) (u,1) (u,3) (u,2) (v,2) (v,3) Consider ((u,1),0,0) - edge(0,0) incident to vertex (u,1). z 2

Edges of G 1 G 2 z (v,1) (u,1) (u,3) (u,2) (v,2) (v,3) Vertex (u,1) and all its neighbors (w,1) 1

Edges of G 1 G 2 = V  E 2  E 2 z (v,1) (u,1) (u,3) (u,2) (v,2) (v,3) Connect ( u,i ) and ( v, j ) iff  i, j such that 1. i and i connected in G 2 2. ( u, i ) and ( v, j ) correspond to same edge of G 1 3. j and j connected in G 2

G = G 1 G 2 G 1 is (N,D,λ 1 )-graph and G 2 is (D,d,λ 2 )-graph |V(G)| = |V( G 1 )||V( G 2 )| = ND Degree of G = deg( G 2 ) 2 =d 2 Edge set of G: a step in G 2 a step in G 2 a step in G 1 a step in G 1 a step in G 2 a step in G 2 λ (G) ≤ λ 1 + λ 2 +λ 2 2 z

ANALYSIS of the ZIG-ZAG Graph Product Intuition

The Eigenvalue Bound Need to show: Random step on G 1 G 2 makes non-uniform probability distributions  closer to uniform. Need to show: Random step on G 1 G 2 makes non-uniform probability distributions  closer to uniform. Random step on G 1 G 2  Random step on G 1 G 2  1. random step within “cloud”. 1. random step within “cloud”. 2. jump between clouds. 2. jump between clouds. 3. random step within new cloud. 3. random step within new cloud. z z

Analysis, Intuition (cont.) A,C – normalized adjacency matrices of G 1,G 2 M – normalized adjacency matrix of G Must show: G1 G2-matrix M shrinks every vector α  ND that is perp. to uniform (Rayeigh- Ritz Thm, for 2-nd eigenvalue). Must show: G1 G2-matrix M shrinks every vector α  ND that is perp. to uniform (Rayeigh- Ritz Thm, for 2-nd eigenvalue). Decompose α=α || + α , where α || is probability distribution, where distribution within clouds is uniform, and α  is a distribution, where probabilities within cloud are far from uniform. Decompose α=α || + α , where α || is probability distribution, where distribution within clouds is uniform, and α  is a distribution, where probabilities within cloud are far from uniform. z

Analysis, Intuition (cont.) Write α  ND as α=α 11, …,α 1D,…, α N1, …,α ND Write α  ND as α=α 11, …,α 1D,…, α N1, …,α ND For i  [N], define: For i  [N], define: (α) i  D, (α) i = α 11, …,α 1D (α) i  D, (α) i = α 11, …,α 1D β i = ∑ j=1,D α ij “ distribution” on clouds themselves β i = ∑ j=1,D α ij “ distribution” on clouds themselves (α) i  = (α) i -(α) i || (α) i  = (α) i -(α) i || (α) i || = 1/ β j 1 D (α) i || = 1/ β j 1 D Evaluate ||Mα||=||CACα|| (not exactly, just intuition) Evaluate ||Mα||=||CACα|| (not exactly, just intuition)

Case I: Non-uniform Distribution (far from uniform –Case I: α very non-uniform (far from uniform) within “clouds” Step 1 makes α more uniform (by expansion of G 2 ). Steps 2 & 3 cannot make α less uniform.

Case I: Non-uniform Distribution, cont. Case I ( α  ): “distribution” within clouds perpendicular to (far from) uniform. Case I ( α  ): “distribution” within clouds perpendicular to (far from) uniform. Decompose M as product M=CAC, corresponding to three steps in definition of G 1 G 2 ’s edges. C applies G 2 -matrix to each row of α . Decompose M as product M=CAC, corresponding to three steps in definition of G 1 G 2 ’s edges. C applies G 2 -matrix to each row of α .  shrinks α  by ( G 2 ). Applying CA cannot increase length. Applying CA cannot increase length. z

Case II: Uniform Distribution, cont. Case II ( α || ): distribution within clouds is uniform. Cα || = α || Cα || = α || Applying A  applying G 1 -matrix to “distribution” on clouds themselves. Applying A  applying G 1 -matrix to “distribution” on clouds themselves. α  ND, α=(α 11, …,α 1D,…, α N1, …,α ND ) α  ND, α=(α 11, …,α 1D,…, α N1, …,α ND ) (α) i  D, (α) i = α 11, …,α 1D (α) i  D, (α) i = α 11, …,α 1D β i = ∑ j=1,D α ij “ distribution” on clouds themselves” β i = ∑ j=1,D α ij “ distribution” on clouds themselves” β  N as ( β 1, …, β N ) β  N as ( β 1, …, β N ) ||A β|| ≤ ( G 1 )|| β|| ||A β|| ≤ ( G 1 )|| β||

Case II: Uniform Distribution Case II: α uniform within clouds. Step 1: does not change α. Step 1: does not change α. Step 2: Jump between clouds  random step on G 1 Step 2: Jump between clouds  random step on G 1  Distribution on clouds themselves becomes more uniform (by expansion of G 1 )

Talk Outline 1. Introduction: notations, definitions, facts. Zig-Zag graph product: 1. Overview 2. Construction 3. Analysis – Intuition 4. Analysis (all the gory details..)

Analysis of λ(G) To show that λ(G) ≤ (λ(G 1 ) + λ(G 2 )+λ(G 2 ) 2 ) suffices to prove that to show that for any α  ND, perpendicular to 1 ND ≤ (λ(G 1 ) + λ(G 2 )+λ(G 2 ) 2 ) ≤ (λ(G 1 ) + λ(G 2 )+λ(G 2 ) 2 ) ≤ (λ 1 +λ 2 +λ 2 2 ) ≤ (λ 1 +λ 2 +λ 2 2 )

Normalized Adj. Matrix of the Product A,C – normalized adjacency matrices of G 1,G 2 M – normalized adjacency matrix of G 1 G 2 M=ĈÂĈ, where M=ĈÂĈ, where Ĉ = I N  C Ĉ = I N  C  is a permutation matrix (length preserving), where element (u,v) goes to the v-th neighbor of v in G 1.  is a permutation matrix (length preserving), where element (u,v) goes to the v-th neighbor of v in G 1. We relate  to A next: We relate  to A next: z

 is.. Given any α  ND, α=α 11, …,α 1D,…, α N1, …,α ND For i  [N], define: For i  [N], define: (α) i  D, (α) i = α 11, …,α 1D distribution within the cloud. (α) i  D, (α) i = α 11, …,α 1D distribution within the cloud. β i =∑ j=1,D α ij “distribution” on clouds themselves. β i =∑ j=1,D α ij “distribution” on clouds themselves. (α) i || = (β i /D) 1 D (α) i || = (β i /D) 1 D (α) i  = (α) i -(α) i || (α) i  = (α) i -(α) i || L:  ND →  N, L(α) = (β 1,…, β N )= β  N L:  ND →  N, L(α) = (β 1,…, β N )= β  N LÂ( β  1 D ))= A β LÂ( β  1 D ))= A β Â( β  1 D ) = A β  1 D Â( β  1 D ) = A β  1 D

Proof (cont.) 1. = = α T ĈÂĈ α = (Ĉα) T Â(Ĉ α) = (Ĉα) T Â(Ĉ α) = 2. α = α || + α  3. Ĉα || = α || 4. Ĉα = Ĉ(α  +α || ) = α || + Ĉα 

Proof (cont.) Ĉα = Ĉ(α  + α || ) = α || + Ĉα  =| |= =| |= = = ≤ + ||Âα || ||||Ĉα  || + ||ÂĈα  |||| α || || + ≤ + ||Âα || ||||Ĉα  || + ||ÂĈα  |||| α || || + = + 2||α || ||.||Ĉα  || + ||Ĉα  || 2

Proof (cont.) ≤ | | + 2||α || ||.||Ĉα  || + ||Ĉα  || 2 ≤ | | + 2||α || ||.||Ĉα  || + ||Ĉα  || 2 Claim1: ||Ĉα  || ≤ λ(G 2 )||α  || = λ 2 ||α  || Claim2: ≤ λ 1 =λ 1 ||α|| 2 α || = β  U D Âα || = Â( β  U D ) = A β  U D = λ 1 β  u D By expansion of G 2 - A β ≤ λ 1 β ≤ λ 1 = λ 1 ||α|| 2 ≤ λ 1 = λ 1 ||α|| 2

Proof (cont.) ≤ | | + 2||α || ||.||Ĉα  || + ||Ĉα  || 2 ≤ | | + 2||α || ||.||Ĉα  || + ||Ĉα  || 2 Claim1: ||Ĉα  || ≤ λ(G 2 )||α  || = λ 2 ||α  || Claim2: ≤ λ 1 =λ 1 ||α|| 2 α || = β  U D Âα || = Â( β  U D ) = A β  U D By expansion of G 2 - A β ≤ λ 1 β = = = = ≤ λ 1 = λ 1 ||α|| 2 ≤ λ 1 = λ 1 ||α|| 2

Proof. = ≤ λ 1 ||α || || 2 +2λ 2 ||α || ||.||α  ||+ λ 2 2.||α  || 2 = ≤ λ 1 ||α || || 2 +2λ 2 ||α || ||.||α  ||+ λ 2 2.||α  || 2 ||α|| 2 =||α  + α || || 2 =||α  || 2 +||α || || 2 / = / ||α|| 2 / = / ||α|| 2 =λ 1 ||α || || 2 /||α|| 2 + 2λ 2 ||α || ||.||α  || /||α|| 2 + λ 2 2 ||α  || 2 /||α|| 2 / ≤ λ 1 +λ 2 + λ 2 2 / ≤ λ 1 +λ 2 + λ 2 2Q.E.D.

The Zig-Zag Graph Product: Theorem 1 Let G 1 be (N,D,λ 1 )-graph and G 2 be (D,d,λ 2 )-graph, then ( G 1 G 2 ) = (ND, d 2, λ 1 + λ 2 + λ 2 2 ). Theorem 2 For every i, G i is (D 4i, D 2, 2/5)- graph. z