1. Limits of Local Algorithms in Random Graphs
Madhu Sudan
MSR
Joint work with David Gamarnik (MIT)
10/07/2014
Local Algorithms on Random Graphs

2. Local Algorithms on Random Graphs
Preliminaries
Terminology:
Graph 𝐺= 𝑉,𝐸 ; 𝐸⊆𝑉×𝑉 symmetric
𝑉: Vertices; 𝐸: edges;
Independent Set : 𝐼⊆𝑉s.t. 𝑢,𝑣 ⊆𝐼⇒ 𝑢,𝑣 ∉𝐸.
Algorithmic Challenge: Given 𝐺, find large independent set 𝐼.
[Karp’72]: NP-complete in worst-case.
10/07/2014
Local Algorithms on Random Graphs

3. Local Algorithms on Random Graphs
Popularized by Erdös-Renyi:
Basic Model: Every edge thrown in independently with probability 𝑝.
Regular Model: Pick 𝐺 uniformly among all 𝑑-regular graphs:
𝑑-regular: Every vertex in exactly 𝑑 edges.
Background: Almost surely, random 𝑑-regular graph on 𝑛 vertices has independent set of size 1+𝑜 1 ⋅ 𝑐 𝑑 ⋅𝑛 for 𝑐 𝑑 = 2 𝑑 log⁡𝑑.
Question: Find such large independent sets?
10/07/2014
Local Algorithms on Random Graphs

4. Random Graphs & Complexity
Worst-case complexity results no longer apply.
Could hope: Some polynomial time algorithm finds ind. sets of size 1−𝑜 1 ⋅ 𝑐 𝑑 ⋅𝑛
Greedy algorithm:
Order vertices arbitrarily.
Run through vertices in order, include 𝑣 in 𝐼 if this keeps 𝐼 independent.
Fact: Finds set of size ≈ 𝑐 𝑑 ⋅ 𝑛 2
10/07/2014
Local Algorithms on Random Graphs

5. Local Algorithms on Random Graphs
Main Result
Our Theorem: “Local algorithms” can not. In fact they fall short by a constant factor.
Extensions/Subsequent results:
[Rahman-Virag]: Fall short by factor of
Locally-guided decimation algorithms (Belief Propagation, Survey Propagation) fail on some other CSPs.
10/07/2014
Local Algorithms on Random Graphs

6. Definition: Local Algorithms
Informally: Local algorithms
Input = Communication network.
Wish to use local communication to compute some property of input.
In our case – large independent set in graph.
Allowed to use randomness, generated locally.
10/07/2014
Local Algorithms on Random Graphs

7. Local Algorithms on Random Graphs
Formally
(Randomized) Decision Algorithm:
𝑓 𝑢,𝐺, 𝑤 ∈{0,1}: Determines if 𝑢∈𝐼.
𝑤 is a weighting, say in 0,1 , on vertices
Correctness:
∀𝑢,𝑣,𝐺, 𝑤 s.t. 𝑢 ↔ 𝐺 𝑣,
𝑓(𝑢,𝐺,𝑤) = 0 or 𝑓(𝑣,𝐺,𝑤) = 0.
Locality:
𝑓 is 𝑟-local if 𝑓 𝑢,𝐺, 𝑤 =𝑓(𝑣,𝐻, 𝑥 ) whenever 𝑟-local weighted neighborhood around 𝑢 in (𝐺, 𝑤 ) and 𝑣 in (𝐻, 𝑥 ) are identical.
10/07/2014
Local Algorithms on Random Graphs

8. Local Algorithms on Random Graphs
Locality ≠ Locality
Locality in distributed algorithms
Usually algorithms try to compute some function of input graph, on the graph itself.
Algorithm uses data available topologically locally.
Leads to our model
Locality a la Codes/Property Testing
Locality simply refers to number of queries to input.
More general model.
We can’t/don’t deal with it.
10/07/2014
Local Algorithms on Random Graphs

9. Motivations for our work
Paucity of “complexity” results for random graphs. Major exceptions:
Rossman: 𝐴 𝐶 0 /Monotone complexity of planted clique.
Feige-Krauthgamer: LP relaxations.
Physicists explanation of complexity
Clustering/Shattering explain inability of algorithms.
Graph Limit theory
Local characteristics of (random) graphs predict global properties (nearly).
10/07/2014
Local Algorithms on Random Graphs

10. Local Algorithms on Random Graphs
Motivations (contd.)
Specific conjecture [Hatami-Lovasz-Szegedy]: As 𝑟→∞, 𝑟-local algorithms should find independent sets of cardinality 𝑐 𝑑 (1−𝑜(1)) 𝑛.
Refuted by our theorem.
10/07/2014
Local Algorithms on Random Graphs

11. Local Algorithms on Random Graphs
Proof
Part I:
A clustering phenomenon for independent sets in random graphs [Inspired by Coja-Oglan].
Part II:
Locality ⇒ Continuity ⇒¬(Clustering).
Both parts simple.
10/07/2014
Local Algorithms on Random Graphs

12. Local Algorithms on Random Graphs
Clustering Phenomena
Generally:
When you look at “near-optimal” solutions, then they are very structured.
⇒ topology of solutions highly disconnected (in Hamming space.
In our context
Consider graph on independent sets (of size ≈ 𝑐 𝑑 𝑛) with 𝐼↔𝐽 if 𝐼 Δ 𝐽 ≤𝜖⋅𝑛.
Highly disconnected?
10/07/2014
Local Algorithms on Random Graphs

13. Local Algorithms on Random Graphs
Clustering Theorem
Theorem: ∀𝑑, ∃0<𝜃<𝜏< 𝑐 𝑑 s.t.:
Almost surely over 𝐺, ∀𝐼, 𝐽 of size ≈ 𝑐 𝑑 𝑛,
𝐼∩𝐽 𝑛 ∉(𝜃,𝜏)
Proof:
Compute expected number of independent sets with forbidden intersection and note it is ≪1.
Second moment proves concentration.
Implies Clustering.
10/07/2014
Local Algorithms on Random Graphs

14. Locality ⇒¬(Clustering)
Main Idea:
Fix 𝑟-local function 𝑓, that usually produces independent sets of size ≈ 𝑐 𝑑 ⋅𝑛
Sample weights twice: 𝑤 , and then 𝑥 ; 𝑝-correlatedly.
Let 𝐼=𝑓(𝐺, 𝑤 ) and 𝐽=𝑓(𝐺, 𝑥 ).
Prove:
whp, 𝐼 , 𝐽 ≈ 𝑐 𝑑 ⋅𝑛
whp, 𝐼∩𝐽 ≈𝛽 𝑝 ⋅𝑛
∃𝑝 s.t. 𝛽 𝑝 ∈(𝜃,𝜏)
10/07/2014
Local Algorithms on Random Graphs

15. Local Algorithms on Random Graphs
Size of Ind. Set
Claim: Size of independent set produced by local algorithms is concentrated.
Let 𝛼=𝛼 𝑓 = 𝔼 𝑤 [f u, 𝕋 𝑑 , 𝑤 ]
(where 𝕋 𝑑 = infinite tree of degree 𝑑)
W.p. 1-o(1), size of ind. set produced ≈𝛼⋅𝑛.
Proof:
Most neighborhoods are trees ⇒ Expectation.
Most neighborhoods are disjoint ⇒ Chebychev.
10/07/2014
Local Algorithms on Random Graphs

16. 𝑝-correlated distributions
Pick 𝑤 , 𝑦 ∈[0,1]^𝑛, independently.
Let 𝑥 𝑖 = 𝑤 𝑖 w.p. 𝑝 and 𝑦 𝑖 otherwise, independently for each 𝑖.
Let 𝛽 𝑝 = 𝔼 𝑤 , 𝑥 [𝑓 𝑢, 𝕋 𝑑 , 𝑤 ∧ 𝑓 𝑢, 𝕋 𝑑 , 𝑥 ]
As in previous argument:
𝔼 𝐼∩𝐽 ≈𝛽 𝑝 ⋅𝑛
𝐼∩𝐽 concentrated around expectation.
10/07/2014
Local Algorithms on Random Graphs

17. Local Algorithms on Random Graphs
Continuity of 𝛽(𝑝)
Fix 𝑤 , 𝑦 , and consider
Pr⁡[𝑓 𝑢, 𝕋 𝑑 , 𝑤 ∧𝑓 𝑢, 𝕋 𝑑 , 𝑥 ]
Above expression is some polynomial in 𝑝, of degree at most 𝑑 𝑟 .
In particular, it is continuous as function of 𝑝.
⇒𝛽(𝑝)=Expectation over 𝑤 , 𝑦 is also continuous.
Suffices to show 𝛽 0 ,𝛽 1 ∩ 𝜃,𝜏 ≠∅.
10/07/2014
Local Algorithms on Random Graphs

18. Local Algorithms on Random Graphs
Continuity (contd.)
𝛽 𝑝 = 𝔼 𝑤 , 𝑥 [𝑓 𝑢, 𝕋 𝑑 , 𝑤 ∧ 𝑓 𝑢, 𝕋 𝑑 , 𝑥 ]
𝛽 1 =𝛼 𝑓 ≈ 𝑐 𝑑
𝛽 0 = 𝛼 2 ≈ 𝑐 𝑑 2
Follows from calculations (also naturally) that
𝛽 0 ,𝛽 1 ∩ 𝜃,𝜏 ≠∅
Conclude:
whp, 𝐼 , 𝐽 ≈ 𝑐 𝑑 ⋅𝑛
whp, 𝐼∩𝐽 ≈𝛽 𝑝 ⋅𝑛
∃𝑝 s.t. 𝛽 𝑝 ∈(𝜃,𝜏)
10/07/2014
Local Algorithms on Random Graphs

19. Local Algorithms on Random Graphs
Extensions-1
Our notion of clustering:
∀𝐼,𝐽 independent: 𝐼 , 𝐽 ≈𝛼 𝑐 𝑑 𝑛, 𝐼∩𝐽 ∉(𝜃⋅𝑛, 𝜏⋅𝑛)
To get 𝜃<𝜏, need 𝛼 close to 1.
To improve [Ramzan-Virag] suggest:
∀ 𝐼 1 , 𝐼 2 ,…, 𝐼 𝑚 with 𝐼 𝑗 ≈𝛼 𝑐 𝑑 𝑛, ∃𝑖,𝑗 s.t. 𝐼 𝑖 ∩ 𝐼 𝑗 ∉ 𝜃⋅𝑛, 𝜏⋅𝑛
Lets them get to 𝛼→ 1 2
10/07/2014
Local Algorithms on Random Graphs

20. Local Algorithms on Random Graphs
Extensions-2
Local algorithms: Makes all decisions locally, in one shot.
Locally guided decimation algorithms:
Compute some local information.
Make one decision (e.g., 𝑣∈𝐼?) and commit
Repeat.
Recent work: Locally guided decimation algorithms also don’t get close to optimum (on other random CSPs).
10/07/2014
Local Algorithms on Random Graphs

21. Local Algorithms on Random Graphs
Conclusions
“Clustering” is an obstacle?
Answer:
At least to local algorithms.
Local algorithms behave continuously, forcing non-clustering of solutions.
Open questions:
Barrier to local algorithms in general sense?
To other complexity classes?
10/07/2014
Local Algorithms on Random Graphs

22. Local Algorithms on Random Graphs
Thank You
10/07/2014
Local Algorithms on Random Graphs