1. A 3-Query PCP over integers a.k.a Solving Sparse Linear Systems Prasad Raghavendra Venkatesan Guruswami

2. Linear Equations Given a system of linear equations over reals, Find a solution.. Easy, Use Gaussian elimination.

3. Noise? Given a set of linear equations for which there is a solution satisfying 99% of the equations, What is the best solution that can be efficiently found? Can we atleast satisfy 1% of the equations?

4. 10 years ago [ Håstad STOC97, JACM 01] For any prime p, ε > 0, given a set of linear equations modulo p, it is NP-hard to distinguish between: (1 – ε) – fraction of the equations can be satisfied. 1/p + ε – fraction of the equations can be satisfied. All equations are of the form X i + X j = X k + c (mod p)

5. X 1 + X 2 = X 3 + 10 (mod p) X 1 + X 3 = X 5 + 17 (mod p) X 9 + X 4 = X 3 + 23 (mod p) X 11 + X 2 = X 31 + 1 (mod p) X 1 + X 2 = X 7 - 1 (mod p) X 1 + X 3 = X 8 + p-10 (mod p) …….. X 9 + X 7 = X 3 + p/2 (mod p) X 5 + X 2 = X 7 + 10 (mod p) It is a 3-Query Probabilistically Checkable Proof system for NP Just have to read values of 3 variables to check an equation. Håstad’s 3-Query PCP [STOC97, JACM 01] Can be verified by 3 queries Reals?

6. NP-hard [Guruswami-Raghavendra 06, Feldman-Gopalan-Khot-Ponnuswami 06] For any ε,δ > 0, Given a set of linear equations over reals, it is NP-hard to distinguish between the following two cases: There is a solution that satisfies 1 – ε fraction of the equations. No solution satisfies more than δ fraction of the equations. Unlike Hastad’s result, equations are not sparse

7. Sparse Equations? Solving sparse systems of equations important for many applications. In the spirit of PCP theorem.. Sparse equations have important connections to PCPs, linearity testing, Unique Games conjecture.

8. Sparse Equations over Reals For any ε,δ > 0, Given a set of sparse linear equations, it is NP-hard to distinguish between: (1 – ε) – fraction of the equations can be satisfied. δ – fraction of the equations can be satisfied. X 1 + X 2 = X 3 + 10 X 1 + X 3 = X 5 + 17 … X 9 + X 4 = X 3 + 23 X 2 + X 6 = X 7 + 27 Some fixed constant accuracy, say ±1

9. Label Cover Problem U, V : set of vertices E : set of edges {1,2… R} : set of labels π e : constraint on edge e An assignment A satisfies an edge e = (u,v) E if π e (A(u)) = A(v) 123..R123..R 123..R123..R πeπe U V u v Find an assignment A that satisfies maximum number of edges 3 π e (3)=2 7 5 2 3 3 1 4 1 2 5 6

10. Label Cover with Long Codes πeπe 7 5 2 3 3 1 4 1 2 5 6 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 Write Long Codes of the Labels instead of the labels itself

11. Long Code A long code over a finite field F is a function: G i : F X F … X F XF F G i (x 1, x 2, … x n ) = x i n different long codes. Long code over F p represented by a table of p n values. Linear Function.

12. Extending Hastad’s result to integers πeπe 7 5 2 3 3 1 4 1 2 5 6 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 X1X1 X2X2 G 2 (x 1, x 2 ) = x 2 A Long Code over integers is an infinite object. Use long code over integers Just Truncate the long code!

13. Core Problem 4 4 4 4 4 3 3 3 1 3 2 2 1 2 2 1 1 1 1 1 0 0 0 0 0 X1X1 X2X2 0 1 2 3 4 X1X1 X2X2 G 2 (x 1, x 2 ) = x 2 G 1 (x 1, x 2 ) = x 1 ?=?= Given two supposed long codes, query 3 locations and test if they are close to some long code If test succeeds, must decode a small set of possible labels

14. Proof Obstacles Linearity Testing Decoding Labels

15. Linearity Testing Given a function from an group G 1 to group G 2 (both abelian) A : G 1 -> G 2 Pick x,y uniformly at random from G 1 Test if A(x) + A(y) = A(x+y) [Blum-Luby-Rubinfeld] With G 1 = {0,1} n,G 2 = {0,1}, if A is δ- far from linear function, then the test rejects with probability at least δ

16. Derandomized Linearity Testing 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 For sufficiently large primes p, Linearity testing on truncated long code = Testing modulo p This should imply a derandomized linearity test. Total randomness used independent of the prime. p

17. Proof Obstacles Linearity Testing Decoding Labels

18. Fourier Analysis One Fourier coefficient corresponding to every linear function P ω (x) = ωx for ω =(ω 1, ω 2,… ω R ) in F p R Â( ω) measures similarity with P ω (x) = ωx Â( ω) = E[ A(x)e -iωx ]

19. Hastad’s Decoding Pick a large Fourier coefficient Â( ω) of the long code, randomly pick a nonzero coordinate ω i Decode to label i Not too many large Fourier Coefficients Parseval’s Identity

20. Obstacle The distribution is not uniform, so a Fourier coefficient that appears is There could be exponentially many large Fourier coefficients!

21. P(x) A(x) P(x)A(x) Large values in Fourier spectrum are clustered. Function Fourier Transform

22. Decoding Labels Pick a large Fourier coefficient A P (ω), randomly pick one of its large coordinate ω i Assign label i to the vertex All large Fourier coefficients in the same cluster, will yield the same label with high probability. There are very few clusters, so there are very few possible choices

23. Conclusion Sparse linear equations over real numbers are hard to solve even with little noise. In a weak sense, complete Derandomization of linearity testing is possible. Two variable linear equations over reals?

24. Thank You

26. 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 π A B Randomly pick a vector x = (x 1,x 2,.. x R ) Define x o π = (x π(1), x π(2), x π(3) ….x π(R) ) Test if a(x o π) = b(x) Testing an Edge I will just assign 0 to everything!

27. Testing an Edge Randomly pick a vector x = (x 1,x 2,.. x R ) Define x o π = (x π(1), x π(2), x π(3) ….x π(R) ) A(x o π) = B(x) 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 π A B For a long code a, a(x + 1 ) = a(x) + 1 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 X1X1 X2X2 G 2 (x 1, x 2 ) = x 2 A(x o π – t 1 1) + t 1 = B(x – t 2 1) + t 2 I will give something that does not look linear at all

28. Testing an Edge Randomly pick a vector x = (x 1,x 2,.. x R ) Define x o π = (x π(1), x π(2), x π(3) ….x π(R) ) Randomly pick y = (y 1,y 2,.. y R ) Test if a(x o π + y) – a(y) = b(x) + c Long Code is a linear function! a( x o π + y ) – a(y) = a(x o π) 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 π A B A(x) = (x 1 + x 2 +...x R )/R

29. (ε,δ) – concentrated distribution All Fourier coefficients of P that are 2πδ away from origin are bounded by ε 4πδ ε

30. Examples Epsilon Biased Spaces over [0,1] n are (ε, ½) – concentrated. Epsilon Biased Spaces over F p are (ε, 1/p) – concentrated. [BenSasson-Sudan-Vadhan-Widgerson] use Epsilon biased spaces to derandomize low degree tests(including linearity) Any sufficiently slowly decaying probability distribution over integers.

31. Hardness of Label Cover There exists γ > 0 such that Given a label cover instance Г =(U,V,E,R,π), it is NP-hard to distinguish between : Г is completely satisfiable No assignment satisfies more than 1/R γ fraction of the edges. [Raz 98]

32. Testing an Edge For a function π : [1,2,.. R] -> [1,2..R] A vector x = (x 1,x 2,.. x R ) Define x o π = (x π(1), x π(2), x π(3) ….x π(R) ) a(x o π) = b(x) 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 πeπe a b 1212 1212 πeπe 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 X1X1 X2X2 A (x 1, x 2 ) = x 2 0 1 2 3 4 X1X1 X2X2 B (x 1, x 2 ) = x 1 a(2, 4) = b(4,2) A Linear Equation on Long code symbols

33. Hastad’s 3 Query PCP Randomly pick a vector x Define x o π = (x π(1), x π(2), x π(3) ….x π(R) ) Randomly pick y Perturb each coordinate of x o π + y independently with probability ε. To perturb just change the value to anything else in F p Test if a(x o π + y+μ) – a(y) = b(x) + c Long codes/Dictator functions are stable against noise in the coordinates

34. Arithmetization Define A(x) = ω a(x) = e 2πia(x)/p B(y) = ω b(y) = e 2πib(y)/p Then : a(x o π + y+μ) – a(y) - b(x) = 0 if and only if 1/p ∑ (A(y)B(x) A(x o π + y+μ) ) j = 1

35. Soundness Argument As Linearity is tested, a(x) must have some similarity to a linear function. There have to be large Fourier coefficients Â( ω) As we force a(x + 1) = a(x) + 1 the function a(x) is not similar to constant function. Thus, there are some nonzero ω with large Â( ω) There are large Â( ω) with ω having few non-zero labels.

36. Obstacles 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 X1X1 X2X2 G 2 (x 1, x 2 ) = x 2 Truncated region no more a group. For a constant fraction of x and y, (x+y) is outside the region. There could be exponentially many (in the dimension of space) large Fourier coefficients.

37. Modified Test P, P’ be decaying and (ε,δ)- concentrated distributions Pick x from distribution P Pick y from distribution P’ Perturb each coordinate of x o π + y independently with probability ε. To perturb, just change the value by a random number < M Test if A(x o π + y+μ) – A(y) = B(x) + c P’ P P much more flatter than P’

38. Fourier Analysis (continued) Inverse Fourier Transform Not too many large Fourier Coefficients Parseval’s Identity

39. Time Domain Fourier Domain P(x) = 1

40. Properties For a long code a, a(x + 1 ) = a(x) + 1 Long codes/Dictator functions are stable against noise in the coordinates. 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 1 1 1 1 1 0 0 0 0 0 X1X1 X2X2 0 1 2 3 4 X1X1 X2X2 G 2 (x 1, x 2 ) = x 2 G 1 (x 1, x 2 ) = x 1 Two dimensional long codes over F 5