1. Pseudorandomness for Approximate Counting and Sampling Ronen Shaltiel University of Haifa Chris Umans Caltech

2. 2 What is this talk about? Main technical result: We define and construct “pseudorandom objects” associated with: Approximate counting of accepting instances of a given circuit. Random sampling of accepting instances of a given circuit. But in fact it all relates to derandomization and it’s a long story: Once upon a time there was an evil magician called Merlin and a handsome prince called Arthur. One day as Arthur was tossing coins he came about a beautiful NP statement…

3. 3 This talk is about derandomization Derandomization of procedures that use both randomness and nondeterminism. –Arthur-Merlin games (by derandomization we mean AM=NP). –Approximate counting of accepting instances. –random sampling of accepting instances Goal: Get rid of randomness (we don’t expect to get rid of nondeterminism). Under what assumptions? We derandomize some randomized procedures using assumptions that seem weaker than those we are “supposed to use”.

4. 4 Approximate counting and sampling of accepting instances Two common computational tasks used frequently in complexity: –approximate counting: given circuit C on n bits output approximation of |C -1 (1)|: –random sampling: given circuit C on n bits output random x in C -1 (1) –Solvable using randomness and nondeterminism [Sto,JVV,BGP]. {0,1} n objects of interest (C recognizes) What do we mean by derandomizing a sampling procedure?

5. 5 Derandomization: Hardness versus Randomness Initiated by [BM,Yao]. Assumption: hard functions exist. Conclusion: Derandomization. A lot of works: [BM82,Y82,HILL,NW88,BFNW93, I95,IW97,IW98,KvM99,STV99,ISW99,MV99, ISW00,SU01,U02,TV02,KI03,GST03]

6. 6 Pseudo-Random Generators A input output pseudo-random bits PRG seed Use a short “ seed ” of very few truly random bits to generate a long string of pseudo-random bits. A input random bits output Pseudo-randomness: no efficient algorithm can distinguish truly random bits from pseudo-random bits. few truly random bits many “ pseudo-random ” bits Nisan-Wigderson setting: The test A can ’ t run PRG. (i.e., for tests that runs in time n 3 the PRG is allowed to run in time n 5 ).

7. 7 Hardness versus Randomness Assumption: hard functions exist. Exists pseudo-random generator Conclusion: Derandomization.

8. 8 The meta-argument Hard function PRG Derandomization Proof takes a distinguishing A and uses it to construct a circuit/algorithm for the supposedly hard function. A Algorithm for function a contradiction The hardness assumption is against procedures at least as complex as A. Meta-Argument: We can’t derandomize the probabilistic version of a complexity class C without a lower bound against C. Assume (for contradiction) that  A that is not fooled by PRG A input output pseudo-random bits PRG seed

9. 9 A brief survey: Achieving the meta argument Meta-Argument: We can’t derandomize the probabilistic version of a complexity class C without a lower bound against C. Actually, we usually require a lower bound against the nonuniform version of C of size 2 Ω(n) [KvM99]. Assumption: There is a function in E=DTIME(2 O(n) ) that cannot be computed for size 2 Ω(n) circuits of a certain type. ClassProb. ClassLower bound for nonuniform size 2 Ω(n) : Ref. PBPPdeterministic circuits.[IW97] NPAM P NP - circuits [KvM99] Nondeterministic circuits[MV99,SU01]

10. 10 Different types of nondeterminsim P NP  coNP NPcoNP P NP || P NP P NP : Poly-time with access to a SAT oracle. P NP || : Poly-time with nonadaptive access to a SAT oracle. …. SAT …. ordinary circuit …. ordinary circuit Adaptive SAT circuit Nonadaptive SAT circuit

11. Our results

12. 12 Beating the Meta-argument ClassProb. classNonunform class PBPPDet. Circuits NPAMNondet circuits P NP ||BPP NP ||Nonadaptive SAT circuits P NP BPP NP Adaptive SAT circuits counting sampling Arthur-Merlin Prvs results: Each can be derandomized using respective hardness. Our results: All can be derandomized using only hardness for non- deterministic circuits. (Same assumption as the one for AM). This results beat the meta-argument! It is known that S 2 P contains P NP. We’ve “derandomized” S 2 P using a lower bound for a weaker circuit class than supposed to! S2PS2P

13. 13 A little bit more formally… Theorem: Assume that there is a problem in E=DTIME(2 O(n) ) that cannot be computed by size 2 Ω(n) (SV-)nondeterministic circuits then: –AM=NP (known result [MV99,SU01], new proof) –Approximate counting and “sampling” can be done in P NP ||. –S 2 P=P NP –BPP path =P NP || –The learning algorithm of Bshouty et al. can be derandomized. –More… Remarks: –E can sometimes be replaced by stronger classes: NE  coNE, E NP ||,E NP.

14. 14 Main technical result Theorem: (boosting hardness): if E requires size 2 Ω(n) nondeterministic circuits then E requires size 2 Ω(n) P NP ||-circuits. Contra-positive: (downward collapse): If E has P NP ||-circuits of size s(n) then E has nondeterministic circuits of size s(n) O(1). (E can be replaced by PSPACE, P #P, E NP, E || NP, NEXP  coNEXP )

15. 15 Quick survey on assumptions implying AM = NP  L worst-case hard for non-det. circuits  L worst-case hard for P NP ||-circuits  L average- case hard for non-det. circuits  L average- case hard for P NP ||-circuits  HSG for co- non-det. circuits  PRG for non-det. circuits  PRG for P NP ||-circuits AM = NP KvM AK MV SU this paper All assumptions are equivalent.

16. 16 Strong PRGs from weak assumptions  L worst-case hard for P NP ||-circuits  HSG for co- non-det. circuits  PRG for (co-) non-det. circuits  PRG for P NP ||-circuits AM = NP KvM “Boosting hardness” MV SU  L worst-case hard for non-det. circuits this paper PRG for stronger circuits than “supposed to”.

17. 17  PRG for adap. P NP -circuits  L worst-case hard for adap. P NP -circuits The current picture of nondeterministic hardness KvM  L worst-case hard for P NP ||-circuits  HSG for co- non-det. circuits  PRG for (co-) non-det. circuits  PRG for P NP ||-circuits AM = NP KvM “Boosting hardness” MV SU  L worst-case hard for non-det. circuits this paper open problem

18. Proof of main result

19. 19 Outline of proof SAT ordinary circuit …. ordinary circuit Note: in general can’t replace small P NP ||- circuit with small nondeterministic circuit (implies, e.g., coNP  NP/poly) Naïve attempt for simulating a SAT query in a nondeterministic circuit: Guess whether the query is answered by “yes” or “no”. If query is answered by “yes”: guess satisfying assignment and verify. If query is answered by “no”: ????????? Assumption:  small P NP ||-circuit C for a complete f in E: (for simplicity assume that it makes only one SAT query). Goal: Show that f has small nondeterministic circuit C’: C We have to use that f is complete for E

20. 20 w.l.o.g. a function in E is a low degree multivariate polynomial Theorem: (low degree extension) [BF] There is a function family f n :F q n  F for q=n O(1) that is complete for E.

21. 21 Simulating C by a randomized nondeterministic circuit C’ On input x: Pass a random low degree curve through x. Field size polynomial => curve has poly many points x 1,..,x q. Suppose we construct a nondeterministic circuit C’ that computes f(x 1 ),..,f(x q ) with at most an  fraction of errors. Then we can compute f(x)! Because f restricted to curve is a low degree univariate polynomial. Use Reed-Solomon decoding. FqnFqn SAT ordinary circuit …. ordinary circuit x low degree f C x1x1 x2x2 x3x3 x4x4 x5x5 xqxq

22. 22 low degree f Using nonuniformity (following [FF91,SU01,BT03,..]) On input x: Pass a random low degree curve through x. Let p = fraction of y’s in F d s.t. the SAT query on y is answered “yes”. Hardwire p to circuit C’. Points on random curve are k-wise independent for k=poly. ∀x with high probability (over curve) the fraction of x i ‘s on curve s.t. the SAT query on x i is answered “yes” is p . FqnFqn SAT ordinary circuit …. ordinary circuit x C x1x1 x2x2 x3x3 x4x4 x5x5 xqxq All points y in F d s.t. the SAT query on y is answered “yes”.

23. 23 Simulating C on all x i ‘s on curve with only few errors. FqnFqn SAT ordinary circuit …. ordinary circuit x f C x1x1 x2x2 x3x3 x4x4 x5x5 xqxq the fraction of x i ‘s on curve s.t. the SAT query on x i is answered “yes” is p . Goal: Simulate C(x 1 ),..,C(x q ) with at most  -fraction of errors. For every x i we simulate C up to the SAT query. Guess fraction of p-  x i ‘s on curve and witnesses showing that all queries of x i ‘s are answered “yes”. Assume queries of other points on curve are answered “no”. <2  errors. By choosing large enough poly degree for curve. There exists a fixed choice of random bits that is good for all x’s.

24. Applications

25. 25 Story so far… ClassProb. classNonunform class PBPPDet. Circuits NPAMNondet circuits P NP ||BPP NP ||Nonadaptive SAT circuits P NP BPP NP Adaptive SAT circuits counting sampling Arthur-Merlin Goal: Derandomize using only hardness for nondeterministic circuits. We’ve seen: can boost hardness: From nondeterministic circuits to nonadaptive SAT circuits. This gives: new proof for AM=NP. “Implies”: derandomizing counting and sampling. What does it mean to derandomize sampling? S2PS2P

26. 26 Sampling accepting instances: given circuit C on n bits. sample random x in C -1 (1) Standard sampling: sample random x in {0,1} n. C -1 (1) A pseudorandom object for sampling accepting instances Sampling accepting instances: given circuit C on n bits. sample random x in C -1 (1) Conditional discrepancy set: given circuit C on n bits. Output x 1,..,x poly(n) in C -1 (1) No circuit of size (say n 2 ) can distinguish a random x i from a random accepting x. Discrepancy set: Output x 1,..,x poly(n) in {0,1} n No circuit of size (say n 2 ) can distinguish a random x i from a random x. {0,1} n x C -1 (1) {0,1} n x

27. 27 More applications ClassProb. classNonunform class PBPPDet. Circuits NPAMNondet circuits P NP ||BPP NP ||Nonadaptive SAT circuits P NP BPP NP Adaptive SAT circuits counting sampling Arthur-Merlin Goal: Derandomize using only hardness for nondeterministic circuits. We’ve seen: new proof for AM=NP. We’ve seen: can boost hardness: From nondeterministic circuits to nonadaptive SAT circuits. “Implies”: derandomizing counting and sampling. Under the same hardness assumption S 2 P=P NP. S2PS2P

28. 28 Derandomizing S 2 P S 2 P  ZPP NP [Cai] Cai’s proof gives that: Every S 2 P language has an algorithm that runs in P NP and uses conditional discrepancy sets. Theorem: if E NP requires exponential size nondeterministic circuits, then S 2 P = P NP.

29. 29 Conclusions conditional discrepancy set generators are “pseudorandom object” for sampling accepting instances. (SV-)nondeterministic hardness assumption sufficient for: –AM = NP (and all assumptions are equivalent) –placing approximate counting in P NP || –placing sampling in P NP || –Placing S 2 P in P NP. Use given assumptions in stronger ways!

30. 30 Open questions strengthen downward collapse to adaptive case? current result: “If E  P NP ||/poly then E  NP/poly” open problem: “If E  P NP /poly then E  NP/poly” uniform version? open problem: “If E  P NP || then E  AM” Our techniques give E  AM/log. Improvement by [KF05], E  NP/log. More examples of beating the meta-argument. Can it be done for weaker classes?

31. That’s it… Thank You!

32. 32 HdHd Tool: low degree extension Every language L  E has a low-degree extension L  E. –extend to f:F q d  F q –f has low total degree (≤ hd) f can be computed in E and is a robust version of f. f:{0,1} n  {0,1} H  F q (e.g. H={0,1}). think of f as f:H d  F q Identify f with low-degree polynomial p:H d  F q FqdFqd

33. 33 A pseudorandom generator for sampling Approximate counting: –given circuit C –output approximation of |C -1 (1)|: –Namely: a number r s.t. |C -1 (1)|(1-  ) ≤ r ≤ |C -1 (1)|(1+  ) Theorem: in P NP || if E requires exponential size (SV-)nondeterministic circuits. {0,1} n objects of interest (C recognizes)

34. 34 Derandomizing Approximate counting Approximate counting: –given circuit C –output approximation of |C -1 (1)|: –Namely: a number r s.t. |C -1 (1)|(1-  ) ≤ r ≤ |C -1 (1)|(1+  ) Theorem: in P NP || if E requires exponential size (SV-)nondeterministic circuits. {0,1} n objects of interest (C recognizes)

35. 35 Approximate counting and sampling –approximate counting: given circuit C output approximation of |C -1 (1)|: |C -1 (1)|(1-  ) ≤ r ≤ |C -1 (1)|(1+  ) Note: PRGs for det circuits give: |C -1 (1)| -  ≤ r ≤ |C -1 (1)| +  Theorem: in P NP || if E requires exponential size (SV-)nondeterministic circuits. {0,1} n objects of interest (C recognizes)

36. 36 Proof sketch Start from weak assumption (hardness for (SV-)nondeterministic circuits). Use boosting theorem to obtain PRG against P NP || circuits. Algorithm for counting works in “ BPP NP ||“. Replace random bits with pseudorandom bits (careful: counting is not a decision problem).

37. 37 –try random hash fn. h into 1, 2, 3, … bits –NP query:  y that has too many preimages? –with high probability when 2 k  |C -1 (1)| no y has too many preimages. Output 2 k. {0,1} n {0,1} 1 … {0,1} k Probabilistic procedure for Approximate counting [S,JVV,BGP]

38. 38 –try hash functions h into 1, 2, 3, … bits that are the outputs of a PRG fooling P NP ||-circuits –NP query: “  y that has too many preimages?” –when 2 k  |C -1 (1)| no y has too many preimages with high probability over all hash functions. –therefore many hash functions that are outputs of the PRG will pass the NP test. Output 2 k. {0,1} 1 … {0,1} k Derandomized procedure for Approximate counting

39. 39 Pseudorandom Sampling Discrepancy set generator: –given s, output T  {0,1} n s.t. for all circuits D of size s: |Pr x [D(x) = 1] - Pr t [D(t) = 1]| ≤  Conditional discrepancy set generator: –given C, s, output T  {0,1} n s.t. for all circuits D of size s: |Pr x [D(x)=1|C(x) = 1] - Pr t  T [D(t)=1|C(t)=1]| ≤  {0,1} n objects of interest (C recognizes)

40. 40 Sampling Conditional discrepancy set generator: –given C, s, output T  {0,1} n s.t. for all circuits D of size s: |Pr x [D(x)=1|C(x)=1] - Pr t  T [D(t)=1|C(t)=1]| ≤  Theorem: in P NP || if E requires exponential size SV-nondeterministic circuits.

41. 41 Proof sketch Start from weak assumption (hardness for SV nondeterministic circuits). Use boosting theorem to obtain PRG against P NP || circuits. Algorithm for sampling works in “BPP NP “. Observe that adaptive NP queries are used mainly to find NP witnesses. (Given NP statement find witness). Replace with non-adaptive witness finding [BCGL90] to get “BPP NP ||”. Replace random bits with pseudorandom bits.

42. 42 Random sampling –pick random y, use NP oracle to enumerate: S y = {x : C(x) = 1 and h(x) = y}(note: |S y | ≤ n 2 ) –pick random i in {1,2,…, n 2 } –output i th item in list, or “fail” if no i th item (requires adaptive queries). {0,1} n {0,1} 1 … {0,1} k 2 k  |C -1 (1)|

43. 43 Pseudorandom Sampling –as before, using nonadaptive NP queries, can obtain hash function h:{0,1} n  {0,1} k such that 2 k  |C -1 (1)| and no y has > n 2 preimages. –idea: use NP oracle to enumerate: S y = {x : C(x) = 1 and h(x) = y} for those y that are the outputs of a PRG fooling P NP ||- circuits –Note: Seems that we require fooling P NP circuits! {0,1} 1 … {0,1} k

44. 44 Sampling –idea: use NP oracle to enumerate: S y = {x : C(x) = 1 and h(x) = y} for those y that are the outputs of a PRG fooling NP||-circuits Two issues: –need to convert small circuit that catches this conditional discrepancy set into small NP||- circuit that catches the PRG. –enumeration step seems to require adaptive use of NP oracle.

45. 45 Non-adaptive witness finding can deal with both issues using non- adaptive NP witness finding –usual technique: given  (x 1, x 2, …, x n ) 2 queries: is  (c 1, x 2, …, x n ) satisfiable for c 1 =0,1 if satisfiable for c 1 =0, then 2 queries: is  (0, c 2, …, x n ) satisfiable for c 2 =0,1 else 2 queries: is  (1, c 2, …, x n ) satisfiable for c 2 =0,1 etc… at most 2n adaptive queries total

46. 46 Non-adaptive witness finding –usual technique if unique satisfying assignment: given  (x 1, x 2, …, x n ) is  (c 1, x 2, …, x n ) satisfiable for c 1 =0,1 is  (x 1, c 2, …, x n ) satisfiable for c 2 =0,1 … is  (x 1, x 2, …, c n ) satisfiable for c n =0,1 assemble into single satisfying assignment 2n non-adaptive queries total

47. 47 Non-adaptive witness finding –Valiant-Vazirani: randomized procedure given  (x 1, x 2, …, x n ), produce  1,  2, …,  n with high probability this is a “good” set: at least one  i has a unique satisfying assignment Key observation (in KvM): there is a small circuit that given  1,  2, …,  n uses non-adaptive NP queries to decide if input is a “good” set the output of a PRG fooling NP||-circuits includes a “good” set use non-adaptive procedure from previous slide in parallel on all formulas in the output of the PRG

48. 48 Putting it all together “pseudorandom object for sampling” Conditional discrepancy set generator: –given C, s, output T  {0,1} n s.t. for all circuits D of size s: |Pr x [D(x)=1|C(x)=1] - Pr t  T [D(t)=1|C(t)=1]| ≤  Theorem: in P NP || if E requires exponential size SV-nondeterministic circuits.

49. 49 Applications S 2 P = those languages L expressible as x  L   y  z R(x, y, z) = 1 x  L   z  y R(x, y, z) = 0 –given x, form matrix: 1 1 1 1 1 1 1 1 1 1 1 0000000000 x  L: cell (y, z) = R(x, y, z) y z x  L:

50. 50 Applications Background BPP  S 2 P known: P NP  S 2 P S 2 P  ZPP NP (Cai) Theorem: if E NP requires exponential size SV- nondeterministic circuits, then S 2 P = P NP. –Proof idea: Cai’s argument can be viewed as non-randomized reduction to sampling. Note: This is the strongest example we have of breaking the barrier. Moral: Make better use of assumptions.

51. 51 Applications BPP path = those languages L with a nondeterministic TM M for which x  L  at least 2/3 of M’s computation paths accept x  L  at least 2/3 of M’s computation paths reject –note: paths need not be same length –[HHT97]: P || NP  BPP path

52. 52 Applications Theorem: if E || NP requires exponential size SV-nondeterministic circuits, then BPP path = P || NP. Proof: –perform approximate counting of computation paths Note: non-adaptive needed to get exact characterization.

53. 53 Conclusions conditional discrepancy set generators are “pseudorandom object” for sampling relative error approximators are “pseudorandom object” for approximate counting SV-nondeterministic hardness assumption sufficient for: –AM = NP (and all assumptions are equivalent) –placing approximate counting in P NP || –placing sampling in P NP || –Placing S 2 P in P NP. Use given assumptions in stronger ways!

54. 54 Open questions strengthen downward collapse to adaptive case? “If E has small adaptive SAT-oracle circuits then E has small SV-nondeterministic circuits.” current result: “If E  P NP ||/poly then E  NP/poly” open problem: “If E  P NP /poly then E  NP/poly” uniform version? open problem: “If E  P NP || then E  AM” Our techniques give E  AM/log. Improvement by [KF05], E  NP/log. More examples of breaking the barrier. Can it be broken for weaker classes?

55. That’s it… Thank You!