May 5, 2010 MSRI 1 The Method of Multiplicities Madhu Sudan Microsoft New England/MIT TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A Based on joint works with: V. Guruswami ‘98 S. Saraf ‘08 Z. Dvir, S. Kopparty, S. Saraf ‘09

of 26 Kakeya Sets K ½ F n is a Kakeya set if it has a line in every direction. K ½ F n is a Kakeya set if it has a line in every direction. I.e., 8 y 2 F n 9 x 2 F n s.t. {x + t.y | t 2 F} ½ K I.e., 8 y 2 F n 9 x 2 F n s.t. {x + t.y | t 2 F} ½ K F is a field (could be Reals, Rationals, Finite). F is a field (could be Reals, Rationals, Finite). Our Interest: Our Interest: F = F q (finite field of cardinality q). F = F q (finite field of cardinality q). Lower bounds. Lower bounds. Simple/Obvious: q n/2 · K · q n Simple/Obvious: q n/2 · K · q n Do better? Mostly open till [Dvir 2008]. Do better? Mostly open till [Dvir 2008]. May 5, 2010 MSRI 2

of 26 Random Uniform Randomness in Computation May 5, 2010 MSRI 3 Alg X F(X) Randomness Processors Distribution A Distribution B Support industry: Readily available randomness Prgs, (seeded) extractors, limited independence generators, epsilon-biased generators, Condensers, mergers,

of 26 Randomness Extractors and Mergers Extractors: Physical randomness (correlated, biased) + small pure seed -> Pure randomness (for use in algorithms). Extractors: Physical randomness (correlated, biased) + small pure seed -> Pure randomness (for use in algorithms). Mergers: General primitive useful in the context of manipulating randomness. Mergers: General primitive useful in the context of manipulating randomness. Given: k (possibly dependent) random variables X 1 … X k, such that one is uniform over its domain, Given: k (possibly dependent) random variables X 1 … X k, such that one is uniform over its domain, Add: small seed s (Additional randomness) Add: small seed s (Additional randomness) Output: a uniform random variable Y. Output: a uniform random variable Y. May 5, 2010 MSRI 4

of 26 Merger Analysis Problem Merger(X 1,…,X k ; s) = f(s), Merger(X 1,…,X k ; s) = f(s), where X 1, …, X k 2 F q n ; s 2 F q where X 1, …, X k 2 F q n ; s 2 F q and f is deg. k-1 function mapping F  F n and f is deg. k-1 function mapping F  F n s.t. f(i) = X i. s.t. f(i) = X i. (f is the curve through X 1,…,X k ) (f is the curve through X 1,…,X k ) Question: For what choices of q, n, k is Merger’s output close to uniform? Question: For what choices of q, n, k is Merger’s output close to uniform? Arises from [DvirShpilka’05, DvirWigderson’08]. Arises from [DvirShpilka’05, DvirWigderson’08]. “Statistical high-deg. version” of Kakeya problem. “Statistical high-deg. version” of Kakeya problem. May 5, 2010 MSRI 5

of 26 List-decoding of Reed-Solomon codes Given L polynomials P 1,…,P L of degree d; and sets S 1,…,S L ½ F £ F s.t. Given L polynomials P 1,…,P L of degree d; and sets S 1,…,S L ½ F £ F s.t. |S i | = t |S i | = t S i ½ {(x,P i (x)) | x 2 F} S i ½ {(x,P i (x)) | x 2 F} How small can n = |S| be, where S = [ i S i ? How small can n = |S| be, where S = [ i S i ? Problem arises in “List-decoding of RS codes” Problem arises in “List-decoding of RS codes” Algebraic analysis from [S. ‘96, GuruswamiS’98] basis of decoding algorithms. Algebraic analysis from [S. ‘96, GuruswamiS’98] basis of decoding algorithms. May 5, 2010 MSRI 6

of 26 What is common? Given a set in F q n with nice algebraic properties, want to understand its size. Given a set in F q n with nice algebraic properties, want to understand its size. Kakeya Problem: Kakeya Problem: The Kakeya Set. The Kakeya Set. Merger Problem: Merger Problem: Any set T ½ F n that contains ² -fraction of points on ² -fraction of merger curves. Any set T ½ F n that contains ² -fraction of points on ² -fraction of merger curves. If T small, then output is non-uniform; else output is uniform. If T small, then output is non-uniform; else output is uniform. List-decoding problem: List-decoding problem: The union of the sets. The union of the sets. May 5, 2010 MSRI 7

of 26 List-decoding analysis [S ‘96] Construct Q(x,y) ≠ 0 s.t. Construct Q(x,y) ≠ 0 s.t. Deg y (Q) < L Deg y (Q) < L Deg x (Q) < n/L Deg x (Q) < n/L Q(x,y) = 0 for every (x,y) 2 S = [ i S i Q(x,y) = 0 for every (x,y) 2 S = [ i S i Can Show: t > n/L + dL ) (y – P i (x)) | Q Can Show: t > n/L + dL ) (y – P i (x)) | Q Conclude: n ¸ L ¢ (t – dL). Conclude: n ¸ L ¢ (t – dL). (Can be proved combinatorially also; (Can be proved combinatorially also; using inclusion-exclusion) using inclusion-exclusion) If L > t/(2d), yield n ¸ t 2 /(4d) If L > t/(2d), yield n ¸ t 2 /(4d) May 5, 2010 MSRI 8

of 26 Kakeya Set analysis [Dvir ‘08] Find Q(x 1,…,x n ) ≠ 0 s.t. Find Q(x 1,…,x n ) ≠ 0 s.t. Total deg. of Q < q (let deg. = d) Total deg. of Q < q (let deg. = d) Q(x) = 0 for every x 2 K. ( exists if |K| < q n /n!) Q(x) = 0 for every x 2 K. ( exists if |K| < q n /n!) Prove that homogenous deg. d part of Q vanishes on y, if there exists a line in direction y that is contained in K. Prove that homogenous deg. d part of Q vanishes on y, if there exists a line in direction y that is contained in K. Line L ½ K ) Q| L = 0. Line L ½ K ) Q| L = 0. Highest degree coefficient of Q| L is homogenous part of Q evaluated at y. Highest degree coefficient of Q| L is homogenous part of Q evaluated at y. Conclude: homogenous part of Q = 0. > <. Yields |K| ¸ q n /n!. Yields |K| ¸ q n /n!. May 5, 2010 MSRI 9

of 26 Improved L-D. Analysis [G.+S. ‘98] Can we improve on the inclusion-exclusion bound? Working when t < dL? Can we improve on the inclusion-exclusion bound? Working when t < dL? Idea: Try fitting a polynomial Q that passes through each point with “multiplicity” 2. Idea: Try fitting a polynomial Q that passes through each point with “multiplicity” 2. Can find with Deg y < L, Deg x < 3n/L. Can find with Deg y < L, Deg x < 3n/L. If 2t > 3n/L + dL then (y-P i (x)) | Q. If 2t > 3n/L + dL then (y-P i (x)) | Q. Yields n ¸ (L/3).(2t – dL) Yields n ¸ (L/3).(2t – dL) If L>t/d, then n ¸ t 2 /(3d). If L>t/d, then n ¸ t 2 /(3d). Optimizing Q; letting mult.  1, get n ¸ t 2 /d Optimizing Q; letting mult.  1, get n ¸ t 2 /d May 5, 2010 MSRI 10

of 26 Aside: Is the factor of 2 important? Results in some improvement in [GS] (allowed us to improve list-decoding for codes of high rate) … Results in some improvement in [GS] (allowed us to improve list-decoding for codes of high rate) … But crucial to subsequent work But crucial to subsequent work [Guruswami-Rudra] construction of rate- optimal codes: Couldn’t afford to lose this factor of 2 (or any constant > 1). [Guruswami-Rudra] construction of rate- optimal codes: Couldn’t afford to lose this factor of 2 (or any constant > 1). May 5, 2010 MSRI 11

of 26 Multiplicity = ? Over reals: f(x,y,z) has root of multiplicity m at (a,b,c) if every partial derivative of order up to m-1 vanishes at 0. Over reals: f(x,y,z) has root of multiplicity m at (a,b,c) if every partial derivative of order up to m-1 vanishes at 0. Over finite fields? Over finite fields? Derivatives don’t work; but “Hasse derivatives” do. What are these? Later… Derivatives don’t work; but “Hasse derivatives” do. What are these? Later… There are {m + n choose n} such derivatives, for n-variate polynomials; There are {m + n choose n} such derivatives, for n-variate polynomials; Each is a linear function of coefficients of f. Each is a linear function of coefficients of f. May 5, 2010 MSRI 12

of 26 Multiplicities in Kakeya [Saraf,S ’08] Back to K ½ F n. Fit Q that vanishes often? Back to K ½ F n. Fit Q that vanishes often? Works! Works! Can find Q ≠ 0 of individual degree < q, that vanishes at each point with multiplicity n, provided |K| 4 n < q n Can find Q ≠ 0 of individual degree < q, that vanishes at each point with multiplicity n, provided |K| 4 n < q n Q| L is of degree < qn. Q| L is of degree < qn. But it vanishes with multiplicity n at q points! But it vanishes with multiplicity n at q points! So it is identically zero ) its highest degree coeff. is zero. > < Conclude: |K| ¸ (q/4) n Conclude: |K| ¸ (q/4) n May 5, 2010 MSRI 13

of 26 Comparing the bounds Simple: |K| ¸ q n/2 Simple: |K| ¸ q n/2 [Dvir]: |K| ¸ q n /n! [Dvir]: |K| ¸ q n /n! [SS]: |K| ¸ q n /4 n [SS]: |K| ¸ q n /4 n [SS] improves Simple even when q (large) constant and n  1 (in particular, allows q < n) [SS] improves Simple even when q (large) constant and n  1 (in particular, allows q < n) [MockenhauptTao, Dvir]: [MockenhauptTao, Dvir]: 9 K s.t. |K| · q n /2 n-1 + O(q n-1 ) 9 K s.t. |K| · q n /2 n-1 + O(q n-1 ) Can we do even better? Can we do even better? Improve Merger Analysis? Improve Merger Analysis? May 5, 2010 MSRI 14

of 26 Concerns from Merger Analysis Recall Merger(X 1,…,X k ; s) = f(s), Recall Merger(X 1,…,X k ; s) = f(s), where X 1, …, X k 2 F q n ; s 2 F q where X 1, …, X k 2 F q n ; s 2 F q and f is deg. k-1 curve s.t. f(i) = X i. and f is deg. k-1 curve s.t. f(i) = X i. [DW08] Say X 1 random; Let K be such that ² fraction of choices of X 1,…,X k lead to “bad” curves such that ² fraction of s’s such that Merger outputs value in K with high probability. [DW08] Say X 1 random; Let K be such that ² fraction of choices of X 1,…,X k lead to “bad” curves such that ² fraction of s’s such that Merger outputs value in K with high probability. Build low-deg. poly Q vanishing on K; Prove for “bad” curves, Q vanishes on curve; and so Q vanishes on ² -fraction of X 1 ’s (and so ² -fraction of domain). Build low-deg. poly Q vanishing on K; Prove for “bad” curves, Q vanishes on curve; and so Q vanishes on ² -fraction of X 1 ’s (and so ² -fraction of domain). Apply Schwartz-Zippel. > < May 5, 2010 MSRI 15

of 26 Concerns from Merger Analysis [DW] Analysis: Works only if q > n. [DW] Analysis: Works only if q > n. So seed length = log 2 q > log 2 n So seed length = log 2 q > log 2 n Not good enough for setting where k = O(1), and n  1. Not good enough for setting where k = O(1), and n  1. (Would like seed length to be O(log k)). (Would like seed length to be O(log k)). Multiplicty technique: Seems to allow q < n. Multiplicty technique: Seems to allow q < n. But doesn’t seem to help … But doesn’t seem to help … Degrees of polynomials at most qn; Degrees of polynomials at most qn; Limits multiplicities. Limits multiplicities. May 5, 2010 MSRI 16

of 26 General obstacle in multiplicity method Can’t force polynomial Q to vanish with too high a multiplicity. Gives no benefit. Can’t force polynomial Q to vanish with too high a multiplicity. Gives no benefit. E.g. Kakeya problem: Why stop at mult = n? E.g. Kakeya problem: Why stop at mult = n? Most we can hope from Q is that it vanishes on all of q n ; Most we can hope from Q is that it vanishes on all of q n ; Once this happens, Q = 0, if its degree is < q in each variable. Once this happens, Q = 0, if its degree is < q in each variable. So Q| L is of degree at most qn, so mult n suffices. Using larger multiplicity can’t help! So Q| L is of degree at most qn, so mult n suffices. Using larger multiplicity can’t help! Or can it? Or can it? May 5, 2010 MSRI 17

of 26 Extended method of multiplicities (In Kakeya context): (In Kakeya context): Perhaps Q can be shown to vanish with high multiplicity at each point in F n. Perhaps Q can be shown to vanish with high multiplicity at each point in F n. (Technical question: How?) (Technical question: How?) Perhaps vanishing of Q with high multiplicity at each point shows higher degree polynomials (deg > q in each variable) are identically zero? Perhaps vanishing of Q with high multiplicity at each point shows higher degree polynomials (deg > q in each variable) are identically zero? (Needed: An extension of Schwartz-Zippel.) (Needed: An extension of Schwartz-Zippel.) May 5, 2010 MSRI 18

of 26 Multiplicities? Q(X 1,…,X n ) has zero of mult. m at a = (a 1,…,a n ) if all (Hasse) derivatives of order < m vanish. Q(X 1,…,X n ) has zero of mult. m at a = (a 1,…,a n ) if all (Hasse) derivatives of order < m vanish. Hasse derivative = ? Hasse derivative = ? Formally defined in terms of coefficients of Q, various multinomial coefficients and a. Formally defined in terms of coefficients of Q, various multinomial coefficients and a. But really … But really … The i = (i1,…, in)th derivative is the coefficient of z 1 i1 …z n in in Q(z + a). The i = (i1,…, in)th derivative is the coefficient of z 1 i1 …z n in in Q(z + a). Even better … coeff. of z i in Q(z+x) Even better … coeff. of z i in Q(z+x) (defines ith derivative Q i as a function of x; can evaluate at x = a). (defines ith derivative Q i as a function of x; can evaluate at x = a). May 5, 2010 MSRI 19

of 26 Key Properties Each derivative is a linear function of coefficients of Q. [Used in [GS’98], [SS’09].] (Q+R) i = Q i + R i Each derivative is a linear function of coefficients of Q. [Used in [GS’98], [SS’09].] (Q+R) i = Q i + R i Q has zero of mult m at a, and S is a curve that passes through a, then Q| S has zero of mult m at a. [Used for lines in prior work.] Q has zero of mult m at a, and S is a curve that passes through a, then Q| S has zero of mult m at a. [Used for lines in prior work.] Q i is a polynomial of degree deg(Q) -  j i i (not used in prior works) Q i is a polynomial of degree deg(Q) -  j i i (not used in prior works) (Q i ) j ≠ Q i+j, but Q i+j (a) = 0 ) (Q i ) j (a) = 0 (Q i ) j ≠ Q i+j, but Q i+j (a) = 0 ) (Q i ) j (a) = 0 Q vanishes with mult m at a Q vanishes with mult m at a ) Q i vanishes with mult m -  j i i at a. ) Q i vanishes with mult m -  j i i at a. May 5, 2010 MSRI 20

of 26 Propagating multiplicities (in Kakeya) Find Q that vanishes with mult m on K Find Q that vanishes with mult m on K For every i of order m/2, Q_i vanishes with mult m/2 on K. For every i of order m/2, Q_i vanishes with mult m/2 on K. Conclude: Q, as well as all derivatives of Q of order m/2 vanish on F n Conclude: Q, as well as all derivatives of Q of order m/2 vanish on F n ) Q vanishes with multiplicity m/2 on F n ) Q vanishes with multiplicity m/2 on F n Next Question: When is a polynomial (of deg > qn, or even q n ) that vanishes with high multiplicity on q n identically zero? Next Question: When is a polynomial (of deg > qn, or even q n ) that vanishes with high multiplicity on q n identically zero? May 5, 2010 MSRI 21

of 26 Vanishing of high-degree polynomials Mult(Q,a) = multiplicity of zeroes of Q at a. Mult(Q,a) = multiplicity of zeroes of Q at a. I(Q,a) = 1 if mult(Q,a) > 0 and 0 o.w. I(Q,a) = 1 if mult(Q,a) > 0 and 0 o.w. = min{1, mult(Q,a)} = min{1, mult(Q,a)} Schwartz-Zippel: for any S ½ F Schwartz-Zippel: for any S ½ F  I(Q,a) · d. |S| n-1 where sum is over a 2 S n  I(Q,a) · d. |S| n-1 where sum is over a 2 S n Can we replace I with mult above? Would strengthen S-Z, and be useful in our case. Can we replace I with mult above? Would strengthen S-Z, and be useful in our case. [DKSS ‘09]: Yes … (simple inductive proof [DKSS ‘09]: Yes … (simple inductive proof … that I can’t remember) … that I can’t remember) May 5, 2010 MSRI 22

of 26 Back to Kakeya Find Q of degree d vanishing on K with mult m. (can do if (m/n) n |K| m n |K| ) Find Q of degree d vanishing on K with mult m. (can do if (m/n) n |K| m n |K| ) Conclude Q vanishes on F n with mult. m/2. Conclude Q vanishes on F n with mult. m/2. Apply Extended-Schwartz-Zippel to conclude Apply Extended-Schwartz-Zippel to conclude (m/2) q n < d q n-1 (m/2) q n < d q n-1, (m/2) q < d, (m/2) n q n < d n = m n |K| Conclude: |K| ¸ (q/2) n Conclude: |K| ¸ (q/2) n Tight to within 2+o(1) factor! Tight to within 2+o(1) factor! May 5, 2010 MSRI 23

of 26 Consequences for Mergers Can analyze [DW] merger when q > k very small, n growing; Can analyze [DW] merger when q > k very small, n growing; Analysis similar, more calculations. Analysis similar, more calculations. Yields: Seed length log q (independent of n). Yields: Seed length log q (independent of n). By combining it with every other ingredient in extractor construction: By combining it with every other ingredient in extractor construction: Extract all but vanishing entropy (k – o(k) bits of randomness from (n,k) sources) using O(log n) seed (for the first time). Extract all but vanishing entropy (k – o(k) bits of randomness from (n,k) sources) using O(log n) seed (for the first time). May 5, 2010 MSRI 24

of 26 Conclusions Method of multiplicities Method of multiplicities Extends power of algebraic techniques beyond “low-degree” polynomials. Extends power of algebraic techniques beyond “low-degree” polynomials. Key ingredient: Extended Schwartz-Zippel lemma. Key ingredient: Extended Schwartz-Zippel lemma. Gives applications to Gives applications to Kakeya Sets: Near tight bounds Kakeya Sets: Near tight bounds Extractors: State of the art constructions Extractors: State of the art constructions RS List-decoding: Best known algorithm [GS ‘98] + algebraic proofs of known bounds [DKSS ‘09]. RS List-decoding: Best known algorithm [GS ‘98] + algebraic proofs of known bounds [DKSS ‘09]. Open: Open: Other applications? Why does it work? Other applications? Why does it work? May 5, 2010 MSRI 25

May 5, 2010 MSRI 26 Thank You