The Method of Multiplicities Madhu Sudan Microsoft New England/MIT TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A Based on joint works with: V. Guruswami ‘98 S. Saraf ‘08 Z. Dvir, S. Kopparty, S. Saraf ‘09 June 16, SIAM Disc. Math.1

of 32 Agenda “Combinatorics = Math – Techniques?” “Combinatorics = Math – Techniques?” Except … it does have techniques! Except … it does have techniques! Probabilistic method, Spectral methods, Polynomial method, Nullstellensatz, … Probabilistic method, Spectral methods, Polynomial method, Nullstellensatz, … Today’s Agenda: A technique: Today’s Agenda: A technique: ( Different ) Polynomial Method + Multiplicity method ( Different ) Polynomial Method + Multiplicity method List-decoding of Reed-Solomon Codes List-decoding of Reed-Solomon Codes Bounding size of Kakeya Sets Bounding size of Kakeya Sets Extractor constructions Extractor constructions June 16, SIAM Disc. Math.

of 32 Part I: Decoding Reed-Solomon Codes Reed-Solomon Codes: Reed-Solomon Codes: Commonly used codes to store information (on CDs, DVDs etc.) Commonly used codes to store information (on CDs, DVDs etc.) Message: C 0, C 1, …, C d є F (finite field) Message: C 0, C 1, …, C d є F (finite field) Encoding: Encoding: View message as polynomial: M(x) =  i=0 d C i x i View message as polynomial: M(x) =  i=0 d C i x i Encoding = evaluations: { M( ® ) }_{ ® є F } Encoding = evaluations: { M( ® ) }_{ ® є F } Decoding Problem: Decoding Problem: Given: (x 1,y 1 ) … (x n,y n ) є F x F ; integers t,d; Given: (x 1,y 1 ) … (x n,y n ) є F x F ; integers t,d; Find: deg. d poly through t of the n points. Find: deg. d poly through t of the n points. June 16, SIAM Disc. Math.

of 32 List-decoding? If #errors (n-t) very large, then several polynomials may agree with t of n points. If #errors (n-t) very large, then several polynomials may agree with t of n points. List-decoding problem: List-decoding problem: Report all such polynomials. Report all such polynomials. Combinatorial obstacle: Combinatorial obstacle: There may be too many such polynomials. There may be too many such polynomials. Hope – can’t happen. Hope – can’t happen. To analyze: Focus on polynomials P 1,…, P L and set of agreements S 1 … S L. To analyze: Focus on polynomials P 1,…, P L and set of agreements S 1 … S L. Combinatorial question: Can S 1, … S L be large, while n = | [ j S j | is small? Combinatorial question: Can S 1, … S L be large, while n = | [ j S j | is small? June 16, SIAM Disc. Math.

of 32 List-decoding of Reed-Solomon codes Given L polynomials P 1,…,P L of degree d; and sets S 1,…,S L ½ F £ F s.t. Given L polynomials P 1,…,P L of degree d; and sets S 1,…,S L ½ F £ F s.t. |S i | = t |S i | = t S i ½ {(x,P i (x)) | x 2 F} S i ½ {(x,P i (x)) | x 2 F} How small can n = |S| be, where S = [ i S i ? How small can n = |S| be, where S = [ i S i ? Algebraic analysis from [S. ‘96, GuruswamiS ’98] basis of decoding algorithms. Algebraic analysis from [S. ‘96, GuruswamiS ’98] basis of decoding algorithms. June 16, SIAM Disc. Math.

of 32 List-decoding analysis [S ‘96] Construct Q(x,y) ≠ 0 s.t. Construct Q(x,y) ≠ 0 s.t. Deg y (Q) < L Deg y (Q) < L Deg x (Q) < n/L Deg x (Q) < n/L Q(x,y) = 0 for every (x,y) 2 S = [ i S i Q(x,y) = 0 for every (x,y) 2 S = [ i S i Can Show: Can Show: Such a Q exists (interpolation/counting). Such a Q exists (interpolation/counting). Implies: t > n/L + dL ) (y – P i (x)) | Q Implies: t > n/L + dL ) (y – P i (x)) | Q Conclude: n ¸ L ¢ (t – dL). Conclude: n ¸ L ¢ (t – dL). (Can be proved combinatorially also; (Can be proved combinatorially also; using inclusion-exclusion) using inclusion-exclusion) If L > t/(2d), yield n ¸ t 2 /(4d) If L > t/(2d), yield n ¸ t 2 /(4d) June 16, SIAM Disc. Math.

of 32 Focus: The Polynomial Method To analyze size of “algebraically nice” set S: To analyze size of “algebraically nice” set S: Find polynomial Q vanishing on S; Find polynomial Q vanishing on S; (Can prove existence of Q by counting coefficients … degree Q grows with |S|.) (Can prove existence of Q by counting coefficients … degree Q grows with |S|.) Use “algebraic niceness” of S to prove Q vanishes at other places as well. Use “algebraic niceness” of S to prove Q vanishes at other places as well. (In our case whenever y = P i (x) ). (In our case whenever y = P i (x) ). Conclude Q zero too often (unless S large). Conclude Q zero too often (unless S large). … (abstraction based on [Dvir]’s work) June 16, SIAM Disc. Math.

of 32 Improved L-D. Analysis [G.+S. ‘98] Can we improve on the inclusion-exclusion bound? Working when n > t^/(4d)? Can we improve on the inclusion-exclusion bound? Working when n > t^/(4d)? Idea: Try fitting a polynomial Q that passes through each point with “multiplicity” 2. Idea: Try fitting a polynomial Q that passes through each point with “multiplicity” 2. Can find with Deg y < L, Deg x < 3n/L. Can find with Deg y < L, Deg x < 3n/L. If 2t > 3n/L + dL then (y-P i (x)) | Q. If 2t > 3n/L + dL then (y-P i (x)) | Q. Yields n ¸ (L/3).(2t – dL) Yields n ¸ (L/3).(2t – dL) If L>t/d, then n ¸ t 2 /(3d). If L>t/d, then n ¸ t 2 /(3d). Optimizing Q; letting mult.  1, get n ¸ t 2 /d Optimizing Q; letting mult.  1, get n ¸ t 2 /d June 16, SIAM Disc. Math.

of 32 Aside: Is the factor of 2 important? Results in some improvement in [GS] (allowed us to improve list-decoding for codes of high rate) … Results in some improvement in [GS] (allowed us to improve list-decoding for codes of high rate) … But crucial to subsequent work But crucial to subsequent work [Guruswami-Rudra] construction of rate- optimal codes: Couldn’t afford to lose this factor of 2 (or any constant > 1). [Guruswami-Rudra] construction of rate- optimal codes: Couldn’t afford to lose this factor of 2 (or any constant > 1). June 16, SIAM Disc. Math.

of 32 Focus: The Multiplicity Method To analyze size of “algebraically nice” set S: To analyze size of “algebraically nice” set S: Find poly Q zero on S (w. high multiplicity); Find poly Q zero on S (w. high multiplicity); (Can prove existence of Q by counting coefficients … degree Q grows with |S|.) (Can prove existence of Q by counting coefficients … degree Q grows with |S|.) Use “algebraic niceness” of S to prove Q vanishes at other places as well. Use “algebraic niceness” of S to prove Q vanishes at other places as well. (In our case whenever y = P i (x) ). (In our case whenever y = P i (x) ). Conclude Q zero too often (unless S large). Conclude Q zero too often (unless S large). June 16, SIAM Disc. Math.

of 32 Multiplicity = ? Over reals: Q(x,y) has root of multiplicity m+1 at (a,b) if every partial derivative of order up to m vanishes at 0. Over reals: Q(x,y) has root of multiplicity m+1 at (a,b) if every partial derivative of order up to m vanishes at 0. Over finite fields? Over finite fields? Derivatives don’t work; but “Hasse derivatives” do. What are these? Later… Derivatives don’t work; but “Hasse derivatives” do. What are these? Later… There are {m+n choose n} such derivatives, for n-variate polynomials; There are {m+n choose n} such derivatives, for n-variate polynomials; Each is a linear function of coefficients of f. Each is a linear function of coefficients of f. June 16, SIAM Disc. Math.

of 32 Part II: Kakeya Sets June 16, SIAM Disc. Math.

of 32 Kakeya Sets K ½ F n is a Kakeya set if it has a line in every direction. K ½ F n is a Kakeya set if it has a line in every direction. I.e., 8 y 2 F n 9 x 2 F n s.t. {x + t.y | t 2 F} ½ K I.e., 8 y 2 F n 9 x 2 F n s.t. {x + t.y | t 2 F} ½ K F is a field (could be Reals, Rationals, Finite). F is a field (could be Reals, Rationals, Finite). Our Interest: Our Interest: F = F q (finite field of cardinality q). F = F q (finite field of cardinality q). Lower bounds. Lower bounds. Simple/Obvious: q n/2 · K · q n Simple/Obvious: q n/2 · K · q n Do better? Mostly open till [Dvir 2008]. Do better? Mostly open till [Dvir 2008]. June 16, SIAM Disc. Math.

of 32 Kakeya Set analysis [Dvir ‘08] Find Q(x 1,…,x n ) ≠ 0 s.t. Find Q(x 1,…,x n ) ≠ 0 s.t. Total deg. of Q < q (let deg. = d) Total deg. of Q < q (let deg. = d) Q(x) = 0 for every x 2 K. ( exists if |K| < q n /n!) Q(x) = 0 for every x 2 K. ( exists if |K| < q n /n!) Prove that (homogenous deg. d part of) Q vanishes on y, if there exists a line in direction y that is contained in K. Prove that (homogenous deg. d part of) Q vanishes on y, if there exists a line in direction y that is contained in K. Line L ½ K ) Q| L = 0. Line L ½ K ) Q| L = 0. Highest degree coefficient of Q| L is homogenous part of Q evaluated at y. Highest degree coefficient of Q| L is homogenous part of Q evaluated at y. Conclude: homogenous part of Q = 0. > <. Yields |K| ¸ q n /n!. Yields |K| ¸ q n /n!. June 16, SIAM Disc. Math.

of 32 Multiplicities in Kakeya [Saraf, S ’08] Fit Q that vanishes often? Fit Q that vanishes often? Good choice: #multiplicity m = n Good choice: #multiplicity m = n Can find Q ≠ 0 of individual degree < q, that vanishes at each point in K with multiplicity n, provided |K| 4 n < q n Can find Q ≠ 0 of individual degree < q, that vanishes at each point in K with multiplicity n, provided |K| 4 n < q n Q| L is of degree < qn. Q| L is of degree < qn. But it vanishes with multiplicity n at q points! But it vanishes with multiplicity n at q points! So it is identically zero ) its highest degree coeff. is zero. > < Conclude: |K| ¸ (q/4) n Conclude: |K| ¸ (q/4) n June 16, SIAM Disc. Math.

of 32 Comparing the bounds Simple: |K| ¸ q n/2 Simple: |K| ¸ q n/2 [Dvir]: |K| ¸ q n /n! [Dvir]: |K| ¸ q n /n! [SS]: |K| ¸ q n /4 n [SS]: |K| ¸ q n /4 n [SS] improves Simple even when q (large) constant and n  1 (in particular, allows q < n) [SS] improves Simple even when q (large) constant and n  1 (in particular, allows q < n) [MockenhauptTao, Dvir]: [MockenhauptTao, Dvir]: 9 K s.t. |K| · q n /2 n-1 + O(q n-1 ) 9 K s.t. |K| · q n /2 n-1 + O(q n-1 ) Can we do even better? Can we do even better? June 16, SIAM Disc. Math.

of 32 Part III: Randomness Mergers & Extractors June 16, SIAM Disc. Math.

of 32 Context One of the motivations for Dvir’s work: One of the motivations for Dvir’s work: Build better “randomness extractors” Build better “randomness extractors” Approach proposed in [Dvir-Shpilka] Approach proposed in [Dvir-Shpilka] Following [Dvir], new “randomness merger” and analysis given by [Dvir-Wigderson] Following [Dvir], new “randomness merger” and analysis given by [Dvir-Wigderson] Led to “extractors” matching known constructions, but not improving them … Led to “extractors” matching known constructions, but not improving them … What are Extractors? Mergers? … can we improve them? What are Extractors? Mergers? … can we improve them? June 16, SIAM Disc. Math.

of 32 Random Uniform Randomness in Computation Alg X F(X) Randomness Processors Distribution A Distribution B Support industry: Readily available randomness Prgs, (seeded) extractors, limited independence generators, epsilon-biased generators, Condensers, mergers, June 16, SIAM Disc. Math.

of 32 Randomness Extractors and Mergers Extractors: Extractors: Dirty randomness  Pure randomness Dirty randomness  Pure randomness Mergers: General primitive useful in the context of manipulating randomness. Mergers: General primitive useful in the context of manipulating randomness. k random variables  1 random variable k random variables  1 random variable June 16, SIAM Disc. Math. (Biased, correlated) (Uniform, independent … nearly) + small pure seed (One of them uniform) (high entropy) (Don’t know which, others potentially correlated) + small pure seed

of 32 Merger Analysis Problem Merger(X 1,…,X k ; s) = f(s), Merger(X 1,…,X k ; s) = f(s), where X 1, …, X k 2 F q n ; s 2 F q where X 1, …, X k 2 F q n ; s 2 F q and f is deg. k-1 function mapping F  F n and f is deg. k-1 function mapping F  F n s.t. f(i) = X i. s.t. f(i) = X i. (f is the curve through X 1,…,X k ) (f is the curve through X 1,…,X k ) Question: For what choices of q, n, k is Merger’s output close to uniform? Question: For what choices of q, n, k is Merger’s output close to uniform? Arises from [DvirShpilka’05, DvirWigderson’08]. Arises from [DvirShpilka’05, DvirWigderson’08]. “Statistical high-deg. version” of Kakeya problem. “Statistical high-deg. version” of Kakeya problem. June 16, SIAM Disc. Math.

of 32 Concerns from Merger Analysis [DW] Analysis: Worked only if q > n. [DW] Analysis: Worked only if q > n. So seed length = log 2 q > log 2 n So seed length = log 2 q > log 2 n Not good enough for setting where k = O(1), and n  1. Not good enough for setting where k = O(1), and n  1. (Would like seed length to be O(log k)). (Would like seed length to be O(log k)). Multiplicity technique: Multiplicity technique: seems bottlenecked at mult = n. seems bottlenecked at mult = n. June 16, SIAM Disc. Math.

of 32 General obstacle in multiplicity method Can’t force polynomial Q to vanish with too high a multiplicity. Gives no benefit. Can’t force polynomial Q to vanish with too high a multiplicity. Gives no benefit. E.g. Kakeya problem: Why stop at mult = n? E.g. Kakeya problem: Why stop at mult = n? Most we can hope from Q is that it vanishes on all of q n ; Most we can hope from Q is that it vanishes on all of q n ; Once this happens, Q = 0, if its degree is < q in each variable. Once this happens, Q = 0, if its degree is < q in each variable. So Q| L is of degree at most qn, so mult n suffices. Using larger multiplicity can’t help! So Q| L is of degree at most qn, so mult n suffices. Using larger multiplicity can’t help! Or can it? Or can it? June 16, SIAM Disc. Math.

of 32 Extended method of multiplicities (In Kakeya context): (In Kakeya context): Perhaps vanishing of Q with high multiplicity at each point shows higher degree polynomials (deg > q in each variable) are identically zero? Perhaps vanishing of Q with high multiplicity at each point shows higher degree polynomials (deg > q in each variable) are identically zero? (Needed: Condition on multiplicity of zeroes of multivariate polynomials.) (Needed: Condition on multiplicity of zeroes of multivariate polynomials.) Perhaps Q can be shown to vanish with high multiplicity at each point in F n. Perhaps Q can be shown to vanish with high multiplicity at each point in F n. (Technical question: How?) (Technical question: How?) June 16, SIAM Disc. Math.

of 32 Vanishing of high-degree polynomials Mult(Q,a) = multiplicity of zeroes of Q at a. Mult(Q,a) = multiplicity of zeroes of Q at a. I(Q,a) = 1 if mult(Q,a) > 0 and 0 o.w. I(Q,a) = 1 if mult(Q,a) > 0 and 0 o.w. = min{1, mult(Q,a)} = min{1, mult(Q,a)} Schwartz-Zippel: for any S ½ F Schwartz-Zippel: for any S ½ F  I(Q,a) · d. |S| n-1 where sum is over a 2 S n  I(Q,a) · d. |S| n-1 where sum is over a 2 S n Can we replace I with mult above? Would strengthen S-Z, and be useful in our case. Can we replace I with mult above? Would strengthen S-Z, and be useful in our case. [DKSS ‘09]: Yes … (simple inductive proof [DKSS ‘09]: Yes … (simple inductive proof … that I can’t remember) … that I can’t remember) June 16, SIAM Disc. Math.

of 32 Multiplicities? Q(X 1,…,X n ) has zero of mult. m at a = (a 1,…,a n ) if all (Hasse) derivatives of order < m vanish. Q(X 1,…,X n ) has zero of mult. m at a = (a 1,…,a n ) if all (Hasse) derivatives of order < m vanish. Hasse derivative = ? Hasse derivative = ? Formally defined in terms of coefficients of Q, various multinomial coefficients and a. Formally defined in terms of coefficients of Q, various multinomial coefficients and a. But really … But really … The i = (i1,…, in)th derivative is the coefficient of z 1 i1 …z n in in Q(z + a). The i = (i1,…, in)th derivative is the coefficient of z 1 i1 …z n in in Q(z + a). Even better … coeff. of z i in Q(z+x) Even better … coeff. of z i in Q(z+x) (defines ith derivative Q i as a function of x; can evaluate at x = a). (defines ith derivative Q i as a function of x; can evaluate at x = a). June 16, SIAM Disc. Math.

of 32 Key Properties Each derivative is a linear function of coefficients of Q. [Used in [GS’98], [SS’09].] (Q+R) i = Q i + R i Each derivative is a linear function of coefficients of Q. [Used in [GS’98], [SS’09].] (Q+R) i = Q i + R i Q has zero of mult m at a, and S is a curve that passes through a, then Q| S has zero of mult m at a. [Used for lines in prior work.] Q has zero of mult m at a, and S is a curve that passes through a, then Q| S has zero of mult m at a. [Used for lines in prior work.] Q i is a polynomial of degree deg(Q) -  j i i (not used in prior works) Q i is a polynomial of degree deg(Q) -  j i i (not used in prior works) (Q i ) j ≠ Q i+j, but Q i+j (a) = 0 ) (Q i ) j (a) = 0 (Q i ) j ≠ Q i+j, but Q i+j (a) = 0 ) (Q i ) j (a) = 0 Q vanishes with mult m at a Q vanishes with mult m at a ) Q i vanishes with mult m -  j i i at a. ) Q i vanishes with mult m -  j i i at a. June 16, SIAM Disc. Math.

of 32 Propagating multiplicities (in Kakeya) Find Q that vanishes with mult m on K Find Q that vanishes with mult m on K For every i of order m/2, Q_i vanishes with mult m/2 on K. For every i of order m/2, Q_i vanishes with mult m/2 on K. Conclude: Q, as well as all derivatives of Q of order m/2 vanish on F n Conclude: Q, as well as all derivatives of Q of order m/2 vanish on F n ) Q vanishes with multiplicity m/2 on F n ) Q vanishes with multiplicity m/2 on F n Next Question: When is a polynomial (of deg > qn, or even q n ) that vanishes with high multiplicity on q n identically zero? Next Question: When is a polynomial (of deg > qn, or even q n ) that vanishes with high multiplicity on q n identically zero? June 16, SIAM Disc. Math.

of 32 Back to Kakeya Find Q of degree d vanishing on K with mult m. (can do if (m/n) n |K| m n |K| ) Find Q of degree d vanishing on K with mult m. (can do if (m/n) n |K| m n |K| ) Conclude Q vanishes on F n with mult. m/2. Conclude Q vanishes on F n with mult. m/2. Apply Extended-Schwartz-Zippel to conclude Apply Extended-Schwartz-Zippel to conclude (m/2) q n < d q n-1 (m/2) q n < d q n-1, (m/2) q < d, (m/2) n q n < d n = m n |K| Conclude: |K| ¸ (q/2) n Conclude: |K| ¸ (q/2) n Tight to within 2+o(1) factor! Tight to within 2+o(1) factor! June 16, SIAM Disc. Math.

of 32 Consequences for Mergers Can analyze [DW] merger when q > k very small, n growing; Can analyze [DW] merger when q > k very small, n growing; Analysis similar, more calculations. Analysis similar, more calculations. Yields: Seed length log q (independent of n). Yields: Seed length log q (independent of n). By combining it with every other ingredient in extractor construction: By combining it with every other ingredient in extractor construction: Extract all but vanishing entropy (k – o(k) bits of randomness from (n,k) sources) using O(log n) seed (for the first time). Extract all but vanishing entropy (k – o(k) bits of randomness from (n,k) sources) using O(log n) seed (for the first time). June 16, SIAM Disc. Math.

of 32 Conclusions Combinatorics does have many “techniques” … Combinatorics does have many “techniques” … Polynomial method + Multiplicity method adds to the body Polynomial method + Multiplicity method adds to the body Supporting evidence: Supporting evidence: List decoding List decoding Kakeya sets Kakeya sets Extractors/Mergers Extractors/Mergers ??? ??? … just needs more creative names … … just needs more creative names … June 16, SIAM Disc. Math.

Thank You June 16, SIAM Disc. Math.32

of 32 Mutliplicity = ? Q vanishes with multiplicity 2 at (a,b): Q vanishes with multiplicity 2 at (a,b): Q(a,b) = 0; \del Q/del x(a,b) = 0; \del Q/\del y(a,b) = 0. Q(a,b) = 0; \del Q/del x(a,b) = 0; \del Q/\del y(a,b) = 0. \del Q/del x? \del Q/del x? Hasse derivative; Hasse derivative; Linear function of coefficients of Q; Linear function of coefficients of Q; Q zero with multiplicity m at (a,b) and C=(x(t),y(t)) is curve passing through (a,b), then Q|C has zero of multiplity m at (a,b). Q zero with multiplicity m at (a,b) and C=(x(t),y(t)) is curve passing through (a,b), then Q|C has zero of multiplity m at (a,b). June 16, SIAM Disc. Math.

of 32 Concerns from Merger Analysis Recall Merger(X 1,…,X k ; s) = f(s), Recall Merger(X 1,…,X k ; s) = f(s), where X 1, …, X k 2 F q n ; s 2 F q where X 1, …, X k 2 F q n ; s 2 F q and f is deg. k-1 curve s.t. f(i) = X i. and f is deg. k-1 curve s.t. f(i) = X i. [DW08] Say X 1 random; Let K be such that ² fraction of choices of X 1,…,X k lead to “bad” curves such that ² fraction of s’s such that Merger outputs value in K with high probability. [DW08] Say X 1 random; Let K be such that ² fraction of choices of X 1,…,X k lead to “bad” curves such that ² fraction of s’s such that Merger outputs value in K with high probability. Build low-deg. poly Q vanishing on K; Prove for “bad” curves, Q vanishes on curve; and so Q vanishes on ² -fraction of X 1 ’s (and so ² -fraction of domain). Build low-deg. poly Q vanishing on K; Prove for “bad” curves, Q vanishes on curve; and so Q vanishes on ² -fraction of X 1 ’s (and so ² -fraction of domain). Apply Schwartz-Zippel. > < June 16, SIAM Disc. Math.

of 32 What is common? Given a set in F q n with nice algebraic properties, want to understand its size. Given a set in F q n with nice algebraic properties, want to understand its size. Kakeya Problem: Kakeya Problem: The Kakeya Set. The Kakeya Set. Merger Problem: Merger Problem: Any set T ½ F n that contains ² -fraction of points on ² -fraction of merger curves. Any set T ½ F n that contains ² -fraction of points on ² -fraction of merger curves. If T small, then output is non-uniform; else output is uniform. If T small, then output is non-uniform; else output is uniform. List-decoding problem: List-decoding problem: The union of the sets. The union of the sets. June 16, SIAM Disc. Math.

of 32 Randomness Extractors and Mergers Extractors: Extractors: Dirty randomness  Pure randomness Dirty randomness  Pure randomness Mergers: General primitive useful in the context of manipulating randomness. Mergers: General primitive useful in the context of manipulating randomness. Given: k (dependent) random variables Given: k (dependent) random variables X 1 … X k, such that one is uniform. X 1 … X k, such that one is uniform. Add: small seed s (Additional randomness) Add: small seed s (Additional randomness) Output: a uniform random variable Y. Output: a uniform random variable Y. June 16, SIAM Disc. Math. (Biased, correlated) (Uniform, independent … nearly) + small pure seed