1. Ryan O’DonnellJohn Wright Carnegie Mellon

2. Picture by Jorge Cham a unit vector v in ℂ d (“qudit”)

3. Any measurement…... can only yield probabilistic info about the vector.

4. May rotate space by your favorite unitary U ∈ℂ d×d. Measurement outcome is i ∈ {1,2,…,d} with probability |e i, Uv| 2.... can only yield probabilistic info about the vector. Details, if you want to know: Any measurement…

5. Apparatus may itself be probabilistic

6. Actual output: p 1 p 2 · · · p d v 1 v 2 · · · v d orthonormal vectors in ℂ d

7. An unknown probability distribution over an unknown set of d orthonormal vectors. And “measuring” only gives you some probabilistic info about the outcome vector. That’s a triple whammy. Actual output: p 1 p 2 · · · p d v 1 v 2 · · · v d

8. It’s expensive, but you can hit the button n times. d=3, n=7 example outcome for the particles: v 1, v 3, v 2, v 2, v 1, v 1, v 3 with prob. p 1 · p 3 · p 2 · p 2 · p 1 · p 1 · p 3

9. It’s expensive, but you can hit the button n times. d=3, n=7 example outcome for the particles: with prob. p 1 · p 3 · p 2 · p 2 · p 1 · p 1 · p 3 v1⊗v3⊗v2⊗v2⊗v1⊗v1⊗v3v1⊗v3⊗v2⊗v2⊗v1⊗v1⊗v3 ∈ ( ℂ 3 ) ⊗ 7 You may measure each particle separately. Or, may do one giant measurement on ( ℂ d ) ⊗ n.

10. Quantum Problems #1: Learn v 1, …, v d, p 1, …, p d. (Approximately, up to some, w.h.p.) (“Quantum tomography”)

11. Quantum Problems #1: Learn v 1, …, v d, p 1, …, p d. #2: Learn the multiset {p 1, …, p d }. (Approximately, up to some, w.h.p.) (“Quantum spectrum estimation”)

12. Quantum Problems #1: Learn v 1, …, v d, p 1, …, p d. #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is or has ≤ r nonzeros (“Quantum spectrum property testing”)

13. Quantum Problems #1: Learn v 1, …, v d, p 1, …, p d. #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s and the associated v i ’s. (“Quantum principal component analysis”)

14. Quantum Problems #1: Learn v 1, …, v d, p 1, …, p d. #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s and the associated v i ’s. Et cetera

15. Quantum Problems #1: Learn v 1, …, v d, p 1, …, p d. #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s and the associated v i ’s. We [OW] have new tight upper and lower bounds. n ≤ O(d 2 ) suffices O(d 2 log d) shown independently in [HHJWY15]

16. Quantum Problems #1: Learn v 1, …, v d, p 1, …, p d. #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s We [OW] have new tight upper and lower bounds. n ≤ O(d 2 ) suffices n ≤ O(k 2 ) suffices

17. Quantum Problems #1: Learn v 1, …, v d, p 1, …, p d. #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s and the associated v i ’s. We [OW] have new tight upper and lower bounds. n ≤ O(d 2 ) suffices n ≤ O(kd) suffices

18. Quantum Problems #1: Learn v 1, …, v d, p 1, …, p d. #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s and the associated v i ’s. We [OW] have new tight upper and lower bounds. n ≤ O(d 2 ) suffices n = Θ(d) nec. & suff. n ≤ O(kd) suffices

19. #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s For simplicity, today’s focus: Just the problems depending on {p 1,..., p d }. What’s the methodology for proving bounds?

20. An unknown probability distribution. An unknown set of d orthonormal vectors. Actual output: p 1 p 2 · · · p d v 1 v 2 · · · v d

21. An unknown probability distribution. Actual output: p 1 p 2 · · · p d v 1 v 2 · · · v d If the vectors v 1, v 2, …, v d are known, you can measure the outcomes exactly. Setup becomes equivalent to: Learning / testing an unknown probability distribution on {1,2,…,d}

22. Classical Distribution Problems #1: Learn p 1, …, p d. (approximately, whp)

23. Classical Distribution Problems #1: Learn p 1, …, p d ) #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s Et cetera

24. Classical Distribution Problems #1: Learn p 1, …, p d ) #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s Last 10 years: some new tight upper and lower bounds.

25. #2: Learn the multiset {p 1, …, p d }. #3: Determine if {p 1, …, p d } satisfies a certain property. E.g., is #4: Learn the k largest p i ’s Last 10 years: some new tight upper and lower bounds. Focus: Problems depending on {p 1,..., p d }. What’s the methodology for proving bounds?

26. Typical sample when n=20, d=5 might be… 54423131423144554251 Idea 1: Permuting the n positions doesn’t matter. Hence may as well only retain histogram. Say we care about a property of {p 1, …, p d } (e.g., “Uniform distribution?” “Support ≤ r?”)

27. 1234512345 Typical sample when n=20, d=5 might be… 54423131423144554251 Say we care about a property of {p 1, …, p d } (e.g., “Uniform distribution?” “Support ≤ r?”)

28. Typical sample when n=20, d=5 might be… 1234512345 Idea 2: For properties we care about, permuting the d symbols doesn’t matter. Hence may as well sort the histogram. Say we care about a property of {p 1, …, p d } (e.g., “Uniform distribution?” “Support ≤ r?”)

29. Typical sample when n=20, d=5 might be… 1234512345 Say we care about a property of {p 1, …, p d } (e.g., “Uniform distribution?” “Support ≤ r?”)

30. Typical sample when n=20, d=5 might be… 1 st most freq: 2 nd most freq: 3 rd most freq: 4 th most freq: 5 th most freq: Say we care about a property of {p 1, …, p d } (e.g., “Uniform distribution?” “Support ≤ r?”)

31. Typical sample when n=20, d=5 might be… λ 1 := 1 st most freq: λ 2 := 2 nd most freq: λ 3 := 3 rd most freq: λ 4 := 4 th most freq: λ 5 := 5 th most freq: (Sorted histogram λ is AKA a Young diagram.) Say we care about a property of {p 1, …, p d } (e.g., “Uniform distribution?” “Support ≤ r?”)

32. Classically learning properties of {p 1, …, p d }, a summary: The problem has two commuting symmetries: S n -invariance (permuting the n outcomes) S d -invariance (permuting d outcome names) “Factoring these out”, WLOG learner just gets a random Young diagram λ (with n boxes, d rows) Remark: Pr[λ] = · m λ (p 1, …, p d ), (some certain symmetric polynomial in p 1, …, p d )

33. Quantumly learning properties of {p 1, …, p d }, a summary: The problem has two commuting symmetries: S n -invariance (permuting the n outcomes) U(d)-invariance (rotating unknown v 1, …, v d ) (some certain symmetric polynomial in p 1, …, p d ) d-dimensional unitary group

34. Quantumly learning properties of {p 1, …, p d }, a summary: The problem has two commuting symmetries: S n -invariance (permuting the n outcomes) U(d)-invariance (rotating unknown v 1, …, v d ) (some certain symmetric polynomial in p 1, …, p d ) “Factoring these out” involves Schur–Weyl duality from the representation theory of S n and U(d). This also involves Young diagrams! Upshot: again, WLOG the learner gets a random Young diagram, but with a weird distribution…

35. Quantumly learning properties of {p 1, …, p d }, a summary: The problem has two commuting symmetries: S n -invariance (permuting the n outcomes) U(d)-invariance (rotating unknown v 1, …, v d ) (some certain symmetric polynomial in p 1, …, p d ) “Factoring these out”, WLOG learner just gets a random Young diagram λ (n boxes, d rows), with : Pr[λ] = f λ · s λ (p 1, …, p d ) (some certain symmetric polynomial in p 1, …, p d )

36. Quantumly learning properties of {p 1, …, p d }, a summary: The problem has two commuting symmetries: S n -invariance (permuting the n outcomes) U(d)-invariance (rotating unknown v 1, …, v d ) (some certain symmetric polynomial in p 1, …, p d ) “Factoring these out”, WLOG learner just gets a random Young diagram λ (n boxes, d rows), with : Pr[λ] = f λ · s λ (p 1, …, p d ) (some certain symmetric polynomial in p 1, …, p d ) “Schur polynomial”# SYTs of shape λ

37. “Factoring these out”, WLOG learner just gets a random Young diagram λ (d rows, n boxes), with : Pr[λ] = f λ · s λ (p 1, …, p d ) There is a combinatorial interpretation of this! λ can be generated as follows: Pick a random word w of length n, each letter randomly drawn from [d] according to the p i ’s Apply the “RSK algorithm” to w Let λ be the ‘shape’ of the resulting tableau

38. RSK Algorithm Gilbert de B. Robinson Craige (Ea Ea) Schensted Donald E. Knuth

39. RSK Algorithm On input w = 54423131423144554251

40. RSK Algorithm On input w = 54423131423144554251 RSK applet by Tom Roby

41. RSK Algorithm On input w = 54423131423144554251 λ 1 = longest incr. subseq λ 1 + λ 2 = longest union of 2 incr. subseqs λ 1 + λ 2 + λ 3 = longest union of 3 incr. subseqs · · · λ 1 + λ 2 + λ 3 + · · · + λ d = n Alternative characterization:

42. RSK Algorithm On input w = 54423131423144554251 Alternative characterization: 54423131423144554251 Cor: Height = longest strictly decreasing subsequence

43. Summary p 1, …, p d are unknown probabilities. Learner specifies n. w ~ [d] n drawn with i.i.d. letters according to p i ’s. Classical case: get to see sorted histogram, μ. Quantum case: get to see “L.I.S. information”, λ. Learner tries to infer things about {p 1, …, p d }.

44. Remark: Majorization Classical case: get to see sorted histogram, μ. Quantum case: get to see “L.I.S. information”, λ. When w = 54423131423144554251 μλ ≻ partial sums of λ ≥ partial sums of μ

45. Example 1: Estimating p Max Classical case: get to see sorted histogram, μ. Quantum case: get to see “L.I.S. information”, λ. Classical case: Output μ 1 /n. Easy analysis: -accurate if n = O(1/ 2 ). Quantum case: Output λ 1 /n. (?) Does the L.I.S. concentrate around p Max ·n ? Not too hard: -accurate if n = Õ(d/ 2 ). Harder (?) [OW15b]: if n = O(1/ 2 ). a universal constant, independent of d, of p Max −p 2ndMax, etc.

46. p 1 = · · · = p d = 1/d ⇔ p Max as small as possible ⇔ λ 1 “as small as possible” ⇔ λ “as rectangular as possible” ⇔ Variance{λ 1, …, λ d } “small” ⇔ “small” Example 2: Testing uniformity Intuition: Can we determine ?

47. Example 2: Testing uniformity No, but we can determine a similar quantity: Can we determine ?, where Why? Goes back to representation theory of S n, Fourier analysis. Up to scaling, is

48. Methodological summary Half the time it’s probabilistic combinatorics of increasing subsequences in random words. Half the time it’s representation theory of S n and theory of (shifted) symmetric polynomials.

49. Conclusion

50. Suppose p 1 = · · · = p d = 1/d. Suppose d ≫ n.In fact, let “d = ∞”. All letters unique ⇔ w is a random permutation. Distribution on λ is the “Plancherel Distribution”. Now as n → ∞, there are many beautiful theorems about the limiting shape of λ.

51. n=1000 example (pic by Dan Romik) LIS = LDS ~ where ζ 1 ~ Tracy–Widom Suppose p 1 = · · · = p d = 1/d,with “d = ∞”. As n → ∞,

52. As n → ∞, scaled curve → Suppose p 1 = · · · = p d = 1/d,with “d = ∞”.

53. Suppose p 1 = · · · = p d = 1/d,with “d = ∞”. The case of “finite d” is less studied. The case of p i ’s not all equal is still less studied. The case when n is not assumed to be sufficiently large as a function of d is still less studied. But these cases are highly motivated by quantum mechanics, and I think many beautiful theorems are waiting to be discovered.

54. Thanks! Gluttons for punishment: I will give a 3-hour blackboard version of this talk next Friday, Oct. 9 at 1:30pm in Harvard’s Maxwell-Dworkin 119. Gluttons for pizza: Apparently there will be pizza at 1:15pm.