1 Communication Complexity מגישים: מיכאל זמור: /2 אבי מינץ: ערן מנצור: ת.ז /9

2 Communication Complexity The Model 2 Computers : A,B All calculations for A are free All calculations for B are free Algorithm costs are measured by cost of communications. Cost is measured per bits

3 Communication Complexity The Model Communication Complexity 14, 29,53,28,284,34839, 67,98,22,35,253 (48) (98) A B

4 Example Input A has array of n numbers B has array of n numbers Output median of all 2n numbers Heuristic numbers are O(log n) bits long

5 Naïve Algorithm 1.A sends all of his numbers to B 2.B calculates median of all 2n numbers Cost –Each number is O(log n) bits –N numbers are sent –Total cost is O(n*log n) bits

6 Communication Complexity Naive algorithm Communication Complexity 14, 29,53,28,284,34839, 67,98,22,35,253 (284) (348) (53) (29) (14) A B Total cost is O(n*log n) bits

7 Communication Complexity Naive algorithm Communication Complexity 14, 29,53,28,284,34839, 67,98,22,35,25314, 29,53,28,284,348 B calculate the median A B

8 Better algorithm 1.A sorts his array and sends his median ( ) to B 2.B sorts his array and sends his median ( ) to A. 3.Exercise : define : r = real median b = MAX{ } s = MIN { } prove :

9 Communication Complexity Better algorithm Communication Complexity 14, 28,29, 53,284,348,50022, 35,39, 67,98,253,300 A B B sort his array and find his median A sort his array and find his median

10 Communication Complexity Better algorithm Communication Complexity 14, 28,29, 53,284,348,50022, 35,39, 67,98,253,300 A B B send his median to A A send his median to B

11 4. If = then return ( = ) 5. If > then A throws top (n/2) elements B throws low (n/2) elements 6. And vice versa We reduces the size of the problem by half 7. Back to step 1, until size of arrays = 1

12 Communication Complexity Better algorithm Communication Complexity, 53,284,348,500 A B 67>53 Then B throws the big half of his array 53<67 Then A throws the small half of his array 14, 28,29 22, 35,39, 67,98,253,300

13 Communication Complexity Better algorithm Communication Complexity, 53,284,348,500 A B 14, 28,29 22, 35,39, 67,98,253,300 Since equal amount of members were thrown from two sides of the median So, new median not changed.

14 Communication Complexity Better algorithm Communication Complexity, 53,284,348,500 A B 14, 28,29 22, 35,39, 67,98,253,300 We will repeat this algorithm until the size of the array will be 1, while every loop the array is cut in half, and log n bits transferred Total cost is O (Log^2(n)) bits

15 Calculation of cost Each number is O(log n) bits Each step 2 numbers are sent log(n) steps Total cost is O (Log^2(n)) bits

16 Communication Complexity A B A calculates his median - B calculates his median another improvement

17 Two assumptions 1)The requested median is between the A’s median and B’s median A’s median B’s median The requested median between A’s and B’s median * (we have an even amount of numbers, so we choose the the 6 th place of the 10 numbers)

18 Two assumptions 2) In binary representation we divide the medians to common segment and different segment. Note: Sometimes there is no common segment, but if there is common for A’s and B’s medians then its common to the request median. Decimalbinary representation Common segment Different segment

19 Communication Complexity A B After those assumptions we continue…. Every side send each bit of his median (start from MSB bit) 27 = b 20 = b b1 b 1 b1 b

20 Communication Complexity A B Every side compares the bit he gets with the bit of his median. If equal, those bits are in the “common segment” (see slide 17) and this common to the request median then every side continue send the next bit of his median. 20 = b 1 b1 b 27 = b 1 b1 b

21 Communication Complexity A B If not equal then A sees that his median is bigger, so he throws the bigger half of his array, B sees that his median is smaller then A’s median, so he throws the smaller half of his array 27 = b 0 b0 b 20 = b 1 b1 b

22 Communication Complexity A B , After throwing half of his array, every side calculates his new median. Every side send bits of the new median, starting from the bit that there was different in the previous iteration ( and not from the MSB bit). Every side continues the algorithm.

23 Communication Complexity A B The algorithm stops when one of two events occur. 1) the arrays of A and B contain one element (each). 2) all bits of the medians were sent.

24 Communication Complexity A B If arrays contain one element so this element is the median. If all bits were sent so the medians are equal and this is the request median.

25 Complexity of algorithm Every iteration one of this events occur Bit is send to the other sides. (log n bits can be sent from every side) The array become shorter by half. (the array can reduce by half, log n times) So, sum of bits can be sent limited by O(log n).

26 Communication complexity The previous subject talked about problem of finding median of array that was divided to two parts. Now we consider a new problem : Each side has a number and we want to know if the numbers are equal. Communication Complexity A B X Y ? X=Y

27 Communication complexity Naïve algorithm A send X to B. B compares X to Y and return yes/no X is logX bits long so cost is logX Communication Complexity A B X Y X

28 Communication complexity New random algorithm - GLOBAL CC Communication Complexity A B X Y ? X=Y In this algorithm we have a random number R, that can be changed and known for both sides.

29 Communication complexity GLOBAL CC Communication Complexity A B X Y Lets define inner product of A and B: A  [0…..n-1], B  [0…..n-1]

30 Communication complexity GLOBAL CC Communication Complexity A B X Y A: calculates (X*R)mod 2 B: calculates (Y*R)mod 2

31 Communication complexity GLOBAL CC Communication Complexity A B X Y b X =(X*R)mod 2 b Y =(Y*R)mod 2 If X=Y then always b X= b Y If x ≠ Y then Prob(b X= b Y )=1/2

32 Communication complexity GLOBAL CC Communication Complexity A B X Y b X =(X*R)mod 2b Y =(Y*R)mod 2 Every side sends his result (b) and compares the results. If the results are not equal so the numbers are not equal. bXbX bYbY

33 Communication complexity GLOBAL CC Communication Complexity A B X Y b X =(X*R new )mod 2 b Y =(Y*R new )mod 2 We choose a new R and repeat the algorithm. We do so C times.

34 Communication complexity GLOBAL CC Communication Complexity A B X Y If X=Y then all C times b X = b Y. If X≠Y the probability that all C times b X = b Y is 2 -c.

35 Communication complexity Complexity of global CC Communication Complexity A B X Y Each time every side transfers b X/Y by length of 1 bit. There are C times so every side transfers C bits. Complexity = O ( C )

36 Communication complexity Another random algorithm Communication Complexity A B X Y In this algorithm every side constructs a polynom from his number according to these steps: Every number consist of n bits. X  [a n-1, a n-2,…..,a 1,a 0 ]Y  [a n-1, a n-2,…..,a 1,a 0 ]

37 Communication complexity Communication Complexity A B X Y Polynom of side A will be: Poly A (T)= a n-1 *T n-1 + a n-2 *T n-2 +…..+ a 0 *T 0 X  [a n-1, a n-2,…..,a 1,a 0 ]Y  [b n-1, b n-2,…..,b 1,b 0 ] Step 1:

38 Communication complexity Communication Complexity A B X Y Like side A polynom B will be: Poly b (T)= b n-1 *T n-1 + b n-2 *T n-2 +…..+ b 0 *T 0 X  [a n-1, a n-2,…..,a 1,a 0 ]Y  [b n-1, b n-2,…..,b 1,b 0 ] Step 1:

39 Communication complexity Communication Complexity A B X Y X  [a n-1, a n-2,…..,a 1,a 0 ]Y  [b n-1, b n-2,…..,b 1,b 0 ] Step 2: A chooses random prime p : n 2 <p < n 3 A sends p to B

40 Communication complexity Communication Complexity A B X Y Step 3: B chooses 1≤R ≤ p-1 and calculates (poly b (R) mod p) Remind: Poly b (R)= b n-1 *R n-1 + b n-2 *R n-2 +…..+ b 0 *R 0 B sends (R, (poly b (R) mod p) ) to A.

41 Communication complexity Communication Complexity A B X Y Step 4: A calculates poly a (R) mod p A compares (poly a (R) mod p) with (poly b (R) mod p)

42 Communication complexity Communication Complexity A B X Y Analysis of algorithm If poly a (R)  poly b (R) Then X  Y

43 Communication complexity Communication Complexity A B X Y Analysis of algorithm But if poly a (R) = poly b (R) there is some probability that X  Y. We will calculate F: F=prob(X  Y and poly a (R) = poly b (R))

44 Communication complexity Communication Complexity A B X Y Analysis of algorithm poly a (R) = poly b )R) poly a (R) - poly b )R)=0 F=prob[X  Y and poly a (R) = poly b )R(]= prob[X  Y and poly a (R)-poly b )R(=0]. poly a (R)-poly b )R(=0 is a polynom of degree n it has n-1 roots.

45 Communication complexity Communication Complexity A B X Y Analysis of algorithm Because there are n-1 R’s s.t. poly a (R)-poly b )R(=0. And 1<=R<=p-1. So the probability to choose one of those R’s is: (n-1)/p.

46 Communication complexity Communication Complexity A B X Y Analysis of algorithm We saw that n 2 <p < n 3 So: n-1/p>(n-1)/ n 2  1/n F=prob(X  Y and poly a (R) = poly b (R))<1/n The probability of mistake is:1/n

47 Communication complexity Communication Complexity A B X Y Analysis of algorithm Communication complexity: The sides transferred p and R only: n2 <p < n3 |p|=O(log n) And R<p So the total complexity is : O(log n)

48 Communication complexity Communication Complexity A B X Y CC and Global CC What is the complexity relation between CC and GCC Note: CC is the last alogorithm we had seen.

49 Communication complexity Communication Complexity A B X Y solution A and B have a deterministic Turing machine which generates (deterministically) K random r’s. (while K= )

50 Communication complexity Communication Complexity A B X Y solution Define R to be the chain of K’s random r’s R

51 Communication complexity Communication Complexity A B X Y Now, instead of A sending r to B, A only has to send index in R. The index can be sent with O(log( ) bits =O(log n)

52 Communication complexity Communication Complexity A B X Y Claim: For each subset of R there is in the subset that is never wrong for each (x,y).

53 Communication complexity Communication Complexity A B X Y Proof:

54 Communication complexity Communication Complexity A B X Y Proof:

55 Communication complexity Communication Complexity A B X Y So:

56 Communication complexity Communication Complexity A B X Y Conclusion: