CSE 20: Discrete Mathematics for Computer Science Prof. Shachar Lovett

Today’s Topics: GCD Euclid’s algorithm Extended Euclid’s algorithm

1. GCD The poor man version of prime factorization

GCD Greatest common divisor Given two positive integers a,b, their GCD is the largest integer n such that n|a and n|b

GCD What is GCD(20,30)? 5 10 20 30 Other

GCD GCD is the same as finding simplified rational expressions, just like you did in school GCD(20,30)=10 means that 20 30 = 2∗10 3∗10 = 2 3 and it can’t be simplified further

GCD What is GCD(101281371,347832984723)?

GCD How can we compute GCD(a,b)? Simple way: Compute prime factorization of a,b Take common primes and prime powers Example: if a=2103859 and b=21735 then GCD(a,b)=21035 However, we believe that computing the prime factorization of large numbers is hard… Euclid’s algorithm provides a much faster way

2. Euclid’s algorithm Fast GCD

Euclid’s algorithm Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While b>0: x=a mod b a=b b=x Return a

Euclid’s algorithm Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While b>0: x=a mod b a=b b=x Return a (a,b) (b,a mod b)

Euclid’s algorithm Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While b>0: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Return a

Euclid’s algorithm Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While b>0: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Return a Example run: a=20, b=30 a 20 30 10 b 30 20 10

Euclid’s algorithm The same basic questions Does it always terminate? Does it return the correct answer? How fast is it?

Euclid’s algorithm: termination Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While 𝑏>0: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Return a Loop invariant (after 1st iteration): 𝑎>𝑏

Euclid’s algorithm: termination Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While 𝑏>0: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Return a Loop invariant (after 1st iteration): 𝑎>𝑏 Loop beginning: (a,b) Loop end: (b, a mod b) By definition, 𝑎 𝑚𝑜𝑑 𝑏∈ 0,…,𝑏−1 and hence (a mod b) < b

Euclid’s algorithm: termination Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While 𝑏>0: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Return a Loop invariant (after 1st iteration): 𝑎>𝑏 The value of a keeps decreasing, which proves termination

Euclid’s algorithm: correctness Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While 𝑏>0: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Return a g=gcd(a,b)

Euclid’s algorithm: correctness Need to prove the following lemma Lemma: ∀𝑎,𝑏≥1, gcd 𝑎,𝑏 =gcd⁡(𝑏,𝑎 𝑚𝑜𝑑 𝑏)

Euclid’s algorithm: correctness Need to prove the following lemma Lemma: ∀𝑎,𝑏≥1, gcd 𝑎,𝑏 =gcd⁡(𝑏,𝑎 𝑚𝑜𝑑 𝑏) Proof: we will actually show ∀𝑚≥1, 𝑚 𝑎∧𝑚 𝑏 ⇔(𝑚 𝑏∧𝑚 𝑎 𝑚𝑜𝑑 𝑏 ) In particular, the largest such m is (by definition) the GCD, and so gcd 𝑎,𝑏 =gcd⁡(𝑏,𝑎 𝑚𝑜𝑑 𝑏)

Euclid’s algorithm: correctness Lemma: ∀𝑎,𝑏≥1, gcd 𝑎,𝑏 = gcd 𝑏,𝑎 𝑚𝑜𝑑 𝑏 Proof (⇒): 𝑚 𝑎∧𝑚 𝑏 ⇒(𝑚 𝑏∧𝑚 𝑎 𝑚𝑜𝑑 𝑏 ) Let 𝑎 = 𝑏𝑝+𝑞, where 𝑝=𝑎 𝑑𝑖𝑣 𝑏, 𝑞 = 𝑎 𝑚𝑜𝑑 𝑏. If 𝑚|𝑎,𝑚|𝑏 then 𝑎=𝑚𝑥, 𝑏=𝑚𝑦, where 𝑥,𝑦∈𝑍. Then: 𝑎 𝑚𝑜𝑑 𝑏=𝑞=𝑎−𝑏𝑝=𝑚𝑥−𝑚𝑦⋅𝑝=𝑚 𝑥−𝑦𝑝 So, 𝑚|(𝑎 𝑚𝑜𝑑 𝑏).

Euclid’s algorithm: correctness Lemma: ∀𝑎,𝑏≥1, gcd 𝑎,𝑏 = gcd 𝑏,𝑎 𝑚𝑜𝑑 𝑏 Proof (⇐): 𝑚 𝑎∧𝑚 𝑏 ⇐(𝑚 𝑏∧𝑚 𝑎 𝑚𝑜𝑑 𝑏 ) Let 𝑎 = 𝑏𝑝+𝑞, where 𝑝=𝑎 𝑑𝑖𝑣 𝑏, 𝑞 = 𝑎 𝑚𝑜𝑑 𝑏. If 𝑚|𝑏,𝑚|𝑞 then b=𝑚𝑥, q=𝑚𝑦, where 𝑥,𝑦∈𝑍. Then: 𝑎=𝑏𝑝+𝑞=𝑚𝑥⋅𝑝+𝑚𝑦=𝑚(𝑥𝑝+𝑦) So, 𝑚|𝑎.

Euclid’s algorithm: correctness Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While 𝑏>0: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Return a g=gcd(a,b) Proved!

Euclid’s algorithm: speed Euclid(a,b): Input: 𝑎,𝑏∈𝑁 Output: 𝑔=gcd⁡(𝑎,𝑏) While 𝑏>0: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Return a How many iterations?

Euclid’s algorithm: speed Consider one iteration: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) We know that a,b become smaller But by how much?

Euclid’s algorithm: speed Consider one iteration: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) We know that a,b become smaller But by how much? Intuition: Since 𝑎 𝑚𝑜𝑑 𝑏∈{0,..,𝑏−1}, on “average” 𝑎 𝑚𝑜𝑑 𝑏≈𝑏/2. Hence, value of b decreases by a factor of 2 at each iteration (so log(b) iterations will be needed) Can we justify this?

Euclid’s algorithm: speed Consider one iteration: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Lemma: 𝑎+𝑏≥ 3 2 (𝑏+ 𝑎 𝑚𝑜𝑑 𝑏 ) Sum decreases by 3/2 at each iteration; So, we need at most ~log(a+b) iterations In fact, at most ~log(b) iterations, since after the first iteration, both values are at most b

Euclid’s algorithm: speed Consider one iteration: 𝑎,𝑏 ←(𝑏, 𝑎 𝑚𝑜𝑑 𝑏) Lemma: 𝑎+𝑏≥ 3 2 (𝑏+ 𝑎 𝑚𝑜𝑑 𝑏 ) Proof: Since 𝑎>𝑏>𝑎 𝑚𝑜𝑑 𝑏, and since b|𝑎−(𝑎 𝑚𝑜𝑑 𝑏), we must have 𝑎− 𝑎 𝑚𝑜𝑑 𝑏 ≥𝑏. So: 𝑎≥𝑏+(𝑎 𝑚𝑜𝑑 𝑏) 𝑏= 𝑏 2 + 𝑏 2 ≥ 1 2 𝑏+ 𝑎 𝑚𝑜𝑑 𝑏 ⇒𝑎+𝑏≥ 3 2 (𝑏+ 𝑎 𝑚𝑜𝑑 𝑏 )

3. Extended Euclid’s algorithm Using algorithms to do math!

Extended Euclid’s algorithm Theorem: ∀𝑎,𝑏∈𝑁 ∃𝑥,𝑦∈𝑍 𝑎𝑥+𝑏𝑦=gcd⁡(𝑎,𝑏) (this is called Extended Euclid’s algorithm) Example: a=3, b=5, gcd(a,b)=1 3*(-3)+5*2 = 1 (solution: x=-3, y=2)

Extended Euclid’s algorithm Theorem: ∀𝑎,𝑏∈𝑁 ∃𝑥,𝑦∈𝑍 𝑎𝑥+𝑏𝑦=gcd⁡(𝑎,𝑏) (this is called Extended Euclid’s algorithm) Example: a=3, b=5, gcd(a,b)=1 3*(-3)+5*2 = 1 (solution: x=-3, y=2) The proof will use our analysis of Euclid’s algorithm So, even though the theorem has nothing to do with algorithms, the proof will use an algorithm!

Extended Euclid’s algorithm Theorem: ∀𝑎≥𝑏≥1 ∃𝑥,𝑦∈𝑍 𝑎𝑥+𝑏𝑦=gcd⁡(𝑎,𝑏) Proof (by strong induction on b): Base case: b=1, so gcd(a,1)=a, can take x=1,y=1 Inductive case: We use the identity: gcd 𝑎,𝑏 =gcd⁡(𝑏,𝑎 𝑚𝑜𝑑 𝑏). Since 𝑏>𝑎 𝑚𝑜𝑑 𝑏, by the inductive assumption ∃ 𝑥 ′ , 𝑦 ′ ∈𝑍, 𝑏 𝑥 ′ + 𝑎 𝑚𝑜𝑑 𝑏 𝑦 ′ =gcd b,a mod b = gcd 𝑎,𝑏 Let 𝑎 = 𝑏𝑝+𝑞, where 𝑝=𝑎 𝑑𝑖𝑣 𝑏, 𝑞 = 𝑎 𝑚𝑜𝑑 𝑏. Then: 𝑏𝑥′+ 𝑎−𝑏𝑝 𝑦′= gcd 𝑎,𝑏 Take 𝑥= 𝑦 ′ ,𝑦= 𝑥 ′ −𝑝𝑦′ so that 𝑎𝑥+𝑏𝑦= gcd 𝑎,𝑏 .