Notes on the analysis of multiplication algorithms.. Dr. M. Sakalli, Marmara University.

Slides:



Advertisements
Similar presentations
A simple example finding the maximum of a set S of n numbers.
Advertisements

Analysis of Algorithms
Design & Analysis of Algorithms COMP 482 / ELEC 420 John Greiner.
CSE 373 Data Structures and Algorithms Lecture 5: Math Review/Asymptotic Analysis III.
CSE 373: Data Structures and Algorithms Lecture 5: Math Review/Asymptotic Analysis III 1.
More on Divide and Conquer. The divide-and-conquer design paradigm 1. Divide the problem (instance) into subproblems. 2. Conquer the subproblems by solving.
Number Theory and Cryptography
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. Problem Reduction Dr. M. Sakalli Marmara Unv, Levitin ’ s notes.
Introduction to Algorithms Jiafen Liu Sept
Grade School Revisited: How To Multiply Two Numbers Great Theoretical Ideas In Computer Science Victor Adamchik Danny Sleator CS Spring 2010 Lecture.
DIVIDE AND CONQUER APPROACH. General Method Works on the approach of dividing a given problem into smaller sub problems (ideally of same size).  Divide.
Divide and Conquer. Recall Complexity Analysis – Comparison of algorithm – Big O Simplification From source code – Recursive.
Nattee Niparnan. Recall  Complexity Analysis  Comparison of Two Algos  Big O  Simplification  From source code  Recursive.
Algorithm Design Strategy Divide and Conquer. More examples of Divide and Conquer  Review of Divide & Conquer Concept  More examples  Finding closest.
CS Section 600 CS Section 002 Dr. Angela Guercio Spring 2010.
Faster Multiplication, Powering of Polynomials
Grade School Revisited: How To Multiply Two Numbers Great Theoretical Ideas In Computer Science Steven RudichCS Spring 2004 Lecture 16March 4,
CSE 421 Algorithms Richard Anderson Lecture 13 Divide and Conquer.
Grade School Revisited: How To Multiply Two Numbers Great Theoretical Ideas In Computer Science S. Rudich V. Adamchik CS Spring 2006 Lecture 18March.
Arithmetic.
Asymptotic Growth Rates Themes –Analyzing the cost of programs –Ignoring constants and Big-Oh –Recurrence Relations & Sums –Divide and Conquer Examples.
MA/CSSE 473 Day 03 Asymptotics A Closer Look at Arithmetic With another student, try to write a precise, formal definition of “t(n) is in O(g(n))”
Prof. Swarat Chaudhuri COMP 482: Design and Analysis of Algorithms Spring 2013 Lecture 12.
Iterative Algorithm Analysis & Asymptotic Notations
Great Theoretical Ideas in Computer Science.
Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 3 Prof. Erik Demaine.
Project 2 due … Project 2 due … Project 2 Project 2.
Télécom 2A – Algo Complexity (1) Time Complexity and the divide and conquer strategy Or : how to measure algorithm run-time And : design efficient algorithms.
Divide & Conquer  Themes  Reasoning about code (correctness and cost)  recursion, induction, and recurrence relations  Divide and Conquer  Examples.
Karatsuba’s Algorithm for Integer Multiplication
Applied Symbolic Computation1 Applied Symbolic Computation (CS 300) Karatsuba’s Algorithm for Integer Multiplication Jeremy R. Johnson.
COMPSCI 102 Introduction to Discrete Mathematics.
Asymptotic Growth Rates  Themes  Analyzing the cost of programs  Ignoring constants and Big-Oh  Recurrence Relations & Sums  Divide and Conquer 
CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014CSC-305 Design and Analysis of AlgorithmsBS(CS) -6 Fall-2014 Design and Analysis of Algorithms.
4/3/2003CSE More Math CSE Algorithms Euclidean Algorithm Divide and Conquer.
CSE 421 Algorithms Lecture 15 Closest Pair, Multiplication.
1 Chapter 4-2 Divide and Conquer Slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved.
ADA: 4.5. Matrix Mult.1 Objective o an extra divide and conquer example, based on a question in class Algorithm Design and Analysis (ADA) ,
Divide and Conquer Andreas Klappenecker [based on slides by Prof. Welch]
1 How to Multiply Slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved. integers, matrices, and polynomials.
Young CS 331 D&A of Algo. Topic: Divide and Conquer1 Divide-and-Conquer General idea: Divide a problem into subprograms of the same kind; solve subprograms.
Divide and Conquer (Part II) Multiplication of two numbers Let U = (u 2n-1 u 2n-2 … u 1 u 0 ) 2 and V = (v 2n-1 v 2n-2 …v 1 v 0 ) 2, and our goal is to.
Divide and Conquer Faculty Name: Ruhi Fatima Topics Covered Divide and Conquer Matrix multiplication Recurrence.
MA/CSSE 473 Day 06 Mathematical Induction Modular Arithmetic Do question 1 on today's quiz (work with another person)
Grade School Revisited: How To Multiply Two Numbers CS Lecture 4 2 X 2 = 5.
CSCI-256 Data Structures & Algorithm Analysis Lecture Note: Some slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved. 16.
MA/CSSE 473 Day 14 Strassen's Algorithm: Matrix Multiplication Decrease and Conquer DFS.
MA/CSSE 473 Day 05 More induction Factors and Primes Recursive division algorithm.
LECTURE 2 : fundamentals of analysis of algorithm efficiency Introduction to design and analysis algorithm 1.
Algorithmic Foundations COMP108 COMP108 Algorithmic Foundations Algorithm efficiency Prudence Wong
MA/CSSE 473 Day 05 Factors and Primes Recursive division algorithm.
Introduction to Algorithms: Divide-n-Conquer Algorithms
Introduction Algorithms Order Analysis of Algorithm
Chapter 4: Divide and Conquer
Great Theoretical Ideas in Computer Science
Applied Discrete Mathematics Week 4: Number Theory
Applied Symbolic Computation
Module #17: Recurrence Relations
Grade School Revisited: How To Multiply Two Numbers
Punya Biswas Lecture 15 Closest Pair, Multiplication
Applied Symbolic Computation
Topic: Divide and Conquer
Module #17: Recurrence Relations
Grade School Revisited: How To Multiply Two Numbers
Lecture 15, Winter 2019 Closest Pair, Multiplication
Topic: Divide and Conquer
Applied Symbolic Computation
Lecture 15 Closest Pair, Multiplication
Introduction to Discrete Mathematics
Presentation transcript:

Notes on the analysis of multiplication algorithms.. Dr. M. Sakalli, Marmara University

3-1 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Integer Multiplication MIT notes and wikipedia Example.. Classic High school math.. Let g = A|B and h = C|D where A,B,C and D are n/2 bit integers Simple Method: gh = (2 n/2 A+B)(2 n/2 C+D) same as given above. 4 multiplication routines. XY = (2 n )AC+2 n/2 (AD+BC) + BD and carriages c. Long multiplication: r j = c + Σ k = i-j g j h k Running Time Recurrence T(n) < 4T(n/2) + 100n, 100 multiplications.??, In-place??.. T(n) =  (n 2 ) Provided that neither c nor the total sum exceed log space, indeed, a simple inductive argument shows that the carry c and the total sum for r i can never exceed n and 2 n : <<?? 2lg n respectively. Space efficiency: S(n)=O(loglog(N)),  (loglog(N)). N= gh.

3-2 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Integer Multiplication MIT notes and wikipedia Pseudo code: Log space multiplication algorithm, multiply( g [0..n-1], h [0..n-1]) // Arrays representing to the binary representations x ← 0 for i= 0 : 2n-1 for j= 0 : i for j= 0 : i k ← i - j k ← i - j x ← x + ( g [j] × h [k]) x ← x + ( g [j] × h [k]) r[i] ← x mod 2 r[i] ← x mod 2 x ← floor(x/2) //I think this is carriage return. Last bit if 1.. x ← floor(x/2) //I think this is carriage return. Last bit if 1.. end endend Lattice method, Muhammad ibn Musa al-Khwarizmi. Gauss's complex multiplication algorithm. Muhammad ibn Musa al-KhwarizmiMuhammad ibn Musa al-Khwarizmi

3-3 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Integer Multiplication MIT notes and wikipedia Karatsuba’s algorithm: Polynomial extensions.. g = g 1 10 n/2 + g 2 h = h 1 10 n/2 + h 2 g h = g 1 h 1 10 n + (g 1 h 2 + g 2 h 1 )10 n/2 + g 2 h 2 (g 1 h 2 + g 2 h 1 ) = (g 1 + h 1 )(g 2 + h 2 ) - (g 2 h 2 + g 1 h 1 ), f(n) = 4sums+1 more final sum = 5n, n>2, suppose it is a constant 100n, and some carriages. XY = (2 n/2 +2 n )AC+2 n/2 (A-B)(C-D) + (2 n/2 +1) BD A(n) = 3A(n/2)+5n, A(n) < O(n lg 3 ) ≈  (n 1.6 ) Base value 7, when n<2,

3-4 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Karatsuba (g, h : n-digit integer; n : integer) // return (2n)-digit integer is // return (2n)-digit integer is a, b, c, d;// (n/2)-digit integer U, V, W; //n-digit integer; begin if n == 1 then if n == 1 then return  g(0)*h(0); ???? return  g(0)*h(0); ???? else else g1  g(n-1)... g(n/2); g1  g(n-1)... g(n/2); g2  g(n/2-1)... g(0); g2  g(n/2-1)... g(0); h1  h(n-1)... h(n/2); h1  h(n-1)... h(n/2); h2  h(n/2-1)... h(0); h2  h(n/2-1)... h(0); U  Karatsuba ( g1, h1, n/2 ); U  Karatsuba ( g1, h1, n/2 ); V  Karatsuba ( g2, h2, n/2 ); V  Karatsuba ( g2, h2, n/2 ); W  Karatsuba ( g1+g2, h1+h2, n/2 ); W  Karatsuba ( g1+g2, h1+h2, n/2 ); return  U*10 n + (W-U-V)*10^n/2 + V; return  U*10 n + (W-U-V)*10^n/2 + V; end if; end if; end Karatsuba; end Karatsuba; FFT and Fast Matrix multiplication.

3-5 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Quarter square multiplier 1980, Everett L. Johnson: gh = {( g + h ) 2 - ( g - h ) 2 }/4= {( g hg + h 2 ) - ( g hg + h 2 ) }/4 Think hardware implementation, with a lookup table (converter), the difficulty is that summation of the two numbers each 8bits, will require at least 9 bits, when squared, 18 bits wide.. But if divided by 2 before squared, (discarding remainder when n is odd). Table lookup from 0 to.. 9+9, from 0 … to 81. O(3n), working S(n) =  (n). i.e. 7 by 3, observe that the sum and difference are 10 and 4 respectively. Looking both of those values up on the table yields 25 and 4, the difference of which is 21.

3-6 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Russian (Egyptian) Peasant’s binary multiplication Shift and add.. In-place algorithm, may be implemented and 2n space.. Try complex examples. 11 3, in binary , , , = T(n) =  (n)+O(n 2 ), think about this?.. Why.. S(n) =  (loglog(n)) which is the carriage. If invertible? Division potential question.

3-7 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Matrix multiplication, 8 multiplications, O(n 3 ) Pseudo code for MM. MM(A, B) for i ← 1 : N for j ← 1 : N for j ← 1 : N C(i, j) ← 0; C(i, j) ← 0; for k ← 1 : N for k ← 1 : N C(i, j) ← C(i, j) + A(i, k) * B(k, j) C(i, j) ← C(i, j) + A(i, k) * B(k, j) end, end, end Time complexity of this algo is n 3 multiplications and additions. Can we do better using divide and conquer?.. Subdividing matrices into four sub-matrices. T(n) = b, n  2, T(n) = 8T(n/2) + c n 2, n>2, which has T(n) = O(n??)

3-8 M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes Strassen’s Algorithm Strassen: 7 multiplies, 18 additions T(n) = b, n  2, T(n) = 7T(n/2) + (7m+18s) n 2, n>2, which has T(n) = O(n 2.81 ) 7n 2 (1/4+1/16+…) Strassen-Winograd: 7 multiplies, 15 additions Coppersmith-Winograd, O(n ) (not easily implementable) In practice faster (not large hidden constants) for relatively smaller n~64, and stable but demonstrated that for some matrices (Strassen and Strassen-Winograd) are too unstable.