Complexity 20-1 Complexity Andrei Bulatov Parallel Arithmetic.

Slides:



Advertisements
Similar presentations
CS 1031 Recursion (With applications to Searching and Sorting) Definition of a Recursion Simple Examples of Recursion Conditions for Recursion to Work.
Advertisements

Lecture 3: Parallel Algorithm Design
Recursive Definitions and Structural Induction
Grade School Again: A Parallel Perspective Great Theoretical Ideas In Computer Science Steven RudichCS Spring 2004 Lecture 17Mar 16, 2004Carnegie.
Computability and Complexity
Prune-and-Search Method
Lectures on Recursive Algorithms1 COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski.
1 Potential for Parallel Computation Module 2. 2 Potential for Parallelism Much trivially parallel computing  Independent data, accounts  Nothing to.
April 9, 2015Applied Discrete Mathematics Week 9: Relations 1 Solving Recurrence Relations Another Example: Give an explicit formula for the Fibonacci.
Advanced Topics in Algorithms and Data Structures Lecture pg 1 Recursion.
Computability and Complexity 20-1 Computability and Complexity Andrei Bulatov Random Sources.
1 Maximal Independent Set. 2 Independent Set (IS): In a graph, any set of nodes that are not adjacent.
Discrete Structures & Algorithms More about sets EECE 320 — UBC.
Complexity 11-1 Complexity Andrei Bulatov Space Complexity.
EE 382 Processor DesignWinter 98/99Michael Flynn 1 AT Arithmetic Most concern has gone into creating fast implementation of (especially) FP Arith. Under.
Divide-and-Conquer1 7 2  9 4   2  2 79  4   72  29  94  4.
VLSI Arithmetic. Multiplication A = a n-1 a n-2 … a 1 a 0 B = b n-1 b n-2 … b 1 b 0  eg)  Shift.
Computability and Complexity 32-1 Computability and Complexity Andrei Bulatov Boolean Circuits.
Algorithm Design Techniques: Induction Chapter 5 (Except Section 5.6)
1 Lecture 6 More PRAM Algorithm Parallel Computing Fall 2008.
Solving the Greatest Common Divisor Problem in Parallel Derrick Coetzee University of California, Berkeley CS 273, Fall 2010, Prof. Satish Rao.
The Euler-tour technique
Complexity 19-1 Parallel Computation Complexity Andrei Bulatov.
CSC 2300 Data Structures & Algorithms January 30, 2007 Chapter 2. Algorithm Analysis.
1 Section 6.1 Recurrence Relations. 2 Recursive definition of a sequence Specify one or more initial terms Specify rule for obtaining subsequent terms.
Fundamental Techniques
Great Theoretical Ideas in Computer Science.
Fall 2002CMSC Discrete Structures1 Let us get into… Number Theory.
Arithmetic Sequences A sequence in which each term after the first is obtained by adding a fixed number to the previous term is an arithmetic sequence.
Applied Discrete Mathematics Week 9: Relations
1 Arithmetic and Logical Operations - Part II. Unsigned Numbers Addition in unsigned numbers is the same regardless of the base. Given a pair of bit sequences.
Mathematics of Cryptography Part I: Modular Arithmetic
Liang, Introduction to Java Programming, Seventh Edition, (c) 2009 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
1 Appendix E: Sigma Notation. 2 Definition: Sequence A sequence is a function a(n) (written a n ) who’s domain is the set of natural numbers {1, 2, 3,
Chapter 3: The Fundamentals: Algorithms, the Integers, and Matrices
Chapter 6-2 Multiplier Multiplier Next Lecture Divider
Computer Science and Engineering Parallel and Distributed Processing CSE 8380 February 8, 2005 Session 8.
Mathematics Review Exponents Logarithms Series Modular arithmetic Proofs.
1 Problem Solving using computers Data.. Representation & storage Representation of Numeric data The Binary System.
Analysis of Algorithms
Chapter 3 Sec 3.3 With Question/Answer Animations 1.
Multiplication of signed-operands
1 Chapter 7 Computer Arithmetic Smruti Ranjan Sarangi Computer Organisation and Architecture PowerPoint Slides PROPRIETARY MATERIAL. © 2014 The McGraw-Hill.
New Mexico Computer Science for All Author: Ed Angel Title: Bits and Bytes.
Data Structure and Algorithms. Algorithms: efficiency and complexity Recursion Reading Algorithms.
Introduction To Number Systems Binary System M. AL-Towaileb1.
Greatest Common Divisors & Least Common Multiples  Definition 4 Let a and b be integers, not both zero. The largest integer d such that d|a and d|b is.
Lecture 9COMPSCI.220.FS.T Lower Bound for Sorting Complexity Each algorithm that sorts by comparing only pairs of elements must use at least 
1 How to Multiply Slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved. integers, matrices, and polynomials.
CompSci 102 Discrete Math for Computer Science March 13, 2012 Prof. Rodger Slides modified from Rosen.
4.2A Arithmetic Explicit and Recursive Sequences
Remarks on Fast Exp (4/2) How do we measure how fast any algorithm is? Definition. The complexity of an algorithm is a measure of the approximate number.
Grade School Again: A Parallel Perspective CS Lecture 7.
Number Theory Lecture 1 Text book: Discrete Mathematics and its Applications, 7 th Edition.
Unary, Binary, and Beyond Great Theoretical Ideas In Computer Science Steven RudichCS Spring 2003 Lecture 2Jan 16, 2003Carnegie Mellon University.
Number Theory. Introduction to Number Theory Number theory is about integers and their properties. We will start with the basic principles of divisibility,
PERFORMANCE EVALUATIONS
Lecture 3: Parallel Algorithm Design
Probabilistic Algorithms
Modeling with Recurrence Relations
Partial Products Algorithm for Multiplication
Exercise: Add these two single precision IEEE 754 numbers: … …0 Left number: 1.101x24 Right number: 1.011x 22= x24.
Algorithms with numbers (1) CISC4080, Computer Algorithms
Unsigned Multiplication
Modular Arithmetic and Change of Base
Divide-and-Conquer 7 2  9 4   2   4   7
Numerical Algorithms Quiz questions
Applied Discrete Mathematics Week 7: Computation
Divide-and-Conquer 7 2  9 4   2   4   7
Presentation transcript:

Complexity 20-1 Complexity Andrei Bulatov Parallel Arithmetic

Complexity 20-2 Arithmetic Operations A sequential algorithm can compute the sum of two integers, n and m, in O(max{log n, log m}) the product of two integers, n and m, in O(log n  log m) Our goal is to find parallel algorithms that compute the sum of two integers, n and m, in O(max{log log n, log log m}) the product of two integers, n and m, in O(log log n  log log m)

Complexity 20-3 Prefix Sums Instance: A sequence of positive integers Objective: Compute the prefix sums: Prefix Sum The straightforward algorithm requires n – 1 additions Unfortunately, it cannot be parallelized

Complexity 20-4 Parallel Prefix Sums Our algorithm is recursive Suppose that n is a power of 2 (otherwise add some 0 s to the sequence) Given a sequence of numbers compute recursively compute the prefix sums for (after this step we have half of the prefix sums for ) compute the remaining sums by adding to

Complexity 20-5 Analysis The depth of recursion for n numbers is log n On each iteration the algorithm performs additions in one step and then addition in another step Thus, the time complexity of the algorithm is 2·log n and the total amount of work is Therefore, by Brent’s principle, the number of processors needed is

Complexity 20-6 Associative Operations Note that this algorithm can be used not only for addition, but also for other operations like multiplication and others The only property of addition we used is associativity Definition An operation  is said to be associative if, for any a, b and c (a  (b  c)) = ((a  b)  c) Definition An operation  is said to be associative if, for any a, b and c (a  (b  c)) = ((a  b)  c)

Complexity 20-7 Addition We compute the sum of two binary numbers carry 1 carry 1 carry 1 carry 0 carry 0 … This algorithm requires a time O(log n) and cannot be parallelized in an obvious way

Complexity 20-8 Fast Addition Let and be binary representations of two integers such that Let be their sum and let the carry out of the i th position If we know the carries, we can compute the sum in two parallel steps:

Complexity 20-9 Computing Carries The carry is 1 if and only if (a) or (b) at least one of them is 1 and the previous carry is 1 If we define and then We also assume that

Complexity We define a binary operation on pairs of bits: Now, denoting we get Then That is where

Complexity Claim. The operation  is associative

Complexity Finally, we define Then

Complexity Analysis The carries can be computed as prefix sums in a parallel time log log n Then we complete the computation in 2 more steps The total amount of work is O(log n) and the number of processors needed

Complexity Fast Multiplication To multiply two numbers and in binary representation, we need to compute the following sum This at most n additions can be performed in parallel by grouping summands in pairs. The depth of such a binary tree is Therefore two numbers can be multiplied in a parallel time