Presentation is loading. Please wait.

Presentation is loading. Please wait.

© David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 1 VSCSE Summer School 2009 Many-core Processors for Science and Engineering.

Published byCody Ward Modified over 8 years ago

Similar presentations

Presentation on theme: "© David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 1 VSCSE Summer School 2009 Many-core Processors for Science and Engineering."— Presentation transcript:

1 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 1 VSCSE Summer School 2009 Many-core Processors for Science and Engineering Applications Lecture 6 Floating-Point Considerations and CUDA for Multi-core

2 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 2 Objective To understand the fundamentals of floating-point representation To know the IEEE-754 Floating Point Standard GeForce 8800 CUDA Floating-point speed, accuracy and precision –Deviations from IEEE-754 –Accuracy of device runtime functions –-fastmath compiler option –Future performance considerations To understand CUDA on Multi-cores

3 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 3 GPU Floating Point Features G80SSEIBM AltivecCell SPE PrecisionIEEE 754 Rounding modes for FADD and FMUL Round to nearest and round to zero All 4 IEEE, round to nearest, zero, inf, -inf Round to nearest only Round to zero/truncate only Denormal handlingFlush to zero Supported, 1000’s of cycles Flush to zero NaN supportYes No Overflow and Infinity support Yes, only clamps to max norm Yes No, infinity FlagsNoYes Some Square rootSoftware onlyHardwareSoftware only DivisionSoftware onlyHardwareSoftware only Reciprocal estimate accuracy 24 bit12 bit Reciprocal sqrt estimate accuracy 23 bit12 bit log2(x) and 2^x estimates accuracy 23 bitNo12 bitNo

4 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 4 What is IEEE floating-point format? A floating point binary number consists of three parts: –sign (S), exponent (E), and mantissa (M). –Each (S, E, M) pattern uniquely identifies a floating point number. For each bit pattern, its IEEE floating-point value is derived as: –value = (-1) S * M * {2 E }, where 1.0 ≤ M < 10.0 B The interpretation of S is simple: S=0 results in a positive number and S=1 a negative number.

5 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 5 Normalized Representation Specifying that 1.0 B ≤ M < 10.0 B makes the mantissa value for each floating point number unique. –For example, the only one mantissa value allowed for 0.5 D is M =1.0 0.5 D = 1.0 B * 2 -1 –Neither 10.0 B * 2 -2 nor 0.1 B * 2 0 qualifies Because all mantissa values are of the form 1.XX…, one can omit the “1.” part in the representation. –The mantissa value of 0.5 D in a 2-bit mantissa is 00, which is derived by omitting “1.” from 1.00.

6 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 6 Exponent Representation In an n-bits exponent representation, 2 n-1 -1 is added to its 2's complement representation to form its excess representation. –See Table for a 3-bit exponent representation A simple unsigned integer comparator can be used to compare the magnitude of two FP numbers Symmetric range for +/- exponents (111 reserved) 2’s complementActual decimalExcess-3 0000011 0011100 0102101 0113110 100(reserved pattern) 111 101-3000 110-2001 111010

7 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 7 A simple, hypothetical 5-bit FP format Assume 1-bit S, 2-bit E, and 2-bit M –0.5 D = 1.00 B * 2 -1 –0.5 D = 0 00 00, where S = 0, E = 00, and M = (1.)00 2’s complementActual decimalExcess-1 00001 110 (reserved pattern) 11 00

8 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 8 Representable Numbers The representable numbers of a given format is the set of all numbers that can be exactly represented in the format. See Table for representable numbers of an unsigned 3- bit integer format 0000 0011 0102 0113 1004 1015 1106 1117 07142356 98

9 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 9 Representable Numbers of a 5-bit Hypothetical IEEE Format 0 No-zero Abrupt underflowGradual underflow EM S=0S=1 S=0S=1S=0S=1 00 2 -1 -(2 -1 ) 0000 01 2 -1 +1*2 -3 -(2 -1 +1*2 -3 ) 001*2 -2 -1*2 -2 10 2 -1 +2*2 -3 -(2 -1 +2*2 -3 ) 002*2 -2 -2*2 -2 11 2 -1 +3*2 -3 -(2 -1 +3*2 -3 ) 003*2 -2 -3*2 -2 0100 2020 -(2 0 ) 2020 2020 01 2 0 +1*2 -2 -(2 0 +1*2 -2 ) 2 0 +1*2 -2 -(2 0 +1*2 -2 )2 0 +1*2 -2 -(2 0 +1*2 -2 ) 10 2 0 +2*2 -2 -(2 0 +2*2 -2 ) 2 0 +2*2 -2 -(2 0 +2*2 -2 )2 0 +2*2 -2 -(2 0 +2*2 -2 ) 11 2 0 +3*2 -2 -(2 0 +3*2 -2 ) 2 0 +3*2 -2 -(2 0 +3*2 -2 )2 0 +3*2 -2 -(2 0 +3*2 -2 ) 1000 2121 -(2 1 ) 2121 2121 01 2 1 +1*2 -1 -(2 1 +1*2 -1 ) 2 1 +1*2 -1 -(2 1 +1*2 -1 )2 1 +1*2 -1 -(2 1 +1*2 -1 ) 10 2 1 +2*2 -1 -(2 1 +2*2 -1 ) 2 1 +2*2 -1 -(2 1 +2*2 -1 )2 1 +2*2 -1 -(2 1 +2*2 -1 ) 11 2 1 +3*2 -1 -(2 1 +3*2 -1 ) 2 1 +3*2 -1 -(2 1 +3*2 -1 )2 1 +3*2 -1 -(2 1 +3*2 -1 ) 11Reserved pattern Cannot represent Zero!

10 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 10 Flush to Zero Treat all bit patterns with E=0 as 0.0 –This takes away several representable numbers near zero and lump them all into 0.0 –For a representation with large M, a large number of representable numbers numbers will be removed. 0 1234 0

11 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 11 Flush to Zero 0 No-zero Flush to Zero Denormalized EMS=0S=1 S=0S=1 S=0S=1 00 2 -1 -(2 -1 ) 00 00 012 -1 +1*2 -3 -(2 -1 +1*2 -3 ) 00 1*2 -2 -1*2 -2 102 -1 +2*2 -3 -(2 -1 +2*2 -3 ) 00 2*2 -2 -2*2 -2 112 -1 +3*2 -3 -(2 -1 +3*2 -3 ) 00 3*2 -2 -3*2 -2 01002020 -(2 0 ) 2020 2020 012 0 +1*2 -2 -(2 0 +1*2 -2 ) 2 0 +1*2 -2 -(2 0 +1*2 -2 ) 2 0 +1*2 -2 -(2 0 +1*2 -2 ) 102 0 +2*2 -2 -(2 0 +2*2 -2 ) 2 0 +2*2 -2 -(2 0 +2*2 -2 ) 2 0 +2*2 -2 -(2 0 +2*2 -2 ) 112 0 +3*2 -2 -(2 0 +3*2 -2 ) 2 0 +3*2 -2 -(2 0 +3*2 -2 ) 2 0 +3*2 -2 -(2 0 +3*2 -2 ) 10002121 -(2 1 ) 2121 2121 012 1 +1*2 -1 -(2 1 +1*2 -1 ) 2 1 +1*2 -1 -(2 1 +1*2 -1 ) 2 1 +1*2 -1 -(2 1 +1*2 -1 ) 102 1 +2*2 -1 -(2 1 +2*2 -1 ) 2 1 +2*2 -1 -(2 1 +2*2 -1 ) 2 1 +2*2 -1 -(2 1 +2*2 -1 ) 112 1 +3*2 -1 -(2 1 +3*2 -1 ) 2 1 +3*2 -1 -(2 1 +3*2 -1 ) 2 1 +3*2 -1 -(2 1 +3*2 -1 ) 11Reserved pattern

12 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 12 Denormalized Numbers The actual method adopted by the IEEE standard is called denromalized numbers or gradual underflow. –The method relaxes the normalization requirement for numbers very close to 0. –whenever E=0, the mantissa is no longer assumed to be of the form 1.XX. Rather, it is assumed to be 0.XX. In general, if the n-bit exponent is 0, the value is 0.M * 2 - 2 ^(n-1) + 2 0 1 2 3

13 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 13 Denormalization 0 No-zeroFlush to Zero Denormalized EMS=0S=1S=0S=1 S=0S=1 00 2 -1 -(2 -1 )00 00 012 -1 +1*2 -3 -(2 -1 +1*2 -3 )00 1*2 -2 -1*2 -2 102 -1 +2*2 -3 -(2 -1 +2*2 -3 )00 2*2 -2 -2*2 -2 112 -1 +3*2 -3 -(2 -1 +3*2 -3 )00 3*2 -2 -3*2 -2 01002020 -(2 0 )2020 2020 012 0 +1*2 -2 -(2 0 +1*2 -2 )2 0 +1*2 -2 -(2 0 +1*2 -2 ) 2 0 +1*2 -2 -(2 0 +1*2 -2 ) 102 0 +2*2 -2 -(2 0 +2*2 -2 )2 0 +2*2 -2 -(2 0 +2*2 -2 ) 2 0 +2*2 -2 -(2 0 +2*2 -2 ) 112 0 +3*2 -2 -(2 0 +3*2 -2 )2 0 +3*2 -2 -(2 0 +3*2 -2 ) 2 0 +3*2 -2 -(2 0 +3*2 -2 ) 10002121 -(2 1 )2121 2121 012 1 +1*2 -1 -(2 1 +1*2 -1 )2 1 +1*2 -1 -(2 1 +1*2 -1 ) 2 1 +1*2 -1 -(2 1 +1*2 -1 ) 102 1 +2*2 -1 -(2 1 +2*2 -1 )2 1 +2*2 -1 -(2 1 +2*2 -1 ) 2 1 +2*2 -1 -(2 1 +2*2 -1 ) 112 1 +3*2 -1 -(2 1 +3*2 -1 )2 1 +3*2 -1 -(2 1 +3*2 -1 ) 2 1 +3*2 -1 -(2 1 +3*2 -1 ) 11Reserved pattern

14 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 14 Arithmetic Instruction Throughput int and float add, shift, min, max and float mul, mad: 4 cycles per warp –int multiply (*) is by default 32-bit requires multiple cycles / warp –Use __mul24() / __umul24() intrinsics for 4-cycle 24-bit int multiply Integer divide and modulo are expensive –Compiler will convert literal power-of-2 divides to shifts –Be explicit in cases where compiler can’t tell that divisor is a power of 2! –Useful trick: foo % n == foo & (n-1) if n is a power of 2

15 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 15 Arithmetic Instruction Throughput Reciprocal, reciprocal square root, sin/cos, log, exp: 16 cycles per warp –These are the versions prefixed with “__” –Examples:__rcp(), __sin(), __exp() Other functions are combinations of the above –y / x == rcp(x) * y == 20 cycles per warp –sqrt(x) == rcp(rsqrt(x)) == 32 cycles per warp

16 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 16 Runtime Math Library There are two types of runtime math operations –__func(): direct mapping to hardware ISA Fast but low accuracy (see prog. guide for details) Examples: __sin(x), __exp(x), __pow(x,y) –func() : compile to multiple instructions Slower but higher accuracy (5 ulp, units in the least place, or less) Examples: sin(x), exp(x), pow(x,y) The -use_fast_math compiler option forces every func() to compile to __func()

17 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 17 Make your program float-safe! G280 and beyond have double precision support –Double precision will have additional performance cost –Careless use of double or undeclared types may run more slowly on G280 and beyond Important to be float-safe (be explicit whenever you want single precision) to avoid using double precision where it is not needed –Add ‘f’ specifier on float literals: foo = bar * 0.123; // double assumed foo = bar * 0.123f; // float explicit –Use float version of standard library functions foo = sin(bar); // double assumed foo = sinf(bar); // single precision explicit

18 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 18 Deviations from IEEE-754 Addition and Multiplication are IEEE 754 compliant –Maximum 0.5 ulp (units in the least place) error However, often combined into multiply-add (FMAD) –Intermediate result is truncated Division is non-compliant (2 ulp) Not all rounding modes are supported Denormalized numbers are supported in G280 and beyond No mechanism to detect floating-point exceptions

19 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 19 GPU Floating Point Features G80SSEIBM AltivecCell SPE PrecisionIEEE 754 Rounding modes for FADD and FMUL Round to nearest and round to zero All 4 IEEE, round to nearest, zero, inf, -inf Round to nearest only Round to zero/truncate only Denormal handlingFlush to zero Supported, 1000’s of cycles Flush to zero NaN supportYes No Overflow and Infinity support Yes, only clamps to max norm Yes No, infinity FlagsNoYes Some Square rootSoftware onlyHardwareSoftware only DivisionSoftware onlyHardwareSoftware only Reciprocal estimate accuracy 24 bit12 bit Reciprocal sqrt estimate accuracy 23 bit12 bit log2(x) and 2^x estimates accuracy 23 bitNo12 bitNo

20 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 20 CUDA Kernels on Multi-core CPU Most application developers do not want to write special code for each architecture – CUDA kernels are currently special code for GPUs –MCUDA allows efficient execution of CUDA kernels on multi-core CPUs A single GPU thread is too small for a CPU Thread –CUDA emulation does this and performs poorly CPU cores designed for ILP, SIMD –Optimizing compilers work well with iterative loops Turn GPU thread blocks from CUDA into iterative CPU loops

21 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 21 Key Issue: Synchronization Suspend and Wakeup –Move on to other threads –Begin again after all hit barrier Matrixmul(A[ ], B[ ], C[ ]) { __shared__ Asub[ ][ ], Bsub[ ][ ]; int a,b,c; float Csub; int k; … for(…) { Asub[tx][ty] = A[a]; Bsub[tx][ty] = B[b]; __syncthreads(); for( k = 0; k < blockDim.x; k++ ) Csub += Asub[ty][k] + Bsub[k][tx]; __syncthreads(); } … }

22 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 22 Synchronization solution Loop fission –Break the loop into multiple smaller loops according to syncthreads() –Not always possible with __synchthreads() in control flow Matrixmul(A[ ], B[ ], C[ ]) { __shared__ Asub[ ][ ], Bsub[ ][ ]; int a,b,c; float Csub; int k; … for(…) { for(ty=0; ty < blockDim.y; ty++) for(tx=0; tx < blockDim.x; tx++) { Asub[tx][ty] = A[a]; Bsub[tx][ty] = B[b]; } for(ty=0; ty < blockDim.y; ty++) for(tx=0; tx < blockDim.x; tx++) { for( k = 0; k < blockDim.x; k++ ) Csub += Asub[ty][k] + Bsub[k][tx]; } … }

23 © David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 23 Bigger Picture Performance Results Consistent speed-up over hand-tuned single-thread code Best optimizations for GPU and CPU not always the same *Over hand-optimized CPU **Intel MKL, multi-core execution

Download ppt "© David Kirk/NVIDIA and Wen-mei W. Hwu Urbana, Illinois, August 10-14, 2009 1 VSCSE Summer School 2009 Many-core Processors for Science and Engineering."

Similar presentations

1/1/ / faculty of Electrical Engineering eindhoven university of technology Introduction Part 2: Data types and addressing modes dr.ir. A.C. Verschueren.

1/1/ / faculty of Electrical Engineering eindhoven university of technology Introduction Part 2: Data types and addressing modes dr.ir. A.C. Verschueren.
L9: Floating Point Issues CS6963. Outline Finish control flow and predicated execution discussion Floating point – Mostly single precision until recent.

L9: Floating Point Issues CS6963. Outline Finish control flow and predicated execution discussion Floating point – Mostly single precision until recent.
Fixed Point Numbers The binary integer arithmetic you are used to is known by the more general term of Fixed Point arithmetic. Fixed Point means that we.

Fixed Point Numbers The binary integer arithmetic you are used to is known by the more general term of Fixed Point arithmetic. Fixed Point means that we.
Fabián E. Bustamante, Spring 2007 Floating point Today IEEE Floating Point Standard Rounding Floating Point Operations Mathematical properties Next time.

Fabián E. Bustamante, Spring 2007 Floating point Today IEEE Floating Point Standard Rounding Floating Point Operations Mathematical properties Next time.
Computer Engineering FloatingPoint page 1 Floating Point Number system corresponding to the decimal notation 1,837 * 10 significand exponent A great number.

Computer Engineering FloatingPoint page 1 Floating Point Number system corresponding to the decimal notation 1,837 * 10 significand exponent A great number.
CENG536 Computer Engineering department Çankaya University.

CENG536 Computer Engineering department Çankaya University.
© David Kirk/NVIDIA and Wen-mei W. Hwu, 2007 ECE 498AL, University of Illinois, Urbana-Champaign 1 ECE 498AL Lecture 11: Floating-Point Considerations.

© David Kirk/NVIDIA and Wen-mei W. Hwu, 2007 ECE 498AL, University of Illinois, Urbana-Champaign 1 ECE 498AL Lecture 11: Floating-Point Considerations.
Topics covered: Floating point arithmetic CSE243: Introduction to Computer Architecture and Hardware/Software Interface.

Topics covered: Floating point arithmetic CSE243: Introduction to Computer Architecture and Hardware/Software Interface.
Princess Sumaya Univ. Computer Engineering Dept. Chapter 3:

Princess Sumaya Univ. Computer Engineering Dept. Chapter 3:
Princess Sumaya Univ. Computer Engineering Dept. Chapter 3: IT Students.

Princess Sumaya Univ. Computer Engineering Dept. Chapter 3: IT Students.
Datorteknik FloatingPoint bild 1 Floating point Number system corresponding to the decimal notation 1,837 * 10 significand exponent a great number of corresponding.

Datorteknik FloatingPoint bild 1 Floating point Number system corresponding to the decimal notation 1,837 * 10 significand exponent a great number of corresponding.
Floating Point Numbers

Floating Point Numbers
University of Washington Today Topics: Floating Point Background: Fractional binary numbers IEEE floating point standard: Definition Example and properties.

University of Washington Today Topics: Floating Point Background: Fractional binary numbers IEEE floating point standard: Definition Example and properties.
L10: Floating Point Issues and Project CS6963. Administrative Issues Project proposals – Due 5PM, Friday, March 13 (hard deadline) Homework (Lab 2) –

L10: Floating Point Issues and Project CS6963. Administrative Issues Project proposals – Due 5PM, Friday, March 13 (hard deadline) Homework (Lab 2) –
L9: Next Assignment, Project and Floating Point Issues CS6963.

L9: Next Assignment, Project and Floating Point Issues CS6963.
CSE 378 Floating-point1 How to represent real numbers In decimal scientific notation –sign –fraction –base (i.e., 10) to some power Most of the time, usual.

CSE 378 Floating-point1 How to represent real numbers In decimal scientific notation –sign –fraction –base (i.e., 10) to some power Most of the time, usual.
Floating Point Numbers

Floating Point Numbers
COMPUTER ARCHITECTURE & OPERATIONS I Instructor: Hao Ji.

COMPUTER ARCHITECTURE & OPERATIONS I Instructor: Hao Ji.
Information Representation (Level ISA3) Floating point numbers.

Information Representation (Level ISA3) Floating point numbers.
© David Kirk/NVIDIA and Wen-mei W. Hwu, 2007-2012 University of Illinois, Urbana-Champaign 1 CSCE 4643 GPU Programming Lecture 8 - Floating Point Considerations.

© David Kirk/NVIDIA and Wen-mei W. Hwu, University of Illinois, Urbana-Champaign 1 CSCE 4643 GPU Programming Lecture 8 - Floating Point Considerations.

Similar presentations

Ads by Google