1. ME 142 Engineering Computation I
Computer Precision
&
Round-Off Error

2. Numerical Error Two major forms of numerical error: Round-off Error
Truncation error
Round-off Error
Is due to the fact that computers can represent only quantities with a finite number of digits
Truncation Error
Is introduced because numerical methods employ approximations to represent exact mathematical operations and quantities

3. Significant Figures
Significant digits of a number are those that can be used with confidence
Typically, 3 to 4 significant digits is common in the final answer of engineering calculations
Keep all available significant figures on intermediate calculations
Round final answer to 3 or 4 significant digits
Significant Figures in Excel
By default Excel keeps all available significant figures and displays a round answer, per user setting

4. Accuracy vs. Precision Two terms commonly used within engineering
Both are important and necessary
Accuracy refers to how closely a computed or measured value agrees with the true value.
Precision refers to how closely individual computed or measured values agree with each other.

5. Increasing Accuracy Increasing Precision Inaccurate & Imprecise
Inaccurate & Precise
Accurate & Precise

6. Computer Representation of Numbers
How do computers represent numbers?
Binary digits or bits or base 2
Groupings of bits into bytes and words
How do we represent numbers?
Decimal system or base 10
Need to understand how to convert between these number systems

7. Base 10 1 2 4 5
5 x 100=
0 x 101=
4 x 102=
2 x 103=
1 x 104=
12405

8. Base 2 1 1 1 1
1 x 20= =
0 x 21= =
1 x 22= =
1 x 23= =
1 x 24=
= 16
11101 29