1. Numerical Computation1
Numerical Computation
Click on the computer image at the bottom right for a direct web link to an interesting Wikipedia Math Site.
Prepared by: Richard Mitchell Humber College

2. 1.1 - THE NUMBER TYPES
1.1-The Number Types

3. 1.1-PLACE VALUE Whole Numbers Decimal Numbers
213 - Nearest Whole Number
Nearest Hundreds
Nearest Tens
52 - Nearest Whole Number
Nearest Thousands
Decimal Numbers
DP’s/Nearest Thousandths
DP’s/Nearest Hundredths
DP’s/Nearest Thousandths
DP’s/Nearest Thousandths
DP’s/Nearest Hundredths
1.1-The Number Types

4. 1.1-EXACT NUMBERS Counted Quantities have no uncertainty.
Whole Numbers and Fractions have no uncertainty.
(Only when not measured and not in Decimal Form)
Defined Numbers have no uncertainty.
4 wheels (exactly counted) 501 roses (exactly counted)
17 letters (exactly counted) washers (exactly counted)
cars (exactly counted) 24 hours in a day (exactly counted)
1.1-The Number Types
1 inch = 25.4 mm (exactly measured by definition)

5. 1.1-APPROXIMATE NUMBERS
Measured Quantities have some degree of uncertainty.
(Last SD and/or Decimal Place is often estimated visually on a scale or meter).
Decimal Numbers have some degree of uncertainty.
(Both measured and non-measured)
Estimates have some degree of uncertainty.
Decimal Form of Fractions and Irrational Numbers.
217 m cm inches m/s
Hz lbs km mph
cm ft
1.1-The Number Types
approximately people about cars were built
2/3 (exact form) equals (approximate decimal form)
∏ (exact form) equals (approximate decimal form)
(exact form) equals (approximate decimal form)

6. 1.1-SIGNIFICANT DIGITS 0.152 - 3 SD’s 0.0177 - 3 SD’s
Whole Numbers
SD’s Ō SD’s
SD’s 31 Ō SD’s
SD’s SD’s
SD’s SD’s
SD’s 4 Ō SD’s
Decimal Numbers
SD’s SD’s
SD SD’s
SD’s SD’s
SD’s SD’s
1.1-The Number Types

7. 1.1-SIGNIFICANT DIGITS SUMMARY
1.1-The Number Types

8. 1.1-ACCURACY vs PRECISION
Accuracy is Determined by Significant Digits
SD’s of Accuracy
SD’s of Accuracy
SD’s of Accuracy
SD’s of Accuracy
SD’s of Accuracy
SD of Accuracy
Precision is Determined by Place Value
DP’s of Precision (Nearest Thousandths)
Nearest Hundred’s of Precision
DP’s of Precision (Nearest Ten Thousandths)
125 - Nearest Whole Number of Precision
DP’s of Precision (Nearest Hundredths)
DP’s of Precision (Nearest Thousandths)
1.1-The Number Types

9. 1.1-ROUNDING SUMMARY
Example 8
Example 9
1.1-The Number Types

10. 1.2 - NUMERICAL OPERATIONS

11. 1.2-RULE of PRECISION
1 Decimal Place
(Approximate)
3 Decimal Places
(Approximate)
USE: The Rule of PRECISION (1 Decimal Place) When adding or subtracting approximate numbers, keep as many decimal places in your answer as the number having the fewest decimal places in the question. Use the least precise place value if there are no decimal places in the question.
1.2-Numerical Operations

12. 1.2-EXAMPLE 13
(Approximate)
Nearest Hundreds
(Exact)
Nearest Ones
USE: The Rule of PRECISION (Nearest Hundreds Place) When adding or subtracting approximate numbers, keep as many decimal places in your answer as the number having the fewest decimal places in the question. Use the least precise place value if there are no decimal places in the question.
1.2-Numerical Operations
When using exact numbers, treat them as if they had more Significant Digits than any of the approximate numbers in the question.

13. 1.2-RULE of ACCURACY
5 Significant Digits
(Approximate)
3 Significant Digits
(Approximate)
USE: The Rule of ACCURACY (3 Significant Digits) When multiplying or dividing approximate numbers, keep as many significant digits in your answer as the number having the fewest significant digits in the question.
1.2-Numerical Operations

14. 1.2-RULE of ACCURACY
3 Significant Digits
(Approximate)
Do not count these as
Significant Digits.
USE: The Rule of ACCURACY (3 Significant Digits) When multiplying or dividing approximate numbers, keep as many significant digits in your answer as the number having the fewest significant digits in the question.
1.2-Numerical Operations
When using exact numbers, treat them as if they had more Significant Digits than any of the approximate numbers in the question.

15. 1.2-RULE of ACCURACY
4 Significant Digits
(Approximate)
3 Significant Digits
(Approximate)
USE: The Rule of ACCURACY (3 Significant Digits) When multiplying or dividing approximate numbers, keep as many significant digits in your answer as the number having the fewest significant digits in the question.
1.2-Numerical Operations

16. 1.2-RULE of ACCURACY
4 Significant Digits
(Approximate)
Do not count these as
Significant Digits.
USE: The Rule of ACCURACY (3 Significant Digits) When multiplying or dividing approximate numbers, keep as many significant digits in your answer as the number having the fewest significant digits in the question.
1.2-Numerical Operations
When using exact numbers, treat them as if they had more Significant Digits than any of the approximate numbers in the question.

17. 1.2-POWERS
Powers
1.2-Numerical Operations

18. 1.2-ROOTS
Roots
1.2-Numerical Operations

19. 1.3 - ORDER of Operations
1.3-Order of Operations

20. 1.3-EXAMPLE extra
1.3-Order of Operations
USE: The Rule of EXACT NUMBERS. Exact numbers produce ‘exact answers’ and do not need to be rounded off.

21. 1.3-EXAMPLE 37
1.3-Order of Operations
USE: The Rule of ACCURACY (3 Significant Digits). Be sure to keep all of the digits used in each calculation and only round off at the end.

22. 1.4 - Scientific and Engineering Notation

23. 1.4-EXAMPLES
Large Numbers
346 = 3.46 x 102 (3 SD’s)
2 700 = 2.7 x 103 (2 SD’s)
= x 106 (4 SD’s)
31Ō 000 = 3.10 x 105 (3 SD’s)
Small Numbers
= 9.31 x (3 SD’s)
= x (4 SD’s)
= 9.50 x (3 SD’s)
1.4-Scientific and Engineering Notation

24. 1.4-CALCULATOR SKILLS Normal Mode (30 000) x (215 000) = 6 450 000 000
(30 000) + ( ) =
( ) + ( ) =
Exponential Mode (EXP or EE or key)
(3.00 x 104) x (2.15 x 105) =
(3.00 x 104) + (2.15 x 105) =
Scientific Mode (Mode or FSE key)
(3.00 x 104) x (2.15 x 105) = 6.45 x 109
(3.00 x 104) + (2.15 x 105) = 2.45 x 105
USE: The Rule of EXACT NUMBERS when using exact numbers.
1.4-Scientific and Engineering Notation
USE: The Rule of ACCURACY when using approximate numbers (Significant Digits).

25. 1.4-EXAMPLES
RULE: Digits from the decimal part of the number are grouped into sets of three.
RULE: For numbers less than 1, separate the digits following the decimal point into groups of three.
1.4-Scientific and Engineering Notation

26. 1.4-SUMMARY Scientific Notation Engineering Notation
1.4-Scientific and Engineering Notation

27. 1.5 - Units of Measurement
1.5-Units of Measurement

28. 1.5-EXAMPLE 47 Conversion Factor: 1 ft. = 0.3048 m
OPTIONAL FORMULA
Multiply by the conversion factor that will cancel the units you wish to eliminate.
654.5 ft. x ( m / 1 ft.)
Conversion Factor: 1 ft. = m
1.5-Units of Measurement

29. 1.5-EXAMPLE 48 Conversion Factor: 1 ha = 2.471 acres
1.5-Units of Measurement

30. 1.5-EXAMPLE 51 Conversion Factor: 1 km = 1000 m
1.5-Units of Measurement

31. 1.5-EXAMPLE 52 Conversion Factor: 1 N = 1.0 x 105 dynes
1.5-Units of Measurement

32. 1.5-EXAMPLE 53 Conversion Factor: 1 cu.ft. = 7.481 gal.(U.S.)
1.5-Units of Measurement

33. 1.6 - Substituting into Equations and Formulas

34. 1.6-EXAMPLE 57
1.6-Substituting into Equations and Formulas

35. 1.6-EXAMPLE 58 Convert all units to pounds and inches.
1.6-Substituting into Equations and Formulas

36. 1.7 - Percentage
1.7-Percentage

37. 1.7-FRACTIONS-DECIMALS-PERCENTS
1.7-Percentage

38. 1.7-EXAMPLES
1.7-Percentage

39. 1.7-STRATEGY A
1.7-Percentage

40. 1.7-STRATEGY B RATE (%) (Change) PART (Change) FORMULA BASE (Original)
1.7-Percentage

41. 1.7-EXAMPLE 59
PART (Change)
1.7-Percentage

42. 1.7-EXAMPLE 60
PART (Change)
1.7-Percentage

43. 1.7-EXAMPLE 61
BASE (Original)
1.7-Percentage

44. 1.7-EXAMPLE 62
BASE (Original)
1.7-Percentage

45. 1.7-EXAMPLE 63
RATE (%)
1.7-Percentage

46. 1.7-EXAMPLE 64
RATE (%)
1.7-Percentage

47. 1.7-STRATEGY Type I: Percent Change Type II: Percent Efficiency
Type III: Percent Error
1.7-Percentage
Type IV: Percent Concentration

48. 1.7-EXAMPLE 65
Type I: Percent Change
1.7-Percentage

49. 1.7-EXAMPLE 66
Type I: Percent Change
1.7-Percentage

50. 1.7-EXAMPLE 67
Type II: Percent Efficiency
1.7-Percentage

51. 1.7-EXAMPLE 68
Type III: Percent Error
1.7-Percentage

52. 1.7-EXAMPLE 69
Type IV: Percent Concentration
1.7-Percentage

53. Copyright