1. Reliability and Channel Coding
Chapter 8
Reliability and Channel Coding
© Bobby Hoggard, Department of Computer Science, East Carolina University
These slides may not be used or duplicated without permission

2. Hamming Algorithm
An error-correcting algorithm with low overhead, that can correct one single-bit error
Data Bits
Check Bits
Overhead 1 2
200%
2 - 4 3 %
5 - 11 4 % 5 % 6 % 7
12 - 6% 8
7 - 3%

3. Hamming Algorithm Overlapping Bits
Data bits are protected by overlapping patterns of check bits
Example - encode the 4-bit data value: 1100
Each circle represents a check bit (A, B, C)
Data bits are places in positions where the circles overlap (thus, being protected by multiple check bits)
The value of the check bit is set so that the total number of 1's in each circle is even or odd – depending on the parity chosen. (this example uses even parity) C A 1 1 1 B

4. Hamming Algorithm Verification
The receiver must verify the correct parity of each circle (this example uses even parity)
If a circle has the wrong parity, then an error has occurred
Parity of circle A =
Parity of circle B =
Parity of circle C =
ODD
EVEN C A 1 1 1
the problem bit is located where the wrong parity circles overlap 1
To correct the problem:
Change the bit to the opposite value
Remove the parity bits B
Data: 1100

5. What If Parity Bit Is Bad?
If a parity bit is bad, then there will be only one circle with the wrong parity
since the parity bits are found in the non-overlapping areas of the circle
Parity of circle A =
Parity of circle B =
Parity of circle C =
EVEN
ODD C A 1 1
the problem bit is located only in circle B 1 1 B

6. What If More Than One Bit Is Bad?
The algorithm is designed to correct only a single bit error in a single block of data
Break the data into multiple blocks if more protection is needed
Parity of circle A =
Parity of circle B =
Parity of circle C =
ODD C A 1
the algorithm indicates that the problem bit is located where all three circles overlap
FAIL! B

7. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #1: Number the bits from left to right, starting with 1

8. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #2: Insert place holders for parity bits; they go in position #s which are powers of 2

9. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #2: Insert place holders for parity bits; they go in position #s which are powers of 2

10. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #2: Insert place holders for parity bits; they go in position #s which are powers of 2

11. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #2: Insert place holders for parity bits; they go in position #s which are powers of 2

12. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #2: Insert place holders for parity bits; they go in position #s which are powers of 2

13. Encoding Example - - 1 - 0 1 1 - 1 0 0 0 1 0 1 - 1 1 1 1 0
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #2: Insert place holders for parity bits; they go in position #s which are powers of 2

14. Encoding Example - - 1 - 0 1 1 - 1 0 0 0 1 0 1 - 1 1 1 1 0
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)

15. Encoding Example - - 1 - 0 1 1 - 1 0 0 0 1 0 1 - 1 1 1 1 0
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 1 belongs to this group:
# of 1's in the group:
1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21 7
Parity bit should be: 1

16. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 1 belongs to this group:
# of 1's in the group:
1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21 7
Parity bit should be: 1

17. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 2 belongs to this group:
# of 1's in the group:
2, 3, 6, 7, 10, 11, 14, 15, 18, 19 6
Parity bit should be:

18. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 2 belongs to this group:
# of 1's in the group:
2, 3, 6, 7, 10, 11, 14, 15, 18, 19 6
Parity bit should be:

19. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 4 belongs to this group:
# of 1's in the group:
4, 5, 6, 7, 12, 13, 14, 15, 20, 21 5
Parity bit should be: 1

20. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 4 belongs to this group:
# of 1's in the group:
4, 5, 6, 7, 12, 13, 14, 15, 20, 21 5
Parity bit should be: 1

21. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 8 belongs to this group:
# of 1's in the group:
8, 9, 10, 11, 12, 13, 14, 15 3
Parity bit should be: 1

22. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 8 belongs to this group:
# of 1's in the group:
8, 9, 10, 11, 12, 13, 14, 15 3
Parity bit should be: 1

23. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 16 belongs to this group:
# of 1's in the group:
16, 17, 18, 19, 20, 21 4
Parity bit should be:

24. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Step #3: Set the values of the parity bits so that each group's parity is even (or odd)
Parity bit 16 belongs to this group:
# of 1's in the group:
16, 17, 18, 19, 20, 21 4
Parity bit should be:

25. Encoding Example
Encode the data:
16-bits data + 5 parity bits = 21 bit codeword (although you do not need to know this in advance)
Hamming Codeword:

26. How To Determine The Bit Groups
Method #1:
Starting with the parity bit, put n bits in the group, then exclude the next n bits
where n is the position number of the parity bit
Hamming Codeword: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 in 1
out 1 in 1
out 1 in 1
out 1 in 1
out 1 in 1
out 1 in 1
out 1 in 1
out 1 in 1
out 1 in 1
out 1 in 1
out 1 in
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21

27. How To Determine The Bit Groups
Method #1:
Starting with the parity bit, put n bits in the group, then exclude the next n bits
where n is the position number of the parity bit
Hamming Codeword: 1 1 1 1 1 1 1 1 1 1 1 1 1 2 in 2
out 2 in 2
out 2 in 2
out 2 in 2
out 2 in 2
out
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19

28. How To Determine The Bit Groups
Method #1:
Starting with the parity bit, put n bits in the group, then exclude the next n bits
where n is the position number of the parity bit
Hamming Codeword: 1 1 1 1 1 1 1 1 1 1 1 1 1 4 in 4
out 4 in 4
out 4 in
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19
Group 4: 4, 5, 6, 7, 12, 13, 14, 15, 20, 21

29. How To Determine The Bit Groups
Method #1:
Starting with the parity bit, put n bits in the group, then exclude the next n bits
where n is the position number of the parity bit
Hamming Codeword: 1 1 1 1 1 1 1 1 1 1 1 1 1 8 in 8
out
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
Group 8: 8, 9, 10, 11, 12, 13, 14, 15
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19
Group 4: 4, 5, 6, 7, 12, 13, 14, 15, 20, 21

30. How To Determine The Bit Groups
Method #1:
Starting with the parity bit, put n bits in the group, then exclude the next n bits
where n is the position number of the parity bit
Hamming Codeword: 1 1 1 1 1 1 1 1 1 1 1 1 1 16 in 16
out
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
Group 8: 8, 9, 10, 11, 12, 13, 14, 15
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19
Group 16: 16, 17, 18, 19, 20, 21
Group 4: 4, 5, 6, 7, 12, 13, 14, 15, 20, 21

31. How To Determine The Bit Groups
Method #2:
A bit, n, is in all parity bit groups where the parity bit position numbers add up to n
A parity bit is in it's own group, and not found in any other groups
Hamming Codeword: 1 1 1 1 1 1 1 1 1 1 1 1 1
Group 1: 1
, 3
, 5
, 7
, 9
, 11
, 13
, 15
, 17
, 19
, 21 1
6=4+2
11=8+2+1 16
Group 2: 2
, 3
, 6
, 7
, 10
, 11
, 14
, 15
, 18
, 19 2
7=4+2+1
12=8+4
17=16+1
Group 4: 4
, 5
, 6
, 7
, 12
, 13
, 14
, 15
, 20
, 21
3=2+1 8
13=8+4+1
18=16+2
Group 8: 8
, 9
, 10
, 11
, 12
, 13
, 14
, 15 4
9=8+1
14=8+4+2
19=16+2+1
Group 16: 16
, 17
, 18
, 19
, 20
, 21
5=4+1
10=8+2
15=
20=16+4
21=16+4+1

32. Reveiver Verification
The receiver has received this Hamming codeword, and now needs to verify correctness
Step #1: Check the parity of each group 1 1 1 1 1 1 1 1 1 1 1 1
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
ODD

33. Reveiver Verification
The receiver has received this Hamming codeword, and now needs to verify correctness
Step #1: Check the parity of each group 1 1 1 1 1 1 1 1 1 1 1 1
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
ODD
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19
EVEN

34. Reveiver Verification
The receiver has received this Hamming codeword, and now needs to verify correctness
Step #1: Check the parity of each group 1 1 1 1 1 1 1 1 1 1 1 1
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
ODD
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19
EVEN
Group 4: 4, 5, 6, 7, 12, 13, 14, 15, 20, 21
EVEN

35. Reveiver Verification
The receiver has received this Hamming codeword, and now needs to verify correctness
Step #1: Check the parity of each group 1 1 1 1 1 1 1 1 1 1 1 1
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
ODD
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19
EVEN
Group 4: 4, 5, 6, 7, 12, 13, 14, 15, 20, 21
EVEN
Group 8: 8, 9, 10, 11, 12, 13, 14, 15
ODD

36. Reveiver Verification
The receiver has received this Hamming codeword, and now needs to verify correctness
Step #1: Check the parity of each group 1 1 1 1 1 1 1 1 1 1 1 1
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
ODD
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19
EVEN
Group 4: 4, 5, 6, 7, 12, 13, 14, 15, 20, 21
EVEN
Group 8: 8, 9, 10, 11, 12, 13, 14, 15
ODD
Group 16: 16, 17, 18, 19, 20, 21
EVEN

37. Reveiver Verification
The receiver has received this Hamming codeword, and now needs to verify correctness
Step #2: If any incorrect parities, the error bit is located where the groups overlap
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
ODD
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19
EVEN
Bit Location: 9
8 + 1 = 9
Group 4: 4, 5, 6, 7, 12, 13, 14, 15, 20, 21
EVEN
Group 8: 8, 9, 10, 11, 12, 13, 14, 15
ODD
Group 16: 16, 17, 18, 19, 20, 21
EVEN

38. Reveiver Verification
The receiver has received this Hamming codeword, and now needs to verify correctness
Step #3: If there's an error bit, correct it by changing it to the opposite value 1
Group 1: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21
ODD
Group 2: 2, 3, 6, 7, 10, 11, 14, 15, 18, 19
EVEN
Bit Location: 9
8 + 1 = 9
Group 4: 4, 5, 6, 7, 12, 13, 14, 15, 20, 21
EVEN
Group 8: 8, 9, 10, 11, 12, 13, 14, 15
ODD
Group 16: 16, 17, 18, 19, 20, 21
EVEN

39. Reveiver Verification
The receiver has received this Hamming codeword, and now needs to verify correctness
Step #4: Remove the parity bits 1 1 1 1
0 1 1
= original data

40. Row And Column (RAC) Parity Algorithm
Another error-correcting algorithm that can correct one single-bit error
Example - encode the 12-bit data value: using odd parity
Step #1: Arrange the data bits into a 4 x 3 grid
(grid size doesn't matter, as long as sender and receiver
agree, and it should be relatively square)

41. RAC Parity Algorithm
Another error-correcting algorithm that can correct one single-bit error
Example - encode the 12-bit data value: using odd parity
Step #2: Add a column of parity bits: the correct parity for each row 1

42. RAC Parity Algorithm
Another error-correcting algorithm that can correct one single-bit error
Example - encode the 12-bit data value: using odd parity
Step #3: Add a row of parity bits: the correct parity for each column 1 1 1
Notice that there is no check bit for this row
since the right most bit is the check bit for the
parity column

43. RAC Parity Receiver Verification
If any row or column has the wrong parity, then an error has occurred
The error is located where the wrong parity row/column intersect
Step #1: Check parity for each row and column 1
ODD
ODD
EVEN 1 1
ODD
ODD
ODD
ODD
EVEN

44. RAC Parity Receiver Verification
If any row or column has the wrong parity, then an error has occurred
The error is located where the wrong parity row/column intersect
Step #2: Error bit is located where the wrong parity row/column intersect 1
ODD
ODD
EVEN 1 1
ODD
ODD
ODD
ODD
EVEN

45. RAC Parity Receiver Verification
If any row or column has the wrong parity, then an error has occurred
The error is located where the wrong parity row/column intersect
Step #3: Change the bit to the opposite value 1
ODD
ODD 1
EVEN 1 1
ODD
ODD
ODD
ODD
EVEN

46. RAC Parity Receiver Verification
If any row or column has the wrong parity, then an error has occurred
The error is located where the wrong parity row/column intersect
Step #4: Remove the parity bits 1 1 1

47. Which Is Better? Hamming Algorithm RAC Algorithm Analysis
Both have the same capability
but Hamming does it with less overhead
Hamming Algorithm
RAC Algorithm
Can correct 1 single bit error
Ability to correct errors
Can correct 1 single bit error 5
# of check bits required
for 12-bits of data 8 17
Codeword size
for 12-bits of data 20
42%
Overhead
for 12-bits of data
67% 5
# of check bits required for 20-bits of data 10
25%
Overhead
for 20-bits of data
50%

48. Internet Checksum Algorithm
Checksum: A sum computed from the bits in a data value for the purpose of detecting if an error has occurred during transmission
The checksum is appended to the end of the data
Checksums have the advantage that the sender and receiver can perform the same calculation on the data (don't have to write different algorithms for each end)
The Internet checksum algorithm has the additional advantage that the checksum itself can be added into the computation on the receiving size, and if the computation produces an answer of zero, then no error has occurred

49. Internet Checksum Algorithm
Checksum: A sum computed from the bits in a data value for the purpose of detecting if an error has occurred during transmission
The checksum is appended to the end of the data
Checksums have the advantage that the sender and receiver can perform the same calculation on the data (don't have to write different algorithms for each end)
The Internet checksum algorithm has the additional advantage that the checksum itself can be added into the computation on the receiving size, and if the computation produces an answer of zero, then no error has occurred

50. Internet Checksum Algorithm
Example - send this data: (binary)
F E (hexadecimal)
Step #1: Group the data into 16-bit units
00FE
0005
(binary)
(hex)

51. Internet Checksum Algorithm
Example - send this data:
Step #2: Add the 16-bit groups 1 1 1 1 1 1 1 1
00FE +
0005 1 1 1 1 3

52. Internet Checksum Algorithm
Example - send this data:
Step #3: If there's carry (>16-bit answer), add the carry to the sum
00FE +
0005
0103
There's no carry in this example

53. Internet Checksum Algorithm
Example - send this data:
Step #4: Invert all the bits
0103
FEFC
checksum

54. Internet Checksum Algorithm
Example - send this data:
Step #5: Append checksum to end of data
checksum
0 0 F E F E F C

55. Internet Checksum Receiver Verification
Example - Received data:
F E F E F C
Note: the verification algorithm is EXACTLY the same as the encoding algorithm
Step #1: Group the data into 16-bit units (including the checksum)
00FE
0005
FEFC
(binary)
(hex)

56. Internet Checksum Receiver Verification
Example - Received data:
F E F E F C
Step #2: Add the 16-bit groups 1 1 1 1 1 1 1 1
00FE
0005 +
FEFC 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 F F F F

57. Internet Checksum Receiver Verification
Example - Received data:
F E F E F C
Step #3: If there's carry (>16-bit answer), add the carry to the sum
00FE
0005 +
FEFC
FFFF
There's no carry in this example

58. Internet Checksum Receiver Verification
Example - Received data:
F E F E F C
Step #4: Invert all the bits
FFFF
0000
If the answer is zero, then
no error has occurred

59. Internet Checksum Algorithm Ex. 2
Example 2 - send this data: F F 4 F 5 F 6 F 7
Step #1: Group the data into 16-bit units
0100
F203
F4F5
F6F7

60. Internet Checksum Algorithm Ex. 2
Example 2 - send this data: F F 4 F 5 F 6 F 7
Step #2: Add the 16-bit groups 2 1
0100
F203
F4F5 +
F6F7 2 D E E F

61. Internet Checksum Algorithm Ex. 2
Example 2 - send this data: F F 4 F 5 F 6 F 7
Step #3: If there's carry (>16-bit answer), add the carry to the sum
0100
F203 1
F4F5
DEEF + +
F6F7
0002 2 D E E F D E F 1
carry
sum

62. Internet Checksum Algorithm Ex. 2
Example 2 - send this data: F F 4 F 5 F 6 F 7
Step #4: Invert all the bits
DEF1
210E

63. Internet Checksum Algorithm Ex. 2
Example 2 - send this data: F F 4 F 5 F 6 F 7
Step #5: Append checksum to end of data
F F 4 F 5 F 6 F E
checksum

64. Internet Checksum Verification Ex. 2
Example 2 - received data: F F 4 F 5 F 6 F E
Step #1: Group the data into 16-bit units
0100 F203 F4F5 F6F7 210E
Step #2: Add the 16-bit groups
F203 + F4F5 + F6F E = 2 FFFD
Step #3: If there's carry (>16-bit answer), add the carry to the sum
FFFD = FFFF
Step #4: Invert all the bits
FFFF
0000
No Error

65. Verification Ex. 3 – Multiple Bit Error
Example 2 - received data: F E 4 F 5 F 6 F A E (checksum didn't change)
Step #1: Group the data into 16-bit units
0107 F203 E4F5 F6FA 210E
Step #2: Add the 16-bit groups
F203 + E4F5 + F6FA + 210E = 2 F007
Step #3: If there's carry (>16-bit answer), add the carry to the sum
F007 = F009
Step #4: Invert all the bits
F009
0FF6
This is a detection algorithm, not a correction algorithm –
therefore, all we can do at this point is request the data again
Error!