1. Hamming Distance & Hamming Code
Lecture 6
Error Correction
Hamming Distance & Hamming Code

2. Something about error correction
Error correction can be handled in two ways:
Forward Error Correction (FEC) that is by correcting the received erroneous code as you cant retransmit data in real time video applications, that means a receiver should have an error correcting code which can automatically correct the code
Reverse Error Correction(REC) that is by retransmitting the data unit
Error correction is done by sending redundant bits with the original data
Since correction requires high no of redundant bits, so most of the times correction is done for 1 bit, 2 bits or max 3 bits.

3. Redundancy
Error detection uses the concept of redundancy, which means adding extra bits for detecting errors at the destination.

4. Redundancy

5. Redundant Bits
To calculate the no of redundant bits we need to find out the relationship between m & r.

6. If the total number of bits in a transmittable unit is m + r, then r must be able to indicate at least m + r +1 different states. Of these, one state means no error and m + r states indicate the location of an error in each of the m + r positions.
So, m + r + 1 states must be discoverable by r bits; and r bits can indicate 2r different states. Therefore, 2r must be equal to or greater than m + r + 1 :
2r ≥ m + r +1
The value of r can be determined by plugging in the value of m (the original length of the data unit to be transmitted). For example, if the value of m is 7 (as in a seven-bit ASCII code), the smallest r value that can satisfy this equation is 4:
24 ≥

7. Number of redundancy bits (r)
Table Data and redundancy bits
Number of data bits (m)
Number of redundancy bits (r)
Total bits
(m + r) 1 2 3 5 6 4 7 9 10 11

8. Coding
Redundancy is achieved through various coding schemes. The sender adds redundant bits through a process that creates a relationship between the redundant bits and the actual data bits. The receiver checks the relationships between the two sets of bits to detect errors. The ratio of redundant bits to data bits and the robustness of the process are important factors in any coding scheme. There are two very popular methods of coding:
Block coding
Convolution coding
10.#

9. BLOCK CODING
In block coding, we divide our message into blocks, each of k bits, called datawords. We add r redundant bits to each block to make the length n = k + r. The resulting n-bit blocks are called codewords.
It uses modulo 2 arithmetic for addition & subtraction operation (for subtraction or division it is equivalent to logical XOR operation)
This unit is on hamming code & distance.
10.#

10. Datawords = 2k
Code words = 2n
Used code words = 2n
Wasted code words = 2n – 2k

11. Example : 4B/5B is good example of coding
Example : 4B/5B is good example of coding. In this scheme k = 4 & n = 5 Data words are 2k = 24 = 16 Codewords are 2n = 25 = 32 Used code words are 2n – 2k = = 16 Wasted codewords are = 16

12. Error Detection
How can errors be detected by using block coding? If the following two conditions are met, the receiver can detect a change in the original codeword.
1. The receiver has (or can find) a list of valid codewords.
2. The original codeword has changed to an invalid one.
10.#

13. Figure: Process of error detection in block coding
10.13
10.#

14. There are two popular block codes
Simple parity check code
Hamming code

15. Example
Let us assume that k = 2 and n = 3. Table given below shows the list of datawords and codewords. Later, we will see how to derive a codeword from a dataword.
Table: A code for error detection in Example 10.1
10.15

16. Example
Let us add more redundant bits to Example 10.2 to see if the receiver can correct an error without knowing what was actually sent. We add 3 redundant bits to the 2-bit dataword to make 5-bit codewords. Again, later we will show how we chose the redundant bits. For the moment let us concentrate on the error correction concept. Table in previous slide shows the datawords and codewords.
Assume the dataword is 01. The sender consults the table (or uses an algorithm) to create the codeword The codeword is corrupted during transmission, and is received (error in the second bit from the right). First, the receiver finds that the received codeword is not in the table. This means an error has occurred. (Detection must come before correction. The receiver, assuming that there is only 1 bit corrupted, uses the following strategy to guess the correct dataword.
Comparing the received codeword with the first codeword in the table (01001 versus 00000), the receiver decides that the first codeword is not the one that was sent because there are two different bits.
By the same reasoning, the original codeword cannot be the third or fourth one in the table.
The original codeword must be the second one in the table because this is the only one that differs from the received codeword by 1 bit. The receiver replaces with and consults the table to find the dataword 01.

17. Types of error correction codes
Error correction can be handled in two ways:
Forward Error Correction (FEC) that is by correcting the received erroneous code as you cant retransmit data in real time video applications, that means a receiver should have an error correcting code which can automatically correct the code
Reverse Error Correction(REC) that is by retransmitting the data unit
Error correction is done by sending redundant bits with the original data
Since correction requires high no of redundant bits, so most of the times correction is done for 1 bit, 2 bits or max 3 bits.

18. Hamming Distance
The Hamming Distance between two words of same size is the number of difference between the corresponding bits. It is a central concept in coding for error control
The hamming distance is denoted between two words x & y is denoted as d(x,y)
It is obtained by XORing the two words.

19. Let us find the Hamming distance between two pairs of words.
Example
Let us find the Hamming distance between two pairs of words.
The Hamming distance d(000, 011) is 2 because 000  011 is 011 (two 1s)
The Hamming distance d(10101, 11110) is 3 because  is (three 1s).

20. Minimum Hamming Distance
The minimum hamming distance is the smallest Hamming distance between all possible pairs in set of words
is denoted as dmin
To guarantee the detection of up to s errors in all cases, the minimum Hamming distance in a block code must be dmin = s+1
To guarantee the correction up to t errors in all cases, the minimum Hamming distance in a block code must be dmin = 2t+1

21. Example
Find the minimum Hamming distance for the pair of values given below.
Solution
We first find all Hamming distances.
d(000, 011) = 2 d(000, 101) = 2 d(000, 110) = 2 d(011, 101) = 2 d(011, 110) = 2 d(101, 110) = 2
The dmin in this case is 2.

22. Find the minimum Hamming distance for the pair of values given below.
Example
Find the minimum Hamming distance for the pair of values given below.
Solution
We first find all the Hamming distances
d(00000, 01011) = 3 d(00000, 10101) = 3 d(00000, 11110) = 4
d(01011, 10101) = 4 d(01011, 11110) = 3 d(10101, 11110) = 3
The dmin in this case is 3

23. Three parameters for error correction
Dataword size k
Codeword size n
Minimum Hamming distance dmin
For example :
Data word = 2
Code word = 5
Dmin = 2

24. Figure : Geometric concept explaining dmin in error detection
To guarantee the detection of up to s errors in all cases, the minimum Hamming distance in a block code must be dmin = s+1
if we want to detect 3 bits in a packet, we need to make the minimum hamming distance 4 bits
10.#

25. Hamming Distance For Error Correction
For error correction, we definitely need more distance than detection. It can be shown that to detect t errors, we need to have dmin = 2t + 1.
In other words, if we want to correct 4 bits in a packet, we need to make the minimum hamming distance 9 bits, which means a lot of redundant bits need to be sent with the data. the geometrical representation of this concept.
10.#

26. Figure : Hamming distance for error correction
10.#

27. Example A code scheme has a Hamming distance dmin = 4
Example A code scheme has a Hamming distance dmin = 4. What is the error detection and correction capability of this scheme? Solution This code guarantees the detection of up to three errors (s = 3), but it can correct up to one error. In other words, if this code is used for error correction, part of its capability is wasted. Error correction codes need to have an odd minimum distance (3, 5, 7, ..)

28. Types of Block Codes Linear codes Non linear codes
Almost all codes we use today are subset of linear block codes
A linear block code is a code n which the XOR(addition 2 modulo) of two valid codewords creates another valid codeword

29. Example Let us see if the two codes we defined in previous tables belong to the class of linear block codes. The scheme in previous Table is a linear block code because the result of XORing any codeword with any other codeword is a valid codeword. For example the XORing of the second and third codewords creates the fourth one. The scheme in second Table is also a linear block code. We can create all four codewords by XORing two other codewords.

30. Minimum Hamming distance for linear block codes is the number of 1s in the non zero valid codeword with the smallest nos of 1s

31. Simple Parity Check Code
The most familiar error detecting code is simple parity check code
In this code, a k-bit dataword is changed to an n-bit codeword where n = k + 1. The extra bit, called the parity bit, is selected to make the total number of 1s in the codeword even.
The minimum Hamming distance for this category is dmin = 2, which means that the code is a single-bit error-detecting code; it cannot correct any error.

32. A simple parity-check code is a single-bit error-detecting code in which n = k + 1 with dmin = 2.
Our first code (Table 10.1) is a parity-check code with k = 2 and n = 3. The code in Table 10.3 is also a parity-check code with k = 4 and n = 5.
Figure shows a possible structure of an encoder (at the sender) and a decoder (at the receiver).

33. The encoder uses a generator that takes a copy of a 4-bit dataword (a0, a1, a2, and a3) and generates a parity bit r0. Te dataword bits and the parity bit create the 5-bit codeword. The parity bit that is added makes the number of 1s in the codeword even.
This is normally done by adding the 4 bits of the dataword (modulo-2); the result is the parity bit. In other words,
r0 = a3 + a2 + a1+ a0 (modulo-2)
s0 = b3 + b2 + b1+ b0 + q0 (modulo-2)
The syndrome is passed to the decision logic analyzer. If the syndrome is 0, there is no error in the received codeword; the data portion of the received codeword is accepted as the dataword; if the syndrome is 1, the data portion of the received codeword is discarded. The dataword is not created.

35. Example 10.7
Let us look at some transmission scenarios. Assume the sender sends the dataword The codeword created from this dataword is 10111, which is sent to the receiver. We examine five cases:
10.35

36. A simple parity check code can detect an odd number of errors

37. Two Dimensional Parity Check A Better Approach
In this method the is organized in a tabular form in rows & columns
In the figure mentioned next we are sending 5 data of 7-bytes in separate rows & columns
We have explained this is block parity case
Here we discuss detection of the places where errors take place

38. This method can detect up to 3 errors.
4 errors can not be detected.

39. Hamming Code A most popular linear block code
Hamming codes are a family of linear error-correcting codes that generalize the Hamming(7,4)-code invented by Richard Hamming in 1950.
Hamming codes can detect up to two-bit errors or correct one-bit errors without detection of uncorrected errors.
By contrast, the simple parity code cannot correct errors, and can detect only an odd number of bits in error.
Hamming codes are perfect codes, that is, they achieve the highest possible rate for codes with their block length and minimum distance called as min hamming distance

40. Hamming Code
A hamming code can be applied to the data unit of any length & uses the relationship between data & redundancy bits. For example, a seven-bit ASCII code requires 4 redundancy bits that can be added to the end of the data unit or interspersed between the original data bits.

41. In Hamming code, the position of r bits in VRC
The position of r1 is chosen for the values which has 1 in the right most position
The position of r2 is chosen for the values which has 1 in the second last position & so on.

42. Hamming Code

43. Hamming Code

44. Example of Hamming Code

45. Detection of Single-bit error using Hamming Code

46. Single bit Error
Detection
It detects
Position of
the corrupted
bits

47. Burst error correction example
Burst Error Detection
Sequence of bits is arranged in rows, then column wise transmission of bits is made from left to right. Burst of error is converted into case of single bit error