1. 8 Coding Theory
Discrete Mathematics: A Concept-based Approach

2. Introduction
The use of codes is in practice since, the ancient rulers of kingdom. In modern era Codes are used in data compression, cryptography, error coding and more recently it is used in network coding. Codes are studied by various scientific disciplines—such as information theory electrical engineering, mathematics, and computer science for the purpose of designing efficient and reliable data transmission methods.
Discrete Mathematics: A Concept-based Approach

3. The source encoding consists of compression of the data in order to transmit more efficiently. This is the practice in the Internet transactions. The channel encoding adds extra data bits to make the transmission of data more robust and secured. Cell phones also use coding techniques to correct for the fading and noisy high frequency radio transmission. Data modems, telephone transmissions and NASA all employ channel coding techniques to get the bits through, for example the turbo code and LDPC codes.
Discrete Mathematics: A Concept-based Approach

4. Some basic coding techniques
Transmitted messages, like data from a satellite, invariably suffer from noise. It is important to encode a message in such a way that in spite of noise one should be able to decode a message to its original form. Most messages are sent as digital-sequences of 0’s and 1’s, such as or Let us assume that we want to send the message 1011 in binary for a real word, such as ‘buy’, or a sentence such as ‘buy stocks’. One way of encoding 1011 would be to attach a binary “tail” to it so that if the message gets distorted to, say, 0011, we can detect the error. One such tail could be a 1 or 0, depending on whether we have an odd or an even number of 1’s in the word.
Discrete Mathematics: A Concept-based Approach

5. This way all encoded words will have an even number of 1’s
This way all encoded words will have an even number of 1’s. So 1011 will be encoded as This is called even parity. Now if this is distorted to 00111, we know that an error has occurred, because we have only received an odd number of 1’s. Similarly, if we look for odd number of 1’s in the word at the time of encoding then it is called odd parity. In general, this error-detecting code is called a parity check and is considered too simple and very useful. The American mathematician Claude E. Shannon ( ) and his friend, namely, Richard Hamming and his colleagues in Bell laboratories, founded the field of coding theory.
Discrete Mathematics: A Concept-based Approach

6. Data compression (also called, source coding) and
Coding theory typically involves the removal of redundancy and the correction (or detection) of errors in the transmitted data. There are essentially two aspects to coding theory, namely,
Data compression (also called, source coding) and
Error correction (also called channel coding).
Communication Process
The process of communication involves transmitting information between source and destination. The three essential parts in an ideal communication system are
Transmitter
Channel and
Receivers.
Discrete Mathematics: A Concept-based Approach

7. The communication channel is subjected to variety of disturbances called noise. It may be due to lightning, folds in a magnetic tape or in any other reasons. Consider a typical communication system with noise. The information is defined by sequences of 0’s and 1’s, each is called a digit. A word is a sequence of digits. For example, the word 1011 is of length of 4 and is of length 5. A word is transmitted by sending its digits, one after the other across a binary channel.
Discrete Mathematics: A Concept-based Approach

8. Definition: consider then is a group under the binary operation modulo 2, denoted as This group is called a group code. Let us consider as a group and is given in the following table. It is necessary to note that 1+21= 0, 0 = 00…0 is the identity and every element is its own inverse. Hence is an Abelian group under the operation +2 +2 00 01 10 11
Discrete Mathematics: A Concept-based Approach

9. Encoding Function
Due to noise, the receiver may make some mistakes that go
undetected, like interchange of 0 or 1. For example, the code
000 may be changed to or or due to noise. The
codes obtained by introducing additional digits called parity
digits to the digits in the original message and are called
Hamming codes. If the original message is a binary string of
length m, the Hamming encoded message consisting of string of
length n, where (n > m). Out of n digits, m digits are being used
to represent the information part of the message and the
Remaining ( n-m) digits are used for the detection and correction of errors in received message.
Discrete Mathematics: A Concept-based Approach

10. Even parity checks
In Hamming’s single-error detection code of length n, the first (n-1) digits contain the information part of the message and last digit is made either 0 or 1. If the digit introduced in the last position gives an even number of 1’s in the encoded word of length n, then the entire message is said to have even parity. Let be an encoding function such that and, . This encoding procedure is called an even parity check. Where
Discrete Mathematics: A Concept-based Approach

11. Code with redundant digit
Example: Let us consider and define The code and redundant digit is given in the following table.
Original code
Code with redundant digit
000
0000
001
0011
010
0101
011
0110
100
1001
101
1010
110
1100
111
1111
Let us consider the word and E(000)=0000. If a single error occurs in 0000, it must be 1000, or or 0010
or With the use of redundant
digits
It is possible to detect and correct a single error occurring in the code.
Discrete Mathematics: A Concept-based Approach

12. Odd parity check
Odd parity check: In the information part of the message, the last digit is appended as either 0 or 1. If the digit introduced in the last position gives an odd number 1’s in the encoded word of length n, then the entire message is said to have an odd parity. Note: i) A code detects at the most k errors if and only if the minimum distance between any two words is at least (k+1). Hence, if the code word x is transformed to the word y due to at least (k+1) errors almost k errors can be detected. ii) A code can correct a set of at the most k errors if and only if the minimum distance between any two code words is at least (2k+1).
Discrete Mathematics: A Concept-based Approach

13. Decoding and Error correction
Let W- denote the set of words to be transmitted and . In order to reduce the number of errors in the transmission of the words, the following procedure is adopted. Choose an Integer (n > m), such that for each word , assign a unique word . This is performed by appending (n - m) extra components to w by means of one-to-one function . , hence c = E(w). The appending of digits helps to detect or correct errors that occur during transmission. The process of appending extra 0’s and 1’s to w through the function E is called the encoding process.
Discrete Mathematics: A Concept-based Approach

14. The word c = E(w) is transmitted through a channel
The word c = E(w) is transmitted through a channel. Let be the word received. The extra components contained are removed with the help of an onto function, to get the word . The process of obtaining the original word w from D(r) is called the Decoding. It is necessary to note the following observations. 1) The word - is called a message and - is called code word assigned to w. 2)The function E – is one-one. This means that different words in W have different code words.
Discrete Mathematics: A Concept-based Approach

15. 3). The set is called the received word and the word is called the decoded word. 4). The function E- is called the encoding function and the function D- is called the decoding function. Example 8.5: Consider the encoding function ( n > m) defined as follows and the corresponding decoding function, . Which is parity code example
Discrete Mathematics: A Concept-based Approach

16. a) Find the code words assigned by E to the following messages in
a) Find the code words assigned by E to the following messages in . 000, 001, 011, 100, 110, 101, 111, 010. b) Find the decoded words assigned by D, to the following received words in , 0001, 0101, 1111, 1010, 1100, 1101, Solution: By the definition of E, we have the following E(000) = 0000, E(001) = 0011, E(011) = 0110, E(100) = 1001 E(1 1 0) = , E(101) = 1010, E(111) = , E(010) = By the definition of D, D(0000) = 000, D(0001) = 000, D(0101) = 010, D(1111) = 111 D(1010) = 101, D(1100) = 110, D(1101) = 110, D(1001) = 100
Discrete Mathematics: A Concept-based Approach

17. Parity and Generator Matrices
The encoding and decoding functions can be expressed as matrices over . Given the encoding function is given by a m x n matrix. The matrix obtained is called the generator matrix for the code and is of the form where Im – unit matrix of order m and A is m x (n- m) matrix to be chosen appropriately. Let be a 3 x 6 matrix over , Then
Discrete Mathematics: A Concept-based Approach

18. Where The augmented matrix G- is called a generator matrix
Where The augmented matrix G- is called a generator matrix. Example : Find the code words generated by the encoding function , with respect to the parity check matrix.
Discrete Mathematics: A Concept-based Approach

19. The generator matrix G – is given by, Now E (w) = WG.
Discrete Mathematics: A Concept-based Approach

20. On simplification we get, The code words generated are as given under
On simplification we get, The code words generated are as given under. C = { 00000, 01011, 10011, }.
Discrete Mathematics: A Concept-based Approach

21. Hamming codes
In telecommunication, Hamming codes are a family of linear error-correcting codes that generalize the Hamming (7, 4) code invented by Richard Hamming in Hamming codes can detect up to two-bit errors or correct one-bit errors without detection of uncorrected errors. By contrast, the simple parity code cannot correct errors, and can detect only an odd number of bits in error. Hamming codes are perfect codes, that is, they achieve the highest possible rate for codes with their block length and minimum distance
Discrete Mathematics: A Concept-based Approach

22. Applications of coding theory
The concern of coding theory is designing codes that help synchronization. A code may be designed so that a phase shift can be easily detected and corrected and that multiple signals can be sent on the same channel. Another application of codes, used in some mobile phone systems, is code division multiple access (CDMA). Each phone is assigned a code sequence that is approximately uncorrelated with the codes of other phones. When transmitting, the code word is used to modulate the data bits representing the voice message.
Discrete Mathematics: A Concept-based Approach

23. At the receiver, a demodulation process is performed to recover the data. The properties of this class of codes allow many users (with different codes) to use the same radio channel at the same time. To the receiver, the signals of other users will appear to the demodulator only as a low-level noise. Another general class of codes is the automatic repeat request (ARQ) codes. In these codes, the sender adds redundancy to each message for error checking, usually by adding the check bits. If the check bits are not consistent with the rest of the message, when it arrives, the receiver will ask the sender to retransmit the message.
Discrete Mathematics: A Concept-based Approach

24. Summary The concern of coding theory is designing codes that help
synchronization. The hierarchy of data processing is presented in terms coding theory using the concept of codes as groups. Encoding and decoding techniques with mathematics perspective are discussed by viewing encoding and decoding as functions.
Parity check is explained lucidly for both odd and even. The encoding function is expressed in terms of matrices called Generator matrices. Hamming codes are explained as family of linear error-correcting codes that generalize the Hamming (7, 4). The applications of coding theory is presented with real world applications.
Discrete Mathematics: A Concept-based Approach