8 Coding Theory Discrete Mathematics: A Concept-based Approach

Introduction The use of codes is in practice since, the ancient rulers of kingdom. In modern era Codes are used in data compression, cryptography, error coding and more recently it is used in network coding. Codes are studied by various scientific disciplines—such as information theory electrical engineering, mathematics, and computer science for the purpose of designing efficient and reliable data transmission methods. Discrete Mathematics: A Concept-based Approach

The source encoding consists of compression of the data in order to transmit more efficiently. This is the practice in the Internet transactions. The channel encoding adds extra data bits to make the transmission of data more robust and secured. Cell phones also use coding techniques to correct for the fading and noisy high frequency radio transmission. Data modems, telephone transmissions and NASA all employ channel coding techniques to get the bits through, for example the turbo code and LDPC codes. Discrete Mathematics: A Concept-based Approach

Some basic coding techniques Transmitted messages, like data from a satellite, invariably suffer from noise. It is important to encode a message in such a way that in spite of noise one should be able to decode a message to its original form. Most messages are sent as digital-sequences of 0’s and 1’s, such as 10101 or 1010011. Let us assume that we want to send the message 1011 in binary for a real word, such as ‘buy’, or a sentence such as ‘buy stocks’. One way of encoding 1011 would be to attach a binary “tail” to it so that if the message gets distorted to, say, 0011, we can detect the error. One such tail could be a 1 or 0, depending on whether we have an odd or an even number of 1’s in the word. Discrete Mathematics: A Concept-based Approach

This way all encoded words will have an even number of 1’s This way all encoded words will have an even number of 1’s. So 1011 will be encoded as 10111. This is called even parity. Now if this is distorted to 00111, we know that an error has occurred, because we have only received an odd number of 1’s. Similarly, if we look for odd number of 1’s in the word at the time of encoding then it is called odd parity. In general, this error-detecting code is called a parity check and is considered too simple and very useful. The American mathematician Claude E. Shannon (1916-2001) and his friend, namely, Richard Hamming and his colleagues in Bell laboratories, founded the field of coding theory. Discrete Mathematics: A Concept-based Approach

Data compression (also called, source coding) and Coding theory typically involves the removal of redundancy and the correction (or detection) of errors in the transmitted data. There are essentially two aspects to coding theory, namely, Data compression (also called, source coding) and Error correction (also called channel coding). Communication Process The process of communication involves transmitting information between source and destination. The three essential parts in an ideal communication system are Transmitter Channel and Receivers. Discrete Mathematics: A Concept-based Approach

The communication channel is subjected to variety of disturbances called noise. It may be due to lightning, folds in a magnetic tape or in any other reasons. Consider a typical communication system with noise. The information is defined by sequences of 0’s and 1’s, each is called a digit. A word is a sequence of digits. For example, the word 1011 is of length of 4 and 00101 is of length 5. A word is transmitted by sending its digits, one after the other across a binary channel. Discrete Mathematics: A Concept-based Approach

Definition: consider then is a group under the binary operation modulo 2, denoted as This group is called a group code. Let us consider as a group and is given in the following table. It is necessary to note that 1+21= 0, 0 = 00…0 is the identity and every element is its own inverse. Hence is an Abelian group under the operation +2 +2 00 01 10 11 Discrete Mathematics: A Concept-based Approach

Encoding Function Due to noise, the receiver may make some mistakes that go undetected, like interchange of 0 or 1. For example, the code 000 may be changed to 100 or 010 or 001 due to noise. The codes obtained by introducing additional digits called parity digits to the digits in the original message and are called Hamming codes. If the original message is a binary string of length m, the Hamming encoded message consisting of string of length n, where (n > m). Out of n digits, m digits are being used to represent the information part of the message and the Remaining ( n-m) digits are used for the detection and correction of errors in received message. Discrete Mathematics: A Concept-based Approach

Even parity checks In Hamming’s single-error detection code of length n, the first (n-1) digits contain the information part of the message and last digit is made either 0 or 1. If the digit introduced in the last position gives an even number of 1’s in the encoded word of length n, then the entire message is said to have even parity. Let be an encoding function such that and, . This encoding procedure is called an even parity check. Where Discrete Mathematics: A Concept-based Approach

Code with redundant digit Example: Let us consider and define The code and redundant digit is given in the following table. Original code Code with redundant digit 000 0000 001 0011 010 0101 011 0110 100 1001 101 1010 110 1100 111 1111 Let us consider the word 000 and E(000)=0000. If a single error occurs in 0000, it must be 1000, or 0100 or 0010 or 0001. With the use of redundant digits It is possible to detect and correct a single error occurring in the code. Discrete Mathematics: A Concept-based Approach

Odd parity check Odd parity check: In the information part of the message, the last digit is appended as either 0 or 1. If the digit introduced in the last position gives an odd number 1’s in the encoded word of length n, then the entire message is said to have an odd parity. Note: i) A code detects at the most k errors if and only if the minimum distance between any two words is at least (k+1). Hence, if the code word x is transformed to the word y due to at least (k+1) errors almost k errors can be detected. ii) A code can correct a set of at the most k errors if and only if the minimum distance between any two code words is at least (2k+1). Discrete Mathematics: A Concept-based Approach

Decoding and Error correction Let W- denote the set of words to be transmitted and . In order to reduce the number of errors in the transmission of the words, the following procedure is adopted. Choose an Integer (n > m), such that for each word , assign a unique word . This is performed by appending (n - m) extra components to w by means of one-to-one function . , hence c = E(w). The appending of digits helps to detect or correct errors that occur during transmission. The process of appending extra 0’s and 1’s to w through the function E is called the encoding process. Discrete Mathematics: A Concept-based Approach

The word c = E(w) is transmitted through a channel The word c = E(w) is transmitted through a channel. Let be the word received. The extra components contained are removed with the help of an onto function, to get the word . The process of obtaining the original word w from D(r) is called the Decoding. It is necessary to note the following observations. 1) The word - is called a message and - is called code word assigned to w. 2)The function E – is one-one. This means that different words in W have different code words. Discrete Mathematics: A Concept-based Approach

3). The set is called the received word and the word is called the decoded word. 4). The function E- is called the encoding function and the function D- is called the decoding function. Example 8.5: Consider the encoding function ( n > m) defined as follows and the corresponding decoding function, . Which is parity code example Discrete Mathematics: A Concept-based Approach

a) Find the code words assigned by E to the following messages in a) Find the code words assigned by E to the following messages in . 000, 001, 011, 100, 110, 101, 111, 010. b) Find the decoded words assigned by D, to the following received words in . 0000, 0001, 0101, 1111, 1010, 1100, 1101, 1001. Solution: By the definition of E, we have the following E(000) = 0000, E(001) = 0011, E(011) = 0110, E(100) = 1001 E(1 1 0) = 1 1 0 0, E(101) = 1010, E(111) = 1 1 1 1, E(010) = 0101. By the definition of D, D(0000) = 000, D(0001) = 000, D(0101) = 010, D(1111) = 111 D(1010) = 101, D(1100) = 110, D(1101) = 110, D(1001) = 100 Discrete Mathematics: A Concept-based Approach

Parity and Generator Matrices The encoding and decoding functions can be expressed as matrices over . Given the encoding function is given by a m x n matrix. The matrix obtained is called the generator matrix for the code and is of the form where Im – unit matrix of order m and A is m x (n- m) matrix to be chosen appropriately. Let be a 3 x 6 matrix over , Then Discrete Mathematics: A Concept-based Approach

Where The augmented matrix G- is called a generator matrix Where The augmented matrix G- is called a generator matrix. Example : Find the code words generated by the encoding function , with respect to the parity check matrix. Discrete Mathematics: A Concept-based Approach

The generator matrix G – is given by, Now E (w) = WG. Discrete Mathematics: A Concept-based Approach

On simplification we get, The code words generated are as given under On simplification we get, The code words generated are as given under. C = { 00000, 01011, 10011, 11000 }. Discrete Mathematics: A Concept-based Approach

Hamming codes In telecommunication, Hamming codes are a family of linear error-correcting codes that generalize the Hamming (7, 4) code invented by Richard Hamming in 1950. Hamming codes can detect up to two-bit errors or correct one-bit errors without detection of uncorrected errors. By contrast, the simple parity code cannot correct errors, and can detect only an odd number of bits in error. Hamming codes are perfect codes, that is, they achieve the highest possible rate for codes with their block length and minimum distance Discrete Mathematics: A Concept-based Approach

Applications of coding theory The concern of coding theory is designing codes that help synchronization. A code may be designed so that a phase shift can be easily detected and corrected and that multiple signals can be sent on the same channel. Another application of codes, used in some mobile phone systems, is code division multiple access (CDMA). Each phone is assigned a code sequence that is approximately uncorrelated with the codes of other phones. When transmitting, the code word is used to modulate the data bits representing the voice message. Discrete Mathematics: A Concept-based Approach

At the receiver, a demodulation process is performed to recover the data. The properties of this class of codes allow many users (with different codes) to use the same radio channel at the same time. To the receiver, the signals of other users will appear to the demodulator only as a low-level noise. Another general class of codes is the automatic repeat request (ARQ) codes. In these codes, the sender adds redundancy to each message for error checking, usually by adding the check bits. If the check bits are not consistent with the rest of the message, when it arrives, the receiver will ask the sender to retransmit the message. Discrete Mathematics: A Concept-based Approach

Summary The concern of coding theory is designing codes that help synchronization. The hierarchy of data processing is presented in terms coding theory using the concept of codes as groups. Encoding and decoding techniques with mathematics perspective are discussed by viewing encoding and decoding as functions. Parity check is explained lucidly for both odd and even. The encoding function is expressed in terms of matrices called Generator matrices. Hamming codes are explained as family of linear error-correcting codes that generalize the Hamming (7, 4). The applications of coding theory is presented with real world applications. Discrete Mathematics: A Concept-based Approach