NOISE-RESISTANT CODING ● Coding Concepts and Technologies of Noise-resistant Coding It's hard to work in a noisy room as it makes it harder to think. Work.

Module 2. “Basics of Transmission Theory and Messages Coding” Lecture 2.1. NOISE-RESISTANT CODING

NOISE-RESISTANT CODING ● Coding Concepts and Technologies of Noise-resistant Coding It's hard to work in a noisy room as it makes it harder to think. Work done in such environment is likely to cause more mistakes. Electronic devices suffer much the same fate. Signals too are apt (склонны) to be misinterpreted in an environment that is "noisy".

During a conversation how do you determine if you heard the other person correctly? We human receivers employ various forms of error detection and correction as we try to make sense of what we are hearing. First of all, we use context and if a word or phrase does not seem to fit the topic of conversation, we automatically replace the confusing section with what we expect to hear. If contextual confusion still exists, we ask the person to repeat what he has just said or we may ask the other person to speak louder. All of these are forms of error correction schemes employed during human communication.

Error detection and correction (EDC) to achieve good communication is also employed in electronic devices and in much the same conceptual way. In fact, without EDC most electronic communication would not be possible. The level of noise and interferences are just way too high in electronic medium. The amount of EDC required and its effectiveness depends on the signal to noise ratio, or SNR.

EDC can be implemented in the five following ways: Increase signal power Decrease signal noise Introduce diversity Retransmission Forward Error Correction

Forward Error Correction When a duplex line is not available or is not practical, a form of error correction called Forward Error Coding (FEC) is used. The receiver has no real-time contact with the transmitter and can not verify if a block was received correctly. It must make a decision about the received data and do whatever it can to either fix it or declare an alarm.

FEC techniques add a heavy burden (нагрузку) on the link either in adding redundant data and adding delay. Another big disadvantage is the lower information rate. However at the same time, these techniques reduce the need to vary power. For the same power, we can now achieve a lower error rate. The communication in this case remains simplex and all the burden of detecting and correcting errors falls on the receiver. The transmitter complexity is avoided but is now placed on the receiver instead.

Digital signals use FEC in these three well known methods: Block Codes – блоковые коды Convolutional Codes – свёрточные коды (загортальне кодування) Interleaving (чередование) – каскадное кодирование

Block codes operate on a block of bits
Block codes operate on a block of bits. Using a preset algorithm, we take a group of bits and add a coded part to make a larger block. This block is checked at the receiver. The receiver then makes a decision about the validity of the received sequence. Convolutional codes are referred to as continuous codes as they operate on a certain number of bits continuously.

Interleaving has mitigating properties for fading channels and works well in conjunction with these two types of coding. In fact all three techniques are used together in many EDC suites such as Digital Video Broadcast, satellite communications, radio and cell phones and baseband systems such as PCs and CD players.

Block Codes Block codes are referred to as (n, k) codes
Block Codes Block codes are referred to as (n, k) codes. A block of k information bits are coded to become a block of n bits. Formal definition A block code is a code which encodes strings formed from an alphabet set S into code words by encoding each letter of S separately.

But before we go any further with the details, let's look at an important concept in coding called Hamming distance. Let's say that we want to code the 10 integers, 0 to 9 by a digital sequence. Sixteen unique sequences can be obtained from four bit words. We assign the first ten of these, one to each integer. Each integer is now identified by its own unique sequence of bits as in Table below. The remaining six possible sequences are unassigned.

The universe of all possible sequences is called the code space
The universe of all possible sequences is called the code space. For 4-bit sequences, the code space consists of 16 sequences. Of these we have used only ten. These are the valid words. The ones not used or unassigned are invalid code words. They would never be sent, so if one of these is received then the receiver assumes that an error has occurred.

Hamming Weight: The Hamming weight of this code scheme is the largest number of 1's in a valid codeword. This number is 3 among the 10 codewords we have chosen. (the ones in the while space) Concept of Hamming Distance In continuous variables, we measure distance by Euclidean concepts such as lengths, angles and vectors. In the binary world, distances are measured between two binary words by something called the Hamming distance. The Hamming distance is the number of disagreements between two binary sequences of the same size. It is a measure of how apart binary objects are.

The Hamming distance between sequences 001 and 101 is = 1 The Hamming distance between sequences and is = 4 Hamming distance and weight are very important and useful concepts in coding. The knowledge of Hamming distance is used to determine the capability of a code to detect and correct errors.

Although the Hamming weight of our chosen code set is 3, the minimum Hamming distance is 1. The codeword 1011 and codeword 1010 differ by only one bit. A single bit error can turn 1011 into The receiver interpreting the received sequence of 1010 will see it as a valid codeword will never know that actually the sent sequence was So it only takes one bit error to turn one valid codeword into another valid codeword such that no error is apparent. Number 2 can then turn into Number 8 with one bit error in the first bit.

Now extend the chosen code set by one parity bit
Now extend the chosen code set by one parity bit. Here is what we do, if the number of 1's is even then we extend the sequence by appending a 1, and if the number of 1's is odd, then we add a zero at the end of the sequence.. The codespace now doubles. It goes from 16 possible codewords to 32 codewords since codespace is equal to 2N, where N is the length of the sequence. We can extend the sequence yet again using the same process and now the code space increases to 64. In the first case the number of valid codewords is 10 out of 32 and in the second case it is still 10 but out of 64, as shown in Column 4.

We now have three code choices, with 4, 5 or 6-bit words
We now have three code choices, with 4, 5 or 6-bit words. The Hamming weight of the first set is 3, 4 for the second set and 5 for the last set. The minimum Hamming distance for the first set is 1. The minimum Hamming distance for the second set is 2 bits and 3 for the last set.

In the first set, even a single bit error cannot be tolerated (because it cannot be detected). In the second set, a single error can be detected because a single bit error does not turn one code word into another one. So its error detection ability is 1. For the last set, we can detect up to 2 errors. So its error detection capability is 2 bits. We can generalize this to say that the maximum number of error bits that can be detected is t = dmin -1, where dmin is Hamming distance of the codewords. For a code with dmin = 3, we can both detect 1 and 2 bit errors.

So we want to have a code set with as large a Hamming distance as possible since this directly effects our ability to detect errors. It also says that in order to have a large Hamming distance, we need to make our codewords larger. This adds overhead, reduces information bit rate for a given bandwidth and proves the adage that you cannot get something for nothing even in signal processing.

Number of errors we can correct Detecting is one thing but correcting is another. For a sequence of 6 bits, there are 6 different ways to get a 1 bit error and there are 15 different ways we can get 2-bit errors ! No. of 2-bit combinations = 2!4! = 15

What is the most likely error event given that this is a Gaussian channel? Most likely event is a 1 bit error then 2 bit errors and so on. There is also a possibility that 3 errors can occur, but for a code of 6 bits, the maximum we can detect is two bits. This is an acceptable risk since 3 bit errors are highly unlikely because if the transition probability p is small (<<1), the probability of getting three errors is cube of the channel error rate, PE(N) = pN

Of course it is indeed possible to get as many as 6 errors in a 6 bit sequence but since we have no way of detecting this scenario, we get what is called a decoding failure. This is also termed the probability of undetected errors and is the limitation on what can be corrected. The number of errors that we can correct is given by t (int)=(dmin-1)/2 For a code with dmin = 3, the correction capability is just 1 bit.

Creating block codes The block codes are specified by (n. k)
Creating block codes The block codes are specified by (n.k). The code takes k information bits and computes (n-k) parity bits from the code generator matrix. Most block codes are systematic in that the information bits remain unchanged with parity bits attached either to the front or to the back of the information sequence. Hamming code, a simple linear block code.

Hamming codes are most widely used linear block codes
Hamming codes are most widely used linear block codes. A Hamming code is generally specified as (2n-1, 2n-n-1). The size of the block is equal to 2n-1. The number of information bits in the block is equal to 2n-n-1 and the number of overhead bits is equal to n. All Hamming codes are able to detect three errors and correct one. Common sizes of Hamming codes are (7, 4), (15,11) and (31, 26). All of these have the same Hamming distance.

Repetition schemes Variations on this theme exist
Repetition schemes Variations on this theme exist. Given a stream of data that is to be sent, the data is broken up into blocks of bits, and in sending, each block is sent some predetermined number of times. For example, if we want to send "1011", we may repeat this block three times each. Suppose we send " ", and this is received as " ". As one group is not the same as the other two, we can determine that an error has occurred. This scheme is not very efficient, and can be susceptible to problems if the error occurs in exactly the same place for each group (e.g. " " in the example above will be detected as correct in this scheme).

Parity schemes A parity bit is an error detection mechanism that can only detect an odd number of errors. The stream of data is broken up into blocks of bits, and the number of 1 bits is counted. Then, a "parity bit" is set (or cleared) if the number of one bits is odd (or even). (This scheme is called even parity; odd parity can also be used.) If the tested blocks overlap, then the parity bits can be used to isolate the error, and even correct it if the error affects a single bit: this is the principle behind the Hamming code. There is a limitation to parity schemes. A parity bit is only guaranteed to detect an odd number of bit errors (one, three, five, and so on). If an even number of bits (two, four, six and so on) are flipped, the parity bit appears to be correct, even though the data is corrupt.

Checksum A checksum of a message is an arithmetic sum of message code words of a certain word length, for example byte values, and their carry value. The sum is negated by means of ones-complement, and stored or transferred as an extra code word extending the message. On the receiver side, a new checksum may be calculated from the extended message. If the new checksum is not 0, an error has been detected. Checksum schemes include parity bits, check digits, and longitudinal redundancy check.

Cyclic redundancy checks More complex error detection (and correction) methods make use of the properties of finite fields and polynomials over such fields. The cyclic redundancy check considers a block of data as the coefficients to a polynomial and then divides by a fixed, predetermined polynomial. The coefficients of the result of the division is taken as the redundant data bits, the CRC. On reception, one can recompute the CRC from the payload bits and compare this with the CRC that was received. A mismatch indicates that an error occurred.

Hamming distance based checks If we want to detect d bit errors in an n bit word we can map every n bit word into a bigger n+d+1 bit word so that the minimum Hamming distance between each valid mapping is d+1. This way, if one receives a n+d+1 word that doesn't match any word in the mapping (with a Hamming distance x <= d+1 from any word in the mapping) it can successfully detect it as an erroneous word. Even more, d or fewer errors will never transform a valid word into another, because the Hamming distance between each valid word is at least d+1, and such errors only lead to invalid words that are detected correctly. Given a stream of m*n bits, we can detect x <= d bit errors successfully using the above method on every n bit word. In fact, we can detect a maximum of m*d errors if every n word is transmitted with maximum d errors.

Hash function Any hash function can be used as a integrity check
Hash function Any hash function can be used as a integrity check. Horizontal and vertical redundancy check Other types of redundancy check include horizontal redundancy check, vertical redundancy check and "double", "dual" or "diagonal" parity (used in RAID-DP).

Contents Introduction credit card error checking Encoding
what is a code purpose of error-correction codes Encoding naïve approach hamming codes Minimum Weight Theorem definitions proof of single error-correction Decoding list all possible messages using vectors syndrome Conclusion perfect codes

Detect Error On Credit Card

Formula for detecting error
Let d2, d4, d6, d8, d10, d12, d14, d16 be all the even values in the credit card number. Let d1, d3, d5, d7, d9, d11, d13, d15 be all the odd values in the credit card number. Let n be the number of all the odd digits which have a value that exceeds four Credit card has an error if the following is true: (d1 + d3 + d5 + d7 + d9 + d11 + d13 + d15) x 2 + n + (d2 + d4 + d6 + d8 + d10 + d12 + d14 + d16) 0 mod(10)

Detect Error On Credit Card
… d15 d16

Now the test (4 + 4 + 8 + 1 + 3 + 5 + 7 + 9) = 41
( ) = 41 ( ) x = 61 = 102 mod (10) = 2 3

Credit Card Summary The test performed on the credit card number is called a parity check equation. The last digit is a function of the other digits in the credit card. This is how credit card numbers are generated by Visa and Mastercard. They start with an account number that is 15 digits long and use the parity check equation to find the value of the 16th digit. “This method allows computers to detect 100% of single-position errors and about 98% of other common errors” (For All Practical Purposes p. 354).

What is a code? A code is defined as an n-tuple of q elements. Where q is any alphabet. Ex n=4, q={1,0} Ex n=16, q={0,1,2,3,4,5,6,7,8,9} Ex. (a,b,c,d,e) n=5, q={a,b,c,d,e,…,y,z} The most common code is when q={1,0}. This is known as a binary code.

The purpose A message can become distorted through a wide range of unpredictable errors. Humans Equipment failure Lighting interference Scratches in a magnetic tape

Why error-correcting code?
To add redundancy to a message so the original message can be recovered if it has been garbled. e.g. message = 10 code =

Send a message Message Encoder Channel Decoder Message 10 101010 noise
001010 10

Encoding Naïve approach Hamming codes

Take Naïve approach Append the same message multiple times. Then take the value with the highest average. Message:= 1001 Encode:= Channel:= Decode: = a1 = Average(1,1,0,1) = 1 a2 = Average(0,0,0,0) = (a1,a2,a3,a4)

Hamming [7,4] Code The seven is the number of digits that make the code. E.g The four is the number of information digits in the code.

Hamming [7,4] Encoding Encoded with a generator matrix. All codes can be formed from row operations on matrix. The code generator matrix for this presentation is the following:

Hamming [7,4] Codes Codes Possible codes 1000011 0100101 0010110
Codes Possible codes

Minimum Weight Theorem
Definitions Proof of Theorem

Definitions The weight of a code is the number of nonzero components it contains. e.g. wt( ) = 3 The minimum weight of Hamming codes is the weight of the smallest nonzero vector in the code. e.g d(G)= 3

Definitions The distance between two codes u and v is the number of positions which differ e.g. u=(1,0,0,0,0,1,1) v=(0,1,0,0,1,0,1) dist(u,v) = 4 Another definition of distance is wt(u – v) = dist(u,v).

Definitions For any u, v, and w in a space V, the following three conditions hold:

Definitions The sphere of radius r about a vector u is defined as:
(0,0,0,0,0,1,1) e.g. u=(1,0,0,0,0,1,1) (1,1,0,0,0,1,1) (1,0,1,0,0,1,1) (1,0,0,1,0,1,1) (1,0,0,0,1,1,1) (1,0,0,0,0,0,1) (1,0,0,0,0,0,1)

Minimum Weight Theorem
If d is the minimum weight of a code C, then C can correct t = [(d – 1)/2] or fewer errors, and conversely.

Proof Want to prove that spheres of radius t = [(d – 1)/2] about codes are disjoint. Suppose for contradiction that they are not. Let u and w be distinct vectors in C, and assume that u v w

Proof By triangle inequality v u w

Proof Since spheres of radius t = [(d – 1)/2] so and this gives
But since We have a contradiction. Showing the sphere of radius t about codes are disjoint.

Result of Theorem Since d(G) = 3 then for t = [(3 – 1)/2] = 1 or fewer errors, the received code is in a disjoint sphere about a unique code word.

Decoding list all possible messages using vectors syndrome

List all messages This is done by generating a list of all the possible messages. For something small like the Hamming [7,4] codes the task is feasible, but for codes of greater length it is not. An example of a list is as follows: Code words … Other Received Words … … …

List all messages For example, if the received code was then it would be decoded to from the list. Code words … Other Received Words … … …

Vector Decoding Let a:=(0,0,0,1,1,1,1), b:=(0,1,1,0,0,1,1), and c:=(1,0,1,0,1,0,1). If then inner product =

Vector Decoding Correct errors by taking inner product of received vector u by a, b, c. We get e.g. recall: a:=(0,0,0,1,1,1,1), b:=(0,1,1,0,0,1,1), and c:=(1,0,1,0,1,0,1). Message Encoder Channel Decoder Message 1001 noise ? Error at 100 = digit 4. Decode to and message equals 1001

syndrome Decodes without having to derive decoding vectors.
In addition to decoding Hamming [7,4] it can decode other codes More feasible than a list of messages

syndrome The cosets of C are determined by Some facts about cosets:
Every coset of C has the same number of elements as C does Any two cosets are either disjoint or identical V is the union of all cosets of C C has cosets

syndrome A Coset leader is the vector with the minimum weight in the coset. The parity check matrix is found by solving the generator matrix for

syndrome The first step is to create a list of syndromes corresponding the coset leaders. The syndrome of each vector y is found by When a code is received, the syndrome is computed and compared to the list of syndromes. Let the coset leader to the syndrome by e. Finally the code is decoded to x = y – e.

Syndrome example Note that G=(I | A) and H = ( | I).

Syndrome example Let x:= 1001100 be the original message
Encoder Channel Decoder Message 1001 noise ? Compute the syndrome of the received code

Conclusion A code of minimum weight d is called perfect if all the vectors in V are contained in the sphere of radius t = [(d – 1)/2] about the code-word. The Hamming [7,4] code has eight vectors of sphere of radius one about each code-word, times sixteen unique codes. Therefore, the Hamming [7,4] code with minimum weight 3 is perfect since all the vectors (128) are contained in the sphere of radius 1.

NOISE-RESISTANT CODING ● Coding Concepts and Technologies of Noise-resistant Coding It's hard to work in a noisy room as it makes it harder to think. Work.

Similar presentations

Presentation on theme: "NOISE-RESISTANT CODING ● Coding Concepts and Technologies of Noise-resistant Coding It's hard to work in a noisy room as it makes it harder to think. Work."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

NOISE-RESISTANT CODING ● Coding Concepts and Technologies of Noise-resistant Coding It's hard to work in a noisy room as it makes it harder to think. Work.

Similar presentations

Presentation on theme: "NOISE-RESISTANT CODING ● Coding Concepts and Technologies of Noise-resistant Coding It's hard to work in a noisy room as it makes it harder to think. Work."— Presentation transcript:

Similar presentations

About project

Feedback