1. Error-Correcting Code
Message
Encoding Algorithm cn
Encoding
Transmission
Received
Decoding Algorithm
Message

2. Performance Measures Encoding and decoding times Decoding guarantee
How many errors can you recover from?

3. Encoding Process
Encoding time is linear
in number of edges.

4. Decoding Process Correct one layer at a time.
WLOG assume previous layer corrected.
Bipartite
graph
message
bits
check
bits
= correct bits
= erroneous bits

5. Decoding Hard decision decoding [Gallager 62]
No probabilities in the state, votes.
All votes are 1 bit, 0/1.
Goal: votes converge to correct values. 1
From my other check bits,
I vote that I am a 0.
From my other message bits,
I vote you are a 1.

6. Mental Picture: A Regular Tree

7. Example: Regular Graphs
Strategy: message bit m will send received bit to c unless all other check bits say otherwise.
Assume initial message all zeroes. m 1 c

8. Round 0, initial received bitsm 1 1
Initial vote = Bit received

9. Round 1, message to check c 1 m 1 1 1

10. Round 1, check to message c m 1 1 1

11. Round 2, message to check c 1 m 1 1

12. Round 2, check to message c m 1 1

13. Threshold Generalization
Strategy: message bit m will send initial vote to c unless at least b other check bits say otherwise.
Optimize the b value for each round.
Initially, b begins high.
Strategy eventually becomes: message bit passes on decision of majority of other check bits (or initial vote if tied).

14. Irregular Graphs Same algorithm
Strategy: message bit m with degree j will send initial vote to c unless the discrepancy (|# 0 votes - #1 votes|) is at least b.
Value b is independent of j!
Vary b according to round as before.

15. Degree of Right Nodes
The higher the degree on the right, the less information
it yields to the nodes on the left.

16. High Degree Nodes on Left
Left nodes of high degree get information from many right
nodes; quickly their beliefs converge to their correct value.

17. Low Degree Nodes on Left
= high degree
Left nodes of low degree get less information; however,
as high degree nodes settle on a value, their info. grows
more accurate, and their beliefs converge correctly.

18. Irregular Decoding High degree nodes get corrected FIRST.
Low degree nodes get corrected
LATER.

19. Analysis
Assume graph looks like a tree (for constant number of levels).
Derive recursive formula for probability pr after r rounds.
Show pr goes to 0. c 1 m

20. Finding Irregular Graphs
Our analysis yields equations
Given graph description, can find percentage of errors that can be corrected
Goal: find graphs with good performance.
Methodology: use equations to design a linear program
Given right hand side, find the best left hand side
Experimented with just one degree on right

21. Experimental Results Best regular graph gets 5.17% asymptotically
Analysis gives Code 10: 5.78%, Code 14: 6.27%
Shannon bound is 11.1%

22. A Better Algorithm Belief propagation.
Send probabilities, instead of 0/1 votes.
Nodes keep track of their conditional probabilities of being a 0/1.
0.78
0.83
From my other check bits,
I believe my probability
of being 0 is 0.78
From my other message bits,
I believe your probability
of being 0 is 0.83

23. One Level of Belief Propagation
message bit
check bits
must be correct!
message bits
A node R looks at neighboring check bits, and their
neighboring message bits. From this it determines
its conditional probability of being correct.

24. Belief Propagation R If R knows the values of the complete tree down
message bit
check bits
If R knows the values
of the complete tree down
L levels, it can determine
its conditional probability
of being correct.
message bits
check bits
message bits
check bits

25. Belief Propagation : Implementation
These conditional probabilities are easy to compute.
Nodes update themselves in parallel over rounds.
Each round equivalent to exploring one level further in graph.
Process stops when current solution is consistent, or fails after preset number of rounds.

26. Results: Left/Right = 2/1

27. Time to Decode
Time is proportional to product of number of edges and number of rounds
Irregular graphs have more edges, but often succeed in fewer rounds.
Current times (for 2/1 ratio) at peak about:
1 second for 2K left nodes
10-25 seconds for 20K left nodes
seconds for 100K left nodes.