1. Factor Graphs and the Sum-Product Algorithm
Raviteja Ganireddy
Shuaifang Zhang
Yashashvini Pakala

2. Brief Introduction Complicated global functions with many variables
The given functions are factored as a product of “local” functions
Each “local” function depends on a subset of the variables
The factorization could be visualized with a factor graph
The sum-product algorithm computes the various marginal functions derived from the global function

3. Marginal functions
Configuration space

4. Factor Graphs
Example X
Factor graph representation

5. Factor Graphs—Expression trees
Expression tree represents x(y+z)
Compute Single Marginal Function
Fig.3. Expression tree for factor graph of marginal function g1(x1) xi
Fig.5 an expression tree for a marginal function at (a) a variable node and (b) a factor node

6. Factor Graphs—Expression trees
Single Marginal Function
Fig.4. Expression tree for factor graph of marginal function g3(x3)

7. Sum-Product Algorithms
The Sum-Product Update Rule -----Compute All Marginal Functions
The message sent from a node v on an edge e is the product of the local function at v(or the unit function if v is a variable node) with all messages received at v on edges other than e, summarized for the variable associated with e

8. Sum-Product Algorithms
Example--Compute All Marginal Functions
Product of all the message functions connected to variable (Similar as a root node)

9. Factor graphs for linear codes
r x n parity check matrix

10. Wiberg Type Graphs
A particular behaviour B with both auxillary (hidden) and visible variables is said to represent a given visible behaviour C if the projection of the elements of B on the visible variables is equal to C.
Any factor graph for B is then also a factor graph for C.

11. Probabilistic Modeling
Codeword from a code C of length n With output sequence
Is proportional to

12. Probabilistic Modeling
Factor graphs represent factorization of probability density function
be joint probability mass function
If n=4, then

13. Markov chain Hidden Markov Chain

14. Forward-Backward Algorithm
Aims to compute marginal distribution for each of the hidden states given all observations in a Hidden Markov Model
Si are state variables
Ui are input variables
Xi are output variables and
Yi the output of a memoryless channel
behavior is defined by local check functions Ti

15. Forward-Backward Algorithm
Initialization : the sum-product algorithm begins at the leaf nodes Each pendant factor node sends a message to the corresponding output
Forward - Backward recursions

16. FACTOR GRAPHS WITH CYCLES
Due to cycles in the graph, an iterative algorithm with no natural termination will result.(with messages passed multiple times on a given edge)
The results of the sum product algorithm operating in a factor graph with cycles cannot in general be interpreted as exact function summaries.
Examples : Decoding of turbo codes, LDPC codes, RA codes

17. Message Passing Schedules
Messages are synchronized with a global discrete time clock with at most one message passed on any edge in any given direction at one time.
Message passing schedule in a factor graph is a specification of messages to be passed during each clock tick.
Ex: Flooding schedule, Serial schedule
Message Pending
Cycle-Free Factor Graph: Sum Product algorithm Halts
Factor graph with Cycles doesn’t terminate.
In Practice all schedules are finite schedules.

18. Forward error correction
A technique used for controlling errors in data transmission over unreliable or noisy communication channels.
The sender encodes the message in a redundant way by using an error-correcting code.
The redundancy allows the receiver to detect a limited number of errors that may occur anywhere in the message, and often to correct these errors without retransmission.
Examples: Turbo Codes, LDPC codes RA codes etc

19. Turbo Codes
Parallel concatenated convolutional code.

20. Factor Graph for Turbo Codes

21. Iterative Decoding of Turbo Codes

22. LDPC codes
Linear error correcting code, a method of transmitting message over a noisy transmission channel.
Constructed using a sparse bipartite graph
(j,k) LDPC code
Check enforces the condition that the adjacent symbols should have even overall parity.
(2,4) LDPC Code

23. RA Codes
An encoder for an RA code operates on k input bits u1, …...uk repeating each bit Q times, and permuting the result to arrive at a sequence z1, …….zkQ.
An output sequence is formed via an accumulator that satisfies x1 = z1 and xi = xi-1+ zi for i > 1.
The checks all enforce the condition that incident variables sum to zero modulo 2.

24. Factor Graph Transformations
Modify a factor graph with an inconvenient structure into a more convenient form.
Ex: Transform a factor graph with cycles into a cycle free factor graph.
CLUSTERING
Cluster nodes of like type.
If v and w are two nodes being clustered -> delete (v,w), any incident edges from the factor graph,->introduce a new node representing the pair (v,w) -> connect this new node to nodes that were neighbours of v or w in the original graph.
v(Ae ) and w(Aw ) are variables -> (v,w) (Ae * Aw )
v(Xv) and w(Xw) are local functions -> (v,w) (Xv U Xw )
Clustering nodes may eliminate cycles in the graph.

25. CLUSTERING EXAMPLE

26. Stretching Variable Nodes
n2(x) - set of nodes that can be reached from x by a path of length two in F.
y belongs to n2(x) we can pair x and y
This is called stretching transformation since we imagine node x being stretched along the path from x to y.
Stretching transformation can be used to break cycles.

27. Spanning Trees F - Connected Factor Graph T - Spanning Tree
For every variable node x of F, n(x) - set of function nodes having x as an argument.
Unique path between x and every element of n(x)
If x is stretched to all variable nodes involved in each path from x to every element of n(x) - F1 be the resultant transformed factor graph.
Every edge of F1 not in T is redundant and all such edges can be deleted from F1

28. Fast Fourier Transform
DFT - used to analyse discrete time signals.
w = (w0,...,wN-1) be a complex valued N tuple
j = (-1)½
DFT of w is
Let

29. In three steps the function wn is converted to the function Wk

30. THANK YOU