Networking Seminar Network Information Flow R. Ahlswede, N. Cai, S.-Y. R. Li, and R. W. Yeung. Network Information Flow. IEEE Transactions on Information Theory, 46(4):1204–1216, (Referred to as this paper throughout the presentation) Presented by Bo Ji

Outline Introduction Max-Flow Min-Cut Theorem Main Result Simple optimal codes Multiple sources case Discussion and Conclusion Some of the slides are from Prof. Chih-Chun & Communications and Multimedia

Outline Introduction Max-Flow Min-Cut Theorem Main Result Simple optimal codes Multiple sources case Discussion and Conclusion

Introduction In existing computer networks, each node functions as a switch either relays or replicates information received. (Store and forward) However, from the information-theoretic point of view, there is no reason to restrict the function of a node to that of a switch. Rather, a node can function as an encoder in the sense that it receives information from all the input links, encodes, and sends information to all the output links. (Store, code and forward) We refer to coding at a node in a network as network coding. Basic idea of network coding - Coding at intermediate nodes

Introduction Contributions: – Introduce a new class of problems called network information flow. – Obtained a simple characterization of the admissible coding rate region for single-source problem. (Max-Flow Min-Cut Theorem for network information flow) – Show that o the traditional technique for multicasting in a computer network in general is not optimal. o Rather, we should think of information as being “diffused” through the network from the source to the sinks by means of network coding. – Show that very simple optimal codes do exist for certain networks. (Actually, Li, Yeung and Cai [1] have devised a systematic procedure to construct linear codes for acyclic networks. ) [1] Ref: S. Y. R. Li, R. W. Yeung, and N. Cai. Linear Network Coding. IEEE Transactions on Information Theory, 49(2):371 – 381, 2003.

Outline Introduction Max-Flow Min-Cut Theorem Main Result Simple optimal codes Multiple sources case Discussion and Conclusion

Max-Flow Problem

Ford, Fulkerson

Notations

Properties of Flow

Max-Flow

The Capacity of A Cut

Min-Cut The cut with the minimum value of capacity in G.

Example

Max-Flow Min-Cut Theorem

Unicast Case: – MinCut Bound, Menger’s Theorem and Path Packing. Broadcast Case: – Edmond’s Theorem and Spanning Tree Packing. Multicast Case: – Steiner Tree Packing and Non-Achievability of MinCut Bound. – Unlike the broadcast case, the upper bound may not be achievable by routing information through a set of edge-disjoint trees. Is there any other way being able to achieve the upper bound?

A single-level diversity coding system The graph representing the coding system Let X 1,…, X i be mutually independent information sources and the information rate of X 1,…,X i is denoted by h 1,…, h i. We drop the subscripts of X 1 and h 1, since there is only one- single source. Let r i be the coding rate of encoder i. In order for a decoder to reconstruct X, it is necessary that the sum of the coding rates of the encoder accessible by this decoder is at least h.

Max-Flow in Multicast

Max-Flow Min-Cut Theorem Conjecture 1: Let G = (V,E) be a graph with source s and sinks t 1,……,t L, and the capacity of an edge (i, j) be denoted by R ij. Then (R,h,G) is admissible if and only if the values of a max-flow from s to t l, l = 1,……,L are greater than or equal to h, the rate of the information source. The spirit of this conjecture resembles that of the celebrated Max-flow Min-cut Theorem in graph theory[2]. Illustrate Conjecture 1 by a few examples [2] Ref : B. Bollobas, Graph Theory, An Introductory Course. New York: Springer-Verlag, 1979.

A one-source one-sink network (L=1) CapacityMax-Flow

A one-source two-sink network without coding (L=2) h ≤ min(5,6)

A one-source two-sink network with coding (L=2)

A one-source three-sink network (L=3) Quantified advantage: Save bandwidth: 10% Increase throughput: 1/3

Outline Introduction Max-Flow Min-Cut Theorem Main Result Simple optimal codes Multiple sources case Discussion and Conclusion

Main Result Let us consider a block code of length n. We assume that x, the value assumed by X, is obtained by selecting an index from a set Ω with uniform distribution. The elements in Ω are called messages. For (i,j) є E, node i can send information to node j which depends only on the information previously received by node i. The paper confines its discussion to a class of block codes, called the α-code.

α-Code An on a graph G is defined by the following components:

Construction of α-Code At the beginning of the coding session, the value of X is available to node s. In the coding session, there are K transactions which take place in chronological order, where each transaction refers to a node sending information to another node. In the kth transaction, node u(k) encodes according to f k and sends an index in A k to node v(k). The domain of f k is the information received by node u(k) so far, and we distinguish two cases: – If u(k) = s, the domain of f k is Ω. – If u(k) ≠ s, Q k gives the indices of all previous transactions for which information was sent to node u(k), so the domain of f k is ∏ k’є Qk A k’ The set T ij gives the indices of all transactions for which information is sent from node i to node j, so η ij is the number of possible index-tuples that can be sent from node i to node j during the coding session. Finally, W l gives the indices of all transactions for which information is sent to t l, and g l is the decoding function at t l.

Theorem 1 Theorem 1:

Outline Introduction Max-Flow Min-Cut Theorem Main Result Simple optimal codes Multiple sources case Discussion and Conclusion

An Example Show that very simple optimal codes do exist for certain cyclic networks. (A kind of convolutional code)

A Simple Optimal Coding Scheme Show a coding scheme that can multicast {x 0 (k), x 1 (k), x 2 (k)} from the source to the sinks. x l (k) = 0 for k ≤ 0. At time k ≥ 1, information transactions occur in the following order:

Outline Introduction Max-Flow Min-Cut Theorem Main Result Simple optimal codes Multiple sources case Discussion and Conclusion

Multiple Sources The multisource problem is not a trivial extension of the single-source problem, and it is extremely difficult in general. A multilevel diversity coding systemThe graph G representing the coding system

Multiple Sources Assume the sources X 1 and X 2 are independent, and h 1 =h 2 =1. For any admissible coding rate triple (r 1,r 2,r 3 ), for i = 1,2,3, we can write where are the subrates associated with sources X 1 and X 2 respectively. X 1 is multicast to all the decoders, and X 2 is multicast to Decoder 2,3,4. So we have following constraints:

Multiple Sources [3] shows that the rate triple (1,1,1) is admissible, but it cannot be decomposed into two sets of subrates as prescribed above. Therefore, coding by superposition is not optimal in general, even when the two information sources are generated at the same node. How to characterize multilevel diversity coding systems for which coding by superposition is always optimal is still an open problem. Although the multisource problem in general is extremely difficult, there exist special cases which can be readily solved by the results for the single-source problem (Such as video-conferencing scenario). [3] Ref: R.W. Yeung, “Multilevel diversity coding with distortion,” IEEE Trans. Inform. Theory, vol. 41, pp. 412–422, Mar

Outline Introduction Max-Flow Min-Cut Theorem Main Result Simple optimal codes Multiple sources case Discussion and Conclusion

Conclusion have characterized the admissible coding rate region of the single-source problem. result can be regarded as the Max-flow Min-cut Theorem for network information flow. the discussion is based on a class of block codes called α-codes. Therefore, it is possible that the result can be enhanced by considering more general coding schemes. prove that probabilistic coding does not improve performance. The problem becomes more complicated when there are more than one source.

Conclusion The most important contribution of this paper is to show that – the traditional technique for multicasting in a computer network in general is not optimal. – Rather, we should think of information as being “diffused” through the network from the source to the sinks by means of network coding. This is a new concept in multicasting in a point- to-point network which may have significant impact on future design of switching systems.

Interesting Problems by imposing the constraint that network coding is not allowed, i.e., each node functions as a switch in existing computer networks, whether a rate tuple R is admissible? under what condition can optimality be achieved without network coding? Besides, the problem of representing codes in graphs has received much attention. ……

Discussion Question: – How to identify when network coding solution is needed and feasible?

Two Simple Unicast Sessions Goal: X1: S1  t1 X2: S2  t2 When can we send X1 and X2 simutaneously? (2-2 case) Ref: C. Wang and N B. Shroff, " Beyond the Butterfly - A Graph-Theoretic Characterization of the Feasibility of Network Coding with Two Simple Unicast Sessions, " IEEE International Symposium on Information Theory, Nice, France, June 2007." Beyond the Butterfly - A Graph-Theoretic Characterization of the Feasibility of Network Coding with Two Simple Unicast Sessions, "

Two Simple Multicast Sessions It can be generalized to two simple multicast sessions (2-m case) – C.-C. Wang, N.B. Shroff, “Intersession Network Coding for Two Simple Multicast Sessions,” in 45-th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, September 26 – 28, 2007

Two Simple Multicast Sessions Settings

Two Simple Multicast Sessions Main Theorem

Further… A more general characterization for problems with more than two multicast sessions is still an on-going research.

Thank you ! !