Daphne Koller Message Passing Loopy BP and Message Decoding Probabilistic Graphical Models Inference.

Slides:

Advertisements

Similar presentations

Another question consider a message (sequence of characters) from {a, b, c, d} encoded using the code shown what is the probability that a randomly chosen.

Advertisements

Chapter 8 Channel Capacity. bits of useful info per bits actually sent Change in entropy going through the channel (drop in uncertainty): average uncertainty.

Error Correction and LDPC decoding CMPE 691/491: DSP Hardware Implementation Tinoosh Mohsenin 1.

Modern Digital and Analog Communication Systems Lathi Copyright © 2009 by Oxford University Press, Inc. C H A P T E R 15 ERROR CORRECTING CODES.

Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California.

Belief Propagation by Jakob Metzler. Outline Motivation Pearl’s BP Algorithm Turbo Codes Generalized Belief Propagation Free Energies.

Belief Propagation on Markov Random Fields Aggeliki Tsoli.

Chapter 6 Information Theory

Cellular Communications

Near Shannon Limit Performance of Low Density Parity Check Codes

Turbo Codes Azmat Ali Pasha.

Sep 06, 2005CS477: Analog and Digital Communications1 Introduction Analog and Digital Communications Autumn

Code and Decoder Design of LDPC Codes for Gbps Systems Jeremy Thorpe Presented to: Microsoft Research

2/28/03 1 The Virtues of Redundancy An Introduction to Error-Correcting Codes Paul H. Siegel Director, CMRR University of California, San Diego The Virtues.

Generalized Communication System: Error Control Coding Occurs In Right Column. 6.

15-853Page :Algorithms in the Real World Error Correcting Codes I – Overview – Hamming Codes – Linear Codes.

The Role of Specialization in LDPC Codes Jeremy Thorpe Pizza Meeting Talk 2/12/03.

Optimizing LDPC Codes for message-passing decoding. Jeremy Thorpe Ph.D. Candidacy 2/26/03.

Noise, Information Theory, and Entropy

Channel Polarization and Polar Codes

Introduction to Information Theory

Lecture 3 Outline Announcements: No class Wednesday Friday lecture (1/17) start at 12:50pm Review of Last Lecture Communication System Block Diagram Performance.

Page 1 of 37 Density Evolution, Capacity Limits, and the "5k" Code Result (L. Schirber 11/22/11) The Density Evolution (DE) algorithm calculates a "threshold.

24 November, 2011National Tsin Hua University, Taiwan1 Mathematical Structures of Belief Propagation Algorithms in Probabilistic Information Processing.

Lecture 7: Channel capacity and multiplexing Anders Västberg

Review of modern noise proof coding methods D. Sc. Valeri V. Zolotarev.

Fundamentals of Digital Communication 2 Digital communication system Low Pass Filter SamplerQuantizer Channel Encoder Line Encoder Pulse Shaping Filters.

Channel Coding Part 1: Block Coding

Wireless Mobile Communication and Transmission Lab. Theory and Technology of Error Control Coding Chapter 7 Low Density Parity Check Codes.

Channel Capacity

III. Turbo Codes.

Redundancy The object of coding is to introduce redundancy so that even if some of the information is lost or corrupted, it will still be possible to recover.

Andrea Montanari and Ruediger Urbanke TIFR Tuesday, January 6th, 2008 Phase Transitions in Coding, Communications, and Inference.

Physical Fuctuomatics (Tohoku University) 1 Physical Fluctuomatics Applied Stochastic Process 1st Review of probabilistic information processing Kazuyuki.

Miss Insah Bhurtah Main Supervisor: Prof. K.M.S. Soyjaudah Associate Supervisor: Dr C Catherine.

Belief Propagation Revisited Adnan Darwiche. Graphical Models Battery Age Alternator Fan Belt Battery Charge Delivered Battery Power Starter Radio LightsEngine.

Shannon Theory Risanuri Hidayat Reference L L Peterson and B S Davie,

Introduction to Digital and Analog Communication Systems

Introduction of Low Density Parity Check Codes Mong-kai Ku.

Computer Vision – Compression(1) Hanyang University Jong-Il Park.

Lecture 2 Outline Announcements: No class next Wednesday MF lectures (1/13,1/17) start at 12:50pm Review of Last Lecture Analog and Digital Signals Information.

Coding Theory. 2 Communication System Channel encoder Source encoder Modulator Demodulator Channel Voice Image Data CRC encoder Interleaver Deinterleaver.

Coding Theory Efficient and Reliable Transfer of Information

Tutorial 8 Mobile Communications Netrowks. Prob.1 Construct 4 Walsh (Orthogonal) codes for 4 different users by two methods. Assume that 4 users transmit.

Daphne Koller Message Passing Belief Propagation Algorithm Probabilistic Graphical Models Inference.

Channel Coding Binit Mohanty Ketan Rajawat. Recap…  Information is transmitted through channels (eg. Wires, optical fibres and even air)  Channels are.

Physical Fuctuomatics (Tohoku University) 1 Physical Fluctuomatics 1st Review of probabilistic information processing Kazuyuki Tanaka Graduate School of.

Part 1: Overview of Low Density Parity Check(LDPC) codes.

1 Lecture 7 System Models Attributes of a man-made system. Concerns in the design of a distributed system Communication channels Entropy and mutual information.

Postacademic Interuniversity Course in Information Technology – Module C1p1 Chapter 4 Communications, Theory and Media.

Tufts University. EE194-WIR Wireless Sensor Networks. February 17, 2005 Increased QoS through a Degraded Channel using a Cross-Layered HARQ Protocol Elliot.

Raptor Codes Amin Shokrollahi EPFL. BEC(p 1 ) BEC(p 2 ) BEC(p 3 ) BEC(p 4 ) BEC(p 5 ) BEC(p 6 ) Communication on Multiple Unknown Channels.

Source Encoder Channel Encoder Noisy channel Source Decoder Channel Decoder Figure 1.1. A communication system: source and channel coding.

Daphne Koller Overview Conditional Probability Queries Probabilistic Graphical Models Inference.

Daphne Koller Overview Maximum a posteriori (MAP) Probabilistic Graphical Models Inference.

Error Detecting and Error Correcting Codes

Information Theory & Coding for Digital Communications Prof JA Ritcey EE 417 Source; Anderson Digital Transmission Engineering 2005.

DIGITAL COMMUNICATION. Introduction In a data communication system, the output of the data source is transmitted from one point to another. The rate of.

Channel Coding and Error Control 1. Outline Introduction Linear Block Codes Cyclic Codes Cyclic Redundancy Check (CRC) Convolutional Codes Turbo Codes.

Institute for Experimental Mathematics Ellernstrasse Essen - Germany DATA COMMUNICATION introduction A.J. Han Vinck May 10, 2003.

UNIT –V INFORMATION THEORY EC6402 : Communication TheoryIV Semester - ECE Prepared by: S.P.SIVAGNANA SUBRAMANIAN, Assistant Professor, Dept. of ECE, Sri.

Chapter 4: Information Theory. Learning Objectives LO 4.1 – Understand discrete and continuous messages, message sources, amount of information and its.

8 Coding Theory Discrete Mathematics: A Concept-based Approach.

VLSI Architectures For Low-Density Parity-Check (LDPC) Decoders

Coordination and computation over wireless networks

Optimizing LDPC Codes for message-passing decoding.

CSCD 433 Network Programming

Sampling Theorems- Nyquist Theorem and Shannon-Hartley Theorem

Grids A1,1 A1,2 A1,3 A1,4 A2,1 A2,2 A2,3 A2,4 A3,1 A3,2 A3,3 A3,4 A4,1 A4,2 A4,3 A4,4.

Presentation transcript:

Daphne Koller Message Passing Loopy BP and Message Decoding Probabilistic Graphical Models Inference

Daphne Koller Message Coding & Decoding U 1, …, U k Encoder X 1, …, X n Noisy Channel Y 1, …, Y n Decoder V 1, …, V k Rate = k/n Bit error rate =

Daphne Koller Channel Capacity Noisy Channel ? Binary symmetric channel Binary erasure channel capacity = 0.531capacity = X capacity =

Daphne Koller Shannon’s Theorem McEliece Rate in multiples of capacity Bit error probability

Daphne Koller How close to C can we get? Signal to noise ratio (dB) Log bit error prob McEliece

Daphne Koller Turbocodes (May 1993)

Daphne Koller How close to C can we get? Signal to noise ratio (dB) Log bit error prob. Shannon limit = McEliece

Daphne Koller Turbocodes: The Idea U 1, …, U k Noisy Channel Encoder 1 Encoder 2 Y 1 1, …, Y 1 n Y 2 1, …, Y 2 n Compute P(U | Y 1, Y 2 ) Y 1 1, …, Y 1 n Y 2 1, …, Y 2 n Decoder 1 Decoder 2 Bel(U | Y 2 )Bel(U | Y 1 )

Daphne Koller Iterations of Turbo Decoding Signal to noise ratio (dB) Log bit error prob

Daphne Koller X7X7 Y5Y5 Y7Y7 Y6Y6 X5X5 X6X6 U1U1 U4U4 U2U2 X4X4 X1X1 X2X2 Y4Y4 Y1Y1 Y2Y2 U3U3 X3X3 Y3Y3 Low-Density Parity Checking Codes Invented by Robert Gallager in 1962! original message bits parity bits

Daphne Koller Decoding as Loopy BP X7X7 Y5Y5 Y7Y7 Y6Y6 X5X5 X6X6 U1U1 U4U4 U2U2 X4X4 X1X1 X2X2 Y4Y4 Y1Y1 Y2Y2 U3U3 X3X3 Y3Y3 X 5 cluster X 7 cluster X 6 cluster U2U2 U3U3 U4U4 U1U1

Daphne Koller Decoding in Action Information Theory, Inference, and Learning Algorithms, David MacKay

Daphne Koller Turbo-Codes & LDPCs 3G and 4G mobile telephony standards Mobile television system from Qualcomm Digital video broadcasting Satellite communication systems New NASA missions (e.g., Mars Orbiter) Wireless metropolitan network standard

Daphne Koller Summary Loopy BP rediscovered by coding practitioners Understanding turbocodes as loopy BP led to development of many new and better codes – Current codes coming closer and closer to Shannon limit Resurgence of interest in BP led to much deeper understanding of approximate inference in graphical models – Many new algorithms

Daphne Koller END END END