Download presentation
Published byLewis Golden Modified over 9 years ago
1
Coding Theory Efficient and Reliable Transfer of Information
Information Theory Algebraic Coding Theory
2
Information Source Encoder Communication Channel Decoder Information Sink Noise
3
What is Information
4
Information Theory Information is a decrease in uncertainty
What is uncertainty?
5
Uncertainty Number of Possibilities 2 ways 3 ways Together: 6 ways
Prefer additive measure take logarithm of number of symbols M
6
Symbols with unequal probability
7
Average uncertainty per symbol: Shannon Entropy
9
Mutual Information Change in uncertainty
Change in uncertainty in one random variable from knowledge of another
10
Molecular Information Theory
Applied to DNA Sequences Pattern of information, or label, to ribosome to start translation at given site
11
Channel Capacity Maximum rate at which information can be sent and recovered with vanishing probability of error Channel Coding Theorem
12
Source Coding Can’t Compress information below entropy
Optimal Prefix-Free Codes for Representation Code Lengths are greater than or equal to entropy of source
13
Rate Distortion Encoding bits of information per symbol bit
How many given a certain distortion (error) p 1 1-p
14
Algorithmic Information Theory
Kolmogorov Descriptive complexity of an object is the length of the shortest computer program that describes the object If equal to length of object then it is random Pi
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.