Presentation is loading. Please wait.

Presentation is loading. Please wait.

Abdallah Kassir 1. Information Theory Entropy: Conditional Entropy: Mutual Information: 2.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Abdallah Kassir 1. Information Theory Entropy: Conditional Entropy: Mutual Information: 2."— Presentation transcript:

1 Abdallah Kassir 1

2 Information Theory Entropy: Conditional Entropy: Mutual Information: 2

3 Abdallah Kassir Optimal Sensor Parameter Selection MMI: Maximum Mutual Information 3

4 Abdallah Kassir Example: 12 Coin Problem 4

5 Abdallah Kassir Problem Need to learn: Need to solve: 5

6 Abdallah Kassir Observation Model Can be learnt over many experiments Or, modelled by recognition system 6

7 Abdallah Kassir Solve argmax problem Integral difficult to compute: Discretise Or, use Monte Carlo methods to estimate Even if we can compute the MI, we also need to maximise. Local maxima possible 7

8 Abdallah Kassir Experimental Results 8 MI Max MI

9 Abdallah Kassir Experimental Results 9

10 Abdallah Kassir Experimental Results 10

Download ppt "Abdallah Kassir 1. Information Theory Entropy: Conditional Entropy: Mutual Information: 2."

Similar presentations

Nonmyopic Active Learning of Gaussian Processes An Exploration – Exploitation Approach Andreas Krause, Carlos Guestrin Carnegie Mellon University TexPoint.

Nonmyopic Active Learning of Gaussian Processes An Exploration – Exploitation Approach Andreas Krause, Carlos Guestrin Carnegie Mellon University TexPoint.
Lecture 16 Hidden Markov Models. HMM Until now we only considered IID data. Some data are of sequential nature, i.e. have correlations have time. Example:

Lecture 16 Hidden Markov Models. HMM Until now we only considered IID data. Some data are of sequential nature, i.e. have correlations have time. Example:
Header, Specification, Body Input Parameter List Output Parameter List

Header, Specification, Body Input Parameter List Output Parameter List
SE503 Advanced Project Management Dr. Ahmed Sameh, Ph.D. Professor, CS & IS Project Uncertainty Management.

SE503 Advanced Project Management Dr. Ahmed Sameh, Ph.D. Professor, CS & IS Project Uncertainty Management.
Learning Inhomogeneous Gibbs Models Ce Liu

Learning Inhomogeneous Gibbs Models Ce Liu
Maximum Likelihood-Maximum Entropy Duality : Session 1 Pushpak Bhattacharyya Scribed by Aditya Joshi Presented in NLP-AI talk on 14 th January, 2014.

Maximum Likelihood-Maximum Entropy Duality : Session 1 Pushpak Bhattacharyya Scribed by Aditya Joshi Presented in NLP-AI talk on 14 th January, 2014.
2004/11/161 A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition LAWRENCE R. RABINER, FELLOW, IEEE Presented by: Chi-Chun.

2004/11/161 A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition LAWRENCE R. RABINER, FELLOW, IEEE Presented by: Chi-Chun.
Efficient Informative Sensing using Multiple Robots

Efficient Informative Sensing using Multiple Robots
Decentralised Coordination of Continuously Valued Control Parameters using the Max-Sum Algorithm Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick.

Decentralised Coordination of Continuously Valued Control Parameters using the Max-Sum Algorithm Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick.
Parameter Estimation: Maximum Likelihood Estimation Chapter 3 (Duda et al.) – Sections 3.1-3.2 CS479/679 Pattern Recognition Dr. George Bebis.

Parameter Estimation: Maximum Likelihood Estimation Chapter 3 (Duda et al.) – Sections CS479/679 Pattern Recognition Dr. George Bebis.
Bayesian estimation Bayes’s theorem: prior, likelihood, posterior

Bayesian estimation Bayes’s theorem: prior, likelihood, posterior
Mean-Shift Algorithm and Its Application Bohyung Han

Mean-Shift Algorithm and Its Application Bohyung Han
Problem Statement Given a control system where components, i.e. plant, sensors, controllers, actuators, are connected via a communication network, design.

Problem Statement Given a control system where components, i.e. plant, sensors, controllers, actuators, are connected via a communication network, design.
Maximum Likelihood. The likelihood function is the simultaneous density of the observation, as a function of the model parameters. L(  ) = Pr(Data| 

Maximum Likelihood. The likelihood function is the simultaneous density of the observation, as a function of the model parameters. L(  ) = Pr(Data| 
CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference (Sec. )

CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference (Sec. )
Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.

Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.
CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference.

CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference.
Minimum Error Rate Training in Statistical Machine Translation By: Franz Och, 2003 Presented By: Anna Tinnemore, 2006.

Minimum Error Rate Training in Statistical Machine Translation By: Franz Och, 2003 Presented By: Anna Tinnemore, 2006.
Nonmyopic Active Learning of Gaussian Processes An Exploration – Exploitation Approach Andreas Krause, Carlos Guestrin Carnegie Mellon University TexPoint.

Nonmyopic Active Learning of Gaussian Processes An Exploration – Exploitation Approach Andreas Krause, Carlos Guestrin Carnegie Mellon University TexPoint.
Reinforcement Learning: Learning algorithms Yishay Mansour Tel-Aviv University.

Reinforcement Learning: Learning algorithms Yishay Mansour Tel-Aviv University.

Similar presentations

Ads by Google