Abdallah Kassir 1. Information Theory Entropy: Conditional Entropy: Mutual Information: 2.

Slides:

Advertisements

Similar presentations

Nonmyopic Active Learning of Gaussian Processes An Exploration – Exploitation Approach Andreas Krause, Carlos Guestrin Carnegie Mellon University TexPoint.

Advertisements

Lecture 16 Hidden Markov Models. HMM Until now we only considered IID data. Some data are of sequential nature, i.e. have correlations have time. Example:

Header, Specification, Body Input Parameter List Output Parameter List

SE503 Advanced Project Management Dr. Ahmed Sameh, Ph.D. Professor, CS & IS Project Uncertainty Management.

Learning Inhomogeneous Gibbs Models Ce Liu

Maximum Likelihood-Maximum Entropy Duality : Session 1 Pushpak Bhattacharyya Scribed by Aditya Joshi Presented in NLP-AI talk on 14 th January, 2014.

2004/11/161 A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition LAWRENCE R. RABINER, FELLOW, IEEE Presented by: Chi-Chun.

Efficient Informative Sensing using Multiple Robots

Decentralised Coordination of Continuously Valued Control Parameters using the Max-Sum Algorithm Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick.

Parameter Estimation: Maximum Likelihood Estimation Chapter 3 (Duda et al.) – Sections CS479/679 Pattern Recognition Dr. George Bebis.

Bayesian estimation Bayes’s theorem: prior, likelihood, posterior

Mean-Shift Algorithm and Its Application Bohyung Han

Problem Statement Given a control system where components, i.e. plant, sensors, controllers, actuators, are connected via a communication network, design.

Maximum Likelihood. The likelihood function is the simultaneous density of the observation, as a function of the model parameters. L(  ) = Pr(Data| 

CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference (Sec. )

Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.

CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference.

Minimum Error Rate Training in Statistical Machine Translation By: Franz Och, 2003 Presented By: Anna Tinnemore, 2006.

Nonmyopic Active Learning of Gaussian Processes An Exploration – Exploitation Approach Andreas Krause, Carlos Guestrin Carnegie Mellon University TexPoint.

Reinforcement Learning: Learning algorithms Yishay Mansour Tel-Aviv University.

CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference.

1 Assessment of Imprecise Reliability Using Efficient Probabilistic Reanalysis Farizal Efstratios Nikolaidis SAE 2007 World Congress.

Pre-Algebra 8-5 Estimating with Percents Learn to estimate with percents.

Problem 1 Given a high-resolution computer image of a map of an irregularly shaped lake with several islands, determine the water surface area. Assume.

Likelihood probability of observing the data given a model with certain parameters Maximum Likelihood Estimation (MLE) –find the parameter combination.

1 Robust HMM classification schemes for speaker recognition using integral decode Marie Roch Florida International University.

Conference Paper by: Bikramjit Banerjee University of Southern Mississippi From the Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence.

A performance analysis of multicore computer architectures Michel Schelske.

1 Theoretical Physics Experimental Physics Equipment, Observation Gambling: Cards, Dice Fast PCs Random- number generators Monte- Carlo methods Experimental.

November 1, 2012 Presented by Marwan M. Alkhweldi Co-authors Natalia A. Schmid and Matthew C. Valenti Distributed Estimation of a Parametric Field Using.

Simple Linear Regression. The term linear regression implies that  Y|x is linearly related to x by the population regression equation  Y|x =  +  x.

Optimum design of optical filters and deposition monitoring methods Dimitris Kouzis - Loukas Supervisor: S. Maltezos Support: M. Fokitis.

Maximum a posteriori sequence estimation using Monte Carlo particle filters S. J. Godsill, A. Doucet, and M. West Annals of the Institute of Statistical.

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Supervised Learning Resources: AG: Conditional Maximum Likelihood DP:

Maximum Likelihood Estimation

Expectation-Maximization (EM) Algorithm & Monte Carlo Sampling for Inference and Approximation.

Landmark-Based Speech Recognition: Spectrogram Reading, Support Vector Machines, Dynamic Bayesian Networks, and Phonology Mark Hasegawa-Johnson

Lecture 3: MLE, Bayes Learning, and Maximum Entropy

Lecture Fall 2001 Controlling Animation Boundary-Value Problems Shooting Methods Constrained Optimization Robot Control.

1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,..., sN Si Sj.

1 Exercise 1. (a) Find all optimal sequences for the scheduling problem 1 ||  w j C j with the following jobs. (b) Determine the effect of a change in.

Mutual Information Brian Dils I590 – ALife/AI

Information Bottleneck versus Maximum Likelihood Felix Polyakov.

Geraint Palmer Optimisation using Linear Programming.

Problem 1: Service System Capacity CustomersServed Customers Queue Server Problem: Can a server taking an average of x time units meet the demand? Solution.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Bayes Rule Mutual Information Conditional.

Dan Roth University of Illinois, Urbana-Champaign 7 Sequential Models Tutorial on Machine Learning in Natural.

CS479/679 Pattern Recognition Dr. George Bebis

Monte Carlo Methods Some example applications in C++

Exercise on the concept of Likelihood

Problem 1: Service System Capacity

Bayesian estimation Bayes’s theorem: prior, likelihood, posterior

High Performance Computing and Monte Carlo Methods

An Iterative Approach to Discriminative Structure Learning

LECTURE 10: EXPECTATION MAXIMIZATION (EM)

J. Zhu, A. Ahmed and E.P. Xing Carnegie Mellon University ICML 2009

7. Scientific Method- = The systematic approach to problem solving that involves observation and experimentation.

Unfolding Problem: A Machine Learning Approach

Acoustic Array Systems: Theory, Implementation and Application

Probabilistic Assessment

The Mutual Information-based Oracle

LECTURE 23: INFORMATION THEORY REVIEW

A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models Jeff A. Bilmes International.

Introduction CSE 541.

A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models Jeff A. Bilmes International.

Maximum Likelihood Estimation (MLE)

Uncertainty Propagation

Presentation transcript:

Abdallah Kassir 1

Information Theory Entropy: Conditional Entropy: Mutual Information: 2

Abdallah Kassir Optimal Sensor Parameter Selection MMI: Maximum Mutual Information 3

Abdallah Kassir Example: 12 Coin Problem 4

Abdallah Kassir Problem Need to learn: Need to solve: 5

Abdallah Kassir Observation Model Can be learnt over many experiments Or, modelled by recognition system 6

Abdallah Kassir Solve argmax problem Integral difficult to compute: Discretise Or, use Monte Carlo methods to estimate Even if we can compute the MI, we also need to maximise. Local maxima possible 7

Abdallah Kassir Experimental Results 8 MI Max MI

Abdallah Kassir Experimental Results 9

Abdallah Kassir Experimental Results 10