Introduction to Probabilities Fall 2010 Dept. Electrical Engineering National Tsing Hua University 劉奕汶

2 What is probability? Literally, how probable an event is to occur. –We live in a random world –Relative-frequency interpretation 機率／概率／或然率 –This interpretation is problematic Involved law of large number Not all experiments could be repeated Not all repeating processes have convergent frequency –Axiomatic approach

3 A bit of History 3500 B.C., Egyptians used bones to gamble –Since then, dice, playing cards, mahjong, etc th centuries: Italy (Galilei et al.) th centuries: Western-central Europe –Pascal, Fermat, Laplace, Poisson, Gauss –Huygens ( ) On Calculations in Games of Chance th : Russia –1900: Hilbert’s 23 problems –1933: Kolmogorov: probability theory axiomatized

4 Probability in EE/CS Signal processing –“Signal” = Random Process –Random because of noise and uncertainty Machine learning –Natural language processing –Pattern recognition Communication –Source coding –Channel coding –Modulation and estimation

5 Probability in Finance/Economics Investment / Gambling –Portfolio theory Advertisement / Pricing

6

7 Probability in Physics (i) Statistical mechanics –Equilibrium –Entropy and 2 nd law of thermodynamics –Definition of temperature

Probability in Physics (ii) Quantum mechanics –Schrödinger’s wave function –“Measurement makes reality” The paradox of Schrödinger’s cat Einstein’s famous comment 8

9 Probability in Biomedicine Genomics Proteomics Neuroscience Ecology Epidemiology

10 Probability and Statistics Law of Large Number Central Limit Theorem –Why Gaussian distribution is “Normal” Counter-example: stock market

11 Syllabus Textbook: S. Ghahramani, Fundamentals of Probability: with stochastic processes, 3 rd Edition –Chapters 1-3: probability space –Chapters 4-5: discrete random variables –Chapter 6: Continuous random variables –Midterm exam (35%) –Chapters 7: continuous random variables II –Chapters 8: bivariate distributions –Chapter 10-11: advanced topics (Correlations, LLN, CLT, etc) –* Measure theory and axioms of probability –Final exam (35%) A4 double-side cheat sheet permitted for both exams –6 homework assignments (30%) Office hours: Monday 5-6 pm, Rm 704B Website:

Statistics of last semester’s grades (N = 37) 期中考 : M = 51.4, SD = 7.9 期末考 : M = 49.3, SD = 10.8 總成績 : M = 78, SD = 11 –36 passed, 1 failed. –4 scored 90 or above (A+) 12