K. Shum Lecture 6 Various definitions of Probability

K. Shum Classical Probability Pierre-Simon de Laplace (1749— 1827) Théorie Analytique des Probilités (1812). Principle of Cogent Reason: “Physical symmetry implies equal probability.” Principle of Insufficient Reason: “In the absence of any prior knowledge, we must assume that the possible outcomes have equal probability.”

K. Shum Limitations of Classical Prob. The outcomes are equally likely. –There is no reason from physics that a coin or a die is fair. –Events such as “A coin landing on the rim”, “Coin lost” is rare but possible. –Biased coins or dice cannot be modeled. The sample space is finite. –Cannot assign probability to integers. –Cannot assign probability to the unit interval [0,1]. –Cannot talk about probability that a coin will turn up head eventually.

K. Shum From statistical physics… N particles, r states. Maxell-Boltzmann statistic –There are r N equally probable states. –It is shown experimentally that this does not occur in nature. Bose-Einstein statistic –The particles are indistinguishable. –There are r+n-1 C n-1 equally probable states. Fermi-Dirac statistic. –No more than one particle in a state. –There are n C r eqally probable states.

K. Shum Frequentist interpretation of probability (1920’s) Richard von Mises (1883—1953) Probability only make sense when the experiment can be repeated unlimitedly many times under the same condition. Probability of an event is the limit of relative frequency.

K. Shum Objections to relative frequency “In the long run, we are all dead.” –We cannot repeat the experiment infinitely many times. There is no ground to the existence of the limit of relative frequency. –von Mises treated the existence of limit as a postulate. –His theory of “collectives” is very complex in compare to Kolmogorov’s approach.

K. Shum Andrey Kolmogorov’s axioms Sample space Ω. Events are subsets of Ω. Probability is a function from events to numbers between 0 and 1 with the properties 1.P(E) >= 0. 2.P(Ω)=1. 3.If events A and B are disjoint, then P(A ∪ B)=P(A)+P(B). 3’. If events A k ’s are disjoint,

K. Shum A Quotation “Kolmogorov (1903—1987) was appointed a professor at Moscow University in His monograph on probability theory G rundbegriffe der Wahrscheinlichkeitsrechnung published in 1933 (translated to F oundations of the Theory of Probability in 1955) built up probability theory in a rigorous way from fundamental axioms in a way comparable with Euclid's treatment of geometry. One success of this approach is that it provides a rigorous definition of conditional expectation.”

K. Shum Kolmogorov’s approach Sample space, probability function etc. are basic building blocks of the theory, like point and line in Euclidean geometry, or mass and force in mechanics. It is an abstract mathematical model, that may refer to non-observable world. This approach does not explain the true meaning of probability, as geometers do not discuss the physical meaning of an ideal point. The axioms establish the foundation of probability as a special case in measure theory (MAT5011/5012). Anyway, the engineering interpretations are: –If an event has very small probability, e.g , and we do the experiment once, then we expect that this event will not occur. –If an event has probability p, and we repeat the experiment sufficiently many times, the relative frequency is practically close to p.

K. Shum Other definitions of Probability Subjective probability –P(KW Tung will be chief executive again) –Advocated by Bruno de Finetti (1931) Geometric probability Algorithmic probability, algorithmic randomness. ….

K. Shum Philosophical questions What is randomness? Can we measure randomness objectively? Is tossing a coin a random phenomenon?