Project Management for Software Engineers (Summer 2017) LECTURE 12 Risk Quantification Basics July 19, 2016 (9:00 am – 11:40 pm PST) University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018

Risk = Uncertainty Risk quantification means: “How much” uncertain are we about a particular outcome Supplier  Input  Process  Output  Customer: Focusing on the I.P.O: If we know “how much” uncertain we are about the Inputs, we can agglomerate all uncertainties to calculate the uncertainty about the output, using statistical techniques (simulation) University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018

A Crash Course on Probabilities A + Ac = 1 P(A or B) = P(A) + P(B) – P (A and B) Mutually Exclusive events: P(A and B) =0 Conditional Probability P(A|B) = P(A and B) / P(B)  P(A and B) = P(A|B)xP(B) Joint Probabilities: Decision Tree  (pp. 257-258) Independent Events: P(A|B)=P(A)  P(A and B) =P(A)xP(B) University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018

A Crash Course on Statistics Descriptive Statistics Current Data attributes (central tendency & dispersion) Histograms: Location, Spread, Shape Inferential Statistics (Enumerative Studies) Sample  Population (Hypothesis Testing) Key: Proper Sampling  Representative Sample Predictive Statistics (Analytic Studies) DOE, Regression, ANOVA University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018

Descriptive Statistics Central tendency: Mean, median, mode Dispersion: Variance, Min, Max, Range Shape: Histogram, Skewness, Kurtosis Others: Count, Sum Excel Example: Tools  Data Analysis  Descriptive Statistics Tools  Data Analysis  Histogram University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018

Statistical Distributions Discrete: Binomial, Hypergeometric, Poisson, … Continuous: Normal, Lognormal, Exponential, … Interrelationships Binomial  Poisson Normal  Lognormal Poisson  Exponential Sources of abnormality: Recording & Measurement errors, Multiplicity, truncation, natural skewness, tampering (Deming experiments) Understand data & sources of variation before sampling Bi (p<0.5 & n>20) == Poisson (np) Bi: Flipping a coin Poisson: # of cars entering an intersection Exponential: Interarrival of of cars entering an intersection University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018

Inferential Statistics Sampling: Select an “unbiased” subset of an entire group Simple Random Sampling (SRS): For homogeneous sets Stratified sampling: Group by some attributes (internally homogeneous) Clustering: Similar groups, different components (“Naturally” clustered, internally heterogeneous) – Less Precise & More Economic Systematic: Every (N/n)th sample, or some other rules (Easy, but prone to bias) Examples: Random, NOT arbitrary! Stratified: Sampling City population, in age groups Stratified: Exponential Distribution Clustered: Sampling random pallets of packed products Systematic: Sampling every 8th house of a street with 120 houses NOT REQUIRED Stratified: Exponential Distribution tail has bigger weight Systematic could be biased

Representative Samples If the sampling is performed properly, the sample “represents” the population Parameter Sample Population Size n N Mean x̄ m St. Dev. s s CLT – For very large population (n/N<0.05) and “Truly” representative samples: E(x̄) = m & s(x̄)=s /√n  x̄ ~ N ( m , s /√n ) Example: P. 280, Examples 6.13 & 6.14 Note: N(x, m, s) == Ns[(x-m)/s, 0, 1]  Ns Table (pp.A3&A4) In CLT: x̄ ~ Ns [(x̄- m)/(s /√n ),0,1] NOT REQUIRED University of Southern California, Industrial & Systems Engineering 9/18/2018

From “Point” To “Range” Estimate Confidence Level (CL) = 1 - a a = Probability of a parameter falling outside a given range. (e.g. probability of defect) Example: “90% CI for m is x̄ + q” means “In 100 samples, for 90 of them: the range [x̄ - q x̄ + q ] contains the true value of m” Equations: P. 281 Examples 6.15 & 6.16 (p. 282) NOT REQUIRED University of Southern California, Industrial & Systems Engineering 9/18/2018

Estimating reality by experimentation What is simulation? Problem: Flipping a coin as many times as we get 3 heads; Cost: $1 per coin-flip, reward: $8 for 3 heads Can you develop a formula to get the answer? Yes you can, but it’s a tedious and complicated process. Can it be done easier? Just walk the talk: Generate a random integer (0, 1) Excel: Int((Rand()+0.5)) Repeat until you get three 1’s (heads) If we repeat this experiment many many times, we can develop a statistical distribution for the sample size (i.e. number of flips that returns 3 heads) Example calculations in Excel (handout) Estimating reality by experimentation University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018

Monte-Carlo Simulation Process: If Y=f(X1,X2,X3,…) and statistical distribution of Xi’s are known as F(Xi), then: U = F(Xi)  Xi = F-1(U), where U is a probability, a random number between 0 and 1 (100%) With numerous iterations, statistical distribution of Y and its statistical characteristics can be modeled and the probability of Y<=Y0 can be estimated with a very high accuracy The more iterations, the more accurate the statistical distribution for Y For each Xi: For the entire model (Y): N=500 N=1000 N=100 N=200 Source: Queueing Methods, Randolph W. Hall (1991), P. 59 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018

A Basic Example A Typical customer service: Poisson Process Trial #5 𝐹 𝑥 =1− 𝑒 −l𝑥 , SET: R[0−1]=𝐹 𝑥  𝑥= ln⁡ 𝑅 −l Assume l=4 customers / hour based on historical data How many customers will arrive in 2 hours? Trial #5 N=5 #5 N=4 #4 Trial #4 N=2 #3 Trial #3 Trial #1 N=6 #1 N=5 #2 Trial #2 Also see Example: Hall. P. 185 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018

Simulating PM Metrics NPV : pp 310-318 CPM/PERT Scheduling: pp 367-372 University of Southern California, IMSC/SSU CERTIFICATION PROGRAM 9/18/2018