1. Some Basic Statistical Concepts
Dr. Tai-Yue Wang
Department of Industrial and Information Management
National Cheng Kung University
Tainan, TAIWAN, ROC
This is a basic course blah, blah, blah…

2. Outline Introduction Basic Statistical Concepts
Inferences about the differences in Means, Randomized Designs
Inferences about the Differences in Means, Paired Comparison Designs
Inferences about the Variances of Normal Distribution

3. Introduction Formulation of a cement mortar
Original formulation and modified formulation
10 samples for each formulation
One factor  formulation
Two formulations:
 two treatments
 two levels of the factor formulation

4. Introduction
Results:

5. Introduction
Dot diagram

6. Basic Statistical Concepts
Experiences from above example
Run – each of above observations
Noise, experimental error, error – the individual runs difference
Statistical error– arises from variation that is uncontrolled and generally unavoidable
The presence of error means that the response variable is a random variable
Random variable could be discrete or continuous

7. Basic Statistical Concepts
Describing sample data
Graphical descriptions
Dot diagram—central tendency, spread
Box plot –
Histogram

8. Basic Statistical Concepts

9. Basc Statistical Concepts

10. Basic Statistical Concepts
Discrete vs continuous

11. Basic Statistical Concepts
Probability distribution
Discrete
Continuous

12. Basic Statistical Concepts
Probability distribution
Mean—measure of its central tendency
Expected value –long-run average value

13. Basic Statistical Concepts
Probability distribution
Variance —variability or dispersion of a distribution

14. Basic Statistical Concepts
Probability distribution
Properties: c is a constant
E(c) = c
E(y)= μ
E(cy)=cE(y)=cμ
V(c)=0
V(y)= σ2
V(cy)=c2 σ2
E(y1+y2)= μ1+ μ2

15. Basic Statistical Concepts
Probability distribution
Properties: c is a constant
V(y1+y2)= V(y1)+V(y2)+2Cov(y1, y2)
V(y1-y2)= V(y1)+V(y2)-2Cov(y1, y2)
If y1 and y2 are independent, Cov(y1, y2) =0
E(y1*y2)= E(y1)*V(y2)= μ1* μ2
E(y1/y2) is not necessary equal to E(y1)/V(y2)

16. Basic Statistical Concepts
Sampling and sampling distribution
Random samples --
if the population contains N elements and a sample of n of them is to be selected, and if each of N!/[(N-n)!n!] possible samples has equal probability being chosen
Random sampling – above procedure
Statistic – any function of the observations in a sample that does not contain unknown parameters

17. Basic Statistical Concepts
Sampling and sampling distribution
Sample mean
Sample variance

18. Basic Statistical Concepts
Sampling and sampling distribution
Estimator – a statistic that correspond to an unknown parameter
Estimate – a particular numerical value of an estimator
Point estimator: to μ and s2 to σ2
Properties on sample mean and variance:
The point estimator should be unbiased
An unbiased estimator should have minimum variance

19. Basic Statistical Concepts
Sampling and sampling distribution
Sum of squares, SS in
Sum of squares, SS, can be defined as

20. Basic Statistical Concepts
Sampling and sampling distribution
Degree of freedom, v, number of independent elements in a sum of square in
Degree of freedom, v , can be defined as

21. Basic Statistical Concepts
Sampling and sampling distribution
Normal distribution, N

22. Basic Statistical Concepts
Sampling and sampling distribution
Standard Normal distribution, z, a normal distribution with μ=0 and σ2=1

23. Basic Statistical Concepts
Sampling and sampling distribution
Central Limit Theorem–
If y1, y2, …, yn is a sequence of n independent and identically distributed random variables with E(yi)=μ and V(yi)=σ2 and x=y1+y2+…+yn, then the limiting form of the distribution of
as n∞, is the standard normal distribution

24. Basic Statistical Concepts
Sampling and sampling distribution
Chi-square, χ2 , distribution–
If z1, z2, …, zk are normally and independently distributed random variables with mean 0 and variance 1, NID(0,1), the random variable
follows the chi-square distribution with k degree of freedom.

25. Basic Statistical Concepts
Sampling and sampling distribution
Chi-square distribution– example
If y1, y2, …, yn are random samples from N(μ, σ2), distribution,
Sample variance from NID(μ, σ2),

26. Basic Statistical Concepts
Sampling and sampling distribution
t distribution–
If z and are independent standard normal and chi-square random variables, respectively, the random variable
follows t distribution with k degrees of freedom

27. Basic Statistical Concepts
Sampling and sampling distribution
pdf of t distribution–
μ =0, σ2=k/(k-2) for k>2

28. Basic Statistical Concepts

29. Basic Statistical Concepts
Sampling and sampling distribution
If y1, y2, …, yn are random samples from N(μ, σ2), the quantity
is distributed as t with n-1 degrees of freedom

30. Basic Statistical Concepts
Sampling and sampling distribution
F distribution—
If and are two independent chi-square random variables with u and v degrees of freedom, respectively
follows F distribution with u numerator degrees of freedom and v denominator degrees of freedom

31. Basic Statistical Concepts
Sampling and sampling distribution
pdf of F distribution–

32. Basic Statistical Concepts
Sampling and sampling distribution
F distribution– example
Suppose we have two independent normal distributions with common variance σ2 , if y11, y12, …, y1n1 is a random sample of n1 observations from the first population and y21, y22, …, y2n2 is a random sample of n2 observations from the second population

33. The Hypothesis Testing Framework
Statistical hypothesis testing is a useful framework for many experimental situations
Origins of the methodology date from the early 1900s
We will use a procedure known as the two-sample t-test

34. Two-Sample-t-Test
Suppose we have two independent normal, if y11, y12, …, y1n1 is a random sample of n1 observations from the first population and y21, y22, …, y2n2 is a random sample of n2 observations from the second population

35. Two-Sample-t-Test
A model for data
ε is a random error

36. Two-Sample-t-Test Sampling from a normal distribution
Statistical hypotheses:

37. Two-Sample-t-Test
H0 is called the null hypothesis and H1 is call alternative hypothesis.
One-sided vs two-sided hypothesis
Type I error, α: the null hypothesis is rejected when it is true
Type II error, β: the null hypothesis is not rejected when it is false

38. Two-Sample-t-Test Power of the test: Type I error  significance level
1- α = confidence level

39. Two-Sample-t-Test
Two-sample-t-test
Hypothesis:
Test statistic:
where

40. Two-Sample-t-Test

41. Two-Sample-t-Test

42. Example --Summary Statistics
Formulation 2
“Original recipe”
Formulation 1
“New recipe”

43. Two-Sample-t-Test--How the Two-Sample t-Test Works:

44. Two-Sample-t-Test--How the Two-Sample t-Test Works:

45. Two-Sample-t-Test--How the Two-Sample t-Test Works:
Values of t0 that are near zero are consistent with the null hypothesis
Values of t0 that are very different from zero are consistent with the alternative hypothesis
t0 is a “distance” measure-how far apart the averages are expressed in standard deviation units
Notice the interpretation of t0 as a signal-to-noise ratio

46. The Two-Sample (Pooled) t-Test

47. Two-Sample-t-Test
P-value– The smallest level of significance that would lead to rejection of the null hypothesis.
Computer application
Two-Sample T-Test and CI
Sample N Mean StDev SE Mean
Difference = mu (1) - mu (2)
Estimate for difference:
95% CI for difference: (-0.547, )
T-Test of difference = 0 (vs not =): T-Value = P-Value = DF = 18
Both use Pooled StDev =

48. William Sealy Gosset (1876, 1937)
Gosset's interest in barley cultivation led him to speculate that design of experiments should aim, not only at improving the average yield, but also at breeding varieties whose yield was insensitive (robust) to variation in soil and climate.
Developed the t-test (1908)
Gosset was a friend of both Karl Pearson and R.A. Fisher, an achievement, for each had a monumental ego and a loathing for the other.
Gosset was a modest man who cut short an admirer with the comment that “Fisher would have discovered it all anyway.”

49. The Two-Sample (Pooled) t-Test
So far, we haven’t really done any “statistics”
We need an objective basis for deciding how large the test statistic t0 really is
In 1908, W. S. Gosset derived the reference distribution for t0 … called the t distribution
Tables of the t distribution – see textbook appendix

50. The Two-Sample (Pooled) t-Test
A value of t0 between –2.101 and is consistent with equality of means
It is possible for the means to be equal and t0 to exceed either or –2.101, but it would be a “rare event” … leads to the conclusion that the means are different
Could also use the P-value approach

51. The Two-Sample (Pooled) t-Test
The P-value is the area (probability) in the tails of the t-distribution beyond the probability beyond (it’s a two-sided test)
The P-value is a measure of how unusual the value of the test statistic is given that the null hypothesis is true
The P-value the risk of wrongly rejecting the null hypothesis of equal means (it measures rareness of the event)
The P-value in our problem is P = 0.042

52. Checking Assumptions – The Normal Probability Plot

53. Two-sample-t-test--Choice of sample size
The choice of sample size and the probability of type II error β are closely related connected
Suppose that we are testing the hypothesis
And The mean are not equal so that δ=μ1-μ2
Because H0 is not true  we care about the probability of wrongly failing to reject H0
 type II error

54. Two-sample-t-test--Choice of sample size
Define
One can find the sample size by varying power (1-β) and δ

55. Two-sample-t-test--Choice of sample size
Testing mean 1 = mean 2 (versus not =)
Calculating power for mean 1 = mean 2 + difference
Alpha = Assumed standard deviation = 0.25
Sample Target
Difference Size Power Actual Power
The sample size is for each group.

56. Two-sample-t-test--Choice of sample size

57. An Introduction to Experimental Design -How to sample?
A completely randomized design is an experimental design in which the treatments are randomly assigned to the experimental units.
If the experimental units are heterogeneous, blocking can be used to form homogeneous groups, resulting in a randomized block design.

58. Completely Randomized Design -How to sample?
Recall Simple Random Sampling
Finite populations are often defined by lists such as:
Organization membership roster
Credit card account numbers
Inventory product numbers

59. Completely Randomized Design -How to sample?
A simple random sample of size n from a finite population of size N is a sample selected such that each possible sample of size n has the same probability of being selected.
Replacing each sampled element before selecting subsequent elements is called sampling with replacement.
Sampling without replacement is the procedure used most often.

60. Completely Randomized Design -How to sample?
In large sampling projects, computer-generated random numbers are often used to automate the sample selection process.
Excel provides a function for generating random numbers in its worksheets.
Infinite populations are often defined by an ongoing process whereby the elements of the population consist of items generated as though the process would operate indefinitely.

61. Completely Randomized Design -How to sample?
A simple random sample from an infinite population is a sample selected such that the following conditions are satisfied.
Each element selected comes from the same population.
Each element is selected independently.

62. Completely Randomized Design -How to sample?
Random Numbers: the numbers in the table are random, these four-digit numbers are equally likely.

63. Completely Randomized Design -How to sample?
Most experiments have critical error on random sampling.
Ex: sampling 8 samples from a production line in one day
Wrong method:
Get one sample every 3 hours not random!

64. Completely Randomized Design -How to sample?
Ex: sampling 8 samples from a production line
Correct method:
You can get one sample at each 3 hours interval but not every 3 hours  correct but not a simple random sampling
Get 8 samples in 24 hours
Maximum population is 24, getting 8 samples two digits
63, 27, 15, 99, 86, 71, 74, 45, 10, 21, 51, …
Larger than 24 is discarded
So eight samples are collected at:
15, 10, 21, … hour

65. Completely Randomized Design -How to sample?
In Completely Randomized Design, samples are randomly collected by simple random sampling method.
Only one factor is concerned in Completely Randomized Design, and k levels in this factor.

66. Importance of the t-Test
Provides an objective framework for simple comparative experiments
Could be used to test all relevant hypotheses in a two-level factorial design, because all of these hypotheses involve the mean response at one “side” of the cube versus the mean response at the opposite “side” of the cube

67. Two-sample-t-test— Confidence Intervals
Hypothesis testing gives an objective statement concerning the difference in means, but it doesn’t specify “how different” they are
General form of a confidence interval
The 100(1- α)% confidence interval on the difference in two means:

68. Two-sample-t-test— Confidence Intervals--example

69. Other Topics
Hypothesis testing when the variances are known—two-sample-z-test
One sample inference—one-sample-z or one-sample-t tests
Hypothesis tests on variances– chi-square test
Paired experiments

70. Other Topics

71. Other Topics

72. Other Topics