32931 Technology Research Methods Autumn 2017 Quantitative Research Component Topic 2: Inference Statistics Background. 30% Quantitative Assignment due on May 24th Wednesday 9pm Lecturer: Mahrita Harahap Mahrita.Harahap@uts.edu.au B MathFin (Hons) M Stat (UNSW) PhD (UTS) mahritaharahap.wordpress.com/ teaching-areas Faculty of Engineering and Information Technology

Statistical Inference Statistical inference is the process of drawing conclusions about the entire population based on information in a sample. Week 2

Statistical Inference: The big picture We usually have a sample statistic and want to use it to make inferences about the population parameter. e.g. average income of a random sample of Australians ( x ) e.g. average income of all Australians (µ) Week 2

Statistical Inference A parameter is a number that describes some aspect of a population. A statistic is a number that is computed from data in a sample. Mean Proportion Standard deviation correlation Statistic x p s r Parameter µ σ ρ Week 2

Where does inference fit in? So far we have looked at how to describe data but we haven’t been able to test our observations. Inference provides the tools to test whether our observations can be applied to the population as opposed to seeing the results by chance. A statistical test uses data from a sample to assess a claim about a population. Week 2

Test of a Hypothesis It is a procedure leading to a decision about a particular hypothesis. Hypothesis testing procedures rely on using the information in a random sample from the population of interest. A hypothesised value for a population’s parameter is reasonable if the corresponding sample statistic is close to the hypothesised value, and not reasonable if the sample statistic is far away from it. Week 2

Hypothesis Testing Hypothesis tests give us an objective way of assessing such questions. They are based on a proof by contradiction form of argument. We formulate a null hypothesis (H0) We formulate an alternative hypothesis (H1) We calculate a test statistic. If the test statistic is close to what we would expect under H0, we DO NOT reject H0. If the test statistic is far from what we would expect under H0, we reject H0. We state our conclusion in context. Week 2

Hypotheses H0: null hypothesis H1: alternative hypothesis Statistical tests are framed formally in terms of two competing hypotheses H0: null hypothesis H1: alternative hypothesis which are two competing claims about a population. Week 2

The Null Hypothesis (H0) H0 is the claim that there is no effect or difference (the status quo), and is always an equality (or could be expressed as one). E.g., The mean height of UTS students is 160cm H0: µ = 160 The null hypothesis is always about a population parameter (e.g. µ = 160), and never about a sample statistic (e.g. x = 160). Week 2

The Alternative Hypothesis (H1) The alternative hypothesis is the claim that we are trying to find evidence for. It should reflect our hopes or suspicions. H1 can be one sided or two sided. The mean is not equal to 160 (H1: µ ≠ 160) The mean is greater than 160 (H1: µ > 160) The mean is less than 160 (H1: µ < 160) How do we decide what H1 should be? It depends on the question that we are trying to answer. Are we looking for differences, or are we looking for larger/smaller values. When in doubt, use a 2-sided alternative Week 2

Hypothesis Testing: An Analogy Suppose that you are trying to convince me that UFOs exist, but I’m sceptical. My null hypothesis will be that there are no UFOs. My alternative hypothesis will be that UFOs exist. If you convince me enough, then I will reject H0 and also believe that UFOs exist. If you don’t convince me enough, then I will not reject H0, and will remain sceptical about UFOs Notice that this process does will not convince me that UFOs do not exist So we can’t accept H0 Week 2

Hypothesis Testing - Examples Medical Example H0: A new treatment is no different to the current treatment H1: A new treatment is better than the current treatment OR H0: µ 1 = µ 2 H1: µ 1 > µ 2 where µ 1 = mean survival time for the new treatment, and µ 2 = mean survival time for the current treatment Week 2

Statistical significance When results as extreme as the observed sample statistic are unlikely to occur by chance alone(assuming the null hypothesis is true), we say the sample results are statistically significant. If our sample is statistically significant, we have convincing evidence against Ho, in favour of H1 If our sample is not statistically significant, our test is inconclusive(we don’t have convincing evidence of H1) Week 2

Statistical significance: Decision making We make a decision based on a p-value The p-value is the probability that we would observe the sample mean or something more extreme, by chance, when the null hypothesis is true We compare the p-value to a level of significance The level of significance is the level of risk of mistakenly rejecting a true null hypothesis By default, we will use 5% = 0.05 significance. That is, we are willing to accept a 5% chance that we incorrectly reject the null hypothesis In some situations we may change the level of significance. If p-value < α(0.05) we reject H0 If p-value ≥ α(0.05) we do not reject H0 Week 2

The Structure of a Hypothesis Test Step 1: Set up Hypotheses Null Hypothesis: H0 Alternative Hypothesis: H1 Step 2: Choose an appropriate test Step 3: Execute the test in SPSS and obtain a p-value P-value = probability that you would observe such a sample when the null hypothesis is true Step 4: Make a decision either Reject H0 if the p-value is small enough, or do not reject H0 – you can’t accept a hypothesis Step 5: State the conclusion in plain language So people who don’t understand statistics can still understand your conclusions Week 2

Hypothesis Testing What hypothesis testing is: Based on randomness. A method that gives us the probability that we see the observed effect by chance. What hypothesis testing is not: A method of determining whether an effect is of practical importance. A quantitative method that allows you to make conclusions with certainty. A binary decision rule. Week 2

Types of Hypothesis Tests The choice of test depends on What type of data you are using. The hypotheses that you are testing. Whether your data satisfies certain assumptions, such as normality. Week 2

Categorical x QUANTITATIVE 1-sample t Test Paired t Test 2-Sample t Test ANOVA Multiple Comparisons Non-Parametric Tests Categorical x QUANTITATIVE Week 2

Categorical x Quantitative: Which Test? Tests One group 1-Sample tests 1-Sample T Two groups 2-Sample tests Data are paired Paired T Data are not paired 2-Sample T Three or more groups K-Sample tests Analysis of Variance Week 2

1-Sample T test The simplest type of test is a one sample T test, where we want to compare the population mean to a specific value. I have one column of data, and I want to compare the population mean to a particular measurement. ASSUMPTION: 1 – Sample T assumes that the data are normally distributed. H0:  = value ≠ value two sided alternative H1:  < value one sided alternative > value one sided alternative Week 2

Example: Pulse Data An experiment was carried out among a group of University students to determine the effect of exercise and various other variables on pulse rate. Each person measured their pulse rate (beats per minute), then tossed a coin. If the coin came up Heads, they ran on the spot for one minute and then took their pulse again. If it came up Tails, they had to sit quietly for one minute and then take their pulse again. Pulse 1 Pulse 2 Ran Smokes Sex Height Weight Activity 64.0 88.0 1 0. 1.68 2 58.0 70.0 1.83 66.0 : 76.0 1.57 49.0 First pulse measurement Second pulse measurement 1 = Yes 0 = No 1 = Male 2 = Female in m in kg 1 = slight 2 = moderate 3 = high

1-Sample T test - Example Suppose that we would like to test whether our group has a starting pulse rate that is different from the typical resting pulse rate of 75bpm. Step 1: Set up the hypotheses H0:  =75 H1:  ≠ 75 Step 2: Choose an appropriate test – 1-Sample T Step 3: Execute the test in SPSS and obtain a p- value Use Analyze> Compare Means > One Sample T test Week 2

1-Sample T test - Example Step 4: Make a conclusion P-value = 0.067 > 0.05 (level of significance) Therefore do not reject H0 Step 5: State the conclusion in plain language Therefore the initial pulse rate measurements do not differ significantly from the typical resting pulse rate of 75bpm at a 5% level of significance. Week 2

Confidence Intervals One of the most common goals when performing an experiment/study is to obtain a good estimate for a population parameter. While it is of some use to have a point estimate, it is better to be able to give a range of plausible values. An interval estimate gives a range of plausible values for a population parameter. A range of values that have probability (1- α) of containing the population parameter is called a (1- α)% Confidence Interval. α is known as the level of significance, i.e. the probability that the confidence interval does not contain the population parameter. Week 2

Confidence Intervals A confidence interval for a parameter is an interval computed from sample data by a method that will capture the parameter for a specified proportion of all samples. The success rate (proportion of all samples whose intervals contain the parameter) is known as the confidence level. A 95% confidence interval will contain the true parameter for 95% of all samples. Week 2

Statistical Inference: Confidence Intervals We can relate the conclusion from a hypothesis test to whether or not the test mean falls in the confidence interval. If the test mean falls in the confidence interval then we do not reject H0. If the test mean does not fall in the confidence interval then we reject H0. Week 2

Confidence Intervals To get a confidence interval for a population mean in SPSS, we perform a 1-sample t test with a test mean of 0, and use the confidence interval given in the output. We can change the level of confidence in the Options menu. Week 2

Testing for Normality One of our assumptions is that our data take a normal distribution. We can test this formally using the Kolmologrov-Smirnov test (KS for short). The hypotheses for this test are H0: The data are normally distributed H1: The data are not normally distributed In SPSS we find this test under Analyze > Nonparametric Tests > 1-Sample KS If we have more than one group, we could either test each sample individually, or subtract the group mean and then test all of the data. Week 2

Testing for Normality - Example Suppose that we want to test whether the initial pulse rates are normally distributed. Step 1: Set up the hypotheses H0 : Initial pulse rates are normally distributed H1: Initial pulse rates are not normally distributed Step 2: Choose an appropriate test – KS Step 3: Execute the test in SPSS and obtain a p-value Week 2

Testing for Normality - Example Step 4: Make a decision P-value = 0.256 > α(0.05) level of significance Therefore we do not reject H0 Step 5: State the conclusion in plain language We conclude that the initial pulse rates do not differ significantly from a normally distribution. Week 2

Errors in Hypothesis Testing Since we are using a sample to infer information about a population, we are going to get it wrong sometimes! We need to quantify this risk. No Error Type II Error P(fail to reject H0|H0 True) = 1- a P(fail to reject H0|H0 False) = b Type I Error No Error P(reject H0|H0 True) = a P(reject H0|H0 False) = 1 - b Week 3

Errors in Hypothesis Testing The probability of a type I error is denoted by  and is called the level of significance of the test. Thus, a test with  = 0.01 is said to have a level of significance of 1%. Typical values are 0.01, 0.05, 0.10 and default value is 0.05 The probability of a type II error is denoted by . The value 1 –  is called the power of the test. The general procedure is to specify a value for the type I error α, then design the test procedure so that the probability of type II error β, has a suitably small value. Week 3

Errors in Hypothesis Testing: Analogy UTS has fire alarms throughout the campus (it’s the law) The alarms will go off if they detect smoke (which usually means that there is a fire). If there is a fire, then we want to evacuate the building. The default situation is that there is no fire Null hypothesis: There is no fire. Alternative hypothesis: There is a fire. We can set this problem up in terms of a hypothesis test. Week 3

Errors in Hypothesis Testing: Analogy Null Hypothesis True – No Fire False – Fire Do not Reject H0 – No Alarm Reject H0 - Alarm No Error Type II Error Type I Error No Error Week 3

Week 2 appendices Week 2

Revision: Populations and Samples The data that we actually have and can use in statistical investigation The complete collection of data from which a sample is collected Actually available, and not usually too large Unavailable, and usually infinite or very large Summarised by statistics, for example, the sample mean X and the sample standard deviation S Summarised by parameters, for example, the population mean µ and population standard deviation σ Statistics are usually different for each sample we take from the population (i.e. they are random) Parameters are fixed for any particular population. Statistics can be accurately calculated from the sample values Parameters cannot be obtained by calculation: their values can rarely be known for certain We actually have values for statistics We are really interested in the values for parameters Week 2

Calculating the p-value Recall that a p-value is the probability that we observe such a sample given that the null hypothesis is true. To find this probability, we need two things: A measure for how much the sample differs from the hypothesised mean, and A distribution for that measure. This measure is known as a test statistic, and is also quoted in SPSS. The test statistic differs from test to test. Week 2

Calculating the p-value - Example Consider a 1-sample t test. We have the hypotheses H0:  = 0 H1:  ≠ 0 We take a sample and obtain a sample mean X and sample standard deviation s. It follows that the test statistic will have a t distribution with n-1 degrees of freedom. Week 2

Calculating the p-value - Example We then need to find the probability that a t distribution with n-1 degrees of freedom will have a value larger than the test statistic in absolute value P-value = P(|Tn-1|> t) The shaded region includes the values of t that will result in a rejected null hypothesis. This is called the rejection region. p-value Week 2

Calculating the p-value We compare the p-value to a level of significance The level of significance is the level of risk of mistakenly rejecting a true null hypothesis By default, we will use 5% = 0.05 significance. That is, we are willing to accept a 5% chance that we incorrectly reject the null hypothesis In some situations we may change the level of significance. If p-value < 0.05 we reject H0 If p-value ≥ 0.05 we do not reject H0 Week 2

Calculating the p-value Suppose that I actually wanted a 2-tailed p-value for a 2-sided alternative. Then I would double the 1-tailed p-value. Suppose that SPSS gives me a 2-tailed p-value, but I want a 1-tailed p-value since I have a 1-sided alternative (< or >) Then I would either halve the p-value if the sample mean agrees with the alternative, or I would halve the p-value and subtract from 1 if the sample mean is not consistent with the alternative hypothesis E.g. The sample mean is larger than the hypothesised mean, but the alternative is less than. Week 2

SPSS: Q-q plots for normality When you are doing a t-test or ANOVA, the assumption is that the distribution of the sample means are normally distributed.  One way to guarantee this is for the distribution of the individual observations from the sample to be normal.  However, even if the distribution of the individual observations is not normal, the distribution of the sample means will be normally distributed if your sample size is about 30 or larger.  This is due to the “central limit theorem” that shows that even when a population is non-normally distributed, the distribution of the “sample means” will be normally distributed when the sample size is 30 or more, for example see Central limit theorem demonstration . Week 2