One Tailed and Two Tailed tests One tailed tests: Based on a uni-directional hypothesis Example: Effect of training on problems using PowerPoint Population figures for usability of PP are known Hypothesis: Training will decrease number of problems with PP Two tailed tests: Based on a bi-directional hypothesis Hypothesis: Training will change the number of problems with PP

If we know the population mean Mean Usability Index Sampling Distribution Population for usability of Powerpoint Frequency Std. Dev =.45 Mean = 5.65 N = Unidirectional hypothesis:.05 level Bidirectional hypothesis:.05 level Identify region

What does it mean if our significance level is.05?What does it mean if our significance level is.05? XFor a uni-directional hypothesis XFor a bi-directional hypothesis PowerPoint example: UnidirectionalUnidirectional XIf we set significance level at.05 level, 5% of the time we will higher mean by chance5% of the time we will higher mean by chance 95% of the time the higher mean mean will be real95% of the time the higher mean mean will be real BidirectionalBidirectional XIf we set significance level at.05 level 2.5 % of the time we will find higher mean by chance2.5 % of the time we will find higher mean by chance 2.5% of the time we will find lower mean by chance2.5% of the time we will find lower mean by chance 95% of time difference will be real95% of time difference will be real

Changing significance levels What happens if we decrease our significance level from.01 to.05What happens if we decrease our significance level from.01 to.05 XProbability of finding differences that don’t exist goes up (criteria becomes more lenient) What happens if we increase our significance from.01 to.001What happens if we increase our significance from.01 to.001 XProbability of not finding differences that exist goes up (criteria becomes more conservative)

PowerPoint example:PowerPoint example: XIf we set significance level at.05 level, 5% of the time we will find a difference by chance5% of the time we will find a difference by chance 95% of the time the difference will be real95% of the time the difference will be real XIf we set significance level at.01 level 1% of the time we will find a difference by chance1% of the time we will find a difference by chance 99% of time difference will be real99% of time difference will be real For usability, if you are set out to find problems: setting lenient criteria might work better (you will identify more problems)For usability, if you are set out to find problems: setting lenient criteria might work better (you will identify more problems)

Effect of decreasing significance level from.01 to.05Effect of decreasing significance level from.01 to.05 XProbability of finding differences that don’t exist goes up (criteria becomes more lenient) XAlso called Type I error (Alpha) Effect of increasing significance from.01 to.001Effect of increasing significance from.01 to.001 XProbability of not finding differences that exist goes up (criteria becomes more conservative) XAlso called Type II error (Beta)

Degree of Freedom The number of independent pieces of information remaining after estimating one or more parametersThe number of independent pieces of information remaining after estimating one or more parameters Example: List= 1, 2, 3, 4 Average= 2.5Example: List= 1, 2, 3, 4 Average= 2.5 For average to remain the same three of the numbers can be anything you want, fourth is fixedFor average to remain the same three of the numbers can be anything you want, fourth is fixed New List = 1, 5, 2.5, __ Average = 2.5New List = 1, 5, 2.5, __ Average = 2.5

Major Points T tests: are differences significant?T tests: are differences significant? One sample t tests, comparing one mean to populationOne sample t tests, comparing one mean to population Within subjects test: Comparing mean in condition 1 to mean in condition 2Within subjects test: Comparing mean in condition 1 to mean in condition 2 Between Subjects test: Comparing mean in condition 1 to mean in condition 2Between Subjects test: Comparing mean in condition 1 to mean in condition 2

Effect of training on Powerpoint use Does training lead to lesser problems with PP?Does training lead to lesser problems with PP? 9 subjects were trained on the use of PP.9 subjects were trained on the use of PP. Then designed a presentation with PP.Then designed a presentation with PP. XNo of problems they had was DV

Powerpoint study data Mean = 23.89Mean = SD = 4.20SD = 4.20

Results of Powerpoint study. ResultsResults XMean number of problems = Assume we know that without training the mean would be 30, but not the standard deviationAssume we know that without training the mean would be 30, but not the standard deviation Population mean = 30 Is enough larger than 30 to conclude that video affected results?Is enough larger than 30 to conclude that video affected results?

Sampling Distribution of the Mean We need to know what kinds of sample means to expect if training has no effect.We need to know what kinds of sample means to expect if training has no effect.  i. e. What kinds of means if  = XThis is the sampling distribution of the mean.

Sampling Distribution of the Mean--cont. The sampling distribution of the mean depends onThe sampling distribution of the mean depends on XMean of sampled population XSt. dev. of sampled population XSize of sample

Cont.

Sampling Distribution of the mean--cont. Shape of the sampled populationShape of the sampled population XApproaches normal XRate of approach depends on sample size XAlso depends on the shape of the population distribution

Implications of the Central Limit Theorem Given a population with mean =  and standard deviation = , the sampling distribution of the mean (the distribution of sample means) has a mean = , and a standard deviation =  /  n.Given a population with mean =  and standard deviation = , the sampling distribution of the mean (the distribution of sample means) has a mean = , and a standard deviation =  /  n. The distribution approaches normal as n, the sample size, increases.The distribution approaches normal as n, the sample size, increases.

Demonstration Let population be very skewedLet population be very skewed Draw samples of 3 and calculate meansDraw samples of 3 and calculate means Draw samples of 10 and calculate meansDraw samples of 10 and calculate means Plot meansPlot means Note changes in means, standard deviations, and shapesNote changes in means, standard deviations, and shapes Cont.

Parent Population Cont.

Sampling Distribution n = 3 Cont.

Sampling Distribution n = 10 Cont.

Demonstration--cont. Means have stayed at 3.00 throughout-- except for minor sampling errorMeans have stayed at 3.00 throughout-- except for minor sampling error Standard deviations have decreased appropriatelyStandard deviations have decreased appropriately Shapes have become more normal--see superimposed normal distribution for referenceShapes have become more normal--see superimposed normal distribution for reference

One sample t test cont. Assume mean of population known, but standard deviation (SD) not knownAssume mean of population known, but standard deviation (SD) not known Substitute sample SD for population SD (standard error)Substitute sample SD for population SD (standard error) Gives you the t statisticsGives you the t statistics Compare t to tabled values which show critical values of tCompare t to tabled values which show critical values of t

t Test for One Mean Get mean difference between sample and population meanGet mean difference between sample and population mean Use sample SD as variance metric = 4.40Use sample SD as variance metric = 4.40

Degrees of Freedom Skewness of sampling distribution of variance decreases as n increasesSkewness of sampling distribution of variance decreases as n increases t will differ from z less as sample size increasest will differ from z less as sample size increases Therefore need to adjust t accordinglyTherefore need to adjust t accordingly df = n - 1df = n - 1 t based on dft based on df

Looking up critical t (Table E.6)

Conclusions Critical t= n = 9, t.05 = 2.62 (two tail significance)Critical t= n = 9, t.05 = 2.62 (two tail significance) If t > 2.62, reject H 0If t > 2.62, reject H 0 Conclude that training leads to less problemsConclude that training leads to less problems

Factors Affecting t Difference between sample and population meansDifference between sample and population means Magnitude of sample varianceMagnitude of sample variance Sample sizeSample size

Factors Affecting Decision Significance level Significance level  One-tailed versus two-tailed testOne-tailed versus two-tailed test