1. summary

2. ANOVA What is it? How does it work? What is an F-ratio?
What is a grand mean?
What are the degrees of freedom for the F-ratio?
k-1, N-k

3. Post hoc tests
F-test in ANOVA is the so-called omnibus test. It tests the means globally. It says nothing about which particular means are different.
post hoc tests, multiple comparison tests
Tukey Honestly Significant Differences
> TukeyHSD(fit) # where fit comes from aov()

4. New stuff

5. ANOVA assumptions normality – all samples are from normal distribution
homogeneity of variance (homoscedasticity) – variances are equal
independence of observations – the results found in one sample won't affect others
Most influencial is the independence assumption. Otherwise, ANOVA is relatively robust.
We can sometimes violate
normality – large sample size
variance homogeneity – equal sample sizes + the ratio of any two variances does not exceed four
Nonparametric equivalent – Kruskal-Wallis test

6. ANOVA kinds
one-way ANOVA (analýza rozptylu při jednoduchém třídění, jednofaktorová ANOVA)
aov(beer_brands$Price~beer_brands$Brand)
two-way ANOVA (analýza rozptylu dvojného třídění, dvoufaktorová ANOVA)
Example: engagement ratio, measure two educational methods (with and without song) for men and women independently
aov(engagement~method+sex)
interactions between factors
dependent variable
independent variable

7. Report statistical results I
Descriptive statistics
mean, s.d.
Confidence intervals
confidence level (e.g., 95%)
lower limit
upper limit
CI on what (e.g., on a mean)?
APA style
See, for example,
Confidence interval on the mean difference; 95% CI = (4,6)

8. Report statistical results II
Hypothesis test
kind of test (e.g., one-sample t-test)
the actual value of the test statistic (e.g., the value of t)
d.f.
p-value
if applicable, give a direction of test (e.g., one-tailed or two-tailed)
𝛼 level!
APA style for reporting results of the hypothesis test
t(df) = X.XX, p = X.XX, direction
e.g. t(24) = -2.50, p = 0.01, one-tailed

9. correlation

10. Introduction
Up to this point we've been working with only one variable.
Now we are going to focus at two variables.
Two variables that are probably related. Can you think of some examples?
weight and height
time spent studying and your grade
temperature outside and ankle injuries

11. Car data x – predictor, explanatory, independent variable
Miles on a car
Value of the car
60 000
$12 000
80 000
$10 000
90 000
$9 000
$7 500
$6 000
x – predictor, explanatory, independent variable
y – outcome, response, dependent variable

12. Car data How may we show these variables have a relationship?
Miles on a car
Value of the car
60 000
$12 000
80 000
$10 000
90 000
$9 000
$7 500
$6 000
How may we show these variables have a relationship?
Tell me some of yours ideas.
scatterplot

13. Scatterplot

14. Stronger relationship?

15. Correlation Relation between two variables = correlation
strong relationship = strong correlation, high correlation
Match these
strong positive
strong negative
weak positive
weak negative

16. Correlation coefficient
r (Pearson's r) - a number that quantifies the relationship.
𝑟= 𝑟 𝑥𝑦 = 1 𝑛− 𝑥 𝑖 − 𝑥 𝑦 𝑖 − 𝑦 𝑠 𝑋 𝑠 𝑌
Miles on a car
Value of the car
60 000
$12 000
80 000
$10 000
90 000
$9 000
$7 500
$6 000

17. +1 -1
+0.14
+0.93
-0.73

18. Guessing Correlations
Try to guess correlation coefficients at

19. Coefficient of determination
Coefficient of determination - 𝑟 2 is the percentage of variation in Y explained by the variation in X.
Percentage of variance in one variable that is accounted for by the variance in the other variable.
r2 = 0
r2 = 0.25
r2 = 0.81
from

20. Crickets
Find a cricket, count the number of its chirps in 15 seconds, add 37, you have just approximated the outside temperature in degrees Fahrenheit.
National Service Weather Forecast Office:
chirps in 15 sec
temperature 18 57 27 68 20 60 30 71 21 64 34 74 23 65 39 77

21. Hypothesis testing
Even when two variables describing the sample of data may seem they have a relationship, this could be just due to the chance. The situation in population may be different.
𝑟 … sample corr. coeff., 𝜌 … population corr. coeff.
How a hypotheses will look like?
𝐻 0 :𝑟=0 𝐻 𝐴 :𝑟< 𝑟> 𝑟≠0
𝐻 0 :𝜌=0 𝐻 𝐴 :𝜌< 𝜌> 𝜌≠0
𝐻 0 :𝑟< 𝑟> 𝑟≠0
𝐻 𝐴 :𝑟=0
𝐻 0 :𝑟< 𝑟> 𝑟≠0
𝐻 𝐴 :𝑟=0 A B C D

22. Hypothesis testing 𝑡= 𝑟 𝑛−2 1− 𝑟 2 with 𝑑𝑓=𝑛−2
test statistic has a t-distribution
Example: we measure the relationship between two variables, we have 25 participants, we get t = Is there a significant relationship between X and Y?
𝛼=0.05, non-directonal test, 𝑡 𝑐𝑟𝑖𝑡 =2.069

23. Correlation vs. causation
causation – one variable causes another to happen
For example, the fact that it is raining causes people to take their umbrellas .
correlation – just means there is a relationship
For example, do happy people have more friends? Are they just happy because they have more friends? Or they act a certain way which causes them to have more friends.

24. Correlation vs. causation
There is a strong relationship between the ice cream consumption and the crime rate.
However, if you stop selling ice cream, does the crime rate drop? What do you think?
So how could this be true?
Outside temperature.
from causeweb.org

25. Correlation vs. causation
Outside temperature is a variable we did not realize to control.
Such variable is called third variable, confounding variable, lurking variable.
The methodologies of scientific studies therefore need to control for these factors to avoid a 'false positive‘ conclusion that the dependent variables are in a causal relationship with the independent variable.

26. Correlation vs. causation
That’s because correlation expresses the association between two or more variables; it has nothing to do with causality.
In other words, just because the level of ice cream consumption and crime rate increase/descrease together does not mean that a change in one necessarily results in a change in the other.
You can’t interpret associations as being causal.

28. Correlation and regression analysis
Correlation analysis investigates the relationships between variables using graphs or correlation coefficients.
Regression analysis answers the questions like: which relationship exists between variables X and Y (linear, quadratic ,….), is it possible to predict Y using X, and with what error?

29. Simple linear regression
also single linear regression (jednoduchá lineární regrese)
one y (dependent variable, závisle proměnná), one x (independent variable, nezávisle proměnná)
𝑦 = 𝑎 + 𝑏𝑥
𝑎 – y-intercept (constant), 𝑏 – slope
𝑦 is estimated value, so to distinguish it from the actual value 𝑦 corresponding to the given 𝑥 statisticans use 𝑦

30. Data set Students in higher grades carry more textbooks.
Weight of the textbooks depends on the weight of the student.

31. strong positive correlation, r = 0.926
outlier
from Intermediate Statistics for Dummies

32. Build a model
Find a straight line y = a + bx

33. Interpretation
y-intercept (3.69 in our case) may or may not have practical meaning
Does it fall within actual values in the data set?
Does it fall within negative territory where negative y-value are not possible? (e.g. weights can’t be negative)
Does a value x = 0 have practical meaning (student weighting 0)?
However, even if it has no meaning, it may be necessary (i.e. significantly different from zero)!
slope
change in y due to one-unit increase in x (i.e. if student’s weight increases by 1 pound, its textbook’s weight increases by pounds)
now you can use regression line to estimate y value for new x

34. Regression model conditions
After building a regression mode you need to check if the required conditions are met.
What are these conditions?
The y’s have normal distribution for each value of x.
The y’s have constant spread (standard deviation) for each value of x.

35. Normal y’s for every x
For any value of x, the population of possible y-values must have a normal distribution.
from Intermediate Statistics for Dummies

36. Homoscedasticity condition
As you move from the left to the right on the x-axis, the spread around y-values remain the same.
source: wikipedia.org

37. Residuals
To check the normality of y-values you need to measure how far off your predictions were from the actual data, and to explore these errors.
residual (residuum, reziduální hodnota predikce)
𝑒=𝑦− 𝑦

38. residual actual value predicted value
from Intermediate Statistics for Dummies

39. Residuals
The residuals are data just like any other, so you can find their mean (which is zero!!) and their standard deviation.
Residuals can be standardized, i.e. converted to the Z-score so you see where they fall on the standard normal distribution.
Plotting residuals on the graph – residual plots.

40. normality of residuals
homoscedasticity
residuals independence

41. Using r2 to measure model fit
r2 measures what percentage of the variability in y is explained by the model.
The y-values of the data you collect have a great deal of variability.
You look for another variable (x) that helps to explain the variability in the y-values.
After you put x into the model and you find it’s highly correlated with y, you want to find out how well this model did at explaining why the values of y are different.

42. Interpreting r2 high r2 (80-90% is extremely high, 70% is fairly high)
A high percentage of variability means that the line fits well because there is not much left to explain about the value of y other than using x and its relationship to y.
small r2 (0-30%)
The model containing x doesn’t help much in explaining the difference in the y-values
The model would not fit well. You need another variable to explain y other than the one you already tried.
middle r2 (30-70%)
x does help somewhat in explaining y, but it doesn’t do the job well enough on its own.
Add one or more variables to the model to help explain y more fully as a group.
Textbook example: r = 0.93, r2 = Approximately 86% of variability you find in textbook weights is explained by the average student weight. Fairly good model.

43. Multiple regression Two (or more) variables are better than one.
y = b0 + b1x1 + b2x2 + … + bkxk
Steps in the analysis
Check the relationships between each x variable and y (using scatterplots and correlations) and use the results to eliminate those x variables that aren’t strongly related to y.
Look at possible relationships between the x variables themselves to make sure you aren’t being redundant (in statistical terms, you’re trying to avoid the problem of multicolinearity). If two x variables relate to y the same way, you don’t need both in the model.
Use selected x variables in a multiple regression analysis to find the best-fitting model for your data.
Use the best-fitting model to predict y for given x-values.

44. Data set Relate plasma TV sales with two types of advertisement .
from Intermediate Statistics for Dummies

45. Pinpoint Possible Relationships
Scatterplot
from Intermediate Statistics for Dummies

46. Correlations All possible correlations TV vs. Sales
Newspaper vs. Sales
Newspaper vs. TV
from Intermediate Statistics for Dummies

47. Is correlation coefficient ρ statistically significant?
What are null and alternative hypotheses?
Ho: ρ = 0, Ha: ρ ≠ 0
Investigate p-values.
If p-value is smaller than α (typically 0.05), reject Ho.
from Intermediate Statistics for Dummies

48. Checking for Multicolinearity
Look at the relationship between the x variables themselves and check for redundancy.
Multicolinearity – two x variables are highly correlated.
If two x variables are significantly correlated, include only one.
If you include both, the computer won’t know what numbers to give as coefficients for each of the two variables, because they share their contribution to determining the value of y.
Multicolinearity can really mess up the model-fitting.
from Intermediate Statistics for Dummies

49. Find the best fitting model
from Intermediate Statistics for Dummies

50. The interpretation of coefficients are a little more complicated then in simple linear regression.
The coefficient of an x variable in a multiple regression model is the amount by which y changes if that x variable increases by one and the values of all other x variables in the model don’t change.
Plasma TV sales increases by million dollars when TV ad spending increases by $1,000 and spending on newspaper ads doesn’t change.

51. Testing the Coefficients
Determine whether you have the right x variables in your model.
Test H0: Coef = 0, Ha: Coef ≠ 0

52. Extrapolation: no-no
Do not estimate y for values of x outside their range!
There is no guarantee that the relationship you found follows the same model for distant values of predictors.

53. Checking the fit of model
The residuals have a normal distribution with mean zero.
The residuals have the same variance for each fitted (predicted) value of y.
The residuals are independent (don’t affect each other).

54. normality of residuals homoscedasticity
residuals independence
from Intermediate Statistics for Dummies

55. Adjusted R2
How well the regression line approximates the real data points is measured by the coefficient of determination
R2 = r2.
It tells you how much the variability in the y-value is explained by the model.
R2 increases as we increase the number of variables in the model.
As such, R2 alone cannot be used as a meaningful comparison of models with different numbers of independent variables – adjusted R2.