1. PSYC512: Research Methods PSYC512: Research Methods Lecture 8 Brian P. Dyre University of Idaho

2. PSYC512: Research Methods Lecture 8 Outline Questions about material covered in Lecture 7 Questions about material covered in Lecture 7 Measures: Reliability, Precision, and Validity Measures: Reliability, Precision, and Validity Defining Variables and Research Designs Defining Variables and Research Designs Describing Data Describing Data Testing Hypotheses Testing Hypotheses Inferential Statistics Inferential Statistics

3. PSYC512: Research Methods Understanding Variability What is variability? What is variability? How is variability related to probability? How is variability related to probability?

4. PSYC512: Research Methods Visualizing Variability: Distributions of Frequency and the Histogram Histograms: used to represent frequencies of data in different classes or categories Histograms: used to represent frequencies of data in different classes or categories BinFrequency 00 10 20 30 43 51 66 74 82 96 101

5. PSYC512: Research Methods Displaying Histograms: Stem and Leaf Plots Stem and Leaf plots are used to display histograms graphically (on their side) using only typed characters Stem and Leaf plots are used to display histograms graphically (on their side) using only typed characters StemLeaf(hypothetical histogram for IQ) 678 735668 8012234445555667777889 900011233333334445566667889999 1001112233334444445566677777888899 1001112233334444445566677777888899 110001122233444566777899 110001122233444566777899 120012569 120012569 1302 1302

6. PSYC512: Research Methods Distributions of Probability Density Similar to frequency histogram except y-axis now represents probability density (mass) rather than frequency Similar to frequency histogram except y-axis now represents probability density (mass) rather than frequency Probability density = Frequency/N Probability density = Frequency/N

7. PSYC512: Research Methods Some Types of Probability Density Distributions Normal (Gaussian)Gamma

8. PSYC512: Research Methods Describing Distributions: Estimators and Parameters Sample statistics estimate population parameters, e.g. Sample statistics estimate population parameters, e.g. Sample mean: M or estimate the mean of a population,  Sample mean: M or estimate the mean of a population,  Sample variance: s 2 estimates the variance of a population,  2 Sample variance: s 2 estimates the variance of a population,  2 Properties of Estimators Properties of Estimators Sufficiency: extent to which statistic uses all information (observations) available in sample Sufficiency: extent to which statistic uses all information (observations) available in sample Unbiasedness: extent to which expected value of statistic approaches population value with increased sampling Unbiasedness: extent to which expected value of statistic approaches population value with increased sampling Efficiency: tightness of cluster of sample statistics relative to the population parameter Efficiency: tightness of cluster of sample statistics relative to the population parameter Resistance: extent of influence of outliers on sample statistic Resistance: extent of influence of outliers on sample statistic

9. PSYC512: Research Methods Measures of the Center of a Population or Sample Measures of center represent the general magnitude of scores Measures of center represent the general magnitude of scores Mode: most frequent score Mode: most frequent score Median: the middle score of an ordered list Median: the middle score of an ordered list Mean (average):where X represents a vector of samples and Mean (average):where X represents a vector of samples and N is the total number of observations; Which measures are the most sufficient? Unbiased? Efficient? Resistant? Which measures are the most sufficient? Unbiased? Efficient? Resistant?

10. PSYC512: Research Methods Measures of the Spread of a Population or Sample Measures of spread are used to assess the consistency of scores in a distribution Measures of spread are used to assess the consistency of scores in a distribution Range = max score – min score Range = max score – min score Interquartile range = score(Q3) – score(Q1) Interquartile range = score(Q3) – score(Q1) Variance (  2  s 2 ) and standard deviation (  s) Variance (  2  s 2 ) and standard deviation (  s) where X is a vector of data,  is the mean of the population, is the mean of a sample, and N is the total number of observations Which measures are the most sufficient? Unbiased? Efficient? Resistant? Which measures are the most sufficient? Unbiased? Efficient? Resistant?

11. PSYC512: Research Methods Standard Deviation Standard Deviation (  ) = sqrt(variance) Standard Deviation (  ) = sqrt(variance) where X is the data, m is the mean of the data, and N is the total number of observations Remembering how to compute variance Remembering how to compute variance “the mean of the squares – square of the means”

12. PSYC512: Research Methods Describing Distributions Parametrically: Statistical Moments Any distribution based on interval or ratio data can be summarized by its statistical moments Any distribution based on interval or ratio data can be summarized by its statistical moments First Moment: Mean—location of distribution on x-axis First Moment: Mean—location of distribution on x-axis Second Moment: Variance—dispersion of distribution Second Moment: Variance—dispersion of distribution Third Moment: Skewness—symmetry of distribution Third Moment: Skewness—symmetry of distribution Fourth Moment: Kurtosis—degree of “peakedness” Fourth Moment: Kurtosis—degree of “peakedness”

13. PSYC512: Research Methods Testing Hypotheses Hypothesis testing is the process by which hypothetical relationships between intervening variables are assessed Hypothesis testing is the process by which hypothetical relationships between intervening variables are assessed Hypotheses are always tested relative to one- another or to a “null” hypothesis Hypotheses are always tested relative to one- another or to a “null” hypothesis Examples Examples Comparing groups Comparing groups Assessing performance interventions Assessing performance interventions Assessing relationships between variables Assessing relationships between variables

14. PSYC512: Research Methods Null-Hypothesis Testing and Inferential Statistics 2 possible realities 2 possible realities Relationship between your variables does not exist—a null relationship (H o, the null hypothesis) Relationship between your variables does not exist—a null relationship (H o, the null hypothesis) Relationship between the two variables in question actually exists (H 1, the experimental or alternative hypothesis) Relationship between the two variables in question actually exists (H 1, the experimental or alternative hypothesis) 2 possible decisions when looking at the data 2 possible decisions when looking at the data Conclude that a relationship exists (reject the null hypothesis, H o  DISCONFIRMATION!) Conclude that a relationship exists (reject the null hypothesis, H o  DISCONFIRMATION!) Conclude that no relationship exists (do not reject the null hypothesis  CONFIRMATION? NO!) Conclude that no relationship exists (do not reject the null hypothesis  CONFIRMATION? NO!)

15. PSYC512: Research Methods Null-Hypothesis Testing and Inferential Statistics H o True H o False Reject H o (conclude there is an effect) Type I error (false alarm) Correct Decision Do not Reject H o (conclude there is NOT an effect) Correct Decision Type II error (miss) Decision True State of the World 2 realities by 2 decisions form a 2 x 2 matrix of 4 possibilites

16. PSYC512: Research Methods Hypothesis Testing: Probability and Statistics Problem: How do we distinguish real differences or relationships from measurement noise? Problem: How do we distinguish real differences or relationships from measurement noise? Probability and statistics may be used to assess (descriptive statistics) or compare (inferential statistics) the relative magnitude of different types of variability Probability and statistics may be used to assess (descriptive statistics) or compare (inferential statistics) the relative magnitude of different types of variability Effect (treatment) Variance Effect (treatment) Variance Variability due to relationship between variables or effect of different levels of independent variable (treatments) Variability due to relationship between variables or effect of different levels of independent variable (treatments) “Good” variance that we want to maximize “Good” variance that we want to maximize Error Variance Error Variance Variability in measure due to factors other than the treatment Variability in measure due to factors other than the treatment “Bad” variance that we want to minimize “Bad” variance that we want to minimize

17. PSYC512: Research Methods Hypothesis Testing: Inferential Statistics All inferential statistics are evaluating this ratio: All inferential statistics are evaluating this ratio: Effect (good) Variance Test statistic = -------------------------------------- Error (bad) Variance Error (bad) Variance Example test statistics: Chi-square, t, F Example test statistics: Chi-square, t, F These test statistics have known distributions that then allow us to estimate p, the probability of a Type I error (inappropriately rejecting the null hypothesis) These test statistics have known distributions that then allow us to estimate p, the probability of a Type I error (inappropriately rejecting the null hypothesis) Decision to reject null is made by comparing p to some generally accepted criterion for Type I error probability,  Decision to reject null is made by comparing p to some generally accepted criterion for Type I error probability, 

18. PSYC512: Research Methods Null-Hypothesis Testing and Inferential Statistics Why might we observe a difference between two groups if no difference actually exists (null is true; samples are drawn from the same population)? Why might we observe a difference between two groups if no difference actually exists (null is true; samples are drawn from the same population)? Each sample may have a unique mean due to sampling error Each sample may have a unique mean due to sampling error Frequency  1 Population 2 samples

19. PSYC512: Research Methods Null-Hypothesis Testing and Inferential Statistics How does this change if a difference actually exists between my groups? How does this change if a difference actually exists between my groups? Each sample has a unique mean that represents both sampling error and the differences between the 2 populations Each sample has a unique mean that represents both sampling error and the differences between the 2 populations Frequency  2 Populations  Frequency

20. PSYC512: Research Methods How is p calculated? It depends on… the scaling properties of your dependent variable (DV) the scaling properties of your dependent variable (DV) DV is interval or ratio  parametric tests DV is interval or ratio  parametric tests DV is nominal or ordinal  non-parametric tests DV is nominal or ordinal  non-parametric tests Research design Research design Experimental – test differences on measure between conditions or groups  t-test, ANOVA, sign test, Mann- Whitney Experimental – test differences on measure between conditions or groups  t-test, ANOVA, sign test, Mann- Whitney Correlational – test relations between different measures  Pearson product-moment correlation, point-biserial correlation, etc. Correlational – test relations between different measures  Pearson product-moment correlation, point-biserial correlation, etc. the manner in which you phrase your hypotheses the manner in which you phrase your hypotheses One tailed vs. two-tailed tests One tailed vs. two-tailed tests

21. PSYC512: Research Methods Next Time… Topic: Normality, Probability, Nuts and Bolts of Testing Hypotheses Topic: Normality, Probability, Nuts and Bolts of Testing Hypotheses Be sure to: Be sure to: Read the assigned readings (Howell chapters 6-7) Read the assigned readings (Howell chapters 6-7) Continue searching and reading the scientific literature for your proposal Continue searching and reading the scientific literature for your proposal