1. Research Methods, Design, and Analysis Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner CHAPTER Twelfth Edition Measuring Variables and Sampling 5

2. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Variable and Measurement Variable –a condition or characteristic that can take on different values or categories –e.g., gender, reaction time Measurement –the assignment of symbols or numbers to something according to a set of rules –gender – male/female –reaction time – minutes or seconds

3. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Scales of Measurement Stevens (1946) –measurement can be categorized by the type of information that is communicated by the symbols assigned to the variables of interest Nominal scale –use of symbols to classify or categorize –e.g., gender, ethnicity, religion, major in college Ordinal scale –rank-order scale of measurement –equal distances on scale not necessarily equal on dimension being measured –e.g., finishing order in a race, letter grades (ABCDF), SES (low, medium, high)

4. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Scales of Measurement Interval scale –equal distances between adjacent numbers –e.g., temperature on Fahrenheit or Celsius scale Ratio scale –highest scale of measurement –same properties of other scales plus absolute zero point –e.g., weight, height, number grades, temperature on Kelvin scale, reaction time, length

5. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Scales of Measurement

6. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Scales of Measurement

7. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Scales of Measurement

8. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Psychometric Properties of Good Measurement Reliability –refers to the consistency or stability of the scores of your measurement instrument Validity –refers to the extent to which your measurement procedure is measuring what you think it is measuring and whether you have interpreted your scores correctly A measure must be reliable in order to be valid, but a reliable measure is not necessarily valid

9. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner A Portrayal of High, Moderate, and Low Reliability

10. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Types of Reliability Reliability –refers to the consistency or stability of the scores of your test, assessment, instrument, or raters Test-retest reliability –consistency of individual scores over time –same test administered to individuals two times –correlate scores to determine reliability –how long to wait between tests  typically an increase in time between testings will decrease reliability Equivalent-forms reliability –consistency of scores on two versions of test –each version of test given to the same group of individuals –e.g., SAT, GRE, IQ

11. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Test–Retest Reliability Good reliabilityBad reliability

12. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Types of Reliability Internal consistency reliability –consistency with which items on a test measure a single construct –e.g., learning, extraversion –involves comparing individual items within a single test –coefficient alpha (Cronbach’s alpha) is common index  should be +0.70 or higher  multidimentional tests will generate multiple coefficient alphas

13. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Types of Reliability Interrater reliability –degree of agreement between two or more observers (raters) –how is interobserver agreement calculated? –nominal or ordinal scale  the percentage of times different raters agree –interval or ratio scale  correlation coefficient

14. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Sampling Methods Sample –a set of elements selected from a population Population –the full set of elements or people from which the sample was selected Sampling –process of drawing elements from population to form a sample Representative sample –a sample that resembles the population Equal probability method of selection method (EPSEM) –each individual element has an equal probability of selection into the sample

15. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Sampling Techniques Simple random sampling –choosing a sample in a manner in which everyone has an equal chance of being selected (EPSEM) –sampling “without replacement” is preferred –random numbers generators simplify the process  www.randomizer.org  www.random.org

16. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Sampling Techniques Stratified random sampling –random samples drawn from different groups or strata within the population  groups should be mutually exclusive  strata can be categorical (nominal or ordinal) or quantitative (interval or ratio)  proportional stratified sampling –involves insuring that each subgroup in sample is proportional to the subgroups in the population

17. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Sampling Techniques Stratified random sampling example (proportional) –strata – gender (males/females) –population – presidents of APA, N = 122  14 female presidents (11%)  108 male presidents (89%) –sample – n = 100  11 female presidents drawn randomly  89 male presidents drawn randomly

18. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Sampling Techniques Stratified random sampling example (proportional)

19. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Sampling Techniques Cluster random sampling –involves random selection of groups of individuals –clusters  a collective type of unit that includes multiple elements (has more than one unit in it)  e.g., neighborhoods, families, schools, classrooms –one-stage cluster sampling  randomly select clusters and using all individuals within  e.g., randomly select 15 psychology classrooms using all individuals in each classroom –two-stage cluster  randomly select clusters AND  randomly choosing individuals within each chosen cluster  e.g., randomly select 30 psychology classrooms then randomly select 10 students from each of those classrooms

20. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Sampling Techniques Cluster random sampling

21. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Sampling Techniques Systematic sampling –involves three steps 1.determine the sampling interval (k)  population size divided by desired sample size 2.randomly select a number between 1 and k, and include that person in your sample 3.also include each kth element in your sample  periodicity  potential, but uncommon, problem  problematic situation in systematic sampling that can occur if there is a cyclical pattern in the sampling frame

22. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Sampling Techniques Systematic sampling example –population N = 100 –sample n = 10 –k = 10 –randomly select a number between 1 and 10  e.g., 5 –the 5 th person in the population will be included in the sample along with person #15, 25, 35, 45, 55, 65, 75, 85, and 95

23. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Sampling Techniques Systematic sampling

24. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Nonrandom Sampling Techniques Generally produce biased, non-representative samples Convenience sampling –using research participants that are readily available –e.g., college students Quota sampling –identifying quotas for individual groups and then using convenience sampling to select participants within each group –e.g., gender – 25 males and 25 females –e.g., year in school – 15 freshman, 15 sophomores, 15 juniors, and 15 seniors

25. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Nonrandom Sampling Techniques Purposive sampling –involves identifying a group of individuals with specific characteristics –e.g., college freshmen who have been diagnosed with ADHD Snowball sampling –technique in which research participants identify other potential participants –particularly useful in identifying participants from a difficult to find population –e.g., Spanish speaking ESL students, parents of children with autism

26. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Random Selection and Random Assignment Random selection –involves selecting participants for research from the population to be included in the sample –purpose is to obtain a representative sample Random assignment –involves how participants are assigned to conditions within the research –purpose is to create equivalent groups to allow for investigation of causality –e.g., 20 college students sign up to be participants in a study. Each is randomly assigned to the treatment or control group of the study

27. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Determining Sample Size Simple rules for determining sample size 1.if less than 100, use entire population 2.larger sample sizes make it easier to detect an effect or relationship in the population 3.compare to other research studies in area by doing a literature review 4.use a sample size calculator (e.g., G-Power)

28. Copyright ©2014 by Pearson Education, Inc. All rights reserved. Research Methods, Design, and Analysis, Twelfth Edition Christensen Johnson Turner Determining Sample Size Larger sample sizes are needed if population is –heterogeneous  composed of widely different kinds of people –you want to breakdown the sample into multiple subcategories  e.g., look at males and females separately –if you want to obtain a narrow or more precise confidence interval –when you expect a small effect or weak relationship –when you use less efficient methods of sampling  e.g., cluster sampling –for some statistical techniques –if you expect a low response rate