1. Statistic & Research Methodology II Week 1, Session 1

2. Recap from SRM I: The research process Statement of the problem Design of research study Measurement of variables Analysis of data Conclusions from research SRM I SRM II Your other theoretical courses

3. Syllabus Goals Grading components Office hours

4. Research in psychology Quantitative Purpose Description Prediction Causation Method Experiments Survey Archival Qualitative (take sociology courses) Purpose Description Method Ethnography Focus group Interviews Archival

5. Psychometrics The theory and technique of psychological measurement

6. What is psychometrics? Psychometrics is that area of psychology that specializes in how to measure what we talk and think about. How to measure psychological constructs? What are the measurement problems involved?

7. What is a measurement? A quantifiable attribute Determines whether the objects fall in the same or different category with respect to an attribute How much of …?How fast …? How long…?How many…?

8. Variable: Some property of an object, phenomenon, or event whose measurement can take on two or more values Independent/dependent Predictor/criterion Continuous/discrete Qualitative/quantitative Recap: Measurement of Variables

9. Recap: Scales of measurement The four basic types of scales are: Nominal scales Ordinal scales Interval scales Ratio scales Recall: These dictate the type of statistical tests you can run.

10. What is a psychological measurement? A quantifiable psychological attribute How much of …?How fast …? How long…?How many…?

11. Can you really measure psychological attributes? Most psychological attributes are: latent: present but invisible abstract Some examples: Intelligence Evilness Memory Hence they need to be inferred.

12. The problem How do you make strong inferences?

13. Reliability and Validity The first step towards measurement

14. Reliability and measurement error Measurements, even in natural sciences, are imperfect. What is your weight? 55? 55.1? 55.13? Wait…did you just have a meal? Did you just empty your bladder (-200g) or stools (-123.6g)? Psychological constructs are always measured with error Observed Score = True score + Error of measurement

15. Sources of error Systematic You always way 5 kg more because of that miscalibrated needle Random You read it wrongly, every time

16. Which error is worse? Systematic Can be corrected mathematically (at what cost?) But you must first know that the error is systematic Random Can be cancelled out if multiple measurements are taken – sometimes you score higher, sometimes you score lower In some cognitive experiments, there can be as many as 1600 trials. Why? But it’s generally better to minimize random error

17. Reliability Reliability refers to: 1.the extent to which measurements are repeatable 2.consistency of measurement 3.stability of measurement over a variety of conditions in which basically the same results should be obtained

18. Different types of reliability Reliability Internal consistency Alpha reliability Inter-rater Temporal stability Test-retest reliability

19. Internal Consistency: Alpha reliability When you use multiple items to measure one construct, how do you know they all measure the same construct? They should “hang” together. Alpha reliability (“Cronbach’s alpha) is a function of the number of items and average correlation among them You want the least number of items that have good intercorrelations (α =.80 -.90) SELF- ESTEEM I feel that I have a number of good qualities. I am able to do things as well as most other people. On the whole, I am satisfied with myself.

20. Replication ≠ Reliability Replicability: “How likely will my next finding be statistically significant?” Reliability implies replicability, but using a reliable scale does not guarantee of replicability Why? Replication failure could be due to low power, low reliability, problematic participants, improper methodology, wrong theory, etc.. WARNING: Sometimes you may see articles use the term ‘statistically reliable’ to mean statistically significant. It is grossly misleading terminology and, taken literally, usually wrong.

21. Validity General definition: “Are your measurements actually measuring the construct you think you are measuring?” While we can obtain specific statistics for reliability (even different types), validity is more of a global assessment based on the evidence available We can have reliable measurements that are invalid The history of science has lots of examples…

22. “Science” is filled with mismeasurements

23. Types of validity Internal validity a manager of a company tests employees on leadership satisfaction. Only 50% of the employees responded to the survey and all of them liked their boss. a job satisfaction survey before Christmas just after everybody received a nice bonus. The results showed that all employees were happy You will practice this in “Analyze a research design 1-5”

24. Types of validity Construct How much is it an actual measure of the construct of interest Criterion degree of correspondence between a test measure and one or more external referents (criteria) Predictive SAT  GPA Convergent testing for convergence across different measures or manipulations of the same thing Discriminant testing for divergence between measures and manipulations of related but conceptually distinct things External: how generalizable is this relationship across persons, settings, etc?

25. How would you know if a measure is valid? Statistical methods Criterion Predictive Convergent Discriminant Logical reasoning Internal Construct External

26. Let’s examine the validity of BART The Balloon Analogue Risk task (BART) claims to be a valid measure risk taking in the lab.

27. Principles of the BART Do you think the task has construct validity?

28. Let’s examine the validity of dermatoglyphics Here’s one theory: Do you think the fingerprints are valid measurements of intelligence?

29. Take home message Reliability and validity are key concerns in psychological research, regardless of the domain of psychology, regardless of methodology Validity issues permeate daily life too, but nonstatisticians call it “interpretation” Is staring at other people’s butts a good measure of lust? Or is it a better measure of visual contrast sensitivity?

30. What do you see?

31. Next class We will analyze reliability of two questionnares. Download and install SPSS before coming to class (SPSS is 1 GB). Download the relevant datafile (see Course Manual). We will start the Questionnaire Development Project in the 2 nd half of the lesson.