Measurements and mismeasurements in Psychology Week 1, Session 1

About me…  Kai Qin (“Kai Chin”)  Professor  Sir  Teacher teachingpsychology.weebly.com

How to enjoy and do well in this course? Class attendance Critical reading, thinking, writing Do not hesitate to ask questions during class! Be punctual

Critical thinking in psychology

Syllabus  Goals  Grading components  Office hours

Science & Measurements

Science  Measurement is a defining characteristics of science

What is a measurement?  A quantifiable attribute How much of …?How fast …? How long…?How many…?

What is a psychological measurement?  A quantifiable psychological attribute How much of …?How fast …? How long…?How many…?

We make statements such as…  Politicians are greedy…  The driver is rude…  The professor is smart.  This class is awesome!!! All these require you to measure the constructs

How would you measure…  Greed  Rudeness  Intelligence  Enjoyment

Psychologists are obsessed with measurements  circa 1000 BC. : Chinese introduced written tests to help fill civil service positions Civil Laws, Military Affairs, Agriculture, Geography  1850 : The United States begins civil service examinations.  1885 : Germans tested people for brain damage  1890 : James Cattell develops a "mental test" to assess college students. Test includes measures of strength, resistance to pain, and reaction time.  1905 : Binet-Simon scale of mental development used to classify mentally retarded children in France.

Psychologists are obsessed with measurements  1914 : World War I produces need in U.S. to quickly classify incoming recruits. Army Alpha test and Army Beta test developed. Looked at psychopathology.  1916 : Terman develops Stanford - Binet test and develops the idea of Intelligence Quotient  : factor analysis, projective tests, and personality inventories first appear.  : vocational interest measures developed  : item response theory and neuropsychological testing developed  Present : Wide spread adaptation of computerized testing. "Smart" Tests which can give each individual different test items develop

What about India?

Can you really measure psychological attributes?  Most psychological attributes are:  latent: present but invisible  abstract  Some examples:  Intelligence  Evilness  Memory  Hence they need to be inferred.

The problem  How do you make strong inferences?

The BIG QUESTION throughout the course  When we measure something, what are we really measuring? Brain activity? Really? Always? Religiosity?

Validity

 General definition: “Are your measurements actually measuring the construct you think you are measuring?”  While we can obtain specific statistics for reliability (even different types), validity is more of a global assessment based on the evidence available  We can have reliable measurements that are invalid  The history of science has lots of examples…(see Course Manual)

One approach of critical thinking Claim  Search for evidence  Example: Touch therapy, magnet therapy WHAT IS THE EVIDENCE?

Does touch therapy work?  Practitioners of TT claim to treat many medical conditions by using their hands to manipulate a “human energy field” perceptible above the patient’s skin.  Claims Made for TT  Relieves burns pain, nausea, fever  Treats ulcers, measles, cancer  Note: TT ≠ Massage therapy Rosa, L., Rosa, E., Sarner, L., & Barrett, S. (1998). A close look at therapeutic touch. J Am Med Assoc, 279, 1005.

Finding evidence for TT  If TT was true, then practitioners should have been able to locate the investigator’s hand 100% of the time.  A score of 50% would be expected through chance alone. Rosa, L., Rosa, E., Sarner, L., & Barrett, S. (1998). A close look at therapeutic touch. J Am Med Assoc, 279, 1005.

Experiment  Sample: 21 practitioners  Double-blinded study  “Patient” had palms under practitioners’ hands: Yes vs. No  280 trials for each practitioners  Correct response? 123 (44%) of 280 trials Rosa, L., Rosa, E., Sarner, L., & Barrett, S. (1998). A close look at therapeutic touch. J Am Med Assoc, 279, 1005.

Does magnet therapy make sense?

The approach of this CTS  We assume the experiment has been done. But we ask…  WHAT IS THE EVIDENCE ACTUALLY TELLING YOU?  Is the measurement… Valid? Reliable? Credible?

Measurements matter in real life  Clinical Psychologists (intelligence, psychopathology)  Counseling Psychologists (career interest, skill assessment)  School Psychologists (readiness for school, social Adjustment)  I/O Psychologists (managerial potential, training needs)  Neuropsychologists (brain damage, neurological impairments)  Forensic Psychology (recidivism risk)

Why do measurements matter?  Often we don’t want a measure that merely describes an attribute.  Often we want a measure to predict.

One can get really rich… even if one is full of s*** (rhyme unintended)

Why do some people use flawed measurements?

A good scientist…  Doesn’t overstate his or her claims  Acknowledges limitations  Tries to rule out alternative explanation  Is not defensive when questioned  Relies on evidence rather than emotional argument

The Story of Facilitated Communication  

Take home message “Any sufficiently advanced technology is indistinguishable from magic.”

My aim for you Be skeptical, but not cynical.

Learning goal  Psychology will not give you answers to many questions.  But it will equip you with the skills needed to evaluate the answers to these questions that others – and yourself – have given.

Announcements  No class next week  Complete the online questionnaires (see Course Manual for url)  Read the Course Manual