Mini Quiz 1. Data that derive from the researcher's direct observation of what the subject does in some predefined context are a. L data. b. I data.

Slides:



Advertisements
Similar presentations
Measurement Concepts Operational Definition: is the definition of a variable in terms of the actual procedures used by the researcher to measure and/or.
Advertisements

The Research Consumer Evaluates Measurement Reliability and Validity
Part II Sigma Freud & Descriptive Statistics
Part II Sigma Freud & Descriptive Statistics
Measurement Reliability and Validity
LECTURE 9.
Test Validity: What it is, and why we care.
VALIDITY.
Assessing Personality
Assessing Personality
Reliability and Validity. Criteria of Measurement Quality How do we judge the relative success (or failure) in measuring various concepts? How do we judge.
Measurement Concepts & Interpretation. Scores on tests can be interpreted: By comparing a client to a peer in the norm group to determine how different.
VALIDITY, RELIABILITY, and TRIANGULATED STRATEGIES
Analyzing Reliability and Validity in Outcomes Assessment (Part 1) Robert W. Lingard and Deborah K. van Alphen California State University, Northridge.
Technical Adequacy Session One Part Three.
Principles of Test Construction
Chapter 4: Test administration. z scores Standard score expressed in terms of standard deviation units which indicates distance raw score is from mean.
Chapter 2 Doing Social Psychology Research. Why Should You Learn About Research Methods?  It can improve your reasoning about real-life events  This.
The Basics of Experimentation Ch7 – Reliability and Validity.
Selecting a Sample. Sampling Select participants for study Select participants for study Must represent a larger group Must represent a larger group Picked.
CHAPTER OVERVIEW The Measurement Process Levels of Measurement Reliability and Validity: Why They Are Very, Very Important A Conceptual Definition of Reliability.
Research Methodology and Methods of Social Inquiry Nov 8, 2011 Assessing Measurement Reliability & Validity.
Validity and Item Analysis Chapter 4.  Concerns what instrument measures and how well it does so  Not something instrument “has” or “does not have”
You can calculate: Central tendency Variability You could graph the data.
Correlations Small =.10 Medium =.30 Large =.50 Basic Steps 1) Create a test 2) Validate the test 3) Use the test.
1 Psychology 305A: Personality Psychology January 14 Lecture 3.
Chapter 7 Measuring of data Reliability of measuring instruments The reliability* of instrument is the consistency with which it measures the target attribute.
Reliability performance on language tests is also affected by factors other than communicative language ability. (1) test method facets They are systematic.
You can calculate: Central tendency Variability You could graph the data.
STATISTICS STATISTICS Numerical data. How Do We Make Sense of the Data? descriptively Researchers use statistics for two major purposes: (1) descriptively.
Measurement Experiment - effect of IV on DV. Independent Variable (2 or more levels) MANIPULATED a) situational - features in the environment b) task.
Chapter 6 - Standardized Measurement and Assessment
©2005, Pearson Education/Prentice Hall CHAPTER 6 Nonexperimental Strategies.
TEST SCORES INTERPRETATION - is a process of assigning meaning and usefulness to the scores obtained from classroom test. - This is necessary because.
Personality Assessment DAPT Barnum Effect People’s willingness to interpret vague, general statements as personally meaningful interpretations of their.
Measurement Chapter 6. Measuring Variables Measurement Classifying units of analysis by categories to represent variable concepts.
WHS AP Psychology Unit 7: Intelligence (Cognition) Essential Task 7-3:Explain how psychologists design tests, including standardization strategies and.
ESTABLISHING RELIABILITY AND VALIDITY OF RESEARCH TOOLS Prof. HCL Rawat Principal UCON,BFUHS Faridkot.
Reliability and Validity
Personality Assessment, Measurement, and Research Design
Data and the Nature of Measurement
Unit 8: Intelligence (Cognition)
Ch. 5 Measurement Concepts.
Mini Quiz 1. Julie is a serious and conscientious employee; however, with her friends at Happy Hour on Friday afternoon she is the "life of the party."
Lecture 5 Validity and Reliability
Mini Quiz 1. Data that derive from the researcher's direct observation of what the subject does in some predefined context are a. L data. b. I data.
Assessing Personality
Concept of Test Validity
Assessment Theory and Models Part II
Overview of Research Process
Tests and Measurements: Reliability
Journalism 614: Reliability and Validity
Reliability & Validity
Remember No Class on Wednesday No Class on Friday.
How Do Psychologists Ask & Answer Questions?
Introduction to Measurement
Human Resource Management By Dr. Debashish Sengupta
پرسشنامه کارگاه.
Is language uniquely human?
Reliability and Validity of Measurement
You can calculate: Central tendency Variability You could graph the data.
Analyzing Reliability and Validity in Outcomes Assessment Part 1
Mini Quiz 1. Data that derive from the researcher's direct observation of what the subject does in some predefined context are a. L data. b. I data.
RESEARCH METHODS Lecture 18
Assessing Personality
You can calculate: Central tendency Variability You could graph the data.
Personality Assessment, Measurement, and Research Design
An Introduction to Correlational Research
Some statistics questions answered:
Analyzing Reliability and Validity in Outcomes Assessment
Presentation transcript:

Mini Quiz 1. Data that derive from the researcher's direct observation of what the subject does in some predefined context are a. L data. b. I data. c. S data. d. B data.

Mini Quiz 2. _____________________ ask a respondent to interpret a meaningless, ambiguous stimulus in order to access the inner workings of the person's mind. a. Rationally constructed tests b. Projective tests c. Factor analytic tests d. Objective tests

Mini Quiz 3. The basis of the _____________ method of test construction is to come up with items that seem directly, obviously, and logically related to what it is you wish to measure. a. rational b. empirical c. philosophical d. factor analytic

Mini Quiz 4. For any rationally constructed personality scale to work, it must satisfy which of the following conditions? a. The items on the form must all be valid indicators of what the tester is trying to measure. b. The person who completes the form must be willing to accurately report his or her self-assessment. c. Each item must mean the same thing to the person who fills out the form as it did to the psychologist who wrote it. d. All of the above conditions must be satisfied for the scale to work.

Mini Quiz 5) Which one? A) B B) D C) A D) C

Methods for creating a test Rational Method Projective Tests Factor Analytic Method Empirical Method Combination of Methods

Basic Steps 1) Create a test 2) Validate the test 3) Use the test

Statistics

Correlation

Positive Correlation

Positive Correlation

Positive Correlation r = 1.00

Positive Correlation . . . . . r = .64

Negative Correlation

Negative Correlation r = - 1.00

Negative Correlation . . . r = - .85 . .

Zero Correlation

Zero Correlation . . . . . r = .00

Correlation Coefficient The sign of a correlation (+ or -) only tells you the direction of the relationship The value of the correlation only tells you about the size of the relationship (i.e., how close the scores are to the regression line)

Which is a bigger effect? r = .40 or r = -.40 How are they different?

Practice Do you think the following variables are positively, negatively or uncorrelated to each other? Alcohol consumption & Driving skills Miles of running a day & speed in a foot race Height & GPA Forearm length & foot length

Interpreting a Correlation What does it actually mean in “people words”? Binomial Effect Size Display (BESD)

BESD 200 subjects (all sick) Drug given to 100 of them At the end: 100 live and 100 die If the effect of the drug was .00 – what does that mean?

BESD When r = .00 Alive Dead Total Drug 50 100 No Drug 200

BESD 200 subjects (all sick) Drug given to 100 of them At the end: 100 live and 100 die What if the drugs effect was .40 – what does that look like?

BESD When r = .40 Alive Dead Total Drug 70 30 100 No Drug 200

BESD Alive Dead Total Drug 70 30 100 No Drug 200 Thus, if you take the drug you have a 70% chance of living compared to only 30% if you do not take the drug! BESD When r = .40 Alive Dead Total Drug 70 30 100 No Drug 200

BESD How to compute: 200 subjects (all sick) Drug given to 100 of them At the end: 100 live and 100 die Drugs effect was .30

BESD When r = .30 Alive Dead Total Drug 100 No Drug 200

BESD Alive Dead Total Drug 100 No Drug 200 1) Compute cell values if r = .00 BESD When r = .30 Alive Dead Total Drug 100 No Drug 200

BESD Alive Dead Total Drug 50 100 No Drug 200 1) Compute cell values if r = .00 BESD When r = .30 Alive Dead Total Drug 50 100 No Drug 200

BESD Alive Dead Total Drug 50 100 No Drug 200 1) Drop the decimal (30) 2) Divide by 2 (30 / 2 = 15) 3) Add to number in upper left cell (50 + 15 = 65) BESD When r = .30 Alive Dead Total Drug 50 100 No Drug 200

BESD Alive Dead Total Drug 65 50 100 No Drug 200 Plug in value Compute other cell values BESD When r = .30 Alive Dead Total Drug 65 50 100 No Drug 200

BESD Alive Dead Total Drug 65 35 100 No Drug 50 200 Plug in value Compute other cell values BESD When r = .30 Alive Dead Total Drug 65 35 100 No Drug 50 200

BESD Alive Dead Total Drug 65 35 100 No Drug 200 Plug in value Compute other cell values BESD When r = .30 Alive Dead Total Drug 65 35 100 No Drug 200

BESD Practice Create BESDs for the following r = .10 r = .55 r = .80

Correlations Small = .10 Medium = .30 Large = .50

Basic Steps 1) Create a test 2) Validate the test 3) Use the test

Validating a Test Is a test measuring what it is suppose to measure? Not a YES or NO answer Types of Validity 1) Predictive 2) Concurrent 3) Content 4) Construct

Predictive and Concurrent Called “Criterion-orientated” validity Does the test predict some type of criterion? Predictive – criterion is in the future Concurrent – criterion is in the present

Predictive and Concurrent Predicting likeability in Internet Chat Rooms

Participants One-on-One Condition 84 participants 71% female 29% male Markey & Wells, 2002

Likeability * p < .05

Content Validity Test items represent the entire “universe” of possible items

Group Activity What qualities does the trait of extraversion contain? e.g., assertiveness Come up with at least 6 qualities Create one item to measure each quality.

Content Validity Extraversion

Content Validity Positive Emotions Warmth Gregariousness Excitement-Seeking Assertiveness Activity Extraversion

Content Validity Have a lot of fun. Make friends easily Positive Emotions Love large parties Warmth Gregariousness Excitement-Seeking Love excitement. Assertiveness Activity Take charge. Am always busy.

Content Validity Not normally established empirically Established by experts in the field

Construct Validity The test actually gauges the personality dimension being measured. How can you do this? First need to determine if its internal structure is correct.

Content Validity Extraversion

Content Validity Positive Emotions Warmth Gregariousness Excitement-Seeking Assertiveness Activity Extraversion

Construct Validity Next, you need to create a “model” of the construct

Extraversion

Reliable Talking Risk taker Extraversion Eye-contact Stimulation Height

Reliable Talking Risk taker Nomological Net Contains both *Criterion-orientated validity *Discriminant validity Extraversion Eye-contact Stimulation Height

Reliable Talking Risk taker -.04 .60 .54 Extraversion .32 .44 .11 Eye-contact Stimulation Height

Construct Validity Uses all types of validity to determine if a test actually gauges the personality dimension being measured There is actually even more than can be done to examine construct validity -e.g., “changes over time” Note: you NEVER get a single number that represents “construct validity”

Basic Steps 1) Create a test 2) Validate the test 3) Use the test

Basic Steps 1) Create a test 2) Validate the test 3) Use the test Rational Method Projective Tests Factor Analytic Method Empirical Method Combination of Methods 2) Validate the test Predictive Concurrent Content Construct 3) Use the test

What is a trait of personality? A “unit” of measurement Allport presents 8 Criteria that define a personality trait

Traits 1) A trait has more than nominal existence Traits are real!

Traits 2) A trait is more generalized than a behavior lie steal cheat

Traits 2) A trait is more generalized than a behavior lie steal Honesty cheat

Traits 2) A trait is more generalized than a behavior There are systems of behaviors Traits my embrace anywhere between 2 to hundreds of behaviors

Traits 3) A trait is dynamic A trait CAUSES behavior, it is not just a summary of behavior lie steal YES! Honesty cheat

Traits 3) A trait is dynamic A trait CAUSES behavior, it is not just a summary of behavior lie steal NO! Honesty cheat

Traits 4) The existence of a trait my be established empirically or statistically Statistical techniques can be used to examine coherence among behaviors

Traits 5) Traits are only relatively independent of each other It will be difficult to isolate “fundamental” traits that are completely independent of each other

Traits 6) A trait of personality, psychologically considered, is not the same as a moral quality.

Traits 7) Acts, and even habits, that are inconsistent with a trait are not proof of the non-existence of the trait Not realistic to expect perfect consistency Some traits not important in some people Traits interact with each other within a person Context also determines behavior

Traits 8) A trait may be viewed either in the light of the personality which contains it, or in the light of its distribution in the population at large. Some traits are unique and some are universal Can examine either Universal traits across people Unique blends of traits within a person

Using Traits to Understand Behavior Different approaches: The Single-Trait Approach The Many-Trait Approach The Essential-Trait Approach The Simultaneous-Trait Approach