Validity = accuracy of studyReliability = consistency of results

VALIDITY- Does your study reveal accurate and representative results?

Reliability Is the measure able to give consistent measures over time?

Reliability checks? Test-retest Split-half Inter-observer Convergence…replications

VALIDITY- Does your study reveal accurate and representative results? Internal Validity and External Validity –External Validity- generalizability Issues »Sampling »Return rates »…..

…External Validity Generalizeability –Across population –Across environments –Across time

Internal Validity

Convergent validity Different manipulations Different measuring devices Different populations…etc Replication

COMMON THREATS TO VALIDITY Selection bias –random selection and assignment –Pretesting and matching –Recruitment technique

History and Maturation Effects Changes in the DV over time Between pretest and post-test Or between test sessions –constancy

Mortality Subject loss Over time May reflect unintended effect of IV or DV Subject loss may result in selection bias

STOP FOR FUN! ESP in PSYCHOLOGY?? –Somewhat critical –Lacking empirical validation –Should we remain open-minded? –Let’s give it a shot

One of these symbols on each trial

STOP until test complete

What is Chance Performance? -1 out of 4 for each trial… -20 trials Chance performance = 5

Just for fun let’s try again..this time seriously!

answers

Statistical Regression Regression to the mean with repeated testing Watch out for extreme scores..they may be due to chance

Instrumentation Changes in DV due to changes in measurement, not due to IV

Sequence Effects-repeated testing effects Practice effects Test sensitization Boredom/fatigue Carry-over –Randomization and counterbalancing

Subject Bias Subjects can have attitude! RESPONSE SETS that produce inaccurate results –Soc desireability –Cooperitivity –Negativity –Volunteerism –-blindin-recruitment -deception or disguise lie detectors

Experimentor Bias Experimentors like to be correct! CONFIRMATIONAL BIAS Demand characteristics –constancy –Automation –Double blinding

How would you use blinding in a MJ drug study?

PLACEBO EFFECTS ARE REAL Expectation What might someone expect MJ to do? Beliefs/expectations can lead to real results….that are not related to the real effects of the IV.

DOUBLE-BLIND PLACEBO CONTROL DESIGNS Fake MJ?/ Ciggarette/ Pill other Constancy Can tease apart placebo effects from real drug effects.

RANGE EFFECTS Don’t forget the inability to see an affect of an IV on a DV does not necessarily mean there isn’t one. Ceiling effects Basement effects

ULTIMATE CONTROL Control procedures may be tailored for unique experiments Ultimate control may never be completely achieved

BEING A REAL PSYCHOLOGICAL SCIENTIST?? The crux- identification of potential confounds AND RIVAL HYPOTHESIS

Alternative explanations for experimental results Alternative to the research hypothesis

Consider the following Hampton Court Maze

The radial arm maze for rats

Training of rats on the RAM Initial freezing Adaptation and exploration Neophobia to food pellet Adaptation to food Rapid acquisition of task

RAM memory errors? Entrance into previously entered arm One foot? Two? Half-way? All the way?

Rats become quite good Rarely make a memory error Win-shift strategy

Maybe they take an algorithmic strategy Right-right-right etc

To control for strategies..forced choice procedures

Using forced-choice procedures No indication of algorithm Performance still very good

SO… What does rat performance in RAM show us?

Rival Hypothesis? Memory or sensory guidance? Intra-maze or extramaze guidance? Spatial memory or…..?

Motivation-deficits Sensory-deficits Attention-deficits