Research Design 10/3/2013

Graduate & Professional School Fair: Oct. 7 Explore Graduate, Law, Medical and Professional Schools THIS coming Monday! – Monday, October 7, 2013 – 10:30 a.m. – 1:30 p.m.. Mabee Ballrooms, 3 rd floor of the Ragsdale Center 48 schools & 2 test prep services will be participating.

Readings Chapter 4 Research Design and the Logic of Control (Pollock)

OPPORTUNITIES TO DISCUSS COURSE CONTENT

Office Hours For the Week When – Friday – Monday – And by appointment

Homework (Due today) Chapter 2 – Question 1: A, B, C, D, E – Question 2: B, D, E (this requires a printout) – Question 3: A, B, D – Question 5: A, B, C, D – Question 7: A, B, C, D – Question 8: A, B, C

Course Learning Objectives 1. First, students will learn the research methods commonly used in behavioral sciences 2.Students will learn the basics of research design and be able to critically analyze the advantages and disadvantages of different types of design.

What a Research Design Includes Unit of Analysis Variables Data Hypotheses Justification for your statistics

The Goal of A Research Design is to create a study that can demonstrate causality

INTERNAL VALIDITY OF RESEARCH DESIGN Working for Causality

Internal Validity Setting up Research Designs Properly Having control over the experiment. Especially the independent variable.

Threat 1:History You cannot account for all previous knowledge and events You cannot control for all potential independent variablesvariables

An Example

Threat 2: Maturation We get older We get wiser We get tired (short term) These are natural changes

Threat 3: Experimental Mortality Participants leave the research study Those who remain, may not be like the target group

Threat 4: Selection Bias Choosing the wrong sample Picking Respondents to favor your results Picking Respondents Excluding cases or respondents that do not fit your goals Using volunteers!

Threat 5: Instrumentation A Bad Measure Changing a Measure to Fit your Needs

Threat 6: Design Contamination People intentionally or unintentionally act differently “Instrument Reactivity” We Guess the test, we share informationinformation

Hawthorne Effect

Which of these are Most Common? History Maturation Contamination is the worst!

EXTERNAL VALIDITY Making it Generalizable

What is External Validity Can we generalize our research Does it apply to other populations, settings and times. Does the test apply to the real world

The Lowest Form “Remember when is the lowest form of conversation” conversation Personal experience is the weakest form of evidence Stories Hypothetical Examples

Unique Program Features/Location Can it only work in one place Does the artificial setting harm validity Creaming (popular with education policy)

Effects of Selection This is also a threat to internal validity Subjects in study are unrepresentative Using Volunteers Poorly Drawn Samples

Reactive Effects of Experiments Taking the experiment out to the real world Participants act differently because they know they are being studied Administrators work differently as well.

Replication Can You Reproduce your findings? Can others reproduce your findings?

Replication

Biggest problems of External Validity Failure to replicate the exam. The Lack of Real World-Applicability To increase external validity, increase your sample!

The Goal Is both kinds of validity. The Social Sciences do better on external than internal

Research Design Experimental vs. Non Experimental

Types of Designs Experimental (mirror the natural sciences Non-experimental- sacrifice internal for external

CLASSIC EXPERIMENTAL DESIGN

The Classic Experimental Design This design is the best way to demonstrate causality You have total control This has 4 key parts

Step 1- Random Selection and Assignment 1.Subjects are selected at random and assigned to an experimental or control Group – You cannot pick people who you want to be in the experiment – You have two equal groups of participants – This ensures an even baseline between your group

Step 2- A pre-test is given to both groups 2. A Pretest measuring the Dependent variable is given to each group

Step 3- The introduction of the independent variable 3.The experimental group receives the independent variable (test stimulus) and the control group does not

Step 4- A Post Test is Given 4.The researcher measures the dependent variable for both groups after the experimental stimulus is given.

What You want to show R= Random assignment O 1 & O2 Observation for the two groups at time 1 X =Introduction of the treatment for the experimental group O3 & O4 Observation of the two groups at time 2 Note change for the two groups (overtime and between)

This type of design is common In the natural sciences In Psychology In Food TestsTests

It is rare in Social Sciences Legal and Ethical Problems Historical Problems Reactivity Problems.