Copyright © Allyn & Bacon (2007) Hypothesis Testing, Validity, and Threats to Validity Graziano and Raulin Research Methods: Chapter 8 This multimedia.

Slides:



Advertisements
Similar presentations
Allyn & Bacon 2003 Social Work Research Methods: Qualitative and Quantitative Approaches Topic 6: Basic Social Work Research Designs View.
Advertisements

Chapter 2: Nonexperimental Research Approaches This multimedia product and its contents are protected under copyright law. The following are prohibited.
Chapter 3: The Experimental Research Approach Introduction Introduction –Goal is to discover the effects of presumed causes This multimedia product and.
Chapter 8: Construct and External Validity in Experimental Research
Chapter 4: Problem Identification and Hypothesis Formation This multimedia product and its contents are protected under copyright law. The following are.
Copyright © Allyn & Bacon (2007) Single-Variable, Independent-Groups Designs Graziano and Raulin Research Methods: Chapter 10 This multimedia product and.
Controls to Reduce Threats to Validity
Correlational and Differential Research
Chapter 11: Quasi- Experimental Designs This multimedia product and its contents are protected under copyright law. The following are prohibited by law:
Copyright © Allyn & Bacon (2007) Research is a Process of Inquiry Graziano and Raulin Research Methods: Chapter 2 This multimedia product and its contents.
Copyright © Allyn & Bacon (2010) Research is a Process of Inquiry Graziano and Raulin Research Methods: Chapter 2 This multimedia product and its contents.
Copyright © Allyn & Bacon (2007) Using a Random Number Table Graziano and Raulin Research Methods: Appendix D This multimedia product and its contents.
Experimental Research Designs
Copyright © 2011 by Pearson Education, Inc. All rights reserved Statistics for the Behavioral and Social Sciences: A Brief Course Fifth Edition Arthur.
Copyright © Allyn & Bacon (2010) Single-Variable, Independent-Groups Designs Graziano and Raulin Research Methods: Chapter 10 This multimedia product and.
Copyright © Allyn & Bacon (2007) Final Preparations Before Data Collection Graziano and Raulin Research Methods: Chapter 14 This multimedia product and.
Copyright © Allyn & Bacon 2008 This multimedia product and its contents are protected under copyright law. The following are prohibited by law: any public.
Sociology: A Down-to-Earth Approach 7/e
Copyright © Allyn & Bacon (2010) The Starting Point: Asking Questions Graziano and Raulin Research Methods: Chapter 3 This multimedia product and its contents.
Copyright © Allyn & Bacon (2007) Controls to Reduce Threats to Validity Graziano and Raulin Research Methods: Chapter 9 This multimedia product and its.
Hypotheses & Research Design
Social Psychology: Sociological Perspectives
Copyright © Allyn & Bacon (2010) Selecting Statistical Procedures Graziano and Raulin Research Methods: Appendix D This multimedia product and its contents.
Validity Lecture Overview Overview of the concept Different types of validity Threats to validity and strategies for handling them Examples of validity.
Copyright © Allyn & Bacon (2007) The Starting Point: Asking Questions Graziano and Raulin Research Methods: Chapter 3 This multimedia product and its contents.
Copyright © Allyn & Bacon (2007) Field Research: Naturalistic and Case-Study Research Graziano and Raulin Research Methods: Chapter 6 This multimedia product.
Chapter 8 Experimental Research
Chapter 1 This multimedia product and its contents are protected under copyright law. The following are prohibited by law: any public performance or display,
Copyright © Allyn & Bacon (2010) Meta-Analysis Graziano and Raulin Research Methods: Appendix H This multimedia product and its contents are protected.
Introduction to Clinical Psychology Science, Practice and Ethics Chapter 5 General Issues in Psychological Assessment This multimedia product and its contents.
Copyright © Allyn & Bacon 2008 Locating and Reviewing Related Literature Chapter 3 This multimedia product and its contents are protected under copyright.
Chapter 3 Copyright © Allyn & Bacon 2008 Locating and Reviewing Related Literature This multimedia product and its contents are protected under copyright.
Copyright © 2010 Allyn & Bacon This multimedia product and its contents are protected under copyright law. The following are prohibited by law: any public.
The Research Enterprise in Psychology
Creswell, Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 4e © 2012, 2008, 2005, 2002 Pearson Education,
Copyright © Allyn & Bacon 2006 by Pearson Education. Reproduced by permission of the publisher. Further reproduction is prohibited without written permission.
Copyright © 2007 Allyn & Bacon Mayer’s Personality: A Systems Approach Part 2: Parts of PersonalityChapter 5: Interior Selves… Interior Selves; Interior.
1 Experimental Research Cause + Effect Manipulation Control.
Sociology Now 1 st Edition (Brief) Kimmel/Aronson *This multimedia product and its contents are protected under copyright law. The following are prohibited.
Getting Started Copyright © 2007 Allyn & Bacon Mayer’s Personality: A Systems Approach PART 1: EXPLORING PERSONALITYCHAPTER 2: RESEARCH IN PERSONALITY.
Copyright © Allyn & Bacon (2010) Research Checklist Graziano and Raulin Research Methods: Appendix E This multimedia product and its contents are protected.
Copyright © Allyn & Bacon 2007 Chapter 2 Research Methods This multimedia product and its contents are protected under copyright law. The following are.
Experimental Research
Copyright © Allyn & Bacon 2000 Research and Evaluation Chapter 7 Public Relations: A Values-Driven Process This multimedia product and its contents are.
Copyright © Allyn & Bacon 2008 Intelligent Consumer Chapter 14 This multimedia product and its contents are protected under copyright law. The following.
Chapter 6 Research Validity. Research Validity: Truthfulness of inferences made from a research study.
Chapter 8 – Lecture 6. Hypothesis Question Initial Idea (0ften Vague) Initial ObservationsSearch Existing Lit. Statement of the problem Operational definition.
Chapter 10 Copyright © Allyn & Bacon 2008 This multimedia product and its contents are protected under copyright law. The following are prohibited by law:
Introduction to Validity True Experiment – searching for causality What effect does the I.V. have on the D.V. Correlation Design – searching for an association.
SOCIOLOGY A Down-to-Earth Approach 8/e SOCIOLOGY Chapter Five: How Sociologists Do Research This multimedia product and its contents are protected under.
This multimedia product and its contents are protected under copyright law. The following are prohibited by law: any public performance or display, including.
Direct Practice in Social Work, 2e Scott W. Boyle Grafton H. Hull, Jr. Jannah Hurn Mather Larry Lorenzo Smith O. William Farley University of Utah, College.
This multimedia product and its contents are protected under copyright law. The following are prohibited by law: any public performance or display including.
Copyright © Allyn & Bacon 2007 Chapter 2 Research Methods This multimedia product and its contents are protected under copyright law. The following are.
Experiments Why would a double-blind experiment be used?
Hypothesis Testing, Validity, and Threats to Validity
Controls to Reduce Threats to Validity
Single-Variable, Independent-Groups Designs
Chapter 6 Research Validity.
The Starting Point: Asking Questions
Chapter 9 Experimental Research: An Overview
Discussions and Conclusions
Hypothesis Testing, Validity &
Introduction to Experimental Design
Chapter 6 Research Validity.
Correlated-Groups and Single-Subject Designs
Understanding Statistical Inferences
Misc Internal Validity Scenarios External Validity Construct Validity
Presentation transcript:

Copyright © Allyn & Bacon (2007) Hypothesis Testing, Validity, and Threats to Validity Graziano and Raulin Research Methods: Chapter 8 This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.

Copyright © Allyn & Bacon (2007) Testing Hypotheses Research tests specific hypotheses Research tests specific hypotheses –Generated from the initial research idea through a series of steps A research idea can generate dozens of research hypotheses depending on how A research idea can generate dozens of research hypotheses depending on how –it is translated into a statement of the problem –the variables are operationally defined

Copyright © Allyn & Bacon (2007) The Initial Idea The initial idea is the starting point The initial idea is the starting point –Often vague or general –It requires refining before research hypotheses can be generated Refinement of the initial idea is based on Refinement of the initial idea is based on (1) a search of relevant research literature (2) initial observations of the phenomenon Narrow and formalize the initial idea into a statement of the problem Narrow and formalize the initial idea into a statement of the problem

Copyright © Allyn & Bacon (2007) Statement of the Problem In the form of a question In the form of a question –Clearly indicates an expected relationship –Nature of the question dictates the required level of constraint of a study Causal questions will require experimental research Causal questions will require experimental research Questions about relationships can be answered with lower-constraint research Questions about relationships can be answered with lower-constraint research Convert into research hypotheses by operationally defining the variables Convert into research hypotheses by operationally defining the variables

Copyright © Allyn & Bacon (2007) Generating Research Hypotheses Ideas lead to Ideas lead to –observations –library research Statement of problem Statement of problem Problem statements become research hypotheses when constructs are operationalized Problem statements become research hypotheses when constructs are operationalized

Copyright © Allyn & Bacon (2007) Operational Definitions Initially covered in Chapter 3 Initially covered in Chapter 3 The procedures used to measure and/or manipulate variables The procedures used to measure and/or manipulate variables Most variables can be operationally defined in many different ways Most variables can be operationally defined in many different ways

Copyright © Allyn & Bacon (2007) Research Hypotheses State clearly the expected relationship between the variables State clearly the expected relationship between the variables The form is a declarative statement, but it is a tentative statement to be tested in research The form is a declarative statement, but it is a tentative statement to be tested in research Variables in research hypotheses are stated in operational definition terms Variables in research hypotheses are stated in operational definition terms

Copyright © Allyn & Bacon (2007) The Role of Theory Theory guides all research planning Theory guides all research planning –Often the primary source of the research hypothesis –Guides the selection of variables –Guides the operational definitions of variables Most research is based on multiple, overlapping, and interacting theories Most research is based on multiple, overlapping, and interacting theories

Copyright © Allyn & Bacon (2007) Testing Research Hypotheses Actually testing three hypotheses Actually testing three hypotheses –The null hypothesis –The confounding variable hypothesis –The causal hypothesis Accept causal hypothesis only if you Accept causal hypothesis only if you –reject null hypothesis (statistical analysis) –rule out each potential confounding variable hypothesis (based on appropriate controls)

Copyright © Allyn & Bacon (2007) Evaluating Hypotheses

Copyright © Allyn & Bacon (2007) Types of Validity Statistical Validity Statistical Validity Construct Validity Construct Validity External Validity External Validity Internal Validity Internal Validity

Copyright © Allyn & Bacon (2007) Statistical Validity Are the statistical tests accurate? Are the statistical tests accurate? Threatened by Threatened by –Unreliable measures –Violations of statistical assumptions Strengthened by Strengthened by –Using well validated measures –Having approximately equal sample sizes in each group (covered in Chapter 14)

Copyright © Allyn & Bacon (2007) Construct Validity Is our theory the best explanation for the results? Is our theory the best explanation for the results? Threatened by Threatened by –Any alternative explanation for the results Strengthened by Strengthened by –Using well-validated constructs to build the theoretical predictions for the study

Copyright © Allyn & Bacon (2007) External Validity Do the results apply to the broader population? Do the results apply to the broader population? Threatened by Threatened by –Unrepresentative samples –Generalizing beyond the limits of the sample Strengthened by Strengthened by –Gathering a representative sample (if possible) –Clearly describing sample, so that other researchers will know the limits of generalization

Copyright © Allyn & Bacon (2007) Internal Validity Is the independent variable responsible for the observed changes in the dependent variable ? Is the independent variable responsible for the observed changes in the dependent variable ? Threatened by Threatened by –Confounding variables Strengthened by Strengthened by –Adding adequate controls to reduce or eliminate confounding

Copyright © Allyn & Bacon (2007) Avoiding Confounding Confounding and internal validity Confounding and internal validity –Many sources for confounding (covered next) –With proper controls, confounding can be virtually eliminated (see Chapter 9) Confounding and construct validity Confounding and construct validity –Make sure that you have considered alternative theoretical explanations for the anticipated phenomenon

Copyright © Allyn & Bacon (2007) Confounding Variables 1 Maturation Maturation –Changes due to growth or predictable changes History History –Changes due to an event that occurs during the study Testing Testing –Changes due to the effects of previous testing

Copyright © Allyn & Bacon (2007) Confounding Variables 2 Instrumentation Instrumentation –Any change in the calibration of the measuring instrument over the course of the study Regression to the Mean Regression to the Mean –Tendency for participants selected because of extreme scores to be less extreme on a retest Selection Selection –Any factor that creates groups that are not equal at the start of the study

Copyright © Allyn & Bacon (2007) Confounding Variables 3 Attrition Attrition –Loss of participants during a study; are the participants who drop out different from those who continue? Diffusion of treatment Diffusion of treatment –Changes in participants’ behavior due to information they obtained about other conditions Sequence Effects Sequence Effects –Effects on performance in one condition due to experience with previous conditions

Copyright © Allyn & Bacon (2007) Subject Effects Participants are not passive Participants are not passive –They try to understand the study to help them to know what they “should do” (termed subject effects) –Respond to subtle cues about what is expected (termed demand characteristics) Placebo effect: treatment effect due to expectations that the treatment will work Placebo effect: treatment effect due to expectations that the treatment will work

Copyright © Allyn & Bacon (2007) Experimenter Effect Based on the expectations of the researcher Based on the expectations of the researcher Affects the outcome of studies if not controlled Affects the outcome of studies if not controlled May be due to the experimenter providing demand characteristics to the participant May be due to the experimenter providing demand characteristics to the participant Not the same as scientific fraud (which is deliberate) Not the same as scientific fraud (which is deliberate)

Copyright © Allyn & Bacon (2007) Validity, Control, and Constraint Three closely-tied concepts Three closely-tied concepts Validity Validity –The accuracy of the study or procedure –Increased by using appropriate control procedures The more controls we employ, the higher the level of constraint The more controls we employ, the higher the level of constraint

Copyright © Allyn & Bacon (2007) Summary Start by building a research hypothesis Start by building a research hypothesis Testing the research hypothesis is actually testing three hypotheses Testing the research hypothesis is actually testing three hypotheses –(1) null; (2) confounding-variable; (3) causal Several types of validity Several types of validity Many potential confounding variables Many potential confounding variables Subject and experimenter effects can also affect the outcome of the study Subject and experimenter effects can also affect the outcome of the study