Texas A&M University EPSY 642 FALL 2009 Victor L. Willson, Instructor.

Slides:



Advertisements
Similar presentations
Jane Long, MA, MLIS Reference Services Librarian Al Harris Library.
Advertisements

Introduction to Meta-Analysis Joseph Stevens, Ph.D., University of Oregon (541) , © Stevens 2006.
Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 6 Finding the Evidence: Informational Sources, Search Strategies, and Critical.
Research Synthesis (Meta-Analysis) Research Synthesis (Meta-Analysis) CHAPTER 1 CHAPTER 10.
Computer Science Department Middle States Assessment Computer Science has 4 programs (minor, bachelor’s, master’s and doctorate) and therefore 4 different.
Topics - Reading a Research Article Brief Overview: Purpose and Process of Empirical Research Standard Format of Research Articles Evaluating/Critiquing.
Beginning the Adventure of Your Thesis/Project (SWRK500) or Advanced Research (SWRK501) & Capstone Course (SWRK502) ~ Culminating Experience ~ Prepared.
Practical Meta-Analysis -- D. B. Wilson 1 Practical Meta-Analysis David B. Wilson.
Writing the Literature Review & In-Text Citations Science 2 Fall 14
©2007 Prentice Hall Organizational Behavior: An Introduction to Your Life in Organizations Chapter 19 OB is for Life.
Chapter One of Your Thesis
APA Types of Articles Dr. Gustafson. Multiple Styles of Journal Articles Empirical Studies Literature Reviews Theoretical Articles Methodological Articles.
Fig Theory construction. A good theory will generate a host of testable hypotheses. In a typical study, only one or a few of these hypotheses can.
Writing a Research Proposal
1 The Literature Review March 2007 (3). 2 The Literature Review The review of the literature is defined as a broad, comprehensive, in- depth, systematic,
Dissertation: Introduction & Literature Review Useful resources: Lunenburg, F. C., & Irby, B. J. (2008). Writing a Successful Thesis or Dissertation: Tips.
Martin F. Lueken Anna M. Jacob Jennifer Ash Prepared for the Campbell Collaboration Colloquium Copenhagen 2012 Thursday, May The Effects of Charter.
Funded through the ESRC’s Researcher Development Initiative
Chapter 3 An Overview of Quantitative Research
Advanced Statistics for Researchers Meta-analysis and Systematic Review Avoiding bias in literature review and calculating effect sizes Dr. Chris Rakes.
Northcentral University The Graduate School February 2014
Program Evaluation. Program evaluation Methodological techniques of the social sciences social policy public welfare administration.
The Effect of Computers on Student Writing: A Meta-Analysis of Studies from 1992 to 2002 Amie Goldberg, Michael Russell, & Abigail Cook Technology and.
How to Write a Critical Review of Research Articles
Copyright © Allyn & Bacon 2008 Locating and Reviewing Related Literature Chapter 3 This multimedia product and its contents are protected under copyright.
Chapter 3 Copyright © Allyn & Bacon 2008 Locating and Reviewing Related Literature This multimedia product and its contents are protected under copyright.
Evaluating a Research Report
EPSY 640 INTRODUCTION TEXAS A&M UNIVERSITY. SYLLABUS Available at Victor L. Willson,
Experimental Research Methods in Language Learning Chapter 16 Experimental Research Proposals.
The Literature Search and Background of the Problem.
LECTURE 2 EPSY 642 META ANALYSIS FALL CONCEPTS AND OPERATIONS CONCEPTUAL DEFINITIONS: HOW ARE VARIABLES DEFINED? Variables are operationally defined.
ScWk 242 Course Overview and Review of ScWk 240 Concepts ScWk 242 Session 1 Slides.
Evaluating Research Articles Approach With Skepticism Rebecca L. Fiedler January 16, 2002.
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
PHY 1405 Conceptual Physics (CP 1) Spring 2010 Cypress Campus.
I know of no more encouraging fact than the unquestionable ability of man to elevate his life by conscious endeavor. Henry David Thoreau.
Information commitments, evaluative standards and information searching strategies in web-based learning evnironments Ying-Tien Wu & Chin-Chung Tsai Institute.
Planning an Applied Research Project Chapter 3 – Conducting a Literature Review © 2014 by John Wiley & Sons, Inc. All rights reserved.
LITERATURE REVIEW  A GENERAL GUIDE  MAIN SOURCE  HART, C. (1998), DOING A LITERATURE REVIEW: RELEASING THE SOCIAL SCIENCE RESEARCH IMAGINATION.
LECTURE 5 HYPOTHESIS TESTING EPSY 640 Texas A&M University.
Developing a Review Protocol. 1. Title Registration 2. Protocol 3. Complete Review Components of the C2 Review Process.
META-ANALYSIS, RESEARCH SYNTHESES AND SYSTEMATIC REVIEWS © LOUIS COHEN, LAWRENCE MANION & KEITH MORRISON.
Copyright © Allyn & Bacon 2008 Intelligent Consumer Chapter 14 This multimedia product and its contents are protected under copyright law. The following.
Anatomy of a Research Article Five (or six) major sections Abstract Introduction (without a heading!) Method (and procedures) Results Discussion and conclusions.
DOING LITERATURE REVIEW DR. FARIZA KHALID. WHAT IS JOURNAL ARTICLE? "Journal articles are usually reports of empirical studies, literature reviews, theoretical.
Beginning the Adventure of Your Thesis/Project (SWRK500) or Advanced Research (SWRK501) & Capstone Course (SWRK502) ~ Culminating Experience ~ Prepared.
In Search of a Valid Research Topic and its Development.
LECTURE 3 EPSY 642 FALL CODING STUDIES Dependent variable(s) Construct(s) represented Measure name and related characteristics Effect size and associated.
Chapter 14 Research Synthesis (Meta-Analysis). Chapter Outline Using meta-analysis to synthesize research Tutorial example of meta-analysis.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 27 Systematic Reviews of Research Evidence: Meta-Analysis, Metasynthesis,
Trouble? Can’t type: F11 Can’t hear & speakers okay or can’t see slide? Cntrl R or Go out & come back in 1 Sridhar Rajappan.
CHAPTER 2 LITERATION REVIEW 1-1. LEARNING OUTCOMES 1.The reasons for a literature review being an essential part of every project. 2.The purpose of a.
Cedric D. Murry APT Instructor of Applied Technology in research and development.
Academic Dishonesty One way to help understand just what constitutes academic dishonesty is to look at what another institution has formalized. The material.
HCS 465 OUTLET Experience Tradition /hcs465outlet.com FOR MORE CLASSES VISIT
Understanding and Critically Appraising the Literature Review
Scholastic Dishonesty
Writing a Research Report (Adapted from “Engineering Your Report: From Start to Finish” by Krishnan, L.A. et. al., 2003) Writing a Research Write the introduction.
EDUC 6540: Data Based Decision-Making for School Leaders
The Literature Search and Background of the Problem
Critically Reviewing the Literature
FINDINGS CUMULATION AND REPORT WRITING
Rehabilitation Research July 27, 2005
Research proposal MGT-602.
The Starting Point: Asking Questions
What is Research? A research study is a study conducted to collect and analyse information in order to increase our understanding of a topic or an issue.
Meta-analysis April 11, 2006.
Problems, Purpose and Questions
Scholastic Dishonesty
META-ANALYSIS PROCEDURES
Presentation transcript:

Texas A&M University EPSY 642 FALL 2009 Victor L. Willson, Instructor

EPSY 642 Meta Analysis of Social Science Research Syllabus FALL 2009 TUESDAYS 4:45 – 7:35 Victor L. Willson, Professor:Office: by appt 704 Harrington /fax: Course description: Theory and methodology of current state of the topic of meta-analysis of research documents. Course objectives: Students will learn meta-analysis methods by conducting a meta-analysis on a circumscribed topic of their choice. Students will be required to present their findings at the end of the course and produce a research paper on their topic. Students will be encouraged to develop a potentially publishable research article from their work. Components for the project will be reported by students during the semester. Texts: Lipsey, M. W., & Wilson, D. B. (2001). Practical Meta-Analysis. Sage. Cooper, H. (2010). Research Synthesis and Meta-Analysis. 4 th Ed. Sage

Date Topic Reading Assignment due SEP 1Introduction: Purposes, Approaches HC 1 Activity: Online Google Scholar search 5 articles: samples description, variable list 8NO CLASS 15Problem finding, Study SelectionLW2, HC Study coding, menu elementsLW4, 5, HC 4, handout 29Outcome variable effect computation LW3, HC 6, handouts OCT 6 Study Design variablesHS 2, HC 5 13Correlational study analysis LW 6-7 Experimental study analysisLW 7 #1 27Findings cumulation and reportingHC 6-9, LW 8 #2 handouts & examples NOV 3Advanced Modelinghandouts# Individual meetings #4 Nov 10, #5 Nov 17 24Presentation of student papers using powerpoint DEC 1continuation of presentations as needed

Grades: Based on completion of intermediate assignments (20%) and quality of paper (80%) U= < 60% S = 61% - 100% Intermediate assignments: 1 Proposed meta-analysis topic with preliminary list of at least 20 articles 2Preliminary list of proposed variables to be extracted from articles 3Proposed Coding menu for variables 4Preliminary analysis of outcomes using Q method, funnel plot of one outcome variable 5Preliminary covariate analysis using QB and QW method (at least 1 covariate) Project and paper: conduct a meta-analysis using the procedures of this course on a topic of your choice; limit the number of studies to be finally included to no more than about 20 but at least 8 as appropriate to the topic. Produce a finished paper in APA format summarizing the procedures and results; produce a powerpoint of your paper for presentation. In order to receive a grade a presentation must be made on the dates listed; otherwise an incomplete will be given and a presentation to me and a group of students must be made before a final grade can be submitted. Final papers may be turned in after the last day to turn in grades for the semester with a grade of I given and no penalty, but after a presentation has been made and all intermediate assignments are completed. After one semester a grade of F will be given if these conditions are not met.

Note: The handouts and web-based files used in this course are copyrighted. By “handouts” I mean all materials generated for this class, which includes but is not limited to syllabi, quizzes, exams, lab problems, in-class materials, review sheets, and additional problem sets, in paper or electronic form. Because these materials are copyrighted, you do not have the right to copy the handouts unless I expressly grant permission. As commonly defined, plagiarism consists of passing off as one’s own ideas, words, writings, etc. which belong to another. In accordance with this definition, you are committing plagiarism if you copy the work of another person and turn it in as your own, even if you should have the permission of that person. Plagiarism is one of the worst academic sins, for the plagiarist destroys the trust among colleagues, without which research cannot be safely communicated. If you have any questions regarding plagiarism, please consult the latest issue of the Texas A&M University Student Rules, under the section “Scholastic Dishonesty”. Honor Code and refers the student to the Honor Council Rules and Procedures on the web: “An Aggie does not lie, cheat or steal, or tolerate those who do.”

Purposes for Meta-Analysis Cumulate findings of studies on a particular topic Examine homogeneity of outcomes Estimate effects of independent variables on outcomes in a standardized format – Evaluate moderator and mediator effects on outcomes – Differentiate different types or classes of outcome effects

Historical background Criticism of traditional narrative reviews of research Exasperation in social sciences with constructs measured different ways in terms of determining consistencies Need to formulate theoretical relationships based on many studies

History part 2 Early 1970s efforts focused on significance testing and “vote counts” of significance Glass (1976) presented a method he called “meta- anlaysis” in Am. Ed. Research Assn. presidential address Others proposed related methods, but Glass and colleagues developed the most widely used approach (Glass, McGaw, & Smith, 1981)

Meta-Analysis as Survey Research Research articles as unit of focus Population defined – Conditions for inclusion of articles Data requirements needed for inclusion Completeness of data available in article or estimable Publication sources available, selected Sample vs. population acquisition – Availability of publications and cost – Time to acquisition (how long to wait for retrieval)

Strengths of Meta-Analysis Definition of effect and effect size beyond “significant or not” Focus on selection threats in traditional reviews (bias in selection of articles for review) Systematic consideration of potential mediators and moderators of effects Data organization of articles for public review

Weaknesses of Meta-Analysis Methodologically sophisticated and expensive Potential ignoring of contextual effects not easily quantified; eg. historical/environmental placement of research Potential improper mixing of studies Averages hiding important subgroupings Improperly weighting studies with different methodological strength/rigor

Research focus for meta-analysis Defining and delineating the construct Determining a research outlet Meta-Analysis as an interactive, developing process

Recent Criticism Suri & Clarke (2009) : Advancements in Research Synthesis Methods: From a Methodologically Inclusive Perspective (Review of Educational Research, pp ) They propose 6 overlapping approaches: 1. Statistical research syntheses (eg. meta-analysis) 2. Systematic reviews 3. Qualitative research syntheses 4. Qualitative syntheses of qualitative and quantitative research 5. Critical impetus in reviewing research 6. Exemplary syntheses

Some critical comments on Suri & Clarke (2009) Systematic reviews- original Glass criticisms hold: what is the basis for inclusion and exclusion; why are certain articles privileged? Qualitative research syntheses- how can these be done with situated contexts, small samples, environmentally- developed variables, sources, etc.? Will there be a review for every reader, or for every researcher? Same limitation as all qual research Qual syntheses of quant and qual research- potentially doable, with an alternating order: qual first to focus emphases in the quant analysis, or quant first to be validated with the qual studies of particular environments and populations- do they fit/match in reasonable ways? Critical impetus- code words for critical theory/Marxist etc. Answer is already known, why do the research? Exemplary syntheses- what is the purpose?

Defining and Delineating the Research Topic Outcome construct definition – Importance to the field to know what has been learned – How big is it? How many potential studies? – Conduct preliminary searches using various databases Refining the construct – How much resource is available? Eg studies = 2-3 years work – Are there specific sub-constructs more important than others? Select them or one of them – Are there time-limitations (no studies before 19xx) – Are there too few studies for the given construct, should it be broadened? Too few-> less than 10?

Defining and Delineating the Research Topic What is the typical research approach for the topic area? All quantitative All qualitative Mixed quantitative and qualitative Are there sufficient quantitative studies to provide evidence for findings? Can qualitative studies be included as a separate part of the study? How?

Determining Research Outlet Does the proposed journal – publish research on the construct? – Publish reviews or meta-analyses? Is there a journal devoted to reviews that your project would fit with? Has a recent similar meta-analysis been published? If so, will yours add anything new? – Ex. Allen, et al (under review) evaluated articles on first grade retention after 1990 focusing on the quality of the research design in each study to determine if the effects were different from a fairly recent meta-analysis by Jimerson (2001)

Meta-Analysis as an interactive, developing process View meta-analysis as evolutionary – As studies are reviewed and included, purpose and scope may change Assume initial conceptualizations about both outcomes and potential predictors may change over time – Definitions, instruments, coding may all change as studies are found and included Plan for revisions to all aspects of the meta-analysis

ACTIVITY Get into groups of 3 or 4 Use GOOGLE SCHOLAR to find research articles from the following: Group 1:elementary school grade retention Group 2: teenage depression treatments Group 3: IQ and socioeconomic status Groups 4+: one of above Select 5 articles that have empirical data For each article define the population being samples List with specific names the variables used in the study Discuss within your group the similarities and differences of the populations and variables; consider difficulties you would have in summarizing research for these Plan to report to the class what you found