(2) Locating Studies

Overview General Information to keep in mind:  A meta-analysis is only informative if it adequately summarizes the existing literature  HOWEVER, it does not have to be comprehensive (fail-safe n) but needs to be close to comprehensive  Techniques - database searches, ancestry approach, descendancy approach, hand searching, invisible college

Available approaches (from Johnson & Eagly, 2000) (1) Database searches  Which databases? Depends on topic Obviously PsycINFO Ask Librarian if other databases are relevant  Which search terms? For studies in hand, see “Descriptors” For studies in hand, see “Identifiers” Use wildcards: juror*

Available approaches (from Johnson & Eagly, 2000) (2) Ancestry approach  Search reference list of articles in hand  What to do about referenced unpublished?  What to do about referenced poster/talk?  What to do about foreign language?

Available approaches (from Johnson & Eagly, 2000) (3) Descendancy approach  Use “cited by” feature in PsycINFO?  What about SSCI?  What if there are differences between the two?

Available approaches (from Johnson & Eagly, 2000) (4) Hand searching  Scan individual journals  Provides a good cross-check  May find “hidden” articles  May find new Descriptors / Identifiers  Don’t spend too much time on this

Available approaches (from Johnson & Eagly, 2000) (5) Invisible college  Using the network of researchers  listservs  When in the process should you send the ? The best time to send the is: (1) after you have finished your first pass through finding ES for each study because then you will have a clear idea of what you need and how to craft the letter appropriately, (2) but before you have your coders start coding, otherwise you will have to re-do the coding for the new studies.

Overarching Principles (1) Inclusion/Exclusion criteria  Locating Studies (Step 2) goes hand-in-hand with Identifying Hypothesis (Step 1)  This is partly an a-priori determination of what to investigate  This is partly an on-going dynamic process where you review the literature and decide what to investigate  This is partly an ad-hoc statement or summary of what you investigated

Overarching Principles (2) Explicit and open to scrutiny  Must detail in “Method” how you found sources so must cover all your bases, otherwise reviewers may argue about your methods  So read “Method” sections of other meta-analyses for information and copy best ones  Keep a record of what search terms you used, what databases you used, etc. See page of my Quals for example

Overarching Principles (3) Comprehensive?  “Garbage in – Garbage out”  If cast too wide a net and need to trim down, then try setting conceptual boundaries such type of IV, type of DV, domain, paradigm, etc.  Ideally comprehensive but doesn’t have to be comprehensive (fail-safe n) but needs to be close to comprehensive

Overarching Principles (4) Inaccessible?  For articles listed as “unpublished” or “in press”, contact the authors  For articles in a foreign language, if there is no English translation, then report in Method section which articles were inaccessible due to language issues

Concrete Steps: Create an excel file (See website “Example-DataSet1”) Why this helps:  Have listing of each article you have found  Sort them into those with relevant data and those without (see bottom of excel file)  For those with relevant data, can start inserting effect sizes (Step 3) and moderators (Step 5)  For those without relevant data, can type in “notes” about why it is not relevant (you will need this information later to report in “Method” section your exclusion/inclusion criteria)  Since listing of each article you found, when come across a new article, can see if you already found it  Within excel file type in article reference (APA format) so can copy/paste whole thing into your papers “Reference” section

Concrete steps (cont.) Things to keep in mind  In example excel file, Step 2 is only columns A, B, C (other columns are Steps 3, 4, 5)  In your excel file, you may create as many columns as you need the example excel file is a cleaned-up version of one I used but the actual one has a lot of information unique to the particular meta. See next slide for my actual excel file by double-clicking