Data Collection

In the Beginning Define Constructs (what is the domain?) Therapy effectiveness Integrity tests Research Question What’s the average? Zero? Impact of management (e.g., Brown*) May or may not be summary of a literature Theoretical Justification of Moderators Pick ONE study type (e.g., experiment, correlational study) or pick all and analyze separately. *Brown, S. (1981). Validity generalization and situational moderation in the life insurance industry. Journal of Applied Psychology, 66,

Sources of Data (ES) 1 Electronic Databases Psych info, APA online SSCI, Web of Science, PubMed Google scholar*; What others? May hire a reference librarian Conference Proceedings & Abstracts Dissertations Local Distant Government Docs & Agencies *So far as I know, you cannot use Boolean AND or NOT in Google Scholar, so you get too many returns.

Sources of Data (2) Hand searches of the literature Refs in books Refs in articles Flip thru journals People in the area (write, , drink) Use your contacts (faculty, friends, etc.) Butter up Review Articles (used to be more) Unpublished manuscripts (prepubs, too)

Eligibility (1) Country Language Measure (specific) NEO vs Big Five Beck vs CESD Measure (general) Reliability N of items Participants (e.g., female cancer survivors; bankers; no students; employed 20 + hr/ week, etc.) Publications (what type) Published vs. not The criteria are your choice. But you may have to defend that choice. Do you have good reasons?

Eligibility (2) Time frame Gains over time Time after treatment (e.g., smoking) Required information Effect Size Moderator Other You can contact authors for missing info Study Design Control group Repeated Measures Random Assignment Specific controls (e.g., dismantling) Methodological Quality? Code for features, not overall quality

Database 1 Keep records of the search process and the elimination process to create a flow chart (replication). Example of flowchart soon. Create a database to hold the effect sizes, data needed to compute them, and potential moderators Excel is good for a database, but you can use whatever you like

Database 2 Prepare for reliability of coding – copy code book and database At least 2 coders who will code ALL the variables independently for at least a sample of studies Create system to track dependencies (effect sizes from the same study, etc.) Code Author and Date of Publication separately for cumulative analysis by date

Example Flow Chart

Example Search Setup Note. We sort the documents into bins. Also see the files ‘SearchAurah.xlsx’ and ‘SearchProcessmb.docx’ for examples. The Excel file is for the flow chart. It has no space for coding ES.