1. Archival Research, Content Analysis, & Meta-Analysis
Research Methods

2. Archival Research Studying existing records
E.g., US Census Bureau, police reports
Developing clear questions or hypotheses
Issues
Gaining access
Completeness of records
Selection of records
Archival research – involves studying existing records (e.g., police records, census data, medical records, transcripts from congressional hearings). These records serve as your data.
Because a vast amount of information may be gathered in archival research, it is important that researchers develop clear, focused questions or hypotheses at the beginning of the process.
Bordens and Abbott’s (2008) textbook) notes some issues that need to be addressed when conducting archival research.
First, researchers must gain access to the records. In some cases this is relatively easy, as some databases are freely available on the internet. However, in other situations it may be very difficult to obtain the records, which may require consent and permission from various parties.
Once, researchers have gained access they’ll have to examine the completeness of the records. This may mean determining whether the questions they are asking can be asked with the data available. It may mean making decisions about specific records that may include only partial information.
Researchers may also make decisions about which records to select and include in the study.

3. Content Analysis Content Analysis Analyze spoken or written record
Sampling
Which materials to use?
Units of analysis
Recording unit
Content unit
Clear operational definitions
Multiple coders (to assess reliability)
Reduce bias issues
A content analysis is a procedure used to analyze information (typically, spoken or written records). This information may be archival or used in conjunction with observational methods. In fact, there are similarities between the methods.
Similar to archival and observational research methods, with content analyses it is important to decide which materials you plan to use. For example, you could examine advertisements in magazines to examine gender differences sexual objectification (i.e., treating people as instruments of pleasure). However, which magazines should the researcher choose to examine? From what time period? How many issues? Which advertisements should be the focus? As you can see, there are a lot of questions to consider. The answers to these questions may influence our judgements about the external validity of the study.
Researchers will also have to make decisions about the units of analysis
Recording unit – element of the material that you are going to record (word, sentences, phrases, themes)
Context unit – context in which recording unit was used (context can give meaning and help with interpretation later)
Researchers must develop clear operational definitions. Reminder, that operational definitions are the procedures used to asses specific variables. In this case, how can we tell if sexual objectification is occurring. In order to do this, researchers could look for instances in which men and women are reduced to body parts rather than presented as whole persons – but we’d want this to be even more specific. We have to be able categorize the records. These categorizes should be exhaustive, mutually exclusive, independent, derived from one classification system.
We may also have more than one researcher coding the records. In this case, we could assess reliability similar to what is done in observational research by looking at interjudge or interrater reliability. Doing so, may reduce potential bias issues and make the process more systematic and objective.

4. Example
Adkins, Covert, & Wasburn (2007) examined the extent of media bias in Time and Newsweek coverage of domestic social issues from through The authors coded 873 articles.
Selection of article topics?
What is liberal? What is conservative?
Inter-rater reliability?
For example, Adkins, Covert, and Wasburn (2007) examined the extend of media bias in Time and Newsweek coverage of domestic social issues from 1975 through The authors codes 873 articles for liberal and conservative bias. These authors had to make decisions about which magazines (Time and Newsweek) to examine, as well as which types of articles (domestic social issues) over a specific time frame ( ). Additionally, they had to make determinations about what articles are classified as domestic social issues. They needed to develop operational definitions to determine what liberal vs. conservative bias looked like. Additionally, they needed to establish interrater reliability.

5. Example
Kramer and Winter (2008) interested in impression management among users of StudiVZ. Researchers randomly sent 150 members an invitation to complete their study. Fifty-eight participants answered questions about their extraversion levels, self-esteem, efficacy of self-presentation. Two researchers coded participants’ user profile page (e.g., number of photos, friends, relationship status, style of text, style of profile photo, etc.).
What type of study is this?
What are the variables of interest?
How could reliability be assessed?
Do we need to be concerned about response rate?
Read though the above example and then answer the questions.
In another study published in 2008, Kramer and Winter examined impression management among users of StudiVZ (Germany social networking site – like facebook). Not only did they perform a content analysis by examining users profile pages, but they combined this method with a survey method by sending some users a survey to assess constructs like personality. Once researchers have coded the data then they would look for patterns by performing correlational research. For example, is there a positive relationship between how extraverted users are and the number of friends listed on their profile page? Notice, that 58 out of the 150 responded to the survey (approximately 39%). How concerned about the response rate should we be?

6. Meta-Analysis
Statistical procedure combining data from multiple studies
Systematic, quantitatively combines data in order to provide information about effect sizes
Steps
Identify studies
File drawer phenomenon
Determine eligibility of studies
Get the data from these studies
Statistically analyze the data
Researchers may also choose to conduct a meta-analysis. This is similar to a literature review in that several previously conducted studies are examined to determine if there are patterns or themes across many studies. However, it is different from a literature review in how those patterns/themes are identified. A literature review would be considered more qualitative, whereas a meta-analysis would be considered more quantitative. The data, the statistics, from the studies are collected and then further analyzed via statistics.
Meta-analyses are systematic and quantitative. They combine the data from many studies to provide information about the average size of some effect. That is, how larger an effect or how strong a relationship is. For example, researchers may compile 50 studies of the effect of violent video games on aggression. A meta-analysis may help to determine if there is a positive relationship between the two variables, and if there is, how strong is that relationship.
When engaging in a meta-analysis researchers would first need to identify studies that fit their specific criteria. Researchers can look through databases (like PsycINFO), consult experts in the field, post requests for studies on their topic via listervs, find abstracts of researcher presented at conferences, etc.
Researchers do have to be concerned about the file drawer phenomenon. In psychology, as is the case in some other fields, there has been a bias to publish research in which there are statistically significant findings. Those that do not find statistically significant relationships may get shoveled in the file drawer. Researchers may then need to try contacting individuals who have conducted similar research, but not published it. There are also specific journals that publish what we call ‘null findings’ (non-significant relationships).
Once we have a set of studies, then researchers must go through those studies more carefully to determine which should be included and which should be excluded. Perhaps studies are excluded because they measured one of the key variables differently (not consistently with the criteria outlined by the researchers) or because the study is lacking necessary information.
Researchers then get the data from these studies and statistically analyze the data.

7. Example
Here is an example of a meta-analysis from a study conducted by Bushman (2016) entitled “violent media and hostile appraisals: A meta-analytic review.”

9. Mini-Review
In what ways are a content analysis and observational research similar?
Compare and contrast meta-analysis to literature reviews.
Provide an example of a content analysis.