Does Media Concentration Lead to Biased Coverage? Evidence from Movie Reviews Stefano DellaVigna, UC Berkeley and NBER Alec Kennedy, San Francisco Fed Moscow Media Conference October 29, 2011 Preliminary, comments most welcome

Introduction Dec. 13, 2007: News Corp. acquires from Bancroft family Dow Jones & Company  Wall Street Journal Unlike Bancroft family, Murdoch's holdings include: Cable channels (i.e., Fox Sports and Fox News) Satellite television (Sky group) Movie distributor (20th Century Fox) … Media conglomerate implies conflict of interest  Coverage of such businesses (or their competitors) in the Wall Street Journal may be biased.

Introduction Wall Street Journal situation hardly unique Media conglomerates: Comcast owns NBC Hearst Corporation owns ESPN and print outlets Time Warner owns AOL and Time magazine  Pervasive conflict of interest But should media consolidation lead to distortion? Cost: Loss of reputation if bias is revealed Benefit: Can persuade audience (if audience naïve about bias) Generally, persuasion rates of 5-10% from the media (DellaVigna-Gentzkow, ARE 2010)

Introduction This paper: Focus on two conglomerates News Corp. Time-Warner Measure how media outlets in these groups review movies distributed by an affiliate 20th Century Fox Warner Bros. Identification of bias transparent (diff-in-diff): Compare review of 20th Century Fox movie by WSJ and NYT Further control group: review of Paramount movie by WSJ and NYT

Introduction Why movie reviews? 1. Frequent (500,000+ reviews in data set) 2. Easily quantifiable (coded on scale by metacritic.com) 3. Large industry ($60bn annual revenue) 4. Some evidence that movie reviews influence movie attendance (Reinstein and Snyder (2005)  benefits to the distributor from increased ticket sales

Data First data set of reviews from metacritic.com, scored 0-100

Data Second data set of reviews from rottentomatoes.com, scored on a 0-1 “freshness” scale

Data Merged data set contains 548,764 reviews from 336 media from 1985 to July 2011

Data Studios which distribute movies in the sample: Major, Independent, and Other

Average Bias: News Corp.

Average Bias: Time Warner

Average Bias: Statistical Test Is the bias for News Corp. statistically significant? Is it robust to introducing controls? OLS regression captures the effect of conflict of interest in Fox captures the effect of conflict of interest in Time Warner Sample of 473,727 reviews (qualitative reviews in RT have no score) Standard errors clustered at the movie level

Average Bias: Statistical Test

In favorite specification, conflict of interest increases rating for News Corp. by 2.6 points out of 100 Estimate of bias increases with extra controls  Unobservables unlikely to bias coefficient upward (Altonji, Elder, and Taber, 2005) Magnitude of bias: Equivalent to 1 extra star every ten reviews Small but still economically significant impact Reinstein and Snyder (2005) estimate 25% higher revenue for two thumbs up by Roger Ebert Are all media conglomerates the same? No evidence of bias for Time Warner, can reject bias of 0.9 points

Average Bias: Robustness Evidence of bias using RT 0-1 freshness indicator

Calculation of benefits Back-of-the-envelope calculation of estimated benefits from distortion of movie review Case of New York Post (NYP) Suppose that NYP gives one extra star per Fox movie 500,000 average readers Persuades an extra 1% of readers to watch movie Ticket sales increase by 5,000, or $8*5,000=$40,000 Studio receives about half of increased sales, plus another half from higher rights from DVD licensing  $40,000 About th Century Fox movies since 1985  Potential benefits to NewsCorp. from one extra star per Fox review by NYP over 25 years : $16,ooo,ooo

Average Bias: Explanations Three main explanations for the results: (E) Explicit editorial policy conveyed to journalists (J) Bias by a journalist ultimately due to the conflict of interest, but lacking editorial pressure (T) Correlation in taste between the media reviewer (or the media audience) and the affiliated studio To separate explanations, we present evidence on: 1. Clustering of bias within a conglomerate (E and T, maybe J) 2. Editorial policies (E only) 3. Selective bias by type of movie (E or J, not T) 4. Omission of reviews (E or J)

Clustering of Bias: By Media News Corp. media: Statistical evidence for NYP, 2-3 point bias for all 6 media No evidence of bias for any of the Time Warner media

Clustering of Bias: By Journalist Most media have a small number of journalists reviewing movies Chicago Sun- Times, News of the World, Wall Street Journal and CNN.com have essentially only one

Clustering of Bias: By Journalist News Corp: Statistical evidence of bias for 3 out of 4 NYP journalists, and one of TV Guide journalists Time Warner: No evidence of positive bias Significant Clustering: Pattern suggestive of editorial bias, but could also be correlated tastes, or similar journalists

Editorial Policies Two tests of editorial policies 1. Personnel Policy: Change of reviewers at change of ownership – No evidence 2. Bias in Assignment: Assign affiliated movies to more generous reviewers Estimate average reviewer generosity in score: Reviewers differ significantly in their generosity Are movies by affiliate studios more likely to be assigned to more generous reviewer? No evidence– assignment quasi-random Bias at Newscorp. is unlikely of editorial origin Could be correlation in journalist bias or in tastes

Editorial Policies

Selective Bias If bias is due to conflict of interest, should be optimal: Bias when marginal return (persuasion) is highest Maximize impact on revenue Proposition 2. Small or no bias for very low quality movies If bias is due to correlated tastes, no such prediction Assumption for this test: Movies with negative reviews by others are unlikely to benefit from a lone positive review Movies with other positive reviews more likely to benefit from more positive review

Selective Bias: News Corp.

Selective Bias: New York Post

Selective Bias: Time Warner

Selective Bias: Statistical Evidence Evidence of selective bias for NYPost and qualitatively for WSJ – Bias due to conflict of interest No evidence for other media

Bias by Omission If bias is due to conflict of interest, and audience not fully rational, bias by omission: Review high-quality affiliated movies Omit review of low-quality affiliated movies Do not expect this pattern if correlated tastes Rare setting to separate bias by omission and by commission Outlets differ substantially in probability of review  Use matching procedure For each media, find ten matching media in terms of average probability of review of a movie Plot local polynomial regression of dummy for review on average review score

Bias by Omission: New York Post

Bias by Omission: Time

Bias by Omission: Entertainment Weekly

Bias by Omission: Statistical Test Newscorp.: No systematic evidence of omission bias Time Warner: Evidence for 2 outlets Bias by omission and by commission substitutes, not complement

Bias in Movie Aggregator So far focus on most obvious conflict, for reviewers Conflict of interest hardly stops there: Rottentomatoes.com, also independent when launched in 1998, was acquired by IGN Entertainment in June 2004, and IGN was purchased by News Corp. in September IGN, and hence RottenTomatoes, was then sold in January of 2010 by Newscorp. Conflict of interest for RT: more positive reviews of the 20th Century Fox movies in Control for rating of same review in MC

Bias in Movie Aggregator Remarkably, no evidence of bias, even for qualitative reviews, where bias is easier to hide Can reject small bias

Bias in Movie Aggregator Local polynomial regression of ‘fresh’ indicator on average movie score – no evidence of bias

Bias in Movie Aggregator Event study comparing residual freshness (after controlling for score) for FOX and non-FOX movies

We documented the extent of bias due to conflict of interest for two media conglomerates Average bias: 2.6 points bias out of 100 for Newscorp. outlets No bias for Time Warner outlets, can reject even small bias Bias is clustered within a media conglomerate No evidence of editorial policy to assign movies Selective Bias: Evidence for New York Post, not for other media Omission Bias: Evidence for two Time Warner outlets Interpretation: Best fits with bias due to conflict of interest for journalists, with clustering of such bias Summary of Results

Overall, remarkably little evidence of distortion from conflict of interest: No distortion in review for Time Warner No distortion for Rottentomatoes No distortion in editorial assignment However, bias still does occur: Small, but significant, bias for Newscorp. Outlets Some omission bias for Time Warner outlets Suggests that transparency and emphasis on reputation (for example because of competition) critical to keep media honest Paper allowed us to decompose potential media bias in novel ways Relates to Conflict of interest in other settings: Significantly less distortion than for analysts Less distortion than for advertising Conclusion

Average Bias

Selective Bias II Different types of movies can have different returns to positive reviews Snyder and Reinstein: Larger effect of movie reviews for independent movies However: Independent movies also have smaller revenue, so bias may be less worthwhile Examine the effect of conflict of interest separately

Selective Bias: Indy movies Significant bias for News Corp. only for mainstream movies