1. The History of the Cross Section of Returns
September 2017
Juhani Linnainmaa, USC and NBER
Michael R. Roberts, Wharton and NBER

2. Introduction Lots of anomalies What is mechanism behind anomalies
314 “factors” Harvey, Liu, and Zhu (2015)
What is mechanism behind anomalies
Unmodeled risk? Mispricing? Data-snooping?
Empirical strategy
Exploit comprehensive accounting data from 1926 to 2016
Pre-sample period (Jaffe et al ’89, Davis et al ‘00)
In-sample period
Post-sample period (Jagadeesh and Titman ’01, Schwert ’03, McLean and Pontiff ‘16)
Finding anomalies is a business and business has been good
314 at last count accourding to Cam and co-authors
But, what is behind these anomalies?
This is an important question the answer to which provides insight into market efficiency (or lack thereof), the portfolio allocation problem, and, ultimately, the efficiency of capital allocations.
Unmodelled risk and mispricing are symptomatic of model smisspecification
Data-snooping is a problem with test size (Type 1 errors, alpha, probability of rejecting the null when it is true.)
Barber & Lion (excluded securities), Fama and French (international markets), Asness et al. (other asset classes)
© Michael R Roberts

3. Key Findings 78% of anomalies “disappear” in pre- and post-periods
Sharpe ratios, alphas, and information ratios all decrease; volatility and covariation increase
Including investment and profitability
Sharpe ratio of 5-factor strategy ≈ Market Sharpe ratio (0.5) in pre-
Choice of in-sample period critical to significance
Small changes attenuate/eliminate many existing results
22% of anomalies survive
Pre-sample: real investment, equity financing, distress, ROE/ROA
Post-sample: Sales and earnings, total financing, distress, ROE/ROA
78% = 28 of 36 anomalies
Information ratio = alpha / SD of residuals
© Michael R Roberts

4. Economic Messages Quantify data-snooping concerns
Even robust anomalies are not robust out-of-sample
True asset pricing model would be rejected using in-sample data
In-sample corrections imperfectly correlated with out-of-sample tests
Anomaly survival tied to underlying macro shifts
1st half of sample  tangible investment and equity financing
2nd half of sample  intangible investment and debt financing
Does academic research lead to death of anomalies?
McLean and Pontiff 2016 test has no power against data-snooping alternative
Information ratio = alpha / SD of residuals
© Michael R Roberts

5. Data CRSP monthly returns 1926 to 2015
Compustat 1962 to 2015 (+ some info back to 1947)
Davis et al. ‘00 book value of equity 1926 to 1980
Moody's Industrial and Railroad Manuals 1918 to 1970
Graham, Leary, and Roberts (2014, 2015)
Limitations:
No financials and utilities
More aggregated than Compustat (e.g., no SG&A or R&D)
Data quality
Multiple checks and verifications (on top of checks in GLR)
© Michael R Roberts

6. Coverage
© Michael R Roberts

7. Illustrative Vehicle Profitability and investment factors
Novy-Marx 2013, Fama and French 2015, Hou et al (2015)
Profitability = OP/BE (FF 2015)
Investment = Asset growth (FF, Hou et al.)
Create HML-like factors for all anomalies
E.g., Investment
Portfolios held constant from July t to June t+1
Avg return on two low portfolios and two high portfolios then difference
Mitigate impact of small/micro firms
The new asset pricing models include protability and investment factors|Fama and French (2015) and Hou, Xue, and Zhang (2015) are very similar to each other
We use just these two factors to illustrate what we are doing
Different studies all measure profitability a bit differently.
The renewed interest towards protability is mostly attributed to Novy-Marx (2013)
In this section, we use Fama and French's (2015) operating protability;
later, we also construct Novy-Marx's (2013) gross profitability.
Hou, Xue, and Zhang (2015) construct their measure from quarterly data, so even they have to start their series in 1972.
© Michael R Roberts

8. Monthly Factor Premiums by Era
Value, profitability, and investment factors earn statistically significant premiums in the data
Size is marginally significant
Size was never as \good" as we thought. A part of it was that the data was not good because of, e.g., missing delisting returns.
Before 1963, only value premium is statistically significant
We knew this from Davis, Fama, and French (2000)
© Michael R Roberts

9. Monthly CAPM Alphas by Era
Here we adjust for market risk. Previous slide was like running regression of factor on a constant. Here we regress factor on a constant and market excess return
The value and investment factors gain a bit of statistical significance in the post-1963 data after we adjust for market risk
In the pre-1963 data, the value factor's CAPM alpha is statistically insignificant
This finding has been discussed in Fama and French (2006) and Ang and Chen (2007)|FF is in response to Ang and Chen
In pre-1963 data, value firms have higher market betas, and so CAPM explains away the value premium
Caution: This finding does not imply that CAPM “works”, average returns still do not increase in CAPM betas
© Michael R Roberts

10. Monthly 3-Factor Alphas
We run three-factor model regressions with the profitability and investment factors on the left-hand side
If the alpha (which we report here) is significant, it means that the factor contains some information about average returns not contained in the three-factor model
An investor would like to trade that additional factor
In the post-1963 data, both profitability and investment factors add something to the three-factor model
Profitability is significant in the pre-1963 data because profitability correlates negatively with value
This finding was highlighted in Novy-Marx (2013) for the post-1963 data; that profitability is really useful when combined with value
This negative correlation exists throughout the sample
In the same data, investment is useless
© Michael R Roberts

11. Characteristic Distributions
© Michael R Roberts

12. The Rest of the Zoo
© Michael R Roberts

13. Statistically Significant Individual Anomalies
In-sample
Every anomaly CAPM or FF-3 alpha
Pre-sample
8 average returns, 8 CAPM alphas, 16 FF-3 alphas
Post-sample
1 average return, 10 CAPM alphas, 9 FF-3 alphas
© Michael R Roberts

14. Average Anomaly across Eras: Returns and Sharpe Ratios
Block bootstrap SEs
The discussion should probably start from the in-sample numbers.
The average anomaly, when it is in-sample, earns 28 basis points per month; this estimate is significant with a t -value of 8.62.
Its annualized Sharpe ratio is 0.55
Now, we can move out-of-sample by moving either to the pre-discovery period or to the post-discovery period
In both cases, average returns and Sharpe ratios fall sharply
Now, let's look (on the next slide) at the dierences to see if we have power to say whether the average anomaly performs differently in-sample
© Michael R Roberts

15. Average Anomaly across Eras: Returns and Sharpe Ratios
Block bootstrap SEs
The discussion should probably start from the in-sample numbers.
The average anomaly, when it is in-sample, earns 28 basis points per month; this estimate is significant with a t -value of 8.62.
Its annualized Sharpe ratio is 0.55
Now, we can move out-of-sample by moving either to the pre-discovery period or to the post-discovery period
In both cases, average returns and Sharpe ratios fall sharply
Now, let's look (on the next slide) at the dierences to see if we have power to say whether the average anomaly performs differently in-sample
© Michael R Roberts

16. Average Anomaly across Eras: Alphas and Information Ratios
The results are the same.
There is very little in the pre-discovery period; there is a big jump up for the in-sample period; and then we fall back down.
The pre-in and post-in comparisons are always statistically signicant; the pre-post comparisons are not.
© Michael R Roberts

17. Average Anomaly across Eras: Alphas and Information Ratios
The results are the same.
There is very little in the pre-discovery period; there is a big jump up for the in-sample period; and then we fall back down.
The pre-in and post-in comparisons are always statistically signicant; the pre-post comparisons are not.
© Michael R Roberts

18. Identification Threats
Unmodeled risk:
Threat: Structural breaks
Changes in risks that matter to investors, information costs
Mispricing:
Threat: Transient fads
Learning:
Investors learning and trade away anomalies
© Michael R Roberts

19. Are Start Dates “Judiciously” Chosen?
All anomalies could have been measured as of 1963
Was there a structural break around this time?
1963
Compustat Release
Gross Profitability
Return on Assets
Profit Margin
Cash Flow-to-Price ●
© Michael R Roberts

20. Are Start Dates “Judiciously” Chosen?
1963
Compustat Release
Gross Profitability
Return on Assets
Profit Margin
Cash Flow-to-Price ●
© Michael R Roberts

21. Structural Break Test
Average return drops by 50%
© Michael R Roberts

22. Structural Break Test Average return decline 40%-80% .
© Michael R Roberts

23. Structural Break Test .
CAPM alpha decline
50%-75%
© Michael R Roberts

24. Structural Break Test
FF-3 alpha decline
30%-90% .
© Michael R Roberts

25. Correlation Structure of Returns
How does an anomaly correlate with other anomalies across eras?
Motivated by Mclean and Pontiff (2016)
anomaly = average return for anomaly i
Post = indicator = 1 in the post-sample period
InSample Index = average return on all anomalies except the i’th anomaly during the in-sample period
PostSample Index = average return on all anomalies except the i’th anomaly during the post-sample period
© Michael R Roberts

26. Correlation structure of returns: Post-sample
© Michael R Roberts

27. Correlation structure of returns: Pre-sample
The correlations change the same when we move from being in-sample to being in the pre-discovery sample
But here, it would be absurd to attribute this change in correlations to the actions of arbitrageurs
Rather, it looks like there is something special about the correlations when an anomaly is in-sample
Connect these results to the main results about the changes in alphas:
The pattern in alphas is consistent with data-mining being important
But it could also be explained by, e.g., well-timed structural breaks
An alternative explanation should also address how the correlations change the way they do
© Michael R Roberts

28. Do In-sample Adjustments Work?
Not really…
Pr(Type I error) = 30%
Pr(Type II error) = 26%
© Michael R Roberts

29. Conclusions and Future Work
Half-empty
Data-snooping is severe
Statistical adjustments have limitations
 Out-of-sample testing (new data, holdout samples)
Half-full
Persistent violations of common AP models
Appear correlated with economic fundamentals
In-progress:
What is the “right” model?
How does this model tie into economic fundamentals?
© Michael R Roberts