META-ANALYSIS

Types of Meta-Analysis Qualitative Quantitative 2

Quantitative Meta-Analysis Data Sources Primary – the ideal data source –Raw data Secondary: –The data and conclusions presented in study reports / papers Tertiary: –Distillations of the data and conclusions presented by the secondary sources 3

Major Flaw in Existing Meta-Analyses Depend on tertiary data (third-hand information) Use extremely broad explanatory variable definitions Generally have very low explanatory power Coefficients frequently have confidence levels of less than 50% 4

Literature Example Lipscomb, Mooney and Kilpatrick “Do Survey Results Systematically Differ from Hedonic Regression Results? Evidence From a Residential Property Analysis” - Forthcoming, Journal of Real Estate Literature 5

Issues No comparison of results to actual market results 6

7 Issues Only peer-reviewed papers used without a test for bias resulting from restricted data selection

From the Paper Comparison of Descriptive Statistics and Meta-Analysis Results FactorDescriptive StatisticsMeta-Analysis Water-6.1%-1.27% Superfund-11.4%-1.78% Air-2.5%-1.81% Linear-10.4%-3.02% Urban-5.7%-1.57% 8

From the Paper Explanatory Coefficients Significant at 95% Confidence Level or Above VariableCoefficientStandard ErrorT-statistic Constant LITIG URBANDIS SUBURBAN UNEMP INFO YEAROFDATA N = 273; Adjusted R-Squared =

From the Paper Explanatory Coefficients Not Significantly at 95% Confidence Level VariableCoefficientStandard ErrorT-statistic STUDY_SURV LANDFILL BELOW THIS POINT THE CONFIDENCE LEVEL IS LESS THAN 85% INTERNATIONAL REALVAL LOTSOFOBS NATIONAL LINEAR BELOW THIS POINT THE CONFIDENCE LEVEL IS THAN 50% URBAN SUPERSITE LNDIST MIXED WATER AIRCAFO REGIONAL POSITIVE NUKEMANUF LNSAMPLE % of coefficients less than 85% confidence 43% of coefficients less than 50% confidence 10

Base Studies A number of the base studies used in the meta-analyses to date can be shown to be flawed 11

Summary Quantitative Meta-analysis: Hedonic analysis of a regression model BUT Meta-analysis has significantly greater issues Third-hand data Low explanatory power Low statistical significance of coefficients Overly-broad variable definitions Virtually complete lack of confirmation 12

META-ANALYSIS IS NOT AND SHOULD NOT BE AN ACCEPTED APPRAISAL METHOD 13

CONTINGENT VALUATION CV CVM POTENTIAL BUYER SURVEYS 14

DEFINITION “According to the World Bank Institute (2002, p. 2), the CVM is a stated preference method used to estimate ‘‘the value that a person places on a good, usually one that is not sold in markets, such as environmental quality or good health.” Lipscomb, Clifford A., Max Kummerow, Will Spiess, Sarah Kilpatrick, John A. Kilpatrick, “Contingent Valuation and Real Estate Damage Estimation,” Journal of Real Estate Literature, Volume 19, Number 2,

REASONS FOR DOING CVM real estate is a market good Argument for CVM based on the idea that those who purchased are ill-informed, imprudent, or plain foolish, and the prices paid do not fully reflect the impact of the issue a CVM survey must be performed to determine how the market should have responded. 16

Dr. Kinnard’s “dictum” has been stated as: “The best information may be found in transactions that did not occur.” The correct statement is: Possibly the best information might be found in failed sales of impaired property. 17

COURT RULINGS The courts do not agree Melody Baker et al v Motorola, Inc, et al, Superior Court of the State of Arizona in and for the County of Maricopa, CV Circuit Court for Baltimore County Case # 03-C IN THE COURT OF APPEALS OF MARYLAND No. 15 September Term, 2012 EXXON MOBIL CORPORATION v. THOMAS M. ALBRIGHT, et al. 18

WHAT IS BEING MEASURED COMPENSATING SURPLUS: “That amount that would be required to make the property owner indifferent between remaining in the property with the issue, and moving to a property without the issue.” Mundy, Bill and David McLean, “The Addition of Contingent Valuation and Conjoint Analysis to the Required Body of Knowledge for the Estimation of Environmental Damages to Real Property,” Journal of Real Estate Practice and Education, Volume 1, Number 1, {Referencing “The straightforward discussion of compensating surplus can be found in Mitchell and Carson (1990), p. 24.”} 19

WRONG RESPONDENTS All of the surveys to date use respondents chosen at random and not qualified with respect to specialized knowledge of the issue. Surveys are normally less than ten minutes. Respondents do not have time to fully consider the issues. Respondents do not have time to consult other players 20

LACK OF CONFIRMATION CVM proponents have not tested the accuracy and reliability of their predictions. The research by Roddewig and Frey noted error rates of 200% when compared to actual market performance 21

TRIMMED MEAN “This method {marginal bidder analysis} eliminates extreme bids and gives more weight to those bids closer to the middle of the bid distribution. This is a form of trimmed mean analysis, which eliminates outlier observations.” Lipscomb, Clifford A., Max Kummerow, Will Spiess, Sarah Kilpatrick, John A. Kilpatrick, ibid. 22

TRIMMED MEAN The buyers and sellers have no interest in taking the “middle of the bid distribution” as a sale price. The seller wishes to maximize his price and only the top few bidders would be considered, all others bids being put in the circular file where the real market participants actually put them. 23

NO BASELINE Simons CVM studies of leaking gas stations provide an indicated discount. HOWEVER, no base line to let us determine the discount for a gas station without a leak. This is a common CVM problem. 24

NOT AN APPROVED METHOD Not in USPAP or Advisory Opinions or Guide notes Not endorsed in AI training materials Plenty of market data Intended only for non-market data and then for amounts generally less than $50 25