2. A Critical Examination of Hedonic Analysis of a Regression Model (HARM) and META-ANALYSIS Albert R. Wilson BSSE, MBA, CRE (Ret) 1

3. Regression Model A model intended to allow an exploration of the hypothetical relationship between possible explanatory variables and the sales price 2

4. Regression Model Reflection of reality The touchstone of that reality? Actual market participants 3

5. “Estimated” versus “Predicted” Estimated = Sale IN database Predicted = Sale NOT IN database 4

6. Predicted Sales Prices At the mean predicted sales price variance is larger than estimated variance by σ 2 (variance in the data) 5

7. Mean Confidence Intervals (MCI) Estimated and Predicted MCI FOR PREDICTED 4.38 TIMES MCI FOR ESTIMATED 6

8. DATABASE EDITING GARBAGE IN => GARBAGE OUT (GIGO) 7

9. Case Example Influence on the Removal of “Flipping Transactions” on the Predicted Prices for 33 Properties PREDICTED SALES PRICES PROPERTY NO.AS PRESENTEDFLIPS REMOVED% CHANGE SUM5,069,2394,018,112(1,051,127) n391379-12 Adj. R-squared0.76840.7593-0.0091 8

10. Editing and Confirmation of Data STEP 1: Edit to identify obvious issues (the desk edit) Case Example Assessor’s Data4,325Removed74717.3% R-Squared0.790.83 MLS Data1,888Removed77944.3% 9

11. Editing and Confirmation of Data STEP 2: Identify sales that are not appropriate to the analysis 10

12. Editing and Confirmation of Data STEP 3: Sales confirmation A values-neutral interview of sale participants OBJECT: to elicit the primary factors motivating the conclusion of the sale price MUST NOT INTRODUCE ANALYST OPINION THIS IS THE ONLY MEANS OF IDENTIFYING/CONFIRMING THE REASONS FOR A CONCLUDED PRICE 11

13. Regression Model Considerations Faithfully represent: Identified concerns of actual market participants Restrictions imposed by the data Estimates of prices the ONLY VERIFIABLE OUTPUT 12

14. Coefficient Calculation Result of iterative calculations designed to provide the most accurate estimates of sales prices in database 13

15. Coefficient Calculation Goodness of Fit Measures of the Goodness of Fit apply only to the relationship between the estimated and actual sales prices in the database They do not apply to the coefficients 14

16. Most commonly-cited Goodness-of-Fit Measure R-Squared (Coefficient of Determination) 15

17. R-Squared Generally-applied interpretation: –R-Squared is the amount of variance “explained” by the model 16

18. Low R-Squared Models Mathematically, as the R-Squared approaches 0.30, it becomes more likely that the model is only measuring random effects 17

19. The Omitted and Additional Variable Problem Omitting generally increases magnitude and statistical significance of the remaining coefficients Adding generally decreases the magnitude and statistical significance of the remaining variable coefficients 18

20. Illustration of Omitting or Adding a Variable Base ModelAdded Variable–APNOmitted Variable–Pool VariableCoeff.t-statCoeff.t-stat% ChangeCoeff.t-stat % Change Intercept67,37017.52-663,632-8.14-1085.06%66,29317.14-1.60% APN.0238.98 Fixtures2,6535.392,5115.15-5.35%2,8865.848.74% NoPatio(12,801)-7.77(5,036)-2.73-60.66%(13,451)-8.135.08% SqFt40.7929.2342.8030.614.93%41.5929.721.96% Pool8,3666.778,9087.286.48% Garage19,38212.9020,15313.543.98%19,98013.243.09% Middle Ring(16,141)-11.24(11,230)-7.38-30.43%(15,276)-10.61-5.36% Inner Ring(8,875)-4.52(7,114)-3.64-19.84%(8,012)-4.06-9.72% 20002070.081,787-0.67763.29%2710.1030.92% 2001(2,017)-0.766650.258-132.97%(2,028)-0.760.55% 2002(719)-0.253,9761.36-652.99%(615)-0.21-14.46% 20037,2132.677,6472.866.02%7,2582.710.62% 200441,14915.5040,38015.37-1.87%40,90115.31-0.60% 2005132,07751.04130,66250.93-1.07%131,12950.43-0.72% 2006160,36745.29159,84245.63-0.33%159,89744.89-0.29% R-Squared 0.83 19

21. Consequences of Variable Selection Including the Assessor’s Parcel Number APN Coefficient Value0.023 t-statistic8.98 Mean Value30,834,360 R-Squared0.83 Mean Sale Price$211,000 Results in an incremental increase in the sales price of 0.023 x 30,834.360 = $709,190 (APN Coef.)x(Mean Value)=(Incremental Increase) 20

22. Consequences of Variable Selection Omission of a Variable: Removal of “Pool”; present in 38% of properties –SQFT Cofficient changed from $40.79 to $41.79 –Approximately the same t-statistic Removal of “Fixtures”; present in 100% of properties –SQFT Coefficient changed from $40.79 to $46.50 –T-statistic = 50.94 21

23. Coefficients Coefficients are simply multipliers for the explanatory variable 22

24. Causation in Real Estate From the Real Estate Appraiser’s perspective: 1.Causation demonstrated through sales confirmation interviews. 2.Causation NEVER proven through a regression. 23

25. Strengths and Weaknesses Can never be better than the data Requires significant amount of data: five to 15 or more sales Upper limit to the amount of data: too much may be worse than too little Guide: Are the sales competitive to the subject? Estimate of sales prices most accurate at the mean value of the data Variance of a predicted sales price larger than variance of estimated Thousands of possible regression models 24

26. Further Considerations Absent standards, the “Rubber Ruler” may apply When recognized and published standards are not used, author must demonstrate the accuracy and reliability of his/her work 25

27. Hedonic Analysis

28. The Hedonic Assumption The coefficient accurately and only represents the contribution of the declared meaning of the explanatory variable to the sale price 27

29. Hedonic Analysis The validity of the hedonic assumption must be demonstrated 28

30. “Revealed Preference” Idea cannot be supported for real estate

31. Supporting Literature Not a single paper demonstrated the validity of the hedonic assumption PLUS NO indication of confirmation of raw data NO indication of adherence to any recognized / published standards NO indication of confirmation of results with the normal or typical market participant THE RUBBER RULER EFFECT IS MUCH IN EVIDENCE. 30

32. Regression Model Accuracy If the regression model is inaccurate, then there is no reason to expect the coefficients to be accurate or meaningful. Therefore the HARM cannot be accurate. 31

33. CASE EXAMPLE TO POOL OR NOT TO POOL Using the data from the previous case. Does a pool influence value? By how much? The Hedonic Approach, the coefficient is the marginal contribution to value. 32

34. COMBINED POOL AND NO POOLS COMBINED POOL AND NO POOLS, POOL COEFFICIENT SET TO ZERO Variable COEFFICIEN T MEAN VALUES EXPECTED VALUES COEFFICIEN T MEAN VALUES EXPECTED VALUES Intercept54,089.83154,09054,089.83154,090 ORIG_FIXTURES2,805.338.7324,4912,805.338.7324,491 ORIG_NOPATIO-14,116.470.34-4,800-14,116.470.34-4,800 ORIG_POOL9,161.980.383,4829,161.9800 ORIG_SQF41.522283.6294,81541.522283.6294,815 ORIG_X_3GARA GE 16,212.830.46,48516,212.830.46,485 SY20005,980.3315,9805,980.3315,980 EXPECTED MEAN SALE PRICE 184,543 181,061 Adj R20.8816 33

35. TO POOL OR NOT TO POOL (CONT.) What are the coefficients if there is no pool? 34

36. COMBINED WITH NO POOL VARIABLE VariableCOEFFICIENTMEAN VALUESEXPECTED VALUES Intercept52788.1063152,788 ORIG_FIXTURES3,087.88018.7326,957 ORIG_NOPATIO-14,724.78430.34-5,006 ORIG_SQF42.39862283.6296,822 ORIG_X_3GARAGE16,924.6910.46,770 SY20005,727.746215,728 EXPECTED MEAN SALE PRICE 184,059 Adj R20.8790 35

37. Comparision Orig Fixt 2,805 3,088 Orig-nopatio -14,116 -14,725 Orig-no pool 9,162 NA Orig-sqf 41.52 42.40 Orig-garage 16,21316,925 SY2000 5,980 5,728 ESP $184,513 $184,059 R-sq0.880.88 36

38. POOL OR NOT TO POOL (CONT.) WHAT HAPPENS IF WE CONSIDER A DATABASE WITH POOLS, AND SEPARATELY A DATABASE WITHOUT POOLS? 37

39. WITH POOL ON PROPERTYWITHOUT POOL ON PROPERTY Variable COEFFICIEN T MEAN VALUES EXPECTED VALUES COEFFICIENT MEAN VALUES EXPECTED VALUES Intercept65,957.891.0065,95854,993.781.0054,994 ORIG_FIXTURES2,505.599.6524,1792,784.148.1622,719 ORIG_NOPATIO-15,415.460.22-3,391-14,838.470.41-6,084 ORIG_POOL ORIG_SQF41.632,586.79107,69041.462,097.2086,956 ORIG_X_3GARA GE 15,768.930.406,30816,308.320.315,056 SY20004,211.371.004,2117,209.871.007,210 EXPECTED MEAN SALE PRICE 204,954 170,850 Adj R20.08711 0.8895 38

40. POOLS AND NO POOLS SEPARATELY ESTIMATED SALE PRICE WITH POOL $204,954 – R-SQUARED0.87 ESTIMATED SALE PRICE W/O POOL $170,805 – R-SQUARED0.89 39

41. The Coefficient – What Counts? ALL THAT STATISTICAL SIGNIFICANCE CAN TELL US IS THAT FOR THIS MODEL AND DATABASE THE COEFFICIENT IS A SIGNIFICANT (OR INSIGNIFICANT) MULTIPLIER FOR THE EXPLANATORY VARIABLE. NOTHING MORE. 40

42. The Appropriate Standard: Economic Significance For us, economic significance is determined by what the normal or typical participant considers important to the conclusion of the transaction. 41

43. A Criticality: NOT ONE hedonic analysis encountered to date has actually asked this question: “What was important to you in concluding your transaction?” 42

44. Hedonic Analysis of a Regression Model (HARM) is: Highly inaccurate and unreliable method Not appropriate for appraisal work Observations apply to hedonic analysis NOT regression models! 43