1. Utah Department of Health 1 1 Identifying Peer Areas for Community Health Collaboration and Data Smoothing Brian Paoli Utah Department of Health 6/6/2007

2. Utah Department of Health 2 Acknowledgments - Dr. Lois Haggard, UDOH - Dr. David Mason, Univ. of Utah - Mohammed Chaara, Univ. of Utah - Michael Friedrichs, UDOH - Kathryn Marti, UDOH

3. Utah Department of Health 3 Outline - Background: –Why Peer Areas? –Data Smoothing –Previous Peer Area Attempts - Methodology and Procedures –Utah’s 61 Small Areas –Demographic Similarity –Producing Smoothed Estimates

4. Utah Department of Health 4 Why Peer Areas? - Community Collaboration –Identify areas that are similar for purposes of comparison –Collaborate on strategies, interventions - Data Smoothing –“Borrow strength” from geographic areas that are similar. –Especially useful when multi-year trend data are not available

5. Utah Department of Health 5 Data Smoothing - Why smooth? –We calculate measures, such as rates of death and disease, to assess the underlying disease risk in a population. –Measures from small populations are inherently erratic – subject to sampling variation. –Rare events such as infant mortality can vary widely from year to year.

6. Utah Department of Health 6

7. 7 Data Smoothing - Most methods smooth data over time, requiring data from multiple years –Pool multiple years (e.g., 3- or 5-year averages) –Moving average –Combine areas to increase the number of cases

8. Utah Department of Health 8 Goal - Produce reliable (smooth, not erratic) and timely estimates. - Make appropriate inferences about the underlying disease risk in each community - Method must be simple to apply – and easy to implement

9. Utah Department of Health 9 29 UT Counties

10. Utah Department of Health 10

11. Utah Department of Health 11 Utah Small Areas - 61 small areas were defined using both ZIP code and County –Each area is either a ZIP code area, –two or more contiguous ZIP codes, or –a combination of ZIP code and county information.

12. Utah Department of Health 12 61 UT Small Areas

13. Utah Department of Health 13

14. Utah Department of Health 14 Previous Attempts - Geographic adjacency –rooks’s adjacency –queen’s adjacency –geographic centroid –population centroid

15. Utah Department of Health 15 Previous Attempts - Cluster analysis –odd-sized groups –odd-ball areas –groups mutually exclusive (okay, but not necessary)

16. Utah Department of Health 16 Develop Methodology to: - Identify Peer Areas –Create “Demographic Distance” matrix - Smooth Data –Median? Pooled? Weighted? - Measure Our Success –Reliability of results –Appropriateness of making inference to index area from smoothed rates

17. Utah Department of Health 17 1. Identify Peer Areas - Demographic Characteristics –Use available demographic information from the U.S. Bureau of the Census –Use demographic variables that are associated with population health –Select a small number of these demographic variables –Produce a methodology others can replicate

18. Utah Department of Health 18 1. Identify Peer Areas - Factor analysis to reduce dimensionality. –Foreign born/Hispanic –Education –Income/Poverty –Employment –Age –Urban and Rural

19. Utah Department of Health 19 1. Identify Peer Areas - Selected 5 variables based on factor analysis and correlation with health outcomes (e.g., infant mortality, heart disease, etc.) –% Hispanic –% age 25+ with Bachelor’s degree –% children in poverty –% owner-occupied housing –% age 65+

20. Utah Department of Health 20 1. Identify Peer Areas - Create distance matrix The distance d(x,y) between two areas with n dimensional observations x=[x 1,x 2,…,x n ]’ and y=[y 1,y 2,…,y n ]’ is: d(x,y)= ([x-y]’S -1 [x-y]) 1/2 The matrix S contains the variances and covariances of the n variables.

21. Utah Department of Health 21 Identify Peer Areas - Demographic distance

22. Utah Department of Health 22 1. Identify Peer Areas - Which are the Peer Areas for purposes of collaboration? –3 (or some #) areas with smallest distances? –All areas within a certain distance?

23. Utah Department of Health 23 1. Identify Peer Areas - Which are the Peer Areas for purposes of data smoothing? –Same areas as for collaboration? - Need to think about the smoothing algorithm.

24. Utah Department of Health 24 2. Smooth the Data - Options –Weighted Median rate using a group of five areas –Pool a selected number of areas together and treat them as a single area (crude rate for the combined areas) –Pool all areas together and weight them by a function of their distances to the index area (closer areas -> more weight)

25. Utah Department of Health 25 Close Neighbors Distant Neighbors

26. Utah Department of Health 26 Areas that are close contribute more to the smoothed rate Areas that are distant contribute little to the smoothed rate

27. Utah Department of Health 27

28. Utah Department of Health 28

29. Utah Department of Health 29

30. Utah Department of Health 30

31. Utah Department of Health 31 3. Measure Our Success - Reliability –Did the data get smoother? –Intraclass Correlation Coefficient (ICC) Ratio of the amount of variance between areas to the sum of the variance within and between areas (MS between – MS within )/( MS between +(k-1)Ms within ) Range from 0 to 1 1 = perfectly smooth and level, only variance in the data is from one area to the next

32. Utah Department of Health 32 3. Measure Our Success - Appropriateness of Inference –Is it appropriate to infer that the smoothed rate represents the true underlying disease risk in the community? (Overall, are the smoothed scores in the ballpark?) –Sum of Squared Differences (SS) from smoothed data to original data. Smoothed estimate should be close to the index area’s crude rate

33. Utah Department of Health 33 3. Measure Our Success - ICC – Want high scores, close to 1 - SS – Want low scores, given high ICC - HOW DID THE SMOOTHED RATES PERFORM?

34. Utah Department of Health 34 Smoothed ICC=.901 Crude ICC=.835

35. Utah Department of Health 35 Smoothed ICC=.901 Crude ICC=.835

36. Utah Department of Health 36 Smoothed ICC=.901 Crude ICC=.835

37. Utah Department of Health 37 Smoothed ICC=.901 Crude ICC=.835

38. Utah Department of Health 38 Summary - A small number of demographic variables were identified –Capture the demographic variability –Related to health outcomes - Peer Areas were identified –Groupings seem intuitive

39. Utah Department of Health 39 Summary - Smoothing algorithm was identified –Had characteristics we liked Index area gets highest weight Peer areas get high weights Dissimilar areas weight=0

40. Utah Department of Health 40 Summary - Smoothed rates performed generally well –They were smooth (ICC ~ 1.0) –They represented the underlying risk in the index area (SS relatively small)

41. Utah Department of Health 41 Summary - Easy to replicate? –Excel spreadsheet You: –Enter your demographic variables –Enter health outcomes for the same areas –Change smoothing parameters (if desired) Excel: –Calculates distance matrix –Generates smoothed rates –Generates performance measures

42. Utah Department of Health 42 Challenges/Limitations - Demographic characteristics change, distance scores will need to be updated (decennial census years?) - How much smoothing to use is a subjective decision. - Smoothing may not seem credible to members of community - Peer Groups are not symmetric

43. Utah Department of Health 43 Excel Spreadsheet The spreadsheet is free and the files can be downloaded from the IBIS website. Go to http://ibis.health.utah.gov Look for “Peer Area Analysis Tool” under the “News and System Enhancements” heading.

44. Utah Department of Health 44 - Contact Information: - Brian Paoli Office of Public Health Assessment Utah Department of Health 288 North 1460 West P.O. Box 14201 Salt Lake City, Utah 84114-2101 email: bpaoli@utah.gov