A Review of the Address Coverage Enhancement Scheme for In-person Household Surveys Michael Jones, Westat Sylvia Dohrmann, Graham Kalton, Jean Opsomer, Westat JSM 2019 • Denver, Colorado • July 27-August 1 The views presented in this paper are those of the author(s) and do not represent the views of any Government Agency/Department or Westat

The Three Steps of the ACE Procedure Step 1: Select area segments for ACE Probabilities of selection Step 2: Field work in ACE segments Canvas the segments Step 3: Verify addresses added in the field through ACE In-house review of addresses Vendor comparison of addresses to the CDS

Illustrative Example – Area Segment vs List Segment

Illustrative Example – Area Segment vs List Segment Illustration of ACE Area Segment List Segment

Step 1: Select ACE Segments ACE is performed in a subsample of the study’s area segments Let P(i) = P(area segment i is selected for ACE) = kiri , Where 1/ki = P(selecting ACE added address j if area segment i is selected) And ri = the within segment sampling rate for list segment i ki can be adjusted depending on the estimated CDS coverage and to control workload

Step 1: Select ACE Segments – Adjusting Probabilities of Selection ki > 1 if CDS coverage is expected to be low E.g., set ki = 3 then P(i) = 3ri if the number of ACE added addresses is 30, we sample 10 ki < 1 if CDS coverage is expected to be high E.g., set ki = 0.75 then P(i) = 0.75ri if the number of ACE added addresses is 3, take all and adjust weights by a factor of 1.333 ki = 1/ri if CDS coverage is expected to be LOW P(i) = 1 and the within segment sampling rate is ri

Step 1: Select ACE Segments – Indicators of Coverage & Improving Probabilities Comparison of segment-level census count and CDS count If census count is much larger, can indicate poor coverage Urbanicity of segments Coverage generally better in urban areas Improving the probabilities of selection for ACE segments How to best define urbanicity? Could some segments with expected good coverage be assigned a zero chance of inclusion for ACE?

Step 2: Field Work – The Canvas Field workers canvas the area segments List segment addresses are provided on a tablet Compare the addresses inside the area segment to the addresses on the tablet, i.e., to the list segment Addresses not in the list segment are added to the tablet Transfer the added addresses back to the home office

Step 2: Field Work – Timing ACE field work can be done before or during the data collection period Preference is to perform it before Avoids conflicts with data collection if interviewers perform ACE Allows for coordinated sampling of ACE and CDS addresses If conducted by interviewers during the data collection period Better for interviewers to complete ACE before interviewing CDS cases

Illustrative Example – No Geocoding Error Illustration of ACE Area Segment List Segment Located and geocoded inside area segment Located inside area segment, not on CDS

Illustrative Example – Geocoding Error Illustration of ACE Area Segment List Segment Located outside area segment, geocoded into the area segment Located and geocoded inside area segment Located inside area segment, on CDS Located inside area segment, not on CDS

Step 2: Field Work – Geocoding Error Enhancement procedures treat geocoding differently How enhancement procedures handle geocoding error Type of Geocoding Error ACE EL & CHUM Address located outside the area segment, geocoded inside Retained on frame Removed from frame Address located inside the area segment, geocoded outside Not added to frame Added to frame

Step 2: Field Work – Geocoding Error The level of geocoding error depends on the geocoding methodology Most implementations of ACE have used our vendor’s geocoding Street address interpolation is the most precise Interpolates a position along a street section based on the street addresses More recently, we have used in-house land parcel geocoding Geocoordinates on the address property is the most precise Street address interpolation is the next level Therefore, at least as accurate as the vendor’s method

Step 2: Field Work – Geocoding Error As part of ACE, the field workers record whether the list segment addresses were located within or outside the segment. The percentage of CDS addresses in the list segments correctly geocoded into the area segments Vendor street address interpolation Westat land parcel geocoding Urban segments 93% 97% Rural segments 78% 92% Overall 91% 96%

Step 2: Field Work – Geocoding Error The greater the amount of geocoding error, The more addresses added by ACE that need to be resolved The more addresses sampled from the list segment that are located outside the area segment Percentage of addresses added in ACE field work that geocoded to another area segment Vendor street address interpolation Westat land parcel geocoding Urban segments 68% 47% Rural segments 37% 20% Overall 52% 32%

Illustrative Example – Canvassing Complete Illustration of ACE Area Segment List Segment Located outside area segment, geocoded into the area segment Located and geocoded inside area segment Located inside area segment, not on the field-worker’s list

Illustrative Example – Differentiating Address Types Illustration of ACE Area Segment List Segment Located outside area segment, geocoded into the area segment Located and geocoded inside area segment Located through ACE, on CDS Located through ACE, not on CDS

Step 3: Verification of Addresses Added in the Field Compare 2nd review addresses to to CDS Field Worker Review addresses not on CDS Vendor Indicate addresses not on CDS Initial review of all added addresses Compare added addresses to CDS Canvas ACE segment and enter addresses not on tablet Addresses not on CDS are added to create an ACE enhanced frame and are ready to be sampled

Step 3: Streamlining Verification of Addresses Added in the Field Consider eliminating the second review Of the ACE added addresses determined to be on the CDS, 95% are identified during the first vendor comparison Do not review all ACE added addresses Instead, select the sample of ACE added addresses after canvassing Verify the address during the screener interview Send the verified addresses to the vendor to determine which are on the CDS If address is on the CDS, adjust the weight by one-half

Illustrative Example – Select the Sample Illustration of ACE Area Segment List Segment Located outside area segment, geocoded into the area segment Located and geocoded inside area segment Located through ACE, on CDS Located through ACE, not on CDS Sampled

Summary Benefits of the ACE procedure include Only a sample of segments undergo the procedure, but provides full coverage Is a separate procedure from data collection, i.e., timing is flexible Retains all CDS addresses geocoded to the area segments Areas for further research Improve methods for identifying areas with high numbers of addresses not on the CDS file Improve the method for checking addresses against the CDS Consider benefits of accepting some addresses on the CDS and halving their weights

Thank you! Questions can be directed to: MichaelJones@Westat.com