1. DRAFT - not for publication Nonresponse Bias Analysis in a Survey of Banks Carl Ramirez U.S. Government Accountability Office ramirezc@gao.gov

2. DRAFT - not for publication Overview Describe NR bias analysis techniques and results in one survey of small, minority-owned financial institutions. Make observations about NR bias analyses in small population surveys Describe emerging NR bias analysis practices at one US Federal research agency

3. DRAFT - not for publication Survey of 196 Minority-Owned Banks Census of all institutions defined by at least one regulator as minority-owned as of March 2006. Web survey, mail/fax options upon request President/CEO was targeted respondent 76% unit response rate (AAPOR RR2). Key estimates: Awareness of and attendance at regulator programs, rating of regulator efforts to preserve minority ownership, financial outlook

4. DRAFT - not for publication Survey Fieldwork Field period: March 14 to April 28 – 6 weeks & 3 days Contacts: –Precontact call in Feb. to determine eligibility and get email –Prenotification email in early March –Survey cover email March 14 –2 email NR followups –National Bankers Association endorsement contacts –Multiple phone NR followups by program analysts –Final reminder and closeout emails in late April

5. DRAFT - not for publication NR Bias Analysis Typology (Groves & Brick) Compare survey estimates to other benchmarks Compare R ’ s to NR ’ s on auxiliary variables (frame, external data, fieldwork observations, etc.) Examine variation within respondents (subgroups, converted nonrespondents, early/late responders) Alter weighting adjustment

6. DRAFT - not for publication Unique Aspects of Establishment Survey NR Bias Analyses More longitudinal sample designs = previous response benchmark opportunities Richer administrative data = more benchmarks and survey variable correlates Smaller populations and subgroups = potential for examining individual nonrespondents

7. DRAFT - not for publication NR Bias Evaluation Methods Applied to Survey of Banks Compare respondents to sample on key variables Compare response rates of subgroups defined by frame information, related to survey variables of interest Level of Effort (Time of Return) Subsample of nonrespondents converted with high- effort followup

8. DRAFT - not for publication Respondents vs. Entire Sample on key auxiliary variable: Regulator R’sSample FDIC54%55% Federal Reserve11 OCC2322 OTS1211

9. DRAFT - not for publication Respondents vs. Sample: Minority Type R’sSample African American26%24 Native American10 Asian3934 Hispanic1724

10. DRAFT - not for publication Respondents vs. Sample: Size R’sSample < $100 Million assets43%41% $101-$300 M32 $301-$500 M129 $501M - $10 billion1115

11. DRAFT - not for publication Differences in Response Rate by Regulator of Bank R’sNR’s Federal Reserve82%18 OTS82%18 OCC77%23 FDIC73%27

12. DRAFT - not for publication Differences in Response Rate by Minority Type R’sNR’s African American81%19 Native American80%20 Asian76%24 Hispanic70%30

13. DRAFT - not for publication Differences in Response Rate by Profitability R’sNR’s Low return on assets81%19 Medium75%25 High73%27

14. DRAFT - not for publication Level of Effort Analysis (Time of Return) Key Estimates Early (n=51) Middle (84) Late (9) Received regulator mailings on minority program 48%5056 Aware of FDIC minority bank web page 55%5144 Attended FDIC minority roundtable or conference 45%4356 Used regulator’s technical assistance program 25%2756

15. DRAFT - not for publication Analysis of Respondents with other NR Characteristics Comparison of low-salience (low response propensity) respondents to all other respondents: 12 banks not considered by primary regulator as “ minority owned ” but so designated by one of the other 3 regulators. Removing these respondents changed key estimates by 1-2%, none by more than 4-5%

16. DRAFT - not for publication Micro-Level Examination of Nonrespondents Small sample allowed in-depth study of individual banks not responding 4 explicit refusals – known reasons appeared largely unrelated to key measures Personal contacts for NR followup yields known reason for delay

17. DRAFT - not for publication Emerging GAO practice on NR Bias Analysis? Groves & Brick typology Suggests hierarchy of methods on 2 dimensions: –Prefer use of data related to survey variable over comparison of subgroups or time of return analysis –Prefer methods using data available for entire population over portion of population or sample If conduct analyses, describe in report