1. Talking about the Flint Lead crisis
Nairanjana Dasgupta(Jan)
Professor, Dept of Math and Stat
Director, CISER
WSU

2. Content A few words about CISER and how it works
The Crisis at Flint Michigan
How an outlier changed things
Analysis of the current data
Some thoughts about dealing with sensitive and high focus data

3. Statistical Help at WSU: Center for Interdisciplinary Statistics Education and Research (CISER)

4. The flint Michigan lead contamination problem
Step 1: What was the problem?

5. Case study: Water Safety in Flint
Brief history of the issue: Flint water
April 2014: Flint switches water source to Flint River
Lead and Copper Rule: 15ppb in 10% of homes tested OR 90th percentile of total sample  Action
Methodological controversies, including exclusion of two outliers that change the outcome of the action
Mike to introduce case study and Rashomon concept
Controversy over statistical analysis
Langkjær-Bain, R. (2017), The murky tale of Flint's deceptive water data. Significance, 14: 16–21. doi: /j x

6. The Flint Water Crisis
The Flint water crisis began in 2014 when the Flint River became the drinking water source for the city of Flint, Michigan. Due to insufficient water treatment, over 100,000 residents were potentially exposed to high levels of lead in the drinking water. A federal state of emergency was declared in January 2016 and Flint residents were instructed to use only bottled or filtered water for drinking, cooking, cleaning, and bathing. As of early 2017, the water quality had returned to acceptable levels; however, residents were instructed to continue to use bottled or filtered water until all the lead pipes have been replaced, which is expected to be completed no sooner than 2020.

7. Lead and Copper Rule (LCR) of 1991
EPA federal regulation
Limit allowed for lead is 15 parts per billion (ppb)
If the limit is exceeded in more than 10% of homes tested (90th percentile ≥ 15), action is required
To make sure problems are caught, sampling is supposed to target the “worst-case” homes – those in areas served by lead pipes.

8. Flint Sampling Errors Did not target worst case homes
They instructed residents to flush cold water for at least 5 minutes before sampling – which potentially decreases levels
Small openings in sampling bottles required gently running water
Regulations required 100 samples (pop >100,000) but decided to use minimum sample size of 60 due to recent population estimates
Removed 2 outliers from the sample of 71 because one came from a home that was “fitted with filters” and one came from a business so don’t qualify although they were both over the limit (104 and 20 ppb respectively).

9. Flint Lead Testing Quantiles
According to this analysis of the full data, the 90th percentile of the sample is 18 ppb, which exceeds the federal limit of 15 ppb. Consequently, the Flint water fails the safety test, and the city must take action to improve the water.
But that is not what happened. Instead, "the MDEQ told the city's water quality supervisors to remove two of the samples" (Langkjaer- Bain, p. 19) that were over 15 ppb. The MDEQ claimed that these data were improperly collected. The two data points that were excluded have the values 20 and 104. Because these values are both higher than the 90th percentile, excluding these observations lowered the 90th percentile of the modified sample, which has 69 observations.

10. Flint Lead Testing Quantiles
90th Pctl (all samples)
From Rick Wicklin, SAS

11. Confidence Limits
The 95% confidence interval for P90 is [8, 43], which is very wide and includes the critical value 15 ppb in its interior. If someone asks, "how confident are you that the 90th percentile does not exceed 15 ppb," you should respond, "based on these data, I am not confident at all."

12. Step 2: Flint Michigan, a follow-up
More data is now available
Let us see what the data says and if things have changed?
Is there any socio-economic relationship we need to think about.

13. Data Source: 23,177 sample test results
Data source: lead and copper test results from water samples in Flint, Michigan, from 9/2015-6/2016
Original source:
Data elements:
Date
Lead Parts Per Billion (ppb)
Copper ppb
Address (street number, street name, city, state, ZIP)
Mike to describe data source

14. The Data sets provided:
Provides a Excel file with Lead reads in ppm and residential addresses and dates collected.
Snapshot:
Sample Number
Date Submitted
Analysis (Lead)
Lead (ppb)
Analysis (Copper)
Copper (ppb)
Street #
Street Name
City
Zip Code
LG38207
4/22/2016
Lead
Copper
525
12TH ST
FLINT
48503
LG18552
2/26/2016
1907
2ND AVE
LG20757
3/6/2016
260
308
4TH AVE
LG07760
2/2/2016 7
2417
ADAIR ST
48506
LG10679
2/8/2016 8 70
2421

15. What is Flint, Michigan? What Zip Codes comprise this city?
Pop Pop
ZIP

16. Additional Flint Lead Tests
Red dots for lead tests >= 15
Data from:

17. Mapping of High Lead Samples
Large Concentration Areas to Start Immediate Remediation

18. Step 1: Plots and EDA The most crucial step: understanding the data
This involves: plotting and asking background questions (if allowed)
Look at potential issues of the data, using plots
Our focus will be Max Lead and we will look at the aggregated data over each zip code.
Zip code rather than street addresses
We would like to see what how high lead concentrations are related to socio economic variables.

19. Summarizing Lead Data over all 8 zip codes

20. Graphing Lead data > 10 ppb (2400 houses)

21. Graphing Lead > 1000 ppb (56 houses)

22. High lead concentration and % of whites and Zip Codes

23. Max Lead in relationship to Socio-Economic Variables

24. Some findings: Maximum lead concentration in Zip 48504: 22905 ppb!!
In the high income zip codes, maximum lead concentration seemed to be lower, but needs more investigation
Less percentage of whites, higher the lead concentration, in general
Population and Median Age for the zip codes do not seem to be related to Max lead concentrations.
VERY rough findings, but this is the line and direction and flavor of analysis I would pursue.

25. Flint Sources
The murky tale of Flint’s deceptive water Data by Robert Langkjaer- Bain in Significance Magazine April 2017
Quantiles and the Flint water crisis in The Do Loop by Rick Wicklin 5/17/2017
An overview of the Flint Water Crisis on Wikipedia
Data downloaded from residenttest

26. Some work policies if we are working on a post-research review or policy work
Try to stay away from media reports on the matter
TRY to focus on the data and not on opinions
Stay as unbiased as possible
ONLY use the data source provided or other government sponsored data source
We look at the numbers and also beyond the numbers.

27. My own research
How do you try to account for and reduce false positives in any study?
Will the addition of a neutral zone help false positives when trying to classify data into groups?
How do you deal with ordinal data?