1. Bulletproofing the investigation JAWS 2014 Jennifer LaFleur, CIR (With artwork from young friends of journalism)

2. Investigative reporting is a powerful tool.

3. Snapshot of big famous story.

6. It can make this

8. Into this

11. It gives us the power to check assumptions.

13. Know the extremes

18. First, we must face a fact.

22. But don’t be afraid…

27. Bulletproof your data and documents Before ever reporting data or building an app Do integrity checks to find the flaws Add caveats where necessary Do your own analysis rather than relying on an agency’s analysis

28. External checks Understand what the data and documents mean. Know how many records you should have. Check counts and totals against reports. Are all possibilities included?

29. Internal checks Compare fields to check for red flags More teachers than students More money going to vendors than to contractors What things just don’t make sense

32. Integrity checks for every data set Check for missing data/documents

35. Integrity checks for every data set Check for missing data Check for duplicates

36. Duplicates may not always be obvious

37. Integrity checks for every data set Check for missing data, misplaced data or blank fields Check for duplicates Take margins of error into account if necessary

38. Median HH Income by Metro Area

39. Integrity checks for every data set Check for missing data, misplaced data or blank fields Check for duplicates Take margins of error into account if necessary (important if you’re using Census data). Check for outliers and extreme ups and downs

40. Truck accidents by year and agency

41. Beyond the basics Keep a notes file If you’re working with data or documents – keep a copy of your original Know the source Check against summary reports As you begin to write, footnote your documents and data

42. Texas test score data official results versus district Duncanville district reported 4 th grade writing Official report for Duncanville 4 th grade writing Courtesy Holly Hacker, The Dallas Morning News

43. Beyond the basics Keep a notes file If you’re working with data or documents – keep a copy of your original Know the source Check against summary reports Use the right tool

44. H/T Matt Waite

45. Disparities in water usage “Water use highest in poor areas of the city” Mapping and statistical analysis

46. Beyond the basics Check with experts Know the standards Find out what others have done Gut check – does it just seem wrong?

48. Beyond the basics Check with experts Know the standards Find out what others have done Gut check Go physically see a record or spot check against documents

50. If you’re matching or backgrounding people… A name is not enough. Lots of people have the same name Get dates of birth and other information to make sure you have the correct person.

51. Even people with seemingly unique names aren’t so unique

53. Be transparent: Bounce your findings off targets and tell users/readers/viewers what you did.

56. From the data source Get the questionnaire and methodology Beware of nonscientific methods: Web surveys, man on the street Know the sample size Account for margin of error and non-response Run statistical tests on the data if possible

57. Erroneous government databases can often be a story themselves

59. 123456789 compared to 123-456-789

61. What appears to be an error may be the story.

63. Why data? Contrasts are in the data Your most powerful figures are in the data You can make connections you might not be able to make otherwise

65. Why data? Contrasts are in the data Your most powerful figures are in the data You can make connections you might not be able to make otherwise Test assumptions

67. When there is no data, you may have to make your own.

69. From the data source Get the questionnaire and methodology Beware of nonscientific methods: Web surveys, man on the street

72. @j_la28 jlafleur@cironline.org A special thanks to: Tracy, Olivia, Iris, Emmeline, Thomas, Silas, Carson, Harper and Erin