1. Sensitivity and Representativeness of the Massachusetts Teens At Work Injury Surveillance System MyDzung Chu, MSPH Beatriz P. Vautin, MPH SangWoo Tak, ScD, MPH Letitia Davis, ScD, EdM CSTE Applied Epidemiology Fellow Massachusetts Department of Public Health Funded through CSTE-CDC’s Cooperative Agreement #5U38HM000414-5.1 and CDC-NIOSH’s Cooperative Agreement #2U60OH008490 (1)

2. 2 Designed by Shari Coté, 1 st Place Winner of MA’s 2013 Safe Jobs for Youth Poster Contest (3) In MA, 900+ teens seek ER care for work injuries every year (2). Nearly 70 die from work- related injuries (2).

3. Teens At Work (TAW) Injury Surveillance System: Population and Sentinel (3,4) 3 Goal: Reduce the incidence of work-related (WR) injuries to young workers, by: Identifying sentinel cases for worker and worksite follow-up Generating summary data to target broad-based interventions Using the data to promote prevention activities Case definition: A traumatic injury to a person under age 18 while the person was working for pay, 1)that was medically treated, or 2)for which a workers’ compensation lost-time claim has been filed, or 3)that was fatal.

4. Multi-source system 4 MA Dept of Public Health TAW Workers’ compensation claims for 5+ loss work days Emergency department: sample and statewide Fatal cases Teen interviewTeen follow-up Data analysis & Dissemination Employer follow- up Broad-based interventions Data sources TAW surveillance activities

5. 5 Targeted attributes (5) 1. Data quality 2. Predictive Value Positive (PVP) 3. Representativeness of sample ED cases 4. “Macro” Sensitivity TAW Surveillance System Evaluation

6. I. Representativeness of sample ED cases 6 Comparison of the data sources CharacteristicsSample ED data (n=9)Statewide ED data (N=68) Case definitionMedically treated cases <18 years old with WC as expected payer Timeliness of reporting MonthlyAnnually + time lag Data quality: Personal and employer identifiers YesNo, requires medical record abstraction Surveillance activities Sentinel: Interviews with teens, worker + worksite follow-up, case reports, inform broad-based interventions Population: provide summary data, inform broad-based interventions

7. 7 Question 1: Should TAW recruit more EDs into its sample to increase its representativeness? Which ones? I. Representativeness of sample ED cases? i. Case-level characteristicsii. Geographic distribution

8. Methods: 2009 sample and statewide ED cases Removed duplicates and illnesses Extracted medical records: Evaluated misclassification of ‘WC as expected payer’ variable TAW coded industry (NAICS) (6) and occupation (COC) (7) Coded injury (OIICS) from ICD-9 crosswalk (8) Compared percent distributions χ 2 Goodness-of-fit, significance at p<0.05 8 i. Are the current sample ED cases representative of state- wide ED cases based on demographic, employment, and injury characteristics?

9. Results: Demographics 9 Sample EDs Statewide EDs i. Are the current sample ED cases representative of state-wide ED cases based on demographic, employment, and injury characteristics?

10. 10 Results: Employment and Injury type Sample EDs Statewide EDs i. Are the current sample ED cases representative of state- wide ED cases based on demographic, employment, and injury characteristics? Yes – all χ 2 p >0.05

11. 11 ii. What regions of the state should TAW target to a) increase geographic representativeness of sample EDs and b) address regions with highest injury rates? Statewide EDs I. Representativeness of sample ED cases (9)

12. 12 Method: 2005-2009 statewide ED Removed duplicates and ineligible cases Calculated rates per Public Use Microdata Areas (PUMAs) PUMA: population of 100,000+ (10) MA has 48 PUMAs Link Census towns/cities to PUMAs Denominator: American Community Survey (ACS) (11,12) 16-17-year old workers Identify: PUMAs with the highest WR injury rates that are not represented by the current sample EDs ii. What regions of the state should TAW target to a) increase geographic representativeness of sample EDs and b) address regions with highest injury rates?

13. Mean rate: 12.2/1000 worker 13 PUMAs (rate ≥ 15.4): 100, 300, 1100, 1700, 1800, 1900, and 4300 ii. What regions of the state should TAW target to a) increase geographic representativeness of sample EDs and b) address regions with highest injury rates? Results: PUMAs with high WR injury rates not represented by current sample EDs, 2005-2009 (9)

14. Conclusions and Recommendations 14 TAW does not need to recruit more EDs Demographic, employment, and injury data of the current sample ED cases were representative of state-wide ED cases. Yet, to improve it’s geographic representativeness, TAW should consider recruiting more EDs From PUMAs with high WR injury rates not currently represented by the sample (i.e. 100, 300, 1100, 1700, 1800, 1900, and 4300) 14 Question 1: Should TAW recruit more EDs into its sample to increase its representativeness? Which ones?

15. II. ‘Macro’ Sensitivity 15 Challenge: Is there a ‘gold standard’? TAW is the only state-based multi-source surveillance system for WR injuries to teens <18 Lack of an appropriate external comparison Considered ‘Macro’ comparison with Bureau of Labor Statistics’ annual Survey of Occupational Injuries and Illnesses (SOII) estimates for MA (13) SOII often used as official est. of occupational safety and health conditions nationwide and for some states (14) Comparison has limitations (14) Question 2: How well does TAW capture non-fatal WR injuries to teens <18 in MA?

16. 2. How well does TAW capture non-fatal WR injuries to teens under 18 in MA? Data systems comparison for non-fatal injuries to teens <18 16 CharacteristicsMA SOII (published) TAW Data source Cases reported from employer OSHA log Multiple sources (WC, ED) Data type Survey estimatesCounts Industry PrivatePublic & Private Injury type 1+ more loss work day(s) with or without job transfer or restriction Medically treated & WC claims for 5+ loss work days (14)

17. 17 2. How well does TAW capture non-fatal WR injuries to teens under 18 in MA? Methods: 2005-2008 TAW and SOII, 16-17-year olds Developed SAS program to identify TAW overlaps Accounted for Est. illnesses (<3.27%) in SOII Overlaps (8.44%) and est. non-WR ED cases (1.33%) in TAW Calculate incidence ratio II. ‘Macro’ Sensitivity 17

18. 18 Data sourcesType 2005 2006 2007 2008 Total TAW Medically treated & WC claims for 5+ loss work days, all industries 855 860 766 566 3046 EDD Medically treated with WC as expected payer 7317626784932663 WCWC claims of 5+ loss work days.201174162131668 OverlapsIdentical cases in EDD & WC 77 76 74 58 285 BLS SOII Employer-reported injuries of 1+ loss work days, private industries 470 360 440 450 1770 Incidence ratio (TAW/SOII) 1.822.391.741.261.72 2. How well does TAW capture non-fatal WR injuries to teens under 18 in MA? 18 Results: Comparison of TAW & SOII's non-fatal WR injuries in MA, 2005-2008

19. Conclusions and Recommendations TAW captured an average of 1.7x more cases than SOII 19 TAW should continue to use multiple data sources Overlaps should be accounted for TAW can use SOII as a crude comparison for capture rate Yearly evaluation, expect TAW cases > SOII est. Public industries available for SOII data 2008+ 19 Question 2: How well does TAW capture non-fatal WR injuries to teens <18 in MA?

20. ‘Evaluating’ the TAW Evaluation 20 1. Representativeness of sample ED cases Limitations 1. PUMAs based on case residence 2. Unable to get ACS est. for FTEs and workers <16 Usefulness Informed sentinel surveillance Identified regions to target outreach and broad-based interventions 2. ‘Macro’ sensitivity Limitations : Unavailable 1. 2009 MA SOII 2. Standard errors 3. Cases <16 years old Usefulness Informed population surveillance Developed protocols to manage multiple data sources and estimate total cases 20

21. 21 Usefulness of TAW Surveillance System Declining rates of WR injuries to 15-17-year olds in MA WC claim data, 1994-2008: -5.5% annual change 1993: TAW starts 1994: OSHA educ. materials dev. through comm. project

22. Thank you! 22 Acknowledgements: Beatriz P. Vautin, MPH & Sara Rattigan, MS Massachusetts Teens at Work: Injury Surveillance & Prevention Project 617-624-5677 | Teens.atwork@state.ma.us www.mass.gov/dph/teensatwork Contact: MyDzung Chu, MSPH MyDzung.chu@state.ma.us 22

23. References 23 (1) National Institute for Occupational Safety and Health. 2013. Health and Safety of Young Workers. Proceedings of a U.S. and Canadian Series of Symposia. Accessed on 05/15/2013 at http://www.cdc.gov/niosh/docs/2013-144/pdfs/2013-144.pdfhttp://www.cdc.gov/niosh/docs/2013-144/pdfs/2013-144.pdf (2) Massachusetts Department of Public Health. 2011. Massachusetts Guide for Working Teens. Accessed on 05/10/2013 at http://www.mass.gov/eohhs/docs/dph/occupational-health/ma-guide-working-teens.pdf http://www.mass.gov/eohhs/docs/dph/occupational-health/ma-guide-working-teens.pdf (3) Occupational Health Surveillance Program. Massachusetts Department of Public Health. http://www.mass.gov/eohhs/gov/departments/dph/programs/health-stats/ohsp/ (4) Protecting young workers: A Guide for Building a State Surveillance System for Work-Related Injuries to Youths. MDPH with Education Development Center, Inc. Spring 2005. Accessed at: http://www.mass.gov/eohhs/docs/dph/occupational-health/how-to-guide.pdfhttp://www.mass.gov/eohhs/docs/dph/occupational-health/how-to-guide.pdf (5) Morbidity and Mortality Weekly Report (MMWR). Updating Guidelines for Evaluating Public Health Surveillance Systems: Recommendations from the Guidelines Working Group. Centers for Disease Control and Prevention. US Department of Health and Human Services 2001. Vol. 50 / No. RR-13: 1-36. (6) Office of Management and Budget, North American Industry Classification System, United States 1997 Manual available at http://www.census.goc/naics http://www.census.goc/naics (7) Bureau of Census 1990 Occupational Classification System Manual available at http://www.census.gov/hhes/www/ioindex/ioindex.htmlhttp://www.census.gov/hhes/www/ioindex/ioindex.html (8) Bureau of Labor Statistics Occupational Injury and Illness Classification www.bls.gov/iif/oshtc.htmwww.bls.gov/iif/oshtc.htm (9) ESRI. ArcGIS software version 7. http://www.esri.com/http://www.esri.com/ (10) Missouri Census Data Center, Missouri State Library, Missouri Secretary of State. All About PUMAs (Public Use Microdata Areas). Site last modified: 11/13/2010. Accessed at: http://mcdc2.missouri.edu/pub/allabout/geo_pumas.shtmlMissouri Census Data CenterMissouri State LibraryMissouri Secretary of Statehttp://mcdc2.missouri.edu/pub/allabout/geo_pumas.shtml (11) United States Census Bureau. American Community Survey, 2005-2009. http://www.census.gov/acs/www/ (12) U.S. Census Bureau DataFerrett web-based application http://thedataweb.rm.census.gov/TheDataWeb/launchDFA.htmlhttp://thedataweb.rm.census.gov/TheDataWeb/launchDFA.html (13) Groenewold MR, Baron SL. The Proportion of Work-Related Emergency Department Visits Not Expected to Be Paid by Workers' Compensation: Implications for Occupational Health Surveillance, Research, Policy, and Health Equity. Health Serv Res. 2013Groenewold MRBaron SLHealth Serv Res. (14) U.S. Bureau of Labor. Occupational Injuries/Illnesses and Fatal Injuries Profiles. Accessed at: http://data.bls.gov/gqt/InitialPage.http://data.bls.gov/gqt/InitialPage 23

24. Extra slides 24

25. Results: i. Are the sample ED cases representative of state-wide ED cases based on demographic, employment, and injury characteristics? 25 EDD (N=377) Sample ED (n=77) Representiv eness of sample EDs* Chi-square goodness of fit n%n% ratio p-value Age0.15 14 and under71.8633.92.10 15133.4556.491.88 1610126.792431.171.16 1725667.94558.440.86 Gender0.54 F16844.563748.051.08 M20955.444051.950.94 Industry0.58 Accomm. & Food Serv.10728.382127.270.96 Arts, Ent., & Rec.184.7756.491.36 Health Care & Social Assist.3910.3467.790.75 Retail Trade4511.9479.090.76 Other4211.141316.881.52 Missing/NA12633.422532.470.97 25

26. Results: i. Are the sample ED cases representative of state-wide ED cases based on demographic, employment, and injury characteristics? 26 EDD (N=377) Sample ED (n=77) Representi veness of sample EDs* Chi-square goodness of fit n%n% ratio p-value Occupation<0.0001 Service Occupations, Except Private Household5815.382532.472.11 Other205.31911.692.20 Missing/NA29979.314355.840.70 Injury0.17 Burns4211.141114.31.28 Open wounds17145.362735.10.77 Sprains strains tears4712.471316.91.35 Surface wounds5815.38911.70.76 Other5915.651722.11.41 26

27. ii. What regions of the state should TAW target to a) increase geographic representativeness of sample EDs and b) address regions with highest injury rates? Results: PUMAs with top ten cumulative WR injury rates to 16-17-year- old workers in the statewide EDs, 2005-2009 27 RankPUMA Rate/1000 workers 95% CI PUMA represented by sample ED? EDD cases ACS est. workers Average--- 12.2 9.2-15.2--- 64 5245 14400 25.6 21.7-29.5Y 166 6487 2300 21.317.5-25.0 N 124 5831 31700 20.715.8-25.5 N 69 3338 44600 19.716.4-23.0 Y 136 6897 5100 19.615.8-23.5 N 101 5140 64700 17.413.3-21.5 Y 69 3965 71900 16.612.7-20.5 N 69 4158 81800 16.113.0-19.3 N 102 6321 94300 15.812.3-19.3 N 78 4938 101100 15.411.7-19.0 N 69 4490 27