1. Evaluating Immunization Coverage in New York City by Geographic Area
María Cecilia Mosquera, MD, MPH; Vikki Papadouka PhD, MPH,
Michael Hansen, MPH; Christopher M. Zimmerman, MD, MPH;
Jane R. Zucker, MD, MSc; Karen Fernandez, MBA
Hello. My name is Maria Cecilia Mosquera; I am a Resident in Public Health and Preventive Medicine in the Bureau of Public Health Training in the New York City Department of Health and Mental Hygiene. Over the past year and a half, I have had the opportunity to work with the Bureau of Immunization on a project. This project was aimed at evaluating immunization coverage in NYC by geographic area.
Today, I will be presenting to you a summary of my project to date, presenting particularly the results of our most recent data analysis.
This presentation will cover the: Objectives of the project. Background on datasets used. Methods and Results obtained. Then, we’ll go through summary/conclusions and go through limitations and next steps.
Resident Project with Bureau of Immunization
New York City Department of Health and Mental Hygiene (DOHMH)
44th NIC, April 21, 2010

2. New York City (NYC) In 2008: 5 Boroughs (counties)
Population of >8.3 million
>2 million children 19 yrs
>127,000 live births
5 Boroughs (counties)
To place this evaluation in context … let me tell you a little about New York City (NYC). As many of you may know, NYC is a large urban area. You may not know, however, how large the City actually is. In 2008, more than 8 million people lived in NYC (Yr 2008 Population of 8,363,710). Approximately 2 million of these were children <=19 years of age (Yr ,144,445 children 19 yrs). And that year the City had more than 127,00 births (Yr ,680 live births). NYC’s population is spread throughout 5 boroughs, which are equivalent to what some areas call counties. They are: the Bronx, Manhattan, Queens, Brooklyn, and Staten Island.
REFERENCES:
2008 data from-- Summary of Vital Statistics 2008: The City of New York at:
Map of NYC Boroughs from DOHMH GIS Center.

3. (where DTP = Diphtheria, Tetanus, and Pertussis)
National Immunization Survey (NIS) – 2008 Coverage for NYC mo olds
Now … in this context … when we look at immunization in New York City children … one way to evaluate immunization coverage is to look at data from the National Immunization Survey (NIS), which many of you may be very familiar with.
NIS allows us to look at data for NYC children months of age, specifically looking at immunization coverage for the 4:3:1:3:3:1 series. 4:3:1:3:3:1 refers to  4 DTP/DT, 3 Polio, 1 Measles, 3 Hib, 3 Hep B, 1 Varicella (where DTP = Diphtheria, Tetanus, and Pertussis)
In this graph, we see NIS 2008 national data in yellow and NYC data in blue. The Y-axis shows percent coverage. The X-axis shows the components of 4:3:1:3:3:1. series You can see 4:3:1:3:3:1 on the right hand column and the individual vaccines in the columns towards the left. NYC 4:3:1:3:3:1 coverage in 2008 was 75.4%, similar to national coverage.
But, in fact if I were to show you a graph of the NYC trends for 4:3:1:3:3:1 coverage in recent years you would see that coverage had shown no significant gains. This stimulated the NYC DOHMH Bureau of Immunization to look at ways to evaluate immunization coverage. So, we considered DTP4 coverage. Because, if you look only at coverage with the 4th DTP, we see something that has been well described (REFERENCE: ). Of all the vaccines within the 4:3:1:3:3:1 series, coverage for 4th DTP is the lowest, the only one below 90%. This is true both nationally and in New York City.
Because of this [next slide ]…
REFERENCES:
4:3:1:3:3:1 refers to
4 or more doses of DTP/DT,
3 or more doses of poliovirus vaccine,
1 or more doses of MCV (1 or more doses of measles-containing vaccine, including MMR)
3 or more doses of Hib. 3 or more doses of Hep B. 1 or more doses of varicella.
4+DTP refers to 4 or more doses of diphtheria and tetanus toxoids and pertussis vaccine, or diphtheria and tetanus toxoids (DTP/DT/DTaP). 3+polio refers to 3 or more doses of poliovirus vaccine.
1+MCV refers to 1 or more doses of measles-containing vaccine, including MMR.
1+MMR refers to 1 or more doses of a measles-mumps- rubella vaccine.
3+Hib refers to 3 or more doses of Haemophilus influenzae type b (Hib) vaccine.
3+HepB refers to 3 or more doses of hepatitis B vaccine.
1+Var refers to 1 or more doses of varicella vaccine received at or after age 12 months, unadjusted for history of varicella illness.
REFERENCE: (From NIS)
Graph made from NIS data available at:
NIS Coverage for New York City (Estimated Vaccination Coverage* with Individual Vaccines and Selected Vaccination Series Among Children Months of Age by State and Local Area US, National Immunization Survey, Q1/2008-Q4/2008), see:
FURTHER INFORMATION:
In the last quarter for CIR’s Quarterly Reports, DTP4 was 71%. It has been consistently lower than the coverage for all other vaccines in the 4:3:1:3:3:1 series.
4:3:1:3:3:1 refers to  4 DTP/DT, 3 Polio, 1 Measles, 3 Hib, 3 Hep B, 1 Varicella
(where DTP = Diphtheria, Tetanus, and Pertussis)
Data from: National Immunization Survey 2008, Table Data, Coverage with Individual Vaccines and Vaccination Series, by State and Local area

4. Objectives and Goal Objectives: Goal:
To evaluate geographic differences in DTP4 immunization coverage in NYC
To identify areas in NYC which have the most under-immunized children
Goal:
To inform programmatic development in the Bureau of Immunization (BOI) aimed at targeting areas of low immunization coverage
Because of this …. the objectives of this project were to evaluate….
This was done in order to inform …

5. Data Sources Data Sources
NYC DOHMH: Citywide Immunization Registry (CIR)
NYC Department of Education: Automate the Schools (ATS)
US Census 2000
Data sources utilized for this project include … . I will give you further detail about the CIR and ATS in the next 2 slides.

6. Background - CIR
NYC population-based Immunization Information System, implemented in 1997
Mandate to report immunizations given to children <19 yrs
Before 2005, mandate covered children <8 yrs
Completeness dependent upon quality of reporting
Contains:
~2,000 child & adolescent immunizing facilities
~90% of facilities have reported in past 6 months
~4.3 million individuals and ~47 million immunizations
Timely = ~85-90% of immunizations within CIR have been reported within 1 month of administration
Birth certificate data loaded weekly
The New York City Department of Health’s Citywide Immunization Registry (also known as the CIR) is a …
FURTHER INFORMATION:
Further information (from the CIR webpage at:
Reporting:
The City Health code section and 11.07(d) requires that all physicians, nurse practitioners, and physician assistants who order the administration of an immunization for any individual age 18 years old and younger in New York City must report the immunizations administered to the Registry within 14 days of administration. Providers are required to report:
All immunizations administered from birth through 18 years of age
Past immunizations given by other health care providers that have not already been reported. There must be proper documentation of past immunizations, including the specific vaccines given and the dates they were administered.
Immunizations administered to persons 19 years of age or over may be reported to the CIR with the individual’s consent documented in the medical record.

7. Background - ATS
NYC public school-based administrative system into which immunization data are collected for the ~1.1 million NYC public school children
Does not include the ~300,000 NYC children in private schools
Immunization records required of all students
Data collected and entered into ATS by public schools
Accountability = Goal for schools in that 98.5% of children have all vaccines required to attend school
Data updated by schools throughout the year to reach goal
ATS is a …
FURTHER INFORMATION:
Further information about ATS at DOE’s website:
NYC DOE Immunization Info website:
See new student immunization requirements at:
See letter to principals with immunization compliance goal of 98.5% for the school year at:

8. Methods DTP4 2 year olds CIR – Children who were 2 years as of 11/09
ATS – Children entering kindergarten in Fall 2009, we looked at their DTP4 status at their 2nd birthday (retrospective analysis)
So, I’ve given you some background on the data sources used … Now, we will discuss the methods used for this particular project. For coverage, looked at children having the 4th DTP. For age, we looked at 2 year olds. In the CIR, we looked at children who were 2 years as of 11/30/09. In ATS, we looked at data collected from children entering kindergarten in Fall 2009, and looked at their DTP4 status at their 2nd birthday through a retrospective analysis.
FURTHER INFORMATION:
Used most recent data available
Age of Cohorts:
2 year olds:
CIR: DOB=12/1/06-11/30/07 (24-35 month olds)
ATS: DOB=1/1/04-12/31/04 (at 2 years old)
Retrospective evaluation using kindergarten school entry data for academic year
“Coverage” = Up to date with DTP4

9. Mapping Mapped by zip code using ArcGIS®
DTP4 Coverage by zip code of 2 y/o using:
CIR
ATS
Population = 2009 CIR birth cohort
Number of children without a 4th DTP
= % Missing DTP4 x Number of children
This was a project looking at coverage by geographic area, so we mapped data using software called ArcGIS. All data were mapped by zip code. We created 3 different sets of maps:
For DTP4 Coverage, we mapped data from both the CIR and ATS by zip code of 2 year old.
For population, we mapped population distribution of the CIR birth cohort of 2009.
Lastly, we mapped the number of children without a 4th DTP. This number was obtained by multiplying the number of children in the CIR 2009 birth population by % of children in either CIR or ATS who were missing DTP4. So, this was a product of the first two parts of our analysis: coverage (from where we obtained the % missing DTP4) and the birth population. We chose to do this in order to take into consideration that if for example, a zip code has 2000 children missing DTP4 that may have more importance for us when considering programmatic interventions than a zip code that has 50 children missing DTP4.
FURTHER INFORMATION:
Other NYC data have been mapped in the past. Many other health indicators have been evaluated geographically, such as poverty and hospitalizations for asthma in children. For this project, we chose to map immunization coverage. Reference: Health Disparities in New York City, DOHMH, Accessed from:
Software:
Used BiQuery, WebUTD program, Stata, EpiInfo, and ArcGIS.

10. Results
Now, let me show you the results …

11. DTP4 Coverage
We first looked at DTP4 coverage in both CIR and ATS by zip code. On this slide, you see CIR data on your left and ATS data on your right. On both maps, the darker shading represents zip codes with the lowest DTP4 coverage. On both maps, the 10 zip codes with the lowest coverage are highlighted in red.
Keep in mind that ATS and CIR maps are to be compared “relatively” because of the differences in the data sets.
Taking that into consideration, you can see relative differences in the two maps. Those zip codes with lowest coverage appear to be grouped in Manhattan and Queens for the CIR and mostly in Bronx, Brooklyn, and Queens for ATS. In fact, the top 10 zip codes with lowest coverage don’t overlap.
As I’ve mentioned before, these two data sources are different which may explain the difference in these maps.
CIR
ATS

12. DTP4 Coverage - Summary
ATS and CIR - absolute differences exist in DTP4 Coverage
CIR: overall= 57.7%, range= 12.5 – 71.6%
ATS: overall= 74.7%, range= %
ATS and CIR - relative differences seen on maps
Likely due to differences in data sets
CIR limited by reporting and moving to/from NYC
ATS limited by population (only public schools)
So, let me summarize the previous slide for you …
Because of the relative differences seen in the DTP4 coverage maps, we decided to consider the population of children within each zip code. [next slide]
FURTHER INFORMATION:
CIR with DOB 12/1/06 to 11/30/07 (newer data) Mapped coverage was 12.5 – 71.61% with overall of 57.7%
ATS (newer data) Mapped coverage was % with overall of 74.7%
Both of the above exclude zips with less than 30 kids each and one unmappable zip using the base layer used for this mapping (zip 10069).
Overall coverage is taken by: kidswith4DTP/n.
For example: in CIR with DOB 12/1/06 to 11/30/07 (newer data), it was divided by is
For older data:
ATS Mapped coverage was 50%-92.2%

13. Distribution of Children
So, on this slide you will see our map of the population distribution of the CIR 2009 birth cohort, available because the CIR is populated with birth certificate data on a frequent basis. Note that the places in NYC with the most children born in 2009 are in the Bronx and Brooklyn and Queens.
In actuality we mapped population distribution using: CIR, ATS, and the US Census. When we compared this map to the maps of ATS and US Census data, they looked very similar. In fact, the pattern of distribution in Census 2000 of 2 year olds and of CIR Children born in 2009 was almost identical. So, realizing that US Census data is from 2000 and now 10 years old and because Vital Records births are continuously entered into the CIR, we chose to focus on the population distribution of children born in 2009.
2009 Birth Population
Source=CIR

14. ATS – Zip Codes with most children without a DTP4
Represent ~15.1% of NYC’s under-immunized children
Neighborhood:
Bronx:
10456 – High Bridge - Morrisania
10457 – Crotona – Tremont
10458 – Fordham - Bronx Park
10467 – Fordham – Bronx Park
Brooklyn:
11207 – East New York
11208 – East New York
11211 – Greenpoint
11219 – Borough Park
11220 – Sunset Park
11226 – East Flatbush – Flatbush
So now that we have looked at both coverage and population, we decided to look at a map that would combine both coverage and population distribution in order to see where the most number of children were with lowest coverage. For this, we put together the birth population information I just showed you along with the coverage information in the maps I just showed you.
The ATS map you see on this slide represents the number of children without a 4th DTP by zip code. So we’re looking at the number of children missing DTP4.
Darker shading represents those zip codes with the most number of children without a 4th DTP. You can see the 10 zip codes with the most number of these children highlighted in red. Listed on the right are United Hospital Fund neighborhoods that correspond to the top ten zip codes with lower coverage. You can see a group in the Bronx, another group in Brooklyn. You’ll remember that this looks very similar to the population map, the first map I showed you. So, it seems that population differences are significant to enough to take into account in NYC when looking at DTP4 coverage.
REFERENCE FOR UHF neighborhoods : DOHMH GIS Center Zip Code List
INFORMATION ABOUT OLDER ATS DATA (NOT THIS DATA):
IN ATS , adding up the number of children calculated by the weighting in all zip codes mappable within NYC who were not up-to-date, I get children. Adding up the number of children calculated by the weighting in the ten zip codes with the largest number of under-immunized children, I get: If it is correct to do the following: / *100, I get= 15.39%. See spreadsheet at: J:\Immunization Data Eval Project\GIS Data\DTP maps 3-09 titled: Copy of zips of vitals of children born in2008MERGEDwithD4coverage_ATS0708CALCULATIONS.xls.
BUT, NOTE: “The U.S. Post Service (USPS) subdivided the zip code, effective July 1, 2007, and created two new zip codes, and ” See: Only zip code is in 2006B ArcGIS baselayer (so zips and weren’t mappable with that base layer). Note that the total number of weighted children in zip plus 10075= So, if subtract this number from the denominator of the following formula (described above) / *100, you get / *100 which gives you 15.49%, slightly different from the 15.39% you get if you include these 2 zips in the denominator.
The range of ATS DTP4 coverage of % excludes:
-- those zips which had a CIR vital records children but weren’t in ATS (41 zips)
-- those zips which had ATS coverage information but weren’t in CIR vital records (12 zips)
-- zips and (only one that matters is which had percent covered of 44.3%). If included, this would affect lower limit of range.
Upper limit of range would not be affected as all the numbers above 92.2% are in those zips which had data in ATS but not in CIR.
INFORMATION ABOUT OLDER CIR DATA (NOT THIS DATA):
CIR DTP4 COVERAGE = / *100 = represents % of NYC’s mappable under-immunized children. This includes zips and 10075, which had weighted counts of = If you exclude these two zips, it gives you= % of NYC’s mappable under-immunized children.
CIR COVERAGE RANGE (UTDPERCENT) = 0-100% (includes zips that have 1 child each. If exclude zips with only one child, lower limit of range is 17.57% and upper limit of range is 87.69%).
IF LOOK AT CIR MISSING DTP4, range is also = 0-100% (includes zips that have 1 child each. If exclude zips with only one child, lower limit of range is 12.3% and upper limit of range is 82.43%).

15. CIR – Advantages/Limitations
More timely
Populated with Vital Records data
Population-based
Incomplete
Reporting-based
Address data not universally available
High coverage indicates better reporting and not necessarily better coverage
Now, after seeing these maps, we are fortunate to have two datasets. But, it is important to consider the limitations and advantages of each one. CIR is …

16. ATS – Advantages /Limitations
More complete
Accountability leads to higher reporting
Less Timely - Older
Based on public schools only
ATS …
FURTHER INFORMATION:
ATS children are older, but we make an assumption that patterns of immunization coverage do not change dramatically in 2-3 years.

17. Conclusions Could look only at coverage
But, goal was to inform programmatic targeting of low coverage
Useful to consider areas in NYC having low coverage that also have the most children
So, to conclude … we could have looked only at DTP4 coverage. But, our goal was to inform programmatic development aimed at targeting low immunization coverage. So, considering areas in NYC that have low coverage but also have the most children may be useful in programmatic interventions to get the biggest “bang for your buck,” particularly when considering the difficult economic times. So, we took the results of this project and added them to the many other things going on in the Bureau [next slide] …

18. Action Steps
Target providers in zip codes identified with lowest coverage
Identify providers with lowest coverage in CIR
Generate recall lists using CIR
Use other data sources to learn more about NYC areas of lowest immunization coverage to:
Target further evaluation
Target messaging/outreach
Unite data sets
Project under way for CIR to become the primary immunization information system for NYC schools
Taking all of this into account …. this information was used to …. In addition, BOI ….
REFERENCE:
Borough Park = 5,126 live births in See: Summary of Vital Statistics 2008: The City of New York at: Data from: Summary of Vital Statistics 2008-The City of New York
FURTHER INFORMATION:
Identified provider facilities within top 10 zip codes
96 facilities (41/96 with  100 patients)
Cross-referenced list with AFIX list of facilities visited and then examined AFIX coverage
Identified facilities needing an AFIX visit
8 of 96 facilities (8.3%) – all visited since
Keeping in mind the limitations and advantages mentioned … after identifying which zip codes were those 10 ten with lowest coverage, then BOI …
AFIX is a quality improvement strategy to raise immunization coverage levels and improve standards of practices at the provider level. The acronym for this four-part dynamic strategy stands for Assessment, Feedback, Incentives, and Exchange (AFIX). (See:
On an from Mike Hansen from 4/8/09, it says “Of the 41 facilities that have at least 100 patients”
On an from Karen Fernandez from 4/10/09, it says: “I was able to review the data and match it to our database.  Out of the original 96 sites with > 30 patients, we conducted AFIX visits to 88 sites (not 65 as indicated during the meeting).  Of the difference, (88-65=23), 15 sites were assessed prior to 2008; 3 sites were assessed in 2008 (but not entered in the PP; I will f/u with the team leaders so they can be entered); and 5 sites were assessed recently in 2009 (I will also make sure that these are entered in the PP ASAP).  Only 8 sites of the list with 96 sites presented during the meeting were not assessed. 
Morris Heights Health Center was in question but I was able to verify that it was never assessed.  This site belongs to a network of 11 sites (most of them being schools).  The first site enrolled in VFC in 2005; the most recent enrollment from this network was in 2009.  We will schedule the network for AFIX site visits ASAP.”
Now that we have mapped this information and identified these zip codes with lowest DTP4 coverage, we can utilize information already compiled by NYC DOHMH about NYC neighborhoods in order to learn more about them to target … .
In addition, because both CIR and ATS have both advantages and limitations, we would only benefit by being able to combine both the CIR and ATS datasets. Therefore, there is actually a project underway between NYC DOHMH and DOE to plan for the CIR to take over the role of ATS in immunization..
See Borough Park Community Health Profile at:

19. Acknowledgments Vikki Papadouka, PhD MPH
Christopher M. Zimmerman, MD MPH
Michael A. Hansen, MPH
Jane R. Zucker, MD MSc
Karen Fernandez, MBA
DOHMH GIS Center:
Susan Resnick
Chris Goranson
School Health/DOE:
Roger Platt, MD
Hiram Szeto
This project would not have been possible without … Thank you for allowing me to present this project.