1. South Dakota Tribal PRAMS: Using Alternative Methods to Reduce Barriers to PRAMS Participation
CDC PRAMS National Meeting
Atlanta, GA
December 9, 2008
Thank you for the opportunity to discuss the alternative methods we implemented to reduce barriers to PRAMS participation.
We hope that our approaches prompt discussion among partners here.
Acknowledge JI and SW, who is in China.
Christine Rinki, MPH; Jennifer Irving, MPH; Ssu Weng, MD, MPH

2. Statements of Need
SDAI communities experience persistent and dramatic disparities in infant mortality, post-neonatal mortality.
Tribes do not have timely access to accurate, population-based maternal / infant health data.
No statewide maternal / infant AI data to supplement vital statistics.
SD Tribal PRAMS has been undertaken to address three critical needs: (read slide).
These challenges are not unique to SD tribes.

3. Low AI PRAMS response rates, 2000-2002
Average response rate AI 63% vs. White 82%
AK, OK, WA have achieved 70% minimum
MN, MT, NE, NM, ND, OR, UT have not reached 70%
PRAMS data have not fully benefited tribes
or AI communities.
Response rates for AI have been historically low.
CDC analysis showed states with high AI pop do not meet 70% minimum and tribes do not benefit.
Low response = low contact, not high refusal.
Developed alternative methods rooted in CBPR to address concerns, challenges, opportunities.
Other states don’t tune out, I believe that what we found can be translated to other rural populations, and even poor urban populations with poor access to postal and landline services.
Kim SY, Tucker M, Danielson M, Johnson CH, Snesrud P, Shulman H. (2008). How can PRAMS Survey Response Rates be Improved Among American Indian Mothers? Data from 10 States. Matern Child Health J, 12(Supp 1):

4. South Dakota Tribal PRAMS: A Statewide, American Indian, Point-in-Time Project
Statewide project focusing on mothers of AI infants, point-in-time, entering our final six months of CDC funding.
Yankton Sioux tribe is the lead.
NPTEC MCH EPI dept was contracted to implement operations.

5. SD Tribal PRAMS Collaboration
Standing Rock Sioux Tribe
Cheyenne River Sioux Tribe
Oglala Sioux Tribe
Rosebud Sioux Tribe
Lower Brule Sioux Tribe
Crow Creek Sioux Tribe
Flandreau Santee Tribe
Sisseton-Wahpeton Oyate
Aberdeen Area Indian Health Service
Aberdeen, SD
South Dakota DOH
Pierre, SD
Northern Plains Tribal
Epidemiology Center
Rapid City, SD
Yankton Sioux Tribe
Reservation land
Other key entities (approximation)
Sioux Falls, SD
North Dakota DOH
Bismarck, ND
Vital Records
Vital Records, Epi, WIC
This map shows the location of participating entities, including nine tribes.
The grant recipient is the Yankton Sioux Tribe.
NPTEC manages project operations.
SD and ND departments of health provide access to vital records and epi support
Of course, tribes retain authority to authorize implementation of PRAMS through tribal approval processes. Resolutions for PRAMS have been obtained from all SD tribes. Tribes also participate in a Tribal Oversight Committee.
Project Management
Grant Recipient
380 miles

6. Tribal Oversight Committee & Steering Committee
TOC: Decision making body
Representation from all 9 SD Tribes
SC: Provided guidance, expertise
SD VR and Epi
IHS, Urban Indian Health
Northern Plains Healthy Start
MCH Programs (Tribal and State)
Oversight is essential to any tribal project.
TOC decides policies and protocol
SC provides guidance and input.

7. Sample: Meeting Tribal Needs
Unique Need
Protocol Modification
Tribe-specific and statewide reports 
Statewide census vs. sample
Allows flexibility for small group analysis
Includes reservation, off-res, urban
All AI infants must be included
Define AI by maternal / paternal race on BC
Border reservation deliveries in neighboring states
Include NE, ND occurrence births to SD residents
One tribe has land in
SD and ND
Develop NDVR agreement to sample 1 ND county
Tribal input was central to developing the sampling scheme for this project.
A key message that I heard is that while state-level reporting serves a purpose, tribes are concerned about what’s happening within their nation and jurisdiction. Tribes want tribe-specific reports of data.
In order to address the need _________________ SDT PRAMS has _________________________.

8. The Data Collection Challenge
Challenges
Long distances from home to post office
Dirt roads, no gas money, no vehicle
Poor telephone coverage, cell phones
Highly mobile, circular migration to cities
Suspicion of data collection activities
No access to state databases
Opportunities
Dense social and familial networks
High level of social program participation
Healthy Start is a trusted program with strong community contacts and knowledge
Early and ongoing consultation with tribes and stakeholders.
Highlighted challenges:
Identified strengths that could facilitate contact with mobile or isolated women.
Developed adaptations:
-successful and within budget
-potential for replication by other PRAMS state
-rejected face-to-face interviewing.
 Our tribes wanted to help other tribes.

9. Mailing Operations: Adapting to Community Context
Standard Mail
SD Tribal Mail
Preletter: postal mail 
Mail 1: postal
M1: postal
Tickler: postal
(NA)
Address verification to TFS
Mail 2: postal
M2: postal (all non-responders)
M2: with WIC (WIC participants only)
Mail 3: postal (optional)
M3: postal (non-reservation residents)
M3: hand delivered or hand pick-up
(reservation residents)
After I provide this overview of mailing operations, then describe our modifications in more detail. Results will follow.
Standard protocol mail data collection steps down the left side of the slide, with corresponding Tribal mail data collection steps down the right. Changes are noted in red.
Pre, m1, tickler—same as standard, on the advised timeline
After tickler, send a worksheet to tribal field staff to verify our mail, physical, and phone contact info, this is not part of the standard protocol.
At m2, all non responders received postal survey. Moms enrolled in wic also received a questionnaire with their wic check.
We chose m3, and moms living off res receive standard m3 questionnaire mailing. Moms living on res received a hand delivered questionnaire. We found that many had a completed questionnaire at home. In those cases, TFS just picked up the completed previous q.
This slide compares standard protocol to SD Tribal protocol for mailing operations.
Only adapted protocol for the mail mode.
While I think our orientation to data collection marks a dramatic shift towards community engagement compared to the standard, I would argue that the substantive modifications to the methods are minor.

10. Mailing Operations: Adapting to Community Context
Standard Mail
SD Tribal Mail
Preletter: postal mail 
Mail 1: postal
M1: postal
Tickler: postal NA
Address verification to TFS
Mail 2: postal
M2: postal (all non-responders)
M2: with WIC (WIC participants only)
Mail 3: postal (optional)
M3: postal (non-reservation residents)
M3: hand delivered or hand pick-up
(reservation residents)
Now will describe our Mail 2 WIC modifications in more detail.
Tribal PRAMS developed the WIC partnership because 75% of our sample receives WIC during pregnancy, and women continue contact with WIC despite frequent moves.

11. SD WIC Partnership WIC enrollment on BC
Confirmed enrollment status and location with SDDOH WIC
Mailed out questionnaires to WIC offices
Questionnaires delivered at appointments by WIC clinical staff
Bi-monthly appointments = contact lag time
Telephone info collected by WIC staff
Return telephone info and tracking data
So what did this partnership look like in practice?
WIC a state-specific variable.
Entered telephone info into database external to PT.

12. Mailing Operations: Adapting to Community Context
Standard Mail
SD Tribal Mail
Preletter: postal mail 
Mail 1: postal
M1: postal
Tickler: postal NA
Address verification to TFS
Mail 2: postal
M2: postal (all non-responders)
M2: with WIC (WIC participants only)
Mail 3: postal (optional)
M3: postal (non-reservation residents)
M3: hand delivered or hand pick-up
(reservation residents)
So now let’s talk about the mail 3 modifications.

13. Tribal Field Staff Partnership Activities 8 hour training
Northern Plains Healthy Start
Tribal Health Administration
Activities
Promote PRAMS on their reservations
Verify address and phone information
Hand deliver & pick up questionnaires
8 hour training
CDC PRAMS Human Subjects Protection
Interactive: role play, brainstorming
Tribal Field Staff Protocol & Manual
Cornerstone of this effort was partnership with Tribal Field Staff. I cannot say enough about the work that these women do.
Provided $$ to NPHS for staff time, large reservations .5 FTE, small .2 FTE.
Not only data collection activities, but strengthening connection with tribes.
Manual as part of our exhibits.

14. Hand Delivery Process
Reservation residence determined by mother’s county of residence on BC
Contact verification worksheets completed by field staff, entered into PRAMTrac
Questionnaires and tracking documentation mailed to field staff
3 delivery attempts, scripted protocol to protect confidentiality
Pick up of completed questionnaires
Additional contact verification
Returned tracking data to PRAMS office
County is imported to pt. Q

15. Additional Activities
Use of Lakota / Dakota language and concepts in promotional and questionnaire materials
Incentives / rewards
30 minute phone card
CD of Lakota / Dakota Honor Songs
$100 monthly drawing
$10 cash reward (not CDC funds)
Extensive promotional plan not fully implemented
Additionally, we tailored project materials to demonstrate our commitment to AI families across SD.

16. Results

17. SDT PRAMS Results Number sampled Respondents Response rate Overall
1300
948
72.9%
Maternal Education
0-11 years
468
324
69.2%
12 years
390
285
73.1%
> 12 years
429
331
77.2%
Age
< 20
303
217
71.6%
20-29
777
561
72.2%
30+
218
170
78.0%
Parity
No previous live births
418
313
74.9%
1+ previous live births
882
635
72.0%
In this first slide we see the overall demographics of our sample and respondents, as well as response rates for each group. As a reminder, we did a census of eligible women.
Overall response rate
Response strong across all groups
Our sample is fairly young, with a large group who have low educational attainment.
As expected, older and more educated women have higher response rates.

18. SDT PRAMS Results (con’t)
Number sampled
Respondents
Response rate
Maternal Race / Ethnicity
White Non-Hispanic
118 76
64.4%
Hispanic 55 30
54.4%
American Indian
1020
764
74.9%
Other
106 77
72.6%
As you recall, we included mothers of AI infants, some mothers are not AI. We saw a lower response among this group.
Complicated identity, cultural and political factors at play, but also all materials strongly resonant with Lakota and Dakota communities.

19. Overall response by mode
Data Collection Step
Number Included
Number of
Respondents
Response Rate
Mail 1
1278
529
40.7%
Mail 2—Postal
845
136
10.5%
Mail 2—WIC
443 46
3.5%
Mail 3—Postal
269 34
2.6%
Mail 3—Hand
412 65
5.0%
Other Mailing NA 3
0.2%
Phone Phase
627
135
10.4%
Total
1299
948
72.9%
For each data collection step down the left of the slide, we see the number who were sent or delivered that mailing, the number who responded in that mode, and the percentage of the overall sample who completed the survey in that mode. So for mail 1, 529 responded, which is 40.7% of For phone phase, this was the number that were forwarded into phone phase.
In all of our operations analyses, our total is 1299 because 1 woman died during data collection and we removed her from these analyses, though she remained in the sample.
You can see that we had about 30% of our sample in each the Mail 2 WIC and Mail 3 Hand Delivery modifications. About 8.5% of women completed the survey in these modes.
But this doesn’t really tell the whole story of the impact of these modifications on response.

20. Modifications Results Response by group
Number
Eligible
Completed
Response Rate
Mail 2
M2-Postal only
467
273
58.5%
M2-WIC
443
317
71.6%*
Mail 3
M3-Postal
269
100
37.2%
M3-Hand Delivery
412
256
62.1%*
First we’ll look at the Mail 2 delivery.
For this analysis, we compared women who were sent a mail 2 questionnaire through the mail ONLY to women who received a mail 2 through WIC and through the mail.
Women in the M2 WIC group, were significantly more likely to complete the questionnaire than women in the standard protocol group for mail 2.
M2-Postal only = 36%; M2-WIC = 34%
M3 Postal = 21%; M3 Hand Delivery = 31.7%
, or no WIC record for mother.
* p<.05
response rate significantly higher in both modification groups

21. WIC Delivery Results Mode of completion by group
M2-Postal
M2- WIC
Mail 1
19.3%
23.3%
Mail 2—Postal
16.3%
13.5%
Mail 2—WIC NA
10.4%
Mail 3—Postal
2.8%
4.7%
Mail 3—Hand
8.8%
Other Mailing
0.2%
0.5%
Phone Phase
11.1%
14.4%
Total
58.5%
71.6%
Here we have a side by side comparison of response for the standard mail 2 postal only and the mail 2 WIC modfication group. This slide shows us where exactly the higher response rate for M2 WIC comes from.
There appears to be a higher response across nearly all data collection steps for the WIC group than for the standard group, with the exception of the standard m2 postal mailing and also Mail 3 hand delivery.

22. WIC Delivery Results Process steps
Number
% of eligible women
% of total sample
Questionnaire
Contacted by WIC
267
60.3%
20.5%
Questionnaire Delivered
192
43.3%
14.7%
Telephone Information
Telephone info provided
206
46.5%
15.8%
Tracking Data
Not documented 95
21.4%
7.3%
So what happened exactly?
There was a documented WIC contact among 60% of WIC eligible women, which is 20% of the overall sample.
Documented questionnaire delivery among 43% of wic eligible women, nearly 15% of the overall sample.
Telephone information was received for 47% of eligible women, 16% of the sample.
We did not receive tracking documentation back from wic for nearly 20% of wic eligible women.
Questionnaires may not be delivered because women declined an additional survey—reported that it was already completed or had them at home already and didn’t need another one.

23. WIC Costs WIC cost per additional response = $20 Budget Item Cost
Duplicate questionnaire packets
$655
Mailings to WIC sites
$340
Mailings from WIC to PRAMS
$170
Staff time (not estimated) $0
Total Cost
$1,165
This slide outlines the cost of this modification.
Overall cost was 1,165.
Cost of each additional response above the standard protocol response rate was $20.
Very cost effective.
WIC cost per additional response = $20

24. Hand Delivery Results M3 eligible women by mode
Mode of completion
M3-Postal
M3-Hand Delivery
Mail 1
5.2%
10.7%
Mail 2—Postal
3.7%
10.4%
Mail 2—WIC
4.8%
4.9%
Mail 3—Postal
12.6% NA
Mail 3—HD
15.8%
Other Mailing
0.4%
0.2%
Phone Phase
20.1%
Total
37.1%
62.1%
We saw a similar slide for wic a minute ago.
This side-by-side comparison of the mail 3 groups shows where the higher response for HD comes from. Much higher across all modes.
We attribute the 10% higher phone response to better phone contacts and promotion of the 800 number and phone interview among HD group.
We attribute the higher mail 1 and mail 2 postal to questionnaire pick ups when staff initially arrived at participant’s homes.

25. Hand Delivery Process Data Delivery or primary pick up
Successful contact
number
% of eligible women
% of total sample
Q delivered
152
36.9%
11.7%
Q delivered/picked up
172
41.7%
13.2%
This slide shows that a q was delivered to 37% of women in the hand delivery group, about 12% of the sample.
The next row shows that 42% of the hd group either received a q or provided a completed questionnaire to the tfs at delivery.

26. Hand Delivery Process Data Questionnaire pick up
Pick up mode
number
% eligible women
% of total sample
Pick up at initial contact 44
10.7%
3.4%
Pick up after delivery 20
4.9%
1.5%
Total Q picked up 64
15.5%
Overall Tribal Field staff picked up completed questionnaires for about 16% of the hd group.
This activity alone contributed nearly 5% of the overall response rate.

27. Hand Delivery Process Data New contact information collected
Type of
information
Number
% of eligible women
% of total sample
New address 27
6.6%
2.1%
New telephone
120
29.1%
9.2%
No new contact
285
69.2%
21.9%
We also received new contact information.
I’ll refer to the far right of the slide—
We received new telephone numbers for 29% of hd group, or 9.2% of the overall sample.

28. Contact Verification (after tickler)
New Information Collected
Number
% eligible women
% of total sample
Address
Mailing 37
6.9%
2.8%
Physical 50
9.3%
3.8%
Phone
New phone
257
47.7%
19.8%
Whose phone info:
Participant
207
38.4%
15.9%
Father / partner 13
2.4%
1.0%
Relative 42
7.8%
3.2%
Friend 10
1.9%
0.8%

29. Hand Delivery Costs HD cost per additional response = $383 Budget Item
Mailings to Tribal Field Sites
$286
Mailings from TFS to PRAMS
$240
TFS personnel
$53,200
Mileage
$2,385
Training (estimated)
$3000
Postage savings
($478)
Total Cost
$58,633
Hand delivery is obviously a more expensive endeavor than the WIC partnership. The overall cost was nearly $60,000, and cost per additional response was $383.
We believe, though, that the results of this partnership extend beyond response rates to supporting data translation efforts.
HD cost per additional response = $383

30. Factors in success
Obtained contact information
Overcame mail and telephone barriers to contact & Q return
Increased motivation
Encouragement from trusted providers
Culturally relevant materials
Desirable rewards

31. Next Steps
Prepare 9 tribe-specific, 1 statewide, and 4 issue-specific reports
Provide data use training for tribes
Develop maternal and infant health task force to use findings to develop new program and policy initiatives
Work with elders and traditional leaders to interpret and communicate findings / develop recommendations

32. Conclusions Protocol modifications were successful and replicable
Community-responsive adaptations could be applied to other groups
CBPR approaches improve PRAMS awareness and demand among stakeholders
Tribes and TECs can lead efforts to improve AI/AN MCH surveillance

33. Contact Acknowledgements Christine Rinki, MPH
Northern Plains Tribal MCH Epidemiology Program
Acknowledgements
SDT PRAMS Staff
Ssu Weng; Jennifer Irving; Lynn Big Eagle; TFS Team/Northern Plains Healthy Start
SDT PRAMS Participants
Yankton Sioux Tribe
Chairman Robert Cournoyer, Glenn Drapeau, Clarence Montgomery
Participating Tribes and Tribal Oversight Committee
SDT PRAMS Steering Committee
South Dakota Department of Health
Jacy Clarke, Kayla Tinker, Kathi Mueller, Anthony Nelson
Everett Putnam
North Dakota Department of Health
Carmell Barth
CDC PRAMS
Denise D’Angelo, Mary Rogers
Funding sources
IHS MCH Epidemiology Grant #H1 U IHS
CDC Cooperative Agreement #1 UR6 DP /02