National Immunization Conference Hyatt Regency Atlanta International Ballroom South Thursday, April 22, :30 – 11:30 AM Lessons Learned from the VTrckS Provider Order Pilot (VPOP): CDC and Grantee Perspectives 0

Agenda 1  Brief VPOP Overview  VPOP Statistics and Evaluation Results  VPOP Lessons Learned  Transition: VPOP to VTrckS  VPOP Questions and Panel Discussion  Open Q&A from Audience

CDC implemented VPOP in May 2009 to address several strategic objectives  Conduct a test run of online ordering processes  Engage the provider community early in the process  Test a simplified version of the implementation process  Capture feedback and lessons learned from grantees and providers to apply to the full VTrckS solution  Reduce the burden of handling paper forms and increase order accuracy 2

3 TopicVPOPVTrckS PilotVTrckS Participants Four grantees and select providers Four VPOP grantees (and providers) and up to two ExIS grantees 1 All grantees; providers at the grantees’ discretion ExIS/IIS IntegrationNoYes VACMAN UseYesNo Order Status Limited to order confirmation Order confirmation to shipment information Grantee AccessNoYes Provider online ordering and submission of accountability data Yes How does VTrckS Provider Ordering Pilot (VPOP) differ from VTrckS? 1 Subject to contract modification

Providers from the four VPOP grantees have used VTrckS to place 1,220 vaccine orders 4 VPOP Statistics as of April 16, 2010 Grantee Go-live (2009) Sites Active in SAMS Users Active in SAMS Total Orders MassachusettsMay CaliforniaJune ChicagoJune ColoradoJuly Total ,220

VPOP Evaluation Results: Overall System Performance 76% of providers agree with the statement In October, provider satisfaction rose from 62% to 76%, with respondents reporting that VTrckS was easy to use 5

71% of providers agree with the statement VPOP Evaluation Results: Overall System Performance In October, 71% of respondents concurred that VTrckS is a satisfactory replacement for existing vaccine ordering methods 6

80% of grantees agree with the statement VPOP Evaluation Results: Overall System Performance Grantee approval rose from 70% in June to 80% in August, with respondents agreeing that their providers are satisfied with VTrckS 7

100% of grantees agree with the statement VPOP Evaluation Results: Overall System Performance In June, 14% of grantees agreed VTrckS was a satisfactory replacement for existing ordering methods, by August 100% agreed 8

49% of respondents indicated that the ID proofing process was easy to complete 9 49% agreed that the ID Proofing process was easy VPOP Evaluation Results: Identity Proofing Process

The survey revealed that the ID proofing process took between 6 to 15 business days 10 “How many business days did it take to have your picture identification reviewed by an authorized agent (Notary Public or Immunization Program representative) for the Vaccine Tracking System (VTrckS)?” 29 respondents (46%) used a notary while 34 used an immunization program representative VPOP Evaluation Results: Identity Proofing Process

VPOP grantee and provider feedback resulted in important changes 11 SummaryImpactsFull Description Add a system timeout notification User This feedback resulted in the addition of a prompt that informs the user that the session will soon end and how to continue the session without losing data. Accountability page column headers User This feedback resulted in a change to the column headings so they are visible at all times, even if scrolling is necessary. Transmit accountability files without an order User/ Grantee This feedback resulted in the inclusion of accountability data submitted into VTrckS without an associated vaccine order from the same date into the Excel file downloaded by grantees. Temporary Address or Hours Change Grantee This feedback resulted in a notification to the grantee that the provider’s address and/or delivery hours have reverted back to the original stored value, which may differ from what was entered into VACMAN for the provider’s most recent order. Automatically move Excel files with no data to the Archive Grantee This feedback resulted in the automatic transfer of any files with a “_0_” in the file name to the Archived Orders directory without any action needed by the grantees.

Lessons learned from VPOP are being incorporated into full VTrckS planning efforts  Develop enhancements for “identity proofing” to simplify access for 100,000+ VTrckS users  Engage the immunization program early to review processes and understand changes  Communicate effectively and often with the user community to update them on the implementation and transition activities  Support end users by balancing implementation plans with other non-VTrckS responsibilities (flu season, back to school, etc.)  Include end users in testing whenever possible 12

VPOP Panel Discussion Members Contact Information  California –Maria Volk - –Claudia Aguiluz -  Chicago –Maribel Chavez-Torres -  Colorado –Nicole Ortiz - –Chris Duggar -  Massachusetts –Robert Morrison - 13