1. THEN and NOW Lessons Learned with Process Change and Management Evaluation in the State of Pennsylvania August 2014

2. Changes in Pennsylvania: Increasing cross-county work –Increasing workload –Statewide system access How to monitor and report? –Sampling and reporting –Corrective Action Planning Pennsylvania Overview

3.  6 Regional Areas  88 county assistance offices (CAOs)  Total # of SNAP households: 894,383 Pennsylvania Overview

4. Deputy Secretary For OIM Bureau of OperationsBureau of Program Evaluation Division of Corrective Action Division of Quality Control Area Managers County Assistance Offices Division of Program Implementation DPW Overview

5. Division of Corrective Action (DCA) has 3 office locations:  Harrisburg- Headquarters (HQ)  Pittsburgh- field office  Philadelphia- field office ME team meets monthly:  Discuss review suggestions  Review changes  Answer questions Background

6. HQ Process 1 Target letter received 2 Meeting scheduled with all ME staff 3 Prior to meeting, HQ staff: Edit worksheets Edit process and procedures documents 4 After meeting, HQ staff: Adjust documents Assemble ME packet for internal approval Forward ME packet to FNS

7. Examiner review duties: –Generate entry letter –Conduct entrance conference –Complete case review assessment (including worksheets) –Complete interviews (local staff, Community Advocates, Blind Calls) –Data enter cases into Rushmore Case Review System –Generate report Examiner Process

8. Prior to 2011: –Conducted by 5 examiners –48 county assistance offices (CAO) –Manually compiled sample list –Traveled monthly to conduct on-site visits over 5 days –Examiner returned to office to reconcile errors, complete worksheets and logs and generate report –Report contained sentence format explanations of each error cited ME Review – THEN

9. FNS mandated targets comprised of: –Customer Service/Program Access –EBT –Recipient Claims –Intentional Program Violation –LEP –Nutrition Education –Employment and Training –Recipient Integrity –Corrective Action Assessment

10. Challenges StaffingBudgetTimingCommunication

11. Challenges Statewide case processing Statewide Customer Service Centers Work with other divisions Staffing Adjustment to desk review- training on electronic case review access Mandate to electronically image all case record information. Budget Provide a report to county office in less than 60 days. Implement corrective action Design new tools and processes Timing Information Memo Presentations Follow-up Communication

12. Since 2011: –Conducted by 2 examiners –24 county assistance offices (CAO) and districts annually –Sample list extracted from Access database –Desk review since 2010 (review electronic case record) –Telephone entrance and exit conference –Telephone interviews –Examiner completes review in DCA field offices, complete worksheets, reconcile errors by data entry into Rushmore database and generates report –Report is Word document format and includes charts outlining errors ME Review – NOW

13. E&T EBT Security Nutrition Education Recipient Integrity Team

14. ME Review – NOW In October 1, 2012 PA changed the county assistance office selection process and review 4 offices from each Area per FFY. Review criteria include: Review number of SNAP households Rank offices Drop offices Select offices Division of Statistical Analysis provides the number of active households

15. October 1, 2012 – ME re-reviews were established. The re-review still involves the same basic review process steps as an ME review. Re-Review process:  One examiner completes re-reviews  Top 4 error elements from previous FFY ME review  Sample size is 35% of original sample  Re-review will determine effectiveness of corrective action activities  Entry/sample letter  Comparison report  Rebuttal  No CAP required if accuracy improved  Final report issued, if rebuttal submitted ME Re-Review – NOW

16. Reports include: Comparison chart Recommendations ME Re-Reviews Re-Review Reports:

17. The ME process started to incorporate more automation: In 2005, the Rushmore Case Review System was designed to: Capture all ME case review results Uniformity of review Statistical information Automation- NOW

18. In 2009, an Access database was created to extract sample cases. Data display follows target letter mandates. Automation - NOW

19. Questionnaires Exported to an Interview Database. Automation - NOW

20. Corrective Action Plan Database: Access database created 2 years ago Includes all benefit programs corrective action activities by county office Database provides one central location to explain internal changes implemented as a result of the ME review Automation - NOW

21. Goals of future ME reviews include: Re-evaluate CAP submission request Incorporate travel to county offices to view signage and client waiting areas Develop macros-based reports to further reduce manual processes Enhance recommendations due to office observations of internal process/procedures Incorporate previous Corrective Assessment Action review criteria Reinstate ME review procedures and areas of concerns in Area Management meetings Resume EBT Security and E&T reviews FUTURE

22. Allison Miles –Director of Corrective Action –amiles@pa.govamiles@pa.gov Amira Milikin –ME Coordinator –amilikin@pa.govamilikin@pa.gov CONTACT INFORMATION