1. Impact Assessment Indicators for Administration of Public Transportation Grants NCHRP 20-65 Task 49 Viktor Zhong Scott Baker Dec 3, 2015

2. Agenda 1.Background and Introduction 2.State Impact Indicators in Use Overall program administration efficiency Effectiveness of funding advocacy and funding programming Timeliness of grant administration activities Provision of grantee support 3.Indicators Reporting 4.Recommendations and Conclusions 12/3/2015

3. Background and Introduction

4. Research Background “How effective is the state’s administration of transit programs?” “How efficiently does the state use resources for transit program administration?” –Two questions of interests to stakeholders of state transit administration, including state transit administrators and external stakeholders –Indicators of state transit administration effectiveness and efficiency - focus of this study –Indicators can be misinterpreted and misused 12/3/2015

5. Research Approach 12/3/2015 Surveyed all 50 states’ public transit administration Conducted more in-depth interviews with three states - Missouri Department of Transportation, Multimodal Division - Texas Department of Transportation, Public Transit Division - Virginia Department of Rail and Public Transportation Compiled performance indicators in use by state transit administration

6. State Administration Activities and Process 12/3/2015 Indicators reported in each of the three areas of state transit administration, as well as overall effectiveness and efficiency Securing State Match/Appropriation Making Federal Grant Application Local transit training State vehicle procurements Inspections Local transit planning, Construction management Drug and alcohol testing Compliance assistance MAP-21 (asset management plans, performance targets, safety plans) Funding and Allocation Receiving and Reviewing Grant Applications Eligibility and Selection of Grant Applications Making Grant Awards Administration and Reimbursement of Grants State Transit Program Design & Management 1. Funding and Program Level 3. Grantee Support 2. Funding Administration Audit Closeout

7. Performance Indicators In Use

8. Overall Administration Effectiveness and Efficiency IndicatorsState Administration cost as % of total annual expenditure FL, MO, OR, VA, WV Program administration cost per trip supported by the program MN Actual expenditure to budget ratioMA, MO Percent of partners indicated “Satisfied” or “Very Satisfied” with DOT in delivering transportation services MO 12/3/2015

9. Administration Cost As A Percentage Of Total Annual Expenditure –A measure of overall administration efficiency; –A lower percentage is desired by some, but… CAVEATS: CAVEATS: when it is compared among states or over time: –Administration cost is not proportional to funding level –Amount and quality of services provided drive the ratio –Consider separating grant administration costs from other program administration expenditures 12/3/2015

10. Funding and Program Level IndicatorsState Total state and federal transit funding received/expended/under contract/available in all open grants/active AZ, IL, MO, NM, VA, TX Percent of federal funds subject to forfeiture at the end of the fiscal year (Sept 30) committed by the department FL 12/3/2015

11. Total Funding Level and Funding Lapsed –Example from Texas DOT’s Public Transit Division 12/3/2015

12. Effectiveness and Efficiency of Grant Administration IndicatorsState Number of grants awarded/opened/managedAZ, IL, TX, VA (Percent of )vendor invoices paid on time or time it takes to pay a requisition AZ, MO, OH, IL Operating/capital payments processed and to Finance on time MI Average number of days from sponsor project selection to project award (MO) Time from grant application until time of grant award contract (IL) IL, MO Staff hours per application reviewWI Number of grants closedAZ 12/3/2015

13. Provision of Grantee Support IndicatorsState Number of subrecipient grantees servedVA Number of vehicles purchased/delivered to granteesAZ, MO Number of compliance program field visit technical support TX Number of site visits for transit grantees’ compliance with federal and state requirements completed MO Number of participants in transit trainingMO Number of grant recipient compliance reviews completedTX Staff hours per procurementWI Average purchased vehicle delivery timeAZ 12/3/2015

14. Indicators Reporting

15. –Purposes: internal and external communication –Internally Audience: state transit administrators Purpose: to evaluate administrative performance and inform management decisions to improve effectiveness and efficiency –Externally Audience: funders (state and federal), grantees, and general public Purpose: to demonstrate to key stakeholders the state transit administration used resources responsibly to provide quality services to their grantees 12/3/2015

16. Indicators Reporting 12/3/2015 –Reported in strategic and/or business plans –Reported in performance report Use a dashboard tool

17. Recommendations and Conclusions

18. Conclusions and Recommendations –Success in tracking indicators in all aspects of state administration, including Overall administration efficiency Effectiveness of funding advocacy and funding programming Timeliness of grant administration activities Provision of grantee support –Consider using a dashboard tool –Consider separating grant administration costs from grantee support costs 12/3/2015

19. Thank You