1. Measuring what matters: Approaches to modernizing grant performance measurement that promote innovation, transformation and deliver impact Alex Sirota, Team Lead, eService Delivery Ministry of Economic Development and Trade @ Results Based Management For Government Conference Ottawa, Ontario, Canada December 2009

2. 2 http://www.ontario.ca/economy December 2009 4 objectives + 1 case study Is your program a candidate for modernization? How people, process and technology work to transform How to achieve credible program outcomes How to measure the results of your program A real-world example

3. 3 http://www.ontario.ca/economy December 2009 A candidate for modernization? Are there specific pains to address? Is there support from senior management for a change? Are there technologies that can be readily reused or applied? Does your organization need a repeatable, agile and measurable process to run a grant program and measure its performance?

4. 4 http://www.ontario.ca/economy December 2009 People, Process & Technology People –Understand what your staff can and cannot do –Understand what management is willing to risk, spend & achieve Process –What are the really painful parts of the process? –Continuous improvement Technology –Technology is readily available as are skill sets –Choose tried and true solutions –Choose what fits your organization and iterate

5. 5 http://www.ontario.ca/economy December 2009 Credible Program Outcomes Can you communicate objectives and results? –Press-release worthy? –Celebrate wins –Survey perception from internal and external stakeholders New challenges or opportunities as a result of transformation? –Grant programs can be perceived as sustainable and scalable –Project team engages to achieve a shared goal –Raises expectations for program (scrutiny, volume, marketing)

6. 6 http://www.ontario.ca/economy December 2009 Measure – Iterate - Innovate Identify what you want to improve, and then do it –Step by step and allow for team to integrate into workflow Things to consider –Process –Review & grant approval processing time –# of grants processed pear year (if budget allows) –Privacy concerns, Accountability & transparency –Ability to measure detailed information pre and post approval –Determining when a proponent is “successful” or “not successful” –People –Satisfaction of program officers, satisfaction of grant recipients –Impact of improvements to daily job –Technology –Flexible and easily adapted application process –Collecting too much, not enough information?

7. 7 http://www.ontario.ca/economy December 2009 There is no such thing as “best practice.”  Best-practice is yesterday’s news and contextual.  Emerging practices are tomorrow’s news and you can claim as your own.  Every year more grant programs are modernizing as technology matures.

8. 8 http://www.ontario.ca/economy December 2009 What does transformation look like?

9. 9 http://www.ontario.ca/economy December 2009 What is Summer Company? An entrepreneurship experience for 15-29 year old students in Ontario - www.ontario.ca/summercompany $3000 award for the duration of the summer to start and operate a business Delivered by a network of program providers across Ontario –Alternate service delivery –Small Business Enterprise Centres + Non profit organizations

10. 10 http://www.ontario.ca/economy December 2009 A good candidate for modernization In 2001, paper-based application and approval process –190 approved awards Inconsistent application and delivery process across Ontario Time & resource intensive pre- and post-approval process Difficult to determine “successful completion” of awards Many young entrepreneurs wait until the last minute to apply, leading to a “python” effect of recommended applications Modernization started 2003

11. 11 http://www.ontario.ca/economy December 2009 Before modernization

12. 12 http://www.ontario.ca/economy December 2009 Summer Company, today 5 years to fully implement process and system, by choice Version 9 is in operation, delivered by 60+ delivery partners Processed over 15,000 application inquiries and over 2,000 approved grants as of 2009 Major issues or opportunities –Avoid resistance to change by involving end users –Privacy Impact Assessment completed, recommendations underway –Improve reporting and business intelligence functionality Next Steps –Business process and technology can be made generic for similar grant programs –Collecting broader public sector interest to adopt business process and/or technology

13. 13 http://www.ontario.ca/economy December 2009 Results 2009 had over 600 approved awards, 200% increase over 2001 Manage more awards with same or less amount of staff Significant workflow improvement Transparency & accountability (Cashflow, Journals, Checklists) Improved standard of service delivery, internally and throughout Ontario, reduced paper burden

14. 14 http://www.ontario.ca/economy December 2009 Our “best-practices” No grand vision for what the program would be in 5 years –build in quality through listening and iteration Get feedback from end users, constantly and implement priority changes Checks and Balances done throughout process leads to improved application rate and reduced default rate on awards

15. 15 http://www.ontario.ca/economy December 2009 After modernization Happy, productive program teams.

16. 16 http://www.ontario.ca/economy December 2009 Thank you for your time! For more information, contact –Alex Sirota, Team Lead, eService –MEDT – Entrepreneurship Branch –416 212 0549 – alex.sirota@ontario.caalex.sirota@ontario.ca Offer –Do you want to see the system in operation to learn more? –Self running “Flash” demo –UAT system is always available via web conference-call

17. 17 http://www.ontario.ca/economy December 2009 DEMO SCREENS

18. 18 http://www.ontario.ca/economy December 2009

19. 19 http://www.ontario.ca/economy December 2009

20. 20 http://www.ontario.ca/economy December 2009

21. 21 http://www.ontario.ca/economy December 2009

22. 22 http://www.ontario.ca/economy December 2009