1. Assessing The Policy Outputs Environment for Effective Programming Mohammed M Tumala (MEMS) Theresa Effa Kaka (ENHANSE) Chinelo Ezeobi (COMPASS)

2. Objective of Presentation Share with participants the Policy Environment Score process as a Monitoring Tool used by some USAID projects in Nigeria

3. PES as a Monitoring Tool 1. Provides a MEASURE of “perceived” overall status of the policy/resources environment in the country/state as it concerns programs When observed over time may indicate effectiveness of existing system strengthening programs 2. Provides a Measure of “perceived” changes in the policy/resources environment between two time periods  This will indicate direction of change in the environment 3. Identify weak/strong areas of policy/program resources  Vital for programmatic decisions

4. Steps in Conducting PES 1. Adapting the PES Questionnaire to the program area 2. Selection of Respondents for the PES Questionnaire 3. Adopting a methodology and administer the questionnaire 4. Data entry and Analysis 5. Writing of Report

5. 1.Questionnaire: Supportive Policy/Resources Environment Political support and commitment including supportive national/state policies, laws and plans; Existence of Policies that meet client’s expressed needs Operational policies that promote access, demand and quality, all the way down to the service level; Adequate financial, infrastructure and human resources Active private sector/CSOs participation Programs that are designed and implemented according to policies.

6. Questionnaire Ctd: Classification of Policy Output & Resource Areas i. Political Support for programs ii. Policy Formulation/Implementation iii. Organizational Structure iv. Legal and Regulatory Environment v. Program Resources vi. Program Component Implemented vii. Evaluation and Research in the program area

7. Questionnaire Ctd : i. Political Support for Programs This looks at level of support/opinion of groups on specific program area and supporting policies:  High Level political Leadership  The public  Political parties  Development plans  Religious organizations  Major traditional/community leaders

8. Questionnaire Ctd : ii. Policy Formulation/Implementation This looks at:  Existence of policies/laws and programs to facilitate provision of services  Dissemination of existing policies at all levels  Adaptation of higher level policies at lower levels  Existence of implementation guidelines or norms and/or protocols of existing policies  Effective implementation of existing policies  NGOs, community and religious leaders involvement in Policy dialogue

9. Questionnaire Ctd: iii. Organizational Structure Should Include:  Existence of a coordinating body that engages CSOs, FBOs, and Communities  Existence of a department/agency with high level placement in government  Existence of Responsible officers on full time responsible for programs  Formal involvement NGOs/FBOs in planning and organizational structure deliberations  Inclusion of Community and Religious leaders in planning organizational structure deliberations  The formal involvement of the private sector in planning organizational structure policy deliberations  Formation of management committees that include CSOs

10. Questionnaire Ctd: iv. Legal and Regulatory Environment  Existence of norms and standards for service provision  Adaptation of norms and standards for the provision of service at all levels of service  Existence of Mechanisms to enforce compliance with norms and standards to ensure quality of service provided  Existence of Mechanisms to enforce the provisions of existing laws

11. Questionnaire Ctd: v. Program Resources  Funding from public sector sources  Funding from external sources  Financial capabilities of communities/CSOs for service provision  Awareness and capacities of community/religious leaders, and CSOs  Resource availability to families/individuals to provide service

12. Questionnaire Ctd: vi. Program Components implemented This will depend on the type of program. OVC example:  Provision and high coverage of all services that meet the needs of OVC Food & Nutrition Education Health Psychosocial Support & Participation Economic Empowerment/IGA Legal Protection Shelter

13. Questionnaire Ctd: vii. Evaluation and Research  Existence of systems for regular/adequate collection of statistics  Existence of systems to monitor secondary data sources (surveys, censuses, local studies, etc.)  Existence of systems to bring assessment/evaluation and research results to policymakers’  Undertaking of Special studies  Use of evaluation and research data for decision making

14. 2. Selection of Respondents Respondents are selected based on perceived knowledge and Involvement in the program area from a wide range of stakeholders:  Government Program managers in ministries and agencies  Donor agencies and Program managers in the private sector  Program managers of international NGOs  Representatives of local NGOs  University-based researchers, academicians  Representatives from CSOs/women’s groups

15. Adopt a Methodology and Administer the Questionnaire There are mainly two ways Postage:  Respondents receive the questionnaire with adequate instructions on the purpose and scoring. Respondents return completed questionnaires Workshop:  Respondents are Invited to a workshop, receive briefing on purpose, use and scoring.  Individually score the items  An FGD may be conducted around the average scores to provide narratives to the observed scores

16. Administer the Questionnaire Whichever method Used, Participants Individually Score Items they have knowledge on. Individuals may differ in their scores, but NO guessing the scores There is a flexible ranking used for scores e.g:  4 means a very strong or a better or more satisfactory rating;  3 means strong,  2 means somewhat strong/somewhat weak (mid-point),  1 means weak, and  0 means very weak or non existent.

17. Data Capture & Analysis The target for Data capture is to find averages for Items and Policy Output Categories An Excel Template is used. Modified alongside the questionnaire ComponentsAverage Score 20062007 Political support Policy formulation Organizational Structure Legal/regulations Resources Programs components Evaluation and Research

18. Use of PES in Nigeria The Policy Project used the PES in 2000 to assess the policy environment as it concerned Reproductive Health in Nigeria (FP, SM, ARH)  The PES tool was adopted to the Nigerian situation  The outputs informed the national population policy review In 2002 the FMOH (CDPA) with TA from Policy Project undertook a PES survey (FP, SM, ARH) COMPASS uses the PES in 5 states; Lagos, Nasarawa, Kano, Bauchi & FCT (BE, RH, CS)to monitor changes at the state level

19. Use of PES Outputs Improve Program planning or implementation for OVC services in the state Advocate for Increased funding of OVC services in the state Inform policy reforms/formulation or review in the state Guide Managers of Public and Private Sector Institutions Provide NGOs with tools to advocate for increased use of quality services