Frameworks 1 March 2011

Session Outline Introduction to frameworks Conceptual frameworks Results Frameworks Logical Framework Hands on activity: developing frameworks Speaker Notes During this module, we will review the components of an M&E plan and the placement of frameworks within that plan and introduce four types of frameworks. Frameworks are probably not new to you; though the terminology that will be used in this course may be. Other terms that are used to describe these types of frameworks will be mentioned. There will be an opportunity to study and develop examples of the different types of frameworks. Lastly, the interface between frameworks and M&E planning will explored. The last two hours of the day will be devoted to work on your group projects that will last the duration of the workshop. A handout containing examples of frameworks has been given to each participant. You may find them helpful for understanding the content and purpose of the different types of frameworks that will be discussed, and may be useful for future reference. Please keep in mind that these frameworks are meant to be examples and are not intended to be interpreted as the “correct” framework for a certain program.

Sound implementation of programs Design sound M&E plans -- Why frameworks Designing M&E Frameworks assists in the development of Clearly understood program/project goals and measurable, long-term, short-term, and intermediate objectives Clearly defined relationships between program/project inputs, processes, outputs, and outcomes, Understanding between program/project activities and the external context (environmental factors) Sound implementation of programs Design sound M&E plans Speaker Notes Frameworks are best understood as useful tools for understanding and analyzing a program, which is crucial for developing sound M&E plans and implementation. Designing program frameworks is one way to develop a clearer understanding of the goals and objectives at the heart of a project, with emphasis on the objective or measurable objectives, both short-term and long-term. Developing M&E frameworks also helps clearly define the relationships among factors key to the implementation and success of a project, both internal and in interaction with the external environment or program context. In other words, developing frameworks helps generate a clearer picture of ideal goals and pragmatic objectives, and the elements both within and external to project operations that will affect its success in the particular context. This design process deepens the understanding of managers, implementers, and other partners in many practical ways, including serving as the foundation for selecting appropriate, useful M&E indicators. Additional Background While frameworks are obviously useful after their completion, providing a clear structure of the indicators at the heart of M&E plans, one of most useful qualities is that discussing and determining their design serves to clarify many of the concepts and assumptions underlying basic project activities. An M&E plan that does not identify appropriate, useful M&E indicators, metrics, and data systems is an M&E plan that will not contribute to good program management or to an intervention’s success. Activities Organize participants into small groups*. Using case studies that provide basic information about a program/project, each group should identify in terms as clear as possible: the long-term goal for their case study, and relevant short-term and intermediate objectives. Reassemble the full workshop and have a representative from each group report on their discussion and decisions. The full group may offer and discuss alternatives to those initial ideas; different goals, different wording, or different levels for targets may be identified. Groups may modify their initial ideas in later sessions. *Small groups working on case studies are recommended as a structure for maximizing the benefits of this workshop.

Characteristics of frameworks All types of M&E frameworks: Inter-relate components, levels and directions of action Enable understanding of how programs influence health outcomes (conceptual framework) programs should operate (logical framework) Programs achieve goals (results) Facilitate consensus building around a common paradigm There is no perfect framework Need consensus building process Use your evaluation team to build consensus, built on your framework Indivdiual M&E M&E Characteristics skills, CF competence (stata, materials) EVALUATION FRAMEWORKS – Purpose is to convey sequence of steps in a project leading to desired end

Types of frameworks Different origins Many types: Specific types Conceptual Results Logical & Logic model Specific types Results Based Management framework

Learning Objectives: Conceptual Framework At the end of this session, participants will be able to: Understand why and how conceptual frameworks are useful for understanding programs and planning M&E Describe the role of conceptual frameworks in program design, program evaluation, and evaluation research. Describe the components of conceptual frameworks Design a conceptual framework to be most useful for M&E planning Design a conceptual framework for an intervention program Speaker Notes Four types of frameworks are introduced in this module: conceptual frameworks, strategic or results frameworks, logical frameworks and logic models. The learning objectives of this Module on Frameworks include understanding the components of these four basic types of frameworks in order to identify and differentiate between them. Participants will be expected to design frameworks for a specific intervention program and to use these frameworks for developing M&E plans. Additional Background Different organizations tend to prefer a selected type of framework. It is not important to convince participants to use any particular type. The gist of this session, however, is the importance of designing a useful framework for a specific project in a specific context, not the titles or particular appearance of any designated kind of framework.

Conceptual framework Research framework

Group work- 1 Why did we have political revolt in Egypt Why do we have poverty in province B Why do we have crime in City Z Why do we have famine in Country X How come we have street abortion in South Africa Why do we have teenage pregnancy in Community B How come Spain won the FIFA soccer world cup How come China has a booming economy How come the people of Thailand are so hospitable How did Uganda manage to control an epidemic

What are concepts

What is happening?

What is happening World Society/community Workplace hospital

When we conceptualise Definitions What is the meaning of the word or term or idea What is the actual use of the word What are the boundaries within which a term operates What are the contrary examples of the word

Concepts Enable interpretation of a subject Analyse complex subject matter Synthesize separate pieces of a subject matter into a unit Perceive similarities and differences Enable us to make value judgments Extend our knowledge

In program design/programming What do you need to know Why do have to know

M&E CONCEPTUAL FRAMEWORKS Conceptual, or “research”, frameworks are diagrams that identify and illustrate the relationships among all relevant systemic, organizational, individual, or other salient factors that may influence program/project operation and the successful achievement of program or project goals. Speaker Notes Conceptual frameworks are sometimes called “research” frameworks. A conceptual framework is a useful tool for identifying and illustrating a wide variety of factors and relationships that may affect the successful outcome of your program. These factors include program plans and operations; its operating environment in terms of target or non-target populations, government or institutional policies, infrastructure, and other characteristics of the socio-economic or political system; and intermediate changes or ultimate goals. Conceptual frameworks take a broad view of the project itself in order to clarify the relationship of its activities and its main goals to the context in which it operates. The design of the framework should shows the interrelationships between all factors that are relevant to achieving the project’s goal. In other words, a conceptual framework identifies, illustrates, and diagrams all of the salient relationships among all of the factors impinging on the operations or targets of a program. These can be systems, organizations, population characteristics, or other features of the operational landscape that may help or hinder the program’s success. Constructing a conceptual framework thus clarifies the complete context affecting the outcome of an program or intervention. It clarifies your assumptions about the causal relationships connecting significant features of the program context, clarifying aspects that your planned intervention may affect as well as other factors beyond your control. Identifying the variables that factor into program performance, and organizing the explicit ways they interact with each other sets the stage for outlining the objective results you can reasonably expect from your program activities. Clarifying all of these issues is a critical step toward designing valid measures for analyzing, or evaluating, the success of those interventions. Additional Background Conceptual frameworks are used in the sciences to select key variables for analysis. By constructing this kind of analytical framework as the foundation within which your program will design, plan, and implement the Monitoring and Evaluation of program performance, real possibilities and limitations become clearer to everyone involved.

Purpose of conceptual frameworks Provides a perspective for understanding program objectives within a complete context of relevant factors in a program’s operating environment Clarifies analytical assumptions and their implications for program possibilities or limitations on success, as well as measuring and analyzing that degree of success

Purpose of Conceptual Framework:cont. Makes explicit connections among relevant contextual (environmental) factors and your program Helps to clarify the “why” and “how” questions of program operation and design: Assumptions that underlie the chosen activities Factors the activities are expected to affect Causal linkages leading to achievement of program objectives Guides identification of appropriate indicators Guides impact analysis There is no perfect framework Need consensus building process Use your evaluation team to build consensus, built on your framework Indivdiual M&E M&E Characteristics skills, CF competence (stata, materials) EVALUATION FRAMEWORKS – Purpose is to convey sequence of steps in a project leading to desired end

Conceptual Frameworks Individual characteristics Technical inputs Program supply Institutional capacity Program sustainability Health status Healthy practices Service utilization Speaker Notes This is a very general example of a conceptual framework in the health field. It has the characteristics described in the previous slide that it includes a number of factors that affect the desired outcome, including those not under the control of the program (such as individual factors). In each box is a factor related to another factor that ultimately affect a health outcome. Arrows indicate the relationship between factors, which in this case include individual, systemic and program variables. As you look at the framework, notice that there are two outcomes: health status and program sustainability. Following health status backward, we see that healthy practices influence health status. Healthy practices are affected by individual characteristics, service utilization and institutional capacity. You can also see service utilization is influenced by program supply and individual characteristics. Going back to institutional capacity, note that, according to this framework, it is affected by program supply and technical inputs. We just looked at this framework by moving backward from the outcomes of interest through all the influencing factors. You could also read it moving forward, however moving backwards allows you to keep the outcomes of interest in your mind when looking at relationships between different factors. Also, note that that one factor can influence or be influenced by more than one other factor. In this example, you can also see that program sustainability and institutional capacity influence each other. Additional Background During the rest of this workshop, many examples of conceptual frameworks may be provided or developed and discussed by participants. The point at this stage is to show that this kind of framework explicitly incorporates factors external to the program in order to examine the kind of impacts that contextual factors may have on outcomes of interest to the program.

Proximate Determinants Model for HIV/STI (Boerma and Weir) Health Outcome Demog. Outcome Underlying Proximate Biological Context Sociocultural Socioeconomic Programmatic Interventions VCT STD control Condom promote IEC Partner acquis. Mixing patterns Concurrency Exposure to infected HIV incidence Health Impacts: Morbidity Mortality Condom use Concurrent STI Risky sex Treatment Efficacy of Transmission Per contact STI incidence Duration of activity Treatment Socioeconomic Impacts:Socio-Economic Decline (household and national) -Orphans and Vulnerable Children

Research Concepts –connotations-defines the problem and constructs for measurement Operationalise-denotations- variables Framework allows integration of the program(research) with existing knowledge ( accepted thinking) of the problem. Concepts are identified and linked to one another

Research cont.. Research starts from this conceptual or theoretical framework Basis to Formulate our hypothesis Expresses Assumptions Testing/Falsification This is the basis of deductive approach in research

Trying it out!! Identify the problem ( concepts) Define drivers of the problem Define elements of your program( concepts) Identify factors(concepts) in your environment identify individual factors(concepts) Establish relationship between the factors Simplify and refine diagram Create and present your framework

summary To show where program fits into wider context To clarify assumptions about causal relationships To show how program components will operate to influence outcomes To guide identification of indicators To guide impact analysis (causal pathways)

Activity Get into your project/program groups Develop a conceptual framework for your intervention After 45 minutes, a member of each group will share the framework with all participants Speaker Notes Arrange yourselves in small groups of up to 5 people. (This does not need to be the group for the group project.) Each group will be asked to develop a conceptual framework for utilization of family planning/maternal and child health services. Use the materials provided to work out a framework. You may want to use on post-it note for each item in your framework, then move the post-it notes around on the large piece of paper until you finish your framework. Finally, copy the framework using the marking pens onto the large paper. One person from the group will be designated to share this framework with the larger group of participants. I will warn groups when there are 10 minutes remaining. Additional Background Monitor the groups to make sure they are progressing and to answer questions. The instructor should have also received a handout with commonly used conceptual frameworks for child health, family planning and quality of care for his/her information. This handout should not be distributed to participants at this time. Activity Recommended materials for this activity are: - Large piece of poster board or flip chart paper, one per group - Post-it notes, one pack per group - Thick marking pens, one or two per group

Strategic Framework

What are strategies What are results

Revisit your conceptual framework What is your vision? What is your mission? Where are your strengths?

M&E STRATEGIC (RESULTS) FRAMEWORKS Strategic/Results frameworks are diagrams that identify steps, or levels, of results, and illustrate the causal relationships linking all levels of a program’s objectives. Speaker Notes Strategic frameworks are a useful tool for identifying and illustrating the focal points of a project for monitoring and evaluation: the results. A strategic or results framework diagrams the relationships between the incremental results of the key activities all the way up to the overall objective or goal of the intervention. Developing a framework focused on program results clarifies the points at which results can be monitored and evaluated. In addition, results frameworks clearly depict the causal relationships that the project design assumes will connect, for instance, the clinical training of providers with the provision of a higher quality of care at facilities where (more) trained providers work, and thus ultimately lead to an improvement in health status or health outcomes for the targeted population. The effectiveness of the project’s related activities can be measured at each step along the way. Additional Background “Strategic” frameworks lay out the strategy being implemented by the program, in terms of the causal paths leading from short-term or lower level results to the achievement of upper-level or long-term goals. They are also referred to as results frameworks for obvious reasons. The terminology is unimportant; the key issue to emphasize here is that these frameworks combine the investigation of causal relationships developed in the Conceptual Framework with the program-specific, activities-oriented approach of the Logical Framework. Combining the two is most helpful for developing a comprehensive Monitoring and Evaluation plan. A more detailed example of Strategic Framework development is provided in Module 3, showing how it can be used to ensure coherence between program activities and the indicators selected for their related M&E.

Purposes: Provides a clarified focus on the causal relationships that connect incremental achievement of results to the comprehensive program impact Clarifies project/program mechanics and factors’ relationships that suggest ways and means of objectively measuring the achievement of desired ends

Strategic framework Results framework Diagrams Summary of results They identify and illustrate the causal relationships linking all levels of program strategy. Objectives to impacts

Results Frameworks Presents program strategy for achieving specific objective Usually presented as diagram, Includes objective and intermediate results Includes assumptions behind hypothesis (why the program/strategy is expected to work) Both a planning and management tool

Goals and Objectives Goal: Objectives: A broad statement of a desired, long-term outcome of the program Objectives: statements of desired, specific, realistic and and measurable program results Speaker Notes Fundamental to the design of type of program and its accompanying framework whether it is a results framework or a logical framework or a logic model is to clearly identify the program’s goals and objectives. A goal is a broad statement of a desired, long-term outcome of the program. Whereas the objectives are the statements of desired, specific, realistic, and measurable program results. The acronym “SMART” is useful to remember when developing objectives. (Review SMART as explained on the slide.) Source: GAP 2003

Terminology !

Results Framework Example – PEPFAR funded ART Program SO: Utilization of ART services IR-1: Availability of quality services IR-2: Demand for services IR-1.1: Increase ART sites IR-2.1: Increase knowledge of ART IR-1.2: Supply sites with ARVs IR-2.2: Increase referral from VCT IR-1.3: Training for providers

EXAMPLE: STRATEGIC OR RESULTS FRAMEWORK Speaker Notes This is an example of a Strategic Framework. Notice that each of the boxes contains a specific result or achievement. Beginning from the lowest levels, the diagram depicts the causal flow the managers of this program expect to lead from their activities to the overall objective, at the top. Additional Background The example uses USAID terminology: SO =“Strategic Objective”, IR =“Intermediate Result”. The subordinate IRs may be called Sub-IRs or Lower Level IRs; as usual, the terms are unimportant. The main points are that each level has clearly-specified, well-defined results, that should be measurable, and that the lower level results should contribute toward achievement of the higher results and the overall objective. Activities At this point, pause and review the frameworks and their various areas of usefulness. When participants have a general grasp of their differences, break the workshop back into their small groups. Some may wish to refine the goals and objectives they developed earlier (at the beginning of Module 2, Slide 30). Depending on the timeframe for these workshop sessions, each small group could develop each type of framework for their case study. At minimum, at this time each group should develop a conceptual framework they consider appropriate for their program/project. When they have come to a consensus, they should prepare a flipchart page/poster with their conceptual framework diagram. Reassemble everyone and let each group present their framework and the group’s reasoning behind their selected factors and the causal relationships they have shown. Debate and discussion should focus on program factors, environmental factors, and the relationship of the factors selected for inclusion to the effects and outcomes judged key to determining the project’s success. If the workshop schedule allows, postpone the groups’ development of their Results Framework until after the introduction of indicators in Module 3.

Hierarchy of Program Dimensions

USAID labelling

Results Frameworks: Key Elements Goal Strategic Objective Should be appropriate & realistic Balance ambition and accountability Intermediate Results Smaller goals Integral to achieving SO Hypothesized cause-effect (based on conceptual theory) linkages Flow from one intermediate result or to many Move “up” in the results framework Critical assumptions Relative to the achievement of the SO General condition which holds true for SO to be achieved Be realistic!

Results Frameworks: National TB Program SO1: Increase tuberculosis case detection rate to 70% IR1: Increased availability of quality services IR2: Increased demand for quality services IR1.1: Services increased IR2.1: Customer knowledge of TB improved Speaker Notes This example is for a tuberculosis control program. Three of the four IRs shown in the previous slide of a generic results framework appear here, two under IR1 (availability of services and quality of services). Notice that in this more developed example, the IRs and sub-IRs are phrased in such a way to indicate the desired results. For example, IR1 is not simply “quality services” as the previous generic example indicated, but “availability of quality services increased”. Also notice that each IR and sub-IR is measurable. That is, indicators can be developed and data collected to calculate those indicators. We will discuss indicators in the next session, but keep this in mind when thinking about results frameworks. IR1.2: Practitioners’ skills and knowledge increased IR2.2: Social support for TB practices increased IR1.3: Improved program management Source: MEASURE Evaluation, 2004 (draft)

Results Frameworks FIVE-YEAR GOAL: Foundation Established for Reducing Famine Vulnerability, Hunger and Poverty SO 14: Human capacity and social resiliency increased IR 14.1: Use of high impact health, family planning, and nutrition services, products, and practices increased IR 14.2: HIV/AIDS prevalence reduced and mitigation of the impact of HIV/AIDS increased IR 14.3: Use of quality primary education services enhanced IR 14.3.1: Community participation in the management and delivery of primary education services strengthened IR 14.1.1: Community support for high impact health interventions increased IR 14.2.1: Reduced risk behavior Speaker Notes Here is an example of a results framework that is multisectoral. This example comes from a country-level strategy adopted by a donor. The objective of improving human capacity and social resiliency is dependent on three intermediate results, two involving the health sector and one involving the education sector. It is important to remember that programs can span more than one sector and that this should be reflected in the framework. IR 14.2.2: Mother-to-child transmission of HIV reduced IR 14.1.2: Availability of key health services and products improved IR 14.3.2: Planning, mgmt and monitoring and evaluation for delivery of primary education services strengthened IR 14.2.3: Access to care and treatment for people living with HIV and AIDS increased IR 14.1.3: Quality of key health services improved IR 14.2.4: Care and support for orphans and vulnerable children expanded IR 14.3.3: Quality of primary education improved IR 14.1.4: Health sector resources and systems improved IR 14.3.4: Equitable primary education services strengthened IR 14.2.5: A more supportive environment for responding to HIV/AIDS Source: USAID/Ethiopia Integrated Strategic Plan, February, 2004 (revised draft)

Results Frameworks: basis for programmatic process Agreement within operating unit and donor expectations Selecting appropriate indicators for the M&E system Use performance information for management decisions Analyzing and reporting on program results

Building a Results Framework Group work Using your conceptual framework as a basis Discuss and draw a strategic framework for your program After 45 minutes, a member of each group will share the framework with all participants

M&E for HIV/AIDS Programs Logical frameworks M&E for HIV/AIDS Programs

LFA Stakeholder analysis Problem analysis Objective analysis selection of preferred implementation strategy

Logical Frameworks-logic matrix Project Description Performance Indicators Means of Verification Assumptions Goal: The broader development impact to which the project contributes - at a national and sectoral level. Measures of the extent to which a sustainable contribution to the goal has been made. Used during evaluation. Sources of information and methods used to collect and report it. Purpose: The development outcome expected at the end of the project. All components will contribute to this Conditions at the end of the project indicating that the Purpose has been achieved and that benefits are sustainable. Used for project completion and evaluation. Assumptions concerning the purpose/goal linkage. Component Objectives: The expected outcome of producing each component's outputs. Measures of the extent to which component objectives have been achieved and lead to sustainable benefits. Used during review and evaluation. Assumptions concerning the component objective/purpose linkage. Outputs: The direct measurable results (goods and services) of the project which are largely under project management's control Measures of the quantity and quality of outputs and the timing of their delivery. Used during monitoring and review. Assumptions concerning the output/component objective linkage. Activities: The tasks carried out to implement the project and deliver the identified outputs. Implementation/work program targets. Used during monitoring. Assumptions concerning the activity/output linkage. Speaker Notes: A Logframe Matrix is a standardized table that summarizes the important aspects of a project. There are two examples of logical frameworks matrices in your handouts, and this slide is replicated there as well. Notice that the matrix contains all elements important to a program: goal, purpose, objectives, outputs (meaning the measurable results produced by the program), and activities that will be carried out to achieve the outputs. For each of these, indicators are specified and the means for verifying these is also listed. In the last column, assumptions made for each program element are also described.

Logical Frameworks-Log frame matrix Present a standardized summary of the project and its logic. Purposes: Summarizes what the project intends to do and how Summarizes key assumptions Summarizes outputs and outcomes that will be monitored and evaluated Other terms used: Logframe matrix Speaker Notes Logical frameworks are similar to results frameworks in that they are program management tools that manage by results. USAID introduced the use of logical frameworks to international development work but is now promoting the results framework. Other donor agencies including Australia AID, DFID, GTZ, and CIDA continue to use logical frameworks. Logical frameworks logically summarize what the project intends to do and how, the key assumptions, and outputs and outcomes that will be monitored and evaluated. A Logframe Matrix is a standardized table that summarizes the important aspects of a project.

M&E LOGIC FRAMEWORKS or Logic model Logical frameworks are diagrams that identify and illustrate the linear relationships flowing from program inputs, processes, outputs, and outcomes. Inputs or resources affect Processes or activities which produce immediate results or Outputs, ultimately leading to longer term or broader results, or Outcomes. Speaker Notes Logical frameworks are diagrams that identify and illustrate the linear relationships flowing from program inputs, through processes, outputs, and leading to outcomes. Inputs, or the program’s resources, provide the fuel for Processes, or the program’s activities. These processes produce immediate or direct Outputs or Effects; in other words, the program’s activities yield some intermediate results. These Outputs lead to long-term or broader, overall results, or the program’s Outcomes. A logical framework is a useful tool for identifying and illustrating operational factors important to achieving a successful ultimate outcome. It can help identify linkages and key processes, highlighting the relationship between resource allocation and success, as well as where problems may lie if goals are not being achieved. Logical frameworks provide a streamlined perspective on the most critical processes contributing directly to program outputs and outcomes, and clarifies the linear relationships between program decisions, activities, and products. Additional Background “Log frames” are used in many organizations, and participants may have specific notions about the labels that should be affixed at each level (e.g. outputs versus effects). It is unlikely to be a productive use of time to debate these semantics from the various organizational perspectives. The point here is to explain and illustrate the general idea and usefulness of logical frameworks so that participants see that they can help clarify the linear flow connecting resources, allocation decisions, and program effectiveness.

Purposes: Provides a streamlined interpretation of planned use of resources and desired ends Clarifies project/program assumptions about linear relationships between key factors relevant to desired ends

Logic Frameworks - Components Inputs Program resources like staff, curricula, money Activities What the program does like outreach, training, testing Outputs Deliverables of program like people trained or tested Outcomes Program results like change in knowledge, service use Impact Long term change in health status like decreased HIV incidence Source: CDC Evaluation Guidance Handbook at: http://www.cdc.gov/hiv/aboutdhap/perb/guidance/chapter4.htm

Log frame logic model IMPACT Inputs Activities Outputs Outcome

Logic Models: Training INPUT PROCESS OUTPUT OUTCOME IMPACT Speaker Notes This logic model presents a straightforward view of a project designed to reduce population morbidity due to untrained or unskilled health care providers. As you can see, it does not try to account for all of the factors that may be influencing operation and results like a conceptual model, but instead focuses on the project’s activities and impacts. This narrow focus assists program managers and M&E planners as they clarify the direct relationships among elements of particular interest within a particular program effort. Also, unlike the LogFrame Matrix, no activities are listed here. Sometimes these models do include assumptions between inputs and processes, processes and outputs, outputs and outcomes, and outcomes and impact. Additional Background Logic models can usefully diagram the flow of program operations, while keeping the ultimate goals in view. Process indicators are sometimes useful for monitoring, but may provide most insight if used in the context of an evaluation -- to pinpoint glitches or breakdowns within the project, for instance, if objectives are not being fully met. Develop clinical training curriculum Conduct training events Practitioners trained in new clinical techniques Increase in clients served by (newly) trained providers Declining morbidity levels in target population

; INPUT PROCESS OUTPUT OUTCOME IMPACT Logic Model: Portion of model for tuberculosis control relating to increasing demand for quality services ; INPUT Human and financial resources to develop and print educational brochure PROCESS Distribute brochure to health facilities Meet with physicians to promote distribution of brochure OUTPUT Brochure distributed to clients of facilities OUTCOME Increased customer knowledge of TB transmission and treatment Increased demand for quality TB services IMPACT Decreased TB infection, morbidity and mortality Speaker Notes This logic model shows how it relates to the results framework for the same program. Each activity related to a results framework could have its own logic model. In this example, the input is the human and financial resources to design and print and educational brochure about tuberculosis. The process, or program activities related to this will be the distribution of the brochure and making sure the physicians know about the brochure and promote it among their clients. The anticipated output is that the brochure is distributed to clients of the facilities. The outcome is where the model relates to the results framework. The aim of these activities are to increase customer knowledge of TB transmission and treatment, the Sub-Intermediate Result, which will lead to an increased demand for quality TB services, the Intermediate Result. The desired public health impact of the program is that TB infection, morbidity and mortality will be decreased as a result.

Logic Models Voluntary Counseling and Testing (VCT) INPUTS PROCESSES Problem Statement: HIV infection rates continue to rise, underscoring the importance for people to know their serostatus, develop personalized risk-reduction strategies, and access care and treatment services. INPUTS PROCESSES OUTPUTS OUTCOMES IMPACTS Financial Resources Supervision and training for VCT personnel Clients (HIV+ and -) develop & adhere to personalized HIV risk-reduction Risk behaviors decreased HIV transmission rates decreased People know their HIV status Demand for Services Provide pre-test counseling Increase in care, prevention, and treatment services for HIV+, HIV-, and discordant couples Infrastructure HIV+ Clients develop & adhere to personalized HIV care, support, and treatment plans Provide HIV testing HIV incidence decreased VCT MIS** HIV+ people are referred to appropriate services VCT protocols, guidelines, and training documents** Provide post-test counseling Refer HIV+ clients to appropriate services (PMTCT, care and support, treatment) HIV morbidity & mortality decreased Counseling and Testing Personnel* Health outcomes of HIV + improve Speaker Notes Within a program, there are several activities with their own inputs and outputs. Collectively the outputs of the activities contribute to the program outcomes and impacts. In some cases the output of one program activity could be an input for another activity. This is an example of a logic model for a VCT Program. the inputs and processes and outputs of VCT service delivery are presented in a logically connected manner. Arrows are used to show connections. In order to make this easier to follow, in this example, only arrows connecting to the process of HIV testing are depicted. From the inputs column financial resources, demand for services, infrastructure, a VCT information system, VCT protocols and guidelines, testing personnel with skills in phlebotomy and testing are joined with the group of people who have gone through the process of counseling. The output of the VCT service requires another step in the process: people who have been tested need to be counselled about their results. The output of the VCT process is that people know about their status. In some cases programs will develop logic models for each activity. For example, if an activity is to develop guidelines, the output of that activity is the guideline, which is an input in this overall logic model for VCT service delivery. Client records are available Continuity of care is available HIV test kits Maintain patient records Referral system for prevention & Tx services** Service reports are produced Program and services are improved Complete reporting requirements

Frameworks for M&E Planning Purposes: clarifying assumptions, goals, and interrelationships between factors relevant to the project or program defining objectives selecting activities defining levels of performance and desired results in terms of planned activities and realistic, objective impacts Monitoring and evaluation plans incorporate: program managers’ assumptions and objectives, in a given context a schematic design displaying the directional linkages between key program elements and/or planned results, and other relevant factors Speaker Notes Now that we have discussed different types of frameworks, let’s turn back to the purpose of designing M&E frameworks. One main purpose is the clarification of the program’s assumptions, goals, and the known or expected relationships among program and environmental factors that may affect the effectiveness of the activities or the outcome of the intervention. Another important purpose of M&E framework design is to define clear levels of results that should occur as the intervention unfolds. These should be realistic and objective impacts that can be measured and assessed. M&E frameworks rest on the assumptions and objectives of the program within its operating environment. Drawing on those expectations, the M&E framework provides a schematic design showing how various relevant factors, results, and overall outcomes are linked. Additional Background Participants should be clear on several points: that the process of framework design is both useful in itself, to clarify project and partner understandings of the key assumptions and other necessary elements for a successful intervention, and that it is essential in order to develop an appropriate and useful M&E plan. Without developing a consensus on the foundations and context of the program and its success, monitoring and evaluation cannot be managed in a coherent or comprehensive way in order to contribute to effective management and the achievement of that success.

M&E FRAMEWORKS Designing an M&E framework assists in determining: Appropriate program elements to measure Appropriate indicators and data Appropriate methodology Speaker Notes In sum, we have covered a number of ways in which the development of certain types of frameworks can be useful for generating a better understanding of the ways elements of the program interact with each other and with relevant factors in the context or project environment. This understanding can greatly improve the development of an appropriate and useful Monitoring and Evaluation plan. Designing an M&E Framework clarifies appropriate program elements to measure, assisting in the identification of appropriate indicators and data for measuring them, and the appropriate methodology to be used in monitoring and evaluating program success. The themes of indicators, data, and methods will be central to the topics introduced and discussed further in Module 3. Additional Background The main theme of this Module has been the usefulness of Frameworks for developing a clear and useful understanding of the expectations embedded in project design. By using frameworks to draw out the elements critical to program success, we can identify the points where some monitoring and/or evaluation will be crucial in order to measure and understand where activities may be effective or may be not as effective as managers had hoped. The next Module shifts the focus from the significance of this broader understanding and analysis to the significance of the more mundane details of M&E -- indicator specifications and definitions, the details and data of their operationalization through metrics, and some of the caveats important for appropriate use of data, or methods.

Summary of Frameworks Type of Framework Brief Description Program Management Basis for Monitoring and Evaluation Conceptual Interaction of various factors Determine which factors the program will influence No. Can help to explain results Results Logically linked program objectives Shows the causal relationship between program objectives Yes – at the objective level Logic model Logically links inputs, processes, outputs, and outcomes, Shows the causal relationship between inputs and the objectives Yes – at all stages of the program from inputs to process to outputs to outcomes/ objectives Speaker Notes: We have covered four types of frameworks.  These four frameworks are used to inform programming and for monitoring and evaluation of the program that is implemented.  The conceptual framework puts the health problem in to context and either a results or logical framework is used to understand the interrelationships between factors relevant the program. USAID funded programs tend to use results  frameworks but many others, such as DFID and UN programs, use logical frameworks. Programs should use the type of frameworks that best suit them.  Logic models help to show the logical connections between the inputs and processes and outputs of an activity and its link to the programs objectives (outcomes) and goals (impact).

References Measure : A trainers guide to the fundamentals of Monitoring and Evaluation for population, health, and Nutrition Programs. 2002. Carolina Population Centre. Chapel Hill. NC. AusGuide. The Logical Framework Approach. http://www.ausaid.gov.au/ausguide/ausguidelines/1-1-1.cfm Bertrand, Jane T., Magnani, Robert J, and Rutenberg, Naomi, 1996. Evaluating Family Planning Programs, with Adaptations for Reproductive Health, Chapel Hill, N.C.: The EVALUATION Project. Global AIDS Program. (2003) Monitoring and Evaluation Capacity Building for Program Improvement Field Guide, Version 1. U.S. Centers for Disease Control and Prevention, Atlanta, GA. Marsh, David. 1999. Results Frameworks & Performance Monitoring. A Refresher by David Marsh (ppt) http://www.childsurvival.com/tools/Marsh/sld001.htm Tsui Amy. 2004. Frameworks (ppt). Presented at the Bill & Melinda Gates Summer Institute. John Hopkins, Townson. Maryland. Tsui, Amy. 1999. Frameworks (ppt). Presented at the Summer Institute, University of North Carolina, Chapel Hill. Bloom S. 2007 Lecture notes . Asia HIV M&E Slides 2007

Activity Return to small groups from previous activities Develop Logical model for one of the activities of your program. After 30 minutes, a member of each group will share one of the logical frameworks Speaker Notes Get back into your small groups for these framework activities. Now you will practice developing logic models. Select one activity for which you will develop a logic model. Then, follow the inputs involved in that activity through the logic model. You will have 20 minutes for this activity, and if there is time, each group will share one framework with the larger group. I will let you know when 10 minutes remain. At the end of this activity, you should have one logic model written out on the large paper to be put on the wall. As you are completing this activity, be sure to refer back to the conceptual framework and the results or logical framework your group developed and to notice the relationships between the different types of frameworks. Additional Background Monitor the groups to make sure they are progressing and to answer questions. Activity Recommended materials for this activity are: Large piece of poster board or flip chart paper, one per group Thick marking pens, each group should have pens from the previous activity