1. Network of Oregon Watershed Councils
2004 – years and counting! 1

2. Mission: Support the work of Oregon’s watershed councils to enhance watershed health and benefit their local communities 2

3. Vision:
Increasingly strong, resilient watershed councils sustaining healthy land, water, and economies
Pride of our local communities and model for our country/world
We help councils be more effective individually - and further watershed health collectively 3

4. Strategic Plan 2014 - 2017 External: Internal:
Grow our impact through partnering and peer to peer
Increasing understanding of councils and their work among policy makers, funders, partners
Internal:
Build our leadership
Enhance our staff capacity
Expand our revenue 4

5. 5

6. Oregon Conservation Partnership6

7. Network is the fiscal agent for the Oregon Conservation Partnership7

8. Network is the fiscal agent for the Oregon Conservation Partnership
Partnership is funded by OWEB (total = $145K this year) 8

9. Network is the fiscal agent for the Oregon Conservation Partnership
Partnership is funded by OWEB (total = $145K this year)
Covers overlapping functions of all partners and half Exec. Director salaries/benefits 9

10. Poll: How are you accessing this webinar?10

11. Where else does NOWC get its funding?
Dues- about $5,000/year currently
Mini contracts with councils
Contract with OACD for .25 FTE of their partnership work ( ) 11

12. In conservation, it used to be enough to report on what we were doing12

13. “We met with 12 landowners and signed 4 agreements”
“We partnered with OSU Extension to conduct watershed education for 400 students at outdoor school” 13

14. Or, measuring and reporting on biological indicators14

15. “We restored ____ miles of riparian habitat”
“We removed ____ acres of invasive plants” 15

16. Standard way to describe our impact:Or
Standard way to describe our impact:
Define biological indicators
Collect data
Analyze data
Write up results 16

17. The conservation community does not have a long history of demonstrating impact of its work
We often have not linked the indicators we observe to our interventions 17

18. Often there is reporting on indicators with little or no consideration of how a project has contributed to the status of the watershed 18

19. If we think of ourselves as “agents of change,”
We expect something to be different as a result of our work.
We carry around assumptions about the things we choose to invest in
If we do ‘this’….. then we believe/trust/hypothesize that ‘this’ will happen 19

20. This set of assumptions are what is called our “theory of change”
We have a theory that if we do X, it will lead to Y, and that will lead to Z.
Sometimes these assumptions are based on research. Other times they are a theory.
Often these assumptions are articulated, sometimes implicit. 20

21. This set of assumptions are what is called our “theory of change”
We have a theory that if we do X, it will lead to Y, and that will lead to Z. 21

22. what we hypothesize will happen as a result
A logic model is a graphic that shows (makes explicit) our underlying assumptions about the link between
what we do
and
what we hypothesize will happen as a result 22

23. The use of logic models dates to late 1960s
Now in widespread use in education, social services, health care sectors.
Public Sector Non-Profit Sector Private for-profit Sector International agencies Many funders now require them in grant proposals
Let’s not think that this Logic Model is brand new. Actually, the concepts have been around since the late 1960’s in the writings of Suchman, 1967 and Wholey’s evaluability assessment model.
It has come to the forefront again, and is being developed and applied in a variety of settings as a result of a variety of factors:
Private sector: part of total quality management and performance measurement movement
Public sector, the GPRA has moved all federal agencies to focus on results and link investments to results, not just activities.
Non-profit sector is concerned with improving programs to produce valued impacts with the United Way being a frontrunner in outcome measurement using the logic model.
International programs. The players in the international arena for a long time have used variations of a logic model. The Log Frame of the US Agency for International Development of the 1980’s is a historical precedent to the current logic modeling discourse.
And, professional evaluators have played a prominent role in using and developing the logic model. This is why it is often called an ‘evaluation framework.’ This is a result of evaluators being asked to evaluate impact and finding, too often, that programs didn’t exist, or weren’t being implemented in a way that would achieve the expected impact. Consequently, evaluators began working with programmers to lay out the logic of programs. We see the outgrowth particularly in Chen’s theory-driven evaluation (1990) and Weiss (1997) theory-based evaluation.

24. Logic model is also called…
Model of change
Conceptual map
Outcome map
Program logic
Theory of change model

25. A simple example from education:
Logic model: A series of ‘if-then’ relationships that express a program’s or organization’s “theory of change”
A simple example from education:
IF then
IF then
IF then
IF then
IF then
We invest time and money
We can provide tutoring 3 hrs/week for 1 school year to 50 children
Students struggling academic-ally can be tutored
They will learn and improve their skills
They will get better grades
They will move to next grade level on time
Program objective: increase % of kids who move on to next grade level on time

26. INPUTS OUTPUTS OUTCOMES What we do What we invest What results
Program investments
Activities
Short
Medium
Long-term
What we invest

27. INPUTS OUTPUTS OUTCOMES What we do What we invest What results
Program investments
Activities
Short
Medium
Long-term
What we invest
What they learned

28. INPUTS OUTPUTS OUTCOMES What we do What we invest What results
Program investments
Activities
Short
Medium
Long-term
What we invest
What they learned
What they did

29. INPUTS OUTPUTS OUTCOMES What we do What we invest What results
Program investments
Activities
Short
Medium
Long-term
What we invest
What they learned
What they did
Longer term impacts of what they did
Often hypothetical

30. Simple logic model for landowner outreach
INPUTS
OUTPUTS
OUTCOMES
Program investments
Activities
Short
Medium
Long-term
Workshops, demonstrations and newsletters for landowners about benefits of streambank fencing and riparian buffers
Increased knowledge about benefits of fencing and buffers
Landowners restrict cattle access with fencing and install buffers
State lottery funds
State agency funds
Local ag experts
Volunteers
Water quality indicators

31. Invasive species education program example
Invasive species in Oregon will decline or not increase
These landowners will gain knowledge and take actions to reduce invasive species
Educational program will be provided to participating landowners
Let’s apply this to a typical Extension example
Council or district staff invest staff and volunteer time and $$

32. Long-term outcome: Conditions/Impacts: changes in environmental, social , civic, or economic condition…
Halt the decline in Mtn. Caribou numbers in 7 years
Snowmobilers reduce or halt disturbance of Mtn. Caribou in Winter Range. (change in snowmobiler practices)
Medium-term outcome: Action: changes in behavior, practice, decisions, policies, social action…
Increased understanding and appreciation of Mtn. Caribou biology/increased knowledge of adverse effects of snowmobiling and alternatives locations to recreate.
Short-term outcome: Learning: changes in knowledge, aspirations, attitudes, skills, perceptions, motivation, opinion…
-Focus groups to learn snowmobiling psychology (research)
-Workshops to educate snowmobilers re: effects on Mtn. Caribou and alternatives
-Budget of $7,500
Outputs/Inputs:
Activities: what we do and what we put into it
A simple logic model for snowmobiler workshops designed to reduce adverse effects of snowmobiles on Mountain Caribou

33. -The world is not this simple or linear
Limitations!
-The world is not this simple or linear
-Your logic model will undoubtedly be more complex with multiple interventions to achieve the same desired outcome 33

34. And……. the Contribution vs. Attribution question

35. Write something in your question box to share!
Comments so far?
Reactions so far?
Questions so far?
Write something in your question box to share! 35

36. What does a logic model look like?
Graphic display of boxes and arrows; vertical or horizontal
Relationships, linkages
Any shape possible
Circular, dynamic
Cultural adaptations; storyboards
Level of detail
Simple
Complex
Multiple models
Multi-level programs
Multi-component programs
Logic model is graphic display
Any shape is possible but importance lies in showing expected causal connections
Level of detail: simple, complex
Multiple models – families of models for multi-level programs; multi-component programs
Reinforce that a logic model needs to be:
visually engaging,
appropriate in its level of detail,
easy to understand,
reflective of the context in which the program operates.

37. Multi-component – a way to describe and link different activities within a comprehensive initiative.

38. Bennett’s Hierarchy of Program Effectiveness “Counting heads” level
Measures impact of program on long-term goals
Level 7
End results
Level 6
Practice change
Change in behaviors or practices
Level 5
KASA change
Change in Knowledge, Attitudes, Skills, and Aspirations (KASAs)
Level 4
Reactions
How did participants react to the program?
Level 3
Participation
“Counting heads” level
Who participated and how many people?
Level 2
Activities
What activities participants were involved in?
Level 1
Inputs
What resources were dedicated to the program?

39. How logic models can help with evaluation
Staff
Money
Partners
Assess parent ed programs
Design & deliver evidence-based program of 8 sessions
Parents increase knowledge of child dev
Parents better understand their own parenting style
Parents use effective parenting practices
Improved child-parent relations
Research
Facilitate support groups
Parents gain skills in effective parenting practices
Parents identify appropriate actions to take
Strong families
Parents
of 3-10
year
olds
attend
Reduced stress
To what extent is stress reduced? relations improved?
To what extent did behaviors
change? For whom? Why? What else happened?
To what extent did knowledge and skills increase? For whom? Why? What else happened?
Did all parents participate as intended? Who did/not not? Did they attend all sessions?...support groups? Level of satisfaction?
Were all sessions delivered? How well? Do support groups meet?
What amount of $ and time were invested?

40. Parent Education Example: Evaluation questions & indicators
Parents increase knowledge of child dev
Staff
Develop parent ed curriculum
Parents identify appropriate actions to take
Reduced stress
Parents better understand their own parenting style
Money
Parents of 3-10 year olds
Deliver series of 8 interactive sessions
Partners
Improved child-parent relations
Parents gain skills in new ways to parent
Parents use effective parenting practices
Research
Facilitate support groups
Parents gain confidence in their abilities
Strong families
EVALUATION QUESTIONS
What amount of $ and time were invested?
How many sessions were held? How effectively?
#, quality of support groups?
Who/how many attended/did not attend? Did they attend all sessions?
Supports groups? Were they satisfied – why/why not?
To what extent did knowledge and skills increase? For whom? Why? What else happened?
To what extent did behaviors
change? For whom? Why? What else happened?
To what extent is stress reduced?
To what extent are relations improved?
Explain how fits with collecting data over course of program; integrate into planning and program delivery
INDICATORS
#,% demonstrating increased knowledge/skills
Additional outcomes
#,% demonstrating changes
Types of changes
#,% demonstrating improvements
Types of improvements
#,% attended per session
Certificate of completion
# Staff
$ used
# partners
# Sessions held
Quality criteria

41. So, why bother? Why learn this?
“This seems like a lot of work.” “Where in the world would I get all the information to put in a logic model? “I’m a right brain type of person – this isn’t for me.” “Even if we created one, what would we do with it?”

42. Benefits: What we are finding:
Provides a common language
Increases understanding about program
Guides and helps focus work
Leads to improved planning and management
Increases understanding about program and how different people view the program
Wonderful technique for starting a conversation
Different stakeholders may have different view of program – all may have the same end goal in mind, but different strategies for getting there. Way to arrive at consensus or understanding
If this is our program, what do we need to measure
Funding – more favorable if you can clearly demonstrate how and why they will succeed

43. Enhances teamwork
Motivates staff
Helps to identify important variables to measure for evaluation
Helps communicate what program is about
More and more is required!

44. Testimonials
“Wow – so that is what my program is all about” “I’ve never seen our program on one page before” “I’m now able to say no to things; if it doesn’t fit within our logic model, I can say no. “ “I can do this” “This took time and effort but it was worth it; our team never would have gotten here otherwise.” “It helped us to think as a team – to build a team program vs. an individual program.”

45. Resources and thanks
University of Wisconsin Extension Program Planning and Development -Logic model templates and examples -on-line course Handbook for Developing Watershed Plans to Restore and Protect Waters, EPA 841-B , March 2008.

46. For more information on logic models
Shawn Morford, Network of Oregon Watershed Councils
THANK YOU!