1. Orientation for Reviewers March 23, 2018
Citizen Review
Orientation for Reviewers
March 23, 2018
Cover slide

2. Goals for Today Understand RUW Goals Understand Community Impact
Understand the RUW Funding Process
Understand Logic Models /Measurement Plans
Know how to access and review applications
Understand how to evaluate applications with RUW goals and objectives in mind

3. To improve lives by mobilizing the caring power of our community.
RUW Mission
To improve lives by mobilizing the caring power of our community.
First, let’s talk about
RUW Goals
Community Impact
To improve lives by mobilizing the caring power of our community.

4. RUW Vision
A community where individuals and families achieve their potential through education, financial stability and healthy living.
Our mission is an over-arching statement. We wanted to refine our efforts, so after consulting with the community, we determined that our goal is that More households emerge from poverty and achieve greater financial stability. We use this, in combination with our mission, to guide our decisions, efforts and activities.

5. RUW Focus Areas
To address the issue of poverty, we are focused on Education, Financial Stability, and Healthy Living – specifically programs and activities that can help us achieve our vision.
Education gives you a shot at a better job, with better income. Financial Stability comes from income and good financial practices. And Healthy living practices can help you avoid time off from work, chronic and expensive conditions, and challenges that arise from risky health behaviors.

6. RUW Outcomes
We take it one step further – again with community input, we have identified 10 Outcomes. We see these as steps toward addressing the root causes of poverty.

7. Community Impact Create lasting change in community conditions
Improve lives
Focus on Outcomes vs Outputs
Address root causes
How are we going to achieve all this?
One major step in reaching our goals has been to become a Community Impact United Way. Instead of divvying up the dollars we raise, we seek out and fund programs that:
Create lasting change in community conditions
Improve lives
Focus on Outcomes vs Outputs
Address root causes

8. The RUW Funding Process

9. Community Impact Fund Timeline
Membership
August/September
Nonprofits apply annually for RUW Membership.
Agencies are evaluated for fiscal health and non-profit best practices
Site Visits
December/January
Funded Programs host staff and volunteers
Progress toward goals is presented
Recommendations are made for future funding consideration
Application to CI Fund
January/February
Agencies receive training in January
Agencies receive technical support in January and February
Applications submitted March 2
Applications audited
Agencies revise by March 16
Citizen Review
March/April
Citizen Review Panels evaluate programs and generate questions
Staff seek clarification from agencies
Agencies respond
Reviewers reconvene and recommend priority level for each program
August/September
Nonprofits apply annually for RUW Membership, when we Agencies are evaluated for fiscal health and non-profit best practices. Citizen Reviewers are not asked to assess the agency – you are asked to focus on the program.
December/January
Funded Programs host staff and volunteers where Progress toward goals is presented
In some cases, we use the visit to make staff Recommendations for future funding consideration
January/February
Agencies receive training and technical support in January and February
Applications submitted March 1 Applications audited Agencies revise by March 13
March/April
Citizen Review Panels evaluate programs and generate questions
Staff seek clarification from agencies
Agencies respond
Reviewers reconvene and recommend priority level for each program

10. Funding Decision Process
Citizen Review
Staff and Volunteers
Committee Review
Staff, Volunteers and Focus Area Champions (Board Members)
Board Review
VP of Community Impact and full Board
RUW Funding decisions are finalized by our Board of Directors. But I cannot emphasize enough how important the Citizen Review step is. Board members do not read all 50 or so program applications. They rely upon your recommendations to determine where our Community Impact dollars go.

11. Citizen Review Volunteers: Panels
Participants
Structure
Donors
Campaign Coordinators
Volunteers
Advocates
Board Members
Campaign Companies
People who are invested in our Community!
RUW Staff Liaison
5-7 Panel Members
1 Panel Representative
Who are our volunteers?
People who are invested in our Community!
What is the structure of each panel?
Staff
2-4 Panel members
1 Panel rep.

12. Reviewing Applications
Login screen - Accessible at or scroll down to any page

13. Reviewing Applications
Login is your address
New reviewers: password is changeme1
Returning reviewers, if you’ve forgotten your password, use the ‘reset password’ option or reach out to me.
Call for help if needed!

14. Volunteer agreement
Essentially this says that you’ll participate with the best interests of RUW donors and RUW goals in mind.

15. Conflict of interest – list of our agencies
Conflict of interest – list of our agencies. Please read it over and if you or an immediate family member serve as staff, consultant, board member etc for any of our member agencies select ‘yes’.

16. If you select yes, you will come to a screen that has a drop down list of our agencies – select the agency and describe the conflict. You can add as many of these as needed.
There are non-member agencies on this drop-down list. You can begin to type the member agency name to jump to it on the list.
Once you are done adding conflicts, click ‘home’

17. Reviewing Applications
Click Full Funding Application. This will present a list of applications for your review.

18. You will see a list of the programs to which you’ve been assigned.
Progress features – note that the system always tries to tell you where you are in the process.
Evaluate – Open an application by clicking on it’s name or the “Evaluate” linke
Print – Print if you wish by clicking on “Print”

19. When you open a program to read, it will display with the evaluation form as an overlay. Use the Pink “Evaluate” tab to tuck it away.

20. Reading the Application
Tuck the Evaluation form away and read through once
Application displays as one long document, but you can jump around to different sections
You can print the application if you wish
Just a few notes about this interface: Tuck the Evaluation form away and read through once
Application displays as one long document, but you can jump around to different sections
You can print the application if you wish

21. Use the drop-down list to jump to another section

22. Program Applications: Content
Program Cover Sheet
Service (Demographic) Data
Program Financial Information
Budget / Service Narratives
Program Narratives
Success Story
FY18 Logic Model & Measurement Plan
Last Full Year Logic Model & Measurement Plan**
Elements of the application for each program

23. The Evaluation Form Demographics Financial Information
Program Information
Measurable Impact (Logic Model)
Reviewer Recommendation

24. Each section has a few questions with a scale (Strongly Agree, agree, neutral, etc) and an area for comments, notes, questions you’d like to bring up during the discussion.

25. Last Full Year Logic Model & Measurement Plan
Programs that were funded in FY17 used the Last Full Year LMMP (funded programs) form (this is actually the year end report they submitted at the end of FY17.)
Programs that did not receive RUW funding were asked to create and submit the Last Full Year Logic Model and Measurement Plan (unfunded) form
Content of the application
Program Narratives
Service (Demographic) Data
Budget
Budget / Service Narratives
FY17 Logic Model & Measurement Plan
Last Full Year Logic Model & Measurement Plan ** -Success Story

26. Evaluation Guidance Is there a need for this program in our community?
Community Need
Target Population
Does the application present a clear picture of what the program will do?
Program Narratives
Logic Model Inputs, Activities and Outputs
What should you be looking for as you read, and where can you find these items/
Is there a need for this program in our community?
Community Need
Target Population
Does the application present a clear picture of what the program will do?
Program Narratives
Logic Model Inputs, Activities and Outputs

27. Evaluation Guidance
Are you confident that the agency can fulfill their program goals?
Program Structure
Budget
History
Will the program have measurable impact?
Logic Model and Measurement Plan
Does the program impact align with RUW goals and outcomes?
Overall application
Are you confident that the agency can fulfill their program goals?
Program Structure
Budget
History
Will the program have measurable impact?
Logic Model and Measurement Plan
Does the program impact align with RUW goals and outcomes?
Overall application

28. View / Ask Questions
At any point, you can use this to enter a question that you might have FOR THE AGENCY about the program.
If we can’t answer that question as a group in our review meeting, this may eventually be released to the agency or it may be rewritten.
You can use the narrative spaces on the evaluation form to note things that you’re curious about, want to explore in discussion, etc.

29. Review elements of Q&A screen

30. Logic Model and Measurement Plan
The RUW application combines the Logic Model and Measurement plan into one form.

31. About Logic Models What is a Logic Model? Elements of a Logic Model
Inputs
Activities
Outputs
Outcomes
What is a logic model
Used for program planning. Ideally, it’s like a business plan for a business.
Good programs should have a planning document that they use to plan, execute, and evaluate their programs
Inputs are the things you need to pull off the program. They will appear as a list or paragraph on the application depending upon how they were entered.
Example: Full time program manager, Program Assistant, 3 Site Coordinators, 15 Volunteers, School Supplies, After School Snacks
Activities are the things you do as you deliver the program. They will appear as a list or paragraph on the application depending upon how they were entered.
Example: Provide homework assistance, Teach character building lessons, Organize and execute community service projects
Outputs are the things you count: They will appear as a list.
Example: After School Sessions, Number of children served, After school meals distributed
Outcomes are what you expect to achieve through the program. These are the changes in behavior, knowledge or situation that come about as a result of the program. We ask for a minimum of 1 and no more than 3 outcomes.

32. Inputs & Activities

33. Outputs

34. About Measurement Plans
What is a Measurement Plan?
Elements of a Measurement Plan
Outcomes
Indicators
Relevant Client
Measurements
Data Sources and Methods
Having a plan for a program is the first step. All good programs also measure their successes and focus their measurements on what it is they hope to achieve. .

35. The RUW Requirements Minimum of 1, maximum of 3 Measurable Outcomes
1 to 3 Indicators of success for each outcome
Relevant Client Description – who has a chance at the goal?
The Measurement
The # of clients that will attempt to reach the goal
The # of clients that will achieve the goal
RUW does not ask for the full logic model and measurement plan from applicants. We ask them to submit their best 1-3 outcomes – those that best demonstrate the impact they will have related to our mission, vision and goals.

36. Outcomes and Indicators
Outcomes are the goals. For RUW funding, the outcome should be a change in Behavior, Knowledge, or Situation. Example Program: After School Tutoring Outcome: Program participants will gain competence in math

37. Outcomes and Indicators
Indicators are how the program will know they’ve succeeded in reaching their outcome. Indicator: Math Grades will be maintained at "C" or above level

38. Outcomes and Indicators: Example
By the end of the workshop, Program participants will gain competence in math and we’ll know we achieved this outcome because Math Grades will be maintained at "C" or above level. .
This is how I like to think of this. We will [Outcome] and we’ll know we’ve succeeded because [indicator]
By the end of the workshop, Individuals will learn how to reduce their risk of Stroke, and we’ll know we achieved this outcome because Individuals will be able to identify at least 4 strategies to reduce their stroke risk.

39. Relevant Clients
Relevant clients are the people in the program who have a shot at demonstrating they have achieved the Outcome.
Could be all program participants
Could be a subset that focus on a particular aspect of the program
Might have a benchmark or participation requirement
For example
In our sample plan, only those students who consistently attend the program will be measured for success. It would not be fair to the program to count students who fail to participate.

40. Outcome, indicator, relevant client and measurement

41. Measurements
This is the language you will see in the application. For each indicator, we ask them to predict how many relevant clients will be evaluated for the indicator and how many will achieve the outcome.
All of our measurements are based on the individual, not the activity. This should always be the number of individuals who pursue and achieve the goal.

42. Data Sources and Methods
Data Sources – where data comes from (clients, staff, community, observers)
Methods – how data is gathered, organized, evaluated (case reviews, surveys, staff observation notes)
Each outcome has Data Sources and Methods – these apply to all the indicators under that outcome.

43. Data sources
Data methods
Term: how long before they expect to be able to measure this outcome.

44. Elements of the evaluation form (sections)
Save feature (Save, Save and mark completed, save and close, close without saving)
Print
Status – once you start editing, it will show as “In Progress”
When you are done evaluating, click “Save and Mark as Completed”. Eval is still editable at this point so you can return to it if needed.

45. Submitting the Evaluation.
When you are satisfied, you select the save and mark as completed button, then use the submit now button. Evaluations can be submitted as you complete them (one-by-one).
Once submitted, you can not edit the form but you can read it.

46. Q&A
Use this feature to submit questions that you would like to as the agency about the program you are reviewing.
After our first panel meeting, we will discuss all the questions and, if needed, release your question to the agency for a response.
They must respond within one week.

48. Timeline Programs are open for review now.
Please review and evaluate no later than 24 hours prior to your first meeting.
Your evaluations will be summarized and ready for your panel meeting.
Submitting your evaluations allows me to create a report showing all the responses. This cuts down on the time we spend at the review meeting reading off our score numbers.