Program Evaluation for Nonprofit Professionals Unit 4: Analysis, Reporting and Use

Analysis When analyzing the data, you will look at each piece of information you collected, and makes sense of what it tells you.

Quantitative Analysis Using statistical methods to describe and summarize components of your data. Avoid buying software. Microsoft Excel will usually be sufficient. Have a plan – who will do the analysis? What specifics do you want to know? Allow plenty of time for analysis. Have someone review the work.

Most Commonly Used Statistics Frequency – the number of times an event occurs. Example: You asked survey respondents if they would recommend your organization’s services to a friend. 36 people said yes, and 12 said no.

Most Commonly Used Statistics Percentage – expresses a number in a per one hundred ratio. Example: You asked survey respondents if they would recommend your organization’s services to a friend. 36 people said yes, and 12 said no. 48 responses 36/48 = 75% 12/48 = 25%

Commonly Used Statistics Ratio – a comparison between two things Example: You want to know how the number of graduates from your parenting class compares to the number of non-graduates. In 2011 you had 35 graduates and 56 non-graduates. Your ratio of grads to non-grads is 35/56.

Commonly Used Statistics Mean – an average of numbers Example: Average age of program participants. Median – the middle number in a list of numerically ordered values. Example: The mid age of program participants. Mode – the value that occurs the most often Example: Being able to so say if most program participants are in their 20’s or in their 40’s. Range – the difference between the smallest and largest numbers. Example: Participants in your program have an age range of

Quantitative Analysis Advantages Precise number values are great for some audiences (especially government and policy) Answers the basic evaluation questions of “what” and “how many” Very little bias is involved Disadvantages Numbers can’t communicate context. Doesn’t tell why something happened. More training may be needed for advances statistics.

Qualitative Analysis Analysis of non-numeric information. It can add further dimension to the numbers. Such as: Open-ended survey responses Interview data Observation data

Qualitative Analysis Steps 1.Review the data. Understand what you have. 2.Organize your data so that it will be easy to sort and use. 3.Code the data. Label themes that correspond with your outcomes. 4.Interpret your data. Attach meaning based on what you know about your program.

Qualitative Analysis Advantages Provides context Fills in information that your quantitative data cannot provide Adds depth to your knowledge of your program Disadvantages Can be time consuming Findings will not be generalizable outside your program Reviewer interpretation introduces bias

Reporting The ways you share evaluation findings. Annual report Staff meetings Funder reports Community briefings Board meetings Grant applications

5 Steps to GREAT Reporting 1.Identify your audience. 2.What details will they want to know? 3.Be transparent about your conclusions and be able to back up each claim with evidence. 4.BE HONEST. People can tell when you overstate your impact.

5 Steps to GREAT Reporting 5. Be careful of language pitfalls: – “Significance” has a statistical meaning. – Random selection is also a statistical term. – It is very difficult and costly to claim causation. Better off to discuss your programs' contribution to change.

Evaluation Use This is the most important part! Get together and talk about what was learned in the evaluation, and what it means for improving your program.

Evaluation Use Gather people from different levels of the organization (program staff, dept. managers and executive leadership) to hear about the results. Discuss: 1.What does this information tell us about our program? 2.What improvements need to be made? How can we make them? 3.What strengths were identified? How can they be built upon? 4.What do we want to know in the next evaluation?