1. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Unit 7: Using your evaluation – Communicating, reporting, improving 1.Who 2.What (Content) 3.How (Format) 4.When

2. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation “Think like a wise man, but communicate in the language of the people.” − William Butler Yeats

3. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Why communicate? The proper function of evaluation is to speed up the learning process by communicating what might otherwise be overlooked or wrongly perceived… Success is to be judged by… success in communication… Payoff comes from the insight that the evaluator’s work generates in others. - L. J. Cronbach Cronbach, L.J. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass, p. 8.

4. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Plan with the end in mind.

5. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Types of communications - reports Written report - long, short, summary Success story Impact statement; spotlight Elevator story Press release Media appearance Public meeting Memo, email, fax, postcard Newsletter Personal discussion Bulletin, brochure Display/exhibit Audio/video presentation

6. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Communication plan WHO? WHAT? HOW? WHEN?

7. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation WHO - Potential Users County board/City council (elected and appointed officials) Funders Program Committee Participants Team / Staff Volunteers Collaborating agencies; partners Businesses; business groups Police – law enforcement Schools boards, parent-teacher organizations Community Church organizations State legislators Professional organizations You

8. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation WHO is the Audience? 1. 2. 3. 4. INTERNAL 1. 2. 3. 4. EXTERNAL Primary audience or secondary audience??

9. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation WHAT – will you say, report, communicate?? What does the audience care about? What do you want the audience to have? What type of information resonates with the audience: numbers, quotes, stories?

10. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation HOW will you communicate – what formats will you use? Report Impact statement Executive summary Personal discussion Oral presentation Photo display Press release Newsletter, bulletin Poster ??????

11. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation HOW – what format, style will you use?? Format depends on purpose and audience Written, oral Short, long How does the audience prefer receiving information? How does the audience best learn? What do you have the resources to do?

12. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation WHEN will you communicate? Quarterly? Opportunistically? When requested? ???

13. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Reporting outline follows your logic model

14. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Formal evaluation reports typically include… Abstract/executive summary Introduction −Purpose of the evaluation; key questions −Program background, description Methods/procedures −Data sources −Data collection procedures −Sampling −Limitations Results Discussion Conclusions/recommendations References Appendices

15. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Effective communications Tailor message to issue and audience Avoid jargon and technical terms Be clear, concise Use active voice Eliminate wordiness Check writing, grammar Be accurate, balanced, impartial Be timely Use graphics, quotes, photos, real stories Consult a communications specialist Write-rewrite-rewrite

16. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Myths One report is enough. People read written reports. Complex analysis and big words impress people. Oral reports have the same effect as written reports. Describing limitations weakens report. Everything should be reported. The audience knows why they are getting the report.

17. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Ingredients of a Good Success Story Involves human interest: catches your attention Demonstrates behavior change; action taken −change in practice that results in benefits to individual, community, environment... Uses words of/meaning for people involved Indicates profit to the individual, if appropriate Based on reliable, credible information Includes numbers and narrative Presents balanced, fair assessment

18. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Make sure the evaluation isn’t ignored Engage stakeholders in doing the evaluation: in the design, data collection, in analysis, interpretation of results… Get the information to the right people Address issues that people think are important Keep it in front of people Present it in time to be useful

19. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Discuss limitations Written reports: Be explicit about your limitations Oral reports: Be prepared to discuss limitations Be honest about limitations Know the claims you cannot make −Do not claim causation without a true experimental design −Do not generalize to the population without random sample and quality administration (e.g., <60% response rate on a survey)

20. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Reporting results to the media All Media: Avoid using too many statistics. Focus on the key points. For quotes, speak more globally about the issue. Always give the source and timeliness of your stats. It’s the “news peg.” Steve Busalacchi Director, News & Information Wisconsin Medical Society

21. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Reporting results to the media Radio and TV: Do not offer exact statistics – ear cannot track. “73.6% of respondents” vs. “Nearly three quarters of those surveyed” Don’t go into great detail. Have backup info ready. Steve Busalacchi Director, News & Information Wisconsin Medical Society

22. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Using graphics What do I want my audience to learn – remember – from my graph or table?

23. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Using graphics Title Clear units of measure Date(s) data collected Simple, straightforward design without “clutter” Font size 10 point or larger Explicit data source(s) Sample size, if applicable for the audience

24. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Graphs Bar graphs: show comparisons −Stacked bar −Multiple bars −Vertical, horizontal bars Pie charts: show parts of a whole Line graphs: show progress over time

25. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Bar graph Compare groups of client outcomes Compare points in time Each group you compare gets a “bar”

26. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

27. Horizontal Graph

28. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Vertical Graph

29. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Column Chart

30. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Pie charts: parts of a whole Useful for displaying proportions −Percentage of clients achieving the outcome −Percentage of participants who are boys −Percentage of sites that are rural Ensure individual parts add to 100%

31. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Pie Chart

32. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Line graphs: time lines X axis conveys the time Y axis is the variable of interest Multiple lines or multiple line graphs show performance of different groups

33. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Line Graph

34. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Area Graph

35. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Pictogram End of November Total!

36. University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Graphing using Microsoft Excel Click on graph icon on toolbar, or select “insert” and “chart” Proceed through chart wizard −Select the type of graph you want −Highlight the range of data to include −Set up titles, axis labels, scales, legend −Finish Edit the chart using drop down menu for charts −Click on chart to highlight it −Use the top toolbar selection “chart” to edit parts of the chart