Reporting in Context Performance management, continual quality improvement and the feedback loop.

Overview Performance management Meta evaluation Quality improvement (QI) Bio-break Information sharing Considerations Negative results Written report Data visualization Your project challenge

Performance Management Strategic Objectives EmployeesSystemsResources

Seeking Excellence Invest in high quality staff Competent Trainable Plan for quality improvement Well-outlined processes Implemented continually Dedicate resources Succeed through failure

Excellence in Evaluation Create data collection mechanisms Leverage data in decision making Learn continually in all processes Focus on most important parts Take immediate and decisive action Evaluate the evaluation

Meta Evaluation

Analysis of Evaluation Process Did your tool measure what you wanted? Do your results answer your question(s) or support your hypothesis? Does your data demonstrate change? As expected? To what extend did stakeholders help or hinder the evaluation process? Will your results influence direction or strategy?

Common Issues Adequate training of staff Methodology/protocols Data collection Use of tools/measurements Staff investment/ownership

Common Issues Participant loss Drop-out Disinterest Poor or out-of-date records Loss, etc.

Common Issues Continuous Quality Improvement Case review Peer/Team meetings Corrective actions Follow-up

Continuous QI Feedback Loop Learn What Works For whom Under what conditions With what resources Plan Set priorities, objectives Establish timeline, responsibilities, protocols Allocate resources Do Implement plan, protocols Serve clients Check Monitor & measure Find & fix Document Act Evaluate Apply lessons learned Take corrective/ learning action(s)

Continuous QI Create/acquire necessary tools Identify achievable and measurable outcomes, measure outcomes Be timely Engage stakeholders

Foster Feedback Appeal to the ego Share findings with participants Ask them to help “make meaning” from findings & generate plausible solutions Demonstrate to participants how their engagement led to improvement/change

Generate Good Karma Participate in surveys when given the opportunity Website reviews Program participant Course evaluations Join a survey panel NPR Listeners Survey Monkey

Information Sharing Your Evaluation Data Instruments Logic Model Continual Learning Corrective Actions Dedicated Resources Considerations: What is/are the mechanism(s)? Who is your audience? What is the political climate? How will you maintain ethical standards? What are the implications?

Negative Results Consider industry standards Are negative results the norm? Has success ever been achieved?

Negative Results, First… Identify the problem Tool or strategy? Methodology? Environment? Inadequate resources? Training/staff? Participant loss?

Negative Results, Next… Take corrective or “learning” action New tool Site change Re-training Re-design Change participants

Add’l Recommendations Exit strategy Investment strategy Partnerships/collaboration

Write with Clarity & Brevity The Hard Truth Executive summary Brief overview of evaluation & purpose Summary of findings Recommendations

Excellence in Reporting Be Brief. Be Visual. Be Accurate. Be Consumable/Digestible.

Use Data Visualization & Infographics to: Understand trends and patterns in your data Make good decisions Communicate: Insights Outcomes Trends Tell your story!

Foster Understanding

Qualitative Data

Pair Qualitative & Quantitative Data

Likert-Scales

Learn from Mistakes

Excellent Visualizations Match visual type with data type Respect White Space Work in color and grayscale Are easily understood

Additional Visualization Resources Wikipedia on Data Visualization Data Visualization 101: How to design charts and graphs David McCandless: The beauty of data visualization Jer Thorp: Make data more human

CONSIDER: YOUR EVALUATION PROJECT… WHAT’S YOUR BIGGEST CHALLENGE? (3) CLARIFYING QUESTIONS (5) RESTATEMENT OF CHALLENGE (1) PEER RECOMMENDATIONS (5) ACTION ITEMS/TIMELINE (1) Small group exercise: