Analyzing Data Module 8

2 Where are we in the Cycle? Resources Establish Need Analyze Data Interpret Data Communicate Results Use Results Plan Collect Data Recommend Action

3 Basic Types of Quantitative Analyses n Descriptive – describes the situation using data n Inferential/Comparative - determines whether groups are the same or different, and if so how much Relational - considers how variables relate to one another

4 Types of Analyses n Descriptive - When you merely want to know the situation (e.g.: Census data) n Inferential/Comparative - When the focus is on whether one program is better than another; or comparing the situation to baseline [Best with true experiments which are difficult in actual programs] n Relational - e.g.: Degree of participation in recreation program and state of health

5 Basic Types of Qualitative Analyses n Content analysis - How many times did something happen [If you think it sounds quantitative, you are right!] n Illustration of situation - Using quotes, pictures, “thick” description n Anecdotes - Short case study vignettes that illustrate a point

6 Interpreting the Data n What do the data say? n What were the challenges in analysis? n Reliability? n Validity? n Ethics

7 How to Make Judgements n Criteria Agree on criteria and assess case with respect to those criteria criteria need to be known and seen n Norm Look at what other good organizations in the setting do and use as benchmark n Expert Panel Eminent people look at data and use their judgement, based on their experience

8 Multiple Lines of Evidence: Triangulation Questionnaire Data Focus Group Data Interview Data

9 Depth versus breadth issues How much data? Striking the Balance

10 Sorting the Data n Look for patterns n Code your data n Weight your data

11 Issues in Data Analysis n Problem A lot of data but difficulty in making use of it Contradictory data Insufficient data Unreliable data

12 Formulating Data n Facts n Findings n Conclusions n Recommendations n Lessons learned

13 Fact versus Finding n A fact is a piece of information that has been verified. There has been a 20% increase in program costs in the last 3 years. n A finding is an analysis of related facts. Although the cost of the program has increased, there has been a 10% increase in productivity.

14 Conclusions n A conclusion covers a major aspect of the evaluation and is generally based on a collection of findings n Conclusions are often saved for the concluding chapter of an evaluation report

15 Recommendations n Directed to a responsible person/body n State clearly what is to be done n State when it is to be done by

16 Lessons Learned A lesson is a hypothesis that is based on the findings of one or more evaluations A lesson is presumed to relate to a general principle that may be applied more widely

17 Lessons Learned n Example from an evaluation of a corporate training program “The outcomes of training are more likely to be transferred to the job when the immediate supervisor supports the transfer process by meeting with the employee and developing a plan.”