This is a continuation of part 2 and is extremely important

A USEFUL TOOL IN MAPPING OUT A PROGRAM IS A “PROGRAM LOGIC MODEL (SEE ONLINE READINGS. WEEK 1) PROGRAM LOGIC MODEL IF YOU CAN SUCCESSFULLY IDENTIFY ALL OF THESE, THEN THE PROGRAM CAN BE EVALUATED. This should be used to see if there actually is a ‘functional’ program. PROGRAM LOGIC MODEL INPUTS OBJECTIVESACTIVITIESOUTPUTSOUTCOMESIndicators* All resources used and allocated for program. Staff, money, housing, packaged treatments etc. What specific results does the program hope to achieve. What are the anticipated consequences of the program. They must be behavioral, specific and measurable somehow. The more specific it is the more easily it is measured What was done within the program. How were the inputs used to produce outcomes? Program components, activities, services, interventions Products of a programs activities. What was produced by the program. (HOW MANY PEOPLE WENT THROUGH. HOW MANY DOES IT SERVE?) What was the condition of the outputs. To what degree were the objectives met. How was the problem(s) reduced by the program? How have the clients benefited? What were the ultimate effects of the program? Achievement of the program’s objectives. DATA!!! Often these are measurements of outcomes that are indirect. For example in a prevention program to reduce teen pregnancy, there is no way to adequately measure the number of girls who DON”T get pregnant. However, if there is a reduction in Risk factors such as lower truancy and better grades, these might be considered ‘indicators’ of success.

HERE IS AN EXAMPLE OF THE PROGRAM LOGIC MODEL USED TO EVALUATE THE M.S.W. PROGRAM. NOTICE WHAT IS MISSING FROM THE M.S.W. PRGRAM LOGIC MODEL. WE MUST REVISE IT!!!!

NOTICE HOW THE TABLE DOES NOT IDENTIFY ALL ASPECTS OF THE M.S.W. PROGRAM. Notice how faculty and clients are included as inputs! Notice how ‘everything we do’ that involves teaching, is included in activities! You do not have to include an evaluation column!!!! Also notice how we confused short-term outcomes with ‘outputs’. THIS IS A MISTAKE!!!!! ALTHOUGH THE MSW PRGRAM LOGIC MODEL IS NOT A GOOD PLM, IT CAN GIVE YOU AN IDEA OF HOW ONE CAN BE USED. CAN YOU SEE HOW THIS TYPE OF “MAPPING” ALLOWS YOU TO DETERMINE IF THERE IS ENOUGH IN THE PROGRAM TO EVALUATE IT? YOU WILL INCLUDE A LESS RIGOROUS VERSION OF THIS IN YOUR FEASIBILITY EVALUATION

Notes on measurement All evaluations use some form of measurement. Generally, I can measure in two ways 1. through the exact and precise use of numbers (quantitative). 2. through the exact and precise use of linguistic description (qualitative). In either case, I might be having to measure “Abstract concepts” such as improvement or motivation.

Whether I use words or numbers to measure, I must first make sure that whatever I am measuring is properly “operationalized ”. An operational definition defines a concept by specifying the activities or operations necessary to measure it – ULTIMATELY AN OPERATIONANL DEFINITION MUST LEAD TO SOME FORM OF MEASUREMENT. SOME OPERATIONAL DEFINITIONS ARE EASY – FOR EXAMPLE, WHEN I SAY “HEIGHT”, YOU AUTOMATICALLY THINK IN FEET AND INCHES. BUT MOST ARE NOT SO EASY TO AGREE ON!!! FOR EXAMPLE, SOMETHING LIKE A PARENT “YELLING AT A CHILD” MIGHT SEEM SIMPLE. BUT IS A LOUD AND EMPHATIC “NO” YELLING. THE PARENT WILL LIKELY SAY IT IS NOT AND THE CHILD WILLSAY IT IS. OPERTATIONALIZING BECOMES EVEN MORE PROBLEMATIC WHEN WE START TALKING ABOUT PROBLEMS THAT ARE MORE COMPLEX. SUCH PROBLEMS ARE USUALLY CONCEPTS OR CONTSTRUCTS.

Concept: an abstract generalization that allows social workers to summarize and order some important aspects of society... Construct – a variable that is not directly observable. Often a collection of concepts. It is a theoretical collection of concepts, based on observation of indicators. INTELLIGENCE, MOTIVATION, SELF-ESTEEM, DEPRESSION, WELL- BEING ARE ALL CONSTRUCTS. THEY ARE CONCEPTS CONTRUCTED BY HUMANS THAT PULL TOGETHER A NUMBER OF PHENOMENA UNDER A CONSTRUCTED LABEL! These phenomena becomes ASPECTS OR ATTRIBUTES WHIICH OFTEN OCCUR TOGETHER UNDER THE CONSTRUCTED LABEL Back to “improvement”: In order to measure improvement I would have to know 1. what improvement looks like (what are the behaviors) 2. what it looks like when I measure it. (what will I be measuring) 3. where to look for it when I measure it. (data sources) 4. when to look for it when I measure it 5. how to look for it when I measure it (methods of measurement) 6. how much of it do I need to call it improvement (what are the standards that determine it has been achieved) THESE SIX CHARACTERISTICVS ARE ESSENTIAL FOR SUCCESSFUL MEASUREMENT. THEY MUST BE ANSWERED.

The concept of motivation presents an even harder problem. Where is it? What does it look like? How do you know its there? How can you tell how much of it is there? YOU CANNOT! MOTIVATION IS NOT DIRECTLY OBSERVABLE. We must measure 1. factors of motivation and possibly indicators of motivation. SOME PARTICULAR BEHAVIORS (ASPECTS/ATTRIBUTES) MAY INDICATE THAT MOTIVATION IS THERE. WHAT ARE SOME ASPECTS OR ATTRIBUTES OF MOTIVATION? NAME SOME! (REMEMBER WE MUST BE ABLE TO MEASURE THEM.) We refer to these aspects as INDICATORS of motivation. Because we cannot directly observe motivation, we select several observable aspects that, if they are there, INDICATE the presence of motivation. But again, these six questions must be answered. 1. what improvement looks like (what are the behaviors) 2. what it looks like when I measure it. (what will I be measuring) 3. where to look for it when I measure it. (data sources) 4. when to look for it when I measure it 5. how to look for it when I measure it (methods of measurement) 6. how much of it do I need to call it improvement (what are the standards that determine it has been achieved