USING LOGIC MODELS OR IMRAD FORMAT Sandra Burge, Ph.D. UT Health Science Center – San Antonio Designing a BFEF Scholarly Project with a Curricular Focus 1

Objectives Describe the Logic Model as a tool for planning and evaluating curricular projects and service programs. Describe the IMRAD Format (research) as a tool for proposing and reporting curricular projects Assist BFEF participants in planning, evaluating, and reporting scholarly projects. 2

Logic Models A logic model is a systematic and visual way to present and share your understanding of the relationships among…  The resources you have to operate your program  The activities you plan  The outcomes you hope to achieve W.K. Kellogg Foundation, Logic Model Development Guide, Jan 2004 Resources Activities Outcomes 3

Logic Models The most basic logic model is a picture of how your program will work. It uses words and/or pictures to describe the sequence of activities for bringing about change, and how these activities are linked to the expected results. Resources Activities Outcomes 4

A Simple Logic Model A Family Vacation Resources Family members Budget Car Camping Equipment Resources Family members Budget Car Camping Equipment Activities Drive to state park Set up camp Cook, laugh, hike, play, swim, talk Activities Drive to state park Set up camp Cook, laugh, hike, play, swim, talk Outcomes Family bonding Good memories Fun time Family learns about each other Outcomes Family bonding Good memories Fun time Family learns about each other 5

A Less-Simple Logic Model ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Goals 6 Problem/ Situation

Problem/Situation: Where You Began ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Goals Problem/ Situation Unhappy or unhealthy patients, poor exam scores, low student interest, insufficient mental health services, unskilled residents, accreditation issues, lack of X curriculum, disorganization, “stuck-ness” 7

Goals: What You Wanted to Achieve ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Problem/ Situation Goals – Improvements in … knowledge, skills, behavior, attitudes, patient satisfaction, board scores, pass rates, function, status, promotions (note: These should be linked to your “Outcomes”) 8

Assumptions ResourcesResourcesActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Problem/ Situation Assumptions Improved resident skills enhance patients’ health. Happy residents are more productive.Assumptions Improved resident skills enhance patients’ health. Happy residents are more productive. 9

Resources – What You Had to Work With AssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Problem/ Situation Goals Resources (or “Inputs”) People Finances Materials Organizations Community resourcesResources (or “Inputs”) People Finances Materials Organizations Community resources 10

Activities – What You Did ResourcesResourcesAssumptionAssumptionOutputsOutputs OutcomesOutcomes External Factors Problem/ Situation Goals Activities Program implementation Actions Interventions Meetings Processes Events TechnologyActivities Program implementation Actions Interventions Meetings Processes Events Technology 11

Outputs – What You Produced ResourcesResourcesAssumptionAssumptionActivitiesActivities OutcomesOutcomes External Factors Problem/ Situation Goals Outputs Curricula Reports Talks Slides Syllabi Websites Services Plans SchedulesOutputs Curricula Reports Talks Slides Syllabi Websites Services Plans Schedules 12

Outcomes – What Changed ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs External Factors Problem/ Situation Goals Outcomes Improvements in… Knowledge Skills Behavior Attitudes Patient Satisfaction Board Scores Pass RatesOutcomes Improvements in… Knowledge Skills Behavior Attitudes Patient Satisfaction Board Scores Pass Rates 13

External Factors: Influences Outside The Scope Of Your Project (May Help Or Hinder) ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors – Hindering: limits placed by hospital, changes in duty hours, decreased funding, decreased time, staff shortages, new regulations, social crises Helping: new opportunities, more time, demand for more, new grant, additional assistance, EMR Problem/ Situation Goals 14

Logic Models ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Problem/ Situation Goals 15

Example: Retaining & Advancing Women & Minorities Resources Committee President AA/EEO IT Services HR 5 SchoolsResources Committee President AA/EEO IT Services HR 5 SchoolsAssumption Diversity is good! Improves healthcare to minoritiesAssumption Diversity is good! Improves healthcare to minoritiesActivities Meetings Research diversity activities Develop website Develop Action PlanActivities Meetings Research diversity activities Develop website Develop Action PlanOutputs Diversity Activity Report Diversity Website Action PlanOutputs Diversity Activity Report Diversity Website Action Plan Outcomes Increase retention of women & minorities Increase women & minorities in senior positionsOutcomes Increase retention of women & minorities Increase women & minorities in senior positions External Factors – External Factors – South Texas is plagued by poverty, low education, poor health, and a dearth of healthcare professionals. This region needs a culturally competent workforce. Problem: Problem: The Committee for the Advancement of Women & Minorities (CAWM) was inactive and stuck. This group – consultants for the university president - recently suffered from lack of leadership, turnover in the President’s office, uncertain funding, and unclear direction Goals: Goals: Retain & Advance women and minorities. Objectives: Objectives: identify current activities to promote diversity; identify gaps; develop programs/resources to create an inclusive work environment. 16

Stop Consider a project idea for yourself. Take 5 minutes to fill in some boxes in the Logic Model. Start anywhere in the model. 17

Process Where did you start? What was easiest to do? What was difficult? 18

How Do You Evaluate Your Project? ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Problem/ Situation Goals 19

Process Evaluation is What You Did & Produced ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Problem/ Situation Goals 20

Outcome Evaluation is What You Changed ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Problem/ Situation Goals 21

Logic Models ResourcesResourcesAssumptionAssumptionActivitiesActivitiesOutputsOutputs OutcomesOutcomes External Factors Problem/ Situation Goals 22

23

Using a Research Paper Format to Report Educational Projects 24 How curious! The BFEF Class is reporting their project results in research posters.

Standard Format for Research Reports IMRaD Introduction/ Background Methods Results and Discussion / Conclusions 25

BACKGROUNDBACKGROUNDMETHODS METHODS RESULTSRESULTS OBJECTIVESOBJECTIVES CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Research Poster 26

BACKGROUNDBACKGROUNDMETHODS METHODS RESULTSRESULTS OBJECTIVESOBJECTIVES CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Introduction – Why do the project? 27

METHODS METHODS RESULTSRESULTS OBJECTIVES Link objectives to your plan for evaluation: “By implementing Project ABC, we aimed to improve … A–attitudes, B–behavior C – knowledge.”OBJECTIVES Link objectives to your plan for evaluation: “By implementing Project ABC, we aimed to improve … A–attitudes, B–behavior C – knowledge.” CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Introduction BACKGROUND What do we know about this topic already? What does the research literature say? What’s missing in your program? What is the problem you want to solve, or gap you want to fill?BACKGROUND What do we know about this topic already? What does the research literature say? What’s missing in your program? What is the problem you want to solve, or gap you want to fill? 28

METHODS METHODS RESULTSRESULTS OBJECTIVES Our aim. We propose to develop and evaluate a new curriculum for SBIRT. The new curriculum will increase residents’ knowledge, practice, and skills in SBIRT.OBJECTIVES CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Introduction - Example BACKGROUND What we know: Alcoholism is prevalent and causes serious medical, psychological and social problems. Research shows that SBIRT is effective as an early intervention to prevent alcohol abuse and its consequences. The gap. Our residency program has no systematic approach in its curriculum about managing alcohol misuse and abuse.BACKGROUND What we know: Alcoholism is prevalent and causes serious medical, psychological and social problems. Research shows that SBIRT is effective as an early intervention to prevent alcohol abuse and its consequences. The gap. Our residency program has no systematic approach in its curriculum about managing alcohol misuse and abuse. 29

BACKGROUNDBACKGROUNDMETHODS METHODS RESULTSRESULTS OBJECTIVESOBJECTIVES CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Methods – What did you do? 30

BACKGROUNDBACKGROUNDRESULTSRESULTS OBJECTIVESOBJECTIVES CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Methods METHODS Participants. Who gets the ‘intervention’? Do you have a control group? Intervention. Describe your project/ curriculum. This should get the thickest description Evaluation. Measurement tools Procedure for collecting data Analysis METHODS Participants. Who gets the ‘intervention’? Do you have a control group? Intervention. Describe your project/ curriculum. This should get the thickest description Evaluation. Measurement tools Procedure for collecting data Analysis 31

BACKGROUNDBACKGROUNDMETHODS METHODS RESULTSRESULTS OBJECTIVESOBJECTIVES CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Results – What did you find? 32

BACKGROUNDBACKGROUNDMETHODS METHODS OBJECTIVESOBJECTIVES CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Results RESULTS Process evaluation: How successful was the implementation? # workshops, attendance, etc. Outcome evaluation: What changed after the implementation? Knowledge Attitudes Confidence Skills/ Behavior Stress Mental health Patient outcomes Graphs, tables, qualitative information is all appropriate here.RESULTS Process evaluation: How successful was the implementation? # workshops, attendance, etc. Outcome evaluation: What changed after the implementation? Knowledge Attitudes Confidence Skills/ Behavior Stress Mental health Patient outcomes Graphs, tables, qualitative information is all appropriate here. 33

BACKGROUNDBACKGROUNDMETHODS METHODS RESULTSRESULTS OBJECTIVESOBJECTIVES CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Conclusions – What do you make of this? 34

BACKGROUNDBACKGROUNDMETHODS METHODS RESULTSRESULTS OBJECTIVESOBJECTIVES REFERENCESREFERENCES Conclusions/Discussion 35 CONCLUSIONS State your most exciting finding. Describe: Lessons learned, or Recommendations for others, or Future plansCONCLUSIONS State your most exciting finding. Describe: Lessons learned, or Recommendations for others, or Future plans

BACKGROUNDBACKGROUNDMETHODS METHODS RESULTSRESULTS OBJECTIVESOBJECTIVES CONCLUSIONSCONCLUSIONS REFERENCESREFERENCES Research Poster 36

37