Designing a BFEF Scholarly Project with a Curricular Focus USING Logic Models Or imrad format Sandra Burge, Ph.D. UT Health Science Center – San Antonio

Objectives Describe the Logic Model as a tool for planning and evaluating curricular projects and service programs. Describe the IMRAD Format (research) as a tool for proposing and reporting curricular projects Assist BFEF participants in planning, evaluating, and reporting scholarly projects.

Excellence built on peer review Defining Scholarship Excellence built on peer review

W.K. Kellogg Foundation, Logic Model Development Guide, Jan 2004 Logic Models A logic model is a systematic and visual way to present and share your understanding of the relationships among… The resources you have to operate your program The activities you plan The outcomes you hope to achieve Resources Activities Outcomes W.K. Kellogg Foundation, Logic Model Development Guide, Jan 2004

Logic Models The most basic logic model is a picture of how your program will work. It uses words and/or pictures to describe the sequence of activities for bringing about change, and how these activities are linked to the expected results. Resources Activities Outcomes

A Simple Logic Model A Family Vacation Resources Activities Outcomes Family members Budget Car Camping Equipment Activities Drive to state park Set up camp Cook, laugh, hike, play, swim, talk Outcomes Family bonding Good memories Fun time Family learns about each other

A Less-Simple Logic Model Problem/ Situation Goals Outcomes Assumption Resources Activities Outputs External Factors

Problem/Situation: Where You Began Unhappy or unhealthy patients, poor exam scores, low student interest, insufficient mental health services, unskilled residents, accreditation issues, lack of X curriculum, disorganization, “stuck-ness” Goals Outcomes Assumption Resources Activities Outputs External Factors

Goals: What You Wanted to Achieve Improvements in … knowledge, skills, behavior, attitudes, patient satisfaction, board scores, pass rates, function, status, promotions (note: These should be linked to your “Outcomes”) Problem/ Situation Outcomes Assumption Resources Activities Outputs External Factors

Assumptions Assumptions Problem/ Situation Outcomes Assumptions Improved resident skills enhance patients’ health. Happy residents are more productive. Resources Activities Outputs External Factors

Resources – What You Had to Work With Problem/ Situation Goals Outcomes Resources (or “Inputs”) People Finances Materials Organizations Community resources Assumption Activities Outputs External Factors

Activities – What You Did Problem/ Situation Goals Outcomes Activities Program implementation Actions Interventions Meetings Processes Events Technology Assumption Resources Outputs External Factors

Outputs – What You Produced Problem/ Situation Goals Outcomes Outputs Curricula Reports Talks Slides Syllabi Websites Services Plans Schedules Assumption Resources Activities External Factors

Outcomes – What Changed Problem/ Situation Goals Outcomes Improvements in… Knowledge Skills Behavior Attitudes Patient Satisfaction Board Scores Pass Rates Assumption Resources Activities Outputs External Factors

External Factors: Influences Outside The Scope Of Your Project (May Help Or Hinder) Problem/ Situation Goals Outcomes Assumption Resources Activities Outputs External Factors – Hindering: limits placed by hospital, changes in duty hours, decreased funding, decreased time, staff shortages, new regulations, social crises Helping: new opportunities, more time, demand for more, new grant, additional assistance, EMR

Logic Models Problem/ Situation Goals Outcomes Resources Activities Assumption Resources Activities Outputs External Factors

Example: Retaining & Advancing Women & Minorities Problem: The Committee for the Advancement of Women & Minorities (CAWM) was inactive and stuck. This group – consultants for the university president - recently suffered from lack of leadership, turnover in the President’s office , uncertain funding, and unclear direction Goals: Retain & Advance women and minorities. Objectives: identify current activities to promote diversity; identify gaps; develop programs/resources to create an inclusive work environment. Outcomes Increase retention of women & minorities Increase women & minorities in senior positions Assumption Diversity is good! Improves healthcare to minorities Resources Committee President AA/EEO IT Services HR 5 Schools Activities Meetings Research diversity activities Develop website Develop Action Plan Outputs Diversity Activity Report Diversity Website Action Plan External Factors – South Texas is plagued by poverty, low education, poor health, and a dearth of healthcare professionals. This region needs a culturally competent workforce.

Stop Consider a project idea for yourself. Take 5 minutes to fill in some boxes in the Logic Model. Start anywhere in the model.

Process Where did you start? What was easiest to do? What was difficult?

How Do You Evaluate Your Project? Problem/ Situation Goals Outcomes Assumption Resources Activities Outputs External Factors

Process Evaluation is What You Did & Produced Problem/ Situation Goals Outcomes Assumption Resources Activities Outputs External Factors

Outcome Evaluation is What You Changed Problem/ Situation Goals Outcomes Assumption Resources Activities Outputs External Factors

Logic Models Problem/ Situation Goals Outcomes Resources Activities Assumption Resources Activities Outputs External Factors

Using a Research Paper Format to Report Educational Projects How curious! The 2011-12 BFEF Class is reporting their project results in research posters.

Standard Format for Research Reports IMRaD Introduction/ Background Methods Results and Discussion / Conclusions

Research Poster BACKGROUND METHODS RESULTS CONCLUSIONS OBJECTIVES REFERENCES

Introduction – Why do the project? BACKGROUND METHODS RESULTS CONCLUSIONS OBJECTIVES REFERENCES

Introduction BACKGROUND OBJECTIVES What do we know about this topic already? What does the research literature say? What’s missing in your program? What is the problem you want to solve, or gap you want to fill? METHODS RESULTS CONCLUSIONS OBJECTIVES Link objectives to your plan for evaluation: “By implementing Project ABC, we aimed to improve … A–attitudes, B–behavior C – knowledge.” REFERENCES

Methods – What did you do? BACKGROUND METHODS RESULTS CONCLUSIONS OBJECTIVES REFERENCES

Methods METHODS Participants. Who gets the ‘intervention’? Do you have a control group? Intervention. Describe your project/ curriculum. This should get the thickest description Evaluation. Measurement tools Procedure for collecting data Analysis BACKGROUND RESULTS CONCLUSIONS OBJECTIVES REFERENCES

Results – What did you find? BACKGROUND METHODS RESULTS CONCLUSIONS OBJECTIVES REFERENCES

Results RESULTS Process evaluation: How successful was the implementation? # workshops, attendance, etc. Outcome evaluation: What changed after the implementation? Knowledge Attitudes Confidence Skills/ Behavior Stress Mental health Patient outcomes Graphs, tables, qualitative information are all appropriate here. BACKGROUND METHODS CONCLUSIONS OBJECTIVES REFERENCES

Conclusions – What do you make of this? BACKGROUND METHODS RESULTS CONCLUSIONS OBJECTIVES REFERENCES

Conclusions/Discussion State your most exciting finding. Describe: Lessons learned, or Recommendations for others, or Future plans BACKGROUND METHODS RESULTS OBJECTIVES REFERENCES

Research Poster BACKGROUND METHODS RESULTS CONCLUSIONS OBJECTIVES What do we know about this topic already? What does the research literature say? What’s missing in your program? What is the problem you want to solve, or gap you want to fill? METHODS Participants. Who gets the ‘intervention’? Do you have a control group? Intervention. Describe your project/ curriculum. This should get the thickest description Evaluation. Measurement tools Procedure for collecting data Data analysis RESULTS Process evaluation: How successful was the implementation? attendance, # work-shops, etc. Outcome eval: What changed? Knowledge Attitudes Confidence Skills/ Behavior Stress Mental health Patient outcomes Data Graphs, Tables, Qualitative info CONCLUSIONS State your most exciting finding. Describe one or more of these: Lessons learned Recommendations for others Future plans OBJECTIVES Link objectives to your plan for evaluation. REFERENCES

Research Poster BACKGROUND METHODS RESULTS CONCLUSIONS OBJECTIVES REFERENCES