Evaluation and Measurement of Inter-professional Geriatric Programs Paul Gould, LCSW, PhD Binghamton University State University of New York Kristina Hash, LICSW, PhD West Virginia University

WHAT DO WE MEAN BY EVALUATION?

Program Evaluation The process of systematically applying research methods to assess program: design implementation improvement outcomes Rossi, P. H., & Freeman, H. E. (1993). Evaluation: A systematic approach (5th ed.). Newbury Park, CA: Sage Publications, Inc.

Program Evaluation Needs Assessment: The process of systematically applying research methods to assess needs, gaps, or areas for improvement

EXAMPLES OF EVALUATION INTER-PROFESSIONAL GERIATRIC PROGRAMS

exaMPLE #1 West virginia university

Example 1: West Virginia University Gerontology Practitioner Certificate Needs Assessment and Program Development K. Hash, J. Englehardt, & Amazing Work Group Funded by: WV Partnership for Elder Living $500o grant from Benedum Foundation/WVLTCP to assess training needs in geriatrics of interdisiclinary professionals

Example 1: West Virginia University Objectives:   1.Evaluate the aging education needs of social workers and related professionals (with associate degrees and higher) in the state 2.Develop a gerontology practitioner certificate and curriculum 3.Develop best practices for recruiting and retaining professionals in the field  The School began by gathering the interdisciplinary, statewide workgroup in August.  A survey would be developed by this workgroup to distribute through various professional mailing lists and electronic listservs, including our own PCE listserv by September.  The survey would assess the aging education needs of direct care workers as well as inquire about the best ways to attract and retain quality workers in the field. 

Example 1: West Virginia University Objectives:   1.Evaluate the aging education needs of social workers and related professionals (with associate degrees and higher) in the state 2.Develop a gerontology practitioner certificate and curriculum 3.Develop best practices for recruiting and retaining professionals in the field  The School began by gathering the interdisciplinary, statewide workgroup.  A web-based survey developed by this workgroup to distribute through various professional mailing lists and electronic listservs, including our own PCE listserv by September.  The survey assessed the aging education needs of direct care workers as well as inquired about the best ways to attract and retain quality workers in the field. 

Example 1: West Virginia University Online Survey, N= 391 Training Needs 77% interested in enrolling in a CEU-based certificate Training needed: cognitive changes/dementia; end of life; legal issues; HC and aging policy Great interest in online training     The majority of respondents who identified a discipline in terms of education were trained as social workers (46%, n=164), followed by nurses and speech pathologists (each 16%, n=57). Professionals with educational backgrounds in education, counseling, sociology and psychology each represented about 3% of the respondents. Most had earned a bachelors (27%, n=97) or masters (56%, n=195) degree their identified discipline and over 85% also held a professional license. In terms of formal geriatric training, 43% (n=153) identified CE offerings as the source of this training. Thirty-three percent (n=118) indicated that they had no formal training in geriatrics. Largely from WV The major curriculum areas of a previously offered certificate were listed and respondents asked to base their interest and need on this general curriculum Cognitive changes and dementia and end of life issues were each selected by almost half of the respondents. Legal issues and health care and social policies and programs were also of great interest to over 30% of respondents. Respondents currently meet their CE needs primarily through employer-based trainings (33%, n=121) and attendance at the NASW-WV annual conference and the Summer Institute on Aging and other Division of Social Work-sponsored workshops (each selected by over 20% of the respondents).

Example 1: West Virginia University “A curriculum is very much needed for our workers to understand the population that we are dealing with and the need is urgent.” “It is very evident that WV NEEDS to offer this certificate.”  

Example 1: West Virginia University Recruitment and Retention What led them to the field? Job opportunity & Personal Experience What will keep them in the field? Career growth, Increased Salary & Benefits, Leadership Opportunities Love the Field and Population   When asked what led them to work in the field, many noted a job opportunity (57%, n=201) as well as personal and family experiences (29%, n=101). Open-ended responses to this question indicate that many respondents have always “enjoyed” working with this population. Increased opportunity for career growth as well as increased salary and benefits were all selected by over 40% of respondents. Increased opportunity for leadership in the field was selected by 28% (n=98) of those who worked in the field.

Example 1: West Virginia University “No motivation is needed. I love the population, and I intend to continue working with them, regardless of pay-rate or continuing education”   “I love what I do and just want to keep doing it.”   Many noted their love for the field and population as reasons to stay in the field

Example 1: West Virginia University Outcomes of Needs Assessment Development of CEU-based certificate Needs-identified curriculum Partnerships with other organizations offering trainings Development of online trainings Recruitment and retention ideas shared Certificate Launching in February 2012!  

EXAMPLE #2 BINGHAMTON UNIVERSITY

Geriatric Consultation Clinic Shawn Berkowitz, MD, CMD Paul R. Gould, LCSW, PhD Youjung Lee, PhD – Evaluator Suk-Young Kang, PhD – Evaluator Supported with funding from the John A. Hartford Foundation

Binghamton University HPPAE Geriatric Consultation Clinic Interprofessional Learning Experience Part of the rotational model for HPPAE MSW students collaborate with Medical Students from Upstate Medical University Conduct a 2-part Comprehensive Geriatric Assessment Home Visit Clinic Visit Team Structure: Clinical Social Worker Geriatrician Students from both disciplines Family

Geriatric Consultation Clinic Student Learning Evaluation Evaluation Methods A mixed-method using a triangulation design-multilevel model with concurrent data collection and analyses was employed for the evaluation of the project. Data analysis for the focus group was conducted in three steps: (1) open coding, (2) axial coding, and (3) selective coding (Strauss, 1987). Tools Post-Experience Focus Groups with interprofessional team members Index of Interprofessional Collaboration (IIC) (Bronstein, 2002)

Geriatric Consultation Clinic Patient Outcomes Evaluation Evaluation Methods Quantitative comparative analysis: Baseline at time of consult 8-12 weeks after consult 6 months after consult Data Collected Mini-Mental State Exam (MMSE) Zarit Caregiver Burden Scale Understanding of Medication Usage & Side Effects FS-8 Health Survey

Binghamton University Community Enhancement Initiative in Aging Preparedness Purpose Binghamton University’s HPPAE in place for 5 years Project placed HPPAE Fellows in non-MSW agencies Assess impact of program upon participating organizations and overall community Methods Individual interviews with agency administration Focus group with task supervisors

Example #3 University of Louisville

Overview: Geriatric Evaluation & Self-Management (GEMS) Project Tested effectiveness of: Interdisciplinary geriatric home-based assessment Self-management support services Target population: Community dwelling older adults with 1+ chronic illnesses Providers: Interdisciplinary teams of social workers and physical therapists (students and professionals) collaborating with PCPs

EXAMPLE #4 BOSTON UNIVERSITY

Center for Aging and Disability Education and Research CADER provides comprehensive, validated, skill-based online training programs to ensure that workers have the knowledge and skills needed to meet the challenges in a rapidly changing health and social services landscape. Demonstrating that learners have mastered the competencies needed for effective practice is a key objective of our program. Competence requires professional judgment and critical thinking to translate knowledge, skills, and values into effective practice behaviors.

Evaluation Tools Pre and Post Competency Self-Assessment of Individual Courses Participants complete pre- & post-test using the approximately 20-item instrument that measures the respondent’s perceptions of their skill level on a scale of 0 to 4 (0=not skilled at all; 4=expert skill) Course Evaluation This evaluation includes questions about the effectiveness of the online course, including whether course objectives were clearly described and met, the quality and applicability of the course, whether the course expanded learner knowledge and understanding of the topic area, and whether the training program will help them to apply practice skills in the topic area.

Evaluation Tools Program Included in the evaluation are questions related to the training program from both the perspective of the individual learner and in relationship to their organization: Individual learner: these include questions related to the expansion of knowledge, development of new skills, and the ability to do a better job as a result of the training program. Supervisor expectations, as well as organizational support are also included. In order to assess the degree to which new learning is maintained, evaluation is repeated at 3 months following completion of the program

EXAMPLE #5 FLORIDA ATLANTIC UNIVERSITY

IPE & SAGE Programs The School of Social Work, College of Nursing, and College of Medicine at Florida Atlantic University developed an interpro­fessional curriculum to bring social work, nursing, and medical students together for collaborative practice-based learning. This interprofessional curriculum included two compon­ents: (a) the Interprofessional Education (IPE) program, and (b) the Senior Aging and Geriatrics Educator (SAGE) Mentoring program.

IPE & SAGE Programs Pretest/posttest evaluation (IPE Session 1) (b) Teamwork IQ Quiz (c) reflective journaling (after the IPE & SAGE Mentoring programs) (d) follow-up assignments (after the IPE & SAGE Mentoring programs) (e) surveys before and after the IPE and SAGE Mentoring programs to measure the effectiveness of the programs and reinforce learning from these experiences.

HPPAE Model is Competency Based Gero Social Work Competencies Developed by social workers and aging experts nationally Field tested at demonstration sites Aligned with the 2008 CSWE Education and Accreditation Standards (EPAS) Competencies are measurable practice behaviors—what all graduates are able to do—that are comprised of social work knowledge, skills, and values learned in foundation class and field curriculum.

Gero Social Work Competencies Five Domains of practice Values, Ethics & Theoretical Perspectives will be used as example to connect to EPAS Assessment Intervention Aging Services, Programs & Policies Leadership in Practice (new domain recently added) Each domain has ten competencies (50 total) Gero SW Competencies considered Practice Behaviors under new EPAS

Benefits of Geriatric Social Work Competencies (Practice Behaviors) Addresses practice at BOTH micro and macro levels Direct practice Policy Research Advocacy Recognizes social workers demonstrating leadership in aging need to be proficient in both micro and macro levels Geriatric social workers providing direct service must be proficient in: identifying service gaps assessing community needs developing programs to help older adults and caregivers cope with aging Macro practitioners must be: knowledgeable about the challenges older adults and families experience able to identify older adult and family strengths and assets

BRAINSTORMING AND RESOURCE- SHARING

Participant Discussion Describe the current or potential Inter-professional Geriatric Programs at your institution How are you currently or what plans do you have to evaluate the program(s)? What successes and barriers have you faced in this evaluation or its planning?

Contact Information Paul Gould, LCSW, PhD pgould@binghamton.edu Kris Hash, LICSW, PhD kmhash@mail.wvu.edu