3. Phoenix Residential Society Who we are? ▫ Carole Eaton – Executive Director ▫ Michael F. Seiferling – Research Assistant The Program ▫ Phoenix Apartment Living Services (P.A.L.S.)

4. Introduction The History ▫ Paradigm shift in mental health services (1990) ▫ Journey of critical self-reflection ▫ The “BACKLASH” ▫ Forging ahead into the forest alone

6. The Forest

8. The NEEDS Assessment Intended to assess the extent to which the intervention is needed Examines the implication of the circumstances for the design of the program Defines what services are needed and how they might be delivered?

9. The NEEDS Assessment What is the level of need among the client groups? What are the staffs needs regarding training? What service areas come up most frequently?

10. The NEEDS Assessment - Clients Used Multnomah Community Abilities Scale Developed by a group of community workers Widely used in many formal health settings (primarily community based settings) Is both... Functional Assessment ▫ Clinical tool Outcome measure ▫ Research tool

11. Who Is It For?

12. MCAS Snapshot

13. NEEDS Assessment - Clients MCAS Scores 40 = inpatients 50-60 = high level of support 60 = lower intensity outpatient care

15. Summary of Findings - Clients Two distinct groups appear PALS 1 – much higher functioning PALS 2 – more emphasis on rehabilitation

16. NEEDS Assessment - Staff Staff should be ‘recovery focused’ What is staff’s knowledge of recovery? What does this mean for the training of our staff?

17. The Recovery Knowledge Inventory 20 questions that group into four factors Roles and Responsibilities – risk-taking, decision-making, responsibility of clients and staff Non-linearity of Recovery Process – role of the illness and symptom management and non- linearity of the process

18. The Recovery Knowledge Inventory Roles of Peers and Self-Definition – person’s activities in defining an identity, moving beyond the role of a mental patient Expectations regarding recovery – including those most effected by the illness

20. NEED Assessment - Staff Reasonable understanding of recovery ×Non-linearity of Recovery ×Expectations regarding recovery  Scores comparable to other Region  Focus on Motivational Enhancement Therapy  Rapp and Gorscha Strengths Model

22. Program Theory Assessment Involves describing the program theory in explicit and detailed form Various approaches are then used to examine how reasonable, feasible, ethical and otherwise appropriate it is

23. Program Theory Assessment Main Theories, treatment modalities ▫ Reality Therapy/Choice Theory ▫ Psychiatric Rehabilitation Specific Program guidelines ▫ Best Practices  Clinical Competencies ▫ Evidence Based Practices/Guidelines/Standards

24. Program Theory Assessment Goals of the Program ▫ To assist individuals with serious and persistent mental illness to live as independently as possible in the community ▫ Promote greater self-reliance and an enhanced quality of life ▫ To build on the existing strengths of the person, to improve skills and increase supports.

25. Case Management Models 1:10 caseload High outreach 24 hour accessibility Emphasis on skill training High degree of direct service provision and frequency of contact Occurs in community SMI high service users 1:20-30 caseload Moderate outreach No 24 hour accessibility Emphasis on skills training Moderate degree of direct service and frequency of contact Mainly occurs in community SMI

26. Evidence based ingredients for CM 1.CM should deliver as much of the help as possible 2.Natural community resources are primary partners 3.Work is in the community 4.Individual and team case management works 5.Case managers have primary responsibility for a person’s services

27. Evidence based ingredients for CM 6.Case Managers can be Para-professionals 7.Case loads should allow for high frequency of contact 8.CM should be time-unlimited 9.People need access 24-7 10.CM should foster choice

28. Caseload size Research does suggest that a caseload of 20:1 - 30:1 ratio may work provided that the people receiving services are: ▫ more stable and independent or ▫ the caseloads are compromised of people normally distributed in terms of severity

29. Frequency of Contact It has been stated that “the strongest predictor of successful engagement [is] frequency of contact” Frequency of contact has been identified as more important that total hours of service, suggesting that “brief visits may be more valuable then less frequent but longer visits”

30. Frequency of Contact A good average to strive for people who are typical of involvement with ACT/ICM programs is 11 contacts per month (2-3 contacts per week) It is suggested that phone contact may also be used as a supplement to face to face contact The quality of contact may play a mitigating role on determining client outcomes. In other words, there must be a strong helping relationship present in order for any frequency of contact to be effective

31. Time-unlimited service It has been stated that ‘time-unlimited’ services are a critical element for evidence based case management There has been some research which suggests that successful transfers can be made to lower intensity services The presents of informal social networks have been considered an important factor for people moving on

32. Program Theory Assessment Other options ▫ Fidelity assessments for EBP  Rapp and Goscha Strengths Case-Management  SAMHSA Toolkits for Illness and Symptom Management, IDDT, ACT ▫ Quality Review Tools  Rapp and Goscha Strengths Case-Management  MET Quality Review (MISC)

34. Program Process Assessment Examines how consistent the services actually delivered are with the goals of the program If services are deliver to appropriate recipients Assess the extent to which a program is implemented as intended and operating up to the standards established for it

35. Program Process Assessment

38. Program Process Findings Significant gaps in programming for community clients (i.e. social, money management and medication management) Frequency and Quality of contact needed to be adjusted (higher needs clients were being under served/need for MET) Sharper focus on how rehabilitation goals were being developed and defined (i.e. Strengths Model)

40. Program Impact Assessment Involves providing an estimate of the net effects of a program Establishing the status of the program recipients on relevant outcome measures and also estimating what their status would have been without the services

41. Program Impact Assessment Able to compare our program to other that have used MCAS This comparison allows early estimation of program impact Allows for more longer term applications to be considered

42. Comparison CMHEI On-going samplePALS Intensive Community Support Program (ACT/ICM) Several contacts per week Caseloads size – 17:1 Age = 43.1 years Schizophrenia = 69.3% Mood Disorder = 28.4% Range (45-85) Intensive Community Support Program Several contacts per week Caseload size - 13:1 Age = 45 years Schizophrenia = 80% Mood/ Anxiety /other = 20% Range (32-80)

44. Comparison

45. Program Impact Assessment What about ‘Recovery’? MCAS measures ‘functional outcome’ but not one’s sense of ‘recovery’ Common misconception that these are directly linked

46. Davidson explains…

47. The Vermont Study

48. Mental Health Recovery Measure Based on research of the consumers views of recovery Developed for Adults with SMI who are receiving services from ▫ criminal justice system ▫ inpatient settings ▫ outpatient service settings ▫ peer-run programs ▫ residential settings Correlations with ▫ Empowerment Scale ▫ Resilience Scale ▫ Community Living Scale

49. MHRM and subscales

50. PALS Comparison

51. Recovery and Functioning Compared MHRM and MCAS scores Found no/little correlation in scores MHRH scores hit ceiling so can’t be used for on- going measurement Decided to move toward Strengths based goal attainment as a ‘recovery measure’

52. Strengths Perspective

53. Implications - Outcomes FunctioningRecovery The ‘Skills’ one need to live in the community as independently as possible Includes symptom management Tends to determine ‘traditional’ outcomes in mental health Measured by the MCAS Requires knowledge in PSR Includes how much personal satisfaction one has in their lives Speaks more to how a person accommodates/manages the disability Can occur even with lower functional scores Measured by Goal Attainment Requires knowledge in MET and Strength Assessment

54. Program Impact Assessment

56. The Efficiency Assessment Considers the relationship between the program cost and its effectiveness This builds on good process and impact assessments One of the most difficult areas

57. The Efficiency Assessment

58. Clinical Leadership Program

59. What we have learnt so far Program Evaluation is not as simple as identifying an outcome measure Must address infrastructure issues along the way (i.e. data management, training) Must be seen as part of a longer term plan Have a plan to deal with resistance

60. What’s ahead Strength's assessment which leads to consistent goal attainment scaling Managing Data Additional tools Ability to clearly identify the impact of specific interventions (i.e. peer support, Social Skills Training)

61. Final Thoughts One program at a time! Don’t forget to plant all the trees Forests don’t grow in a day!!! Even small scale programs can start the process of program evaluation Try, try, try…