1. Techniques for Overcoming Resistance to Program Evaluation
Stanley Capela│VP Quality Management & Corporate Compliance Officer │ HeartShare Human Services of New York

2. Agenda About the Speaker Knowing Your World
Dealing with Ethical Issues
Using Positive Language
Collaboration with Difficult Colleagues
Overcoming Personality Differences
Summary
Q&A

3. About the Speaker
Internal evaluator in the non-profit human services sector for 38 years.
VP for Quality Improvement and Corporate Compliance at HeartShare Human Services of New York
Council on Accreditation (COA) peer reviewer with over 100 COA site visits
completed

4. Knowing Your World
I started a job as a research associate in an organization’s Evaluation and Staff Development department. Over a period of time, I noticed that employees from other departments seemed to try to avoid coming into contact with me.
I spent time talking with a variety of people at the organization and discovered that when my department was created, four people had been fired at the conclusion of their first evaluation.

5. Lessons Learned:
Before undertaking any evaluation, take the time to understand the environment you have walked into.
Who are they?
What is the world they work in?
What is the organizational culture and norm?
What is their understanding of program evaluation and your role?
What were their past experiences with program evaluation like?
Asking these questions and understanding their answers will help you decide on your approach in undertaking an evaluation.

6. Dealing with Ethical Issues
I was asked to conduct an evaluation of a re-organization. The evaluation included interviews with all key stakeholders who would be impacted. In this organization, the service areas were broken up into specialty areas such as aging, developmental disabilities, housing, and mental health services.
When I presented the findings at a senior management meeting that included many of the people I had interviewed, I was confronted with an ethical dilemma. One of the major findings of our evaluation was that the re-organization had resulted in a silo effect. After presenting the findings, the #2 person in the organization stated that the Executive Director had told him that I would disclose what certain individuals in the organization had said.
My response was that it would not be possible, because I had clearly stated at the beginning of the evaluation process that responses would remain anonymous. If I were to reveal what the individual had said, it would be unethical. Even when the # 2 person told me that my job would be at risk if I did not give them the information, I stood my ground. In the end, I was able to stay in my position without compromising my values.

7. Lessons Learned:
There will be occasions where you will be confronted with an ethical dilemma. Do you have strategies in place to deal with them? In my situation, my responses came naturally.
First, I had a very supportive Director who gave me the impression she had my back.
Second, I realized if I were to compromise my position, it would further erode the department’s reputation.

8. Using Positive Language
As a COA Team Leader and Peer Reviewer, one of the areas that is emphasized is how you communicate findings when doing a site visit. During the exit conferences, one of the things I have found helpful to keep in mind is the use of language.
For instance, as opposed to using the terms deficits or out of compliance, I start with the word strengths and counter them with challenges.
I have found that staff listen differently because there is an understanding that there are challenging areas.

9. Lessons Learned:
When communicating, how you present your findings may foster greater resistance.
Use of words like deficits may give the impression that the staff have failed.
Challenges, on the other hand, can be overcome. This helps them understand that the evaluator is there to help the program overcome its challenges, not to shame or blame anyone.
Focusing on the strongest areas first will set the tone for the rest and make individuals more responsive to feedback.

10. Collaboration with Resistant Colleagues
Over the years, I identified a number of issues with the organization’s adoption performance. I would share the findings with the appropriate director as well as the Executive Director. The Director of the Adoption division repeatedly made excuses and dismissed my findings as inaccurate.
Our program was achieving 20 to 30 adoptions a year, which was significantly below the benchmark. Eventually, the Executive Director was called into a meeting with the funding source and was told that we needed to either meet the goal for the year, or face the consequences. The overall goal was 245 adoptions. The Executive Director told us to do whatever needed to be done in order to reach the goal.
First, I convened key stakeholders.
Second, we reached a consensus on terminology.
Third, we agreed on a process, specifically how the reports would be designed and how information would be shared.
Fourth, the Executive Director made it clear that I would be in charge of this project.
In the final analysis, we did not meet the target — we exceeded it by 2. We achieved 247 adoptions, of which less than 5% returned to care. Over the past 15 years, we have maintained an outstanding adoption record.

11. Lessons Learned:
Make sure you present the information in a way that gets the point across. Ultimately, we reached an understanding on how to present results in a way that made sense.
It is important to have support of the key powerbroker in the organization.
Don’t focus on blaming others. As long as you achieve the ultimate outcome that you need, it is okay to let others get the credit sometimes.
Once the key stakeholders realize your value, it makes your role that much easier.

12. Overcoming Personality Differences
When two organizations merge, it is also the merger of two organizational cultures. As a result, you have to find ways to deal with some very different personalities and have to learn to adapt to new situations.
After our organizational merger, a couple of directors from the other agency did not understand the way I operated as an evaluator. Initially, I made the effort to understand their world, but they did not understand mine. During an accreditation process for which I was the agency site coordinator, we had to set aside our egos to understand each other and how we could work together to strengthen both organizations. Over time, the directors of the new organization came to realize the value of my style of evaluation and understood that program evaluation is more than just metrics.
The ultimate outcome of the new organization’s accreditation was phenomenal. We had to meet standards and were out of compliance on only 2. As a result, our professional relationships began to change for the better.

13. Lessons Learned: Don’t get caught up in your ego.
Active listening is key.
Take the time to understand the key powerbrokers within the organization.
Do an environmental scan of strengths. Specifically, identify the strengths that they bring to the table and how your strengths can further enhance the relationship.
Find opportunities to show that you are an asset, and how you can not only strengthen the organization, but also make key power brokers look good in front of their Board of Directors.

14. Identifying the Key Secrets to Overcoming Resistance
Learn the organization’s history with program evaluation.
Look at the organization contextually, with an emphasis on identifying its strengths.
When doing a group of evaluations in a specific area, focus on the strongest one first to set the tone for the rest.
Give up your ego.
Eliminate negativity.
Let others get the credit.

15. Realize the importance in keeping it simple.
Focus on the beauty of making a difference in the lives of the clients, families and individuals who seek out services.
Take the time to understand the world staff work in.
Emphasize the importance of being utilization focused.
Identify and understand the key powerbrokers within the organization.
Turn 64, look back at the road you traveled, and realize you made a difference!

16. Program Evaluation does NOT solve Management problems.

17. Thank you! Any Questions?