1. What Do We Want from Evaluation? Methodology: THE WORK PROCESS
Trust US: If We Developed It, We Can Evaluate It
Angela Harnden, PhD, Nancy Ward, Rose Ann Percival, Ellen Bannister, Linda Gonzales and Valerie N. Williams, PhD
Oklahoma People First and the Center for Learning and Leadership, Oklahoma UCEDD
What Do We Want from Evaluation?
Methodology: THE WORK PROCESS
Results and Lessons Learned
STEP 2: Develop Training Materials
Use flip chart paper to expand ideas
Self-advocates review material researched by others
Prepare power point presentations: use symbols & pictures to address all reading levels
Develop trainer’s manual with understandable language using pictures & symbols
Pilot training in various locations
LESSONS FOR PARTNERS
It took a lot of time, patience and editing.
Learned to trust self-advocates.
Honored self-advocates’ wishes to “Teach Us to Do It Ourselves”.
Took a willingness on the part of partners and funders to let self-advocates build their own evaluation process based on their experiences and knowledge.
Learned to respect self-advocates’ ability to develop their own materials and processes.
Learned how to support and not try to control the process.
All of us listen better to our peers.
STEP 1: Agreeing on Topics for Training
Trained committee on Leadership
Asked committee what they really wanted to learn and what they thought other OPF members wanted to learn
Used flip chart paper to record answers
Narrowed down the topics.
STEP 1: Agreeing on the Need for Evaluation
Self-Advocates wanted:
A review and evaluation process that people with disabilities could understand to evaluate the training created and taught by self-advocates in the Oklahoma Self-Advocacy Network (OKSAN).
Development of Training & Evaluation Process
Formed a training committee of self-advocates
Facilitated by CLL and OPF
Used flip chart paper to list skills they currently possess
Used flip chart paper to list all the areas the self-advocates were interested in learning
Sorted list until we decided the most important things to learn first.
Reviewed different evaluation tools and process
Tried different versions until we agreed on what we thought people with disabilities could understand and value.
STEP 2: Developing a Method for Evaluation: Trying what is available
Group Consensus on Action
Developing Training Materials
Training the Trainer
Mentoring
Certification of Trainers
Evaluation and Improvement
STEP 3: Training-the-Trainer
Manuals completed.
Conduct 2-day train-the- trainer class using various materials.
STEP 3: Edit prior evaluation forms to find what is needed. Used Person Centered Thinking (PCT) tool, “Working/Not Working” to create new evaluation tools.
STEP 6: Evaluation
Used Person Centered tools to evaluate trainings.
After each training, reviewed what worked and didn’t work.
Developed a training satisfaction survey.
First, trainers were evaluated by project staff.
Now Master Trainers evaluate new trainers.
STEP 6: Final evaluation process using universal language
“We did it. I am proud of the evaluation process we developed.”
Nancy Ward, trainer
“I feel like people want to know how I feel about something. I loved the Worked/Didn’t Work part of the training.” Duncan participant
“Finally, I understand the form I am being asked to fill out and I see why it is important.”
Ardmore participant
LESSONS FOR SELF-ADVOCATES
If given the right training and tools, we can “do it ourselves.”
We took the time to develop an evaluation that made sense to people with I/DD.
We realized we needed more information than what we could gather on paper, so we started using “What Worked/What Didn’t Work” to gather more information from the training participants and trainers.
The “Worked/Didn’t Work” process gave us valuable information on revising the trainings to meet the needs of the participants.
The evaluation we developed makes sense to people with I/DD and they are more willing to fill out the evaluation.
We have pride in the training and the evaluation, and feel we gained the respect of our partners.
q  We can be important and successful partners.
STEP 4: Mentoring
CLL staff mentored initially.
Self-advocates assume role of mentor to new trainers.
Then assumed role of Master Trainers.
STEP 4: Self-Advocates are consultants to CLL staff to determine evaluation needs. They train staff to know what is useful and needed for self-advocates.
Wow! Somebody cares what I think.”
Oklahoma City participant
STEP 5 : Certification of Trainers
Complete all steps to become certified trainer.
Attend a 2-day class and train- the-trainer session.
Successfully train twice.
Attend 75% of scheduled trainer meetings.
STEP 5: We had to try various evaluation formats to figure out the appropriate evaluation tool(s).
TRAINING EVALUATION QUESTIONS
TECHNICAL ASSISTANCE EVALUATION QUESTIONS
ORIGINAL
NOW
NOW
ORIGINAL
OKSAN is a partnership between the Center for Learning and Leadership/OK UCEDD and Oklahoma People First, and receives funding through the Oklahoma Developmental Disabilities Council. The Center for Learning and Leadership/ Oklahoma UCEDD is a collaborative initiative of the University of Oklahoma Health Sciences Center, sponsored by the OU College of Medicine, with support from the U.S. Department of Health and Human Services, Administration on Intellectual and Developmental Disabilities grant #90DD0685.