1. Consumer Experience Research 2019 Q2 Results August 2, 2019
Florida Rehabilitation Council
Division of Vocational Rehabilitation Florida Department of Education
Consumer Experience Research 2019 Q2 Results August 2, 2019
Brian Robertson
Mark Noyes
Jotham Illumanati

2. Contents Survey Methodology Trends to Watch
Consumer Experience Core Metrics
Problems Experienced by VR Consumers and Feedback
Understanding the Process if You Disagree with VR About Your Services
Satisfaction with Outside Vendors
Current Employment Status
Those Whose Cases Were Unsuccessfully Closed
Youth in Transition Analysis
Upcoming Research 2

3. Current Quarter Survey Methodology
Data collection took place from April 5, 2019 through July 17, 2019.
Over the course of data collection, 751 consumers completed surveys.
302 general VR consumers with open cases
154 general VR consumers - closed successfully
132 general VR consumers - closed unsuccessfully
163 transition youth consumers 3

4. Current Quarter Response Rates%
APPOR Response Rate (RR3)
15.3%
AAPOR Respondent Cooperation Rate (COOP3)
80.7%
AAPOR Respondent Refusal Rate (REF3)
7.0%
AAPOR – The American Association for Public Opinion Research.
Response Rate – The response rate is the number of completed interviews divided by the number of eligible reporting consumers in the sample.
Cooperation Rate – This represents the proportion of all cases interviewed out of all eligible consumers ever contacted. That is, the percent of identified respondents that ended up completing the interview.
Refusal Rate – The refusal rate represents the proportion of all cases in which the respondent refused to be interviewed, or broke off an interview, out of all respondents that were contacted and spoken with. 4

5. Trends to watch 5 Trending Positive in 2019 Q2
Communication with VR staff
Consumer control and involvement
Trending Less Positive in 2019 Q2
Satisfaction with current employment
Satisfaction with outside vendors
A number of metrics measuring understanding of the dispute resolution process trended downward
The percentage reporting problems did increase
The percentage reporting problem resolution decreased
Increase in those reporting switching counselors is a problem 5

6. 2019 Q2 Survey Results 6

7. Consumer Experience Core Metrics7

8. Florida VR Consumer Experience Dashboard
Measure
2018 Q3
2018 Q4
2019 Q1
2019 Q2
Overall Satisfaction and Expectations 79 78
Experience with Services Provided by VR 81 80
Experience with VR Staff and Counselors 90
Communications with VR Staff 77
Consumer Control and Involvement
Outcomes and Meeting Goals 84 82
Satisfaction with Current Employment 73
Satisfaction with Outside Vendors 74
All things considered, would you tell your friends with disabilities to go to the VR program for help? (% Yes) 94 91 93
Have you experienced any problems with VR or the services they have provided to you? (% No) 76 83 8

9. Overall Satisfaction and Expectations:
Q01: Overall, how satisfied are you with the Florida VR’s program? Q33: How satisfied would you say you are with the services provided by VR? Q35: Please rate how well VR services met your expectations. 9

10. Experience with Services Provided by VR:
Q09: How satisfied are you with the choice of services?
Q11: How satisfied are you with the choice of service providers? 10

11. Experience with VR Staff and Counselors:
Q19: How helpful is the staff of VR in helping you achieve your VR goal?
Q29: The VR staff treats me with dignity and respect. 11

12. Communications with VR Staff:
Q13: How satisfied are you with the information you were given about the choices you had?
Q15: How satisfied are you with how long it took your counselor to answer your questions and address your concerns?
Q21: How easy is it for you to contact your VR counselor? 12

13. Consumer Control and Involvement:
Q05: How satisfied are you with your involvement in your VR experience?
Q07: How satisfied are you with your choice of a vocational goal? 13

14. Outcomes and Meeting Goals:
Q25: The VR services I am receiving will help me become more financially independent.
Q27: The VR services I am receiving will help me become more independent, in general.
Q31: The VR staff will help me reach my job goal. 14

15. Satisfaction with Current Employment:
Q51: If working, thinking about your current job, how satisfied are you with what you are doing? Would you say you are...? 15

16. Satisfaction with Outside Vendors:
Q46: How satisfied were you with the services provided by OUTSIDE VENDOR? 16

17. Recommend VR:
All things considered, would you tell your friends with disabilities to go to the VR program for help?
(% reporting yes) 17

18. Experienced Problems:
Have you experienced any problems with VR or the services they have provided to you?
(% reporting no) 18

19. Problems Experienced by VR Consumers and Feedback
Consumers were asked if they experienced problems specifically with VR or the services they received from VR and reasons for dissatisfaction. 19

20. 22% of consumers experienced problems with VR or the services provided.20

21. The Most Commonly Mentioned Problems (2019 Q2):%
Changing counselors, switching too much, causes problems
25%
Counselor was not helpful or supportive
22%
Received no help in reaching plan or goals
19%
Didn't receive job search help
17%
Counselor needs to make more effort, consumer does all the work
16%
Listen to consumer, understand needs, wants, ability
14%
% among those experiencing problems 21

22. % among those experiencing problems
29% of consumers reporting problems indicated that VR worked to resolve their problem(s).
% among those experiencing problems 22

23. Consumer Feedback
Consumers were asked to provide additional feedback when they were not positive in their assessment.
Feedback was collected throughout the survey in response to 25 questions.
Consumers could provide more than one response.
The following table summarizes the most common responses from 527 consumers that provided additional feedback. 23

24. % among those providing feedback
Consumer Feedback %
Changing counselors, switching too much, causes problems
48%
Counselor did not return calls, s or follow up
37%
Counselor needs to make more effort, consumer does all the work
33%
Counselor was not helpful or supportive
Did not receive employment, VR could not find me a job
25%
Received no help in reaching plan or goals
22%
Listen to consumer, understand needs, wants, ability
18%
% among those providing feedback 24

25. Understanding the process if you disagree with VR about your services
Consumers were asked if they were aware of the options available if they disagreed with VR’s decisions about the services they did or did not receive. 25

26. 83% of consumers were aware they could contact their VR counselor in cases of disagreement.26

27. 65% of consumers were aware they could contact the VR Unit Supervisor.27

28. 48% of consumers were aware they could contact the VR Ombudsman Office.28

29. 50% of consumers were aware they could contact Disability Rights Florida, Client Assistance Program.29

30. 67% of consumers were aware they could appeal the decision(s) they disagreed with, if necessary.30

31. Satisfaction with Outside Vendors
Consumer were asked to provide feedback on the services they received from outside vendors. 31

32. % among those receiving services from an outside vendor
75% of consumers with an outside vendor were satisfied with the services provided.
% among those receiving services from an outside vendor 32

33. % among those receiving services from an outside vendor
29% of consumers using an outside center reported problems with their vendor.
% among those receiving services from an outside vendor 33

34. Feedback about Vendors (2019 Q2):%
Vendor no help or help not effective
43%
Vendor would not listen, dismissed concerns
37%
Vendor did not return calls, s or follow up
33%
Did not receive or received little help from Vendor
32%
Negative experience or dissatisfied
23%
% among those not satisfied or that experienced problems with an outside vendor 34

35. % among those receiving services from an outside vendor
71% of consumers felt that both the VR and their vendor(s) kept them informed and up-to-date.
% among those receiving services from an outside vendor 35

36. What worked well during experiences with outside vendors:
VEND05: In thinking about your experience with the outside vendor what worked well for you?
(Multiple responses accepted) %
Having help finding a job and with applications
11%
Nothing in particular
Communication and accessibility of counselor
10%
All, everything
Having support system with a counselor to determine goal and achieve that goal 9%
% among those receiving services from an outside vendor 36

37. Challenges consumers faced while working with outside vendors:
VEND06: What has been the most challenging part of your experience working with the vendor?
(Multiple responses accepted) %
Communication in general
21%
Finding desirable position or school
17%
Process of seeking employment 8%
Personal barriers, being independent, coping with disability
Staff not being helpful in job search
Working with certain staff members in general 7%
% among those receiving services from an outside vendor 37

38. Current Employment Status38

39. 37% of current and former consumers are working full or part time.
Q50: Finally, these last few questions ask about what you are currently doing. Are you currently...? %
Working full time, that is, more than 35 hours per week
14%
Working part time
23%
In school or receiving job training
46%
Currently looking for a job
30%
Keeping house
18%
Volunteering my time
15%
Currently unable to work 5% 39

40. 86% of working consumers were satisfied with their current job.
% among those with currently employed 40

41. Feedback on Employment%
Not a career move, only temporary job
35%
Low pay, does not meet financial need
23%
Dissatisfied with aspects or parts of job
18%
Unpleasant work environment
16%
Too few hours
15%
Need a job aligned with skills, training
13%
% among those with not satisfied with current employment 41

42. 78% of working consumers find their job meaningful.
% among those with currently employed 42

43. Those Whose Cases Were Unsuccessfully Closed
Consumers were asked if they requested their case be closed and reasons for doing so. 43

44. % among those with cases closed unsuccessfully
37% of consumers with unsuccessful cases asked for their case to be closed.
% among those with cases closed unsuccessfully 44

45. (Multiple responses accepted, most common responses)
Consumers asked for their case to be closed due to an injury or disability, or they had other priorities and needed a break.
(Multiple responses accepted, most common responses) %
Unable to work due to injury or disability
15%
Have other priorities, needed a break
13%
Dissatisfied with VR, services not effective
12%
Found employment, have a job
Moved away from VR
% among those with cases closed unsuccessfully asking for case to be closed 45

46. UNSUC01B: Why do you believe your case was closed?
Among those that did not ask for their cases to be closed, the most frequent reason they felt their case was closed was a lack of communication.
UNSUC01B: Why do you believe your case was closed? %
Lack of communication
24%
VR couldn't help, never found jobs
16%
None of these
10%
Completed program, received all expected services 9%
Consumer did not qualify for services 7%
Consumer relocated 4%
% among those with cases closed unsuccessfully that did not ask for case to be closed 46

47. % among those with cases closed unsuccessfully
56% of consumers whose cases closed unsuccessfully received a closure letter with the reason for closure and their rights to appeal.
% among those with cases closed unsuccessfully 47

48. Youth in Transition Analysis48

49. Florida VR Consumer Experience Dashboard Comparisons
Measure
General VR
Youth in Transition
Overall Satisfaction and Expectations 78 79
Experience with Services Provided by VR 81
Experience with VR Staff and Counselors 89 91
Communications with VR Staff 77
Consumer Control and Involvement 80 82
Outcomes and Meeting Goals 86
Satisfaction with Current Employment 72 87
All things considered, would you tell your friends with disabilities to go to the VR program for help? (% Yes) 93
Have you experienced any problems with VR or the services they have provided to you? (% No) 84 49

50. % among youth in transition experiencing problems
Youth report problems at a slightly lower rate than consumers overall (16% vs 22%). But they experience different problems. %
Did not receive needed education or training
16%
Had to leave multiple messages before getting a call back
13%
Changing counselors, switching too much, causes problems
10%
Counselor did not return calls, s or follow up
Nothing achieved, VR not effective 8%
Broken promises, no follow through 7%
Dissatisfied with VR counselor
Time lags to get into the program
Counselor would not listen, dismissed concerns
% among youth in transition experiencing problems

51. % among youth in transition
About half of Pre-ETS recognized at least one of the services they were receiving. However, a fifth reported not using any of the services FLVR provides. %
Workplace readiness to help get ready for the challenges of work
57%
Work based learning experiences including a job, internship, or volunteering
49%
Job exploration counseling or career counseling
Instruction in self-advocacy or peer-mentoring to help you live the life you want
48%
Counseling on opportunities to attend college, learn a trade, get professional or industrial certification
34%
Did not use any of these services
21%
% among youth in transition

52. % Finding Service Valuable % among youth in transition using service
Almost all youth using these services programs found the services valuable.
% Finding Service Valuable
Job Exploration and Career Counseling
98%
Work-based Learning Experiences
92%
Opportunities for further education
90%
Workplace Readiness Services
93%
Instruction in Self-Advocacy
% among youth in transition using service

53. An Update of Upcoming Research53

54. MDR and Florida are collaborating on a presentation and roundtable discussion at the upcoming PEQATAC Summit Conference.
Understanding Pre-ETS Populations: What to Ask and What to Know
The focus is to identify what should be asked of YIT and the best ways to gather the data.
It’s time to think about any changes to the survey.
FRC and FLVR will need to think about what they would like to see added and what can be eliminated.
Any changes will be incorporated into the October 2019 survey.
We are scheduled to conduct two focus groups in 2020.
FRC and FLVR will need to think about the focus for these groups.
FRC and FLVR will also need to think about timing and location.
Update 54

55. I’d be happy to answer any questions you might have.
Thank you
Thank You
I’d be happy to answer any questions you might have.