1. The Quest for Quality: Monitoring and Measuring Outcomes in Continuing Nursing Education Session 2: Selecting Provider Unit Outcomes Measures Pamela S. Dickerson, PhD, RN-BC, FAAN Director of Continuing Education, Montana Nurses Association President, PRN Continuing Education Presented by Ohio Nurses Association and Montana Nurses Association 1

2. Disclosures Purpose: Participants will effectively develop learning and provider unit processes that lead to measurable outcomes demonstrating quality patient care and/or enhanced professional development of registered nurses. Criteria for Successful Completion: Participate in the entire webinar. Complete the evaluation and registration information and return to ONA as directed. Conflict of Interest: There is no conflict of interest for any planner or presenter for this activity. 2

3. Objective Determine appropriate outcome measures to guide the work of your approved provider unit. 3

4. Self-Assessment I know how toStrong = 3 Moderate = 2Weak = 1 Set goals for my provider unit Develop a quality outcome measure Assess learning activities in relation to provider unit goals Evaluate quality outcome measures as a way to determine provider unit effectiveness 4 Total Score =

5. My total score was closest to a: 4 7 9 12 5

6. 6

7. Our Ultimate Goal Reflected in ANA/NNSDO Nursing Professional Development Scope and Standards of Practice Model Reflected in ANCC Accreditation Program Model 7

8. © 2010 ANA & NNSDO – Used with permission 8

9. Accreditation Conceptual Framework, ANCC 2013 (used with permission) Nursing Professional Development : Continuing Nursing Education Educational Design Process Assessment of Learning Needs Planning and Faculty Design Principles Achievement of Objectives Quality Outcomes Evaluation Process Evaluation Participation Goals for Improvement Value/Benefit to Nursing Professional Development Structural Capacity Commitment Accountability Leadership Resources 9

10. What Is A Quality Measure? “Quality measures are metrics, indicators or ratings which provide some insight into the quality of care that patients are receiving in a particular setting or place”. - Pennsylvania Health Care Quality Alliance Following that line of thought, quality measures in continuing education are metrics that provide some insight into the quality of the work of our provider units that contributes to quality patient care and nursing professional development. 10

11. Quality Measures Abound Common examples of quality measures: o Your most recent performance appraisal o Amount of money in your checking account o Number of dust bunnies under your bed o Amount of gas in your car o Your child’s PSAT score o Amount of chocolate in your house 11

12. Examples of Outcomes Measures: Health Care Organizations Morbidity Mortality Healthcare Facility-Acquired Infections Readmission Rates Patient Satisfaction (HCAHPS) Quality of Life 12

13. Examples of Outcomes Measures: Nursing (NDNQI) Clinical Falls Symptoms, e.g. pain Central line infections Catheter-associated UTIs Ventilator associated pneumonia rate Pressure ulcers Other… 13 Professional Nurse hours per patient day Nursing skill mix RN education and certification Retention Work environment Other…

14. Measuring Outcomes Outcome: “The impact of structure and process on the organization…and the value/benefit to nursing professional development.” (ANCC, 2013). Outcome Measurement: “The process of observing, describing, and quantifying predefined indicators of outcomes of performance.” (ANA/NNSDO, 2010) 14

15. Quality Outcomes Measure A measurable component of the structure or process of a provider that demonstrates the effectiveness of the provider unit. Examples: o Number of nurse planners (PU outcome measure) o Number of learning activities that include active learner participation (PU outcome measure) o Patient care outcome related to educational content (professional development/impact on care outcome measure) 15

16. Relationship of Quality Outcome Measures to Goals Goal: What the provider unit will accomplish in the year Outcome Measure: What evidence will be collected to determine goal achievement – how will we know if we’re successful? 16

17. Example PU Goal: Improve the diversity of learning activities available to nurses in our target market area. Outcome measures might include: o Number of “live” activities compared to previous year o Number of web-based activities compared to previous year o Number of interprofessional activities compared to previous year o Number of published independent studies compared to previous year 17

18. Comparison with a Learning Activity For a learning activity, the goal might be that learners will provide more person-centered care to patients with dementia. Measurable outcomes might be: o Observation of behaviors o Report of family members o Decreased number of care-related complaints o Calmer behavior of patients 18

19. Evaluation in Context: Activity Target Audience Needs Assessment Gap Analysis PurposeObjectives Content/ Teaching Methods Short-term Evaluation Outcome Long term evaluation ) 19

20. 20 Name of Session:Facilitator: Date:Location: Attendance (to be inserted for summative) Contact Hours: Please indicate your achievement of each objective: YesNo 1.______ 2.______ Please provide your response to each item: The facilitator was effective in teaching the subject.______ This session provided a good way for me to learn this content.______ What did you learn today that will strengthen your practice or personal behavior? What one change will you make in your practice as a result of participating in this activity? Generic Evaluation

21. Generic Evaluation (cont’d) What barriers do you think will make it difficult to implement what you have learned? What needs to happen to overcome these barriers? What more would you like to learn about this topic? Comments: Suggestions for future topics (please indicate what you would like to know about and WHY this is important in your practice): 21

22. Evaluation in Context: Provider Unit Goals for Provider Unit Outcome Measures Provider Unit Structure & Function Learning Activities Activity Evaluation Outcome Measurement for Provider Unit 22 PSD, 2013

23. Building Blocks Activity outcomes are specific to one learning activity. (EDP 13) Activities should be designed with provider unit goals in mind. Outcomes measures allow you to look at data in the aggregate to determine if provider unit goals are met. (QO 4, 5) 23

24. Your Turn Write a goal for your provider unit for 2014. 24

25. Your Turn Based on the goal you wrote, write the outcome measure or measures that will provide evidence of achieving that goal (or not). 25

26. Provider Outcomes Measures: Ideas Cost-effectiveness of PU operations (may include return on investment) Turnover of nurse planners Number of educational activities provided Types of educational activities offered Quality of learning activities o Learner perception o Faculty feedback o Effective in filling the identified gap 26

27. Example Provider unit goal for 2013: Increase the number of nurse planners so that there is a nurse planner for every clinical area. Outcome measures: o Number of nurse planners o Representation of clinical areas among nurse planners 27

28. What Was The Outcome? December, 2012 – 6 nurse planners for hospital with 1400 nurse employees; ED, critical care areas, OR/PACU, and palliative care did not have nurse planner representation January, 2014 – 10 nurse planners plus primary nurse planner – all clinical areas have nurse planner representation 28

29. Outcomes - Visually 29

30. Professional Development Outcomes Measures: Ideas Leadership skills Healthy work environment Healthy lifestyle / role modeling Critical thinking skills Nurse advocacy o For patients o For the profession o For the community – health promotion 30

31. Goal Example Goal Example Provide quality continuing education to all of our nurses. Is this an effective goal statement? o Yes o No 31

32. 32

33. Goal Example Provide quality continuing education This is very broad – what is quality? What kind of measurement could I use to evaluate it? What does success look like? Commonly used: Maintain approved provider status – so what? Hint: If you can’t think of an outcome measure for it, it probably isn’t a strong PU goal! 33

34. Goal Example Goal Example Provide targeted continuing education to hospice nurses in 2014 based on gap analysis data to increase patient’s perceptions of quality of care at the end of life by the end of the year. Is this an effective goal statement? o Yes o No 34

35. 35

36. Why Is This Goal Stronger? Provide targeted continuing education to hospice nurses in 2014 based on gap analysis data to increase patient’s perceptions of quality of care at the end of life by the end of the year. Outcome is clear Outcome measures can be selected: o Number of hospice nurses engaged (PU) o Number of activities based on gap analysis data (PU) o Patient perceptions of QOL (NPD) 36

37. Revisit Your Goal Is the outcome clear? Will you know when success has been achieved? Are there clearly evident outcome measures? Do you know what data you will collect to demonstrate success? Are your outcomes related to o The work of your provider unit? (QO 4) o Professional development of nurse learners? (QO 5) 37

38. Our Accountability in Nursing Professional Development Standard 3 (Outcomes Identification): o Develops outcomes that reflect professional role competence, learning and change o Documents outcomes, including those that demonstrate learning and program impact. ANA/NNSDO, 2010 38

39. Our Accountability in Nursing Professional Development Standard 6 (Evaluation): o Selects valid, reliable, and relevant methods and instruments to measure processes and outcomes. o Synthesizes evaluation data, trends, and expectations to guide decision-making about changes and improvement of all components of NPD practice ANA/NNSDO, 2010 39

40. Our Accountability in Nursing Professional Development Standard 16 (Leadership): o Influences decision-making bodies to maintain and improve quality nursing and professional development programs o Promotes the professional development program mission, goals, action plans, and outcome measures. ANA/NNSDO, 2010 40

41. Self-Assessment 41 I know how toStrong = 3Moderate = 2Weak = 1 Set goals for my provider unit Develop an outcome measure Assess learning activities in relation to provider unit goals Evaluate outcome measures as a way to determine provider unit effectiveness Your Score:

42. My total score was closest to a: 4 7 9 12 42

43. 43

44. Summary Outcomes measures for your provider unit help you to define and measure goal achievement. Quantifying and sharing of outcomes data is expected as part of our role as nursing professional development specialists. 44

45. Questions and Discussion 45

46. For Additional Information Pam Dickerson, PhD, RN-BC, FAAN pdickerson@aol.com pam@mtnurses.org Zandra Ohri, MA, MS, RN zohri@ohnurses.org 46

47. References American Nurses Association & National Nursing Staff Development Organization (2010). Nursing Professional Development: Scope and Standards of Practice. Silver Spring, MD. Nursesbooks.org American Nurses Credentialing Center. (2013). 2013 ANCC Primary Accreditation Application Manual for Providers and Approvers. Revision 3.0. Silver Spring, MD. Author. National Database for Nursing Quality Indicators. (2013). Data. Retrieved from https://www.nursingquality.org/data.aspxhttps://www.nursingquality.org/data.aspx Pennsylvania Health Care Quality Alliance. (2013). Our Measures. Retrieved from http://www.phcqa.org/measures/http://www.phcqa.org/measures/ 47