1. Designing Valid Reliable Grading Tools Using VALUE Rubrics
Joyce O’Reilly
Department of Nursing Education
Franklin Pierce University
Portsmouth, NH

2. Objectives Define student learning outcome (SLO)
Understand why it is important to assess SLO
Identify components of an analytic rubric
Understand VALUE Rubrics
Create an assessment tool using VALUE Rubrics

3. What is a student learning outcome – SLO?
Glad you asked!
The Council of Higher Education Accreditation defines SLO as
“the knowledge, skills, and abilities that a student has attained at the end of his or her engagement in a particular set of higher education experiences” (2003, p. 5)
Why is it important to assess SLO in higher education?

4. Why assess SLO?
Accrediting bodies and other stakeholders require institutions of higher learning to
Provide evidence students are learning what we say they are learning
Use the evidence to inform program changes
Of the 1400 Association of American Colleges & Universities (AAC&U) member institutions
91% use rubrics to assess student work
Of the 91%, 42% use VALUE Rubrics

5. What is a VALUE Rubric and why should I care?
VALUE = Valid Assessment of Learning in Undergraduate Education
Collaboration of >100 Association of American Colleges & Universities (AAC&U) members
Resulted in 15 rubrics with high reliability and validity
Three major categories
written communication
quantitative reasoning
critical thinking

6. VALUE Rubrics Civic Engagement
Foundations and Skills for Lifelong Learning
Oral Communication
Problem Solving
Quantitative Literacy
Reading
Teamwork
Integrative Learning
9. Creative Thinking 10. Critical Thinking 11. Ethical Reasoning 12. Information Literacy 13. Inquiry and Analysis 14. Intercultural Knowledge and Competence 15. Written Communication

7. Department of Nursing Education Evolution of Assessment Plan
Accreditation Commission for Education in Nursing (ACEN)
Identified lack of evidence of SLO assessment in undergraduate program
Concluded our assessment plan was lackluster
Faculty researched nursing program assessment plans
Faculty researched assessment tools
Decision made to develop assessment tools for benchmark assignments using VALUE Rubrics

8. Process used by nursing faculty
Examined all assignments in all undergraduate courses
Identified assignments from all courses that “matched” one or more of the program SLO
Dissected the assignment
Broke assignment into steps
Matched a “trait” from a VALUE Rubric to the assignment step
For example
“Identify the socioeconomic and health issues of a family”
Define Problem (Problem Solving Rubric)
“Critique five evidence based articles and identify the gaps in knowledge”
Analysis (Inquiry & Analysis Rubric)

9. What did the process look like?
I’m so glad you asked!

10. SLO # 7 Practice as accountable leaders of conscience demonstrating ethical and professional principles in the care of all individuals, families, and populations at risk in a variety of settings.
Ethics Paper Assignment (NR 320)
The purpose of this assignment is to assess understanding and analysis of ethical and professional principles in the care of all individuals, families, and populations at risk in a variety of settings.  

11. SLO #7 Practice as accountable leaders of conscience demonstrating ethical and professional principles in the care of all individuals, families, and populations at risk in a variety of settings.
Ethics Paper Assignment
The purpose of this assignment is to assess understanding and analysis of ethical and professional principles in the care of all individuals, families, and populations at risk in a variety of settings.  

12. Ethics Paper Description
Students will demonstrate ethical and professional principles by:
Discussing the issues of respect, beneficence, and justice relevant to a specific vulnerable population who could be in a research study.
Exploring special considerations that may be required for informed consent for the chosen vulnerable population.
Describing how they (the student) values the need for ethical conduct in practice and research when dealing with a vulnerable patient and informed consent.

13. Ethics Paper Description, cont.
4. Describing the interprofessional and intraprofessional resources used to solve the ethical dilemmas of your vulnerable population. 5. Applying the ethical perspectives of respect, beneficience and justice when acting to prevent unethical practice. 6. Analyzing implications for nurses involved in the research involving vulnerable population and discuss barriers or limitations that might be encountered in practice.

14. Ethics Paper, cont.
Students will demonstrate ethical and professional principles by:
Discussing the issues of respect, beneficence, and justice relevant to a specific vulnerable population who could be in a research study.
Exploring special considerations that may be required for informed consent for the chosen vulnerable population.
Describing how they (the student) values the need for ethical conduct in practice and research when dealing with a vulnerable patient and informed consent.

15. VALUE Rubrics Civic Engagement
Foundations and Skills for Lifelong Learning
Oral Communication
Problem Solving
Quantitative Literacy
Reading
Teamwork
Integrative Learning
9. Creative Thinking 10. Critical Thinking 11. Ethical Reasoning 12. Information Literacy 13. Inquiry and Analysis 14. Intercultural Knowledge and Competence 15. Written Communication

16. Trait 4
Mastering 3
Proficient 2
Developing 1
Emerging
Ethical Reasoning:
1. Ethical Issue Recognition
Discusses the issues of respect, beneficence, and justice relevant to a specific vulnerable population who could be in research study.
Recognizes ethical issues when presented in a complex, multilayered (gray) context and can recognize cross-relationships among the issues.
Recognizes ethical issues when issues are presented in a complex, multilayered (gray) content or can grasp cross-relationships among the issues.
Recognizes basic and obvious ethical issues and grasps (incompletely) the complexities or interrelationships among the issues
Recognizes basic and obvious ethical issues but fails to grasp complexity or interrelationships.

17. Trait 4
Mastering 3
Proficient 2
Developing 1
Emerging
Ethical Reasoning:
1. Ethical Issue Recognition
Discusses the issues of respect, beneficence, and justice relevant to a specific vulnerable population who could be in research study. (20%)
Recognizes ethical issues when presented in a complex, multilayered (gray) context and can recognize cross-relationships among the issues. (20 pt)
Recognizes ethical issues when issues are presented in a complex, multilayered (gray) content or can grasp cross-relationships among the issues. (16-19 pt)
Recognizes basic and obvious ethical issues and grasps (incompletely) the complexities or interrelationships among the issues. (14-15 pt)
Recognizes basic and obvious ethical issues but fails to grasp complexity or interrelationships. (0-13 pt)

18. Ethics Paper, cont.
Students will demonstrate ethical and professional principles by:
Discussing the issues of respect, beneficence, and justice relevant to a specific vulnerable population who could be in a research study.
Exploring special considerations that may be required for informed consent for the chosen vulnerable population.
Describing how they (the student) values the need for ethical conduct in practice and research when dealing with a vulnerable patient and informed consent.

19. VALUE Rubrics Civic Engagement
Foundations and Skills for Lifelong Learning
Oral Communication
Problem Solving
Quantitative Literacy
Reading
Teamwork
Integrative Learning
9. Creative Thinking 10. Critical Thinking 11. Ethical Reasoning 12. Information Literacy 13. Inquiry and Analysis 14. Intercultural Knowledge and Competence 15. Written Communication

20. Trait 4
Mastering 3
Proficient 2
Developing 1
Emerging
Ethical Reasoning:
1. Ethical Issue Recognition
Discusses the issues of respect, beneficence, and justice relevant to a specific vulnerable population who could be in research study. (20%)
Recognizes ethical issues when presented in a complex, multilayered (gray) context and can recognize cross-relationships among the issues. (20 pt)
Recognizes ethical issues when issues are presented in a complex, multilayered (gray) content or can grasp cross-relationships among the issues. (16-19 pt)
Recognizes basic and obvious ethical issues and grasps (incompletely) the complexities or interrelationships among the issues. (14-15 pt)
Recognizes basic and obvious ethical issues but fails to grasp complexity or interrelationships. (0-13 pt)
2. Ethical Issue Recognition
Explores special considerations might be required for informed consent for the vulnerable population you choose? (15%)
Student can recognize ethical issues when presented in a complex, multilayered (gray) context AND can recognize crossrelationships among the issues. (15%)
Student can recognize ethical issues when
issues are presented in a complex, multilayered (gray) context OR can grasp crossrelationships among the issues. (12-14 pt)
Student can recognize basic and obvious
ethical issues and grasp (incompletely) the
complexities or interrelationships among the
issues. (10-11 pt)
ethical issues but fails to grasp complexity or Interrelationships. (0-9 pt)

21. Ethics Paper, cont.
Students will demonstrate ethical and professional principles by:
Discussing the issues of respect, beneficence, and justice relevant to a specific vulnerable population who could be in a research study.
Exploring special considerations that may be required for informed consent for the chosen vulnerable population.
Describing how they (the student) values the need for ethical conduct in practice and research when dealing with a vulnerable patient and informed consent.

22. VALUE Rubrics Civic Engagement
Foundations and Skills for Lifelong Learning
Oral Communication
Problem Solving
Quantitative Literacy
Reading
Teamwork
Integrative Learning
9. Creative Thinking 10. Critical Thinking 11. Ethical Reasoning 12. Information Literacy 13. Inquiry and Analysis 14. Intercultural Knowledge and Competence 15. Written Communication

23. Trait 4
Mastering 3
Proficient 2
Developing 1
Emerging
Ethical Reasoning:
1. Ethical Issue Recognition
Discusses the issues of respect, beneficence, and justice relevant to a specific vulnerable population who could be in research study. (20%)
Recognizes ethical issues when presented in a complex, multilayered (gray) context and can recognize cross-relationships among the issues. (20 pt)
Recognizes ethical issues when issues are presented in a complex, multilayered (gray) content or can grasp cross-relationships among the issues. (16-19 pt)
Recognizes basic and obvious ethical issues and grasps (incompletely) the complexities or interrelationships among the issues. (14-15 pt)
Recognizes basic and obvious ethical issues but fails to grasp complexity or interrelationships. (0-13 pt)
2. Ethical Issue Recognition
Explores special considerations might be required for informed consent for the vulnerable population you choose? (15%)
Student can recognize ethical issues when presented in a complex, multilayered (gray) context AND can recognize crossrelationships among the issues. (15 pt)
Student can recognize ethical issues when
issues are presented in a complex, multilayered (gray) context OR can grasp crossrelationships among the issues. (12-14 pt)
Student can recognize basic and obvious
ethical issues and grasp (incompletely) the
complexities or interrelationships among the
issues. (10-11 pt)
ethical issues but fails to grasp complexity or Interrelationships. (0-9 pt)
3. Ethical Self-Awareness
Describes how they value the need for ethical conduct in practice and research when dealing with a vulnerable patient and informed consent. (10%)
Student discusses in detail/analyzes both core beliefs and the origins of the core beliefs and discussion has greater depth and clarity. (10 pt) 
Student discusses in detail/analyzes both core beliefs and the origins of the core beliefs. (8-9 pt)
Student states both core beliefs and the origins of the core beliefs. (7 pt)
Student states either their core beliefs or
articulates the origins of the core beliefs but not both. (0-6 pt)

24. Trait 4
Mastering 3
Proficient 2
Developing 1
Emerging
Ethical Reasoning:
1. Ethical Issue Recognition
Discusses the issues of respect, beneficence, and justice relevant to a specific vulnerable population who could be in research study. (20%)
Recognizes ethical issues when presented in a complex, multilayered (gray) context and can recognize cross-relationships among the issues. (18-20 pt)
Recognizes ethical issues when issues are presented in a complex, multilayered (gray) content or can grasp cross-relationships among the issues. (16-17 pt)
Recognizes basic and obvious ethical issues and grasps (incompletely) the complexities or interrelationships among the issues. (14-15 pt)
Recognizes basic and obvious ethical issues but fails to grasp complexity or interrelationships. (0-13 pt)
2. Ethical Issue Recognition
Explores special considerations might be required for informed consent for the vulnerable population you choose? (15%)
Student can recognize ethical issues when presented in a complex, multilayered (gray) context AND can recognize crossrelationships among the issues. (15 pt)
Student can recognize ethical issues when
issues are presented in a complex, multilayered (gray) context OR can grasp crossrelationships among the issues. (12-14 pt)
Student can recognize basic and obvious
ethical issues and grasp (incompletely) the
complexities or interrelationships among the
issues. (10-11 pt)
ethical issues but fails to grasp complexity or Interrelationships. (0-9 pt)
3. Ethical Self-Awareness
Describes how they value the need for ethical conduct in practice and research when dealing with a vulnerable patient and informed consent. (10%)
Student discusses in detail/analyzes both core beliefs and the origins of the core beliefs and discussion has greater depth and clarity. (9-10 pt) 
Student discusses in detail/analyzes both core beliefs and the origins of the core beliefs. (8 pt)
Student states both core beliefs and the origins of the core beliefs. (7 pt)
Student states either their core beliefs or
articulates the origins of the core beliefs but not both. (0-6 pt)

25. Lessons Learned
Matching assignments (benchmark) to SLO required tweaking of both assignments and SLO
Process forced faculty to understand what the SLO measures
Developing rubrics helped identify important pieces of assignments
Process provided valuable insight into gaps in the curriculum
Allowed specific feedback to students
Provided feedback that has been used for course/program improvement
Promoted consistency with evaluation (particularly with adjuncts)

26. References
Accreditation Commission for Education in Nursing. (2013). Accreditation manual. Retrieved from Association of American Colleges & Universities. (2007). VALUE. Retrieved from Council for Higher Education Accreditation. (September 2003). Statement of mutual responsibility for student learning outcomes: Accreditation, institutions, and programs. Retrieved from Finley, A. (2011). How reliable are the VALUE rubrics? Peer Review, 13/14(4/1). Retrieved from Gleason, B. L., Gaebelein, C. J., Grice, G. R., Crannage, A. J., Weck, M. A., Hurd, P.,…Duncan, W. (2013). Assessment of students’ critical-thinking and problem-solving abilities across a 6-year doctor of pharmacy program. American Journal of Pharmaceutical Education, 77(8), Renjith, V., George, A. G. R., & D’Souza, P. (2915). Rubrics in nursing education. International Journal of Advanced Research, 3(1),