1. Assessing Essential Learning Outcomes with the “VALUE” METARUBRICs Using Rubrics for Teaching, Learning, & Assessment

2. Introductions Office of Academic Assessment 101 Hillcrest Hall (0157) – Ray Van Dyke, 231-6003, rvandyke@vt.edurvandyke@vt.edu – Steve Culver, 231-4581, sculver@vt.edusculver@vt.edu – Kate Drezek, 231-7534, kmdrezek@vt.edukmdrezek@vt.edu Others here today

3. Today’s agenda Rubrics as an assessment tool – Definition and Overview of Rubrics – Using Rubrics for Program Assessment – Pros/Challenges VALUE METARUBRICS – Assessment toolkit – Background – Purpose – Application at Virginia Tech

4. Overview: What is Assessment of Learning Outcomes? “Assessment of student learning is the systematic gathering of information about student learning, using the time, resources, and expertise available, in order to improve the learning.” – Walvoord A student learning outcome states a specific skill/ability, knowledge, or belief/attitude students are expected to achieve through a course, program, or college experience. Example: Upon completion of a B.A. degree in English, a student will be able to read critically and compose an effective analysis of a literary text.

5. What is The Process for Assessing Student Learning Outcomes? 2. Gather and Analyze Information About Student Achievement Of Outcomes 3. Use Information Gathered To Improve Student Learning 1.Identify And Articulate Student Learning Outcomes

6. Rubric as an Assessment Tool A rubric is an explicit summary of the criteria for assessing a particular piece of student work, plus levels of potential achievement for each criterion. Not only can rubrics can be used to assess an individual student’s performance for a course grade, rubrics can be used to assess the achievement of course, program, or college/ institutional-level student learning Outcomes

7. Rubric as an assessment tool To the extent that a given student assessment measures or relates to a programmatic outcome, it can potentially be aggregated and used for program assessment (course- embedded assessment) The same student product (e.g., essay, capstone project) is used twice: once for individual student assessment and again for program

8. Rubric as an assessment tool Pros – Course-embedded assessment helps to overcome student motivation problem – Inherent efficiencies – “two birds, one stone” – Authenticity of student learning “artifact”

9. Rubric as an assessment tool Challenge – the rubric! – Think-pair-share Take a moment and jot down as many challenges associated with rubrics as you can Pair up with a neighbor and share your list

10. Rubric as an assessment tool Challenge – the rubric! Needs to be well-designed – Time – to create, to test, to implement, to assess – Reliability/validity – needs to be well-designed

11. Well-designed rubrics increase an assessment’s reliability by setting criteria that raters can apply consistently and objectively. Well-designed rubrics increase an assessment’s construct and content validity by aligning evaluation criteria to standards, curriculum, and instruction tasks. Rubric as an assessment tool

12. If, in theory, rubrics can be a useful assessment tool… AAC&U’s VALUE METARUBRICS ARE A TANGIBLE “TOOL KIT” TO FACILITATE THE APPROPRIATE AND MEANINGFUL ASSESSMENT OF THE ESSENTIAL LEARNING OUTCOMES THAT ARE THE HALLMARK OF A 21 st CENTURY LIBERAL EDUCATION

13. VALUE METARUBRICS: ASSESSMENT TOOLKIT Assessing essential learning outcomes

14. AAC&U – Organization – role in assessment, accountability – “LEAP” project – Liberal Education & America’s Promise VALUE METARUBRICS: Assessment toolkit Background

15. VALUE METARUBRICS: Assessment toolkit Empowers individuals and prepares them to deal with complexity, diversity, and change. Provides students with broad knowledge of the wider world (e.g. science, culture, and society) as well as in-depth study in a specific area of interest. Helps students develop a sense of social responsibility, as well as strong and transferable intellectual and practical skills such as communication, analytical and problem-solving skills, and a demonstrated ability to apply knowledge and skills in real-world settings usually includes a general education curriculum that provides broad learning in multiple disciplines and ways of knowing, along with more in-depth study in a major. Liberal Education is an approach to learning that: http://www.aacu.org/leap/What_is_liberal_education.cfm

16. VALUE METARUBRICS: Assessment toolkit http://www.aacu.org/leap/What_is_liberal_education.cfm AAC&U's LEAP Campaign has defined a robust set of "Essential Learning Outcomes" that students develop during an excellent contemporary liberal education. Beginning in school, and continuing at successively higher levels across their college studies, students should prepare for twenty-first-century challenges by gaining all these outcomesLEAP CampaignEssential Learning Outcomes

17. VALUE METARUBRICS: Assessment toolkit Valid Assessment of Learning in Undergraduate Education seeks to contribute to the national dialogue on assessment of college student learning. builds on a philosophy of learning assessment that privileges multiple expert judgments of the quality of student work over reliance on standardized tests administered to samples of students outside of their required courses. The assessment approaches that VALUE advances are based on the shared understanding of faculty and academic professionals on campuses from across the country. VALUE project: PURPOSE http://www.aacu.org/value/

18. VALUE METARUBRICS: Assessment toolkit to achieve a high-quality education for all students, valid assessment data are needed to guide planning, teaching, and improvement; colleges and universities seek to foster and assess numerous essential learning outcomes beyond those addressed by currently available standardized tests; learning develops over time and should become more complex and sophisticated as students move through their curricular and co-curricular educational pathways toward a degree; VALUE project: ASSUMPTIONS http://www.aacu.org/value/

19. VALUE METARUBRICS: Assessment toolkit good practice in assessment requires multiple assessments, over time; well-planned electronic portfolios provide opportunities to collect data from multiple assessments across a broad range of learning outcomes while guiding student learning and building self-assessment capabilities; e-portfolios and assessment of work in them can inform programs and institutions on progress in achieving expected goals. VALUE project: ASSUMPTIONS http://www.aacu.org/leap/What_is_liberal_education.cfm

20. VALUE METARUBRICS: Assessment toolkit VALUE project: Connections to VT http://www.aacu.org/value/ Essential Learning Outcomes – Incorporated into purpose statement for cle: http://www.cle.prov.vt.edu/purpose.html http://www.cle.prov.vt.edu/purpose.html – Charge to QEP from Provost – Alignment with SCHEV Core Competencies – VT “Partner Institution” in VALUE project

21. VALUE METARUBRICS: Assessment toolkit VALUE project: Connections to VT http://www.aacu.org/value/ VT “Partner Institution” for Value Project – Opportunity to use, provide feedback, shape rubrics – Join Google Group with access to rubrics as they are revised – Final rubrics – fall 2009

22. VALUE METARUBRICS: Assessment toolkit VALUE project: Why META????? http://www.aacu.org/value/ Best thinking of higher ed faculty professionals Specific enough to be useful, broad enough to be useful, meaningful in multiple contexts, at multiple levels for assessment

23. To the METARUBRICS! 1. Civic engagement 2. Creative thinking 3. Critical thinking 4. Ethical reasoning 5. Foundations and skills for lifelong learning 6. Information literacy 7. Inquiry and analysis 8. Integrative learning 9. Intercultural knowledge and competence 10. Oral communication 11. Problem solving 12. Quantitative reasoning 13. Teamwork 14. Written communication

24. To the METARUBRICS! Identify learning outcome(s) of interest to program? Which essential learning outcome(s) are in alignment with program goals/outcomes? Which rubric(s)/elements of the rubric(s) can be incorporated into assessment efforts?

25. The Big Message Avoids Re-Inventing the wheel, staring at a blank page Potential connections to other campuses, best practices for using metarubrics More sophisticated, authentic approach to capturing information on our students’ learning

26. Rubrics help us to Reflect Before we React to our work “We are being pummeled by a deluge of data and unless we create time and spaces in which to reflect, we will be left with only our reactions.” – Rebecca Blood

27. Questions/Comments? (Please take a Few minutes to complete the session feedback form. Thanks!)