Tia McNair

Use Q&A for: Panel discussion Use Chat for: Technology support #aacuVALUE Slides, recording, and webinar Q&A will be posted online: www.aacu.org/webinar/adding-value

Webinar Presenters Terrel Rhodes Vice President for Quality, Curriculum, and Assessment and Executive Director of VALUE,  Association of American Colleges and Universities (AAC&U) Rhodes@aacu.org Kate McConnell Assistant Vice President for Research and Assessment, Association of American Colleges and Universities (AAC&U) mcconnell@aacu.org

Our Agenda Background and Context Implementing VALUE on a local level VALUE Scoring and capacity building Beyond campus boundaries: Landscape of Learning Your questions—and answers!

Who’s Here Today? Assessment coordinator Department chair Faculty member Institutional research staff General education assessment coordinator Center for teaching and learning leader Other (tell us your role in the chat feature)

Knowledge, Skills, Abilities Poll

Assessing student learning? Exceedingly Expert Skilled Practitioner Enthusiastic Novice Completely Clueless

Course-embedded assessment? Exceedingly Expert Skilled Practitioner Enthusiastic Novice Completely Clueless

Assessing student learning using rubrics? Exceedingly Expert Skilled Practitioner Enthusiastic Novice Completely Clueless

Assessing student learning using the AAC&U VALUE rubrics? Exceedingly Expert Skilled Practitioner Enthusiastic Novice Completely Clueless

Context—AAC&U AAC&U—Association of American Colleges and Universities (crowdsourcing and translation!) LEAP—Liberal Education and America’s Promise VALUE—Valid Assessment of Learning in Undergraduate Education (AKA, the RUBRICS!)

Evidence for Reporting Authentic Learning Strategies Expert/Novice Divide Guided Practice with Feedback Scaffolded Learning High-Impact Practices (HIPs) Guided Learning Pathways Signature Work VALUE Rubrics Authentic Assessment Direct Assessment Curricular Mapping Assignment Alignment Educational Research AAC&U “Translations" Evidence for Reporting

Institutions are using a variety of data collection approaches to yield actionable evidence: Trending towards greater use of authentic measures of student learning: rubrics, classroom-based performance assessments and capstones JK

Rubric = Accountability Tool

Rubric=Pedagogical Innovation with Assessment Potential

To Rubric or Not to Rubric? A legitimate question.

Plus Ça Change… “Rubrics are all the rage these days. It’s difficult to attend an educational conference without running into relentless support for the educational payoffs of rubrics.” —W. James Popham in Educational Leadership’s October 1997 issue

What does a VALUE rubric look like?

The VALUE Approach to Assessment

On Solid Ground shares results from first two years of VALUE Initiative data collection, a nationwide project to examine direct evidence of student learning. “This project represents the first attempt to develop a large-scale model for assessing student achievement across institutions that goes beyond testing.” — Lynn Pasquerella, President, AAC&U TR

Origin of AAC&U VALUE Rubrics 2007-2009 teams of faculty and other educational professionals developed 16 VALUE rubrics for the LEAP Essential Learning Outcomes Each rubric developed from research and practice for core identified characteristics or criteria of learning for each outcome Rubrics tested by faculty with their own students’ work TR

VALUE Assumptions Student work is representation of student motivated learning Focus on what student does in terms of key dimensions of learning outcomes Faculty and educator expert judgment Results are useful and actionable for improvement of learning Raise up, not wash out, inherent diversity–race, ethnicity, and socioeconomic status and diversity of courses, credit-levels, and disciplinary backgrounds JK

But it’s much more than just downloading rubrics…

VALUE is philosophically, pedagogically, and methodologically complex

For VALUE, a Careful Balancing Act

Nature and implications of complexity Establishing the validity and reliability of VALUE is a key priority Comparing the validity & reliability of the VALUE process to standardized tests will always be an “apples” to “oranges” proposition.

JK refer

“Many rubric problems are not (just) methodological, but pedagogical…”

“For classroom assessment to generate meaningful information, all aspects of the classroom experience must align pedagogically: course outcomes, course content, the learners themselves, the instructional and performance contexts, and the pedagogical strategies used. In other words, assessment must be related to the what, who, and how of teaching and learning.” (McConnell & Doolittle, 2017)

(and nationwide context) Local implementation (and nationwide context)

The case for using the VALUE rubrics “as is”

JK refer

VALUE Institute invites the higher education community writ large to engage in a nuanced, robust examination of the quality of student learning and to explore measures of success for all students.

VALUE Rubrics Civic Engagement Integrative Learning Creative Thinking Intercultural Knowledge & Competence Critical Thinking Oral Communication Ethical Reasoning Problem Solving Foundations/Skills for Lifelong Learning Quantitative Literacy Global Learning Reading Information Literacy Teamwork Inquiry and Analysis Written Communication

How does the VALUE Institute work? JK

Reasons for Participating in VALUE Institute University X: General Education assessment. Campus is reviewing proposals for changes to core curriculum requirements that could affect how written communication and critical thinking skills are developed. Wanted to move from current end of program assessment to an Early and Late model and identify how these skills are developing. University Y: To gain an outside-of-the-institution scoring of student learning goals. Hoping to have the ability to benchmark student performance to better understanding students' strengths and weaknesses. JK

Big Picture For each learning outcome selected, artifacts are collected Demographic information about the students and some metadata about the assignment/course/source of the work is also collected These data are combined for scoring and data analysis reporting The results are provided back to the submitting participant in reports and a database JK

Scoring Each artifact scored by two trained, certified VALUE scorers VALUE Institute trained and certified scorers for specific rubric are invited to score Artifacts are anonymized by campus Artifacts for which the two scorers notably disagree (in that the majority of dimensions received scores more than two apart) undergo an additional third scoring from an “expert” VALUE scorer.

Institutional Reports Critical Thinking results SAMPLE Results breakdown of student scores overall by rubric criteria and levels JK

Institutional Reports Critical Thinking results SAMPLE Results by Pell Eligibility

Registration—October 15

Webinar Presenters Terrel Rhodes Vice President for Quality, Curriculum, and Assessment and Executive Director of VALUE,  Association of American Colleges and Universities (AAC&U) Rhodes@aacu.org Kate McConnell Assistant Vice President for Research and Assessment, Association of American Colleges and Universities (AAC&U) mcconnell@aacu.org

Use Q&A for: Panel discussion Use Chat for: Technology support #aacuVALUE Slides, recording, and webinar Q&A will be posted online: www.aacu.org/webinar/adding-value

Forthcoming AAC&U Webinars Tuesday, October 22 3:00-4:00 p.m. Strategies for Supporting Faculty Planning to Advance Curricular Coherence: Four Case Studies Wednesday, October 30 2:00-3:00 p.m. The Emerging Landscape of Learning: A Review of Five Years of AAC&U’s VALUE Initiative Thursday, November 14 1:30-2:30 p.m. Fostering Civic Consciousness through Creativity-Based Pedagogy Wednesday, December 11 STEM Matters: The Formula for Driving Innovation

Tia McNair