2. AAC&U is working in collaboration with Indiana University’s Center for Postsecondary Research (IUCPR) and the Multi-State Consortium of State Higher Education Executive Officers (SHEEO) to establish the VALUE Institute, which is a continuing resource for higher education institutions to document, report, and use learning outcomes evidence to improve student success in college. VALUE INSTITUTE Registration Now Open

3. Use Q&A for:
Panel discussion
Use Chat for:
Technology support
#aacuVALUE

4. Speakers Moderator Terrel R. Rhodes
Vice President, Office of Quality, Curriculum, and Assessment and Executive Director of VALUE  Association of American Colleges and Universities (AAC&U)
Thomas Nelson Laird Director, Indiana University’s Center for Postsecondary Research (IUCPR)
Robert J. Awkward
Director of Learning Outcomes Assessment Massachusetts Department of Higher Education
Cara Smith Director of Institutional Effectiveness Georgia College

5. Terrel R. Rhodes
Vice President, Office of Quality, Curriculum, and Assessment and Executive Director of VALUE  Association of American Colleges and Universities (AAC&U)

6. Origin of AAC&U VALUE Rubrics
Critical Thinking, Creative Thinking, Written Communication, Oral Communication, Quantitative Literacy, Information Literacy, Teamwork, Problem Solving, Civic Knowledge & Engagement, Intercultural Knowledge & Competence, Ethical Reasoning & Action, Global Learning, Lifelong Learning, Integrative Learning, Inquiry and Analysis, Reading
2007–2009 teams of faculty and other educational professionals developed 16 VALUE rubrics for the LEAP Essential Learning Outcomes. Each rubric was developed from core identified characteristics or criteria of learning for each outcome. Rubrics tested by faculty with their own students’ work. TR

7. VALUE Institute Assumptions
Student work is representation of student motivated learning
Focus on what student does in terms of key dimensions of learning outcomes
Faculty and educator expert judgment
Results are useful and actionable for improvement of learning
Raise up, not wash out, inherent diversity—race, ethnicity, and socioeconomic status and diversity of courses, credit-levels, and disciplinary backgrounds—on campuses. JK

8. Where Authentic Measures of Student Learning Come From: Assignments and Artifacts
Course-based assignments
Papers, lab reports, performances
General education outcome measures
Writing-Intensive sequence artifacts
Program-level (Major) capstone products
Portfolios
Cocurricular learning products/artifacts TR

9. Expert Faculty Scorers
Scorer training sessions that are equal parts calibration to reach a consensus score and a rich faculty development opportunity, and that are open to all faculty whether they are contingent or tenure-track, two-year or four-year, curricular or cocurricular. TR

10. VALUE Rubrics and Equity
The VALUE rubrics were purposefully designed to reflect an assets-based—versus deficit-focused—approach to assessing student learning (i.e., let’s focus on what students can do and build from that solid base).
The rubric “descends” from the level-four Capstone to the level-one Benchmark when reading from left to right.
Scorers are trained to assess student work by beginning at the highest levels of the rubric, working from the assumption that all students have the potential for achieving Capstone-level work.
Scorers immediately orient themselves to the learning that is possible.
TR Sullivan and McConnell in next issue of Change

11. VALUE Institute: Changing the Conversation about Learning Outcomes AssessmentJK

12. Reasons for Participating in VALUE Institute
Baylor: General Education assessment. Wanted to move from current end of program assessment to an Early and Late model and identify how these skills are developing.
University of North Dakota: To gain an outside-of-the-institution scoring of student learning goals. Hoping to have the ability to benchmark student performance to better understanding students' strengths and weaknesses. JK

13. Thomas Nelson Laird
Director, Indiana University’s Center for Postsecondary Research (IUCPR)

14. 2018 Spring Inaugural VALUE Institute:
26 Institutions
5 Outcomes
>5,000 artifacts
127 scorers
TNL

15. 2018 Spring Inaugural VALUE Institute Participants
Baylor University
Bluegrass Community and Technical College
Community College of Rhode Island
Gateway Community College
Georgia College
Grand Canyon University
Itasca Community College
Ivy Tech Community College
New College of Florida
Norfolk State University
North Shore Community College
Northern Kentucky University
Queen’s University
Quinnipiac University
Ryerson University
Southeast Missouri State University
Southern Connecticut State University
Southern Oregon University
State Fair Community College
Suffolk University
University of Hawaii at Manoa
University of Kentucky
University of Maine
University of Massachusetts Amherst
University of North Dakota
Vincennes University

16. VALUE Institute: Learning Outcomes (100 artifacts per outcome)
Outcome/Rubric
Number of Institutions
Written communication 20
Quantitative literacy 11
Critical thinking 22
Civic engagement
Ethical reasoning
Intercultural knowledge and competence
Global learning 1 JK *

17. How Does the VALUE Institute Work?
For each learning outcome selected, 100 artifacts can be scored
Demographic information about the students and some information about the assignment/course/source of the work is also collected
These data are combined for scoring and data analysis reporting
The results are provided back to the submitting participant in reports and a database JK

18. Institutional Reports
Critical Thinking results SAMPLE Results breakdown of student scores overall by rubric criteria and levels JK

19. Institutional Reports
Critical Thinking results SAMPLE Results by Pell Eligibility JK

20. A Few Words on Scoring
Each artifact scored by two trained, certified VALUE scorers
VALUE Institute-certified score is the calculated average of the two scores, rounded up to the nearest whole number
Artifacts that received a zero from either of the two scorers received a score of zero overall, as there was an absence of evidence of student learning on that dimension
Artifacts for which the two scorers notably disagreed (in that the majority of dimensions received scores more than two apart) underwent a separate process -- a third score from an “expert” VALUE scorer. These triple-scored artifacts were then analyzed for patterns to determine the nature of the “true” score - averaged the two original scores and rounding up to the nearest whole number JK

21. Critical Thinking
No Evidence
13%
21%
27%
15%
20%

22. Written Communication
No Evidence 1% 2% 4%
14%

23. Robert J. Awkward
Director of Learning Outcomes Assessment Massachusetts Department of Higher Education

24. Commonwealth of Massachusetts
Public higher education is comprised of:
Five University of Massachusetts campuses
Nine state universities
15 community colleges

25. Student Learning Outcomes Assessment
Brief History
External factors
Have faculty drive assessment
Avoid standardized tests
Address lack of assessment culture
Create assessment initiative
Working Group on Student Learning Outcomes and Assessment established the following in January 2010:

26. Massachusetts Organizing Structure
Student Learning Outcomes Assessment
Massachusetts Organizing Structure
Massachusetts Task Force on Statewide Assessment
Chair: Dr. Lane Glenn, President, Northern Essex Community College
Providing broad oversight for Massachusetts
State Level
Amcoa Team
Advancing assessment of student learning on public campuses; strengthening campus assessment capabilities
Chair: R. J. Awkward
VALUE Institute Steering Committee
Representing MA in the VALUE Institute/MSC to develop a multistate program of assessment
Chair: R. J. Awkward

27. How Did We Get Here? Massachusetts Pilot in 2012-2013
Student Learning Outcomes Assessment
How Did We Get Here?
Massachusetts Pilot in
Massachusetts joined with the Multi-State Collaborative (Mass. Pilot Phase II) in
AAC&U and SHEEO provided a common framework:
Authentic Student Work
LEAP VALUE Rubrics
• Critical Thinking • Quantitative Literacy • Written Communication
Common faculty training and scoring

28. Student Learning Outcomes Assessment
How Did We Get Here?
Multi-State Collaborative to Advance Quality Student Learning continued:
Demonstration Year ( )
Refinement Year ( )
VALUE Institute Inaugural Year ( )
Significant Learning Acquired as documented in “On Solid Ground” (VALUE Report, AAC&U, 2017)

29. Massachusetts Compared to Project Level Data
VALUE Rubrics
Two-Year Institutions
Four-Year Institutions
Critical Thinking +
Written Communication
+/-
Quantitative Literacy
-/+ -

30. Suggested Items for Individual Campuses & Statewide Action
Written Communications
More activities related to Written Communications in 3rd and 4th year courses, programs, and majors (except in Syntax and Mechanics)
Resume or enhance existing “writing across the curriculum” activities
Quantitative Literacy
Employ quantitative literacy across the curriculum in addition to general education courses
Increase activities related to Quantitative Literacy in 3rd and 4th year courses, programs, and majors
Massachusetts Department of Higher Education
DHE to provide regional assignment design workshops for faculty linked to VALUE Rubrics to improve teaching effectiveness, increase student learning, and improve outcomes results
DHE to provide training for faculty on how to employ Quantitative Literacy in curriculum at all levels by launching regional Quantitative Literacy faculty workshops in

31. Cara Smith Director of Institutional Effectiveness Georgia College cara.smith@gcsu.edu

32. Georgia College & State University
Located in rural Milledgeville, Georgia
4-year public, SACSCOC accredited, Master’s (larger programs)
Fall 2018:
4,329 applications, 3,367 admitted, 1,483 enrolled
Total enrollment: 6,989
Undergraduate: 5,958
Graduate: 1,031
Average SAT: 1,211
Average ACT: 25
17/1 student to faculty ratio

33. AAC&U ELO: Critical Thinking

34. GC Core Curriculum University System of Georgia (Areas A-F)
Area B, “Institutional Options”
GC1Y: Critical Thinking:_________________
Into the Wilderness, Ability/Disability, Social Problems, Music and Conflict, Utopia/Dystopia, Research in the Google Age, We the People, Growing up with Literature
GC2Y: Global Perspectives: _________________
The AIDS Pandemic, Arts & Literacy, Big Data, Black Women Artists of the African Diaspora, Captivity and Freedom, Global Connections, History of Global Health

35. Assessing Area B Customized hybrid rubric aligned to:
AAC&U Critical Thinking ELO rubric elements 1, 2 and 5
Area B core outcome (GC SLO 3)
GC Critical thinking outcome, element 2 (GC SLO 13)
However, faculty are supposed to utilize AAC&U Critical Thinking framing language, definition, and entire rubric when designing an Area B course

36. Administration Summary, Spring 2018
31 sections taught by 20 different professors
14 departments/disciplines represented
Employed a simple random sample within courses, soliciting 5 samples per section
Submitted 112 (out of a possible 155) artifacts from 14 unique assignments from 13 professors

37. Very positive for 1st year, first semester

38. How Will We Use Our Data?
Through dialogue, validate faculty teaching and student learning
Validate and strengthen internal assessment processes and practices (Spring 2019, Fall 2019)
Refine assignment alignment
Create faculty development opportunities
Train new GC1Y instructors

39. Benefit for Georgia College and Beyond
Benefit for Georgia College and Beyond? What was the Return on Our Investment?
VERY validating for what we are already doing
Inform future practice
Increased appreciation and awareness of assessment
Increased faculty buy-in
Develop faculty leaders
Coupled with this years’ participation, add to the national conversation of assessing high-impact practices (NASH TS3 HIPS Project)

40. JK

41. Use Q&A for:
Panel discussion
Use Chat for:
Technology support
#aacuVALUE

42. Speakers Moderator Terrel R. Rhodes
Vice President, Office of Quality, Curriculum, and Assessment and Executive Director of VALUE  Association of American Colleges and Universities (AAC&U)
Thomas Nelson Laird Director, Indiana University’s Center for Postsecondary Research (IUCPR)
Robert J. Awkward
Director of Learning Outcomes Assessment Massachusetts Department of Higher Education
Cara Smith Director of Institutional Effectiveness Georgia College

43. Forthcoming AAC&U Webinars
Wednesday, March 6, 2019 Truth, Racial Healing & Transformation Campus Centers
Tuesday, April 16, 2019 Integrating Interfaith Engagement with Curricular and Cocurricular Initiatives