Origins and Development of the Lumina “Degree Qualifications Profile (DQP)” Peter Ewell National Center for Higher Education Management Systems (NCHEMS) SHEEO Annual Meeting July 13, 2012

To increase the proportion of Americans with high-quality degrees and credentials to 60 percent by 2025.

 Increasing the number of degrees requires attention to quality and transparency  Learning is valued by employers  High-quality degrees are essential element to a knowledge economy. How Is Quality Reflected in Goal 2025?

Background  Qualifications Frameworks in Many Other Countries  Bologna Process Common Outcomes Benchmarks (e.g. “Dublin Descriptors”)  AAC&U LEAP Outcomes Statements and Rubrics  State-Level Outcomes Frameworks in U.S. (e.g. UT, WI, CSU, ND, VA)  Some Alignment of Cross-Cutting Abilities Statements Among Institutional Accreditors

Lumina Degree Profile Three Degree Levels: Associate, Bachelor’s, and Master’s Five Learning Areas: Specialized Knowledge, Broad/Integrative Knowledge, Intellectual Skills, Applied Learning, and Civic Learning Framed as Successively Inclusive Hierarchies of “Action Verbs” to Describe Outcomes at Each Degree Level Intended as a “Beta” Version, for Testing, Experimentation, and Further Development Beginning this Year

How the Panel Approached Its Work Wide Literature Review (Other National QFs and Outcomes Adopted by U.S. Colleges and Universities) Emphasis on Application and Integration (as Distinctively “American” Undergraduate Attributes) But Confined to Things that Institutions Actively Teach (Therefore Few Values or Attitudes Included) Emphasized Civic Learning as an Area Particularly Important for a Functioning Democracy

An Example: Communication Skills Associate Level: The student presents substantially error-free prose in both argumentative and narrative forms to general and specialized audiences Bachelor’s Level: The student constructs sustained, coherent arguments and/or narratives and/or explications of technical issues and processes, in two media, to general and specialized audiences Master’s Level: The student creates sustained, coherent arguments or explanations and reflections on his or her work or that of collaborators (if applicable) in two or more media or languages, to both general and specialized audiences

An Example: Engaging Diverse Perspectives Associate Level: Describes how different cultural perspectives would affect his or her interpretations of prominent problems in politics, society, the arts, and/or global relations Bachelor’s Level: Constructs a cultural, political, or technological alternative vision of either the natural or human world, embodied in a written project, laboratory report, exhibit, performance, or community service design; defines the distinct patterns in this alternative vision; and explains how they differ from current realities Master’s Level: Addresses a core issue in his/her field of study from the perspective of either a different point in time, or a different culture, political order, or technological context, and explains how the alternative perspective contributes to results that depart from current norms, dominant cultural assumptions, or technologies—all demonstrated through a project, paper, or performance

What Happens Next?  Growing Number of Lumina-Funded Follow-On Projects Designed to “Test Drive” the DQP (HLC, WASC, SACS, CIC, AASCU, AAC&U, etc.) Involving More than 120 Institutions  Other Efforts Consistent with DQP that are Not Directly Funded by Lumina (e.g. MA)  Results of Projects (and other efforts) Will be Used to Refine the DQP Further in 2014.

Testing The DQP : The Context, Opportunities, and Challenges Debra Humphreys Association of American Colleges and Universities

National Context

1) Increasing Demand for Public Higher Education 2) Declining Funding Sources 3) Declining Public Confidence and Skepticism about Value 4) Lack of Clarity about What a Degree Actually Represents 5) Lack of Solidly Researched Assessment Tools for Full Range of Competencies at High Levels DQP and Quality Collaboratives Respond Directly to 4 and 5.

The Quality Collaboratives Project and the DQP Also Respond to Long-term Trends: The World is Demanding More There is a demand for more numbers of college educated workers. There is a demand for engaged and informed citizens, who are knowledgeable about themselves and the world around them There is also a demand that those educated workers and citizens have higher levels of learning and knowledge, and some new and different skills and abilities.

Quality Collaboratives QC is a three-year project funded by Lumina Foundation and the Hewlett Foundation and based on the DQP. 9 states and 20 individual campuses working in state systems and/or 2-year/4-year partnerships Campuses and systems are testing the use of the DQP to map, assess, and document student achievement across levels of learning. Project outcomes: new assessment frameworks and approaches; policy frameworks to track student progress and achievement; and models of faculty development and leadership.

Opportunities DQP builds on research about how people learn and is designed to ensure both knowledge and ability to integrate and apply DQP builds on LEAP and other efforts that have credibility among faculty and includes outcomes already endorsed by accreditors and employers DQP Combines Vision with Strategy—Focuses on what students actually are required to do (projects, papers, research) Opportunity to Do Assessment Right and Increase Achievement and Completion at the Same Time!

Challenges Progress on articulating outcomes, but not enough connecting of outcomes explicitly to gen ed or major requirements; We aren’t building faculty capacity/support fast enough; Assessment approaches still localized rather than systemic; more disciplinary than cross-cutting; Progress on assessment, but methods are still very local and reporting of results too confusing for public; Pressures to advance completion at reduced costs present dangerous opportunity to skip over defining quality or assessing achievement in meaningful ways.

What is the Role of SHEEOs? Becoming Facilitators as well as Regulators Use participation in DQP projects as opportunity to facilitate intercampus collaboration about common goals and assessment approaches (statewide mtgs, task forces, working groups); Draw positive attention to the work of faculty and academic administrator change agents (focus on mid-level campus leaders); Use participation in nat’l initiatives to draw public attention to the need for broad learning outcomes in the knowledge economy; Use or modify LEAP VALUE Rubrics to provide statewide frameworks for assessing student work.

Assessing Student Learning: The Work in Progress in Massachusetts SHEEO Annual Meeting | July 13, 2012 Richard M. Freeland, Commissioner

The Work in Progress in Massachusetts  Develop a system-level program of learning outcomes assessment that: Allows Massachusetts to compare educational results with other states Does not depend on standardized testing The Challenge

The Accountability Movement The Educational Imperative The Problems with Standardized Tests The Work in Progress in Massachusetts Why This Is Important

The Work in Progress in Massachusetts  Part of A comprehensive initiative to strengthen public higher education in Massachusetts  Relationship of college completion agenda to student learning agenda The Context

The Work in Progress in Massachusetts  Working Group on Student Learning Outcomes and Assessment  Two phases of work:  Campus-level Assessment: 2009–2010  System-level Assessment: 2010–2011 The Approach

The Work in Progress in Massachusetts  Acknowledge pre-eminent role of faculty  Must become integral to teaching/learning process  Should include common elements but allow room for variation  Must be feasible for wide use in terms of cost and faculty workload  Must be useful for program improvement  Must be useful for communicating results to public Working Group Report: Key Principles

The Work in Progress in Massachusetts  Grounded in embedded assessment  Value of LEAP Framework and VALUE Rubrics  Allow for use of other measures (direct and indirect assessments) The Concept

The Work in Progress in Massachusetts  In Massachusetts:  Becoming a LEAP State  Building a collaborative planning structure  Acquiring external support Implementing the Concept

The Work in Progress in Massachusetts  State Partners:  The Boulder Conference  Support of SHEEO and AAC&U  Crafting a multi-state compact  Ongoing partnership with SHEEO Implementing the Concept

Questions