What’s in the works for General Education Assessment? Dan McCollum, Ph.D. Associate Director of Academic Assessment

Objectives Define Assessment Explain the Assessment Cycle of Improvement Explore Purposes of Assessment Review Six Assessment Terms Identify Present Elements Describe Present into Future Elements Describe Desired Future Elements Address Ambiguities & Challenges of the Future Ask, what’s with these new G.E. outcomes? Leave time for Q & A

Define Assessment Trudy Banta – “the systematic collection, review and use of information about educational programs for the purpose of improving student learning and development” By definition, all assessment is formative assessment Logically, if it’s not formative, it’s not assessment Expanded from: Commentary by Trudy Banta, 2008 IUPUI Assessment Institute

Explain the Assessment Cycle of Improvement PLANNING (Re) Invent Outcomes (Re) Design Curriculum (Re) Develop Measures LEARNING Opportunity to Learn ASSESSING Use Measures Analyze Data Discuss Results Adapted from: Suskie, L. (2004). Assessing student learning: A common sense guide.

Explore Purposes of Assessment 1. Committing to student success What are students learning? How well? (baseline) Comparison for improvement (cyclical/longitudinal) Comparison to local standards (local criteria for success) Planning curriculum improvements & innovations 2. Meeting accreditation standards of NWCCU How do we know what and how well students are learning? Accreditation – a catalyst for student success 3. Informing outside constituents of program/university performance Accountability – a catalyst for student success Comparison for external standards (e.g., national criteria/ranks) Adapted in part from: Linda Suskie, Middle States Commission on Higher Education (MSCHE)

Review Six Assessment Terms 1. Direct Measure 2. Indirect Measure 3. (Course) Embedded Measure 4. Common Measure 5. Local Measure 6. External Measure Adapted in part from Andrea Leskes of AAC&U:

Identify Present Elements A set of desired Learning Outcomes Course-embedded, local and external, direct measures A common, local, indirect measure

Identify Present Elements Learning Outcomes English Composition World Literature Constitutions Mathematics Multicultural International Distribution: (1) Humanities & Fine Arts, (2) Life and Physical Sciences & Analytic Thinking, (3) Social Sciences Source: UNLV Undergraduate Catalog Fall 2008 – Spring 2010

Identify Present Elements Course-embedded, local, direct measures Aligned to student learning objectives within courses, but sometimes aligned to learning objectives within sections of a course (a faux pas) Aligned with learning outcomes? Currently being expanded Studying online tracking, analysis, and reporting systems

Identify Present Elements Course-embedded, external, direct measure: A Standardized Test Collegiate Assessment of Academic Proficiency (CAAP) – Writing Essay and Writing Skills Used to assess English Composition course- sequence objectives aligned to the English Composition General Education Outcome For more on CAAP:

Identify Present Elements Common, Local, Indirect measure: Graduating Senior Exit Survey (GSES) includes items to address how well students believe they achieved in: 1. Improving writing skills (English Composition) 2. Comparing literature of different cultures and times (World Literature) 3. Able to interpret the U.S. and Nevada constitutions in broad contexts (Constitutions) 4. Learn about mathematical/statistical reasoning (Math) 5. Analyze multiple, contemporary U.S. Cultures (Multicultural) 6. Learn about sociopolitical complexities of interconnected world cultures (International) 7. Appreciate fine arts (Distribution – Fine Arts) 8. Understand individual & group behaviors and the impacts of policies on them (Distribution – Social Sciences) 9. Learn about the role of science & engineering in society (Distribution – Life and Physical Sciences & Analytic Thinking)

Describe Present into Future Elements Sampling plans being developed for senior pilot of CAAP’s Writing Essay, Critical Thinking, & Math A Baseline Measurement Sampling plans being developed for first full, freshman implementation of same CAAP tests for VSA Learning Outcomes being re-invented Will be the focus of future General Education Assessment See: For more on Voluntary System of Accountability (VSA):

Describe Desired Future Elements Common, Local, Indirect Measure of self- perceptions of outcomes (entering, graduating, alumni) Common, External, Indirect Measure of engagement (entering & graduating) Common, Local, Direct Measure of Outcomes (entering, mid-level, graduating) Common, External, Direct Measure of Writing, Critical Thinking & Math via CAAP for VSA (entering & graduating) For more on Voluntary System of Accountability (VSA):

Describe Desired Future Elements: Full Model with Timing Entry (1) External Direct (2) Local Direct (3) Local Indirect (4) External Indirect Mid-Level (Entry for Transfers) (2) Local Direct Graduating (1) External Direct (2) Local Direct (3) Local Indirect (4) External Indirect Alumni (3) Local Indirect For more on External Direct and Indirect Measures:

Address Ambiguities and Challenges - CAAP Pilot Senior Sampling (Spring 2009) ~600 students will be sampled Proportional Quota Sampling Sample students from each College (college is the Condition) proportionate to the College’s overall part of the University’s graduating population Motivators for Students Required by course policy Explain use of data for public “portrait” of UNLV Rewards: (1) gift cards (2) graduation fee rebates (3) cash (4) extra credit (5) public recognition of performance (6) raffle

Address Ambiguities and Challenges - CAAP Full Entry Sample (Fall 2009?) ~ 750 students needed to represent the population Simple Random Sample Proportional Quota Sample Stratified Random Sample No sample  Census Motivators for Students Required part of orientation/registration by university policy Scores on transcripts Rewards

Address Ambiguities and Challenges - CAAP Full Senior Sample (Spring 2010?) ~ 750 students needed to represent population Simple Random Sample Proportional Quota Sample Stratified Random Sample No sample  Census Motivators for Students Required for graduation Scores on transcripts Rewards

Address Ambiguities and Challenges – Desired Full Model Development of Local Measures by Assessment Measures Development Committees One committee per outcome To include developing blueprints and/or rubrics To include item writing and/or assignment development To include standard setting for each level Assessment Days to administer measures to entering, mid-level, and graduating students Scoring of Local, Direct Measures by Assessment Measures Scoring Committees One committee per outcome

Address Ambiguities and Challenges – Committees Gaining faculty participation in measure development and scoring Having expert measurement people to train and guide the developers and scorers

Address Ambiguities and Challenges: Costs Faculty stipends and/or releases Proctors Rewards for students CAAP: ~ $39/student for test materials Indirect, external engagement measure

Ask, what’s with these new G. E. outcomes? What will they be? See: Possibly implemented alongside the curriculum that follows the current General Education outcomes Quasi-experimentation can be used to compare students’ performance in new and current curriculums

Leave time for Q & A Questions?