1. U.S. Participation in the Assessment of Higher Education Learning Outcomes (AHELO) OECD’s Study of the Scientific and Practical Feasibility of Assessing Baccalaureate-Level Student Learning Outcomes Across Nations Association for Institutional Research Orlando, Florida May 29, 2014 www.sheeo.org

2. U.S. Participants’ Roles National level – U.S. Dept of Education is member of OECD Education Governing Board – SHEEO—National Project Manager (NPM) and representative on project advisory board- -Group of National Experts (GNE) – NCHEMS prepared sample files and will analyze national data – Foundations provided initial funding and remain interested State level – SHEEO agency provided project leadership, coordination, and oversight in Connecticut, Missouri and Pennsylvania Institutional level – AHELO Institutional Coordinator and “team” – IR office prepared student/faculty population files – Test Administration—recruitment, scheduling, monitoring – President, provost, faculty, media relations, graduate students

4. US Feasibility Study Results Institutional participation  Extremely small, voluntary, non-representative sample, but still highly variable participation (hard to generalize domestically or internationally) Student motivation and participation  Middling at 31% (719/2296), U.S. high of 68% ( highly variable internationally 95%) Contextual factors dominate, highly variable across nations and difficult to examine

5. IR Challenges Raised by AHELO Lack of consistency in institutional and national data on students and faculty Limitations in institutional and system-level analytic capacities and variable interests Theory-application gap—Disconnects between psychometric feasibility/requirements and practical limitations/policy applications  Relevance to teaching and learning  Implications for higher education policy

6. Should there be an AHELO be in our future, then—one participant’s perspective Assessment needs focused purpose clearly understood and communicated International input and development of assessment design and instruments Greater involvement by data providers and explicit attention to ultimate users Stronger international leadership, adequate financing and demonstrable value

7. Thank You Charles S. Lenth State Higher Education Executive Officers (SHEEO) Boulder, CO clenth@sheeo.org AHELO project repository and links: http://www.sheeo.org/projects/assessment- higher-education-learning-outcomes-ahelo