Building bridges to constituents: Communicating through performance indicators and electronic portfolios Trudy Banta, Victor Borden, & Susan Kahn Sharon Hamilton Policy and governance issues for electronic student portfolios

Overview Planning for improvement IUPUI The tip of the assessment iceberg iPort: The institutional portfolio Placing PIs in context ePort: The electronic student portfolio Demonstrating student learning Completing the cycle Linking accountability to improvement

Planning for Improvement Assessable Outcomes Instrumentation Data Collection Analysis Application of Findings Culture Of Evidence Improvement 1. Reporting to internal constituents 2. Demonstrating accountability to external stakeholders 3. Applying findings in campus improvement initiatives 4. Proposing improvement initiatives 5. Improving assessment methods  Web-based data  Electronic portfolios

Planning and Budgeting 1.Mission, Vision, Goals developed 2.Unit goals aligned 3.Annual reports on web 4.Programs based on assessable goals, with performance indicators 5.Biennial planning/budgeting hearings conducted

Implementation Everyone on campus implements goals

Evaluation 1. Academic and administrative program reviews 2. Evaluation of process effectiveness 3. Assessment of learning outcomes in major in general education 4. Course evaluations 5. Student assessment 6. Constituent surveys 7. Management information and analysis 8. Program cost analysis 9. Web-based evaluation tools 10. Annual campus performance report 11. NCA accreditation

Improvement 1.Reporting to internal constituents 2.Demonstrating accountability to external stakeholders 3.Applying findings in campus improvement initiatives 4.Proposing improvement initiatives 5.Improving assessment methods Web-based data Electronic portfolios

Planning for Improvement Assessable Outcomes Instrumentation Data Collection Analysis Application of Findings Culture Of Evidence Improvement 1. Reporting to internal constituents 2. Demonstrating accountability to external stakeholders 3. Applying findings in campus improvement initiatives 4. Proposing improvement initiatives 5. Improving assessment methods  Web-based data  Electronic portfolios

IUPUI Performance Indicators as the tip of the assessment iceberg

IUPUI: Background Status of PIs in the early 1990s Kells (OECD) Failure of PIs for national accountability in Europe and elsewhere. Movement toward quality assurance Borden & Banta (NDIR 82) Greater possibilities for PIs if they emerge from comprehensive planning efforts Gaither, Nedweck, & Neal (ASHE ERIC) Increasing demand for accountability and continuing inability of HEIs to communicate about performance to external audiences The Rise of CSFs, KSFs, KPIs Rockhart (1979) Critical Success Factors (CSFs) NACUBO/ Peat Marwick – 67 Indicators Dolence (1989) KPIs for enrollment management

Balanced Score Card (BSC) Kaplan & Norton propose business model Financial performance Customer service and satisfaction Process effectiveness and efficiency Organizational learning Ruben (1999) applies BSC to HEIs Teaching/Learning - Programs/Courses, Student Outcomes Service/Outreach - University, profession, alumns, state, prospective students, families employers Scholarship/Research - Productivity/Impact Workplace satisfaction - Faculty/staff Financial - Revenues/expenditures

Problems with Current Systems Despite Ruben’s challenge to focus on mission critical activities, most focus on administrative functions There is a disconnect between performance indicators and core performance objectives What is the organizational link between the accountability focus of campus executives and the performance of mission critical functions?

Making the Connection Doing assessment right (Banta, 2002) Participative planning Assessment must be explicitly tied to institutional functions and be associated with goals that faculty and administrators value Implementation Assessment is most powerful at the unit level where changes will most immediately affect the faculty and students Improving and sustaining while the results of assessment may be used to respond to various external accrediting and stakeholder constituents, it will first and foremost provide the means by which the organization can continuously assess and improve itself

IUPUI: Performance Objectives Excellence in Teaching and Learning Attract and support a better prepared, more diverse student population. Support and enhance effective teaching Enhance undergraduate student learning and success Provide effective professional and graduate programs and support for graduate students and post-doctoral fellows. Excellence in Research, Scholarship, and Creative Activity Conduct world-class research and creative activities relevant to Indianapolis, the state, and beyond. Provide support for scholarly activity and external funding. Enhance infrastructure for research and creative activities. Excellence in Civic Engagement, Locally, Nationally, and Globally Enhance capacity for civic engagement. Enhance civic activities, partnerships, and patient and client services. Intensify commitment and accountability to Indianapolis, Central Indiana, and the state. Collaboration – Diversity – Best Practices

PI Development Process Vested interested groups (handout) Brainstorming session What evidence would convince a (neighbor, prospective student, legislator) that you we were doing well? What evidence do you (or could you) use in your unit to assess performance? 3-4 performance indicators per objective Three level rubric (green, yellow, red) Adapted Nominal Group Technique for voting

Status and Results In Handout Summary of performance objectives and performance indicators, with measures shown for one set of indicators Published in… Campus annual performance report Chancellors State of Diversity address Institutional portfolio

iPort The Electronic Institutional Portfolio

“Institutional Portfolio” A focused selection of authentic institutional work, combined with narrative analysis, that demonstrates specific institutional achievements and shows learning and improvement over time—i.e., “institutional effectiveness”

IUPUI’s iPort Begun in 1998 as part of national grant- funded project Organized around main components of institutional mission and major strategic priorities Intended for purposes of both internal improvement and accountability to multiple stakeholders Used as platform for accreditation self-study in 2002 Is on the Web—available to all

What do we gain from work on an electronic institutional portfolio? Occasions for focused, campus-wide discussions of areas needing improvement A means of demonstrating institution’s commitment to accountability and effectiveness An easy, accessible medium for communicating with stakeholders about accomplishments and effectiveness

Next Steps Develop new iteration/update as annual performance report Update the design and technological infrastructure Foreground links to other relevant campus Web sites Incorporate more direct evidence of student learning through interface with student electronic portfolio

ePortfolio The Electronic Student Portfolio

Authn/Authz SecurityWorkflow Comm. Tools Storage “Unbundling” In Touch Syllabus Testing Gradebooks Schedule Ereserves Digital content Full Text articles Federated Searching

Learning Profile Learning Matrix Advising Resume/Vita Builder Research Manager Content Manager Knowledge Mapper Authn/Authz SecurityWorkflow Comm Tools. Storage Other Services Other Services

ePortfolio Goals Support students' learning of and engagement with the PULs over their entire undergraduate experience at IUPUI beginning in the freshman learning community and culminating in the capstone experience Assist both faculty and students to reach a clearer, more coherent understanding of how various aspects of the curriculum support students' increasing mastery of the PULs Contribute to assessment of student learning of the PULs at the levels of the individual student, the course, program, and institution

ePortfolio and the PULs The ePortfolio is organized around IUPUI's Principles of Undergraduate Learning 1. Core communication and quantitative skills 2. Critical thinking 3. Integration and application of knowledge 4. Intellectual depth, breadth, and adaptiveness 5. Understanding society and culture 6. Values and ethics

ePortfolio Levels of Competence Introductory: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 26 credit hours Intermediate: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 56 credit hours Advanced: What all baccalaureate students at IUPUI should know and be able to do in relation to PULs in their major or profession or academic program

Learning Matrix

Planning for Improvement Assessable Outcomes Instrumentation Data Collection Analysis Application of Findings Culture Of Evidence Improvement 1. Reporting to internal constituents 2. Demonstrating accountability to external stakeholders 3. Applying findings in campus improvement initiatives 4. Proposing improvement initiatives 5. Improving assessment methods  Web-based data  Electronic portfolios