Assessing Student and Institutional Learning Using Electronic Portfolios Trudy Banta Sharon Hamilton Susan Kahn The Assessment Institute in Indianapolis November 3, 2003

Our Questions: How can electronic portfolios be used to assess 1. student learning? 2. institutional learning and effectiveness?

What have you accomplished in the last 4 years?

Did you have any difficulty summarizing your accomplishments? What would have helped you do this?

How would you organize a presentation on student learning on your campus? Is there a conceptual framework that you could use?

The Search for Accurate Measures of Student Learning 1980 – Estimated gain in Tennessee 1992 – Federal proposal for national test 2000 – Report Card (Incomplete in learning) 2002 – New proposals for assessing learning

ePort at IUPUI Connecting and Assessing Curricular and Co-Curricular Learning

What is ePort A collection of student work throughout their academic career at IUPUI, selected to show learning of the Principles of Undergraduate Learning in relation to course work. An opportunity for students to develop their own learning profile and learning matrix, knowledge maps, advising record, and resumes.

Authn/Authz SecurityWorkflow Comm. Tools Storage “Unbundling” In Touch Syllabus Testing Gradebooks Schedule Ereserves Digital content Full Text articles Federated Searching

Learning Profile Learning Matrix Advising Resume/Vita Builder Research Manager Content Manager Knowledge Mapper Authn/Authz SecurityWorkflow Comm Tools. Storage Other Services Other Services

ePortfolio Goals Assist both faculty and students to reach a clearer, more coherent understanding of how aspects of the curriculum support students' increasing mastery of the PULs. Contribute to assessment of student learning of the PULs at the levels of the individual student, the course, program, and institution. Support student engagement with the PULs over their entire undergraduate experience, beginning in the freshman learning community and culminating in the capstone experience.

ePortfolio The ePortfolio is organized around IUPUI's Principles of Undergraduate Learning. 1.Core communication and quantitative skills 2.Critical thinking 3.Integration and application of knowledge 4.Intellectual depth, breadth, and adaptiveness 5.Understanding society and culture 6.Values and ethics

ePortfolio Levels of Competence –Introductory: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 26 credit hours. –Intermediate: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 56 credit hours. –Advanced: What all baccalaureate students at IUPUI should know and be able to do in relation to PULs in their major or profession or academic program. –Experiential: Connecting curricular and co- curricular learning

Complete PendingReady Locked

Learning Matrix View by Category View by Matrix Click on cell to view/edit Core Communications & Quantitative Skills Written Communication Analyzing Texts Oral Communication Quantitative Problem Solving Information Literacy

Learning Matrix View by Category View by Matrix Click on cell to view/edit Core Skills > Oral Communication Persuasive Speech Add this to another Cell. 1. a. b. c. Course Development Project Poster One Page document 2.

Learning Matrix View by Category View by Matrix Click on cell to view/edit Core Skills > Oral Communication > Introductory Students communicate orally with different audiences and purposes. To demonstrate your understanding of this PUL at the Introductory level, the oral communication and documents you upload and your reflection should show the following: a)you identify the characteristics of your intended audience and adapt your speech to this assessment and analysis b)you create a specific purpose or function for your oral communication c) you organize and develop your main ideas to accomplish your purpose for your intended audience; d)you locate and identify credible sources of information on your topic e)you identify appropriate communication aids (for example, power point slides, handouts, audio tape or video tape) that will aid in accomplishing your specific purpose f)you use effective oral (rate, volume, pauses, articulation) and nonverbal (eye contact and body language) delivery skills g)you identify and understand critical listening skills h)you identify your own and others strengths in oral communication, which includes strengths in organization, content, delivery and audience adaptation

Learning Matrix View by Category View by Matrix Click on cell to view/edit Core Skills > Oral Communication > Introductory Students communicate orally with different audiences and purposes. To demonstrate your understanding of this PUL at the Introductory level, the oral communication and documents you upload and your reflection should show the following: a)you identify the characteristics of your intended audience and adapt your speech to this assessment and analysis b)you create a specific purpose or function for your oral communication c) you organize and develop your main ideas to accomplish your purpose for your intended audience; d)you locate and identify credible sources of information on your topic e)you identify appropriate communication aids (for example, power point slides, handouts, audio tape or video tape) that will aid in accomplishing your specific purpose f)you use effective oral (rate, volume, pauses, articulation) and nonverbal (eye contact and body language) delivery skills g)you identify and understand critical listening skills h)you identify your own and others strengths in oral communication, which includes strengths in organization, content, delivery and audience adaptation Audience: These are the intended viewers/ listeners of your oral communication. Your intended viewer/listener may be one person or a group, which will require you to consider who they are and what their perspectives are on the topic. The oral communication you choose should demonstrate how you can adapt your speaking to your viewers/listeners.

Learning Matrix View by Category View by Matrix Core Communication & Quantitative Skill Critical Thinking Integration & Application of Knowledge Intellectual Breadth, Depth and Adaptiveness Understanding Society and Culture Values and Ethics

Learning Matrix View by Category View by Matrix Core Communication & Quantitative Skill Critical Thinking Integration & Application of Knowledge Intellectual Breadth, Depth and Adaptiveness Understanding Society and Culture Values and Ethics Written Communication Analyzing Texts Oral communication Quantitative Problem Solving Information Literacy IntroductoryIntermediateAdvancedExperientialAll

Can we assess institutional learning and effectiveness using an electronic institutional portfolio?

Urban Universities Portfolio Project (UUPP)  California State University, Sacramento  Georgia State University  IUPUI  Portland State University  University of Illinois at Chicago  University of Massachusetts Boston Sponsor: AAHE Funded by: The Pew Charitable Trusts ( )

“Institutional Portfolio” A focused selection of real work, combined with interpretation and reflection, that demonstrates specific institutional achievements and shows learning and improvement over time—i.e., “institutional effectiveness.”

Lessons Learned: Advantages of Electronic Institutional Portfolios  Facilitate campus involvement  Bring new perspectives that catalyze learning and change  Focus on evidence and alignment  Demonstrate institutional effectiveness and improvement over time  Demonstrate accountability  Useful for multiple purposes and audiences

Lessons Learned: Disadvantages of Electronic Institutional Portfolios  More work than a paper report!  Scarcity of models  Need for infrastructure  Accreditation/accountability in transition—organizations/teams may need to be oriented to this approach  Blur “boundaries” of self-study/report

Next Steps  Develop new iteration/update as annual performance report  Update the design and technological infrastructure  Incorporate more examples and aggregated evidence of student learning through interface with student electronic portfolio