Using Electronic Portfolios to Assess Learning at IUPUI Trudy Banta Sharon Hamilton Susan Kahn Indiana University-Purdue University Indianapolis

Plan Culture of Evidence Implement Improve Evaluate

PLANNING Campus mission, goals Unit goals aligned Programs based on assessable goals with PIs Annual reports on the Web

PAII Evaluation Services Assessment of learning Surveys Program reviews Performance indicators Program cost analysis Web-based evaluation tools Program evaluation/action research Accreditation

IMPROVEMENT AGENDA 1. Reporting to internal constituents 2. Demonstrating accountability www.iport.iupui.edu 3. Proposing improvement initiatives based on findings 4. Improving assessment methods

Assessment of Individual Student Development Assessment of basic skills for use in advising Placement Counseling Periodic review of performance with detailed feedback End-of-program certification of competence Licensing exams External examiners

Key Results of Individual Assessment Faculty can assign grades 2. Students learn their own strengths and weaknesses 3. Students become self-assessors

A Second Look 1. Across students 2. Across sections 3. Across courses

1. Where is learning satisfactory? 2. What needs to be retaught? 3. Which approaches produce the most learning for which students?

Group Assessment Activities Classroom assignments, tests, projects Questionnaires for students, graduates, employers Interviews, focus groups Program completion and placement Awards/recognition for graduates Monitoring of success in graduate school Monitoring of success on the job

Use of Results of Group Assessment Program improvement Institutional and / or state peer review Regional and / or national accreditation

Assessment of Learning Faculty agreement on outcomes and performance standards Evidence in syllabi and assignments that outcomes are taught Collective faculty review of group performance Use of findings to improve instruction and curriculum

Methods of Assessment Paper and pencil tests Individual or group projects Portfolios Observation of practice Observation of simulated practice Analysis of case studies Attitude or belief inventories Interviews and focus groups Surveys

Press to assess with a test Now We Have the Press to assess with a test

2006 Commission on the Future of Higher Education Need simple way to compare institutions Institutions should be more accountable for student learning

What happens to outcomes assessment undertaken to guide improvements in teaching and learning?

Using Electronic Assessment Methods We can Track progress in assignments Evaluate contributions to group projects Conduct immediate process checks to evaluate instruction Assess the quality of written work

What is an e-portfolio? A selection of purposefully organized artifacts that support retrospective and prospective reflection and that document, assess, and enhance student learning over time.

Why e-portfolios? For students Track growth and development Develop capacities for metacognition and self-directed learning Integrate and apply learning For faculty Enrich learning experiences Base assessment on authentic performances For programs and institutions Assessment for admissions, improvement and effectiveness, accreditation

Principles of Undergraduate Learning Core communication and quantitative skills Critical thinking Integration and application of knowledge Intellectual depth, breadth, and adaptiveness Understanding society and culture Values and ethics

ePort Learning Matrix

ePort levels of competence What all undergraduate students at IUPUI should know and be able to do within their: Introductory: First 26 credit hours Intermediate: First 56 credit hours Advanced: Major, profession, or academic program Experiential: Co-curricular, extra-curricular, work-based learning

Inside a Matrix Cell Goal: To document introductory competence in written communication Expectations: 1.You have identified a different audience and purpose for at least 2 documents 2.You have selected information appropriate to your audience and purpose. 3.You have used two different organizational styles appropriate to audience and purpose 4. You have used appropriate language, style, writing conventions, and formats.

Features of ePort Resources Learner Profile Matrices PUL Customized Presentation Tool Reporting Tool

Dashboard/Learner Profile Tasks Profile Pre-Survey Preflection Upload resources Reflection Profile Information Connections to self-assessment tools Connections to Career Center; Student Life and Diversity Connections to other helpful sites

Objects Aggregate Criteria • Time/level/Linear (Reports) Intro Intro interact interact interact interact Objects assess assess assess assess Crs grade Crs grade Crs grade Crs grade reflect reflect reflect reflect Aggregate (Reports) Artifact Artifact Artifact Artifact Communication Skills Critical Thinking Integration & Application of Knowledge Understanding Society &Culture Values & Ethics Etc …. Intro Intermed Adv. Exper Intro Intermed Adv. Exper Criteria Intro Intermed Adv. Exper Intro Intermed Adv. Exper Intro Intermed Adv. Exper • Time/level/Linear

Adoption strategies Small grants to departments and to freshmen learning communities Collaboration with users to develop needed features and modify or customize existing ones Extensive faculty development and technology support

Student Self-Assessment in the IUPUI ePort From Gateway Through Graduation

From Gateway Through Graduation First year: Preflection Reflection Capstone: Retrospective Prospective

Preflection What are your important goals for your life after college? Think about your goals in areas like career, citizenship, relationships, personal life, and spiritual/religious development. Thinking of one or two of these goals, what do you need to do and learn between now and graduation to get there? What abilities, skills, knowledge, and other characteristics do you need to improve or develop? Which of the PULs seem(s) most important for you to improve upon to reach your goals? Why?

Reflection Reflect upon the goals you had during your first weeks at IUPUI. Have your goals, or your ideas about how to reach them, changed? What have you learned that is relevant to the goal(s) discussed in your preflection? What is the evidence for this learning? Are there additional PULs that you now see as relevant to your goals?

Capstone Matrix LIFELONG LEARNING POSSIBLE CAREERS CURRENT ROLE IN SOCIETY PRINCIPLE OF UNDERGRADUATE LEARNING Artifacts REASON FOR BECOMING AN ENGLISH MAJOR

Matrix Thinking Movement from focus on original document to focus on original document in relation to a new conceptual framework See work in new focus Results in greater depth of understanding of original work Students can assess their intellectual growth since the original writing. Development in reflective thinking

Development in Reflection Ability to self assess Awareness of how one learns Developing lifelong learning skills

Ability to Self-Assess Observing One’s Own Performance Using Feedback and Evidence Finding and Analyzing Patterns Making Judgments

Islam Y107.doc exemplifies my ability to be a critical thinker because I had to put forth significant effort to separate my emotion from the facts and research….I believe that objectivity and rationality are at the core of every serious student – this paper shows me that I can be a serious student. Every class that I have taken in political science, English, and philosophy has emphasized the importance of looking past the surface of things. Additionally, my education in the liberal arts has taught me that there is much more to things than what my emotions tell me there are….My emotions are central only to my own experiences, and my critical thinking skills allow me to leap outside of my own experiences.

Critical thinking is my most important PUL because, no matter what goal I am pursuing, critical thinking will be a necessary step before that goal will ever be reached….Thinking about working as an advocate of the law in a globalizing world is intimidating. The documents in this cell, however, have calmed my angst so that I feel as though I am capable of accomplishing my goals. Two of the papers in this cell are only one page in length. I’ve come to learn that brevity and conciseness coupled with poignancy and clarity serve as valuable evidence that one has thought well and critically. To be brief is difficult if there is a lot to say. There is also a very long document in this cell.…Longevity of thought is sometimes necessary before brevity of a statement can be made clear.

I no longer see what I have to offer as an English job hunter in mere terms of degree possessed and years of experience…I look at what I have to offer in a larger context. Beyond the essentials in my resume that I share with all other graduates, I now see capacities in critical thinking, communications, and multi-project analyses. All these capacities can be supported with the creative and scholarly material in my matrix.”

Objects Aggregate Criteria • Time/level/Linear (Reports) Intro Intro interact interact interact interact Objects assess assess assess assess Crs grade Crs grade Crs grade Crs grade reflect reflect reflect reflect Aggregate (Reports) Artifact Artifact Artifact Artifact Communication Skills Critical Thinking Integration & Application of Knowledge Understanding Society &Culture Values & Ethics Etc …. Intro Intermed Adv. Exper Intro Intermed Adv. Exper Criteria Intro Intermed Adv. Exper Intro Intermed Adv. Exper Intro Intermed Adv. Exper • Time/level/Linear