Using Sakai and OSP to Influence Teaching and Learning Sharon Hamilton and Susan Kahn Sakai Conference, June 2006 Vancouver, BC

Outline of workshop  Overview of IUPUI Portfolio  ePortfolios and learning: reflection and metacognition First year Learner Profile Capstone  NPEC/AIR Grant  Lessons learned  Looking to the future

The IUPUI Portfolio  Brief history  Principles of Undergraduate Learning  The OSP context  Current Status  New directions

Brief history  Begun three times: Honors portfolio First year portfolio Undergraduate portfolio  Stand alone  Prototype for Epsilen portfolio  Prototype for Open Source Portfolio 2.0  Now developing its institutional destiny

One student’s perspective  “So you get here and they start asking you, ‘What do you…want to major in? …what courses [do] you want to take?’ and you get the impression that’s what it’s all about – courses and majors. So, you take the courses. You get your card punched. You try a little this and a little that. Then comes GRADUATION. And you wake up and you look at this bunch of courses and then it hits you: They don’t add up to anything. It’s just a bunch of courses. It doesn’t mean a thing.”

Principles of Undergraduate Learning (PULs)  Core communication and quantitative skills  Critical thinking  Integration and application of knowledge  Intellectual depth, breadth, and adaptiveness  Understanding society and culture  Values and ethics

Features of ePort 1. Resources 2. Matrices PUL Customizable 3. Reviewers 4. Presentation Tool 5. Reporting Tool

The PUL matrix A means to: Document student work over time. Enhance student learning. Assess growth over time Encourage reflection.

Inside a Matrix Cell Goal: To document introductory competence in written communication Expectations: 1.You have identified a different audience and purpose for at least 2 documents 2.You have selected information appropriate to your audience and purpose. 3.You have used two different organizational styles appropriate to audience and purpose 4. You have used appropriate language, style, writing conventions, and formats.

ePort in the first year: Welcome  Welcome to the IUPUI electronic portfolio (ePort)! ePort will help you to plan and make the most of your education here at IUPUI. It will allow you to track your learning and progress over time and develop evidence of your learning that you can use for many purposes.  Please begin your ePort by creating your Learner Profile.

My Profile Complete form and then choose 'Save' at the bottom. A '*' means it is required information.  Public Information  First name *  Last name *  Nickname  Position *  Department *  School *  Room  Hide my entire Profile

Personal Information  Picture None Use University ID picture Use Picture URL (browse feature) Hide just my personal information   Home Page  Work Page  Home Phone  Other

Dashboard/Learner Profile  Tasks Profile Pre-Survey Preflection Upload resources Reflection Profile Information Connections to self-assessment tools Connections to Career Center; Student Life and Diversity Connections to other helpful sites

Reflection in ePort  Old definition: Reflecting involves connecting evidence of learning to expectations for learning to discover and describe intellectual change.  Elements of reflection: Evidence Connections Intellectual growth

Preflection  What are your important goals for your life after college? Think about your goals in areas like career, citizenship, relationships, personal life, and spiritual/religious development.  Thinking of one or two of these goals, what do you need to do and learn between now and graduation to get there? What abilities, skills, knowledge, and other characteristics do you need to improve or develop?  Which of the PULs seem(s) most important for you to improve upon to reach your goals? Why?

Reflection  Reflect upon the goals you had during your first weeks at IUPUI.  Have your goals, or your ideas about how to reach them, changed?  What have you learned that is relevant to the goal(s) discussed in your preflection? What is the evidence for this learning?  Are there additional PULs that you now see as relevant to your goals?

Sample Matrix for Capstone Career Choice Lifelong Learning Who I Am in Today’s Global Society Most Important English Goal Most Important Principle Why I Became an English Major

Matrix Thinking  Reflecting on a discipline-specific or interdisciplinary concept in relation to a larger conceptual framework. Example: reflecting on assignments completed in English, history, math, and biology in relation to critical thinking.

Development in Reflection  Ability to self-assess Observing own performance Using feedback to find patterns  Awareness of how one learns Moving from misconceptions to conceptions Metacognition  Developing lifelong learning skills Transferring learning to other contexts Understanding learning as a lifelong process

“Enhancing Student Success Through Electronic Portfolios”  Funded by AIR and NPEC Student Success Initiative in  Research question: What is the impact of ePort on the success of first-year students?

Project Design First semester:  Piloted ePort in 8 Themed Learning Communities  6 other TLCs served as control group  Compared outcomes for experimental and control groups

Multiple assessment approaches  End-of-semester survey administered to students in both the experimental group and the control group  Comparison of retention rates  Comparison of grade point averages

ePort Advisory Board  Made up of faculty teaching experimental classes  Met monthly to exchange materials and discuss experiences with pilot  Also acted as de facto users’ group that provided input to technology developers

Retention to second semester (adjusted)

Grade point averages (adjusted) ePort participants (n=144) Average fall GPA: 2.64 Adjusted GPA:2.70 Non-participants (n=71) Average fall GPA: 2.73 Adjusted GPA:2.60

PUL/NSSE Survey  ePort students indicate greater “engagement” in NSSE questions  Statistically significant differences in: Writing and revising papers Integrating ideas from different courses Working harder than expected Hours preparing for class Support needed to succeed

PUL/NSSE Survey  Statistically significant differences in: Perceptions of importance of 7 of 10 PULs Self-ratings of competence in 3 of 10 PULs (quantitative reasoning, information literacy, integration and application of knowledge)

Lessons Learned So Far  Be prepared to provide extensive faculty development and technical support  Expect some frustration with new technology  Be prepared to answer questions about “why are we doing this?” from faculty and students

Lessons Learned So Far  Be aware of faculty workloads  Keep expectations simple  Expect uneven levels of participation and interest at first—an e-portfolio initiative needs time to grow  Ensure the technology is ready so that people have a good experience the first time around

Looking to the Future  Incorporate what we’ve learned into faculty development for new cohorts  Continue to assess student outcomes; eventually develop longitudinal assessment  Continue to work toward gradual campus- wide adoption by targeting departments (Integrative Department Grants) and large multi-section courses

Looking to the future  Matrices for qualifications in the following: Honors Undergraduate Research Civic Engagement Global Competence Vocation & Values Others…  Meta-tags: one document; many purposes  Presentations Resumes and career applications Graduate school applications; scholarships

“I no longer see what I have to offer as an English job hunter in mere terms of degree possessed and years of experience…I look at what I have to offer in a larger context. Beyond the essentials in my resume that I share with all other graduates, I now see capacities in critical thinking, communications, and multi-project analyses. All these capacities can be supported with the creative and scholarly material in my matrix.”

Contacts  Sharon Hamilton, IUPUI,  Susan Kahn, IUPUI,