The Problem with Reflection: Lessons Learned Alfredo Gaitan University of Bedfordshire Cathy Buyarski Indiana University-Purdue University Indianapolis Presentation delivered at the AAEEBL meeting July 2011.

Session Overview Introduction of presenters and portfolios Three sections: – Using reflection – institutional contexts and overview of portfolios – Assessing reflection – how, when, how well – What we learnt about reflection – in our projects and from each other We will aim to share two different projects/ perspectives, e.g. – Campus versus department – Freshman versus across enrolment – Summative assessment versus formative assessment – Insights on the students’ experiences of constructing eportfolios

Institutional contexts of PDP IUPUI Personal development planning is a process which will enable first year students at IUPUI to understand, implement, and mark progress toward a degree and career goal by creating and following a personalized plan that is open to revision and reevaluation every semester in collaboration with an academic advisor, faculty member, or other mentor. RISE Each undergraduate student is challenged to include at least two of the four RISE experiences - research, international, service learning, and experiential learning - into their degree programs. Principles of Undergraduate Learning UoB E-portfolios fit with the PDP model of learning, support the personalisation agenda and can also serve to structure learning for students studying remotely from the University. “We will further develop the use of e-portfolios as a framework to enable all students to record and review their progress and e-PDP as a means for structured and supported development.” CRe8. The ‘realistic learning’ dimension emphasises a learning experience which is meaningful, active, reflective, collaborative and challenging.

Overview of ePortfolios IUPUI Implemented in first-year seminar course across the entire campus (342 students in fall 2010). Reflection (with very specific guided prompts) in seven areas: About Me Educational Goals and Plan Career Goals Academic Showcase Campus and Community Connections My College Achievements Resume UoB Implemented at levels 1, 2, 3 of the Psychology curriculum with 225 students In Year 1 and 342 in Year 2. Reflection (with prompts) in a optional structure: – Background, – My Learning Experiences – My Learner Development – My Employability (includes a resume) – Showcasing my performance

Portfolio Examples Example Examples IUPUI UoB

Assessment of Reflection IUPUI Assessment to date has been formative and designed to improve prompts and rubrics. Main assessment of the First Year Seminar (1 credit): 100%. Assessed at the level of each prompt. Rubrics based Bloom’s Taxonomy. Feedback provided to students on each section of portfolio. Conducted through content analysis process. UoB Assessment was summative of students’ planning, experience, reflection, prospective view. Part of the assessments in each of three units (30-credit modules): 5%, 20%, 10%. Assessed at the level of sections or whole portfolio. Rubric loosely based on experiential learning (e.g. Kolb). Feedback: comments written on each page/section and the portfolio. Moderation in some units.

Lessons Learned IUPUI N Avg. H.S. GPA Avg. Fall GPA % Fall GPA below a 2.0 Fall DFW Rate Fall – Spring Retention Rate ePDP %12.10%91% Not e-PDP %17.23%89% Overall %16.45%89% Note 1: Missing cases were excluded from analyses. Note 2: Students who Withdrew or who were Administratively Withdrawn from Seminars were excluded (N=66 students). Note 3: Bolded items are significantly different based on independent samples t-test or chi-square results (p < 05). Table 1: 2010 e-PDP Compared to Not E-PDP First-Year Seminar Sections: Student Characteristics and Academic Success Indicators.

Lessons Learning University of Bedfordshire Table 2. Submissions of e-portfolios over the two years of the project. Year 1 (2008-9)Year 2 ( ) Level 149 (50%) (n=96) 134 (75%) (n=184) Level 264 (72%) (n=89) 66 (91.7%) (n=72) Level 38 (11%) (n=70) 74 (86%) (n= 86) Total121 (53.8%) N= (80.1 %) N=342

Lessons Learned IUPUI Assessing reflection in developmentally appropriate ways Scaffolding the use of reflection Faculty development Rubrics need to include cognitive development AND critical thinking Grading versus assessment

Lessons Learning University of Bedfordshire An emergent ‘grounded theory’ 1 of the students’ experiences: interconnected themes (1) Glasser & Strauss, 1967; Pigeon, 1996; Pigeon & Henwood, 1997)

What We Learnt from Collaborating Cathy Moving the PDP to embed in academic programs is an important next step Faculty buy-in and development will be key factors in our success Use of themes from UoB will inform our thinking about student participation Alfredo The eporfolios at UoB could be more directly linked to the coursework. Our portfolios are snapshots of a year, and only the Year 3 takes a long view (3 portfolios rather than one). The strong ethos and framework (e.g. RISE) may provide an environment favorable to e-portfolios. What can eportfolios do for your institution?

Let’s talk! Any questions? What have you learnt from your own experiences that you can offer us? Thank you!