1. Identifying the key issues that will persuade staff and students to engage with ePortfolios: the results of nine pilot modules Peter Chalk London Metropolitan University ‘Telling stories’, Wolverhampton, June 2008

2. or ‘ Stuck in the mud ’ A very mixed story…

3. Background Autumn 2007 London Met ePortfolio pilot Nine modules from Foundation Degree to Honours Degree 1-3 to Masters Existing PDP framework includes specified tasks and PDP related assessment in core spine modules (www.londonmet.ac.uk/ugstudy/pdp)

4. Why go electronic? Increasing expectation by employers – will improve students’ employability (currently low at London Met) Part of London Met’s e-enablement policy (all modules to use VLE) Students’ evidence/ artefacts increasingly digital – multimedia/ CV – as are tools Improves sharing with multiple viewers

5. Preparation Summer of 2007 tutors introduced to the new ePortfolio system by workshop, online and face to face support Encouraged to integrate it into their module preparation, materials, teaching, learning strategy, classroom organisation and assessment (pre-pilot findings, Chalk 2007)

6. Examples of ePortfolio use In addition to those following: –Levels 1 and 2 to upload CV in HE orientation (HEO) and employability modules –Level 3 project blog shared with supervisor –Level 1 Film Studies HEO module, to store a log of film criticisms –Level 2 maths employability module with six PDP related tasks.

7. ePortfolio evidence Evidence of student (and staff) engagement includes –Examples of student work –Templates, tools and other resources developed by staff –Results of observations and discussions with staff and students

8. Foundation Degree Community Sports Coaching students

9. Evidence of prior achievement

10. Evaluation of coaching session – evidence of reflection

12. SMART analysis – use of a PDP tool

13. Learning object (assignment) – a collection of files

14. Positive image – role model for non-traditional student – FDSc students enthusiastic compilers of ePortfolios

15. Tutor’s initial template for Events Management (M level) ePortfolio

16. Evaluation Evaluation of the Autumn pilot was carried out by –Two questionnaires to students, –Two workshops for staff, Nov & Jan –Email discussion, –Observation, and –Analysis of actual student ePortfolios

17. Selective feedback from 3 staff Very few students not in the pilot tutorial group chose to use the ePortfolio. Very few students have engaged any more than was absolutely necessary. 30% experienced a significant degree of difficulty in using the system The ePortfolio was not assessed in any way so little engagement: one uploaded their CV, two provided some evidence of reflection

18. Staff feedback 2 (events management) 71% of the students in one tutorial group used the ePortfolio for their reflective commentary, and 28% of these had invited guests to view it (but not the module leader), In another group only 37.5% used the ePortfolio to record their reflections and none invited in guests, Students’ reasons for non-engagement: costly, confusing, unstable internet, too much effort.

19. Staff feedback 3 (film studies) ePortfolio is an improvement over paper-based PDP, customisable by student, told “this is London Met’s Facebook”, but it’s not as user- friendly, Tutor can see lots of potential but needs to be integrated into teaching and learning throughout the course and not just on one module, Can use multimedia binder to store short films, only place to store seminar logs throughout their course.

20. Student feedback 1 (events) “I don’t think I will find it useful since a lot of my courseworks have to be scanned if I want it in my portfolio and that’s quite time consuming for me”, “I am using the reflections tool… it will be useful”, “I use the blog, not the CV tool as I use my own version, I don’t like the layout”

21. Student feedback 2 (Sports FDSc) “I find this portfolio system safe and also fun… I can post my work on it and add other accessories, e.g. poems, pictures, videos etc. Using this portfolio has been one of my greatest experiences” Non-traditional students compiled ePortfolios – introduced across whole course, assessed and supervised in lab.

22. Student feedback 3 (maths employability) 16 students filled in November questionnaire with 90% stating they had uploaded files and added guests, 70% posted a comment and adapted the presentation, only 40% using the CV tool. The most positive ratings were for guest control, showcasing achievement, improving employability, developing self and subject awareness, reflecting on progress. The least positive ratings were for developing creativity and learning about ePortfolios.

23. Results of evaluation questionnaires In November generally positive on a Likert scale questionnaire about features In January, at the end of the modules, a free text email drew 26 responses, 6 positive and 20 negative Evidence from ePortfolios seen by author is patchy (CVs uploaded here, blog written there) except Sports Coaching – see earlier slide – for showcasing/ feedback & session evaluation/ SMART/ standards evidence/ learning object design

24. Conclusions London Met has decided not to adopt one single ePortfolio system at present, Insufficient evidence of ‘stickability’, Need user friendly ePortfolio system, integrated into all aspects of the student’s life, including regular assessment related tasks and enthusiastically supported by academic staff, Considering alternatives (new Blackboard, Pebble Pad, ELGG/Moodle, Web 2 Services), Researching staff/student opinion…

25. Future work Working with the IV Cohort of the International Coalition for ePortfolio Research, using repertory grid analysis technique of personal construct theory (Kelly 1955, Steed & McDonnell 2003) Identifying the key features required for the future ‘stickability’ of ePortfolios in HE

26. Repertory Grid Analysis Conference delegates may wish to consider its possible adoption, its strengths and weaknesses. First, identify a category such as ‘storing, presenting and reflecting upon our own evidence of achievement’ (e.g. research papers, reports, other products)

27. RGA – selecting elements to compare What elements in the above discourse are important? Examples: –Web site - PC disc – USB - protected web site –Blog - Wiki – Stored emails - Diary –Professional portfolio – CV tool –Brainstorming tool - Action plan tool

28. RGA (continued) constructs Choose any 3 elements (and select two alike and the other different ‘in some way’ – this is the personal construct (& its opposite) – and repeat Grade all the elements in a grid for each pair of constructs (including the ‘ideal’) Analyse the pattern of graded constructs to identify important features required for ‘ideal’ ePortfolio

29. References Chalk, P (2007) ’The WebLearn Portfolio and HEO pilot – implications for a blended learning approach’, workshop at London Met Teaching & Learning Conference 10/7/07. URL: homepages.north.londonmet.ac.uk/~chalkp/res/pd p-july-conf.ppt.workshop Kelly, G A (1955) The psychology of personal constructs, vol 1 and 2, Norton, New York.

30. References (continued) Steed, A., & McDonnell, J. (2003). Experiences with repertory grid analysis for investigating effectiveness of virtual environments. In Proceedings of the 6th International Workshop on Presence. Aalborg, Denmark, 6-8 October 2003, www.cs.ucl.ac.uk/staff/A.Steed/presence20 03_fourpage.pdf (accessed 15.2.08) www.cs.ucl.ac.uk/staff/A.Steed/presence20 03_fourpage.pdf

31. Acknowledgements The author would like to thank the contributions of the module leaders to this pilot project and to this paper: Mehryar Adibpour, Sarah Atchia, David Blundell, Dafna Hardbattle, Maureen Kendal, Justin Lance, Tom Lunt, Rosemary Stott and Heather Wanstall. The author is a member of Cohort IV of the International Coalition for Electronic Portfolio Research (http://ncepr.org/).