1. OSP User Experience: Current Examples and Calls for Future Collaboration Teggin Summers, Assistant Director ePortfolio Initiatives, Virginia Tech Lynn Ward, Principle Systems Analyst, Indiana University Purdue University Indianapolis Steve Foehr, Director of Application Services, Rhode Island Network for Educational Technology Janice Smith, Consultant, Three Canoes Consulting Tiffany Marra, Project Manager, Univeristy of Michigan Noah Botimer, ePortfolio Technical Lead, University of Michigan

2. Introductions July 2009210th Sakai Conference - Boston, MA, U.S.A. Teggin SummersAssistant Director of Electronic Portfolio Initiatives, Virginia Tech Lynn WardPrinciple Systems Analyst, Indiana University Purdue University Indianapolis Steve FoehrDirector of Application Services, Rhode Island Network for Educational Technology Janice SmithConsultant, Three Canoes Consulting 2009 Sakai Fellow Tiffany MarraProject Manager, University of Michigan Noah BotimerePortfolio Technical Lead, University of Michigan Sakai Product Council, Member

3. Panel Objectives Share a wide variety of the user experiences we have encountered Start a conversation to discuss how we might broaden the OSP community to include more discussion of user experience (and perhaps teaching & learning?) July 200910th Sakai Conference - Boston, MA, U.S.A.3

4. Brief History of OSP at Virginia Tech OSP tools integrated into our instance of Sakai (“Scholar”) since 2003 Virginia Tech OSP system based on the University of Michigan Scaffolding Portfolio However, we have extended the schemes, layouts, navigation capability, and functions of the UMSCP to satisfy our local needs July 200910th Sakai Conference - Boston, MA, U.S.A.4

5. Different Uses for our eP Tools eP@VT Learning, Assessment, and Professional Development Training for Academic Support Groups Co-curricular ePortfolios July 200910th Sakai Conference - Boston, MA, U.S.A.5

6. eP@VT July 200910th Sakai Conference - Boston, MA, U.S.A.6

7. eP@VT July 200910th Sakai Conference - Boston, MA, U.S.A.7

8. eP@VT July 200910th Sakai Conference - Boston, MA, U.S.A.8

9. Learning, Assessment, & Professional Development July 200910th Sakai Conference - Boston, MA, U.S.A.9

10. Learning, Assessment, and Professional Development July 200910th Sakai Conference - Boston, MA, U.S.A.10

11. Learning, Assessment, and Professional Development July 200910th Sakai Conference - Boston, MA, U.S.A.11

12. Learning, Assessment, and Professional Development July 200910th Sakai Conference - Boston, MA, U.S.A.12

13. Learning, Assessment, and Professional Development July 200910th Sakai Conference - Boston, MA, U.S.A.13

14. Training for Academic Support Groups July 200910th Sakai Conference - Boston, MA, U.S.A.14

15. Co-curricular ePortfolios July 200910th Sakai Conference - Boston, MA, U.S.A.15

16. Challenges Our Users Have Encountered Definitely affected by the form slow down issue (with the help of Michigan, we found a solution to this) Multiple tools in multiple sites leads to confusion FCK Editor “Submitting evidence” can be confusing July 200910th Sakai Conference - Boston, MA, U.S.A.16

17. Additional Steps to Facilitate Use Documentation Videos Coordination with our support groups (i.e., Innovation Space) Channel student energy Website: http://eportfolio.vt.eduhttp://eportfolio.vt.edu July 200910th Sakai Conference - Boston, MA, U.S.A.17

18. OSP AND PROGRAM ASSESSMENT: LOWERING THE BAR TO PARTICIPATION Lynn Ward, Principle Systems Analyst Indiana University July 200910th Sakai Conference - Boston, MA, U.S.A.18

19. OSP Use @IU > 20 active portfolio projects underway, many in professional schools Majority focus on outcomes assessment at the program level Some disciplines (Education, Nursing, Visual Communication) embrace portfolio pedagogies Others (Engineering, Computer Science) do not) July 200910th Sakai Conference - Boston, MA, U.S.A.19

20. Barriers to Adoption @ IU Difficult to achieve faculty consensus and buy-in across entire program Perceptions that application is too complicated, difficult to use Concerns about faculty workload Reports and templates require custom programming July 200910th Sakai Conference - Boston, MA, U.S.A.20

21. Removing the Barriers Difficult to achieve faculty consensus and buy-in across entire program Perceptions that application is too complicated, difficult to use Concerns about faculty workload Reports and templates require custom programming July 200910th Sakai Conference - Boston, MA, U.S.A.21

22. Design and Development Goals Simplicity - simplify workflows for all roles, especially faculty and student participation in program assessment projects Flexibility - accommodate a wide range of approaches to portfolio use and assessment Standardization – define and implement standards to facilitate reporting July 200910th Sakai Conference - Boston, MA, U.S.A.22

23. Simplified Workflow for Program Assessment Instructor and student never leave course site/tools: 1.Coordinator associates course site(s) with portfolio site 2.Students in associated site automatically added to portfolio site 3.Instructors create and link assignments to matrix cells 4.Students submit linked assignments 5.Evaluator(s) review and evaluate linked assignment submissions. 6.Report results July 200910th Sakai Conference - Boston, MA, U.S.A.23

24. July 200910th Sakai Conference - Boston, MA, U.S.A.24 Associate Course Sites 1 2

25. Roster Syncing July 200910th Sakai Conference - Boston, MA, U.S.A.25

26. Roster Sync Auto Groups July 200910th Sakai Conference - Boston, MA, U.S.A.26

27. Synced Roster July 200910th Sakai Conference - Boston, MA, U.S.A.27

28. Instructor Links an Assignment: 1 July 200910th Sakai Conference - Boston, MA, U.S.A.28

29. Instructor Links an Assignment: 2 July 200910th Sakai Conference - Boston, MA, U.S.A.29

30. Student Submits Assignment July 200910th Sakai Conference - Boston, MA, U.S.A.30

31. Submitted Assignment in Matrix: 1 July 200910th Sakai Conference - Boston, MA, U.S.A.31

32. Submitted Assignment in Matrix: 2 July 200910th Sakai Conference - Boston, MA, U.S.A.32

33. Submitted Assignment in Matrix: 3 July 200910th Sakai Conference - Boston, MA, U.S.A.33

34. Evaluating Linked Assignments Add evaluator(s) to class group in portfolio site Convert cell to pending for entire class July 200910th Sakai Conference - Boston, MA, U.S.A.34

35. Pending Cells in Evaluations Tool July 200910th Sakai Conference - Boston, MA, U.S.A.35

36. Evaluations can be hidden… from students July 200910th Sakai Conference - Boston, MA, U.S.A.36

37. Evaluation can be hidden… from other evaluators July 200910th Sakai Conference - Boston, MA, U.S.A.37

38. Report and Analyze Results July 200910th Sakai Conference - Boston, MA, U.S.A.38

39. Additional Workflow Support Participant can “request feedback” Email notifications: For Reviewer when Participant requests feedback For Evaluator when Participant Submits for Evaluation For Participant when feedback or evaluation is added to cell. July 200910th Sakai Conference - Boston, MA, U.S.A.39

40. Future Directions Evaluation directly within Assignments tool Enable linking in other Sakai tools Long term storage for all portfolio evidence, including linked items Improved scalability and performance Advanced and custom reporting Advanced multimedia support (including a turn-key video solution) Simplified site management and scaffolding authoring tools. July 200910th Sakai Conference - Boston, MA, U.S.A.40

41. RI Proficiency Based Graduation Requirements Standards Based Applied Learning Problem solving, critical thinking, research, communication, and reflection/evaluation Student choice/Responsibility Proficiency for Graduation July 20094110th Sakai Conference - Boston, MA, U.S.A.

42. RI Proficiency Based Graduation Requirements Standards Based Curriculum Development Student Artifacts July 20094210th Sakai Conference - Boston, MA, U.S.A.

43. Ballston Spa Central School District OSP Implementation Janice Smith Three Canoes Education Consultant

44. Ballston Spa Central School District BSCSD located near Albany, New York Hands-on coordination by Superintendent Joe Dragone, Ph.D. Planning group includes: IT Coordinator School Principals Lead Teachers for Each Grade Guidance Counselors July 200910th Sakai Conference - Boston, MA, U.S.A.44

45. Project Background / History IBM Innovation Retreat in Albany in 2007 October 22 nd 2008 IBM Open Education Services Briefing December 4 th 2008 OSP / Site Visit in RI (RINET) Spring 2009 Phase I Ballston Spa Competency Mapping Summer 2009 Phase II OSP Customization Future / TBD – OSP Training/Support & Staff Professional Development

46. BSCSD Competency Mapping Project High Level Project Plan

47. Competency Mapping Project Phases Phase I Curriculum Mapping Mapping BSCSD Graduate Competencies to NYS Performance Indicators Mapping NYS Performance Indicators to BSCSD 9th Grade ELA and 5 th Grade curriculum Phase II Sakai/OSP Customization 9 th Grade ELA (English Language Arts) 5 th Grade (4 integrated content areas)

48. Phase I Deliverables: Curriculum Maps 9th Grade English Language Arts Curriculum Map 9 th Grade ELA NYS Performance Indicator Code 9 th Grade ELA NYS Performance Indicator Text Ballston Spa Central School District Graduate Competencies Sample Topics / Activities Sample ELA Assessments Available Resources 5th Grade Curriculum Map 5 th Grade NYS Performance Indicator Code 5 th Grade NYS Performance Indicator Text Ballston Spa Central School District Graduate Competencies Sample Topics / Activities Sample 5 th Grade Assessments Available Resources Plus calendar component

49. Phase II Project Objectives Gather OSP requirements via Ballston Spa site visit Confirm OSP requirements via follow-up report Design OSP data structures according to requirements Set up OSP data structures and workflow on the Ballston Spa server (running rSmart Sakai CLE) Conduct on-site user acceptance testing

50. OSP Workflow Design Portfolio Site Goal Site Assignments Goal 2 Portfolio Report Goal 1 Goal 3 Matrix Reflection Form Graduate Competencies Evaluation Form NYS Performance Indicators

51. Portfolio and Goal Sites Portfolio sites for each grade level course and class Goal sites (using project sites) for each grade level

52. Assignments Assignments preloaded in draft form according to grade level curriculum maps Assignments linked to NYS Performance Indicators Teachers select and flesh out assignments Students complete and submit assignments

53. Matrices and Forms Submitted assignments appear in matrix cells according to BSCSD Graduate Competencies mapped to linked NYS Performance Indicators Students reflect upon their work for each Graduate Competency and submit reflection cells for evaluation Teachers use evaluation forms and evaluations tool to rate and comment on student work for each Graduate Competency

54. Portfolio Templates Students create a portfolio to share their work on BSCSD Graduate Competencies Exact design of portfolio template for each grade level to be determined Students will be able to display their matrix reflections and evaluations in relation to assignments and other artifacts for each Graduate Competency

55. Reporting Options Plans for BSCSD Reporting Fall 2009 – Community development work will allow direct query of form information for reporting Reports design will focus on student mastery of BSCSD Graduate Competencies accessed via evaluation forms Reports will include data on Individual students Aggregation of all students within Each course Each grade level within and across schools Each school Across the Ballston Spa Central School District

56. Administrative Concerns: My Workspace Tools Fewer to begin with User account features No use of email (point to nowhere to begin) Read-only account tool Teacher collaboration sites Sharing of assignments Communication tools

57. Looking to the Future Proof of concept project Intention to expand to more grade levels Importance of teacher and student buy-in 9 th grade ELA 5 th grade Modeling use of Sakai/OSP for K-12 Making customized OSP data structures available to similar K-12 projects

58. Contact Us July 20095810th Sakai Conference - Boston, MA, U.S.A. Teggin SummersAssistant Director of Electronic Portfolio Initiatives, Virginia Tech teggin@vt.edu Lynn WardPrinciple Systems Analyst, Indiana University Purdue University Indianapolis leward@iupui.edu Steve FoehrDirector of Application Services, Rhode Island Network for Educational Technology steve@staff.ri.net Janice SmithConsultant, Three Canoes Consulting 2009 Sakai Fellow janice.smith@threecanoes.com Tiffany MarraProject Manager, University of Michigan tmarra@umich.edu Noah BotimerePortfolio Technical Lead, University of Michigan Sakai Product Council, Member botimer@umich.edu