OSP User Experience: Current Examples and Calls for Future Collaboration Teggin Summers, Assistant Director ePortfolio Initiatives, Virginia Tech Lynn Ward, Principle Systems Analyst, Indiana University Purdue University Indianapolis Steve Foehr, Director of Application Services, Rhode Island Network for Educational Technology Janice Smith, Consultant, Three Canoes Consulting Tiffany Marra, Project Manager, Univeristy of Michigan Noah Botimer, ePortfolio Technical Lead, University of Michigan

Introductions July th Sakai Conference - Boston, MA, U.S.A. Teggin SummersAssistant Director of Electronic Portfolio Initiatives, Virginia Tech Lynn WardPrinciple Systems Analyst, Indiana University Purdue University Indianapolis Steve FoehrDirector of Application Services, Rhode Island Network for Educational Technology Janice SmithConsultant, Three Canoes Consulting 2009 Sakai Fellow Tiffany MarraProject Manager, University of Michigan Noah BotimerePortfolio Technical Lead, University of Michigan Sakai Product Council, Member

Panel Objectives Share a wide variety of the user experiences we have encountered Start a conversation to discuss how we might broaden the OSP community to include more discussion of user experience (and perhaps teaching & learning?) July th Sakai Conference - Boston, MA, U.S.A.3

Brief History of OSP at Virginia Tech OSP tools integrated into our instance of Sakai (“Scholar”) since 2003 Virginia Tech OSP system based on the University of Michigan Scaffolding Portfolio However, we have extended the schemes, layouts, navigation capability, and functions of the UMSCP to satisfy our local needs July th Sakai Conference - Boston, MA, U.S.A.4

Different Uses for our eP Tools Learning, Assessment, and Professional Development Training for Academic Support Groups Co-curricular ePortfolios July th Sakai Conference - Boston, MA, U.S.A.5

July th Sakai Conference - Boston, MA, U.S.A.6

July th Sakai Conference - Boston, MA, U.S.A.7

July th Sakai Conference - Boston, MA, U.S.A.8

Learning, Assessment, & Professional Development July th Sakai Conference - Boston, MA, U.S.A.9

Learning, Assessment, and Professional Development July th Sakai Conference - Boston, MA, U.S.A.10

Learning, Assessment, and Professional Development July th Sakai Conference - Boston, MA, U.S.A.11

Learning, Assessment, and Professional Development July th Sakai Conference - Boston, MA, U.S.A.12

Learning, Assessment, and Professional Development July th Sakai Conference - Boston, MA, U.S.A.13

Training for Academic Support Groups July th Sakai Conference - Boston, MA, U.S.A.14

Co-curricular ePortfolios July th Sakai Conference - Boston, MA, U.S.A.15

Challenges Our Users Have Encountered Definitely affected by the form slow down issue (with the help of Michigan, we found a solution to this) Multiple tools in multiple sites leads to confusion FCK Editor “Submitting evidence” can be confusing July th Sakai Conference - Boston, MA, U.S.A.16

Additional Steps to Facilitate Use Documentation Videos Coordination with our support groups (i.e., Innovation Space) Channel student energy Website: July th Sakai Conference - Boston, MA, U.S.A.17

OSP AND PROGRAM ASSESSMENT: LOWERING THE BAR TO PARTICIPATION Lynn Ward, Principle Systems Analyst Indiana University July th Sakai Conference - Boston, MA, U.S.A.18

OSP > 20 active portfolio projects underway, many in professional schools Majority focus on outcomes assessment at the program level Some disciplines (Education, Nursing, Visual Communication) embrace portfolio pedagogies Others (Engineering, Computer Science) do not) July th Sakai Conference - Boston, MA, U.S.A.19

Barriers to IU Difficult to achieve faculty consensus and buy-in across entire program Perceptions that application is too complicated, difficult to use Concerns about faculty workload Reports and templates require custom programming July th Sakai Conference - Boston, MA, U.S.A.20

Removing the Barriers Difficult to achieve faculty consensus and buy-in across entire program Perceptions that application is too complicated, difficult to use Concerns about faculty workload Reports and templates require custom programming July th Sakai Conference - Boston, MA, U.S.A.21

Design and Development Goals Simplicity - simplify workflows for all roles, especially faculty and student participation in program assessment projects Flexibility - accommodate a wide range of approaches to portfolio use and assessment Standardization – define and implement standards to facilitate reporting July th Sakai Conference - Boston, MA, U.S.A.22

Simplified Workflow for Program Assessment Instructor and student never leave course site/tools: 1.Coordinator associates course site(s) with portfolio site 2.Students in associated site automatically added to portfolio site 3.Instructors create and link assignments to matrix cells 4.Students submit linked assignments 5.Evaluator(s) review and evaluate linked assignment submissions. 6.Report results July th Sakai Conference - Boston, MA, U.S.A.23

July th Sakai Conference - Boston, MA, U.S.A.24 Associate Course Sites 1 2

Roster Syncing July th Sakai Conference - Boston, MA, U.S.A.25

Roster Sync Auto Groups July th Sakai Conference - Boston, MA, U.S.A.26

Synced Roster July th Sakai Conference - Boston, MA, U.S.A.27

Instructor Links an Assignment: 1 July th Sakai Conference - Boston, MA, U.S.A.28

Instructor Links an Assignment: 2 July th Sakai Conference - Boston, MA, U.S.A.29

Student Submits Assignment July th Sakai Conference - Boston, MA, U.S.A.30

Submitted Assignment in Matrix: 1 July th Sakai Conference - Boston, MA, U.S.A.31

Submitted Assignment in Matrix: 2 July th Sakai Conference - Boston, MA, U.S.A.32

Submitted Assignment in Matrix: 3 July th Sakai Conference - Boston, MA, U.S.A.33

Evaluating Linked Assignments Add evaluator(s) to class group in portfolio site Convert cell to pending for entire class July th Sakai Conference - Boston, MA, U.S.A.34

Pending Cells in Evaluations Tool July th Sakai Conference - Boston, MA, U.S.A.35

Evaluations can be hidden… from students July th Sakai Conference - Boston, MA, U.S.A.36

Evaluation can be hidden… from other evaluators July th Sakai Conference - Boston, MA, U.S.A.37

Report and Analyze Results July th Sakai Conference - Boston, MA, U.S.A.38

Additional Workflow Support Participant can “request feedback” notifications: For Reviewer when Participant requests feedback For Evaluator when Participant Submits for Evaluation For Participant when feedback or evaluation is added to cell. July th Sakai Conference - Boston, MA, U.S.A.39

Future Directions Evaluation directly within Assignments tool Enable linking in other Sakai tools Long term storage for all portfolio evidence, including linked items Improved scalability and performance Advanced and custom reporting Advanced multimedia support (including a turn-key video solution) Simplified site management and scaffolding authoring tools. July th Sakai Conference - Boston, MA, U.S.A.40

RI Proficiency Based Graduation Requirements Standards Based Applied Learning Problem solving, critical thinking, research, communication, and reflection/evaluation Student choice/Responsibility Proficiency for Graduation July th Sakai Conference - Boston, MA, U.S.A.

RI Proficiency Based Graduation Requirements Standards Based Curriculum Development Student Artifacts July th Sakai Conference - Boston, MA, U.S.A.

Ballston Spa Central School District OSP Implementation Janice Smith Three Canoes Education Consultant

Ballston Spa Central School District BSCSD located near Albany, New York Hands-on coordination by Superintendent Joe Dragone, Ph.D. Planning group includes: IT Coordinator School Principals Lead Teachers for Each Grade Guidance Counselors July th Sakai Conference - Boston, MA, U.S.A.44

Project Background / History IBM Innovation Retreat in Albany in 2007 October 22 nd 2008 IBM Open Education Services Briefing December 4 th 2008 OSP / Site Visit in RI (RINET) Spring 2009 Phase I Ballston Spa Competency Mapping Summer 2009 Phase II OSP Customization Future / TBD – OSP Training/Support & Staff Professional Development

BSCSD Competency Mapping Project High Level Project Plan

Competency Mapping Project Phases Phase I Curriculum Mapping Mapping BSCSD Graduate Competencies to NYS Performance Indicators Mapping NYS Performance Indicators to BSCSD 9th Grade ELA and 5 th Grade curriculum Phase II Sakai/OSP Customization 9 th Grade ELA (English Language Arts) 5 th Grade (4 integrated content areas)

Phase I Deliverables: Curriculum Maps 9th Grade English Language Arts Curriculum Map 9 th Grade ELA NYS Performance Indicator Code 9 th Grade ELA NYS Performance Indicator Text Ballston Spa Central School District Graduate Competencies Sample Topics / Activities Sample ELA Assessments Available Resources 5th Grade Curriculum Map 5 th Grade NYS Performance Indicator Code 5 th Grade NYS Performance Indicator Text Ballston Spa Central School District Graduate Competencies Sample Topics / Activities Sample 5 th Grade Assessments Available Resources Plus calendar component

Phase II Project Objectives Gather OSP requirements via Ballston Spa site visit Confirm OSP requirements via follow-up report Design OSP data structures according to requirements Set up OSP data structures and workflow on the Ballston Spa server (running rSmart Sakai CLE) Conduct on-site user acceptance testing

OSP Workflow Design Portfolio Site Goal Site Assignments Goal 2 Portfolio Report Goal 1 Goal 3 Matrix Reflection Form Graduate Competencies Evaluation Form NYS Performance Indicators

Portfolio and Goal Sites Portfolio sites for each grade level course and class Goal sites (using project sites) for each grade level

Assignments Assignments preloaded in draft form according to grade level curriculum maps Assignments linked to NYS Performance Indicators Teachers select and flesh out assignments Students complete and submit assignments

Matrices and Forms Submitted assignments appear in matrix cells according to BSCSD Graduate Competencies mapped to linked NYS Performance Indicators Students reflect upon their work for each Graduate Competency and submit reflection cells for evaluation Teachers use evaluation forms and evaluations tool to rate and comment on student work for each Graduate Competency

Portfolio Templates Students create a portfolio to share their work on BSCSD Graduate Competencies Exact design of portfolio template for each grade level to be determined Students will be able to display their matrix reflections and evaluations in relation to assignments and other artifacts for each Graduate Competency

Reporting Options Plans for BSCSD Reporting Fall 2009 – Community development work will allow direct query of form information for reporting Reports design will focus on student mastery of BSCSD Graduate Competencies accessed via evaluation forms Reports will include data on Individual students Aggregation of all students within Each course Each grade level within and across schools Each school Across the Ballston Spa Central School District

Administrative Concerns: My Workspace Tools Fewer to begin with User account features No use of (point to nowhere to begin) Read-only account tool Teacher collaboration sites Sharing of assignments Communication tools

Looking to the Future Proof of concept project Intention to expand to more grade levels Importance of teacher and student buy-in 9 th grade ELA 5 th grade Modeling use of Sakai/OSP for K-12 Making customized OSP data structures available to similar K-12 projects

Contact Us July th Sakai Conference - Boston, MA, U.S.A. Teggin SummersAssistant Director of Electronic Portfolio Initiatives, Virginia Tech Lynn WardPrinciple Systems Analyst, Indiana University Purdue University Indianapolis Steve FoehrDirector of Application Services, Rhode Island Network for Educational Technology Janice SmithConsultant, Three Canoes Consulting 2009 Sakai Fellow Tiffany MarraProject Manager, University of Michigan Noah BotimerePortfolio Technical Lead, University of Michigan Sakai Product Council, Member