A Showcase of Open Source Portfolio Innovations OSP Community Members Monday, June 14, 2010

OSP Community Presenters Janice Smith, Three Canoes, USA Tiffany Marra, University of Michigan, USA Brian Dashew, Marist College, USA Lynn Ward, Indiana University, USA Nancy O’Laughlin, University of Delaware, USA Riyuichi Matsuba, Kumamoto University, Japan Hugo Jacobs, LOI, The Netherlands

Workshop Agenda Introductions and Needs Assessment OSP Overview (for Newbies) Page Composer: Introduction and Innovations Break Teaching/Learning/Assessment Portfolios Innovations from Kumamoto University and LOI OSP Reports Creating a Portfolio Culture at Your Institution

Introductions and Needs Assessment Please share Your name Your institution Current status of Sakai/OSP at your institution Specifics on what you need from this workshop We will meet in small groups later in the workshop according to your needs.

OSP Overview Janice Smith, Ph.D. Consultant Three Canoes June 14, 2010

Sakai 2.6 Portfolio Tools Portfolio Sites Resources Matrices Wizards Glossary Evaluations Forms Portfolios Portfolio Templates Styles Portfolio Layouts Reports

Portfolio Sites Differentiated from course / project sites Specifically dedicated to portfolio work Specialized tools/roles /permissions Portfolio tools also available for course and project sites

Resources Based on the principle of entering and updating data once to use it for multiple purposes Provides a My Workspace folder for each user to collect artifacts Offers site owners a place to distribute content Collects interaction for each user in each portfolio site in a portfolio interaction folder accessible to that user

My WorkSpace Portfolio Resources

Forms Created with xsd via Forms tool xsd may be generated using Form Builder tool Available for export/import Accessed through and stored in Resources Used for Providing reflection prompts Offering feedback and evaluation Structuring participant content Matrices Wizards Portfolios

Form Design and Use Form Builder Optional use of contrib Form Builder tool to create xsd for use in Forms Tool.

Wizards A series of screens to "scaffold" users through the process of entering portfolio data Two types of wizards: Sequential Wizards A set of pages presented one after the other. Hierarchical wizards A nested tree of categories and pages Wizard pages can include user guidance (instructions, rationale and examples) Wizard creators add forms to prompt for Evidence and documentation of learning Reflections Feedback Evaluations

Wizards

Wizard Pages The designer may also allow users to attach files to individual forms.

Matrices Matrices are a type of wizard Matrix cells are almost identical to wizard pages Matrices allow: Application of styles Customization of Rows and columns For each cell Instruction, rationale, examples Progression across cell Reflection Feedback Evaluation

Matrix Cells

Glossary Global entries for instance or local entries for sites Local entries override global entries Long and short descriptions (definitions) Defines terms for published matrices and wizards May also be accessed directly (for use as a dictionary) Export/Import

Evaluations Instructor resource for receiving student submissions of matrix cells and wizard pages Allow instructors to submit evaluations without accessing matrix or wizard Instructors with permission may also access matrices and wizards of individual students via drop-down menus in matrix and wizard tools Aggregation of evaluations across sites is available in My Workspace

Portfolios May be created from portfolio templates May also be created freeform by user User selects content and applies style and layout Choice of styles and layouts supplied by site organizer or portfolio admin Aggregation of portfolios across sites is available in My Workspace Resume Portfolios created using a portfolio template

Matrix Portfolio Created from the data within a matrix

Co-Curricular Transcript May be created with or without a matrix

Portfolio Templates Created with.xml and used to structure and display student evidence of learning according to the student’s choice of audience..

Portfolio Templates

Template-based Portfolio

Portfolio Layouts Optionally combined with a style for the creation of a freeform portfolio – a freely standing that does not require a portfolio template

Free Form Portfolio

Styles Applicable to Matrices and matrix cells Wizards and wizard pages Portfolio layouts

Reports Require use of report templates (definitions) Templates specify data to be collected Data may be displayed, printed, and/or exported May be created using Sakai Reports tool Tool donated by Serensoft to extract data from forms Direct SQL query of OSP data warehouse tables

Portfolio Admin Site with admin tools for portfolios Allows membership in site for use of admin tools Houses global Forms Glossary items Portfolio Layouts Styles Report templates (definitions)

OSP Tool Interactions

Barriers to Using OSP Functional knowledge of the tools and how they work together Takes significant time to understand Much of that time will be spent unproductively Technical knowledge Not every shop understands the XML stack, or how to apply the knowledge to OSP Where the technical knowledge exists, the connection to the pedagogy may not be there Portfolio implementations are much more than tech projects Ongoing leadership is essential in sustaining the effort into the future

OSP Community Resources OSP Community Weekly phone conference, listserv, OSP documentation and resources OSP Community Library Use cases and data structures for export Sakai Confluence OSP documentation and resources rSmart CLE Portfolio Showcase Three archetypal portfolio implementations with data structures for export Three Canoes Services for Sakai/OSP implementation

Introduction to Page Composer (via internet) Tiffany Marra, U of Michigan Brian Dashew, Marist College June 14, 2010

Variations on a Theme A New Take on UMich’s Page Composer Lynn Ward Principal Systems Analyst Indiana University June 14, 2010

Portfolio IU OSP community member and a lead developer since 2004 Initial focus on integrative learning, program assessment, and institutional assessment Development resources focused exclusively on building out OSP’s assessment capabilities Michigan’s Page Composer - potential to fill a significant void at IU 11th Sakai Conference - June 15-17,

IU Presentation Maker Project funded jointly by IUPUI and UITS, IU’s central IT organization UMich Page Composer serves as starting point for development IUPUI ePort Executive Committee develops customization requirements Contract with Three Canoes for customizations 11th Sakai Conference - June 15-17,

Customization Goals Flexibility – facilitate simple web site construction for other purposes Course content Student projects Etc. Usability – tweaks to forms for content creation and required settings Themes with IU/IUPUI branding Build your own theme 11th Sakai Conference - June 15-17,

What We Gave Up Auto-save Ability to autofit FCK editor to content and toggle Sakai navigation on/off Ability to mix and match content pages in Work Showcase section 11th Sakai Conference - June 15-17,

Flexibility Up to seven single or multi-page sections Ability to group pages in multipage sections into categories Ability to use a predefined list of categories or allow user to define 11th Sakai Conference - June 15-17,

Single or Multipage Sections 11th Sakai Conference - June 15-17,

Single or Multipage Sections 11th Sakai Conference - June 15-17,

Categorizing Pages 11th Sakai Conference - June 15-17,

Usability Tweaks: Required Settings 43

Usability Tweaks: Page Creation Removed theme selection Removed controls to size editor and toggle Sakai navigation on/off Renamed fields 11th Sakai Conference - June 15-17,

Page Creation 11th Sakai Conference - June 15-17,

46 Themes

Build Your Own Theme Options Banner Area Banner graphic Color Font face Font color Main Menu Font Face Font Color Side Menu Font colors Background colors Page Content Font face Text color Heading color Background color Page Background Graphic Graphic Repeat Pattern Color 47

48

Build Your Own Theme 11th Sakai Conference - June 15-17,

Next Steps Tweak based on feedback from pilot, including FCK upgrade Train teaching center consultants to customize Pilot in Fall 2010 and further refinement Deploy global templates for showcase portfolio and generic web site Expand theme library Bundle IU version for distribution via th Sakai Conference - June 15-17,

Teaching/Learning/Assessment UD Nancy O’Laughlin University of Delaware June 14, 2010

Early pilots Secondary Science Education Assessment portfolio Presentation portfolio (used UMich – Page Composer) Nutrition and Dietetics Program Assessment portfolio – primarily a type of journaling, reflection Secondary English Education - assessment Nursing Practitioners – presentation 12 Grant projects

Grant projects For teaching, learning and assessment 12 projects – represent a variety of departments Programmatic assessment, assess at least 3 of our General Education goals One project just implemented Others implemented this fall

Grant Projects Art Conservation Business Marketing English Engineering Technology Fashion Hotel, Restaurant & Institutional Management Sport Management Human Services Early Childhood Education Music Educational Leadership* School Psychology* Undergraduate Research**

Example row for matrix development

Example matrix

Contents of cells – capture reflection User forms Upload artifacts Reflect – students provided with prompts Instructors – Evaluate Use simple form Own rubric VALUE rubrics For institutional column – using VALUE rubrics

Notification Tool for Matrices and Power Link to WebCT (Look for these slides in the session on Ja Sakai Updates ) Ryuichi Matsuba Kumamoto University June 14, 2010

OSP IU Lynn Ward Principal Systems Analyst Indiana University June 14, 2010

OSP > 25 active portfolio projects underway, many in professional schools Majority focus on outcomes assessment at the program level Highly individualized approach to assessment Individual differences accommodated via custom scaffoldings and forms 61

OSP IU.5 – 1.5 FTE developers FTE designer.25 FTE combined consultant, analyst, tech writer, trainer, QA specialist Faculty support through teaching centers Outsource some QA and development work Growing knowledge among consultants in campus teaching centers 62

The Conundrum Project coordinators need to aggregate, analyze, and summarize evaluation (and other types of) data Evaluation forms differ widely in terms of # of rating fields # rating scale # labels assigned to each point on scale XML Programmer required to develop custom reports for each project – DOES NOT SCALE 63

Our Solution Define local form standards Flexible Predictable Publish and promote standards Define report definitions based on those standards Reports “discover” the XSD structure on the fly and render the results 64

Evaluation Form Standard Field name conventions: Rating fields (rating, rating 1, rating2, rating3 …rating20) Comments (comments) Rating fields must be represented as a selection field (drop-down menu) All rating fields in the form (except rating) must use the same rating scale. Enumeration values must be a number, “n/a”, or null Rating fields may not be repeatable 65

Customization Options Rating fields and comments fields can use any display name Rating scales can have any number of points The display label for each point on the scale can be any valid string Evaluation forms can have any valid name Each matrix cell can use a different evaluation form. July th Sakai Conference - Boston, MA, U.S.A.66

Examples 67

Woodrow Wilson Fellowship Matrix 11th Sakai Conference - June 15-17,

WW Sample Rubric 11th Sakai Conference - June 15-17,

Ratings Report: Main (Matrix) View 70 Click to see summary report for cell

Ratings Report: Summary View 71 July 2009 Click to see detailed report for cell Click to return to main (matrix) view.

Ratings Report: Details View 72 Click to return to summary view Click to return to main (matrix) view. Click NetID to open cell in new window.

Attachments Report: Main Menu 11th Sakai Conference - June 15-17,

Attachments Report: Summary 11th Sakai Conference - June 15-17,

Cell Status Summary 75

Cell Status Detail 76

Matrix Link Summary 77

Next Steps Refinement of exiting reports Develop additional reports: Text area and WYSIWYG content for qualitative analysis Merge Reports tool enhancements to trunk Post report definitions to OSP library so other institutions can use them. 78

For More Information Lynn Ward Chris Maurer Bryan Holladay 79

Creating a Portfolio Culture at Your Institution Janice A. Smith, Ph.D., Three Canoes with significant contributions by Shelley Smith, Darren Cambridge, and rSmart

Presentation Outline 1.Applying Folio Thinking to Learning 2.Understanding Campus Culture 3.Getting Campus Buy-In 4.Developing a Portfolio Culture on Campus

Applying Folio Thinking to Learning Folio thinking starts with what we know about learning by identifying: Measurable outcomes Processes that work toward these outcomes Software that enables the processes Section 1

From Science to Software Learning science defines dimensions of deep learning Folio thinking defines behaviors linked to these dimensions The “Portfolio Process” promotes these behaviors Section 1

Dimensions of Deep Learning Reflective Learning Integrative Learning Social Learning Section 1

Reflective Learning Professionals use reflection in the workplace – Learning and performance is a cycle of reflection and action (Schon, The Reflective Practitioner) Information becomes knowledge when situated in the cycle by a knower (Brown and Duguid, Social Life of Information) The cycle is hard-wired into our brains (Zull, The Art of Changing the Brain) Section 1

Dimensions of Reflection Reflection-in-action “reviewing, projecting, revising” Constructive reflection “developing a cumulative, multi-selved, multi-vocal identity” Reflection-in-presentation “articulating the relationships between and among” creation, creator, and context of creation” Reflection as conversation with artifacts, with self, with others (— Kathleen Yancey, Reflection in the Writing Classroom) Section 1

Integrative Learning as Expert Thinking Experts – Develop a conceptual framework for information Notice features and patterns Organize their knowledge to reflect deep understanding Apply their knowledge to new situations Monitor their own understanding as they work by modifying concepts identifying information gaps taking control of their learning Section 1 (Donovan, et. al., How People Learn)

Integrative Learning for Careers Scholarship of integration – an important component of research (Boyer, Scholarship Reconsidered) Systems thinking – crucial for professional excellence (Partnership for 21st Century Skills) Multiple careers require continuous learning (Chen and Mazow, Stanford Center for Innovation in Learning) Increasingly, we have career trajectories, not careers (Brown 2004 AAHE keynote) In the USA – 75% of students are older, independent, work full time, attend part time (NCES 2002) 58% attend multiple institutions (NCES 2002) Section 1

Social Learning Our identities are formed through participation in communities of practice (Wenger, Communities of Practice) Learning to be a member of a disciplinary or professional community is as important or more important than learning “content” (Brown and Duguid) Multiple identities Are formed in the context of multiple communities Are integrated into learning career trajectories The social dimension is the most difficult of the three to accomplish via portfolios Section 1

Summarizing Folio Thinking Academia values the creation, organization, and refinement of knowledge Folio thinking gets to the heart of these academic values by asking students to define and refine their relationship to knowledge Folio thinking promotes deep learning through Reflection on learning Integration of learning Sharing of learning Folio thinking leads to the outcome of students explaining and predicting their development as learners The results of folio thinking are available for the assessment and evaluation of learning Section 1

Pause to Reflect Find someone you do not know and discuss: What aspects of folio thinking are most significant for you? How do portfolios at your institution support folio thinking? Section 1

Understanding Campus Culture Motivators for Students, Faculty, and Administrators Section 2

What is culture? Culture is the shared processes which organize our differences A campus is a system of subcultures that create a dynamic whole Each subculture shares certain values and beliefs that are Unique to them Common to the campus culture Changing any part of the cultural system requires an adjustment to the whole system Section 2

Why is culture important? Understanding your campus cultures helps you to: Motivate and persuade students, faculty, and administrators Develop specific strategies for each portfolio stakeholder group Develop a plan of action for implementing ePortfolios on campus Section 2

Beliefs and Values in Higher Education Beliefs (personal, social, spiritual, political): what is true, including: Academic and research paradigms The nature of knowledge and how students learn The purpose of education Values: What is right and wrong; ideals rather than actions, including: What is good teaching? What is ethical teaching and scholarly practice? What is the purpose of a learning community? Section 2

Analyzing Student Culture Success as a student The importance of grades Positioning for a career Relevance of learning to real world tasks and goals Improved self-esteem Good teaching What are student beliefs and values regarding: Section 2

Analyzing Student Culture Available power sources Student evaluation of teaching Paying or withdrawing tuition Affinity to faculty and institution Complaints and grievances Workload expectations and concerns Peer influence/student grapevine What are student beliefs and values regarding:

Identifying Student Motivators Improved learning outcomes Enjoyment of learning through creativity and increased control Better positioning for a career Relevance to “real world” concerns and goals Improved student resources Enhancement of self-esteem and confidence Receiving credit and recognition for co-curricular activities Others? Section 2

Analyzing Faculty Culture Teaching and learning Faculty rewards Peer influence Available power sources Knowledge Charisma Disciplinary status Faculty status Research record Ability to reward or sanction others Ability to provoke change Complaints and grievances What are faculty beliefs and values regarding: Section 2

Identifying Faculty Motivators Faculty workload Time release for faculty development or pursuing desired goals Less time required for necessary tasks Faculty rewards Promotion and Tenure Awards and Recognition Faculty accountability for student learning Personal goals and values regarding education and scholarship Intellectual curiosity about how students learn Others? Section 2

Analyzing Administrative Culture Management in higher education Available power sources Money Positional power Charisma Ability to reward or sanction others Ability to institute change Administrative rewards Workload expectations and concerns Peer influence What are administrator beliefs and values regarding: Section 2

Identifying Administrator Motivators Efficient use of funds to extend resources Improved institutional reputation Attract students Increase funding Enhance administrator influence, status, & position Improved student learning addresses personal goals and values regarding quality education Others? Section 2

Pause to Reflect Identify several motivators on your campus that - Are similar for faculty, administrators, and students Differ across faculty, administrators, and students Section 2

Getting Campus Buy-In ePortfolio Benefits and Challenges Section 3

ePortfolio Benefits to Students ePortfolios: Emphasize reflection, analysis, and continuous improvement Integrate learning across the full educational experience Integrate knowledge and experience Assist with managing educational and professional records Maintain a catalog of accomplishments over time Help prepare for career selection and the job search Enhance self-esteem and confidence Section 3

ePortfolio Benefits to Faculty and Administrators Course-level assessment for student learning Program-level assessment for student learning and accreditation Enhanced student-faculty relationships Improved advisement Enhanced faculty development Streamlined promotion and tenure Section 3

ePortfolio Benefits to Parents, Employers, and Colleagues A window for parents into their children’s education by providing an opportunity to: View learning presentations and products Share more directly in experiences like study abroad and service learning A resume-enhancing tool that provides employers with critical information about applicant skills and learning A means for colleagues to exchange information and feedback about research, teaching, and service ePortfolios can be: Section 3

Challenges to ePortfolio Use Limited faculty and student time and energy Lack of funding Lack of understanding of the value of ePortfolios in enhancing student learning The willingness of faculty and administrators to promote ePortfolios and folio thinking The need for faculty and students to develop the skills to use ePortfolios Inadequate technological resources and support Fear of technology Resistance to change Educating ePortfolio audiences for viewing Section 3

Addressing Challenges to ePortfolio Use I Time and Energy: Portfolios use the same amount of time differently to: Document educational and professional progress Seek and receive feedback Become more self-directed in learning Money: Once implemented, ongoing costs are minimal Value of Reflection: Portfolios focus on higher levels of thinking Skills: The software is relatively easy to implement; learning to document and reflect upon one’s learning is more difficult Section 3

Addressing Challenges to Portfolio Use II Technological resources: Portfolios do require equipment and expertise Fear of technology: User-friendly technology Resistance to change: Folio thinking requires new ways of thinking about learning, with the opportunity for greater transparency and success Educating ePortfolio audiences: It is important to encourage opportunities to interface using ePortfolios

Pause to Reflect What specific challenges to portfolio culture do you face on your campus? What are some concrete ways to persuade students, faculty, and administrators on your campus of the value of ePortfolio use? Section 3

Implementing ePortfolios on Campus Integrating ePortfolios into Academic Processes Section 4

Integrating ePortfolios on Your Campus Student Development Admission Student orientation Advisement Assessment of Learning Course and Program level Professional certification Institutional Assessment Learning Beyond the Classroom Study Abroad Co-Curricular Activities Service or Experiential Learning Career Development Faculty Concerns Faculty development Promotion and tenure Curriculum development Address ePortfolio use through its potential for : Section 4

Campus Activities to Introduce ePortfolios Initiate conversations among faculty, administrators, and students about Benefits of ePortfolios Challenges to ePortfolio use Provide hands-on learning about ePortfolios through Faculty and student orientation sessions Master student classes & first year seminars General education courses Advisement Faculty development Provide resources for pilots in various campus units Section 4

Motivating Faculty to Use ePortfolios Reward faculty for implementing ePortfolios with: Release time Summer institutes or retreats Small grants from institutional money Assistance in seeking external grants Showcasing effective faculty portfolio use Encourage faculty to use their own faculty development activities to learn and model ePortfolio use Demonstrate how ePortfolios can streamline faculty workload by: Decreasing paper flow Minimizing lost assignments Assisting with assignment review and feedback Improving assessment of student learning Enhancing benefits already present in a course management system Section 4

Motivating Administrators to Support ePortfolio Use Strengthening and streamlining advisement Providing students with read-only access to student records Encouraging students to document and appreciate learning in and beyond the classroom Developing and accessing data on learning assessment Encouraging interdisciplinary collaboration in curriculum development Developing and submitting promotion and tenure files Streamlining applications for grants and awards Demonstrate ePortfolio efficiency and effectiveness in: Section 4

Motivating Students to Use Portfolios Enhancement of self-esteem Awareness of how much and how deeply one has learned Demonstration of progress to faculty, peers, family, and employers Improved learning outcomes Incorporation of standards Enhanced feedback from faculty and peers Increased creativity Integration of learning in and outside of the classroom Real-world relevance Better positioning for careers and job search Section 4

Strategies for Success Begin small With pilot groups Specific colleges, programs, or activities Cohorts of students By discipline By year or semester Expand as demand builds Incrementally by adding units, activities, or cohorts To include as many common interest groups as possible Section 4

Pause to Reflect Identify at least one educational process into which your campus can integrate ePortfolios. Choose one or more effective strategies to promote effective use of ePortfolios within that process. Section 4