Sakai in Language Courses: Present Uses and Future Possibilities Ken Romeo, Ph.D. Academic Technology Specialist ::

Outline Background The Present: Sakai in the Stanford Language Center Objectives and Results Summative Assessment Formative Assessment The future: What this could be How students study How teachers (would like to) teach A framework for curriculum Heads up! This is not just about language teaching. July 9, 2009Stanford Language Center / Academic Computing2

Background Me ESL instructor (20 years) Academic Technology Specialist (2006) CourseWork (Sakai) team meeting observer / participant (2008) Stanford Language Center Language requirement 1995 – new director: Prof. Elizabeth Bernhardt Emphasis on assessment and professional development This presentation American Council on the Teaching of Foreign Languages (ACTFL 2007) Bernhardt, Molitoris, Miano, Gelmetti, Tsethlikai, Romeo Sum of experience 3July 9, 2009Stanford Language Center / Academic Computing

SAKAI AND THE STANFORD LANGUAGE CENTER The Present: July 9, 2009Stanford Language Center / Academic Computing4

Assessment Program Objectives Improve student performance Enhance credibility (students and the public) Programmatic consistency Methods ACTFL Oral Proficiency Standards (ACTFL, 1999) Oral Placement and Exit exams More face-time for instructors  put diagnostic assessment online (CourseWork) 5July 9, 2009Stanford Language Center / Academic Computing

Overview: Results More highly trained staff 95% go through ACTFL interview training Over 1/3 certified Professional conversation 20% first year and 24% second year increase Highly positive student reaction Use Sakai to: Deliver formative assessments (SAMigo) Connect to exit assessment (enrollment & archiving) 6July 9, 2009Stanford Language Center / Academic Computing

Foreign Language Placement 800 unregistered students: not in Sakai Online tests during the summer Oral assessment on campus: 1 hour, cassette tapes 7July 9, 2009Stanford Language Center / Academic Computing

Exit Assessment: SOPI Definition Oral Proficiency Interview (OPI): Live Nationally standard format – used everywhere Simulated Oral Proficiency Interview (SOPI): Recorded 1st year: ~10 items, 2nd year: ~20 items: English instructions + line drawing Thinking time Native prompt Response recording Go to next item (NO USER CONTROL) From 2008: Add a writing assessment 8July 9, 2009Stanford Language Center / Academic Computing

SOPI Delivery Scheduling, content creation – human hours Software requirements High stakes: content security and reliability Enrollment from existing courses (Sakai) Non-standard roles: Coordinator, instructor No student control = no web delivery Playback or recording Test progression Securely archive audio recordings (Sakai) 9July 9, 2009Stanford Language Center / Academic Computing

SOPI Software (Project Manager: Makoto Tsuchitani) Application (Developer: Casey Dunn) Desktop Java application Quicktime for Java – play and record WebServices to communicate with Sakai Sakai Monitor Tool (Developer: Zach Thomas) Realtime progress of each student UI depends on role Packaging for further dissemination 10July 9, 2009Stanford Language Center / Academic Computing

SOPI Application July 9, Stanford Language Center / Academic Computing

SOPI Monitor Tool July 9, Stanford Language Center / Academic Computing

Formative Assessments in Sakai 1 st year: Oral diagnostic assessments Collaboratively developed content (audio / video / text) SAMigo: Audio Recording applet Resource (course) sites 2 nd year: Writing Diagnostic assessments Collaboratively developed content (images / text) SAMigo: Timed, short answer Resource (course) sites 13July 9, 2009Stanford Language Center / Academic Computing

Technology Integration: Key Points Each part has a pedagogically valid purpose and is not focused on technology. Uses a standalone application with connection to Sakai to do what the web can’t. All instructors create material based on the same standards-based framework. Implementation takes a huge number of human hours and coordination across many different groups. 14July 9, 2009Stanford Language Center / Academic Computing

WHAT THIS COULD BE The Future: July 9, 2009Stanford Language Center / Academic Computing15

Prologue: Why use an LMS at all? Privacy and authentication Scalability: class  department  university (  ?) Modularity: Centrally stored, clone-able units One portal to existing technology (connection, organization, computation, audio, video, telephony, etc.) 16July 9, 2009Stanford Language Center / Academic Computing

HOW STUDENTS STUDY July 9, 2009Stanford Language Center / Academic Computing17

Increased levels of connection Cell phones: voice and text Sharing – small groups, whole class, future classes (Submission? Grading? Feedback?) Online office hours Social networking – managing multiple identities, authentication to protect privacy 18July 9, 2009Stanford Language Center / Academic Computing

Familiarity with digital environment New formats for work (video, etc.) Don’t always carry laptops – often use clusters, do carry cell phones Use multiple resources – search / self-study / scaffolding Note: Find the least common denominator 19July 9, 2009Stanford Language Center / Academic Computing

HOW TEACHERS (WOULD LIKE TO) TEACH My Wishlist July 9, 2009Stanford Language Center / Academic Computing20

Different tools for different tasks In class teaching Homework Self-study 21July 9, 2009Stanford Language Center / Academic Computing

No-brainer: Video, everywhere Upload  ingest to streaming Clip creation, indexing, delivery, annotation, collaboration Control playback – just once, twice Why streaming? Too large to download. Why not YouTube? No privacy / authentication. (which is what Sakai does very, very well!) 22July 9, 2009Stanford Language Center / Academic Computing

Organization (schedule, groups) Course structure: Linear by definition Tracking students Integration with University registrars Arbitrary groups of students In/across courses/programs Requirements, milestones, electives 23July 9, 2009Stanford Language Center / Academic Computing

Identities (roles) Need a departmental or other arbitrary level Social networking (or not) … Multiple identities Retaining pedagogical control: Assessment In/out groups: Fairness? Motivation? 24July 9, 2009Stanford Language Center / Academic Computing

Assessment (SAMigo) Control response format: Limit and time chances to view/listen Limit and time chances to respond Enable large scale assessment Get rid of all those blue books Work with infrastructure groups: machines, space Telephone delivery of Sakai is a killer app: Accessibility Very controlled linear assessment Anyone want to be partners in a grant? 25July 9, 2009Stanford Language Center / Academic Computing

Content creation (SAMigo) Authoring: basic desktop tools, conversion, definitions, spell checking Break up into modules: Re-usable, sharable, organize-able Changing order of delivery Changing details of content: Randomization of items / variables Downstream control of shared content 26July 9, 2009Stanford Language Center / Academic Computing

Reports (gradebook, etc.) Grading on an arbitrary curve Item analysis: Stats, test theory Assessments / items across departments / years (aggregate-able) Log files: Reportable numbers to stakeholders 27July 9, 2009Stanford Language Center / Academic Computing

A MODULAR CURRICULUM A Framework for the Future July 9, 2009Stanford Language Center / Academic Computing28

How? Homework: collaboration, or not – pedagogical control Self-study: Students who need it Students who want to do self-study Publicly available “open-source” set of online activities Classroom: more in a minute … 29July 9, 2009Stanford Language Center / Academic Computing

Why? Textbooks = information + activities + order Textbooks unbound  Teacher must decide order Information Multiple sources Multiple formats Students can independently supplement Basic unit = activity Requires / allows creativity 30July 9, 2009Stanford Language Center / Academic Computing

Basic curricular unit: the Activity Re-conceptualize interactive lecture / seminar Control information in the classroom Elicit search / curiosity outside of the classroom Facilitate all departments’ products Linguists’ tree diagrams Engineering simulations Video Etc. 31July 9, 2009Stanford Language Center / Academic Computing

In the Classroom Physical: personal interactions, hardware (or not) Virtual: distance learning Small group activities: Quick, arbitrary, but airtight membership Posting materials for each group separately Posting product of each group separately Quick, but controlled access to multimedia Easy creation of multimedia product An example: DIY fill in the blanks 32July 9, 2009Stanford Language Center / Academic Computing

Classroom Example: DIY Fill in the Blanks 2 groups 2 short (30 sec) audio clips 2 paper transcripts 2 pairs of scissors Each group cuts out 15 words to make a fill in the blanks problem Exchange transcripts Play clips (x2-3) Group with most correct answers wins July 9, 2009Stanford Language Center / Academic Computing33

Digitize DIY Fill in the Blanks Arbitrary groups Secure content: audio and text Modify text Exchange text – without revealing original Play audio Check answers Group collaboration July 9, 2009Stanford Language Center / Academic Computing34

Epilogue: Take home messages Focus on the users: how can we change education? (ref: introduction of textbooks) Understand what the important characteristics of pedagogy are (and are not). Don’t just facilitate pedagogy as it is, find out where pedagogy is going by talking to expert teachers. Focus on pedagogically valid activities – not on the tools. Motivated students / creative teachers are NOT the problem: We need to reach everyone. 35July 9, 2009Stanford Language Center / Academic Computing

THANK YOU VERY MUCH! Ken Romeo :: :: July 9, 2009Stanford Language Center / Academic Computing36