2. Open Source Learning Tools Charles Kerns Education Technology Manager Academic Computing, Stanford University charles.kerns@stanford.edu

3. The Presentation Learning tools and open source development CourseWork, Stanford’s CMS The Assignment and Assessment Manager, a new, modular CourseWork tool

4. What is Open Source? Licensed software that can be freely distributed License allows users to modify and adapt to their own needs May have restrictions on redistribution Community of developers and users

5. Open Source Programs that We Already Know and Love Linux sendmail Apache Tcl/Tk Perl Emacs MySQL PHP In fact, a lot of the internet runs on open source software!

6. The Vision Institutions sharing open source tools Basic open source frameworks that accept tools and integrate them with: Campus SIS, Portal, Authentication System, File Servers, Calendar, etc. Modular, interoperable, open source tools Rich set of open source tools to support: discussion, quizzes, learning communities, problem sets, project-based learning, syllabi,etc.

7. The Need-Control of the Campus Learning Tools to integrate learning tools into campus infrastructure SIS, file servers, authentication, email, portal, calendar to modify tools to fit pedagogical methods of faculty to add new tools requested by faculty to develop content with faculty using learning tools. to always retrieve faculty content - content is NEVER locked in proprietary formats to have tool continue even if original developer “dies”

8. Why an Open Source CMS? Port software to your environment: no longer restricted by vendor requirements Modify and expand software to meet your needs and integrate with existing campus systems Easy to integrate features, customizations, and fixes developed by other users Work with a common technical framework to share reusable learning objects

9. More Reasons Why Open standards allow interoperability of applications and services Non-proprietary data formats mean easier conversion to or from the system Reduced acquisition costs: no need to pay for evaluation software, and can take as long as you need You don’t have to pay for upgrades!

10. But that doesn’t mean you get something for nothing…. Must have or develop internal expertise to manage the system and development tools Extensive customization may result in development and support costs equal to or greater than the cost of commercial software…. …but at least you have control: you’re no longer dependent on vendors for upgrades and integration

11. Recent History of Learning Tools Hundreds of Course Management Systems and Learning Tools with many as Open Source Thelma Looms Listing at GWU  160 with a few “free” from 98 to 02 THOT  235 plates-formes e-formation, plates-formes e-learning with 28 Open Source - current EduTools Listing  32 with 10 Open Source - current

12. Some Open Source CMS’sOpen Source CMS’s Open Learning Management System Open Learning Management System - Dept of Psych-Utah – Basic System FLE3 Future Learning Environment FLE3 Future Learning Environment –Helsinki – Collaborative Knowledge Bldg Moodle Moodle by Martin Dougiamas, Australia – Week-by-week Courses

13. Some more Open Source CMS’sOpen Source CMS’s CHEF CHEF –a full portal with research and instructional support system OnCourseOnCourse - Indiana’s reworked CMS CourseWork CourseWork – Stanford’s 80% solution. Supports lectures and seminars.

14. The History - Research and Non-Education-Specific Apps Blogs Wikis Collaborative Knowledge Building Virtual Worlds for Chat Structured Discussion Content mixed with Tools

15. The History - Commercial and Open Source Learning Tools No standards No modularity Little collaboration among developers Much redundancy Some unique applications collaborative knowledge building, structured discussion, problem-based learning

16. The History - Commercial and Open Source Learning Tools All are vertical apps leading to: Much effort for Sys Admins  Many separate apps to maintain and integrate Much time for faculty  Multiple student lists, Separate systems for assessment, difficult to relate components Much confusion for students  Multiple sites, multiple passwords, no integration between activities

17. What’s Needed Standards – OKI, IMS Organization for tech support & distribution A Merlot++ for infrastructure tools?? User community Many users helping each other Development community Several developers of tools using same framework

18. What’s Happening-Standards OKI Collaboration of many universities, funded by Mellon Foundation. IMS Content Packaging, QTI, etc. DTD’s SCORM

19. What’s Happening - Open Knowledge Initiative (OKI) Collaboration of many universities, funded by Mellon Foundation. Goal Define open architectural specifications to support the development of educational software. Provide a modular and extensible development platform for building both traditional and innovative educational applications while helping institutions leverage existing infrastructure. More information: http://web.mit.edu/oki

20. What’s Happening - OKI Develop standard APIs for: Common Services  Authentication  Authorization  Database Connectivity  Filing  Logging Class Info (student lists) Assessment and Grades

21. What’s Happening - User Groups & Distribution Support Content Merlot (Learning Object Sharing) Content and Tools LON CAPA and WebWork have user groups LON CAPA WebWork CourseWork Evaluation at Denison & other colleges CHEF Workshops Hewlett/Mellon looking at distribution

22. What’s Happening - Developers starting to collaborate First goal is to use each others’ tools in proof- of-concept demonstrations OKI users working together (It takes more than APIs) Umich CHEF Indiana OnCourse Stanford CourseWork MIT Stellar Tufts Concept Mapping Tool

23. Evaluation for Open Source Release 5 Universities/colleges (already selected) are setting up local server and testing. They will report on features, admin issues, compatibility with their campus systems. Two have servers running with little assistance; third needed help in setting up Tomcat Denison University is evaluating CourseWork for reporting to CLAC Grant proposal submitted to Mellon Foundation for support for outreach to liberal arts colleges Changes will be made in open source version based on evaluation.

24. License Source is available with free download Can modify or adapt without notifying Stanford Can redistribute original or modified if do not charge Stanford holds copyright on CW Cannot sell CW

25. Questions on State of Open Source Tools... Tech Docs http://www.stanford.edu/group/coursework/docs/

26. CourseWork

27. Design Goals/Methods Easy to use by novice faculty Flexible for use by expert designers Student and Admin views Wizard-based admin 80% Solution for first pass Good for supporting most courses Lecture and seminar best supported

28. Component Tools Public/Private Homepage Announcements Public/Private Syllabus Schedule Course Materials Assignments (Quiz, Problem Sets, etc.) Discussion Grade Book

29. History and Near Future ’98-Research Project for HumBio Program ’00-Extended for Language Quizzing ’01-Full CMS Developed ‘02 Winter-Deployed for all Stanford faculty ‘02 Fall-Version 2 with links to Registrar ‘03-14,000 users; 1200 Courses Total ‘03-Evaluation by other institutions ‘03 Summer-Open Source Release

30. excellent very good good passable poor very poor Ratings of Experience with CourseWork by Discipline

31. not useful Somewhat useful very useful Usefulness of CourseWork Tools by Discipline

32. Tech Info CourseWork System Requirements Applications Web Server (Apache) Java Servlet Application Server (Tomcat 3) SQL-conformant database (Oracle 8) DTL (a free HTML template application) …running on Sun Solaris Other Tested Environments: Linux Tomcat 4 PostGreSQL

33. CourseWork Demo

34. CourseWork Questions...

35. Assignment & Assessment Tool

36. Assignment & Assessment Tool- AAM Gives online tests, quizzes, problem sets, formative assessments Extensible and modular Utilizes OKI Common Service APIs & Class API Informs development of Assessment API Utilizes IMS Question and Test Interoperability (QTI) metadata specification.

37. Pedagogical Features Supports discipline and teaching method-specific extensions Supports formative assessment (results used immediately to help learning) “Chunkable” assessments-questions not locked into independent tests

38. Templates Method for giving maximum flexibility with maximum usability Created by instructional technologists and advanced users – no programming involved Hide complexity from instructors and students Give flexibility to end user in features available

39. Templates - Flexibility in the layout of an assessment All questions, responses, and feedback can be in multiple media Questions can be accessed randomly or restricted to a fixed sequence with no returns to earlier questions Instructor can allow students to bookmark questions and return to them Web pages can be configured to contain a single question, a set of questions, or the full assessment Number of submissions by each student can be fixed or unlimited

40. Templates - Flexibility in evaluation views and procedures Instructor can select type of evaluation: comments (at part, question, or assessment level) or scores Scoring can be done with numeric, alphabetical, check/minus values Evaluation can be done with students remaining anonymous (for surveys or blind scoring) Evaluation can be limited by role (e.g., section scorer, peer review) Multiple options for handling late submissions (allowed-tagged, disallowed, allowed-no penalty)

41. Templates – Flexible access to assessments, scores, and feedback Release of assessment to learner can be immediate, time based, or contingent upon completion of another assessment Retraction of assessment can be by date or upon completion of assessment Release of feedback can be immediate or by date Release date, retraction date, feedback release date, score release date, and due date can be set independently. Assessment can be released to specific groups in class (e.g., sections) Assessment or question-taking duration can be set to specific number of minutes

42. Discipline-specific Extension: Language Instruction Multimedia questions Recorded oral responses Virtual conversations Oral proficiency testing Formative assessment Easy, quick preparation of tests with oral questions by instructors Grading flexibility,ease, speed

44. Teaching-method SpecificSupport for Large Lecture Science Courses Support for sections and TA workflow Annotated question pools for continuity as TA’s change Grading and Reporting flexibility Integrated e-mail alert system Special Question Type that has rationale with multiple choice M-Choice auto-scored; Rationales sorted for easy review; Most frequently missed questions analysis.

45. Type your rationale for your answer here.

46. “Chunked” Assessments Questions and response input can be included in other activities Stand-alone tests are non-mandatory Central processing and reporting of results Asynch Class Assessment Techniques in schedule, email, announcement (probe, muddiest part, etc.) Fuzzy line between tutorial authoring and materials creation when assessments can be included

47. Interactions with other Tools Display or get a question requested by another tool (poll in discussion, question in content, announcement, etc.) Pass a score to other tool Send feedback message to student(s) in email, announcement, FAQ, IM Post test date on schedule, syllabus Link to a discussion

48. Final Questions... More info: Send e-mail to coursework-info@stanford.edu Web Page http://aboutcoursework.stanford.edu charles.kerns@stanford.edu