Designing, Developing, and Evaluating an Interdisciplinary Digital Library Curriculum NSF: Interested in education History: DLs dev for UG ed Ed use of cyberinfrastructure DLs as tools for realizing potential of cyberinf Jeffrey Pomerantz School of Information & Library Science University of North Carolina at Chapel Hill pomerantz@unc.edu Tapping the vast reservoir of human knowledge --Louis Round Wilson, founder, 1931

Acknowledgements Project team: UNC: Barbara Wildemuth, Sanghee Oh VT: Ed Fox, Seungwon Yang Project advisory board This material is based upon work supported by the National Science Foundation under Grant Nos. IIS-0535057 (VT) and IIS-0535060 (UNC-CH).

Introduction to the Project Develop curriculum materials for teaching digital library topics For use in both LIS and CS programs Lesson plans, exercises, assignments, etc. For the classroom, not online Where is the line between DL curriculum and LIS curriculum?

Development and Evaluation Process

Computing Curriculum 2001 11. IM. Information Management (10 core hours) IM1. Information models and systems (3) IM2. Database systems (3) IM3. Data modeling (4) IM4. Relational databases IM5. Database query languages IM6. Relational database design IM7. Transaction processing IM8. Distributed databases IM9. Physical database design IM10. Data mining IM11. Information storage and retrieval IM12. Hypertext and hypermedia IM13. Multimedia information and systems IM14. Digital libraries ACM/IEEE-CS IM = 11 of 14 core areas Note that IM1-3 are the only core, others optional

Topics of Readings in DL Courses Pomerantz, J., Oh, S., Yang, S., Fox, E. A., & Wildemuth, B. M. (2006). The Core: Digital Library Education in Library and Information Science Programs. D-Lib Magazine, 12(11). http://dx.doi.org/10.1045/november2006-pomerantz

Topics of Conference Papers 20 40 60 80 100 120 140 160 180 200 1 2 3 4 5 6 7 8 9 Module ID Number of conference papers JCDL 05 JCDL 04 JCDL 03 JCDL 02 JCDL 01 ACM DL 00 ACM DL 99 ACM DL 98 ACM DL 97 ACM DL 96 Pomerantz, J., Wildemuth, B., Fox, E. A., & Yang, S. (2006). Curriculum Development for Digital Libraries. In Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (pp. 175-184). New York: Association for Computing Machinery. http://doi.acm.org/10.1145/1141753.1141787 Services: LIS vs. CS Data Visualization Collection Development Digital Objects Metadata Architecture Services Intellectual Property Future of DLs Preservation

Topics of Papers in D-Lib Magazine 20 40 60 80 100 120 140 160 180 200 1 2 3 4 5 6 7 8 9 Module ID Number of D-Lib articles D-Lib 06 D-Lib 05 D-Lib 04 D-Lib 03 D-Lib 02 D-Lib 01 D-Lib 00 D-Lib 99 D-Lib 98 D-Lib 97 D-Lib 96 D-Lib 95 Pomerantz, J., Wildemuth, B., Fox, E. A., & Yang, S. (2006). Curriculum Development for Digital Libraries. In Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (pp. 175-184). New York: Association for Computing Machinery. http://doi.acm.org/10.1145/1141753.1141787 Services: LIS vs. CS Data Visualization Collection Development Digital Objects Metadata Architecture Services Intellectual Property Future of DLs Preservation

2006 Curriculum Framework

2007 Curriculum Framework

2008 Curriculum Framework

Module Template Module name Scope Learning objectives Level of effort required Relationships with other modules Prerequisite knowledge required Introductory remedial instruction Body of knowledge Resources Exercises / Learning activities Evaluation of learning outcomes Glossary Contributors Completed modules: Ask: Which do you want to see? Reference Completed modules

Module Development Process First draft written by a single author. Module is reviewed by the research team; feedback is provided to the author. Author makes revisions to the module. Module is posted on the project wiki for expert evaluation. (Evaluators have been previously recruited.) Evaluators post comments to the wiki. Modules are available to be implemented in the classroom.

Module Evaluation Process Modules are available to be implemented in the classroom. Instructor decides to use a module, modifies it to suit their teaching. After the module is used in class: Students are emailed to fill out an online survey. Instructors are interviewed. Graded student work is collected, if any.

Evaluation: Student Survey Clearly outlined objectives and outcomes were provided. The module was well-organized. The amount of work required for this module was appropriate. The assigned readings helped me better understand the subject matter. Given the module’s objectives, the learning activities and/or assignments were appropriate. The learning activities and/or assignments required thinking and understanding. The learning activities and/or assignments were stimulating. Assignments for this module helped me understand what will be expected of me as a professional. I learned useful professional skills from this module. I know significantly more about this subject than before I took this module. Class lectures added to my understanding of the subject. I gained a good understanding of the basic concepts related to this subject. I learned to interrelate important issues related to this subject. This module stimulated me to think critically about the subject matter. I feel that this learning module served my needs well. I was very satisfied with this learning module. Overall, considering its content, design, and structure, this module was effective. 1-5 Likert scale

Evaluation: Instructor Interview Objectives Body of knowledge Readings Learning Activities Logistics Overall structure of the module 1. Objectives Are the objectives observable? If not, how should they be changed? Are they appropriate within the context of your course? If not, how should they be changed? Were your students able to achieve the objectives, given the content in the body of knowledge? 2. Body of knowledge Does the module address all areas of the topic that need to be addressed? Are there any topics that you think are critical to add to the body of knowledge? Did you add these topics to your own presentation of this module? Are there any topics on that you would remove from the body of knowledge? Did you skip any topics in your own presentation of this module? Did the body of knowledge enable students to achieve the objectives? 3. Readings Are the readings the best and most appropriate for the topic? Are there any readings that you think are critical to add to the list? Did you assign these readings in your class? Are there any readings on the list that you would remove? Did you assign these to your class? 4. Learning Activities Which activities did you implement in your class? Were they successful in achieving your objectives? Did you make any changes in the activities as you implemented them? Are the activities appropriate for the topic? Were students able to complete the activities successfully? Did the activities enable students to achieve the module’s objectives? Did you lead the class in any learning activities, other than those described in the module? Can you think of any other class activities that would be appropriate for this module? 5. Logistics: Is it feasible to teach the module as it is currently constructed? Is the level of effort required in class appropriate to the scope of the body of knowledge? The level of effort required prior to class? Is the prerequisite knowledge required sufficient for students to comprehend the body of knowledge? 6. Overall structure of the module: Is current module well structured? Can the topics and their corresponding resources be easily divided? If you used this module in conjunction with another module, was their articulation graceful? Is there a clear mapping between the objectives and the content of the body of knowledge section? If not, how could the objectives be mapped to the body of knowledge more clearly?

Evaluation: Assigned Work Analyzed with respect to the objectives of the module. Triangulated with the student survey data as a check on both.

Administrative Issues Biannual meetings of the advisory board & others, at ASIST & JCDL conferences Recruiting module authors, expert evaluators, and instructors

Future Work Continue to develop modules Develop community of interest How to keep a community of interest interested? Where to host modules in the long term? Wikibooks, Wikiversity, learning object repositories?

Future Work NSF Workshop in November EU-US ATLANTIS Programme Transatlantic Degree Consortia Project IMLS and/or Mellon Foundation Problem-based learning curriculum, internships

Project URLs Project site: curric.dlib.vt.edu Includes links to all publications Project wiki: curric.dlib.vt.edu/wiki/ Includes all modules ready to be used & being evaluated This presentation: www.ils.unc.edu/~jpom/conf/Pomerantz_LIDA2008.ppt

Thank You! Jeffrey Pomerantz School of Information & Library Science University of North Carolina at Chapel Hill pomerantz@unc.edu Tapping the vast reservoir of human knowledge --Louis Round Wilson, founder, 1931