1. Creating Tutorials for the Web: a Designer’s Challenge Module 4: Checking for Effectiveness

2. Learning Process – a quick reminder! What do you want the student to be able to do? - outcomes What does the student need to know? - curriculum What is the learning activity? - pedagogy How will the student demonstrate the learning activity? - assessment How will you know the student has done this well? - rubric

3. Why do Assessment? Necessary to add to the educational experience Part of the learner centered approach Satisfies accountability Important for fulfilling goals of strategic plans etc Can be relevant for accreditation in an academic setting

4. Performance Assessment Requires the student to do something more than just fill in a questionnaire or give true/false answers Student performs a task and generates their own response Task is usually a performance or extended response that requires the application of critical thinking skills

5. Rubrics Scoring guides which define criteria for judging student performance allow for consistency in rating performance give students a set of expectations of what will be assessed and standards that need to be met Can help avoid bias

6. Sample Tutorial Lesson & Outcomes Lesson: - Teaching relevant subject searching Outcomes: - Students will be able to apply subject searches in a catalog that retrieve results with a high degree of relevance to their chosen topic - Students will be able to retrieve records in the catalog doing a keyword search - Students will be able to identify subject headings in catalog records - Students will be able to identify suitable headings - Students will search catalog with LC subject headings relating to their topic

7. Sample Tutorial Curriculum Curriculum: - Students need to know what types of information catalog records contain - Students need to understand what happens in a catalog/database when the user executes a search - Students need to know where to locate LC subject headings in a catalog record - Students need to know the difference between a keyword and LC subject search - Students need to conduct subject searches of their own

8. Sample Tutorial Pedagogy Pedagogy: - Working with a local area telephone directory the student is asked to find the telephone numbers of the local movie theatre and a local area plumbing company - The student then repeats the process using the Yellow Pages - The student is asked to comment on the differences in searching for the same information in two different source arrangements - Explain the differences by showing how the Yellow Pages “indexes” its entries using specialized terms, “controlled vocabulary” - Show a keyword search in the catalog using natural language, “Internet and security” - Examine a record from the list of items returned - Highlight the LC subject headings and run another search on a selected subject to find all other records in the catalog that share that same subject heading - Students will repeat this process starting with a keyword search on a topic of their choice and execute a subject search based on their findings - Students will maintain a log of their search terms, results, and process

9. Sample Tutorial Assessment Assign each student a random topic Student will have to note three keywords for their topic (some may be synonyms) Using their self selected keywords they will do a keyword search in the catalog They will select records that will give them at least three subject headings that area appropriate to their topic/keywords List subject headings that best match the original topics

10. Sample Tutorial Rubric CriteriaFully Understands Basic Understanding No Understanding Analyze topic and devise keywords Can derive at least 3 keywords that match topic Can derive at least 2 keywords with moderate relevance to topic Can derive only 1 keyword from topic with tenuous relevance to topic Active catalog searching Can do keyword and subject searching to find records relevant to topic Can do keyword and subject search to find records of moderate relevance to topic Can perform keyword searches but unable to refine search with subject headings Identification of controlled vocabulary Can find 2 relevant subject headings Can find 1 subject heading Cannot use keywords to find relevant subject headings Retrieving relevant records Can retrieve 2 or more records with direct relevance to topic Can find 1 or more records with moderate relevance to topic Can find 1 or no records with even slight relevance to topic

11. Taking it Online Existing tutorials can be adapted for the Web Activities such as in the Sample Tutorial can be refined - the telephone directory activity could consist of scanned pages from the directories to illustrate the points being made - students could be required to do this activity themselves and report their findings via a Web form in the tutorial - eg: matching a given set of criteria with check boxes etc

12. Taking it Online In an online tutorial students could be asked to submit their sample searches and research log via e-mail The assessment activity can be conducted by presenting the student with a list of possible topics The results of the assessment can be submitted via a Web based form

13. And Finally… Many of the suggestions for putting the Sample Tutorial online involve a degree of involvement from the librarian - reviewing and commenting on Web form submissions It is possible to design a self-contained tutorial where the student is able to gauge their own progress – cannot continue to the next level until the previous skill has been mastered Have colleagues or a control group do the tutorial before you release it to a larger audience Change is a constant