© E. Kowch Telecommunications in Education: Examining Distributed Teaching and Learning Environments (EDER 677 L.91 ) teaching and Learning Environments: Guides for Critical Reviews and Evaluation for the Asynchronous Class and for the Web Site on November 7 from Calgary

© E. Kowch Some pointers If you would like to discuss your online project with me, I will be sure to be available at a time when you need… If you would like to discuss your DL portfolio, I would be delighted to discuss that with you too… just let me know… Housekeeping

© E. Kowch Presentation Overview In this lesson, we will: Explore the issues around designing, developing and utilizing web based assessment techniques for online learning environments or their parts - from the teacher or designer perspective. Discuss the importance of formative and summative program/course evaluation in the online learning context - from the teacher or designer perspective.

© E. Kowch Assessment and Evaluation - Definitions Assessment - determine changes in learner performance Evaluation - make judgments and decisions about learners Program or course evaluation - appraise the success of your program or course

© E. Kowch Assessment and Evaluation - Self Activity (something to ponder as you create evaluations for your own online learning project) List examples of web based assessment activities that you prefer (see the resource section of this lesson) What types of student learning do these activities support (e.g. Bloom’s Taxonomy – knowledge, comprehension, application, analysis, synthesis, evaluation)? Identify issues surrounding the use of web based assessment activities that you feel strongly about, and address those issues in your project.

© E. Kowch Triangle of Assessment The multidimensional approach to assessment Knowledge Transfer and Performance Knowledge Constructs and Reflection Concrete Knowledge Acquisition Bloom’s Taxonomy

© E. Kowch Concrete Knowledge Acquisition Concept "Factual Foundation Knowledge" - provides the ground floor in a learning hierarchy. You can evaluate the things that must be known - facts. The working knowledge or language of a subject discipline can be considered as a foundational knowledge to succeeding in a learning event, online or anywhere.

© E. Kowch Concrete Knowledge Acquisition Working Knowledge Examples Calculus The student can recognize and count numbers The student can add, subtract, multiply and divide Literary criticism and creative writing The student can apply the alphabet and the basic rules of grammar

© E. Kowch Concrete Knowledge Acquisition Assessment Techniques Measuring student mastery of the factual “foundation knowledge” Some in instructional methodologies where you use specific learning evaluation techniques (Great source book: Gronlund: The Assessment of Student Learning” 7th Ed.) We take this up in EDER 673 in detail. Drill and practice is one assessment technique:\ with components designed to check the level of mastery of basic knowledge. Drill and Practice multiple choice true and false matching ordering short answer

© E. Kowch Concrete Knowledge Acquisition More Assessment Ideas Give a Preview quiz – on preparatory textbook, article or online readings Give a Review quiz – on lecture content Give a Collaborative quiz – online discussion about the questions and answers on a quiz Students can develop their own quiz questions – learning how to design tests Online, other students can respond and evaluate the documents. For stronger students in the class – the writing of mid-term and final exam questions can also result in credit for this online “assignment”. Online quizzes - Integrity test questions – first or last question – Did you do this quiz on your own? (on site supervised testing is a simple and effective option).

© E. Kowch Concrete Knowledge Acquisition Web Based Tools for Evaluation: See the Resource Section in the Lesson Plan for November 7th for more of these: Hot Potatoes Quiz Lab TeAch-nology.com's Crossword Puzzle Maker Discovery School's PuzzleMaker (Word Searches)

© E. Kowch Knowledge Constructs and Reflection The Concept of Knowledge Constructs This is the process of students constructing and/or discovering their own knowledge. ( Self evaluation with cognitive (model) direction). Internalization and integration of new information with previously held knowledge (in the good old bad old days, we called this “synthesis” ;-)

© E. Kowch Knowledge Constructs and Reflection The Concept of Reflection Students reflecting on the construction of their knowledge as part of the evaluation process: Zone of proximal development can be created online too: the instructor watches and guides students to evaluate each other after they evaluate themselves. Students are comparing thoughts and ideas with others.

© E. Kowch Knowledge Constructs and Reflection Assessment Techniques Structured journals (e.g. responding to specific questions) Open-ended reflection journals (e.g. responding to broad or open questions) Collaborative discussions and summaries

© E. Kowch Knowledge Constructs and Reflection Assessment Techniques for Online Discussions See our home page links

© E. Kowch Knowledge Constructs and Reflection Assessment Examples Example One - Class Discussion Scoring Guide a rubric for a 100-point project (60% for responses to weekly readings, 40% for responses to classmates postings) Example Two - Grading Online Discussions various models based on support/clarity/helpfulness, an A-F scale, and groups grading each other

© E. Kowch Knowledge Constructs and Reflection Assessment Examples Example Three - Evaluation and Grading Criteria for WebBoard Discussion Assignments an outline of four evaluation criteria for discussion assignments Example Four - Discussion Guidelines and Grading Standards discussion grade guidelines on an A to F scale, as well as a 10-point guideline for participation

© E. Kowch Knowledge Constructs and Reflection Recall the Modes of Web Based Computer Mediated Communication Tools Asynchronous File transfers Threaded discussion forums Synchronous Virtual chats Shared whiteboards, URLs, and applications MUDs(multi-user dimensions) and MOOs (multi-user dimension - object oriented)

© E. Kowch Knowledge Constructs and Reflection Web Based Resources that help you evaluate your online learning environs, considering the mode of delivery / interaction chosen by the designer: Using Asynchronous Conferencing to Promote Critical Thinking Instructional Design Principles for Teaching in Computer Conferencing Environments Shaping Cyberspace Into Human Space Handbook for Instructors on the use of Electronic Discussions Check for Plagiarism

© E. Kowch Knowledge Transfer & Performance Concept Performance-based assessment - students demonstrate the (learned) behavior to be measured (by the evaluator). Authentic assessment - students demonstrate the acquisition and mastery of a 'real world' skill or concept. Learners applying knowledge and skills rather than simply remembering facts

© E. Kowch Knowledge Transfer & Performance Web Based Resources: Hotlinks to Assessment tools: Performance Assessment Performance Assessment Educational Issues Series Strategic Questions - What to Consider When Planning for Electronic Portfolios Electronic Teaching Portfolios Electronic Portfolio Checklist Portfolio Assessment Student Portfolios: Classroom Uses Student Portfolios: Administrative Uses Alternative Assessment and Technology Assessment Terminology: A Glossary of Useful Terms Collected Assessment Resources Center for Research on Evaluation, Standards, and Student Testing

© E. Kowch Online Program/Course Evaluation - Remember this Model from earlier in the course? Now we look at the “Evaluate” loop: (More detail will come in EDER 673)

© E. Kowch Online Program/Course Evaluation - Definitions Formative Evaluation The ongoing analysis of your program/course design during development and implementation. Much more on this in EDER 673 and EDER 675 Summative Evaluation Is conducted to determine whether your course or program is worthy of adoption or continuation leads to a “go/no go” decision on the program.

© E. Kowch Online Program/Course Evaluation - Formative Some of the on-going process that designers use at each stage of the course development process: They have colleagues and students constantly reviewing the development and implementation of your web course site – internal review They develop an evaluation checklist or form They ensure that each module is piloted They make the necessary revisions based on informal and formal feedback mechanisms

© E. Kowch Online Program/Course Evaluation (Summative) Judging the worth or merit of a program/course: This evaluation is conducted after a program/course/design has been in place for a complete cycle – usually an external review to the institutional course distributor Key Questions used in this type of program evaluation: Effectiveness – Do students learn the basic learning objectives or exceed them? Appeal – Does the program/course/design engage learners and students – do they like it? (Affective) Efficiency – Is the program/course cost effective?

© E. Kowch Program/Course Evaluation - Resources Article Shrock, S.A. and Geis, G.L. (1999). Evaluation. In H.D. Stolovitch and E.J. Keeps (Eds.) Handbook of human performance technology (2 nd ed.) ( Chapter 10 – Evaluation, pp ). San Francisco, Jossey-Bass Pfeiffer. Web sites See the resources section in the lesson plan for hundreds of resources.

© E. Kowch For next session Contribute to this week’s discussion forum. Check out the Web Site Home Page for further instructions (core readings). Your online project production should have started by now - your professor will review anything that you like ahead of time… Your Personal DL portfolio web site should be under construction now. Adieu from Calgary..