Evaluating Open Educational Resource (OER) Objects Rubric II: Quality of Explanation of Subject Matter CC BYCC BY Achieve 2013

2 Open : Containing an open license - no restrictions on remixing or reusing [Having no enclosing or confining barrier; not restricted to a particular group or category of participants] Open Educational Resources: [With Webster’s Definitions] O E R Educational: Used for teaching and learning [Pertaining to the action or process of educating or being educated] Resource: Object used to support an effort or task [ A source of support or aid, especially one that can be readily drawn upon when needed]

3 What are OER? OER are teaching, learning, and research resources that contain an open license. They provide extraordinary opportunities for educators to freely share knowledge and resources. They offer great potential for instructional innovation and networks for sharing best practices. There are millions of OER objects available online. Open Educational Resources (OER)

4 OER objects may include (but are not limited to): Images Applets Games Worksheets Lesson plans Original-source texts Assessments Units Textbooks Teacher/learner support materials o NOTE: Any smaller component of a more complex object, that can exist as a stand-alone, also qualifies as an “object” (e.g. a unit in a textbook, a lesson in a unit, or an activity in a lesson). Open Educational Resources (OER)

5 The Achieve OER Evaluation Rubrics How and why were the Achieve OER Rubrics created? Educators using the vast system of sharing need a method for filtering OER to meet the needs of their students. The purpose of the rubrics is to provide a structure for evaluating an online resource in a systematic, purposeful and comprehensive way. There are two ways of approaching the evaluation of a resource: o Holistically evaluating the object with a single rating, (similar to the star-rating systems used on Netflix, Yelp and Amazon) o Separately evaluating each of the components of quality, as with the Achieve OER Rubrics. The rubric criteria are based on Achieve protocols used to assist states in alignment and quality review studies. The Achieve OER Evaluation Tool, hosting the rubrics on OERCommons.org, allows a user to identify, evaluate and sort objects based on the specified essential elements of quality.

6 How do the Achieve OER Rubrics work? The rubrics represent an evaluation system for objects found within Open Educational Resources. They are hosted as an online evaluation tool on the repository, OERCommons.org, but can also be used independently. They are applied to any content area. o NOTE: At this stage only Common Core Standards for English Language Arts/Literacy and Mathematics are available in the Achieve OER Evaluation Tool. Content standards for History, Social Studies, Sciences, and technical subjects are not yet available. Each rubric is applied independently to an object. They rate the potential, not the actual, effectiveness of an object. They use a five-point scoring system to describe levels of quality. Applying the Achieve OER Rubrics

7 This presentation will focus on Rubric II: Rubric I. Degree of Alignment to Standards Rubric II. Quality of Explanation of the Subject Matter Rubric III. Utility of Materials Designed to Support Teaching Rubric IV. Quality of Assessment Rubric V. Quality of Technological Interactivity Rubric VI. Quality of Instructional Tasks and Practice Exercises Rubric VII. Opportunities for Deeper Learning o NOTE: Sometimes one or more of the rubrics may not be relevant to a particular object. In those cases, a rating of N/A is appropriate. The Achieve OER Evaluation Tool Rubrics

8 Rubric II: Quality of Explanation of Subject Matter How and Why is Rubric II Applied? Rubric II is used to rate how well the subject matter is explained or otherwise revealed in the resource. Subject explanation could be aimed at either the teacher or the student. This rubric could apply to objects used for a whole class, small groups, or an individual student (e.g. self-tutoring object). Sometimes the subject matter explanation is included in teacher support material and is intended to help a teacher learn or be refreshed on a specific topic (the teacher becomes the learner). Do not use Rubric II to rate the quality of the explanation of how to present the lesson (planning instructions) for an object. Use an N/A-rating only if there is no explanation of subject matter included in the object or for objects not designed to explain subject matter (e.g. a formula sheet or a map), even if it might be possible to use the object to explain a subject.

9 Rubric II: Quality of Explanation of Subject Matter 3: An object is rated superior only if all of the following are true: The object effectively provides comprehensive information so that the target audience should be able to understand the subject matter. The object connects the subject matter with important associated concepts. The object does not need to be augmented with additional explanation or materials. The main ideas of the subject matter addressed in the object are clearly identified for the learner. For example a lesson on multi-digit addition provides a clear explanation of multiple strategies and makes connections with place value, rather than simply showing procedure. Or a lesson clearly explains why and how to analyze how an author develops ideas across extended text would make connections among the various developmental steps and the various purposes the author has for the text. These examples would be considered superior for explanation of subject matter.

10 2: An object is rated strong if it explains the subject matter in a way that makes skills, procedures, concepts, and/or information understandable. It falls short of superior in that it does not make the connections among important associated concepts within the content. For example if an object’s focus is to compare decimals to hundredths, it would be considered strong if it thoroughly explains how to compare 0.25 and 0.6 by comparing digits in the same place value position. However it may not make the connection to number line placement or conversion of the two decimals to fractions with a common denominator and then comparing the numerators. Rubric II: Quality of Explanation of Subject Matter

11 1: An object is rated limited if it explains the subject matter correctly but in a limited way. This is a cursory treatment of the content that is not developed enough to serve someone attempting to learn the content for the first time. The explanations are not thorough enough to serve as more than a review for most learners. For example an object addressing comparing the mean to the median and mode for a set of data provides procedure but does not explain how the measures of center are used or why one is preferable to another in a given situation. This is of limited quality for explanation of subject matter. Rubric II: Quality of Explanation of Subject Matter

12 0: An object is rated very weak or no value if its subject matter explanations are confusing or contain errors. There is little likelihood that this object will contribute to understanding. For example an object designed to teach synthesis of information gathered from narrative texts, data collections, and simulations found in various science publications, is missing a clear explanation of the process and elements of synthesis. It is very weak for quality of explanation of subject matter. Rubric II: Quality of Explanation of Subject Matter

13 N/A: This rubric is not applicable (N/A) for an object that is not designed to explain subject matter. It may be possible to apply the object in some way that aids a learner’s understanding, but that is beyond any obvious or described purpose of the object. For example Rubric II would be not applicable for a sheet of mathematical formulae, a set of raw data, or a map. Rubric II: Quality of Explanation of Subject Matter

Using the Achieve OER Evaluation Tool On OERCommons.org Rubric II

o NOTE: Specific instructions for Using the OERCommons Website to search for and evaluate objects, using all of the rubrics, can be found in another slide presentation of this training package. 15 OERCommons.orgOERCommons.org hosts the rubrics as the Achieve OER Evaluation Tool Achieve OER Evaluation Tool on OERCommons.org

16 Using OERCommons.org to evaluate an object for its quality of the explanation of the subject matter:OERCommons.org After you have aligned and/or rated alignment using Rubric I, you are ready to apply Rubric II. Begin by clicking on at the end of Rubric I or on the title line for Rubric II, but remember that your ratings for Rubric I will need to be saved. Rubric II: Quality of Explanation of Subject Matter Achieve OER Evaluation Tool

17 To view a video about Rubric II click here… Rubric II: Quality of Explanation of Subject Matter Achieve OER Evaluation Tool

18 Hover over the rating numbers to see an abbreviated version of the rating descriptors. Review the object with your focus on how well it explains the subject matter and select the appropriate rating. Rubric II: Quality of Explanation of Subject Matter Achieve OER Evaluation Tool

19 To download a full text version of the rubrics click here: Rubric II: Quality of Explanation of Subject Matter Achieve OER Evaluation Tool

20 Rubric II: Quality of Explanation of Subject Matter Achieve OER Evaluation Tool Across the bottom of Rubric II you see options to: Leave a [+ Comment], Clear rating, [Save & Go the Next Rubric] (You can also click on any rubric title to leave Rubric II, but your ratings will not be saved.)

21 Now you are ready to rate the object using the Achieve OER Evaluation Tool’s other rubrics. Rubrics can be used in any order and can be selected by simply clicking on the rubric title. Remember to click after each rubric to ensure that your work is saved. You can tell if your work was saved by the symbol at the beginning of each rubric title: indicates not yet rated or rating was not saved indicates your rating has been saved Achieve OER Evaluation Tool on OERCommons.org: Rubric II

Slides developed by Achieve: th Street, NW / Suite 510 Washington, DC 20036