Evaluating OER
Learning Objectives
At the end of this chapter, you should be able to:
- Describe the aspects of an OER that should be assessed before use.
- Explain why it is necessary to assess an OER’s adaptability.
You should always evaluate the resources you implement in your classroom, no matter where they come from. Some of the evaluation criteria listed below are universal, and others are specific to OER. The criteria for evaluation has been adapted from Achieve’s Rubrics for Evaluating OER Objects, Under a CC BY 3.0 License.
Alignment to Course Objectives
In order to successfully search for OER, you must have a clear and detailed list of course objectives that you are hoping to meet with your OER. While you may have to do some remixing of different OER in order to meet specific objectives, it is important to consider how closely found OER actually align with your course content.
Considerations
- Does the OER comprehensively align course content and learning objectives?
- Are the performance expectations from students focussed on in the OER? Many topics may be covered within the OER, but it is important that the specific learning outcomes are the emphasized content within the OER.
Explanation of Subject Matter
Beyond simply meeting course objectives, effective learning resources must comprehensively and clearly explain course material at a reading level appropriate to students. The subject matter should be addressed through multiple means (i.e. text, images, videos, audio), and should also be revised for culturally sensitive and age appropriate content. Instructors should also consider if the OER is comprehensive enough to stand alone, or whether it must be augmented with additional materials.
Considerations
- Will your intended audience be able to understand subject matter presented in the OER?
- Is the content culturally appropriate?
- Are there any factual, grammatical, or typographical errors?
- Are connections provided between various course outcomes?
- Is supplementary material required for students to gain a full understanding?
Utility for Instruction
An ideal OER should be easy and straightforward to use in multiple contexts. How an OER is meant to be implemented is an important factor to think about. Are the OER you are looking for meant for in class use, or outside of class for projects, supplementary understanding, or something else? A comprehensive OER may include instructions on intended use for students and instructors alike. This category also encompasses any software considerations that may need to be made.
Considerations
- Are instructions for use provided?
- Does the licensing allow for reuse and remixing?
- Is any specific software required for use or revision of the OER?
Quality of Assessment
This topic may apply to OER assessments that are intended to assess student understanding before, during, or after learning course content. However, considerations of assessment, even if instructor-made and not OER, should also be taken into account when bringing other OER into the course. Using new course material may require updating former assessments to better align with the OER that has been brought in. Consider comparing your current assessments to possible OER and see if the two intuitively connect.
Considerations
- Is the assessment closely aligned to the content?
- Does the assessment appropriately weigh the course content and objectives?
- Does the structure of assessment appropriately weigh the proficiency of learning?
Quality of Technological Interactivity
This criteria pertains specifically to OER that have a digital, interactive component. Most OER are available in digital form, however this criteria is not applicable unless there is a specific interactive component to the OER. Interactive learning resources should actively encourage learning, and it is important to consider how the resource may help students gain a deeper understanding of course materials.
Considerations
- Does the OER allow for individualized learning and control?
- Is the OER well designed? Does it function as intended on it’s platform?
- Does the OER functionality invite student use or encourage student learning?
Quality of Instructional and Practice Exercises
Many OER, especially textbooks, may include sections where students are encourage to apply what they’ve learned through practice questions and exercises. This criteria may also apply to open lesson plans, or any material intended for instructor use to improve student learning and engagement.
Considerations
- Does the OER provide students with more than enough examples to master a concept?
- Are one or two examples of complex course ideas presented to enrich understanding?
- Do the exercises include an answer key or scoring rubric for student reference?
- Are a variety of exercises available? This will allow students to apply course knowledge if multiple different contexts.
Opportunities for Deeper Learning
This criteria should be applied to any resource meant enrich knowledge and show proof of deeper learning. Skills of deeper learning can be applied across all subject areas, and may include critical thinking, effective communication and collaboration, learning how to learn, abstract reasoning, critique and construction of effective arguments, real world application, and the construction and use of models.
Considerations
- Are at least three deeper learning skills required for use of the resource?
- Does the OER offer a range of cognitive demand appropriate and supportive of the material?
- Are appropriate scaffolding and direction provided?
Accessibility
No matter what resources you plan to adopt, accessibility should always be a part of your assessment process. Many published-provided homework products are not accessible to students and can cause unexpected issues. Similarly, some OER may not be optimized for students with visual or auditory impairments. See our Accessibility & Usability chapter for more details.
Considerations
- Is the content accessible to students with disabilities through the compatibility of third-party reading applications?
- If you are using Web resources, does each image have alt text that can be read? Do videos have accurate closed-captioning?
- Are students able to access the materials in a quick, non-restrictive manner?
For a condensed rubric of this criteria, see Rubrics for Evaluating Open Education Resources Objects by Rodney Birch of George Fox University.
This chapter covers content assessment, or how an instructor can assess OER for quality and fit in their class. For a better understanding of assessing course outcomes from using OER, see our Assessing Course Outcomes chapter.
