Feedback from other instructors is a meaningful component of professional growth as an educator. Peer review or evaluation can be a part of the tenure and promotion process, a department- or program-specific expectation, or an instructor-initiated professional learning experience. Whatever the context, this type of peer feedback can be invaluable, providing instructors with actionable feedback and ideas for improving elements of a course and enhancing student learning.
While Ohio State departments and colleges have standard processes for peer review of classroom teaching, not all units have guidance in place for peer review of online classes.
Key Considerations and Differences
An online class, especially if it’s fully asynchronous, may look and feel very different from a typical face-to-face class. This presents both challenges and opportunities when conducting peer review.
A tool designed specifically for peer review of online courses, like the one included below, can help you navigate these differences.
Course Format
Becoming oriented to the format and setup of an online course requires time and effort. You need to explore the CarmenCanvas course and syllabus to get a sense of what’s happening (say, in a given week). You must figure out how to locate content across different technology platforms.
Your experience navigating the course will be valuable to share with the instructor as you are seeing it with fresh eyes, much like a student.
Instructor Role
In a classroom setting, many elements of instruction occur in one place at one time, including familiar teaching methods—lecture, for example, or facilitating a class activity or discussion. In an online class, these same elements may occur at different times and be mediated by various technologies and platforms.
-
The instructor’s informal reminders and helpful context for students—what might occur in a classroom at the very beginning or end of a class meeting—might occur in Carmen announcements or emails, perhaps a week or unit at a time
-
Instructional content, such as lectures and other course materials, may or may not have been visibly created by the instructor—in fact, sometimes these materials were created by an altogether different instructor
-
Class discussion and interactions with students may occur in CarmenZoom, asynchronous Carmen discussions, or other venues
When and How to Observe
Decisions around when and how to evaluate an online class are somewhat complex. For a fully asynchronous class, you may simply log in to Carmen, navigate to the module that corresponds to the current week, and view the range of instructor and student activity. In this case, you could even review the course after the fact.
If the class has synchronous meetings or activities, especially group meetings or discussions, you will need to navigate platforms beyond Carmen and plan to observe on a set day and time.
Areas for Feedback
A meaningful peer review of an online class aligns to the main categories of focus for a classroom observation, though the specific areas for feedback may look a bit different.
Peer review of an online class should address the following:
- Course setup
- Learning outcomes
- Subject matter expertise/content
- Instructor presence
- Assessment
Example Peer Review Rubric
The following rubric is being developed by faculty through a collaboration between the Office of Academic Affairs and the Office of Technology and Digital Innovation (OTDI). The rubric draws on common practices for peer review of teaching in any context, and it is aligned to Effective Practices for Online Teaching.
The rubric addresses the five categories above: course setup, learning outcomes, subject matter expertise/content, instructor presence, and assessment.
It has a three-level rating scale for each item (Exceeds Expectations, Meets Expectations, Needs Improvement), and open-ended comments are encouraged for each category.
Download Example Peer Review Rubric Template (PDF)
Customizing the rubric and making it accessible
Customize document content in Microsoft Word and export to PDF
- Open the provided DOCX template file and make any changes necessary to customize the content for your department or college.
- You will want to place any form field labels (such as Name, Email, etc.) in the DOCX file, but the form fields (where reviewers will enter their own information and ratings) will be added later using Adobe Acrobat.
- In the Home tab, be sure to use the headings in the Styles Pane to ensure document accessibility and consistent styling.
- In the Review tab, select Check Accessibility and follow the instructions provided by Word to resolve any accessibility issues.
- Select File, then Save As and replace or remove the word "template" in the file name.
- In Microsoft Word, from the Acrobat tab select Create PDF, then select Yes to generate the customized PDF file from your Word document. The PDF should open in Adobe Acrobat. If not, you will need to open the PDF file in Acrobat.
Edit PDF in Adobe Acrobat
- Once your customized PDF is open in Acrobat, select File, then Save As, then select the Reduce File Size checkbox and save the file by naming it.
- Select File, then Create and Create Form. The current file will be the default and select Start.
- You should now see the Prepare Form panel.
- Open the provided PDF template file in Adobe Acrobat. This provided PDF template will allow you to copy and paste form fields into your customized PDF instead of creating them from scratch.
- In the PDF template file, select View, Tools, Prepare Form, then Open to see the Prepare Form panel and form fields. If you do not see Prepare Form in the Tools menu, you may need to add Prepare Form as a shortcut from the Tools Center.
- From the Fields list in the Prepare Form right-hand panel, select all of the Page 1 fields and copy/paste them into the first page of your customized PDF. Depending on the extent to which you've customized your document, the form fields should paste in roughly the correct location. However, you may need to nudge some fields, move them to another page, or even rename the fields to refer to the associated content.
- Repeat step 4b for all pages.
- Select and remove and "undefined" fields. Note: If you close and then reopen the Prepare Form panel after this step is complete, these undefined fields may reappear.
- Ensure all field names are next to the appropriate content on each page.
Ensure your PDF is accessible
- In Adobe Acrobat, select File and Save. Your customized PDF file should now be around 1MB in size.
- Select View, Tools, Accessibility, then Open to display the Accessibility panel. If you do not see Accessibility in the Tools menu, you may need to add Accessibility as a shortcut from the Tools Center.
- Select Accessibility Check and select all check boxes except Attach report to document, then select Start Checking.
- You should now see the Accessibility Checker panel. Accessibility violations will be marked.
- Drill down to see what each violation is. The likely failures will be that the form fields are not tagged.
- To tag all of the form fields:
- From the Accessibility panel, select Reading Order to display the Reading Order tools.
- Uncheck Display like elements in a single block. You should now see that your cursor is a cross-hair.
- Use the cross-hairs to draw a box around the first form field in the document.
- Once the form field is highlighted, from the Reading Order tools select Form Field .
- Continue with all form fields in the entire document.
- In the Accessibility Checker panel, right click on the Forms section and select Re-check.
- Continue until no violations/failures remain. Visit the Adobe website for additional guidance on fixing accessibility issues.
- Ensure correct reading order by selecting View, then Show/Hide, then Navigation Panes, then Order to view the Order panel.
- In the Order panel, select the first item listed at the top of the first page.
- Then select the down arrow key to key through each item of each page to ensure it's in the proper reading order.
- If an item is not in the correct order, drag the item to the appropriate order in the list.
- Save the PDF.
- Finally, use JAWS (PC only), NVDA (PC only), or VoiceOver (Mac) to screen read the document to ensure that it is accessible. Do not use Acrobat's Read Aloud feature as it is not accurate.
- Use the tab key to navigate through the document to ensure the form fields are getting focus in the correct order.
If you have feedback or are interested in a consultation about establishing peer-review processes for online classes, please contact OTDI.