As we determine Remote Assessments, one important option to consider is using open-book exams. This format reduces exam stresses on students and allows instructors to worry less about policing the exam process. This approach is common in many disciplines, utilizing more conceptual or applied questions that students cannot quickly lookup in a textbook.
For many faculty, the shift came when they realized that they have to look up information, formulas, and details in their work. But they also remember the broader concepts, remember enough points to go look up the specifics and synthesize or apply them in my given situation. (Who didn’t do that by the time you got to your dissertation or thesis?)
That’s the reality many of our students will face in the future, so how can we structure our assessments to measure their ability to do that synthesis and application, rather than test on discrete pieces of information they will forget a month from now anyway? If your discipline does require significant amounts of factual recall, open-book exams may not be for you. Still, they are an excellent option for many of us with learning outcomes that focus on higher-order thinking.
Ask higher-order thinking questions (Bloom's Taxonomy)
Closed-book exams can ask factual questions that demand students recall discrete pieces of information, but looking up those bits is simple in an open-book format. So, if you consider open-book exams, work toward higher-order thinking, as defined by Bloom’s Taxonomy.
In the most basic sense, you need to avoid factual questions (“Who invented X?” or “What is the third step in Y’s theory?”) and ask questions that require students to apply that information to unique new situations, such as in the examples below.
Focusing on conceptual problems is seemingly more straightforward in open-ended questions, as it asks students to apply and synthesize multiple pieces of knowledge to answer a question.
- Describe the next step in this process
- Define X within context Y
- Explain this situation through the lens of theory Z
- What would have happened if…? (could be historical situations, chemical reactions, public policy analyses, and more)
- Identify the error in a proof or computation. Or better, explain and fix it.
You can write multiple-choice questions that require more conceptual knowledge, moving students from the recall of information to applications of it.
Provide a mini case study and ask students questions related to it.
- “Based on the case study, which of the following is the most likely cause of the patient’s pain?” or
- “Based on the client’s needs above, which of the following is the preferred course of action?”
Describe a physical/chemical interaction and ask students to predict the outcome.
- “What would happen to the reaction if we added compound X?” or
- “Given the provided diagram of the collision, in which direction will the projectile head?”
- Even better if you can include an open-ended opportunity for them to explain their reasoning.
Ask students to identify an example of a concept or principle.
- “Which is an example of a norm-referenced interpretation?” or
- “Which of the following best exemplifies the principle of synchronicity?”
Use charts that students have to explain or interpret.
Include in a “what if” question to push their analysis and prediction skills.
For a good overview of question writing that includes questions testing higher-order thinking, see How to Write Better Tests from Indiana University-Bloomington Evaluation Services and Testing (BEST).
STEM Essentials for open-book exams
STEM and other quantitative courses face a particular challenge in creating practical online exams because it appears to be easier to cheat, and so many questions are computational. Joe Guadagni, Rutgers University, has compiled this advice from the Mathematics department:
- Ask more conceptual questions (e.g., “what is the next step in this problem?”, “state the definition of…”, “explain why this hypothesis in the theorem is necessary”).
- Ask students to identify an error in a proof or computation (this is particularly effective since students will be unable to Google it).
- Eliminate multiple-choice and fill-in questions in favor of show-all-work questions where students have to scan and upload their work.
- If using a textbook’s problems, change not only the numbers but also the names (e.g., Chris to José), and the scenario (e.g., pulling a boat into letting a kite string out). This one-step helps because popular textbooks will probably have many of their problems already solved online somewhere (e.g., Chegg).
- Use letters and variables in place of specific numbers.
- When randomizing the exam, don’t just randomize numbers. Also, randomize discrete parts of the problem. For instance, one version might have a question like “maximize the volume of the box given its surface area,” whereas another version might have “minimize the surface area of a box given its volume.” (The numbers can even be the same for the two versions.)
- Avoid questions that consist of only simple computations. For example, instead of “calculate this integral,” present students with some application in which they also have to set up a proper integral.
- “Write an integral expression that is equal to the probability that…” or “Write a triple integral which is equal to the mass of the region” are good alternatives.
- Some online calculators will solve not only many computational problems but also give step-by-step solutions. Adding more words and applications to a problem makes it more difficult to cheat and tests the real learning goal: do students know how to apply basic principles? (Ultimately, anyone can use a calculator, but only if you know what you want to calculate.)
Tips for open-book exams
- Instead of multiple-choice or fill-in questions, have students show their work by scanning/uploading it to the exam/assessment.
- Create a video submission assignment. You can use the Studio in Canvas tool to provide a demonstration for students to address or record their answers, which could be their demonstration.
- If you use a textbook publisher’s problems, be sure to change names, numbers, and the scenario, since popular textbooks may have many of their assessments already solved online.
- Instead of asking students to do a simple calculation, ask them a more complex problem that forces them to figure out what they need to calculate, maybe sorting through other information that isn’t necessary or relevant. Present them a question and require them to figure out how to calculate the solution. That way, you make them choose decisions surrounding the calculation, not just doing the computation (or finding an online calculator to do it for them).
- Use multilevel thinking, using questions that include phrases like “most appropriate” or “most important.” This approach forces students to make judgments and demonstrate a fuller understanding of concepts and the subtleties between different answers’ correctness levels.
- Work to find the right amount of time to give students complete the exam–enough time to complete it with minimal stress, but not so much that they will obsess over their answers. Since students’ schedules may be disrupted during this crisis, provide the flexibility of when they can take the exam.
Accommodations for quizzes
Canvas makes it easy to incorporate extended time accommodations on timed assignments, by adding students with such accommodations to a group within Canvas and specifying exceptions in the assessment settings (See the Student Accommodations in Canvas guide).
A meaningful and purposeful assessment paired with constructive feedback can help learners understand and address their achievement gaps and organize future learning. Canvas quizzes provide a way for instructors to deliver helpful feedback, assess and accommodate learning. View the Low and High-Stakes Quizzes in Canvas web guide.
Use Canvas Assignments for essay exams (recommend File Upload option). Do not ask multiple graders to enter grades at the same time in SpeedGrader, or they will overwrite each other’s work; download all files and have one person submit the scores in Gradebook.
Replicate bubble sheet tests
Use the multiple-choice exam in Canvas. Create a Canvas quiz with multiple versions of the same question to increase exam integrity. For previously scheduled ISU Online Testing Center exams, consider:
Center for Excellence in Learning and Teaching (CELT), Iowa State University’s Open Book Exam, used under BY-NC-SA, retrieved on April 17, 2020, is derivative of the following resources: Indiana University-Bloomington’s Tips for Creating Open Book Exams post (https://blogs.iu.edu/citl/2020/03/27/open-book-exams/) along with Rutger University’s Remote Exams and Assessments website (https://coronavirus.rutgers.edu/resources-for-faculty/remote-exams-and-assessments/). This work, Remote Assessment, is by Center for Excellence in Learning and Teaching (CELT), Iowa State University is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.