Source: Designing Meaningful and Measurable Outcomes: A First Step in Backwards Design
I’ve sat on the Curriculum Committee at two different higher education institutions. I’ve also participated in college assessment committees and accreditation committees at both the school level and institutional level. I’ve designed courses and entire programs from scratch and have revised courses and programs to meet either accreditation or institutional needs. One activity all these endeavors has in common is the development or re-development of meaningful and measurable outcomes.
Unfortunately, what I’ve discovered is that most faculty are not well-versed in curriculum design, and therefore unable to have the forethought to consider what they want their learners to know and be able to do upon completion of their course or the program as a whole. Outcomes, when considered, become like the paper tail in the game pin the tail on the donkey. They are an afterthought, and one that is attached blindly to a course or program. When working with faculty on their course or program development, I utilize the practice of backwards design in which you start with the end in mind. Outcomes are the end we have in mind.
Backwards Design
The focus of Backwards Design (or Understanding by Design) is that of student learning and understanding. Wiggins and McTighe (1998) contend that faculty typically design their courses around a set of activities and content first (forward design), and then try to connect those activities to learning goals of their course. Often this is a forced fit, artificial at best and completing missing the mark at worst. In order for instruction to be both meaningful and measurable, it is important that instructors begin their course/curriculum design process with outlining the course level student learning outcomes. The outcomes are the final destination on the journey students in the course will take.
For new faculty and those inexperienced with course design that focuses on student learning outcomes, the idea of creating measurable, yet meaningful, outcomes may seem daunting and even unnecessary. There may be resistance to this endeavor as it feels like unwanted oversight or an infringement on faculty’s academic freedom.
In this era of accountability, and a public that questions the legitimacy of higher education, we in the academe must provide the rationale for every credit-bearing course in the catalog. Meaningful and measurable outcomes help provide credible evidence of student learning; well-designed outcomes and objectives provide the roadmap to student success.
What makes an outcome or objective meaningful and measurable?
Consider these characteristics of Well-Designed Learning Objectives:
- Objectives should identify a learning outcome.
- Objectives should be consistent with course goals. When objectives and goals are not consistent, two avenues of approach are available: change or eliminate the objective, or change the course goal.
- Objectives should be precise. It is sometimes difficult to strike a balance between too much and too little precision in an objective. There is a fine line between choosing objectives that reflect an important and meaningful outcome of instruction, objectives that trivialize information into isolated facts, and objectives that are extremely vague. Remember, the purpose of an objective is to give all learners the same understanding of the desired instructional outcome (Smith, 2012).
One effective tool to use during the design phase of creating course outcomes or module level objectives is Bloom’s Revised Taxonomy Verbs. Bloom’s Revised Taxonomy outlines six levels of learning. Moving up the list of levels allows for deeper learning to occur, however, it is important to include objectives that meet multiple levels across the taxonomy. Each level of learning can be described with verbs when developing an objective. For example, when you write an objective aimed at the Remember level, use a verb like list or define.
Another way to consider the level of learning is by the level of the course. Is your course a 100 level course or a 400 level course? Is it a survey course or a skill application course? Where is it situated in the scope and sequence of the program? Does it have prerequisites? Is it a culminating or capstone experience? Design outcomes based on the level of the course using verbs that align with the level of learning required. The following table 1 illustrates examples of how to create this alignment.
Table 1: Outcomes and Levels of Learning
| Level of Learning |
Objective Action |
| (Lowest) Remember: Recall facts and knowledge from long-term memory |
List, Recognize, Recall, Identify, Define, Describe,State |
| Understand: Assign meaning to given information |
Summarize, Classify, Clarify, Predict, Interpret, Infer, Compare, Explain, Discuss, Distinguish |
| Apply: Use learned information in a given situation |
Respond, Provide, Carry Out, Use, Execute, Implement, Apply, Demonstrate, Prepare, Conduct, Utilize, Show |
| Analyze: Break down the material and determine how the pieces relate to one another or the larger picture of the information |
Select, Differentiate, Integrate, Deconstruct, Organize, Attribute, Analyze, Develop, Design, Illustrate, Defend |
| Evaluate: Judge the results of something based on the information learned |
Check, Determine, Judge, Reflect, Critique, Conclude, Compare, Interpret, Justify, Test, Verify |
| (Highest) Create: Put learned information together to generate new structures/patterns |
Generate, Assemble, Design, Create, Plan, Produce, Integrate, Revise, Edit, Transform, Organize, Reorganize
|
If you already have a set of program or course outcomes and realize that they may be in need of an ‘extreme makeover,’ the best place to begin the renovation is with the verb. Consider the following outcome from a 300 level history course and see how you can apply the wisdom of Bloom’s Taxonomy to make it more meaningful and measurable.
Original Outcome: “Understand different historical time periods from different geographic areas.”
Is this outcome precise enough to outline what students need to know and be able to do within the context of this course? Is it measurable? Take a look at the verb used – understand. What does understand look like? How do you measure understand? Since this is a 300 level course, the verb used should align with that level of learning.
Extreme Makeover of Outcome: “Learners will compare and contrast different processes, modes of thought, and modes of expression from different historical time periods and in different geographic areas through an applied reflection.”
Use of compare and contrast elevates the outcomes from the lowest level of learning to a moderate level of learning. In addition, it is precise and reflects the context of the course. Finally, a method of assessment is included through the applied reflection which in turn makes this outcome measurable.
For better or worse, amid rising tuition costs, and the value of a college education being questioned, students and their families are increasingly turning their gaze towards outcomes. The question on everyone’s lips is, “Is college worth it?” When you design meaningful and measurable program and course outcomes, you’ve answered this question with a resounding, “Yes, it is!”
Vicki Caruana is an associate professor of education at Mount Saint Mary College in New York. Caruana’s PhD is in Curriculum & Instruction, and she has mentored faculty at several institutions to create meaningful learning experiences in both traditional and online contexts. She has authored more than 30 books for both teachers and parents, including the best-selling Apples & Chalkdust: Inspirational Stories and Encouragement for Teachers.
References
Smith, T. (2012, July 2). Writing measurable learning objectives. Retrieved January 25, 2019, from https://teachonline.asu.edu/2012/07/writing-measurable-learning-objectives/
Wiggins, G., & McTighe, J. (1998). Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.
One of the questions CELT receives on a regular basis is, “How can I engage my students in the course content?” Or, “How do I help my students engage with course content more meaningfully?”
