It almost goes without saying that collecting the right data is essential for effective assessments in higher education. Assessment at the course level usually takes one of two forms: direct or indirect.
Both types of measurement are essential for a successful assessment strategy. Finding effective ways to combine the two will help you gather the information you need to continuously improve teaching and learning.
Although the aim of both direct and indirect measures is to show student learning, each type approaches this task in different ways.
Direct measurements provide hard evidence of student learning and take place throughout a course or program. These assessments require students to demonstrate their proficiency in the specific skills, knowledge, or behaviors they are learning in a course.
Some examples include:
Faculty typically score these artifacts with rubrics to ensure fairness and standardize learning across sections. Over time, direct measurements help you gain greater insight into what students know or can do. You can then use these measurements to build a body of evidence of student learning.
It’s essential to remember that your quantitative data isn’t the only piece in the puzzle. You can learn a lot from looking at test scores, but you’ll miss the essential context behind those scores that can help you identify specific areas for improvement.
Indirect measurements assess each student’s opinions about the knowledge or skills they encounter in class, which helps supplement data from direct measures of learning. An indirect assessment usually takes place at a single point in time rather than throughout a course. Some examples include:
While this information can be useful for understanding the hard data you collect, it’s important to remember that it won’t give you insight into whether students are making measurable progress in their courses. That’s when you’ll need quantifiable data from direct measurements.
Ultimately, the difference between direct and indirect methods of measurement depends on what you are attempting to measure.
To determine whether a measurement is direct or indirect, consider whether the assessment is measuring tangible proof that an outcome is achieved.
If your assessment provides a hard number indicating student knowledge or skill proficiency, that’s a direct measurement. If you are asking for qualitative feedback on a topic, you’re making an indirect measurement.
Although the two go hand in hand, each method has different pros and cons. You have to keep in mind what you want to know before conducting an assessment, as your goals will directly influence which type of measurement you should use.
Here’s a quick breakdown of how you can use each assessment type most effectively:
The main advantage of direct vs. indirect assessment measures is that direct measurements more clearly connect to specific learning outcomes. Therefore, they provide the clearest insight into a student’s level of mastery.
A term paper, for example, will reveal whether students have mastered the course material enough to apply it to a specific theoretical situation. Pop quizzes throughout the semester can show which concepts students struggle with early on, so instructors can adjust their lesson plans to focus on the most challenging elements.
On its own, a direct assessment measure lacks the proper context to explain why a student received the results they did. A whole class of students may perform poorly on the midterm exam, but without qualitative feedback showing where the class is stuck, the instructor may have trouble addressing the problem effectively before the final.
The main advantage of indirect measures is that they provide the necessary context for a more holistic view of student learning. For example, while a student may have mastered a concept according to their exam score, they may not feel confident in their understanding. A self-reflection essay can reveal where that student may need additional help and provide valuable insights into the instructor’s teaching effectiveness.
The problem with indirect assessment, though, is that it doesn’t provide enough evidence on its own to demonstrate what a student can do. You need direct measures of assessment to understand how well a student can apply what they’ve learned in class.
Taking a direct measurement requires you to pull data from actual student behavior or work. This data can come from many sources, including:
Many of these assessments are embedded assessments, which students typically complete as part of a course or program, making it easy for faculty to collect data from the entire class. However, students usually undergo larger assessments like capstone projects outside of their usual class schedules. Having a robust analytics system in place is essential for collecting and making sense of all that data.
Indirect assessments require additional planning and time. You can take indirect assessment data from:
The main challenge of collecting indirect assessment data is getting students, alumni, and other subjects to take the extra time to submit the assessments. Course evaluations, for example, have notoriously low response rates when instructors ask students to complete them outside of class. Making the process easy and incentivizing evaluations can help you bring in more responses.
The short answer is yes, you do need both types of measurement to create an effective assessment strategy. A 2:1 ratio of direct to indirect measurements helps ensure you’re actually meeting your desired learning outcomes — and accreditors often require you to use a combination of the two anyway.
Here’s an example. Let’s say you conduct a course evaluation and find that students overwhelmingly feel that they understand the material, which is an indirect measurement of student learning. However, the direct evidence from recent exam results contradicts this evaluation. The instructor can take this information and use it to reexamine their course or program content to determine what might have gone wrong.
Understanding the differences between direct and indirect measurement is essential for collecting assessment data that helps you accomplish your goals. In order to get the most valuable insights from that assessment data, you need a user-friendly software solution that works with your existing tech stack. That’s where we can help.
Watermark’s suite of fully integrated solutions makes it simple to gather and analyze both direct and indirect measurements. Watermark Planning & Self-Study, our planning and accreditation prep platform, integrates with your LMS, student information system, and other Watermark solutions to create a seamless flow of data across your institution’s tech stack. Easily import direct measurement data from any electronic source and get started right away.
Simplified survey administration with Watermark Course Evaluations & Surveys lets you easily capture indirect measurements of student learning. Customizable reports allow you to generate and send actionable insights to all your relevant stakeholders, while rich visualizations and interactive dashboards help you communicate the full story behind those insights.
The tool makes it simple to gather real-time student feedback and create customizable reports that surface valuable insights. Rich visualizations and interactive dashboards help all relevant stakeholders understand the data so they can take action, and integrations with other Watermark solutions create a continuous flow of data across your ecosystem.
Want to learn more? Download our eBook, Student assessment: Go beyond test scores.
To see our assessment and accreditation prep solutions in action, request a personalized demo here.
