Reimagining the role of technology in end-of-course evaluation with Baylor University

In 2006, Baylor’s Institutional Research office was processing 50,000 to 60,000 sheets of paper course evaluations every semester. “We relied heavily on students to return the packets to our office. We were finding that we were getting an increasing number of either packets that were not returned at all or packets that were returned completely blank,” said Meaghann Wheelis, Senior Research and Planning Associate, Institutional Research and Testing at Baylor.

It took about six weeks to get course feedback back to faculty. “From the fall semester we were providing reports to faculty into the spring semester. And at that point it’s really too late to make any changes if they’re teaching the same course,” Wheelis said.

In addition, the results weren’t in a digestible format. Instructors were responsible for compiling the free-response items from the manual scan sheets. “It was up to the instructor to go through and try to compile that feedback. And you can imagine if you’re teaching a section or two of 100 students, that can take a while, especially if you’re already into the next semester,” Wheelis said.

Finally, because their process was paper-based, Baylor could only offer course evaluations to programs on its main campus. That excluded programs based in San Antonio, Dallas, Austin, and abroad.

Baylor needed a digital solution for course evaluation to reduce the administra-tive burden of processing paper forms, increase the speed of delivering feedback to faculty, and ensure students across all campuses had a voice in the process. In Fall 2011, they launched a homegrown system, but it “was pretty clunky and didn’t really do what we needed it to do,” Wheelis said. In Spring 2012, a few areas of the institution volunteered to pilot test digital course evaluations, so Baylor part-nered with EvaluationKIT by Watermark They moved to a fully digitized course evaluation process in Fall 2013.

Fast forward to today: “We now have ten different survey forms and six groups of targeted surveys that we add on at the end for specific programs. And that’s some-thing that we really couldn’t have done with the paper process,” Wheelis said.

Baylor has found several benefits to moving course evaluation online:

Fewer resources needed: From paper to office space to time spent, Baylor uses fewer resources—despite providing course evaluation to more programs and courses. “We had an entire room full of shelves with all of the different packets that we’ve been able to convert to an office,” Wheelis said. “Instead of having two student workers and staff in our office processing papers, we now have one person who spends about 75 percent of her time on course evaluations. That makes a huge difference in what we’re able to do.”

Strong response rates: Initially, faculty were concerned that there would be a drop in response rates. Their average rate for paper evaluations was around 73 percent. Online, it’s around 65 percent, and trending upwards.

More accurate data: Because Baylor can use real-time data syncs with the student information system to identify students who are enrolled in courses after the withdrawal date, all students who should receive a survey, do, and all students who shouldn’t, do not. “Faculty feel more confidence in the data. They can look at their report and know that these are my students, and this is what they have to say about the course,” Wheelis said.

Better reporting: “Our results format has improved substantially,” Wheelis said. As an example, she shared a report from the paper-based days that required instructors to flip back and forth to compare the mean and comparison group mean to the actual distribution of responses. The report excluded free responses, which instructors had to compile themselves.

“Now we have all of the information in one place. We can see the responses for the course with the comparison group. We can also see the distribution in a pretty chart. And at the end of the report, we have all of the free response items in one place. So it’s really easy to go through to pick out themes,” Wheelis said. “We also have the ability to combine across sections. So, if

“Instead of having two student workers and staff in our office processing papers, we now have one person who spends about 75 percent of her time on course evaluations. That makes a huge difference in what we’re able to do.”

Meaghann Wheelis, Senior Research and Planning Associate
Institutional Research and Testing
Baylor University

I’m teaching four sections of the same course, I can get one report for all of those sections. That has been really useful for faculty.”

Timely feedback: Baylor is now able to give results to instructors immediately after grades are due—the day of gradu-ation in Spring, and around December 15 for the Fall semester. “That gives faculty plenty of time to look at their reports and, if they’re teaching it again the next semester, to make any changes they might want to make for next semester,” Wheelis said.

More programs served: Baylor now offers course evaluation for study abroad and programs based off of the main campus, as well as online programs.

Deeper engagement: Faculty and students are more deeply engaged in course evaluation now that the process is online. Faculty monitor their response rates, and “some will pull this up on screen when they’re teaching and say, ‘I’m at 33 percent. I’d really like at least 90 percent for my students. Please give me your feedback.’ Students know that their feedback is being used and it also dispels the mentality of ‘well, everyone else will do it so I don’t need to,’” Wheelis said. Faculty can now ask for very specific feedback, such as rating the texts required for the course, or seeking input on the size of discussion groups. Questions like these have led directly— and quickly—to changes in courses.

Baylor continues to raise the profile of course evaluation on campus. “We have a marketing campaign with a link baylor.edu/course_evaluation—it’s everywhere. It’s on screen savers, and as we move forward in the online process, I think it’s just becoming part of the culture for students,” Wheelis said. In addition, Baylor is now working to incorporate course evaluation data into Digital Measures by Watermark, their faculty activity reporting system, to inform the promotion and tenure review process and provide a more holistic picture of faculty achievement.