Managing Student Feedback for a Large Institution with Swinburne University

Prior to 2015, Swinburne relied on paper surveys for student evaluations of teaching (SETs). They were handed out, completed, and collected in class, then scanned into the system, which was “obviously quite time consuming,” noted Pam Jones, coordinator of survey and credit arrangements in the office of Education & Quality Services at Swinburne. The one advantage of their paper-based process was an incredibly high response rate. Swinburne needed to move its unit evaluation process online while ensuring a high response rate.

Swinburne has a wide range of programs, including vocational training, traditional undergraduate and graduate studies, and professional development programs. When Swinburne adopted EvaluationKIT, they first began using it for unit evaluations in the traditional undergraduate and graduate student cohorts. Today, all 60,000 Swinburne students receive surveys through EvaluationKIT.

Over five years of use, Swinburne has identified 5 key benefits of using EvaluationKIT:

LMS integration:

“When we first transitioned from paper surveys to EvaluationKIT, there was a slight drop in our response rates because we were moving online,” Jones said. To improve on those initial response rates, Swinburne took advantage of EvaluationKIT’s connection to their learning management system to use prompts and pop-ups within the system to encourage survey completion. “There was a massive increase in response rates. We moved from something like a 20% response rate to a 48% response rate,” Jones said.

When Swinburne began using the system, they used the Blackboard LMS, but in 2018 they began a transition to Canvas. “We quickly spoke to Watermark about it, and it was one of the easiest things. It was amazing because nothing changed from our users’ perspective and our students’ perspective. It was seamless — it just slotted straight in,” Jones said.

Ease of use for students:

“EvaluationKIT allows respondents to complete surveys quickly and easily with their preferred device, usually their mobile phone. Our analytics show that nearly 50% of students use this method to complete their survey.”

Pam Jones, Coordinator in the Office of Education & Quality Services, Swinburne University of Technology

In addition, prompts within the LMS encourage students to complete surveys. “When you tie that in with the learning management system, you capture everybody. They can’t get away from that pop-up without completing the survey, which is brilliant. You get the responses because it’s all in the system where they’re already working,” Jones said. She believes it’s helpful to students that surveys are available within the LMS, allowing students to complete the surveys in their own time.

Meaningful data:

Increased response rates mean that Swinburne gets more, and therefore better, data to inform improvements. “Obviously, the higher the response rate, the more valid that data is that you get in from students,” Jones said. “Previously we had instances where course coordinators would say, ‘Do you really expect me to act on 15% of people’s opinion about this course?’” Now, with a nearly 50% response rate, “They say, okay, well, I’ve got to sit up and take notice,” Jones said. “So that’s been a huge success for us.”

Rich, rapid reporting:

Swinburne conducts multiple surveys each term, beginning with a check-in conducted at week four. “This survey is used to get initial thoughts on the unit and allow the conveners to make changes based on the feedback they receive,” Jones said. They leverage EvaluationKIT’s rapid reporting capabilities to get feedback to faculty immediately. “That survey closes on a Sunday at midnight, and at one minute past midnight, those reports are available for faculty to self-serve. They get a PDF with charts to break down the responses and this shows the early indication of student satisfaction. It’s very easy to interpret,” Jones said.

This was particularly useful in early 2020, when the check-in survey coincided with the disruptions caused by COVID-19. “We Technologydecided it was best to adapt our survey to the current situation and acknowledge that these are challenging times,” Jones said. “The questions in the adjusted survey focus on gathering feedback on how students were finding that online experience so far, and if there was any other support we could provide or improvements that we could make to the existing support being offered.”

Data from EvaluationKIT also informs the academic promotion process. “Faculty have to have good student satisfaction scores, which hopefully therefore leads to better quality in our teaching provided,” Jones said. In addition, Jones has also set up a dashboard for deans, providing automatic access to survey results for each dean’s college.

Superior service and support:

Jones values her regular check-ins with her client success manager, which have opened her eyes to capabilities in EvaluationKIT that she hadn’t considered before. “After we’ve used something for five years, you think you know it back to front, and then all of a sudden it’s like, ‘Oh, I didn’t realize we could do that’ because we’d never looked there,” Jones said.

Swinburne has recently begun to explore other evaluations, including single-unit surveys rather than just course surveys, and has also tested live stream surveys through EvaluationKIT with faculty from the schools of science, engineering, and technology.

Jones is now testing using branching within surveys and display settings, and running surveys outside of the LMS with an anonymous link that can be shared with any respondent. “We’re testing these things because we’re picking up a lot of new skills and seeing new possibilities,” Jones said.