There’s been a lot of conversation across higher ed about the value of assessment data collected in an academic term defined by disruption on a scale not seen in living memory. In an “asterisk semester” filled with a lot of anxiety and extra work for everyone, does it make sense to continue assessment as usual? We brought together a panel of educators responsible for assessment at their institutions to discuss this in a conversation moderated by Natasha Jankowski, the Executive Director of the National Institute of Learning Outcomes Assessment (NILOA).
Panelists Bliss Adkinson, Associate Director for Academic Affairs and SACSCOC Liaison, University of Northern Alabama; Tracey Floto, Executive Director of Assessment and Accreditation, Trine University; and Susan Brooks, Assistant Professor of Teaching in Education and Team Leader for the Intervention Specialist Program, University of Findlay, as well as a lively group of attendees shared their experiences, insights, and planned approaches to assessment this term. This thoughtful (and hopeful!) discussion touched on the many issues experienced in the abrupt shift to emergency remote teaching, and shared a new mantra — keep calm and keep collecting — as well as a vision for assessment in the time of COVID.
The unprecedented nature and effects of the pandemic are undisputed. That being the case, attendee Michael raised an obvious issue. “When working with statistical data, any numbers that fall well outside of the norms are typically discarded because they would ‘taint’ the data,” attendee Michael noted. “Why bother with data that will not fit into your normal assessment statistics? As important as assessment data is, what could you hope to prove in a “one-off” (we hope!) situation?”
While the panel maintained that there may be some statistically relevant data to come from assessment of the asterisk semester, they agreed there was a more pressing need for this term’s assessment data: real-time learning to inform planning for the next semester. That’s the approach Trine University is taking.
“One of the big priorities coming from this is discovering what worked well and what didn’t work so well, and how can we use that information going forward. We’re going to have a new normal and we don’t know exactly what that’s going to look like,” Floto said. “We want to use that information to create what we’re thinking are going to be hybrid classes for the fall.” Trine plans to consider this term’s learning outcomes as they plan to ensure that students will get the information for the learning outcomes to be successful in their fall courses.
Attendee Julie raised a key question: “How do we balance the need for consistency (because people hate when assessment changes all the time) with the need for adaptation? Especially when we don’t know when we might be able to go back to our pre-COVID processes.”
All of the institutions represented on the panel had to switch from in-person instruction to emergency remote instruction over a weekend or overnight. With so much chaos introduced into learning — and the lives of students and faculty — University of Findlay has looked for ways to provide order and stability. “We wanted to keep data collection going, to continue to do that because learning continues. Our motto is ‘keep calm and keep collecting,’” Brooks said. The Findlay team chose to make adjustments to assessment so they could continue to document and report on student learning to inform instruction going forward.
Adkinson concurred. “People find comfort in consistency. The more we can communicate the support to faculty, and continue to offer resources that help support them, the better!” she said. “We have encouraged faculty to keep with their assessment strategies as much as possible during this time. In cases where they are not able, we know this will be a caveat in our reporting.”
When it became clear that in-person instruction would end for the term, these institutions quickly recognized a disproportionate impact on certain types of students. “One of our first things that we talked about was, how are we going to make sure that students have additional time and all of the kinds of accommodations and modifications that were provided for them on campus. How will we do that remotely?” Brooks said. “We had our director of accommodation and inclusion coming in and talking with all of us, giving pointers on how we could continue those opportunities for our students with disabilities.”
In conjunction with ADA and disability support, institutions also needed to ensure technology access for students. “How can we get our students the technology they need to be able to participate in their courses and to learn?” Adkinson said.
Scott, an attendee, shared that “Our top priorities in assessment have been students’ technology needs and the capacity for learning in a remote environment. This has allowed us to adapt and move forward with goal planning for future semesters.”
In the abrupt shift to remote learning, the usual co-curricular supports were no longer available. “It’s interesting and heartening to see a greater awareness of our students’ needs and ensuring that we are building structures and being responsive not to who we think our students are and how they operate, but who they really are and what they really need,” Jankowski noted.
