Improving the Quality of Assessment with The University of Arizona

In 2009, interest in assessment was gaining momentum across the country, and accreditors began looking more closely at outcomes. In response, the University of Arizona formed its Assessment Coordinating Council. The council, comprised of academic Associate Deans or their designees, was charged with bringing assessment to the forefront for the colleges and getting units in the college (typically departments or schools) to engage in assessment of student learning. In response to the Council, the UA designed and built a website to help track assessment. The site included pages for every unit. Each unit was expected to updated its pages annually, reporting on student learning outcomes, how the outcomes were measured, what the findings were, and changes to be made in response to the findings.

Every seven years, each unit at the UA undergoes an Academic Program Review, which includes reporting on program assessment and review efforts and updating the unit’s assessment webpages. Assessment reports are scored using a rubric. If necessary, the unit and the Office of Instruction and Assessment (OIA) reviewer develop an improvement plan.

By the time work with cohort six began, it was clear that the university’s assessment website had a number of issues, including poorly organized information and loss of previous years’ findings, which were deleted when units posted new findings. Because previous findings were lost, OIA reviewers had to go back to earlier versions and compare them to find the changes. In addition, there was no consistency from page to page, so it was impossible to run analytics across web pages. This made finding the needed information for accreditation reports, as well as progress reports that go out to the colleges and the provost for review, time consuming.

“It was so hard to review units’ assessment web pages that we became discouraged,” said Dr. Elaine Marchello, OIA’s Assistant Director of Assessment. The website was cumbersome and unintuitive, leaving users frustrated and unmotivated. People needed to be continually reminded and prodded to put in the necessary assessment-related information.

Marchello and the team knew they needed a more intuitive, easily trainable assessment solution that reflected how the university reported assessment information. After reviewing a couple of options, the UA selected Taskstream AMS by Watermark. For Marchello, the choice was simple. “Once I saw their Accountability Management System [Taskstream], I wanted to stop looking,” Marchello said. “I called my colleague and told her, ‘You have to see this.”

What so impressed Marchello? The system was easy to use, provided better reporting, and had useful analytics.

And then there was the Watermark team. “Customer service is important to us,” Marchello said. “We wanted a company that was easy to work with, had a good help desk, and offered helpful tutorials, and Taskstream by Watermark checked all of those boxes.”

In December of 2016, the UA began using Taskstream by Watermark. The OIA assessment team spent the rest of that calendar year working with volunteer units to pilot the program, helping users get accustomed to the software, and designing a workshop to bring the rest of the university units into the new system. In addition, to reach some units, the OIA assessment team did one-on-one orientations.

The full rollout of the program began in early 2018. From January to May of 2018, the OIA assessment team conducted two training workshops each month with two to three colleges invited to each session.

The OIA assessment team was “blown away” by attendee response. Based on survey results, nearly 76 percent of attendees described their workshop experience as either good or awesome; just under 80 percent said they were comfortable using the system or believe they were but had not yet used it. Dr. Ingrid Novodvorsky, Director of Teaching, Learning, and Assessment, knew the workshops went well before seeing the results of the survey, “A common comment was, ‘Oh this is so much easier than reporting on the website,’” Novodvorsky said.

The positive response to the workshop has driven system use. With more than 50 percent of the workspaces populated, the OIA assessment team estimates that they already have double the information they had with the previous system. In addition, the information is clear, uniform, and aligned.

Using Taskstream allows the OIA assessment team to more efficiently handle their workload. “It’s so much easier to get my work done since we got Taskstream because everything is now in one place,” said Marchello.

Novodvorsky concurred, “Before we had to hunt around to find the necessary information but now with uniform reporting in Taskstream, I know exactly where to look for things.”

Getting the work done in a more efficient manner has given the OIA assessment team the time to do more outcomes mapping and analytics, making it easier to report on progress.

Implementing the new system has also given the UA an opportunity to renew its focus on assessment of student learning. People gained a refresher of the process and a greater understanding of the assessment cycle.

The university now uses Taskstream for tasks beyond assessment, including aligning program learning outcomes with institutional learning outcomes, setting up templates within Taskstream for reporting to program accreditors, and creating Higher Learning Commission reports.

With the successful launch of Taskstream by Watermark, the UA is striving to bring on more users and is encouraging units to include additional program review materials in their system workspaces. The OIA assessment team continues to conduct one-to one-trainings with new users. With a more intuitive system and a renewed focus on assessment, the OIA assessment team expects an increase in units’ use of Taskstream. This will lead to more information and analytics being collected, ultimately improving the quality of assessment.

In addition, the UA has discussed using the system for strategic planning and the Academic Program review Self-Study Reports.