Assessment allows your institution to make meaningful improvements by capturing the information needed to analyze course, program and institutional outcomes. But in many cases, the assessment process is painfully inefficient and time consuming. And, despite the substantial effort put into assessment, it often yields results that are difficult to interpret, validate, track, and compare.
That was the case at Craven Community College. “The same data was being touched four or five times by different people,” says Cynthia Bellacero, executive director of institutional effectiveness. “Along the way, we would lose some of the formatting.” Although the data was complete when it got to Bellacero, it wasn’t easily comparable and wasn’t being used to make informed decisions about teaching and learning.
An assessment technology solution can streamline the process, reduce manual tasks, and capture information in ways that make assessment results more useful and transformative. This, in turn, eases accreditation reporting, another traditionally time-consuming task. So it’s crucial to choose the right solution.
Finding the assessment solution best suited to meet your institution’s objectives takes time— plan on spending three months or more—and the right team of campus stakeholders. Consider these four steps when evaluating assessment technology solutions.
On most campuses, numerous individuals have a stake in assessment and its results — and it’s important to include them in improving your assessment process, beginning with evaluating software solutions.
Start by identifying your stakeholders, including academic leaders, administrative leaders, and faculty members. Involve them from the beginning of your evaluation process, including them in committee meetings along with faculty leaders and other stakeholders. You will also want to identify a project sponsor as early in the process as possible—preferably someone at the dean or provost level—who can commit resources and eventually promote adoption.
For example, Waldorf University involved the university president and the vice president for business affairs in their search for an assessment solution from the beginning, says Rick Axelson, director of institutional effectiveness. Doing so helped these key administrators understand the importance of the system and made the budget approval process easier.
Engage larger numbers of stakeholders in surveys and focus groups to ask for their input and gather feedback.
Key questions include:
Next, identify potential challenges or obstacles you will face in gaining stakeholder support. Common challenges include budgetary restraints and lack of faculty engagement.
Quick checklist to success
CASE STUDY
Waldorf was accustomed to a manual assessment process, which required faculty members to fill in blanks on a PDF. Only a handful of academic departments “were really doing a good job with assessment, and the others were sporadic,” says Rick Axelson, director of institutional effectiveness.
Every time Axelson’s team needed to compile data, they struggled to locate, aggregate, and assemble assessment data from each department. “We needed to automate the data, so we could use it instead of just putting it in these reports [for accreditation],” Axelson says.
“Some faculty are really getting into taking a deeper look at their curriculum design and curriculum maps.”
Rick Axelson, director of institutional effectiveness
Vince Beach, vice president of academic affairs, had used an assessment solution at another institution before he arrived at Waldorf, and he regularly encouraged his colleagues to implement an assessment system to streamline and simplify their processes. After evaluating three software systems, the Waldorf team chose to implement a Watermark assessment solution the following year.
After training faculty on the new tool, all assessment data now lives online. “Now it has a sense of being more real to faculty,” Axelson says. “It’s not just a report they’re filling out, but it’s online. They can share the information with their colleagues easily. And I think some faculty are really getting into taking a deeper look at their curriculum design and curriculum maps.” In addition, the assessment solution’s interface was similar to the manual process Waldorf was already using, which led to quick and easy buy-in from faculty.
At Waldorf University, institutional effectiveness leaders sometimes found themselves scrambling to find assessment data for accreditation purposes
“If there had been changes in faculty, sometimes departments didn’t know where their assessment materials were,” Axelson says. “So, we were hunting down materials. And because there was no standard format for submitting the data from each department, it was hard to get a good sense of where overall learning assessment was for the institution.”
Once all data is collected, institutional leaders sometimes face “data analysis paralysis” due to the difficulty of assembling and aggregating assessment data from manual systems across campus. That makes it difficult to turn assessment data into valuable, actionable insights.
At Craven Community College, Bellacero would work to compile data from a variety of sources in several different formats, but even when the data was assembled, it wasn’t easy to make accurate comparisons or make more informed decisions.
“Faculty members were only looking at [one] piece of data and not looking at it comparatively to other people’s data, or data from five years ago, or even last semester,” Bellacero says. “It was labor intensive and not giving faculty what they needed to make better teaching decisions.”
Look for a vendor who is knowledgeable about the needs of higher education, has the expertise you require, and is invested in your success.
Finding the right technology solution can help institutions relieve staff from the burden of manual processes, simplify the accreditation process, and provide easy access to actionable assessment results for stakeholders and decision makers.
Consider the common issues above, plus those you’ve identified on your campus as you research potential assessment solutions. Look for technology that’s robust and has the features and capabilities you need. In addition, look for a vendor who is knowledgeable about the needs of higher education, has the expertise you require, and is invested in your success. Lay out criteria, starting with these best practices.
Follow these 7 tips to form your vendor criteria:
Get more details on the Watermark blog.
An assessment technology solution can streamline the process, reduce manual tasks, and capture information in ways that make assessment results more useful and transformative.
By engaging your full range of stakeholders from the start, you’ve developed a well-rounded vision of the assessment goals and software needs across your institution. Based on information gathered in steps 1 and 2, begin developing your selection criteria.
Match your institution’s assessment goals with the capabilities of potential vendors.
Ask the team:
CASE STUDY
At Craven Community College, each full-time faculty member was responsible for measuring student learning outcomes in their courses. Faculty members wrote reports each semester, then gathered with fellow department members to write summary reports. Those reports went to department heads, who then completed spreadsheets with the information, and sent them to unit leaders. Finally, the unit leaders sent reports to Cynthia Bellacero, executive director of institutional effectiveness.
“The same data was being touched four or five times by different people,” Bellacero explains. “Along the way we would lose some of the formatting.” Although the data was complete when it got to Bellacero, it wasn’t easily comparable and wasn’t being used to make informed decisions about teaching and learning.
“Because faculty members were involved in the selection and they bought into the process, it’s… something that they find beneficial to themselves.”
Cynthia Bellacero, executive director of institutional effectiveness
A selection committee of about 15 people undertook a rigorous process to evaluate and select an assessment software program. After studying several and inviting three vendors to conduct demonstrations, the committee unanimously chose Watermark because it fit Craven’s goals at the course, program and institutional level.
Craven implemented the new solution campus-wide in Fall 2018, while a group of faculty members began using it earlier with positive results. “Because faculty members were involved in the selection and they bought into the process, it’s not another thing they have to do for me, but something that they find beneficial to themselves,” Bellacero says.
Bellacero expects that full implementation will go smoothly because faculty members will not have to learn a new process for assessment. Because the platform is flexible, “We can create our portal to fit our current process,” she says. “That will just make our process easier to navigate as opposed to changing our process to fit the software.”
You’ve set your priorities and requirements, so you’re now ready to evaluate assessment solutions with an eye to your needs and wants. To find the system that’s right for your institution, ask your counterparts at other institutions for their recommendations. And talk to vendors at conferences—they’re there specifically to answer questions, learn about your requirements, and provide the information you need to make the right decision for your institution.
Don’t be afraid to ask tough questions.
Potential vendors should be happy to tell you:
Once you’ve become familiar with some key vendors, attend webinars to learn about their products. Look at their thought leadership content, such as articles and reports, as well as case studies and testimonials to get a comprehensive view of their expertise.
When you’re ready to begin evaluating individual vendors, reach out to their representatives with your priorities and timeline and, if needed, a formal request for proposals (purchasing).
Enlist your committee to review vendor responses and select at least two to conduct demonstrations and question-and-answer sessions for your committee to help make your decision.
When you select a vendor, take a moment to congratulate yourselves— you’ve achieved a substantial milestone! Then, prepare for success with the solution you chose
CASE STUDY
When they arrived at the Fresno State campus, Lisa Bennett and Cathy Yun, teacher credential program coordinators for the university’s Literacy, Early, Bilingual, and Special Education Department, realized there was no centralized data management strategy. When the time came to work on an important grant proposal, they couldn’t find the data they needed.
“It was scattered among 20 different people and 20 different offices, who had 20 different systems for documenting and keeping track of the data they were in charge of,” Yun says.
“No one was monitoring the progress of our candidates in the field,” Yun explains. “No one was monitoring the quality of our mentor teachers, based on candidate evaluations. No one was monitoring the quality of the coaching and feedback that our candidates were getting. That stuff was all collected at the end of the semester, filed away in a cabinet and never looked at again.”
“We talk about the data and what it’s saying and how we need to improve. We’re really trying to cultivate a culture of continuous improvement by using an assessment solution daily.”
Cathy Yun, teacher credential program coordinator
With a grant to fund a new assessment system, Yun and Bennett set out to find the right solution. They examined several different options and identified three software vendors that offered a candidate data management product. They invited each vendor to provide an online demonstration, and, with input from their faculty members, they selected a Watermark assessment solution to help them develop a digital database and move into the 21st century.
After one year of using the assessment solution, Fresno State’s School of Education has accessible data that is regularly analyzed and used in decision making. “We have changed the structure of our program meetings,” Yun says. “We talk about the data and what it’s saying and how we need to improve. We’re really trying to cultivate a culture of continuous improvement by using an assessment solution daily.”
After you’ve selected your vendor and purchased a system, you can confidently move into the implementation phase. Consult your partner about next steps and begin preparing your team for a richer, more robust assessment process.
Don’t wait until you expect users to begin using a new assessment system to inform them about the change. Task your project champion with helping to communicate benefits of the solutions and expectations for use as early as possible, and find out what resources your vendor has to support your efforts building investment with different stakeholders on campus.
Get your committee members involved in promoting the solution to their peers and leading demos. Training committee members as “power users” can speed adoption and ease the learning curve for users across campus.
Consider pilot testing the solution with a few departments before implementing it to all intended users. A pilot test gives you the opportunity to roll out your new assessment solution to a small group and then gather feedback so you can improve delivery to your larger group of users. Make refinements to your system and training materials before rolling out to all users.
Share successes along the way. Streamlined processes and reduced manual tasks encourage users to engage with the system; communicating frequently with key stakeholders about why they’re using the system and how it relates to their goals can help sustain engagement. In addition, explore opportunities to connect with peer institutions to share best practices, and learn how others are using the solution you’ve chosen. Finally, celebrate implementing a solution that will help your institution make meaningful improvements!
See how our tools are helping clients right now, get in-depth information on topics that matter, and stay up-to-date on trends in higher ed.
