The essential guide: What data can do for your university with Watermark Insights

Colleges and universities are grappling with numerous challenges in higher ed such as declining enrollment, changing demographics, and outdated technology. Data is essential for institutions to better understand where they are successful, and where they need to improve. With the right insights on your side, you can increase your accountability to stakeholders, better understand where to make investments, and ensure that students are receiving the best education your institution can offer.

But it’s important to remember that software solutions are simply tools. Just as the hammer in your toolbox can’t build a house by itself, no solution will magically transform your institution on its own. Instead, it’s important to work closely with your institution’s leadership to build a data-driven campus culture.

A data solution can provide six key benefits to your university:

1. Improving efficiency and access to information
2. Easing accreditation, assessment and other reporting
3. Increasing data accuracy
4. Measuring institutional impact
5. Supporting student success
6. And, yes, transforming your university

“Colleges and universities are big places, often decentralized, where success depends on activity at all levels. So any change truly takes a village, but simply peppering that village with data will not lead to better decisions or action. Members of the campus community have to engage in a process of interpretation to collectively assign meaning to it. From there, planning and action can take place.”

Angela Baldasare
Assistant Provost for Institutional Research, University of Arizona

Efficiency is a fairly humble-sounding outcome of investing in a data solution, but don’t underestimate its importance. Entering data once and then reporting on it from all angles is no small matter. Consider how many times people within your institution reach out to faculty and others for the same information. Capturing commonly requested information in a central database ensures that you have answers at your fingertips — instead of having to initiate the gathering process again.

Based on research from hundreds of universities, the number of faculty activity requests range between 8 and 12 per person. Consider the additional administrative time spent sending out requests and aggregating responses, and the hours add up fast.

“We no longer have the problem of having files in multiple locations. We have everything for eight campus locations at our fingertips no matter where we are. That is a huge benefit for us.”

Elizabeth Heise Associate Professor,

University of Texas Rio Grande Valley

According to findings from the first phase of the Time Allocation Workload Knowledge Study (TAWKS) conducted by researchers at Boise State University, faculty work 61 hours per week on average, with more than 19 hours per week spent on administrative tasks. “Efficiency” doesn’t have the curb appeal of “transformation.” But when you consider the value of freeing your faculty and administrative staff to do the work that matters most — teaching, research, and service — the importance of efficiency becomes crystal clear.

Campus data is captured for many uses, including assessment, accreditation, course evaluations, institutional research, institutional effectiveness, program reviews, and more. Unfortunately, this data is often restricted to the department that captured it, meaning someone working in institutional effectiveness may have trouble accessing data from course evaluations. These data silos prevent others from leveraging valuable information that could lead to improved decision-making.

Only 23% of chief academic officers rate their institution as very effective at using data to aid and inform campus decision-making.

Inside Higher Ed 2020 Survey of Chief Academic Officers

Reporting requirements for many stakeholders, from professional and regional accreditors to state legislatures and federal funding sources, continue to grow in complexity. The bar for accurate evidence is high, as are the stakes — consider the cost of lost funding or failing to demonstrate compliance with an accrediting body. Gathering the information needed to provide accurate and timely reporting to these stakeholders from binders, file cabinets, and departmental spreadsheets is onerous, time consuming, and prone to errors and oversights.

Capturing this data in one place means it’s readily available when needed. The reporting capabilities of the right database will allow you to pull the same data into reports for a range of requirements without asking for information again from your department chairs or faculty. This ensures that calculations and analysis are consistent over time, so your reports are not only easier to assemble — they’re also more reliable. That reliability is key to conducting longitudinal analysis to show continuous improvement.

Regional accreditation
Texas Wesleyan University streamlined outcomes assessment and their SACSCOC reaffirmation by implementing a data solution to map goals, house annual assessment plans, findings, and status reports, and foster discussion among the faculty involved. Previously they relied on a paper-based system to manage their assessment and accreditation needs

Program review
Learn how Embry-Riddle Aeronautical University found a tool to help gather feedback to support program review, accreditation, and faculty review.

Federal reporting for extension
Implementing a data solution helped Purdue University create central data collection and reporting to meet its federal reporting obligations as a land grant institution.

Accreditation and other reporting from your data solution requires some work upfront. Consider what you’ll need from your system to be sure you’re collecting data that satisfies accreditor and funder requirements. Capture data in a format that allows you to use it for the full range of accountability reporting your institution does. Then, use your data solution to generate the reports needed to fulfill the requirements of your stakeholders. Common requirements include:

Accreditation reporting for regional and program accreditors in areas of accountability such as assessment of student learning and outcomes, faculty qualifications, institutional effectiveness, program review, continuous improvement initiatives, and more.

Federal reporting such as Title II teacher preparation and certification reporting and National Institute of Food and Agriculture (NIFA) reporting that is required of land-grant universities. (NIFA supports initiatives that advance agriculture-related sciences, and looks to institutions they fund to report on their research, education and extension programs.)

State reporting requirements such as Texas HB 2504, which requires a syllabus and instructor CV to be posted online for every undergraduate course offered by University of Texas system institutions within seven days of the start of each semester.

Actionable insights that power data-driven decisions come from quality data. Achieving a high level of data quality means you can trust the analysis and reporting from your campus systems to inform decisions, whether you’re considering program outcomes, faculty accomplishments, or your institution’s strategic priorities.

“Reports are great, but what more can you do to aid decision-making processes? What does your data mean in the process you’re aiding? When we look at a number, we ask, ‘So what?’ and ‘What more?’” said Sam Russell, Business Analyst at University of Michigan School of Public Health (MPH). “We’ve moved away from manual processes to do more value-added and strategic mission work. A number is just a number. The value comes from context. Now we can provide the relative value.” In other words, MPH uses good data to drive smart decisions.

“We’ve moved away from manual rocesses to do more value-added and strategic mission work. A number is just a number. The value comes from context. Now we can provide the relative value.”

How can you evaluate the quality of data in your campus systems?

Commonly accepted dimensions of data quality include:

• Completeness, or whether the data needed to feed your reports is present in the system.
• Consistency in data collection, so all activities of a certain type are entered the same way, in a single source field, and can be consistently extracted to feed any report— vital to breaking down “data silos” on campus.
• Timeliness, or how up-to-date your data is, and therefore how current your analysis and reports are that use that data.
• Accuracy, which depends entirely on correctly and completely entering data.

Your data solution provider can measure the first three criteria and guide you through improving them. When it comes to accuracy, a data solution should provide the framework for entering data in the most useful format to answer your key questions and ensure that all stakeholders who need to analyze or report on that data have the information they can use. It should also allow you to easily draw in data from other systems, including those on campus and other data repositories.

The time savings here is obvious — enter a faculty member’s credentials into the human resources system on campus, then tap that data for use in your faculty activity reporting system rather than requiring each faculty member to enter it again. Or pull data directly from your learning management system (LMS) and student information system (SIS) into your assessment management system rather than entering the information twice to populate both systems.

A hidden but substantial benefit of drawing data from existing systems is improved accuracy. Or pull data directly from your learning management system (LMS) and student information system (SIS) into your assessment management system rather than entering the information twice to populate both systems. Accuracy is further bolstered when information from a system such as the SIS shows up in an assessment tool so you aren’t relying on users to enter this information. Using data from a “source of truth” software improves the accuracy of all reports that utilize data from that system.

What is your institution doing, and what difference does it make for students, the university, and the community? The right data solution ensures that you capture the data and develop the reporting needed to answer these questions. You’ll be better able to satisfy stakeholders and share compelling stories of your institution’s activities and impacts.

Let’s dig deeper into each of the questions to understand the role data plays in answering them.

There’s so much happening on your campus that this should be easy to answer, right? And it is easy, once you begin to capture all of those activities in a central database. Consider the answers you need in order to demonstrate a point in your strategic plan or accreditation self-study, such as student outcomes or continuous improvement efforts. What data would help you measure and evaluate this across your institution? And what data will help you share stories with your full range of stakeholders, such as prospective students and faculty, legislators, the media, and others?

Make sure your solution has fields in which to capture the data you need, in a format that can be analyzed and reported on across the institution. This makes your data accessible for other kinds of reporting and analysis. Campuses that work to make data available across the institution make remarkable discoveries about what’s actually happening today that inform decision-making at all levels.

“To be able to say, ‘Your $5 million made a breakthrough in terms of a new fertilizer application that is healthier for the water and the soil,’ that’s [what] they want to know about.”

Dawn Parks
Assistant Director of Special Programs,
Purdue University

Evaluating the impact of your institution’s work is a more complex undertaking, likely requiring data from multiple campus systems. “When we think about impact, we’re trying to describe what happened with the research funding dollars that we’ve received,” said Dawn Parks, Assistant Director of Special Programs at Purdue University. Parks works closely with Purdue’s School of Agriculture and Extension program. “To be able to quote numbers is really important … but to be able to say, ‘Your $5 million made a breakthrough in terms of a new fertilizer application that is healthier for the water and the soil,’ that’s [what] they want to know about,” Parks said. Purdue uses campus data to surface individual stories of the impact of Extension research and outreach on the citizens of Indiana.

It takes careful consideration to report on impact, especially since impact often emerges over time, but smart use of campus data can reveal powerful evidence. Begin by considering how you want to track progress against key initiatives — such as changes to the general education curriculum, or a first-year experience initiative — and who has a stake in understanding your outcomes. This will help you understand the reporting required by stakeholders, such as an accreditor, a federal or state agency, or your institution’s board. The analysis and reports you need to generate determine the data you should collect

Forward-thinking institutions have moved beyond viewing data and reporting through the limited lens of compliance. They engage in a mission to discover the current state of their institution — from what’s working well and how to support those activities, to how they can improve in order to better serve students.

“With the targeted outreach and texting potential available in our new solutions, we are able to interact with the students better, which means creating campaigns to proactively influence decision-making,” said one student success champion, Michael Coleman, the former CIO and Vice President of Student Services & Technology at Haywood Community College. “Before this technology, we weren’t utilizing text messages and we weren’t able to realize the potential such communication strategies could have on student engagement and success.”

Used well, a data solution can unearth surprising findings that open the door for further inquiry and true innovation. University of Texas at Austin (UTA) is using data to provide funding to support their summer experience internships. This helps low-income and middle-income students complete their degrees in four years and get the professional experience needed to start their careers right after graduation. UTA has also used this data to give low-income students a campus meal plan, even if they live off-campus.

“[Data solutions] help us keep the students we’ve worked so hard to recruit and see them across the finish line, which in turn improves the bottom line of the college.”

Ian Slater, Vice President for Student Development at Eastern Nazarene College (ENC) in Massachusetts

Using data well starts with defining what to measure, and how. Involving a wide range of campus stakeholders, ensures that goals aren’t met by reducing standards — a common concern among data skeptics.

“People are realizing that when we are looking at outcomes that there is a direct link between using data — using it well — and better outcomes. Those institutions that have been able to see gains in college completion, for example, or have closed equity gaps, have been ones that have embraced a culture of data.”

Michelle Asha Cooper, President. Institute for Higher Education

Best practices for ensuring that data is used to truly improve student success — rather than just hit a benchmark on paper — include:

• Disaggregating data to ensure that overall numbers don’t mask disparities among students from different groups on campus. If your graduation rate isn’t relatively uniform across demographics, it’s vital to find and fix the root cause.
• Challenging assumptions made in the pre-data era, such as whether success in a particular “gatekeeper” course actually predicts success in a major.
• Broadening your view to take into account factors not previously considered when measuring student success. For example, Georgia State University discovered that registration errors were a significant driver of D-Fwithdraw rates.
• Focusing on current student populations. Many of today’s students are older, and balancing jobs and family commitments in addition to their coursework, so they have different needs than the students of previous eras.

Data alone won’t transform an institution. But when a university uses data to drive decisionmaking, transformation is possible. Consider Fayetteville State University: during the 2008 economic recession, FSU lost 22 percent of its state funding and cut 150 faculty positions as the costs and benefits of the University of North Carolina system came under public scrutiny. To survive, FSU had to improve its degree productivity on a substantially reduced budget. FSU used information gleaned from campus data to increase degree productivity from 15 percent in 2008 to 21 percent in 2015, earning the American Council on Education (ACE) Award for Institutional Transformation in 2016.

This success required a cultural shift within the university. Chancellor James Anderson worked in concert with faculty to develop metrics for evaluating departments on their own terms, then codified those metrics to ensure evidence-based decision-making and outcomes assessment. Using the carrot of additional funding to most-improved departments, Anderson and FSU continue down the path of real transformation informed by sound data and reporting.

FSU isn’t alone. Between 2009 and 2017, the student population at the University of Alabama nearly doubled in size, reaching 38,563 students enrolled in the fall. As student enrollment continually increased, so did the exponential demand on resources throughout the institution. In the face of such unprecedented growth, UA’s disparate collection of tools for capturing and organizing data across campus — from paper in binders to siloed single-point solutions — quickly became an unsustainable way to track student learning and institutional outcomes.

Better data leads to better decision-making. A solution that can put your data in context to surface insights will help you make the best decisions for your institution.

Purposefully shifting to a centralized assessment management system has cultivated a new, growing culture of assessment throughout the university.

A process once driven by compliance-based actions with limited perceived value has been transformed into one of meaningful, measurable, and manageable quality improvements driven by better data.

“We’re changing the culture to make it about doing things that are useful and measurable, and we have seen a gradual shift in the way people talk about how we assess things. Now they ask, ‘How is this impacting students and the community?’”

Ginger Bishop Institutional Effectiveness University of Alabama

A quality data solution is just the first step to transforming a university. Ultimately, transformation requires that your institution develops a culture of discovery — informed by data, and fueling a commitment to celebrate successes and strive for continuous improvement. It helps to have a technology partner on the journey to share best practices and examples from peer institutions, while guiding you through an implementation to set you up for success and campus-wide engagement.

It’s also important to know that, with a data solution, you must walk before you run. The more you use your data solution, and the longer you’re gathering data, the more useful it becomes. Knowing what to expect from your data solution at each step of implementation is vital to your success. A path to gathering the data needed for institutional transformation may progress like this:

• Annual processes, such as faculty reviews and student assessment: This requires just one year’s worth of data, allowing everyone to get up to speed with entering or importing data into the system before requiring deeper reporting from the system.
• Outcomes assessment: Formative, ongoing assessment provides the necessary and frequent feedback students need to progress throughout their learning journeys. Summative assessment provides valuable data on student learning at the end of an instructional unit by comparing it against some standard or benchmark. Data from both methods of assessment can be gathered and reported on in as little as one assessment cycle or academic term.
• Accreditation reporting: Accreditors typically require up-to-date information on faculty qualifications as well as current and previous assessment cycles, accessible on the spot during a visit in order to demonstrate consistent continuous improvement efforts across the institution.
• Continuous improvement and program review: With a wide range of data available, as well as a variety of visualization tools, you can build reports that allow you to measure the progress of a full department, a college, the entire institution, or even a particular group of students based on demographic filters. In as little as two assessment cycles or several years or academic terms worth of data, you can have the longitudinal reporting, analysis, and insights needed to improve student learning outcomes and institutional quality.

The data solution you choose is only part of what it takes to turn the statistics into action. Ultimately, carefully curated data allows you to accurately report on the current state of your university, and chart a course for transformation. But as always, the transformation is up to you.