Ten North Carolina community colleges participated in a large-scale experimental study of technology-enabled success coaching funded through the U.S. Department of Education’s First in the World program. Results reveal positive impacts on longer-term retention, especially for Black students, suggesting that technology-enabled success coaching can be an effective tool for strategic enrollment management.
All students should be supported to attend—and finish—college. Most of today’s students live complex lives, balancing multiple responsibilities that include working while enrolled and raising children. In addition, nearly 40 percent of undergraduate students are from low-income households, defined as households earning less than 200 percent of the official poverty threshold, which represents more than seven million students (GAO 2018). As documented in a 2018 report from the U.S. Government Accountability Office, today’s college students are increasingly at-risk of food insecurity, as three-fourths of low-income students are also single parents, first-generation students, homeless, or at risk of being homeless.
Providing holistic supports that can help students address both academic and non-academic needs is an increasingly critical focus for strategic enrollment management (SEM). A major goal of enrollment management is connecting students to campus and building relationships with students to ensure they get involved in campus life and are engaged with support services to increase retention (Henderson 2017). Building relationships with students is a whole-college effort, as relationships are essential to meeting the full range of student needs; how faculty, staff, and administrators respond to and support students in building rapport and connections on campus is essential for institutional enrollment health (Henderson and Pollock 2021). To address the complex lives of students, colleges nationwide are increasingly taking a more holistic, personalized, and proactive approach to providing support services to students in an effort to increase persistence and raise completion rates (Kalamkarian, Boynton, and Lopez 2018; Karp and Stacey 2013).
Several studies published in the last decade point to the effectiveness of intensive advising or coaching to improve student outcomes (Evans, et al. 2020; Mayer, et al. 2019; Richburg-Hayes, et al. 2009; Weiss, et al. 2011).
While there is no universal model of success coaching, most are informed by a case management approach to advising that is tailored to each individual student; incorporates aspects of mentoring and counseling in addition to academic advising; and is designed to provide sustained support for students over time to help them meet their longer-term academic and life goals (Pierce 2016; Richardson 2008). The most widely-cited study of success coaching—based on a large-scale, multi-institution experiment—showed that students receiving services from a coach were significantly more likely than other students to remain enrolled in college and to complete a postsecondary credential (Bettinger and Baker 2014).
Building on this evidence-base, the First in the World (FITW) Validation Study of Carolina Works is the first-ever large-scale, multi-institution experiment to isolate and test the impacts of technology-enabled success coaching within a community college setting. Studying promising strategies in community colleges is important because approximately nine million students are enrolled in public two-year institutions, which offer a lower-cost, open-access entry point to postsecondary education and credentials (Ginder, Kelly-Reid, and Mann 2018). Community colleges also disproportionately enroll students of color and low-income students, with 55 percent of all Hispanic undergraduates and 45 percent of all Black undergraduates enrolled in community colleges, according to recent estimates. In addition, approximately 55 percent of dependent students with family incomes below $50,000 in 2011–12 began their higher education journey at a community college (CCRC 2021). Unfortunately, nearly two-thirds of community college students do not earn a postsecondary degree or credential of any kind (Shapiro, et al. 2019).
The FITW study provided an opportunity to rigorously examine technology-enabled success coaching as a strategy to provide personalized and proactive support services that are informed by real-time data on students’ academic and non-academic progress and challenges, including the use of predictive analytics to prioritize outreach and engagement. The research results summarized in this article indicate that technology-enabled success coaching can be an effective approach for strategic enrollment management to build reliable and trusting relationships with students that impact retention and completion. The results also point to several implications for practice in terms of how coaching can be used to address persistent equity gaps in higher education, and in terms of the institution-level factors that can make coaching interventions most effective.
According to the Department of Education (2016), “the FITW program is designed to support the development, replication, and dissemination of innovative solutions and evidence for what works in addressing persistent and widespread challenges in postsecondary education for students who are at risk for not persisting in and completing postsecondary programs, including, but not limited to, adult learners, working students, part-time students, students from low-income backgrounds, students of color, students with disabilities, and first-generation students.”
Carolina Works was one of two Validation Grants awarded by the U.S. Department of Education FITW program in 2015 and sought to improve students’ retention and credential completion outcomes through the provision of proactive, data-informed success coaching. Led by Central Carolina Community College, the geographically diverse Carolina Works consortium included Caldwell Community College and Technical Institute, Carteret Community College, Cleveland Community College, College of the Albemarle, Isothermal Community College, Pamlico Community College, Randolph Community College, Roanoke-Chowan Community College, and Southwestern Community College.
By utilizing a holistic approach to coaching informed by Appreciative Inquiry (AI) (Cooperrider and Whitney 2005), success coaches—professionals who build relationships with a caseload of students—served as a single point of contact for students to guide and support them on their educational pathway. Success coaches leveraged a Student Success Platform and equity solution that provided predictive analytics and early alert systems to all ten participating colleges. By using predictive analytics, success coaches had the ability to prioritize their outreach, proactively engage and support their students before they reach a crisis point, and provide follow-ups with faculty and staff to “close the loop” so everyone who supports these students knows they are getting the support they need to be successful.
As noted in the introduction, students live complex lives that compete with their educational pursuits, and colleges need to scale innovative solutions—like technology-enabled success coaching—to help students address food and housing insecurity, transportation, childcare needs, and other life conditions alongside academic supports that focus on courses and programs. Success coaches can provide access to include both campus resources like food pantries, emergency grant aid, tutoring, and writing supports as well as community resources such as housing and food assistance, transportation services, utility assistance, and internet access. In other words, success coaches’ responsibilities extend beyond those of a traditional academic advisor.
An independent evaluator for Carolina Works designed and executed an in-depth implementation study and a randomized controlled trial to examine the impact of technology-enabled success coaching. Throughout the five-year FITW study period, ten North Carolina community colleges participated in an experimental study in which new students were randomly assigned to a treatment condition (assignment of a success coach) or a control condition (no coach, business as usual) at the start of their first semester. All students in the study had access to any services already provided at their college such as academic advisors and other student support services; however, only students in the treatment group were assigned to success coaches.
More than 10,700 students across these ten community colleges were randomly assigned to the study sample in fall 2016, fall 2017, or fall 2018—50 percent of whom were assigned a success coach. The timeframe of the study allowed for observation of the impact of success coaching over time, as outcomes of all treatment and control students in the study were tracked for at least two academic years.
Success coaches worked to develop personal relationships with students, serving as a main point of contact as well as a connector to other key supports and resources at the college and in the community. In addition to providing direct support to students, coaches referred students to other college personnel and resources, following up with students to help with any next steps. At each of the ten North Carolina community colleges, success coaches utilized technology including predictive analytics, performance tracking, and automated and early alerts. Through this technology, coaches could monitor student grades and attendance, and utilize comprehensive predictive algorithms that focus on course-level risks as well as the likelihood to return to the next term, all to help prioritize real-time outreach. Notably, all students in the study (both treatment and control) received automated alerts on the basis of attendance patterns and grades; in addition, faculty at all colleges could generate early warning alerts about any student, which were routed to a campus-specific coordinator to handle these alerts. For students in the treatment group, their success coach also received these automatically-generated alerts as well as faculty alerts and followed up with the students proactively. In other words, the only difference between these two groups of students (treatment and control) was the presence of a success coach.
Success coaches in Carolina Works were employed by the college, worked on-site, and had the opportunity to engage students in-person as well as to become familiar with institution-specific policies and procedures affecting student success. In total, 37 coaches were employed across the ten colleges at some point during the five-year study period. Almost all coaches possessed master’s-level degrees, and although many coaches had backgrounds in counseling or education and extensive prior experience working within a community college setting, depth of experience in a student service role varied considerably among coaches.
With the exception of Central Carolina Community College, the other nine institutions were implementing success coaching for the first time as a requirement for this study. All coaches underwent a two-day training provided by the lead college, Central Carolina Community College in collaboration with a student success platform and equity solution, which included training on use of the case management technology to inform coaching practice. The program trained coaches to provide email and text outreach to all students on their caseloads on the first day of each semester, followed by more individualized, proactive outreach throughout the first weeks of the term intended to intervene with students before any issues arose. In addition, the program trained coaches to reach out to students based on automatically-generated notifications that included both positive achievements as well as challenges associated with attendance or grades and early alerts generated by faculty and staff at their institution.
To complement the experimental study, the evaluation of Carolina Works included an in-depth qualitative assessment of implementation across the ten colleges. Through site visits to participating colleges that included interviews and focus groups with success coaches, faculty and staff, administrators, and students, the evaluation assessed six interrelated measures of implementation fidelity. These implementation fidelity measures included key institutional factors such as strength of campus leadership, campus-wide communication and engagement, and technology adoption and use—all of which influence colleges’ and coaches’ ability to implement success coaching most effectively. Based on an assessment of these metrics, the evaluation identified three colleges as high-performers in terms of implementation fidelity. These colleges demonstrated strong leadership support for success coaching, successfully folding coaches into their existing staff and building institution-wide buy-in through campus-wide communication and engagement. Researchers further evaluated these high-fidelity implementation institutions as a sample in order to explore differences in impact based on implementation factors including the level of involvement from the institution and its staff.
Over the course of the grant period, the FTW research study documented the value of the student-coach relationship; the overall impact of having a technology-enabled success coach on student retention and credential completion within community colleges; and the variation of results based on implementation fidelity, which included qualitative measures of institutional commitment and buy-in to this model of student support. The study also found that technology-enabled success coaching can be an equity solution for Black students. More details on the results are included below.
A core component of success coaching is relationship building between the student and coach. Focus groups with students, which were conducted as part of the implementation study, revealed five key attributes that students valued in their success coaches. Across the ten colleges, more than 5 students discussed their experiences and interactions with success coaches, the extent to which success coaches had impacted their college experience or their lives in any way, and their perceptions on the benefits of having a coach as well as any challenges or areas for improvement.
Students indicated that success coaches were perceived as a reliable, caring, and proactive personal connection; a mentor identifying barriers and building their personal capacity to address them; a source of encouragement and accountability; a bridge to resources on- and off-campus; and a guide for academic and career goal setting (Curtis and Valentine 2020). These student perspectives align with the importance of combining both transactional services with deeper bonds of belonging so the twin pillars of SEM—recruitment and retention—are more effective (Henderson and Pollock 2021). The following student quotations are illustrative of the importance of strong relationships as a foundation for additional supports and assistance that coaches can provide to help engage and retain more students (Curtis and Valentine 2020).
“I was scared when I decided to go to college but after meeting with my Success Coach, the college did not look so big—a friendly face goes a thousand miles. Knowing you have this person, this go-to person, made college easier for me. It was no longer overwhelming, having a Coach was calming.”
“I think we should have more people like [my Coach], because she’s awesome, she cares about students academically and personally. If we had more of her, it would be really good. She helped me do a 180. My first semester was really horrible, I got into academic probation. But she helped me turn it all around. Now I’ve been on honor roll two semesters in a row.”
“I think the biggest thing [about having a Success Coach] is that if you are juggling jobs and school, they help remind you what you need to do to be successful. Not for the rest of the semester but for the next year.”
The overall results from the experimental study showed that benefits of success coaching grow over time, as relationships with success coaches take time to develop. For the full sample, students assigned a coach were 4 percent more likely to stay enrolled for two academic years (Valentine and Price 2020). (All findings reported in this article are statistically significant at p<0.05.) In addition, and further emphasizing the importance of relationships, the study found that students benefit more from success coaching when their coaches do not change over time. When assessing outcomes for students within the subset of colleges where coaches were in place for the duration of the entire study period, students assigned to coaches experienced a 6 percent increase in fall-to-fall retention and an 8 percent increase in fall-to-second-spring retention (Valentine and Price 2020). Students assigned to success coaches who remained in place for the full study were also 12 percent more likely to complete a credential compared to students without a coach (Valentine and Price 2020).
As discussed earlier, institutions were evaluated on key measures of implementation fidelity, including key institutional factors that support effective coaching such as strong leadership and campus-wide communication, collaboration, and engagement of faculty and staff in addition to student services personnel. Those institutions that demonstrated high fidelity of implementation generated larger positive impacts on both retention and credential completion. According to the study, success coaching—when combined with high-fidelity implementation—resulted in a 9 percent increase in students’ fall-to-second-spring retention as well as a 9 percent uptick in credential completion (Valentine and Price 2020). The importance of implementation fidelity identified in the FTW study reinforces how organizational change management is critical for effective SEM implementation. Adopting a collaborative approach to change, with formal and structured processes among faculty, staff, and administrators, enables a more constructive management of the inevitable differences of perspectives that will arise (Smith et al., 2020).
As previously stated, the goal of the FTW research study was to validate the effectiveness of technology-enabled success coaching among the ten North Carolina community colleges that participated. In addition, the study examined the extent to which the impacts of coaching may vary for different student groups, with a focus on student groups typically underserved within higher education. Importantly, results from the experimental FTW study found significant benefits of success coaching for Black students, a group that has not been served well by higher education institutions (Garibaldi 2014). In particular, the study shows that among Black students, those assigned to a success coach were 8 percent more likely to stay enrolled in the college for one year and 13 percent more likely to stay enrolled for two academic years (Valentine and Price 2020). A follow-up case study at Roanoke-Chowan Community College small, rural, and predominantly Black institution participating in the study, provides an explanation point for this finding: among Black students, who comprise the majority of all undergraduates at this institution, those assigned to a coach were 59 percent more likely to be retained from fall-to-fall and were 40 percent more likely to be retained after two academic years (Valentine and Price 2021).
The FITW study was designed to analyze the impact success coaching has on student retention and student completion among community colleges. The findings demonstrate that technology-enabled success coaching is an effective strategy to increase student retention and credential completion, especially when coaches remain in place and can work with the same students across multiple semesters. The results indicate that combining technology solutions with a highly-trained success coach who can build trusting and reliable relationships with students can be an effective component of SEM. The use of technology and the implementation of success coaching can ensure that students are provided timely and proactive support, guidance, and assistance to facilitate their path to success in higher education.
The findings from this study underscore the importance of strong institutional support, including widespread communication and engagement supported by college leaders, in order to build campus-wide buy-in for success coaching that can lead to better results for students. Regular and frequent communication about success coaching by college leadership increases faculty and staff awareness of the coaching intervention and ways to support it; a coach’s effectiveness, in turn, is greatly influenced by faculty and staff actions, understandings, and behaviors. For example, coaches cannot respond to early warning alerts from faculty if faculty are not entering them into the student success platform, and coaches cannot respond to the platform’s automated alerts unless faculty are entering students’ grades and attendance, which is the basis for these alerts.
An equally important consideration for successful implementation of coaching is the stability of the success coach position. Results from the FITW study point to the benefits of coach continuity, with impacts on retention growing over time, and with larger impacts of coaching within institutions where coaches were in place for the duration of the study period. Colleges can ensure the stability of a dedicated coach role by using operating revenues to protect the coach position from the swings of grant-funded projects.
Results from the FITW study also suggest that success coaching can be a solution to help close racial equity gaps. Black students in the FITW sample persist at notably lower rates than the whole-sample average, but the impact of coaching on their longer-term retention outcomes is large. Black students too often confront systemic racism within institutions of higher education, which can affect their sense of belonging, their mental health, and ultimately their success in college (Massey and Fischer 2005). Research also points consistently to a lower perceived sense of belonging reported by students of color versus white students within the same campus environments (Johnson et al., 2007; Rankin and Reason 2005). Success coaches in the FITW study focused on developing trusting relationships with students based on a respectful and non-judgmental understanding of students’ circumstances, which may be particularly important for students who are underserved or who confront systemic racism within higher education. More broadly, results from the FITW study suggest that intentional targeting of coaching services could help close equity gaps in student attainment.
With more intentionality, resources, and dedication of enrollment managers, student services professionals, faculty, and executive leaders, colleges can successfully implement technology-enabled success coaching to achieve the results they want—more students earning a credential who are better prepared for life and a career.
Bettinger, E., and R. Baker. 2014.
The effects of student coaching: An evaluation of a randomized experiment in student advising. Educational Evaluation and Policy Analysis, 36(1): 3–19.
CCRC. See Community College Research Center.
Community College Research Center. 2021. Community College FAQs: Community College Enrollment and Completion. New York: Teachers College, Columbia University. Retrieved from <ccrc.tc.columbia.edu/Community-College-FAQs.html>.
Cooperrider, D. L., and D. Whitney. 2005. Appreciative Inquiry: A Positive Revolution in Change. San Francisco: Berrett-Koehler.
Curtis, D., and J. Valentine. 2020. Student Perspectives on the Value of Success Coaching. Indianapolis, IN: DVP-PRAxis Ltd. Available at <dvp-praxis.org/wp-content/uploads/2020/05/Student-Perspectives-on-Success-Coaches-Web-FINAL-May-2020.pdf>.
Evans, W. N., M. S. Kearney, B. Perry, and J. X. Sullivan. 2020. Increasing community college completion rates among low-income students: evidence from a randomized controlled trial evaluation of a case-management intervention. Journal of Policy Analysis and Management, 39(4): 930–965.
GAO. See Government Accountability Office.
Ginder, S. A., J. E. Kelly-Reid, and F. B. Mann. 2018. Postsecondary Institutions and Cost of Attendance in 2017–18; Degrees and Other Awards Conferred 2016–17; 12-Month Enrollment. Washington, DC: National Center for Education Statistics. Retrieved from <nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2018067rev>.
Government Accountability Office. 2018. Food Insecurity: Better Information Could Help Eligible College Students Access Federal Food Assistance Benefits (Report 19-95). Washington, DC: Author.
Henderson, S. 2017. SEM and the student journey: The role of strategic enrollment management in student engagement. Strategic Enrollment Management Quarterly, 4(1).
Johnson, D. R., M. M. Soldner, J. B. Leonard, R. Alvarez, K. K. Inkelas, H. Rowan-Kenyon, and S. Longerbeam. 2007. Examining sense of belonging among first-year undergraduates from different racial/ethnic groups. Journal of College Student Development, 48(5): 525–542. Available at <doi.org/10.1353/csd.2007.0054>.
Kalamkarian, H. S., M. M. Boynton, and T. Lopez Salazar. 2018. Redesigning Advising with the Help of Technology: Early Experiences of Three Institutions. New York: Community College Research Center, Teachers College, Columbia University. Retrieved from <ccrc.tc.columbia.edu/publications/advising-technology-early-experiences-three-institutions.html>.
Kalamkarian, H. S., M. M. Karp, and R. Ganga. 2017. What We Know about Technology-Mediated Advising Reform. New York: Community College Research Center, Teachers College, Columbia University. Retrieved from <ccrc.tc.columbia.edu/publications/what-we-know-technology-mediated-advising-reform.html>.
Karp, M. M., and W. G. Stacey. 2013. What We Know about Nonacademic Student Supports. New York: Community College Research Center, Teachers College, Columbia University.
Mayer, A., H. S. Kalamkarian, E. B. Pellegrino, M. Boynton, and E. Yang. 2019. Integrating Technology and Advising: Studying Enhancements to Colleges’ iPASS Practices. New York: Community College Research Center (Columbia University, Teachers College) and MDRC. Available at <ccrc.tc.columbia.edu/media/k2/attachments/integrating-technology-advising-ipass-enhancements.pdf>.
Massey, D. S., and M. J. Fischer. 2005. Stereotype threat and academic performance: New findings from a racially diverse sample of college freshmen. Du Bois Review, 2(1): 45–67.
Pierce, L. 2016. The case for a case management approach in advising academically underprepared students. Academic Advising Today, 39(4).
Rankin, S. R., and R. D. Reason. 2005. Differing perceptions of campus climate by race/ethnicity and gender: A multiple perspective approach. Journal of College Student Development, 46(1): 43–61. Retrieved from <doi.org/10.1353/csd.2005.0003>.
Richardson, R. 2003. A case management approach to academic advising for minority students. The Mentor: An Academic Advising Journal. Retrieved from <doi.org/10.26209/MJ168581>.
Rucks-Ahidiana, Z., D. Miller, L. LeBlanc, E. Paasewe, C. E. Russo, and L. Barone. 2019. Understanding the Effects of a Performance-Based Scholarship Program for College Students in California. New York: MDRC.
Shapiro, D., A. Dundar, F. Huie, P. Wakhungu, A. Bhimdiwala, and E. Wilson. 2016. Completing College: A State-Level View of Student Completion Rates (Signature Report No. 12a). Herndon, VA: National Student Clearinghouse Research Center.
Shircliff, J., and R. T. Deil-Amen. 2020. The role of supplemental instruction and tutoring centers in the transformation of a campus. Learning Assistance Review, 8(2): 31–40.
United States Department of Education. 2016. First in the World Program. Washington, DC: Author. Retrieved from <www2.ed.gov/programs/fitw>.
Valentine, J., and D. Price. 2020. Improving Student Success through Proactive Coaching.
Indianapolis, IN: DVP-PRAIS Ltd. Available at <dvp-praxis.org/wp-content/uploads/2020/11/Success-Coach-Final-RCT-Results-November-2020.pdf>.
———. 2021. Improving Student Success through Proactive Coaching: A Case Study of Roanoke-Chowan Community College. Indianapolis, IN: DVP-PRAIS Ltd. Available at <dvp-praxis.org/wp-content/uploads/2021/2021-Brief-Roanoke_Web.pdf>.
Weiss, M. J., A. Ratledge, C. Sommo, and H. Gupta. 2019. Supporting community college students from start to degree completion: Long-term evidence from a randomized trial of CUNY’s ASAP. American Economic Journal: Applied Economics, 11(3): 253–297. Available at <doi.org/10.1257/app.20170430>.
Derek V. Price
Derek V. Price, Ph.D., is Principal and Founder of DVP-PRAIS LTD. Over the past sixteen years, his firm has evaluated projects funded by the U.S. Departments of Labor and Education, including the FITW Study that is the focus of this article, as well as private philanthropists such as Lumina Foundation, Kresge Foundation, Ford Foundation, and W.K. Kellogg Foundation. Price is a national leader in strategic thinking for institutional transformation and systems change and works on a broad range of policy and practice issues around post-secondary education and workforce development. Previously, he served as director of higher education research at Lumina Foundation and helped launch the national initiative, Achieving the Dream: Community Colleges Count. Dr. Price holds a Ph.D. in sociology from American University, a master’s in Russian and East European studies from the University of Michigan, and a bachelor’s in comparative area studies from Duke University.
Jessa L. Valentine
Jessa L. Valentine, Ph.D., is Director of Research, Evaluation, and Analytics at DVP-PRAIS LTD and brings more than fifteen years of research and evaluation experience in higher education and workforce development systems. Since 2015, Dr. Valentine has overseen data collection and analysis for the independent evaluation of First in the World grant, funded by the U.S. Department of Education, to measure the impacts of success coaching on students’ academic outcomes across ten North Carolina community colleges. Prior to joining DVP-PRAIS, she participated as a fellow with the U.S. Department of Education’s Training Program in the Education Sciences, and she was senior researcher at a Wisconsin-based think tank focused on regional higher education and workforce solutions. Dr. Valentine holds a Ph.D. in sociology from the University of Wisconsin-Madison, a master’s in Latin American studies from the University of Chicago, and bachelor’s degrees in economics and Spanish from Stanford University.
Alexander Leader
Alexander Leader is the creative and technical inspiration responsible for the Aviso Enterprise student success platform. As Founder and Chief Executive Officer, Leader guides the direction of Aviso Retention, ensuring that it meets the needs of Aviso’s institutional partners and continues to thrive in its identification of underserved students where colleges can make the biggest difference with their engagement. Prior to Aviso Retention, Leader partnered with the Ohio Board of Regents to build Ohio’s e-Transcript exchange system. This effort lowered barriers for student transfer across Ohio’s public and two-year institutions, ensuring that every student received the credit they deserved through Ohio’s articulation processes. These same institutions approached Leader in 2010 searching for a better way for advisors to prioritize their student outreach. Leader holds a bachelor’s degree from Miami University in Computer Science & Systems Analysis.
See how our tools are helping clients right now, get in-depth information on topics that matter, and stay up-to-date on trends in higher ed.
