Using Data to Put Student Support at the Center of the Academic Alert Process
A few years ago, inspired by what she learned participating in the EDUCAUSE Data Literacy Institute, Tanya Brown began to dig into the data from the student success platform she administers for Arapahoe Community College. Right away, she had three concerning observations: When faculty used the system, “recommend withdraw” was the second most common alert they sent to advisors; they used that alert disproportionately for students of color; and, in general, the alerts didn’t do enough to facilitate conversations about getting students the support they needed to succeed.
Those insights stemmed in part from prioritizing disaggregated data at the college. Brown is the Director of Strategy and Analytics in the Division of Student Affairs, and she and her colleagues had been talking about the need to disaggregate data to better understand how students in different populations were doing and how student support professionals serve them.
In fact, Brown says, “Our Vice President of Student Affairs has told all the directors of departments that no data should come out that is not disaggregated.”
Arapahoe Community College uses EAB Navigate, a student success platform that provides a coordinated care model for scheduling, degree-path planning, academic support intervention, and communication. One way the school uses it is for instructors to send alerts such as “stopped attending” with automated messages to advisors in Student Affairs. Administrators like Brown at individual institutions customize these alerts and the workflow.
The data disaggregated by race, ethnicity, and gender that Brown could see in the platform showed that a disproportionate number of “recommend withdraw” alerts were being used for students of color. That produced “one of those moments for me that I saw something needs to be done,” she says. “As a community college, that is not what we do. We're open access. We support everyone who walks through our door. So I really started to dig into the numbers, reevaluating the process I had implemented.”
Too blunt
The alert workflow at Arapahoe Community College was set up so only advisors and other professional staff saw the “recommend withdraw” language. Individual students received a supportive message saying their instructor mentioned a concern and that an advisor would be in touch to schedule an appointment, as well as outlining resources available to them.
Part of the problem was that the process for academic alerts had grown unnecessarily blunt. While some students were failing and were unlikely to pass, there are others for whom “recommend withdraw” was too broad a brush. It wasn’t common to include more information in the optional comment field, so advisors didn’t have the context they needed to bring their expertise and the college’s resources to bear and possibly help the student.
Brown knew the “recommend withdraw” alert was often inappropriate because she could see in the data that some students didn’t follow the recommendation and still went on to pass the class and graduate. “The student was able to regain their momentum,” she says.
What was unknown was the in-between cases — students who did withdraw and might not have had a conversation about the possibility of regaining their momentum. Brown and her colleagues wondered what effect the language of “recommend withdraw” had on the advisors working with those students. “Were those words creating a bias?” she asks. “Was that triggering ‘Well, they're supposed to withdraw from the class, so this is what I'm going to tell them to do’ instead of finding ways to support the student?”
Meanwhile, Brown could also also see that some departments were using the “recommend withdraw” alert more than others: “It did identify a systemic problem that we had at the college.”
Too many
A related issue is that the academic alert system had grown more complex over time in ways that weren’t helpful. Faculty could select from 28 different alerts, many of them duplicates, and many of them about issues advisors couldn’t help with.
In the spring of 2022, after her data analysis and conversation with faculty and other colleagues, Brown was ready to overhaul the academic alert system. The most apparent change was replacing the 28 alerts with just two — “recommend advising” and “recommend tutoring” — and faculty were encouraged to use the comment field to give more context.
Behind the scenes, Brown and her advising colleagues in Student Affairs were collaborating with the director of tutoring services and others in the Academic Affairs division to develop a new workflow. The alerts would be sent to the appropriate office, which would get in touch with the student directly to learn more about their issue.
The goal was to connect the student with supports that would enable them to persist and graduate. A conversation with a student might lead to them withdrawing from a course, but for many more the conversation focused on accessing the college’s existing resources such as individual tutoring, group tutoring, online tutoring, the writing center, disability support, counseling, TRIO grant-funded support services, or emergency funds.
“Sometimes in those conversations,” Brown says, “we find things like, ‘I just was told that I'm going to be evicted,’ or ‘I've been looking for a job.’ That's a different conversation from ‘This just isn't the class for me.’ This changes the structure to really think through what is the most appropriate connection point for that student to be able to pass the class.”
Framing matters
The hope of this reform of the student success platform implementation is that without the priming of “recommend withdraw,” support professionals and faculty around the college will be engaged in conversations about what will help students succeed and that at least one systemic contribution to equity gaps will be removed. The college hasn’t had these changes in place long enough to see results in the data, but Brown says the institution is benefiting in several ways in the meantime.
For example, some of the comments faculty include in the alerts are about a student sleeping in class or a student being disruptive. The former has started conversations about classroom management and the limits of what advisors, counselors, and tutors can help with. The latter raises the need to clarify when the Dean of Students should be involved.
As the 2023-24 academic year approached, Brown and her Student Affairs colleagues developed a rubric to guide the instructor and the advisor on effective uses of the alert system and support process, and that rubric is being introduced as part of the college’s pre-term professional development week, along with other training on using the student success platform and related data.
Brown is watching for other trends in the comment fields in the alert process, one of which is about students not engaging with the required courseware. She says this shows the need to ensure courseware products are affordable and accessible, and also that students are familiar with using them.
“If COVID taught us anything, it's that not everyone has the technology at home to access what they need to,” she says. “It’s important to go through the courseware with them in class, and if a student has a concern, be open and listen so they can be more successful in the course.”
Another realization is that a student success platform has some limitations to disaggregating data. For example, for confidentiality reasons, Brown can’t identify which students are eligible for Pell grants, which is a proxy for being low income. She now has a process for asking the financial aid office to tell her what number and percentage of students in a set she is analyzing are Pell eligible.
One of the biggest takeaways is the importance of language choices throughout the alert process. “We have become very intentional about the language we use in the platform,” Brown says. “We no longer call a student ‘at risk.’ We emphasize support and getting the student to the next goal. We've tried to be cognizant of not planting a bias in the users of the platform. We’re working with EAB to make sure that if the language is something we can customize on Navigate, we do.”
Collaboration creates opportunities
The uneven use of “recommend withdraw” across departments has led to more conversations between Student Affairs and the academic deans, Brown says: “I didn’t want to call out any one group, so I provided the data I'm seeing, the courses I'm seeing it in, and asked if there is anything I am missing. It did lead to conversations about professional development to support our students.” One way has been reintroducing the resources of the Arapahoe Community College Inclusive Excellence Council.
The initial work to reform the use of alerts required teamwork between advising and tutoring, and those relationships are resulting in new creative collaboration.
For example, Brown says, “Advisors and tutors are starting to look at the data to ask ‘What else can we do?’ They’re thinking about workshops on writing résumés or writing scholarship essays. They're looking for ways to work together to serve more students with their expertise. It's become an effort to create more opportunities.”
The data in the academic alert system ultimately alerted the institution itself to opportunities for continuous improvement, but Brown believes this process is bringing out the best in Arapahoe Community College. “We have many departments really working with students to help them get through gateway courses, because they understand that is a prerequisite to other courses down the road,” she says.
“Every higher education institution has some of these challenges,” Brown explains. “What’s important is how you respond to support the student when you identify a challenge. How you address the challenge says a lot about the institution, and I’m proud of the work we have done to support our common goal of student success.”