Prioritize Disaggregated Data When Selecting Courseware
Higher education is experiencing a growing tide of calls for data that is disaggregated by race, ethnicity, economic circumstances, and other factors relevant to the student experience. Equity advocates like Estela Mara Bensimon and organizations like the American Council on Education are making the case that too much data and discourse in higher education bundles together minoritized students and students from low-income backgrounds in ways that are counterproductive and misleading.
Aggregated data reinforces notions of “typical” students—notions that privilege the experiences of middle-class white students and erase the experiences, needs, and strengths of students who are racially or ethnically minoritized, who come from low-income backgrounds, or who are first-generation. Likewise, aggregated data obscures other important facets of identity that matter to students and that relate to the barriers to equity they might face, including sexual orientation and gender identity, immigration status, home language, or religion.
While working on the report Toward Ending the Monolithic View of “Underrepresented Students”: Why Higher Education Must Account for Racial, Ethnic, and Economic Variations in Barriers to Equity for Every Learner Everywhere, I learned from students who shared their experiences with me and explained that disaggregated data can
- illuminate differences within groups;
- illuminate what is and isn’t effective to support specific groups of students;
- elevate the voices and experiences of individual students;
- empower faculty to refine teaching practices and develop data-informed partnerships across their institutions; and
- empower institutions to confront the effects of systemic inequities.
The students clearly stated that the equity barriers they encounter in higher education vary widely and emerge in unanticipated and idiosyncratic ways. Those barriers include racial battle fatigue, cultural mismatch, imposter syndrome, stereotype threat, racist attacks, student debt, “colorblind” pedagogy, information gaps, inflexible institutional policies, and limited access to technology and other resources. But students’ experiences of those barriers are not easily bundled and characterized in ways that meaningfully apply to all “underrepresented” students.
Often, minoritized students and students from low-income backgrounds do not have the equitable learning barriers that they are assumed to have. For example, first-generation status is often conflated with low-income and conflated again with race in ways that are misleading. Incorrectly aggregating Black or Latino/a students with first-generation students erases the support networks and cultural wealth the former can draw on and soft-pedals how racism inside the institution remains a factor in their learning experience.
If higher education is committed to equity for “underrepresented” students, institutions must stop homogenizing them as one monolithic group and end the dependency on aggregated data. This must happen both at the broader sector level and at the local institutional level. Nationwide data, even when it is disaggregated, is only a starting prompt for inquiry about what happens within an institution. For example, data about the experiences of Latino/a students at four-year colleges in Florida is unlikely to predict the experiences of the growing population of Latino/a students at community colleges in Iowa.
Heterogeneity Within Minoritized Student Populations
Even when data is disaggregated by race, ethnicity, or income, it can still obscure significant intergroup heterogeneity. For example:
- Almost 20% of the U.S. population identifying as Black are immigrants or the children of immigrants.
- Among Latino/a students (who can have roots in more than 15 different countries), college enrollment rates range between 27% and 55%.
- Many Indigenous students are intertribal, meaning they have ancestry in or citizenship in more than one tribe, and some identify as multiracial.
- While Asian American students are more likely than White students to complete college (up to 56% for Korean Americans), among the fastest-growing Asian American groups—Hmong and Cambodian, for example—less than 20% have college degrees.
Each of these populations have intergroup variations in community support, language practices, religion, racial identity, colorism, and experiences with immigration and colonialism. Those variations will contribute to the learning barriers that higher education presents to them and to how they experience those barriers.
As a chemistry professor originally from the Philippines told me, “I went to UCLA, and I saw Asian people, but they didn’t have two parents working graveyard shifts. They were second- and third-generation UCLA students. That was my first exposure to what it means to aggregate communities and to the perception that it matters where you’re coming from.”
Because disaggregated data will take an institution only so far toward illuminating intergroup heterogeneity, data practices must be part of an institutional culture of understanding individual experiences, needs, and assets. Higher education must create mechanisms for hearing and learning about individual students’ voices.
Courseware to Support Analyzing Disaggregated Data
Prioritizing disaggregated data is particularly important as digital courseware becomes a more common part of the higher education experience. The potential benefit of digital learning technologies is that they give faculty, instructional designers, academic support professionals, and administrators powerful data to inform targeted and timely interventions and to inform programmatic and institutional strategy.
However, that data can also powerfully mislead if it reinforces discourse about “typical students.” A program review that shows the implementation of a new courseware resulted in a 5% improvement in course outcomes for all students doesn’t help much to illuminate and confront equity barriers for specific student populations.
Digital courseware is more likely to be used in high-enrollment gateway courses with high DFWI rates that disproportionately impact minoritized students and students from low-income backgrounds. That’s why several of the criteria on CourseGateway refer to disaggregated data. For example, independent reviewers of courseware on CourseGateway consider the following categories:
- Outcomes: Research is transparent and includes student outcomes evidence by race and income (e.g., case studies, quasi-experimental studies, experimental studies).
- Evidence across institution types: Research is conducted across a diversity of institutional settings, and outcomes data is disaggregated across those institutional setting types.
- Data as support: Data is used in a positive and responsible way to promote data equity and ethical data uses.
- Privacy, data privacy, and ownership: Students’ data is protected, and students can decide how their data is to be shared, with no gap for Black, Latino/Latina, Indigenous students, and students from low-income backgrounds.
Other criteria are less explicit about disaggregating data, but by inference, if a program objective is to close equity gaps for particular student populations, disaggregating assessment data will be necessary:
- Measurement and structure: Learning can be assessed in relation to learning objectives/competencies.
Courseware can help address what one literature review about digital learning and equity called the “black box” problem: Institutions may have disaggregated data about the inputs (high school graduation, high school test scores, and the demographic profile of the admitted student body), and they may have disaggregated data about the outputs (graduation and career results), but they rarely have data about what goes on in the classrooms in between. Knowledge about which digitally enabled teaching practices work and for whom they work is obscured.
Disaggregating student data by race, ethnicity, or income is by no means easy. Doing so involves collaboration with institutional research and information technology colleagues as well as with courseware vendors. Ultimately, however, institutions aren’t working with meaningful student data if the data isn’t about the institution’s actual students. Making progress on equitable teaching and learning depends on confronting this challenge.