Heterogeneous effects in education 🔄

TL;DR

Synthesizes the methodological literature on approaches to capture heterogeneous effects as a methodological advancement to examining intersectionality in education, challenges for implementation, and considerations to overcome those challenges.

Summary

The paper by Lauren Schudde discusses the incorporation of intersectionality into quantitative research methods in education. Intersectionality theory, which posits that various dimensions of identity (race, class, gender, sexuality) intersect to shape individual experiences and social inequalities, has traditionally guided qualitative research. Schudde argues that quantitative approaches can also capture the varied impacts of identities on educational outcomes, thus moving beyond singular dimensions of individual characteristics.

However, this introduces four critical challenges for researchers using quantitative methodologies:

• First is where data with adequate sample sizes could be acquired as there are few opportunities in education to sample populations large enough to withstand sample size reductions capable of modeling individuals representative of these intersections.
• The second is whether to include interaction effects, as interaction terms make models more challenging to interpret.
• Third is which interaction effects to include as theory is needed to support, interpret, and assess covariates within and test a statistical model for competing explanations for differential effects.
• Even if the researcher chooses to include interaction effects and uses theory to support the analysis, the fourth challenge becomes how to present the findings beyond simple population-level statistics.

Schudde synthesizes some statistical methods used to examine intersectionality and discusses the potential for education researchers to learn from other fields, developing a crosswalk between the literature on heterogeneous effects (HE) and intersectionality. The paper outlines the commonalities between HE and intersectionality, noting that both anticipate that background characteristics, skills, and group memberships influence individuals' experiences and the effects of those experiences.

Despite this alignment, the connection between quantitative HE research and intersectional theory has received little attention. Schudde calls for greater dialogue between quantitative and qualitative researchers to enrich both approaches, suggesting that incorporating intersectionality could strengthen the toolkit for examining HE and support theory for statistical models.

This got me thinking...

What are some pros (+) and cons (-) of different statistical methods used to evaluate intersectionality?

• Generic Intersectionality:

  • (+) Addresses issues that affect 'everybody,' not only marginalized groups.

  • (-) No attention to power and marginality. 

• Pan Equality:

  • (+) It avoids deciding in advance which issues affect which social groups and

  • (+) Enables more structural understandings of intersectionality.

  • (-) Disregard differences and prevent work on issues that are not "common."

• Cross-tabulations:

  • (+) Provides a clear visual representation of data distribution across intersectional categories.

  • (-) It may not be suitable for continuous variables. 

  • (-) It can become unwieldy with multiple categories and intersections.

• Main Effects Regression:

  • (+) Simple to implement with existing statistical software. 

  • (+) Can identify the main effects of each category on the outcome.

  • (-) It does not allow for effects to vary across intersections. 

  • (-) Treats effects of social identities/positions as an additive, a misapplication of intersectionality.

• Regression with Interactions:

  • (+) Allows for heterogeneity across intersections.

  • (+) Can capture interaction effects between different social categories.

  • (-) It is more complex to interpret. 

  • (-) Requires larger sample sizes to detect interaction effects.

• MANOVA:

  • (+) Can assess multiple dependent variables simultaneously. 

  • (+) Can handle the effects of multiple independent variables and their interactions.

  • (-) It is complex and requires assumptions about the distribution of variables. 

  • (-) Not specifically tailored to intersectional analysis without proper model specification.

• Stratification: The method aligns with the concept of Heterogeneous Effects.

  • (+) Can analyze data within strata to observe effects within specific intersectional groups.

  • (+) Groups individuals into more homogeneous strata based on specific characteristics.

  • (+) Allows for estimating treatment effects within each stratum, revealing variation across different subgroups while avoiding exhausting degrees of freedom.

  • (+) Can detect patterns of treatment heterogeneity across the strata.

  • (-) This may lead to many strata with sparse data, making it difficult to conclude.

  • (-) This may lead to small sample sizes within strata, making it difficult to draw robust conclusions.

  • (-) Risk of bias if stratification is done endogenously, such as using outcomes to create strata.

I would love feedback or to hear your thoughts!

About Cuvette Collective:

Cuvette Collective represents a community of educators and scientists dedicated to empowering the societal impact of STEM research.

We provide tailored tools, strategies, and insights to help scientists craft authentic, impactful broader impact plans that align with NSF priorities and their research focus.

Our engaging blog posts, interactive resources, and personalized consulting services support researchers in leveraging their scientific expertise to make a meaningful difference in society.

How to get involved:

1️⃣ Subscribe to our blog for the latest tips and insights.

2️⃣ Apply to participate in the Broader Impacts Excellence Program.

3️⃣ Reach out and let us know how we can help to enhance your broader impacts efforts.