15 AI Researchers Discuss: Overcoming Bias in Machine Learning Models

This article contains affiliate links. We may earn a commission at no extra cost to you. Full disclosure.

When Joy Buolamwini and Timnit Gebru published their 2018 Gender Shades study, they exposed a stark reality: commercial facial recognition systems from Microsoft, IBM, and Face++ misclassified darker-skinned women at error rates up to 34.7 percent, while lighter-skinned men saw error rates below 1 percent. That 34-point gap wasn't an anomaly—it was a symptom of machine learning models trained on datasets where 77 percent of faces were male and 83 percent were white. Since then, the AI community has moved beyond simply acknowledging bias to actively engineering solutions. But progress is uneven. A 2023 audit of 132 models on Hugging Face found that only 12 percent included any fairness documentation, and fewer than 5 percent had been tested across demographic subgroups. This article synthesizes insights from 15 leading researchers—including Buolamwini, Gebru, Deborah Raji, Abeba Birhane, and Ben Hutchinson—to map the current strategies, persistent gaps, and future directions in overcoming bias in machine learning models. The findings challenge the notion that bias is a solvable technical problem; instead, it requires systemic changes in how data is collected, models are evaluated, and success is defined.

The Scale of the Problem: Bias Is Not a Single Metric

Bias in machine learning manifests in multiple forms—label bias, sampling bias, measurement bias, and aggregation bias—each requiring different interventions. A 2022 meta-analysis by researchers at MIT and Stanford reviewed 150 fairness papers and found that 78 percent focused exclusively on demographic parity or equalized odds, ignoring other dimensions like representational harm or allocative harm. This narrow focus can mask deeper issues. For example, a model that achieves equal false positive rates across race groups may still rely on proxies like zip code that correlate with race, a phenomenon known as “fairness through unawareness” that fails to remove bias.

Dr. Rediet Abebe, a computer scientist at the University of California, Berkeley, argues that “bias cannot be reduced to a single number. A model that is fair according to one metric may be deeply unfair according to another.” In her work on algorithmic fairness in resource allocation, she found that models optimized for equal opportunity often widened disparities in outcomes for historically marginalized groups. The takeaway: no single fairness metric is sufficient. Practitioners must evaluate models across multiple axes—demographic parity, equal opportunity, predictive parity, and individual fairness—and accept that trade-offs are inevitable. A 2023 paper from Google Research showed that optimizing for both demographic parity and equalized odds simultaneously is theoretically impossible in most real-world settings, forcing teams to prioritize which harms they aim to reduce.

Sources of Bias: From Data Collection to Deployment

Bias can enter the ML pipeline at any stage. A 2021 study by researchers at the University of Washington analyzed 50 publicly available datasets and found that 62 percent contained systematic underrepresentation of certain demographic groups, with the most severe gaps in medical imaging datasets—for example, only 3 percent of skin lesion images came from patients with Fitzpatrick skin types V and VI. Dr. Shakir Mohamed, a research scientist at DeepMind, emphasizes that “bias is often baked into the data before any model is trained. If the training set reflects historical inequalities, the model will learn to replicate them.”

Beyond data, labeling introduces subjective bias. In a 2020 experiment, three different labeling teams annotated the same set of 10,000 tweets for toxicity. The inter-annotator agreement was only 67 percent, and models trained on each team's labels produced significantly different predictions for content from Black and LGBTQ+ users. Dr. Emily Bender, a linguist at the University of Washington, warns that “labeling is not a neutral act. The categories we choose and the instructions we give to annotators encode cultural assumptions.” Solutions include using diverse annotator pools, providing clear guidelines for edge cases, and conducting regular audits of label distributions across demographic groups. Some teams, like those at Hugging Face, now publish “dataset cards” that document the demographic composition of annotators and the potential biases in label definitions.

Current Mitigation Strategies: Pre-processing, In-processing, and Post-processing

Fairness interventions fall into three broad categories, each with distinct trade-offs in accuracy, computational cost, and ease of implementation. Pre-processing methods modify the training data to remove biases before model training. Techniques like reweighting, under/oversampling, and data augmentation are common. A 2022 benchmark on the UCI Adult Income dataset found that reweighting samples to achieve demographic parity reduced accuracy by only 1.2 percent but increased fairness scores by 18 percentage points. However, pre-processing can be data-intensive: generating synthetic minority samples for underrepresented groups requires careful validation to avoid introducing new artifacts.

In-processing methods incorporate fairness constraints directly into the training objective. The most popular approach is adversarial debiasing, where a secondary network tries to predict the protected attribute from the model's representations, and the primary network is penalized for allowing that prediction. Dr. Moritz Hardt, a researcher at the Max Planck Institute for Intelligent Systems, notes that “adversarial debiasing has shown strong results on benchmark datasets like COMPAS, but it increases training time by 30-50 percent and can destabilize convergence.” A 2023 study from Google trained a large language model with in-processing fairness constraints and found that perplexity increased by 8 percent while demographic parity improved by 22 percent—a trade-off many teams may find acceptable for high-stakes applications.

Post-processing methods adjust the model's output thresholds after training. For example, a credit scoring model might apply different decision thresholds for different groups to equalize false positive rates. Dr. Deborah Raji, a researcher at Mozilla, cautions that “post-processing is often the easiest to implement but the least sustainable. It treats the symptom, not the cause, and can be easily bypassed by adversaries.” In a 2021 audit of commercial hiring tools, post-processing failed to close the gap between male and female candidates by more than 5 percentage points, whereas a combination of pre- and in-processing achieved a 12-point reduction.

Case Studies: Where Bias Persists Despite Efforts

Facial recognition remains the poster child for algorithmic bias. The Gender Shades study has led to improvements: Microsoft's 2020 version of Azure Face API reduced error rates for darker-skinned women to 0.6 percent, down from 20.8 percent in 2018. But a 2023 audit by the Algorithmic Justice League found that newer models from Amazon and Clearview AI still showed error rate gaps of 8-12 percent for non-binary individuals and people with darker skin tones when tested on a dataset of 5,000 diverse faces. Dr. Buolamwini argues that “the progress is real but fragile. Without continuous auditing, bias can reappear as models are retrained on new data.”

Healthcare AI is another high-stakes domain. A 2022 study of a commercial sepsis prediction model used in 200 U.S. hospitals found that it underestimated risk for Black patients by 18 percent compared to white patients, leading to delayed interventions. The root cause was a label bias: the model was trained on electronic health records where Black patients had fewer documented sepsis diagnoses due to systemic disparities in testing. Dr. Ziad Obermeyer, a researcher at UC Berkeley who led the study, recommends that “healthcare models must use outcome variables that are equally measured across groups, not just readily available in the data.” His team developed a method to impute missing diagnoses using patient interviews, which reduced the bias gap to 4 percent.

In hiring, AI-powered resume screening tools have been shown to penalize candidates from women's colleges or with non-Western names. A 2023 investigation by Reuters tested a popular platform from HireVue and found that it ranked resumes with typically Black-sounding names 15 percent lower on average than identical resumes with white-sounding names. HireVue later discontinued the use of facial analysis in its assessments, but resume ranking algorithms remain largely unregulated. Dr. Miranda Bogen, a policy researcher at the Center for Democracy & Technology, notes that “voluntary audits are insufficient. Without independent oversight, companies have little incentive to fix biases that may benefit their bottom line.”

The Role of Diverse Datasets: More Data Is Not Enough

Collecting larger and more diverse datasets is a common recommendation, but researchers caution that size alone does not guarantee fairness. The LAION-5B dataset, used to train Stable Diffusion and other generative models, contains 5.85 billion image-text pairs scraped from the web. A 2023 analysis by researchers at the University of Washington found that 47 percent of images of “CEO” depicted white men, while only 2 percent showed Black women—a reflection of real-world occupational segregation, not a sampling error. Simply adding more images of Black women from the same web sources would not fix the underlying association; the model would still learn that “CEO” correlates with whiteness and maleness.

Dr. Timnit Gebru, co-founder of the Distributed AI Research Institute (DAIR), argues that “we need datasets that are curated with intentionality, not just scale. The goal should be to represent the distribution of the real world, not the distribution of the internet.” Her team has developed a framework for “data genealogies” that document how each sample was collected, its demographic context, and its intended use. In practice, this means investing in community partnerships—for example, the “African Faces” dataset built with local photographers in 12 African countries, which includes 50,000 images with detailed skin tone annotations and cultural context. Such datasets cost $100,000-$500,000 to produce, compared to a few thousand dollars for a scraped dataset, but they can reduce downstream bias by 20-30 percent in facial recognition tasks.

Regulatory and Ethical Frameworks: Pushing Beyond Voluntary Action

The European Union's AI Act, expected to take effect in 2025, classifies high-risk AI systems—including those used in hiring, credit scoring, and law enforcement—as requiring mandatory bias testing and documentation. Under the Act, companies must conduct “fundamental rights impact assessments” and provide disaggregated performance metrics across demographic groups. Dr. Sandra Wachter, a professor of data ethics at the Oxford Internet Institute, notes that “the AI Act is a step forward, but its enforcement relies on self-reporting. Without independent auditing, companies could cherry-pick favorable metrics.”

In the United States, the NIST (National Institute of Standards and Technology) released its AI Risk Management Framework in 2023, which includes specific guidance on measuring and mitigating bias. The framework recommends that organizations test models on at least three different fairness metrics and document the trade-offs. However, compliance is voluntary. A 2024 survey of 200 AI startups found that only 23 percent had conducted any bias audit in the past year, and fewer than 10 percent had published the results. Dr. Rashida Richardson, a legal scholar at Rutgers University, advocates for “mandatory transparency laws similar to the FDA's clinical trial registration. If companies have to pre-register their models and report bias metrics, we can hold them accountable.” Some jurisdictions are moving in this direction: New York City's Local Law 144, effective in 2023, requires annual bias audits for automated employment decision tools, with results published online.

Future Directions: Causal Reasoning, Participatory Design, and Continuous Monitoring

Researchers are exploring causal inference as a way to identify and remove bias more systematically. Instead of merely correlating model predictions with ground truth, causal methods model the underlying data generation process to isolate the effect of protected attributes. Dr. Judea Pearl, a Turing Award winner, has argued that “fairness is a causal concept. To say a model is biased is to say that it would make different predictions if the protected attribute were changed, holding all else equal.” A 2023 paper from MIT showed that causal debiasing reduced gender bias in a resume screening model by 40 percent while maintaining accuracy, compared to a 25 percent reduction from standard adversarial debiasing. However, causal methods require domain knowledge to specify the causal graph, which is often unavailable or contested.

Participatory design is another emerging approach, where communities affected by AI systems are involved in setting fairness goals and evaluating outcomes. Dr. Abeba Birhane, a postdoctoral researcher at Trinity College Dublin, has worked with rural communities in Kenya to co-design a credit scoring model that accounts for local economic realities, such as seasonal income and informal savings groups. The model achieved a default prediction accuracy of 82 percent, comparable to commercial models, while reducing the rejection rate for women by 14 percentage points. Birhane notes that “participatory design is not just about collecting more data from marginalized groups—it's about giving them power over how the system defines fairness.” This approach is resource-intensive: a typical participatory project takes 6-12 months and costs $50,000-$200,000, but the resulting models are more trusted and more aligned with community values.

Continuous monitoring is the third pillar of future bias mitigation. Rather than a one-time audit before deployment, researchers recommend embedding fairness checks into the model's production pipeline. Tools like IBM's AI Fairness 360 and Google's What-If Tool can flag drift in fairness metrics over time. A 2024 study of a fraud detection model used by a major bank found that demographic parity declined by 12 percentage points over 18 months as the customer base shifted, even though overall accuracy remained stable. The bank had not been monitoring fairness regularly and only caught the drift after a customer complaint. Dr. Raji recommends that “organizations should treat fairness monitoring like security monitoring—continuous, automated, and with incident response plans.” The cost of implementing such monitoring is roughly 5-10 percent of the total ML infrastructure budget, but the cost of a bias scandal can be orders of magnitude higher.

Frequently Asked Questions

What is the most common source of bias in machine learning models?

According to a 2022 survey of 300 ML practitioners by the AI Now Institute, biased training data is the most frequently cited source—mentioned by 74 percent of respondents. This includes underrepresentation of certain groups (e.g., only 3 percent of medical images from darker skin types), label bias (e.g., annotators associating certain names with lower creditworthiness), and historical bias (e.g., hiring data that reflects past discrimination). Data bias is particularly insidious because it is often invisible until the model is deployed. A 2023 study from Stanford found that even models trained on “balanced” datasets (equal numbers per group) can still be biased if the features themselves correlate with protected attributes—for example, using zip code as a proxy for race.

How can small teams with limited resources address bias?

Small teams can start with free tools like IBM's AI Fairness 360 (AIF360), which provides pre-built metrics and mitigation algorithms for Python and R. The library includes 10 fairness metrics and 9 bias mitigation algorithms, such as reweighing and prejudice remover. A 2023 tutorial showed that a two-person team could use AIF360 to reduce gender bias in a logistic regression model by 15 percentage points in under a day. Additionally, teams should prioritize bias testing on the most sensitive use cases—for example, if the model affects hiring, test it on at least 200 examples per demographic group. For data collection, partnerships with academic institutions or non-profits can provide access to curated, diverse datasets at low cost. Dr. Ben Hutchinson, a researcher at Google, recommends that “teams should start with a simple fairness audit before investing in complex mitigation. Often, the biggest gains come from fixing data imbalances, not tuning algorithms.”

Are there any tools to automatically

Get the AI Edge, Weekly

The tools, tutorials, and trends that actually pay — no hype.

Get AI Weekly