# The Impossibility of Fair LLMs

**Authors:** Jacy Anthis, Kristian Lum, Michael Ekstrand, Avi Feller, Alexander D'Amour, Chenhao Tan  
**arXiv:** [2406.03198](https://arxiv.org/abs/2406.03198)  
**Date:** 2024-05-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.03198v1/

## Key points

- The need for fair AI is evident in the era of general-purpose systems like ChatGPT and Gemini.
- Machine learning researchers have developed technical frameworks for evaluating fairness, such as group fairness and fair representations.
- Guidelines have been proposed to achieve fairness in specific use cases, emphasizing context, developer responsibility, and stakeholder participation.
- Large language models (LLMs) present challenges for fairness evaluation due to diverse populations, sensitive attributes, and varied use cases.
- Recent research on LLM fairness focuses on association-based metrics and practical challenges rather than nuanced metrics.
- Three general guidelines are proposed for addressing challenges posed by LLMs: considering context critically, emphasizing developer responsibility, and engaging in iterative participatory design processes.
- Interest in LLMs has surged since 2020 with models like GPT gaining popularity, leading to studies exploring bias and discrimination in LLM-generated text across various domains.

## Summary

In the era of general-purpose systems like ChatGPT and Gemini, the need for fair AI is becoming increasingly evident. However, as human-AI interactions become more complex and their social impacts more pronounced, questions arise about how fairness standards can be effectively applied. Machine learning researchers have developed technical frameworks to evaluate fairness, such as group fairness and fair representations. However, applying these frameworks to large language models (LLMs) presents inherent limitations due to the multitude of populations affected, sensitive attributes involved, and diverse use cases. To address these challenges, guidelines have been proposed for achieving fairness in specific use cases. These guidelines emphasize the importance of context and highlight the responsibility of LLM developers in promoting fairness. They also stress the need for stakeholder participation in the design and evaluation process. Before delving into recent work on LLM fairness, it's important to consider key features of LLMs that impact fairness evaluation. LLMs offer exceptional flexibility with their ability to handle a wide range of content in natural language and even multimodal inputs like text and images. At a social level, there are diverse stakeholders involved in LLM systems with evolving relationships - from dataset creators to end-users to researchers analyzing societal impacts. Recent research on LLM fairness has focused on association-based metrics and practical challenges rather than nuanced metrics. This highlights a fundamental logical mismatch between existing frameworks and modern LLM systems. The flexibility of LLMs across data, tasks, stakeholders, and populations makes guaranteeing a fair LLM impractical. Moving forward, three general guidelines are proposed: considering context critically, emphasizing developer responsibility,and engaging in iterative participatory design processes.These guidelines aim to address the challenges posed by large language models while promoting ethical AI development practices. Interest in LLMs has surged since 2020 with models like GPT gaining popularity. Recent studies have explored bias and discrimination in LLM-generated text across various domains such as financial lending predictions or criminal justice recidivism analysis. As generative AI continues to advance, addressing bias and promoting fairness in large language models remains a critical area of research focus for ensuring ethical AI development practices.