Invisibilization as a Source of Probability Bias in Language Models
How to prepare an accurate paper press kit? Illustrated guide
Christian Arnold, Ernesto Calvo, Guadalupe Gonzalez, Sebastián Vallejo Vera, Tiago Ventura · arXiv: 2610.30785v1 · Summary page
Why cover this story this week?
When caution makes evidence disappear
An effort to avoid racial bias can produce another kind of unequal treatment, according to a preprint built around a probability puzzle with an exact Bayesian answer. Across 6,600 sessions, researchers asked models from OpenAI and Anthropic how much to trust an unverifiable report, varying the reporter’s description. At the widest observed gap, models declined to assess a Black-described reporter 46% of the time, compared with 9% for a white-described reporter. Yet among the reports they did assess, the Black-described reporter received slightly more trust.
That distinction matters for auditing: comparing only assessed cases, or simply averaging trust, can conceal unequal rates of discarding information. The researchers also found that the effect depended on wording and could worsen with greater reasoning effort or warning instructions. A more encouraging result emerged when models were scored against revealed truth over repeated rounds: in four of five tested configurations, the racial gap in trust fell below 0.02 by the final rounds. This is evidence from a controlled game, but it exposes a broader measurement problem: fairness checks must examine what a system refuses to evaluate, not just the judgments it returns.
Editorial context written by AI from abstracts, not independent scientific validation. The kit and its verified quotes rely on the source paper, not on this letter.
Talk about a paper in public without overstating it: yours, one you are asked to comment on, or one you write about as a journalist.
- Your paper: a draft release for your press office, the questions journalists will ask with what the paper lets you answer, the likely misreadings, a lay summary and a short thread.
- Asked to comment: what the paper shows and does not, its strengths and weaknesses with their passages, and what would settle its main claim.
- Writing about it as a journalist: reporting angles, the wording to check before publication, and the questions to ask the authors.
- For everyone: findings with their passages checked word for word, the novelty set against the closest earlier papers, a hype check, figures with their credit, and researchers for an outside view who have not signed with the authors.