System Prompts Affect Response Bias More Than Discriminability in LLM Safety
How to prepare an accurate paper press kit? Illustrated guide
Mohan W. Gupta, Sandy Tanwisuth · arXiv: 2611.11074v1 · Summary page
Why cover this story this week?
Stricter instructions are not necessarily better judgment
When a system prompt makes a language model refuse more requests, has the model become safer—or simply more reluctant? A controlled evaluation of 13 models used signal detection theory to separate the ability to distinguish safe from unsafe requests from the threshold for refusing them. Keeping benchmark questions fixed, the researchers varied the prompt family and its clarity. System prompts mainly moved the refusal threshold; changes in discrimination were smaller and less consistent.
Clearer instructions likewise increased willingness to refuse without reliably improving the underlying distinction. Prompts also shifted responses toward behaviors such as clarification and safe assistance, showing why a simple comply-or-refuse score is incomplete. For people deploying these systems, the implication is practical: a more restrictive assistant should not automatically be interpreted as a more discerning one.
Editorial context written by AI from abstracts, not independent scientific validation. The kit and its verified quotes rely on the source paper, not on this letter.
Talk about a paper in public without overstating it: yours, one you are asked to comment on, or one you write about as a journalist.
- Your paper: a draft release for your press office, the questions journalists will ask with what the paper lets you answer, the likely misreadings, a lay summary and a short thread.
- Asked to comment: what the paper shows and does not, its strengths and weaknesses with their passages, and what would settle its main claim.
- Writing about it as a journalist: reporting angles, the wording to check before publication, and the questions to ask the authors.
- For everyone: findings with their passages checked word for word, the novelty set against the closest earlier papers, a hype check, figures with their credit, and researchers for an outside view who have not signed with the authors.