Do Modern Difference-in-Differences Estimators Change Published Findings in Business Research?
How to prepare an accurate paper press kit? Illustrated guide
Davide Proserpio · arXiv: 2611.11003v1 · Summary page
Why cover this story this week?
When a new estimator changes the conclusion
Many policy studies compare outcomes before and after different groups adopt a treatment at different times. A common estimator, two-way fixed effects, can assign negative weights when treatment effects vary. A new preprint asks how much that problem matters in published business research, using language-model agents to help identify eligible analyses and then replicating and re-estimating 355 specifications from 40 papers.
The alternatives do not merely fine-tune the published results. Among comparisons passing a leverage screen, estimates change sign in 25.1 and 18.0 percent of comparisons for the respective modern estimators. Their standard errors are also larger. Including comparisons that fail the screen leaves the overall picture essentially unchanged. For readers outside econometrics, the practical message is that an apparently technical choice of estimator can change both a finding’s direction and its apparent precision.
Editorial context written by AI from abstracts, not independent scientific validation. The kit and its verified quotes rely on the source paper, not on this letter.
Talk about a paper in public without overstating it: yours, one you are asked to comment on, or one you write about as a journalist.
- Your paper: a draft release for your press office, the questions journalists will ask with what the paper lets you answer, the likely misreadings, a lay summary and a short thread.
- Asked to comment: what the paper shows and does not, its strengths and weaknesses with their passages, and what would settle its main claim.
- Writing about it as a journalist: reporting angles, the wording to check before publication, and the questions to ask the authors.
- For everyone: findings with their passages checked word for word, the novelty set against the closest earlier papers, a hype check, figures with their credit, and researchers for an outside view who have not signed with the authors.