How Different are Different Differences-in-Differences? A Large-Scale Meta-Analysis
Auteurs : Peter Hull
Résumé scientifique
I use AI-assisted replication to assess the empirical importance of negative weights in two-way fixed effects (TWFE) estimation, drawing on 1,346 staggered-adoption specifications from 108 economics papers. TWFE estimates are highly correlated with estimates from modern difference-in-differences methods, with observed sign reversals in 10.8-14.5% of comparisons and greater precision for TWFE. Hierarchical models that account for estimation noise imply a 91-98% correlation between estimands and a 0.6-2.1% probability of obtaining a statistically significant TWFE estimate with the opposite sign from the modern estimand. These findings suggest negative weights may be a less serious problem in practice, and illustrate how AI can support large-scale empirical assessments of new econometric methods.
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
When Can We Use Two-Way Fixed Effects (TWFE)? A Comparison of TWFE and Novel Dynamic Difference-in-Differences EstimatorsTobias Rüttenauer, O. Aksoy · 2026 · Sociological methodology · Pas encore résumé
Cette comparaison par simulations présente les biais possibles de TWFE et les limites de cinq estimateurs dynamiques de différences-en-différences.
Résumé scientifique
The conventional two-way fixed effects (TWFE) estimator has come under increasing scrutiny in settings with staggered treatment adoption. Recent research shows that TWFE can be biased when treatment effects vary across time or groups, prompting the development of several alternative dynamic difference-in-differences (DiD) estimators. Yet for applied researchers, it often remains unclear when TWFE is biased, what the newer estimators solve, and what limitations they all have. The authors provide an accessible overview of this literature and compare conventional TWFE with five dynamic DiD estimators using Monte Carlo simulations. The simulations consider realistic scenarios with time- and grou…
-
Two-Way Fixed Effects Estimators with Heterogeneous Treatment EffectsC. de Chaisemartin, Xavier D'Haultfœuille · 2018 · The American Economic Review · cité 4 281 fois · Pas encore résumé
Cet article établit que TWFE peut attribuer des poids négatifs aux effets de traitement hétérogènes, problème évalué empiriquement ici.
Résumé scientifique
Linear regressions with period and group fixed effects are widely used to estimate treatment effects. We show that they estimate weighted sums of the average treatment effects (ATE ) in each group and period, with weights that may be negative. Due to the negative weights, the linear regression coefficient may for instance be negative while all the ATEs are positive. We propose another estimator that solves this issue. In the two applications we revisit, it is significantly different from the linear regression estimator. (JEL C21, C23, D72, J31, J51, L82)
-
Two-way fixed effects, the two-way mundlak regression, and difference-in-differences estimatorsJeffrey M. Wooldridge · 2025 · Empirical Economics · cité 573 fois · Pas encore résumé
Cette étude relie TWFE à une régression Mundlak et à une approche POLS pour autoriser l’hétérogénéité des effets.
Résumé scientifique
I derive a result on the equivalence between the two-way fixed effects (TWFE) estimator and an estimator obtained from a pooled ordinary least squares regression that includes unit-specific time averages and time-period-specific cross-sectional averages—the two-way Mundlak (TWM) regression. The equivalence between TWFE and TWM implies that various estimators used for intervention analysis can be computed using pooled OLS that controls for time-constant treatment cohort indicators, time-period indicators, covariates, and interactions among them—allowing for considerable treatment effect heterogeneity. An extended version of TWFE (ETWFE) is equivalent to the POLS approach. I show that an imput…
-
Two-stage differences in differencesJ. Gardner · 2022 · cité 231 fois · Pas encore résumé
Cette étude propose une estimation en deux étapes pour identifier les effets moyens lorsque l’adoption est échelonnée et les effets hétérogènes.
Résumé scientifique
A recent literature has shown that when adoption of a treatment is staggered and average treatment effects vary across groups and over time, difference-in-differences regression does not identify an easily interpretable measure of the typical effect of the treatment. In this paper, I extend this literature in two ways. First, I provide some simple underlying intuition for why difference-in-differences regression does not identify a group × period average treatment effect. Second, I propose an alternative two-stage estimation framework, motivated by this intuition. In this framework, group and period effects are identified in a first stage from the sample of untreated observations, and average treatmen…
-
What’s trending in difference-in-differences? A synthesis of the recent econometrics literatureJ. Roth, Pedro H. C. Sant'Anna, A. Bilinski et al. · 2022 · Journal of Econometrics · cité 1 782 fois · Pas encore résumé
Cette synthèse classe les avancées en différences-en-différences, notamment celles sur les périodes multiples et le calendrier variable du traitement.
Résumé scientifique
This paper synthesizes recent advances in the econometrics of difference-in-differences (DiD) and provides concrete recommendations for practitioners. We begin by articulating a simple set of ``canonical'' assumptions under which the econometrics of DiD are well-understood. We then argue that recent advances in DiD methods can be broadly classified as relaxing some components of the canonical DiD setup, with a focus on $(i)$ multiple periods and variation in treatment timing, $(ii)$ potential violations of parallel trends, or $(iii)$ alternative frameworks for inference. Our discussion highlights the different ways that the DiD literature has advanced beyond the canonical model, and helps to…
-
Interpreting event-studies from recent difference-in-differences methodsJon Roth · 2024 · Japanese Economic Review · cité 33 fois · Pas encore résumé
Cette note compare l’interprétation des graphiques d’étude d’événement des méthodes récentes à ceux de TWFE.
Résumé scientifique
This note discusses the interpretation of event-study plots produced by recent difference-in-differences methods. I show that even when specialized to the case of non-staggered treatment timing, the default plots produced by software for several of the most popular recent methods do not match those of traditional two-way fixed effects (TWFE) event-studies. The plots produced by the new methods may show a kink or jump at the time of treatment even when the TWFE event-study shows a straight line. This difference stems from the fact that the new methods construct the pre-treatment coefficients asymmetrically from the post-treatment coefficients. As a result, visual heuristics for evaluating vio…
-
Causal Panel Analysis under Parallel Trends: Lessons from a Large Reanalysis StudyAlbert Chiu, Xingchen Lan, Ziyi Liu et al. · 2023 · American Political Science Review · cité 48 fois · Pas encore résumé
Cette réanalyse compare TWFE à six estimateurs robustes à l’hétérogénéité dans des études causales sur données de panel.
Résumé scientifique
Two-way fixed effects (TWFE) models are widely used in political science to establish causality, but recent methodological discussions highlight their limitations under heterogeneous treatment effects (HTE) and violations of the parallel trends (PT) assumption. This growing literature has introduced numerous new estimators and procedures, causing confusion among researchers about the reliability of existing results and best practices. To address these concerns, we replicated and reanalyzed 49 studies from leading journals that employ TWFE models for causal inference using observational panel data with binary treatments. Using six HTE-robust estimators, diagnostic tests, and sensitivity analy…
-
Negative Weights are No Concern in Design-Based SpecificationsK. Borusyak, P. Hull · 2024 · Social Science Research Network · cité 11 fois · Pas encore résumé
Cet article montre que les poids négatifs ne posent pas problème dans certaines spécifications fondées sur le plan d’étude.
Résumé scientifique
Recent work shows that popular partially-linear regression specifications can put negative weights on some treatment effects, potentially producing incorrectly-signed estimands. We show this is not an issue in design-based specifications, in which low-dimensional controls span the conditional expectation of the treatment. Specifically, the estimands of such specifications are convex averages of causal effects with ex-ante weights that average the potentially negative ex-post weights across possible treatment realizations. This result extends to design-based instrumental variable estimands under a first-stage monotonicity condition and applies to formula treatments and instruments such as shi…
Les travaux antérieurs sur lesquels il s’appuie
-
Difference-in-Differences with multiple time periodsBrantly Callaway, Pedro H. C. Sant'Anna · 2018 · Journal of Econometrics · cité 7 276 fois · Pas encore résumé
Cette étude traite l’identification et l’estimation en différences-en-différences avec plusieurs périodes et des dates de traitement variables.
Résumé scientifique
Difference-in-Differences (DID) is one of the most important and popular designs for evaluating causal effects of policy changes. In its standard format, there are two time periods and two groups: in the first period no one is treated, and in the second period a "treatment group" becomes treated, whereas a "control group" remains untreated. However, many em- pirical applications of the DID design have more than two periods and variation in treatment timing. In this article, we consider identification and estimation of treatment effect parameters using DID with (i) multiple time periods, (ii) variation in treatment timing, and (iii) when the "parallel trends assumption" holds potentially only…
-
Finitely Heterogeneous Treatment Effect in Event-studyMyung-Gil Shin · 2022 · Review of Economics and Statistics · cité 1 fois · Pas encore résumé
Cette étude propose un estimateur DiD par types latents pour analyser l’hétérogénéité des effets dans les études d’événement.
Résumé scientifique
A key assumption of the differences-in-differences designs is that the average evolution of untreated potential outcomes is the same across different treatment cohorts: parallel trend assumption. In this paper, we relax the parallel trend assumption by assuming a latent type variable and developing a type-specific parallel trend. With a finite support assumption on the latent type and long pretreatment time periods, an extremum classifier consistently estimates the type assignment. Based on the classification, we propose a type-specific DiD estimator for type-specific ATT. By estimating the type-specific ATT, we study heterogeneity in treatment effect, in addition to heterogeneity in baselin…
-
Difference-in-Differences Designs: A Practitioner's GuideA. Baker, Brantly Callaway, Scott Cunningham et al. · 2025 · cité 47 fois · Pas encore résumé
Ce guide organise les choix de conception et d’estimation en DiD, notamment les poids, les périodes multiples et les traitements échelonnés.
Résumé scientifique
Difference-in-differences (DiD) is arguably the most popular quasi-experimental research design. Its canonical form, with two groups and two periods, is well-understood. However, empirical practices can be ad hoc when researchers go beyond that simple case. This article provides an organizing framework for discussing different types of DiD designs and their associated DiD estimators. It discusses covariates, weights, handling multiple periods, and staggered treatments. The organizational framework, however, applies to other extensions of DiD methods as well.
-
Difference-in-differences with variation in treatment timingA. Goodman-Bacon · 2018 · Journal of Econometrics · cité 7 120 fois · Pas encore résumé
Cet article décompose TWFE en moyennes pondérées de comparaisons DiD et précise les conditions d’interprétation causale.
Résumé scientifique
Abstract The canonical difference-in-differences (DD) estimator contains two time periods, ”pre” and ”post”, and two groups, ”treatment” and ”control”. Most DD applications, however, exploit variation across groups of units that receive treatment at different times. This paper shows that the two-way fixed effects estimator equals a weighted average of all possible two-group/two-period DD estimators in the data. A causal interpretation of two-way fixed effects DD estimates requires both a parallel trends assumption and treatment effects that are constant over time. I show how to decompose the difference between two specifications, and provide a new analysis of models that include time-varying…
-
Identification of and Correction for Publication BiasMaximilian Kasy, Isaiah Andrews · 2017 · The American Economic Review · cité 373 fois · Pas encore résumé
Cette étude propose des méthodes pour corriger le biais de publication à partir de réplications systématiques ou de méta-études.
Résumé scientifique
Some empirical results are more likely to be published than others. Selective publication leads to biased estimates and distorted inference. We propose two approaches for identifying the conditional probability of publication as a function of a study’s results, the first based on systematic replication studies and the second on meta-studies. For known conditional publication probabilities, we propose bias-corrected estimators and confidence sets. We apply our methods to recent replication studies in experimental economics and psychology, and to a meta-study on the effect of the minimum wage. When replication and meta-study data are available, we find similar results from both.(JEL C13, C90, …
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 2 novembre 2026.