The Delta Gap: Cellular World Models for Perturbation Response Prediction Are Far Harder Than Global Correlation Implies
Auteurs : Junhui Gao
Résumé scientifique
Cellular world models aim to predict how cells respond to genetic perturbations, analogous to world models in reinforcement learning. Perturbation response models are evaluated almost exclusively by global Pearson correlation between predicted and observed transcriptomes. We show this metric is dominated by thousands of non-responsive genes and severely inflates apparent performance. On the Replogle K562 screen (509 unseen perturbations, 5000 genes), GEARS achieves global Pearson 0.977, but its delta Pearson is only 0.076. A trivial baseline predicting mean training delta achieves 0.267, which is 3.5x better than GEARS. Our primary contribution is methodological: we identify and quantify this delta gap as a systematic evaluation artifact, and propose delta-space evaluation as the primary benchmark. We provide two post-hoc corrections (RWR on co-expression graphs and kNN matching) and show that 2.8x of the remaining gap lies in perturbation representation geometry, not decoder capacity. We further show the gap is partially addressable end-to-end: delta-aligned training (lambda = 3.0) raises delta Pearson to 0.179 on Replogle (2.35x over GEARS) and surpasses the trivial baseline on direction accuracy (0.680 vs 0.567) and gene recovery (1.52 vs 1.19); combined with kNN it reaches 0.217. An amplitude-calibration variant (eta = 0.3) further raises gene recovery to 2.16 while reaching 0.218. Critically, these representations transfer zero-shot across cell lines: from Replogle K562 to unseen Norman A549, the delta-aligned model reaches 0.328 on 741 shared genes, 4.7x a trivial baseline and 2.1x standard GEARS. This is the first systematic quantification of the delta gap, and the first diagnostic framework localizing the residual gap to perturbation representation geometry via an oracle upper bound.
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
GEARS: Predicting transcriptional outcomes of novel multi-gene perturbationsYusuf H. Roohani, Kexin Huang, J. Leskovec · 2022 · bioRxiv · preprint, non relu par les pairs · cité 35 fois · Pas encore résumé
GEARS prédit les réponses transcriptionnelles à des perturbations inédites, modèle que l’étude évalue sur des perturbations non vues.
Résumé scientifique
Cellular response to genetic perturbation is central to numerous biomedical applications from identifying genetic interactions involved in cancer to methods for regenerative medicine. However, the combinatorial explosion in the number of possible multi-gene perturbations severely limits experimental interrogation. Here, we present GEARS, a method that can predict transcriptional response to both single and multi-gene perturbations using single-cell RNA-sequencing data from perturbational screens. GEARS is uniquely able to predict outcomes of perturbing combinations consisting of novel genes that were never experimentally perturbed by leveraging geometric deep learning and a knowledge graph o…
-
Gradient-aware modeling advances AI-driven prediction of genetic perturbation effectsDixian Zhu, Livnat Jerby · 2025 · bioRxiv · preprint, non relu par les pairs · cité 1 fois · Pas encore résumé
GARM emploie une supervision alignée sur les gradients pour prédire les effets de perturbations, tandis que l’étude aligne l’entraînement sur les deltas.
Résumé scientifique
Predicting the transcriptional effects of genetic perturbations across diverse contexts is a central challenge in functional genomics. While single-cell perturbational assays such as Perturb-seq have generated valuable datasets, exhaustively profiling all perturbations is infeasible, underscoring the need for predictive models. We present GARM (Gradient Aligned Regression with Multi-decoder), a machine learning (ML) framework that leverages gradient-aware supervision to capture both absolute and relative perturbational effects. Across multiple large-scale datasets, GARM consistently outperforms leading approaches—including GEARS, scGPT, and GenePert—in predicting responses to unseen perturba…
-
COMPASS: Component-Wise Inference of Shared and Gene-Specific Perturbation ResponseHuan Liang, Rohit Singh · 2026 · bioRxiv · preprint, non relu par les pairs · cité 2 fois · Pas encore résumé
COMPASS étudie la réponse transcriptionnelle aux perturbations et montre que la moyenne des réponses masque des réponses plus ciblées, comme l’étude examine les limites des évaluations globales.
Résumé scientifique
Predicting how a genetic perturbation reshapes a cell’s transcriptome is a central goal of computational biology. Previous studies report that the mean response across training perturbations rivals specialized models on standard accuracy metrics, even though it cannot distinguish which perturbation occurred. Across 2,270 CRISPRi perturbations measured in each of six cell lines, we show that this apparent paradox reflects a conserved organization of perturbation responses. Perturbations span a continuum from responses strongly aligned with the mean to more targeted responses that depart from it. Crucially, a perturbation’s position along this continuum is conserved across cell lines (Kendall’…
-
Diversity by Design: Addressing Mode Collapse Improves scRNA-seq Perturbation Modeling on Well-Calibrated MetricsGabriel M. Mejía, Henry E. Miller, Francis J. A. Leblanc et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 27 fois · Pas encore résumé
Diversity by Design attribue les performances trompeuses aux métriques fondées sur les deltas et propose des métriques pondérées par les gènes différentiellement exprimés.
Résumé scientifique
Recent benchmarks reveal that models for single-cell perturbation response are often outperformed by simply predicting the dataset mean. We trace this anomaly to a metric artifact: control-referenced deltas and unweighted error metrics reward mode collapse whenever the control is biased or the biological signal is sparse. Large-scale \textit{in silico} simulations and analysis of two real-world perturbation datasets confirm that shared reference shifts, not genuine biological change, drives high performance in these evaluations. We introduce differentially expressed gene (DEG)-aware metrics, weighted mean-squared error (WMSE) and weighted delta $R^{2}$ ($R^{2}_{w}(\Delta)$) with respect to a…
-
Reliable single-cell perturbations explain and improve model performanceXi Wang, J. Kuipers, Florian Hugi et al. · 2026 · bioRxiv · preprint, non relu par les pairs · cité 3 fois · Pas encore résumé
Reliable single-cell perturbations examine la fiabilité des mesures de perturbation, qui conditionne l’interprétation des performances prédictives évaluées ici.
Résumé scientifique
Predicting single-cell transcriptional responses to perturbations is central to building the virtual cell, yet recent benchmarks show that simple baseline methods often outperform complex models, and model comparisons depend on the evaluation metric. Most studies assume that preprocessed RNA sequencing data are reliable ground truth for both training and evaluation. Here, we test this assumption by measuring the reliability of perturbations and their alignment with shared perturbation responses, classifying each perturbation as specific, shared, or unreliable. Among 7,170 perturbations from 29 datasets, 65% are unreliable, 11% shared, and 24% specific. Applying these quality labels to publis…
-
Signal, Bounds, and Baselines: Principles for Evaluating Virtual Cell Perturbation ModelsMichael Vollenweider, P. Bühlmann · 2026 · bioRxiv · preprint, non relu par les pairs · cité 6 fois · Pas encore résumé
Signal, Bounds, and Baselines propose des principes d’évaluation des modèles de perturbation, notamment des métriques sensibles au signal biologique.
Résumé scientifique
Foundation models and deep learning systems are increasingly proposed as core components of “virtual cells” capable of forecasting transcriptomic responses to unseen perturbations. Yet rigorous evaluation of perturbation prediction in high-dimensional gene expression space remains challenging, raising concerns of reported performances. Here, we introduce the SBB principles (Signal, Bounds, and Baselines) for evaluating biological perturbation prediction. The Signal pillar introduces diagnostic meta-metrics and promotes differentially expressed gene (DEG)-based weighting or filtering to verify and recover metric sensitivity to biological signal masked by high-dimensional noise. The Bounds pil…
-
Evaluating Single-Cell Perturbation Response Models Is Far from StraightforwardMahshid Heidari, M. Karimpour, Sumana Srivatsa et al. · 2026 · bioRxiv · preprint, non relu par les pairs · cité 3 fois · Pas encore résumé
Evaluating Single-Cell Perturbation Response Models étudie comment les métriques usuelles surestiment les performances des modèles de réponse aux perturbations.
Résumé scientifique
Predicting cellular responses to genetic and chemical perturbations remains a central challenge in single-cell biology and a key step toward building in silico virtual cells. The rapid growth of perturbation datasets and advances in deep-learning models have raised expectations for accurate and generalizable prediction. We show that these expectations are overly optimistic, largely due to the failure modes of existing evaluation metrics. In this study, using cross-splitting, controlled noise experiments, and synthetic data, we systematically evaluate both prediction models and evaluation metrics. We demonstrate that widely used metrics, including correlation-based measures and common distrib…
-
Systema: a framework for evaluating genetic perturbation response prediction beyond systematic variationRamon Viñas Torné, Maciej Wiatrak, Zoe Piran et al. · 2025 · Nature Biotechnology · cité 129 fois · Pas encore résumé
Systema montre que les métriques courantes surestiment la prédiction en raison de variations systématiques, tandis que l’étude met en évidence l’influence des gènes non réactifs.
Résumé scientifique
Predicting transcriptional responses to genetic perturbations is challenging in functional genomics. While recent methods aim to infer effects of untested perturbations, their true predictive power remains unclear. Here, we show that current methods struggle to generalize beyond systematic variation, the consistent transcriptional differences between perturbed and control cells arising from selection biases or confounders. We quantify this variation in ten datasets, spanning three technologies and five cell lines, and show that common metrics are susceptible to these biases, leading to overestimated performance. To address this, we introduce Systema, an evaluation framework that emphasizes p…
Les travaux antérieurs sur lesquels il s’appuie
-
D2R2: Discrete Diffusion with Regulation Reinforcement for Single-Cell Perturbation PredictionNing Fan, Qi Liu, Xunuo Zhu et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 2 fois · Pas encore résumé
D²R² prédit les réponses transcriptomiques par génération progressive des gènes, autre approche de modélisation des perturbations.
Résumé scientifique
Predicting single-cell transcriptomic responses to genetic perturbations is central to functional genomics and virtual-cell modeling. Existing approaches, however, typically predict an entire expression profile as a whole, leaving the order in which individual gene responses are generated unmodeled. To address this problem, we introduce \textbf{$D^{2}R^{2}$} (\textbf{D}iscrete \textbf{D}iffusion with \textbf{R}egulation \textbf{R}einforcement), which reformulates perturbation prediction as regulation-guided gene-wise progressive generation. A Masked Discrete Diffusion Model represents expression as ordinal tokens and reconstructs a fully masked profile step by step, allowing generated gene r…
-
Modeling and predicting single-cell multi-gene perturbation responses with scLAMBDAGe-Fei Wang, Tian-Yu Liu, Jia Zhao et al. · 2024 · bioRxiv · preprint, non relu par les pairs · cité 34 fois · Pas encore résumé
scLAMBDA modélise et prédit les réponses transcriptionnelles à des perturbations simples ou combinées, sans traiter spécifiquement l’écart entre corrélations globales et deltas.
Résumé scientifique
Understanding cellular responses to genetic perturbations is essential for understanding gene regulation and phenotype formation. While high-throughput single-cell RNA-sequencing has facilitated detailed profiling of heterogeneous transcriptional responses to perturbations at the single-cell level, there remains a pressing need for computational models that can decode the mechanisms driving these responses and accurately predict outcomes to prioritize target genes for experimental design. Here, we present scLAMBDA, a deep generative learning framework designed to model and predict single-cell transcriptional responses to genetic perturbations, including single-gene and combinatorial multi-ge…
-
Benchmarking virtual cell models for in-the-wild perturbation responseXin-Jie Mao, Song-Ming Zhang, Qi-rong Wen et al. · 2026 · cité 8 fois · Pas encore résumé
Benchmarking virtual cell models cite l’étude et propose un cadre standardisé pour évaluer les prédictions en conditions plus proches du réel.
Résumé scientifique
Virtual cell (VC) models aim to predict cellular responses to any perturbations in silico and have emerged as a promising approach for drug discovery and precision medicine. Yet, a clear gap still remains: while models routinely reported impressive results on standard benchmarks, it is unclear whether their predictions are truly meaningful in practice. This is mainly due to limitations in current evaluation setups, which are often overly simplified or inconsistent, and do not reflect the complexity and variability of real biological systems. Here, we introduce a standardized and modular benchmarking framework for virtual cell prediction. Our framework evaluates diverse models under in-the-wi…
-
PRiMeFlow: Capturing Complex Expression Heterogeneity in Perturbation Response ModellingZi-Chao Yan, Yan Wu, M. Ji et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 5 fois · Pas encore résumé
PRiMeFlow modélise les distributions d’expression après perturbation par flow matching, plutôt que de se concentrer sur les métriques de delta.
Résumé scientifique
Predicting the effects of perturbations in-silico on cell state can identify drivers of cell behavior at scale and accelerate drug discovery. However, modeling challenges remain due to the inherent heterogeneity of single cell gene expression and the complex, latent gene dependencies. Here, we present PRiMeFlow, an end-to-end flow matching based approach to directly model the effects of genetic and small molecule perturbations in the gene expression space. The distribution-fitting approach taken by PRiMeFlow enables it to accurately approximate the empirical distribution of single-cell gene expression, which we demonstrate through extensive benchmarking inside PerturBench. Through ablation s…
-
scDFM: Distributional Flow Matching Model for Robust Single-Cell Perturbation PredictionCheng-Lei Yu, Chuan-Rui Wang, Bangyan Liao et al. · 2026 · International Conference on Learning Representations · cité 27 fois · Pas encore résumé
scDFM prédit les distributions de cellules perturbées avec un objectif de flow matching conditionnel, une approche distincte de l’évaluation des deltas.
Résumé scientifique
A central goal in systems biology and drug discovery is to predict the transcriptional response of cells to perturbations. This task is challenging due to the noisy and sparse nature of single-cell measurements, as well as the fact that perturbations often induce population-level shifts rather than changes in individual cells. Existing deep learning methods typically assume cell-level correspondences, limiting their ability to capture such global effects. We present scDFM, a generative framework based on conditional flow matching that models the full distribution of perturbed cells conditioned on control states. By incorporating a maximum mean discrepancy (MMD) objective, our method aligns p…
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 16 novembre 2026.