The Delta Gap: Cellular World Models for Perturbation Response Prediction Are Far Harder Than Global Correlation Implies

Authors: Junhui Gao

Submitted to arXiv on 07 Nov. 2026
License: CC BY 4.0
Abstract

Cellular world models aim to predict how cells respond to genetic perturbations, analogous to world models in reinforcement learning. Perturbation response models are evaluated almost exclusively by global Pearson correlation between predicted and observed transcriptomes. We show this metric is dominated by thousands of non-responsive genes and severely inflates apparent performance. On the Replogle K562 screen (509 unseen perturbations, 5000 genes), GEARS achieves global Pearson 0.977, but its delta Pearson is only 0.076. A trivial baseline predicting mean training delta achieves 0.267, which is 3.5x better than GEARS. Our primary contribution is methodological: we identify and quantify this delta gap as a systematic evaluation artifact, and propose delta-space evaluation as the primary benchmark. We provide two post-hoc corrections (RWR on co-expression graphs and kNN matching) and show that 2.8x of the remaining gap lies in perturbation representation geometry, not decoder capacity. We further show the gap is partially addressable end-to-end: delta-aligned training (lambda = 3.0) raises delta Pearson to 0.179 on Replogle (2.35x over GEARS) and surpasses the trivial baseline on direction accuracy (0.680 vs 0.567) and gene recovery (1.52 vs 1.19); combined with kNN it reaches 0.217. An amplitude-calibration variant (eta = 0.3) further raises gene recovery to 2.16 while reaching 0.218. Critically, these representations transfer zero-shot across cell lines: from Replogle K562 to unseen Norman A549, the delta-aligned model reaches 0.328 on 741 shared genes, 4.7x a trivial baseline and 2.1x standard GEARS. This is the first systematic quantification of the delta gap, and the first diagnostic framework localizing the residual gap to perturbation representation geometry via an oracle upper bound.

The papers around it, by date and closeness

Higher is closer to this paper; left is older. Lines join papers close to each other: groups of lines are sub-topics. Select a point to see why it is there. Back to the summary of this paper

How to read this map? Illustrated guide

Closest papers Prior work it builds on Papers that cite it

Papers around this one

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

  1. Close Same method, another problem
    GEARS: Predicting transcriptional outcomes of novel multi-gene perturbations
    Yusuf H. Roohani, Kexin Huang, J. Leskovec · 2022 · bioRxiv · preprint, not peer reviewed · cited 35 times · Not summarized yet

    GEARS predicts single- and multi-gene perturbation transcriptomes, and the paper evaluates GEARS while measuring its performance on perturbation-induced changes.

    Abstract

    Cellular response to genetic perturbation is central to numerous biomedical applications from identifying genetic interactions involved in cancer to methods for regenerative medicine. However, the combinatorial explosion in the number of possible multi-gene perturbations severely limits experimental interrogation. Here, we present GEARS, a method that can predict transcriptional response to both single and multi-gene perturbations using single-cell RNA-sequencing data from perturbational screens. GEARS is uniquely able to predict outcomes of perturbing combinations consisting of novel genes that were never experimentally perturbed by leveraging geometric deep learning and a knowledge graph o…

  2. Close Another approach to the problem
    Gradient-aware modeling advances AI-driven prediction of genetic perturbation effects
    Dixian Zhu, Livnat Jerby · 2025 · bioRxiv · preprint, not peer reviewed · cited 1 time · Not summarized yet

    GARM uses gradient-aware supervision to predict unseen perturbation effects, whereas the paper evaluates delta-aligned training and corrections to GEARS.

    Abstract

    Predicting the transcriptional effects of genetic perturbations across diverse contexts is a central challenge in functional genomics. While single-cell perturbational assays such as Perturb-seq have generated valuable datasets, exhaustively profiling all perturbations is infeasible, underscoring the need for predictive models. We present GARM (Gradient Aligned Regression with Multi-decoder), a machine learning (ML) framework that leverages gradient-aware supervision to capture both absolute and relative perturbational effects. Across multiple large-scale datasets, GARM consistently outperforms leading approaches—including GEARS, scGPT, and GenePert—in predicting responses to unseen perturba…

  3. Close Same question
    COMPASS: Component-Wise Inference of Shared and Gene-Specific Perturbation Response
    Huan Liang, Rohit Singh · 2026 · bioRxiv · preprint, not peer reviewed · cited 2 times · Not summarized yet

    COMPASS examines when mean perturbation responses conceal gene-specific effects, while the paper tests whether unchanged genes inflate global correlation.

    Abstract

    Predicting how a genetic perturbation reshapes a cell’s transcriptome is a central goal of computational biology. Previous studies report that the mean response across training perturbations rivals specialized models on standard accuracy metrics, even though it cannot distinguish which perturbation occurred. Across 2,270 CRISPRi perturbations measured in each of six cell lines, we show that this apparent paradox reflects a conserved organization of perturbation responses. Perturbations span a continuum from responses strongly aligned with the mean to more targeted responses that depart from it. Crucially, a perturbation’s position along this continuum is conserved across cell lines (Kendall’…

  4. Close Same question Cited by this paper
    Diversity by Design: Addressing Mode Collapse Improves scRNA-seq Perturbation Modeling on Well-Calibrated Metrics
    Gabriel M. Mejía, Henry E. Miller, Francis J. A. Leblanc et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 27 times · Not summarized yet

    Diversity by Design attributes mean-prediction success to metric artifacts and proposes DEG-aware metrics, while the paper diagnoses inflated global correlation from unchanged genes.

    Abstract

    Recent benchmarks reveal that models for single-cell perturbation response are often outperformed by simply predicting the dataset mean. We trace this anomaly to a metric artifact: control-referenced deltas and unweighted error metrics reward mode collapse whenever the control is biased or the biological signal is sparse. Large-scale \textit{in silico} simulations and analysis of two real-world perturbation datasets confirm that shared reference shifts, not genuine biological change, drives high performance in these evaluations. We introduce differentially expressed gene (DEG)-aware metrics, weighted mean-squared error (WMSE) and weighted delta $R^{2}$ ($R^{2}_{w}(\Delta)$) with respect to a…

  5. Close Broader context
    Reliable single-cell perturbations explain and improve model performance
    Xi Wang, J. Kuipers, Florian Hugi et al. · 2026 · bioRxiv · preprint, not peer reviewed · cited 3 times · Not summarized yet

    Reliable single-cell perturbations assesses measurement reliability and shared response alignment, addressing another source of uncertainty in perturbation-model performance.

    Abstract

    Predicting single-cell transcriptional responses to perturbations is central to building the virtual cell, yet recent benchmarks show that simple baseline methods often outperform complex models, and model comparisons depend on the evaluation metric. Most studies assume that preprocessed RNA sequencing data are reliable ground truth for both training and evaluation. Here, we test this assumption by measuring the reliability of perturbations and their alignment with shared perturbation responses, classifying each perturbation as specific, shared, or unreliable. Among 7,170 perturbations from 29 datasets, 65% are unreliable, 11% shared, and 24% specific. Applying these quality labels to publis…

  6. Close Same question
    Signal, Bounds, and Baselines: Principles for Evaluating Virtual Cell Perturbation Models
    Michael Vollenweider, P. Bühlmann · 2026 · bioRxiv · preprint, not peer reviewed · cited 6 times · Not summarized yet

    Signal, Bounds, and Baselines proposes DEG-based evaluation diagnostics to expose masked biological signal, as the paper advocates delta-space evaluation.

    Abstract

    Foundation models and deep learning systems are increasingly proposed as core components of “virtual cells” capable of forecasting transcriptomic responses to unseen perturbations. Yet rigorous evaluation of perturbation prediction in high-dimensional gene expression space remains challenging, raising concerns of reported performances. Here, we introduce the SBB principles (Signal, Bounds, and Baselines) for evaluating biological perturbation prediction. The Signal pillar introduces diagnostic meta-metrics and promotes differentially expressed gene (DEG)-based weighting or filtering to verify and recover metric sensitivity to biological signal masked by high-dimensional noise. The Bounds pil…

  7. Close Same question
    Evaluating Single-Cell Perturbation Response Models Is Far from Straightforward
    Mahshid Heidari, M. Karimpour, Sumana Srivatsa et al. · 2026 · bioRxiv · preprint, not peer reviewed · cited 3 times · Not summarized yet

    Evaluating Single-Cell Perturbation Response Models tests metric failure modes in perturbation prediction, including correlation-based measures that the paper finds misleading.

    Abstract

    Predicting cellular responses to genetic and chemical perturbations remains a central challenge in single-cell biology and a key step toward building in silico virtual cells. The rapid growth of perturbation datasets and advances in deep-learning models have raised expectations for accurate and generalizable prediction. We show that these expectations are overly optimistic, largely due to the failure modes of existing evaluation metrics. In this study, using cross-splitting, controlled noise experiments, and synthetic data, we systematically evaluate both prediction models and evaluation metrics. We demonstrate that widely used metrics, including correlation-based measures and common distrib…

  8. Close Same question
    Systema: a framework for evaluating genetic perturbation response prediction beyond systematic variation
    Ramon Viñas Torné, Maciej Wiatrak, Zoe Piran et al. · 2025 · Nature Biotechnology · cited 129 times · Not summarized yet

    Systema measures systematic variation and metric bias across perturbation datasets, while the paper shows unchanged genes inflate global performance scores.

    Abstract

    Predicting transcriptional responses to genetic perturbations is challenging in functional genomics. While recent methods aim to infer effects of untested perturbations, their true predictive power remains unclear. Here, we show that current methods struggle to generalize beyond systematic variation, the consistent transcriptional differences between perturbed and control cells arising from selection biases or confounders. We quantify this variation in ten datasets, spanning three technologies and five cell lines, and show that common metrics are susceptible to these biases, leading to overestimated performance. To address this, we introduce Systema, an evaluation framework that emphasizes p…

Prior work it builds on

  1. Close Another approach to the problem Used in depth by this paper
    D2R2: Discrete Diffusion with Regulation Reinforcement for Single-Cell Perturbation Prediction
    Ning Fan, Qi Liu, Xunuo Zhu et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 2 times · Not summarized yet

    D2R2 generates perturbation responses through regulation-guided discrete diffusion, whereas the paper evaluates and modifies GEARS using delta-focused methods.

    Abstract

    Predicting single-cell transcriptomic responses to genetic perturbations is central to functional genomics and virtual-cell modeling. Existing approaches, however, typically predict an entire expression profile as a whole, leaving the order in which individual gene responses are generated unmodeled. To address this problem, we introduce \textbf{$D^{2}R^{2}$} (\textbf{D}iscrete \textbf{D}iffusion with \textbf{R}egulation \textbf{R}einforcement), which reformulates perturbation prediction as regulation-guided gene-wise progressive generation. A Masked Discrete Diffusion Model represents expression as ordinal tokens and reconstructs a fully masked profile step by step, allowing generated gene r…

  2. Close Another approach to the problem Used in depth by this paper
    Modeling and predicting single-cell multi-gene perturbation responses with scLAMBDA
    Ge-Fei Wang, Tian-Yu Liu, Jia Zhao et al. · 2024 · bioRxiv · preprint, not peer reviewed · cited 34 times · Not summarized yet

    scLAMBDA models single- and multi-gene perturbation responses with a deep generative framework, while the paper studies delta-based evaluation and GEARS corrections.

    Abstract

    Understanding cellular responses to genetic perturbations is essential for understanding gene regulation and phenotype formation. While high-throughput single-cell RNA-sequencing has facilitated detailed profiling of heterogeneous transcriptional responses to perturbations at the single-cell level, there remains a pressing need for computational models that can decode the mechanisms driving these responses and accurately predict outcomes to prioritize target genes for experimental design. Here, we present scLAMBDA, a deep generative learning framework designed to model and predict single-cell transcriptional responses to genetic perturbations, including single-gene and combinatorial multi-ge…

  3. Related Same question Used in depth by this paper
    Benchmarking virtual cell models for in-the-wild perturbation response
    Xin-Jie Mao, Song-Ming Zhang, Qi-rong Wen et al. · 2026 · cited 8 times · Not summarized yet

    Benchmarking virtual cell models evaluates whether perturbation predictions remain meaningful under realistic conditions, complementing the paper’s critique of standard evaluation.

    Abstract

    Virtual cell (VC) models aim to predict cellular responses to any perturbations in silico and have emerged as a promising approach for drug discovery and precision medicine. Yet, a clear gap still remains: while models routinely reported impressive results on standard benchmarks, it is unclear whether their predictions are truly meaningful in practice. This is mainly due to limitations in current evaluation setups, which are often overly simplified or inconsistent, and do not reflect the complexity and variability of real biological systems. Here, we introduce a standardized and modular benchmarking framework for virtual cell prediction. Our framework evaluates diverse models under in-the-wi…

  4. Related Another approach to the problem Used in depth by this paper
    PRiMeFlow: Capturing Complex Expression Heterogeneity in Perturbation Response Modelling
    Zi-Chao Yan, Yan Wu, M. Ji et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 5 times · Not summarized yet

    PRiMeFlow uses flow matching to model heterogeneous perturbation responses, whereas the paper evaluates delta-aligned training and post-hoc corrections to GEARS.

    Abstract

    Predicting the effects of perturbations in-silico on cell state can identify drivers of cell behavior at scale and accelerate drug discovery. However, modeling challenges remain due to the inherent heterogeneity of single cell gene expression and the complex, latent gene dependencies. Here, we present PRiMeFlow, an end-to-end flow matching based approach to directly model the effects of genetic and small molecule perturbations in the gene expression space. The distribution-fitting approach taken by PRiMeFlow enables it to accurately approximate the empirical distribution of single-cell gene expression, which we demonstrate through extensive benchmarking inside PerturBench. Through ablation s…

  5. Related Another approach to the problem Used in depth by this paper
    scDFM: Distributional Flow Matching Model for Robust Single-Cell Perturbation Prediction
    Cheng-Lei Yu, Chuan-Rui Wang, Bangyan Liao et al. · 2026 · International Conference on Learning Representations · cited 27 times · Not summarized yet

    scDFM uses conditional flow matching to predict perturbed-cell distributions, rather than the paper’s evaluation and delta-alignment approach for GEARS.

    Abstract

    A central goal in systems biology and drug discovery is to predict the transcriptional response of cells to perturbations. This task is challenging due to the noisy and sparse nature of single-cell measurements, as well as the fact that perturbations often induce population-level shifts rather than changes in individual cells. Existing deep learning methods typically assume cell-level correspondences, limiting their ability to capture such global effects. We present scDFM, a generative framework based on conditional flow matching that models the full distribution of perturbed cells conditioned on control states. By incorporating a maximum mean discrepancy (MMD) objective, our method aligns p…

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

Updated on 16 November 2026.

The illustrated guide to the papers around a paper