AI summary · arXiv: 2611.03985v1 · cs.AI

Thinking Fast, Trading Late: System One Decision Models vs the Market

Amir Dehkordi (Independent Researcher), Alireza Kargarzadeh (TailState Intelligence Ltd), Nariman Khaledian (Independent Researcher), Navid Parvini (Zanista AI Ltd), Sid Ghatak (Increase Alpha, LLC), Arman Khaledian (Zanista AI Ltd)
Original paper Submitted to arXiv on 02 Nov. 2026 40 pages, 12 figures, 12 tables; pre-registered (analysis plan frozen by hash before the test period was unblinded); code at https://github.com/AmirDehkordi/NewsAction License: NONEXCLUSIVE-DISTRIB 1.0 Also in: fr

How to read this summary? Illustrated guide

The essentials

Under a blinded, point-in-time evaluation, AI news signals weakened out of sample, and commercial-model signals mostly reflected price moves already realized by the market open.

Based on the abstract only: check the details in the original paper.

Question

The paper asks how much stock-return predictability attributed to language models persists when evaluation addresses look-ahead, memorization, and selection bias. It also tests whether news-based signals remain actionable after the overnight market move.

Method

The authors propose OLIVIA, using point-in-time inputs with company identities masked, validation-only model selection, a cryptographically frozen analysis plan, and a single logged unblinding of the test period. They evaluate an open-weights AI decision model fine-tuned on 2024–2025 data, three commercial models, and standard baselines on S&P 500 company news; the abstract reports two validation quarters and a blinded January–July 2026 test.

Findings

  • Result The fine-tuned model’s daily information coefficient declined from 0.055 and 0.065 on the two validation quarters to 0.020 on the blinded test, with t = 1.7.
    See the passage in the paper
    The fine-tuned model's signal replicated on two validation quarters (daily information coefficients 0.055 and 0.065) but fell to 0.020 (t = 1.7) on the blinded January-July 2026 test.
  • Authors’ interpretation Commercial-model readings tracked the overnight move to the open but not subsequent price movement, suggesting the signal was no longer actionable by the open.
    See the passage in the paper
    The commercial models' readings line up with the overnight move to the open (information coefficient 0.074 to 0.077) but not with the move after it, so the news is priced by the time a trader can act on it at the open.
  • Result The models did not distinguish overreaction from under-reaction, and no preregistered strategy remained viable after costs and the deflated Sharpe ratio.
    See the passage in the paper
    No model can tell over- from under-reaction, and no pre-registered strategy survives costs and a deflated Sharpe ratio.
  • Result Masking company names did not reliably conceal company identities from commercial models.
    See the passage in the paper
    Commercial models identify masked companies in 62% to 74% of cases against a 28% baseline: masking hides names, not companies.

Limitations

  • The reported scope is limited to news about S&P 500 stocks and the specified models and test period.
  • The abstract reports a single blinded test period, January through July 2026, so the findings do not establish performance across other periods.
    See the passage in the paper
    the blinded January-July 2026 test

Why it matters

The authors argue that rigorous, blinded evaluation substantially changes how apparent LLM-based return predictability should be interpreted: signals may weaken out of sample or be priced before traders can act. Their results also emphasize that masking names alone may not prevent company identification.

Written by AI from the paper. Passages are checked word for word against the text we analysed; the rest may contain errors.

AI-generated keywords: language models financial news stock returns out-of-sample testing market efficiency

Read further

  • Comprehensive Summary
  • Layman's Summary
  • Blog article
  • Original abstract
“Thinking Fast, Trading Late: System One Decision Models vs. the Market” examines whether language models’ assessments of company news can predict stock returns when evaluated under safeguards against look-ahead bias, memorization, and selective reporting. The paper introduces OLIVIA, a testing protocol designed to make such evaluations more rigorous. It uses information available at the time, masks company identities, restricts model and strategy selection to validation data, and freezes the analysis plan using a cryptographic hash. Researchers then unlock the final test period once, through a logged process. The study compares an open-weights decision model fine-tuned on 2024–2025 data with three commercial models and standard baselines. These models are non-generative: they return probabilities in response to structured questions rather than producing free-form text. The fine-tuned model showed a positive signal in two validation quarters, with daily information coefficients of 0.055 and 0.065. On the blinded January–July 2026 test, its information coefficient fell to 0.020, with a t-statistic of 1.7. The abstract characterizes this as a substantial weakening of the validation results, not as evidence of a robust out-of-sample signal. The commercial models’ assessments were associated with stock movements from the previous close to the market open, with information coefficients ranging from 0.074 to 0.077. They did not predict returns after the open. The authors interpret this pattern as evidence that the news had already been reflected in prices by the time traders could act at the open. The study also reports that no model could distinguish market overreaction from underreaction. None of the preregistered trading strategies remained successful after transaction costs and evaluation with a deflated Sharpe ratio. Thus, the reported predictive associations did not demonstrate a profitable strategy under the tested conditions. Masking company names was imperfect. Commercial models identified the masked companies in 62% to 74% of cases, compared with a 28% baseline. This suggests that masking did not reliably prevent company recognition, leaving memorization or identity inference as concerns. The paper’s central conclusion is that model assessments of news may correlate with price movements, but much of the apparent signal is either fragile out of sample or reflected in prices before traders can respond. The title’s phrase “the market reads the news first” captures this interpretation. The authors present OLIVIA as a way to strengthen predictive-performance claims by limiting opportunities for information leakage and selective reporting. This summary is based only on the title, authors, and abstract, not the full paper. The abstract does not provide enough detail to assess the complete experimental design, including the precise return definitions, model prompts, baselines, transaction costs, or statistical procedures. Nor does it establish how broadly the findings generalize beyond the studied models, S&P 500 news, and January–July 2026 test period.

Was this synthesis useful?

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Figures and plots

Ask about a figure by number or use its question button to request visual analysis. Up to two figures and four image panels per question. Values read from plots are approximate; check the original data.

AI assistant instructions?

Tools for this paper

Explain it differently 1 free rewrite left today
Save to a collection Group papers in collections to compare them, chat with them and export them. Continue with Google Create a free account Log in

Advanced tools Premium

For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium

Zotero
Citation alerts Citation alerts guide
Weekly letter Presented in "Artificial intelligence this week" of 9 November 2026. Artificial intelligence this week: When better scores miss the point

Papers around this one

See them on a map

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

  1. Very close Another approach to the problem
    Trader-Company Method: A Metaheuristic for Interpretable Stock Price Prediction
    Katsuya Ito, Kentaro Minami, Kentaro Imajo et al. · 2020 · Trading and Market Microstructure · cited 16 times · AI summary available

    Trader-Company Method uses an interpretable metaheuristic to predict stock prices, while this paper tests language-model news signals under stricter evaluation controls.

    Abstract

    Investors try to predict returns of financial assets to make successful investment. Many quantitative analysts have used machine learning-based methods to find unknown profitable market rules from large amounts of market data. However, there are several challenges in financial markets hindering practical applications of machine learning-based models. First, in financial markets, there is no single model that can consistently make accurate prediction because traders in markets quickly adapt to newly available information. Instead, there are a number of ephemeral and partially correct models called "alpha factors". Second, since financial markets are highly uncertain, ensuring interpretability…

  2. Very close Another approach to the problem
    From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets
    Tao-Jie Zhu, Wen-Tao Zhao, Rui Sun et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 14 times · Not summarized yet

    KTD-Fin controls for memorized market knowledge and separates stock-selection skill from raw returns, addressing evaluation risks in LLM trading.

    Abstract

    Evaluating whether large language model (LLM) agents can profit in capital markets is increasingly framed as end-to-end trading: place an agent in a historical market, let it trade, and measure portfolio returns. This setup is vulnerable to two evaluation failures. First, long backtests often overlap with the knowledge cutoffs of frontier LLMs, allowing memorized tickers, dates, prices, and market narratives to substitute for investment reasoning. Second, raw returns are a noisy proxy for stock-selection ability, since positive performance may come from market beta, style exposure, or favorable regimes rather than genuine alpha. We introduce KTD-Fin (Knowing-To-Doing Financial Benchmark), an…

  3. Very close Another approach to the problem
    Learning to Trade Like an Expert: Cognitive Fine-Tuning for Stable Financial Reasoning in Language Models
    Yuchen Pan, S. Liew · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet

    Cognitive Fine-Tuning trains LLMs with textbook and historical-market questions, whereas this paper tests news-based return signals with point-in-time controls.

    Abstract

    Recent deployments of large language models (LLMs) as autonomous trading agents raise questions about whether financial decision-making competence generalizes beyond specific market patterns and how it should be trained and evaluated in noisy markets lacking ground truth. We propose a structured framework for training and evaluating such models. Central to our approach is a curated, multiple-choice question (MCQ) dataset derived from classic textbooks and historical markets, verified by an AI committee, enriched with structured reasoning traces, and augmented to reduce shortcut learning. To evaluate whether performance on isolated MCQs generalizes to real-world trading, we introduce a two-st…

  4. Close Broader context
    Going All-In on LLM Accuracy: Fake Prediction Markets, Real Confidence Signals
    Michael Todasco · 2025 · arXiv.org · preprint, not peer reviewed · cited 2 times · Not summarized yet

    Fake Prediction Markets tests whether betting-game framing improves LLM forecasts of other models’ answers, rather than stock returns.

    Abstract

    Large language models are increasingly used to evaluate other models, yet these judgments typically lack any representation of confidence. This pilot study tests whether framing an evaluation task as a betting game (a fictional prediction market with its own LLM currency) improves forecasting accuracy and surfaces calibrated confidence signals. We generated 100 math and logic questions with verifiable answers. Six Baseline models (three current-generation, three prior-generation) answered all items. Three Predictor models then forecasted, for each question-baseline pair, if the baseline would answer correctly. Each predictor completed matched runs in two conditions: Control (simple correct/i…

  5. Close Another approach to the problem
    StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?
    Yanxu Chen, Zijun Yao, Yantao Liu et al. · 2025 · Machine Learning · preprint, not peer reviewed · cited 49 times · AI summary available

    StockBench evaluates LLM agents in realistic iterative trading, while this paper tests whether news-based signals predict returns and remain actionable at the open.

    Abstract

    Large language models (LLMs) have recently demonstrated strong capabilities as autonomous agents, showing promise in reasoning, tool use, and sequential decision-making. While prior benchmarks have evaluated LLM agents in domains such as software engineering and scientific discovery, the finance domain remains underexplored, despite its direct relevance to economic value and high-stakes decision-making. Existing financial benchmarks primarily test static knowledge through question answering, but they fall short of capturing the dynamic and iterative nature of trading. To address this gap, we introduce StockBench, a contamination-free benchmark designed to evaluate LLM agents in realistic, mu…

  6. Close Same question
    Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?
    Weixian Waylon Li, Hyeonjun Kim, Mihai Cucuringu et al. · 2025 · Knowledge Discovery and Data Mining · cited 32 times · Not summarized yet

    FINSABER tests whether LLM investing strategies outperform across longer periods and broader stock universes, examining robustness to biases in backtests.

    Abstract

    Large Language Models (LLMs) have recently been leveraged for asset pricing and stock trading applications, enabling AI agents to generate investment decisions from unstructured financial data. However, most evaluations of LLM timing-based investing strategies are conducted on narrow timeframes and limited stock universes, overstating effectiveness due to survivorship and data-snooping biases. We critically assess their generalizability and robustness by proposing FINSABER, a backtesting framework evaluating timing-based strategies across longer periods and a larger universe of symbols. Systematic backtests over two decades and 100+ symbols reveal that previously reported LLM advantages dete…

  7. Close Another approach to the problem
    AlphaForgeBench: Benchmarking End-to-End Trading Strategy Design with Large Language Models
    Wen-Tao Zhang, Ming-Xuan Zhao, Jin-Cheng Gao et al. · 2026 · Knowledge Discovery and Data Mining · cited 6 times · Not summarized yet

    AlphaForgeBench evaluates end-to-end strategy design and trading-agent stability, rather than point-in-time prediction from masked company news.

    Abstract

    The rapid advancement of Large Language Models (LLMs) has catalyzed the proliferation of diverse financial benchmarks, progressively evolving from static knowledge evaluation to increasingly sophisticated interactive trading simulations. Nevertheless, existing frameworks that assess real-time trading performance largely overlook a fundamental failure mode: the severe behavioral instability exhibited by LLMs in sequential decision-making under financial uncertainty. Through extensive empirical investigation, we demonstrate that when deployed as direct trading agents, LLMs manifest extreme run-to-run variance, produce inconsistent action sequences even under strictly deterministic decoding con…

  8. Close Broader context
    Reflexivity as Prompt: Does Awareness of Self-Reinforcing Market Dynamics Improve LLMs as Financial Market Forecasters?
    Eugene W Park · 2026 · Not summarized yet

    Reflexivity as Prompt studies whether explaining self-reinforcing market dynamics improves LLM forecasts across historical boom-bust episodes.

    Abstract

    We study how frontier large language models (LLMs) behave as financial forecasters during boom-bust market cycles when made progressively aware of Soros's theory of reflexivity. Standard AI-assisted forecasting treats the market as an exogenous system. Reflexivity theory holds otherwise: prices shape fundamentals, and every forecaster is a participative agent in the loop it analyzes. We evaluate three frontier models - GPT5, Claude Sonnet 4.6, and Gemini 3 Pro - under four accumulating zero-shot conditions across two historically distinct episodes: the dot-com bubble (1996-2001) and the global financial crisis (2004-2009). The primary metric is directional forecasting accuracy; we also repor…

Prior work it builds on

  1. Close Same question
    Pervasive underreaction: Evidence from high-frequency data
    Hao Jiang, Sophia Zhengzi Li, Hao Wang · 2020 · Journal of Financial Economics · cited 42 times · Not summarized yet

    Pervasive underreaction tests whether stock prices drift after firm news, while this paper finds news priced by the open and no reliable evidence of underreaction.

    Abstract

    We propose a novel high-frequency decomposition of daily stock returns into news- and non-news-driven components. With it, we uncover evidence of pervasive stock market underreaction to firm news, in the post-millennium period when stocks appear to be priced with increased efficiency. We show that stock prices tend to drift in the same direction as the initial market response, for several days after the news arrival without reversals. A trading strategy exploiting the return drift generates a monthly return of 2.10% after trading costs. To understand the economic mechanism, we find that the return drift is stronger when investors are distracted. Using analyst earnings forecasts as a proxy, w…

  2. Close Same method, another problem
    Approaching Human-Level Forecasting with Language Models
    Danny Halawi, Fred Zhang, Chen Yueh-Han et al. · 2024 · Machine Learning · cited 165 times · AI summary available

    Approaching Human-Level Forecasting uses retrieval-augmented language models to generate forecasts, applying language-model prediction to future events rather than stock news.

    Abstract

    Forecasting future events is important for policy and decision making. In this work, we study whether language models (LMs) can forecast at the level of competitive human forecasters. Towards this goal, we develop a retrieval-augmented LM system designed to automatically search for relevant information, generate forecasts, and aggregate predictions. To facilitate our study, we collect a large dataset of questions from competitive forecasting platforms. Under a test set published after the knowledge cut-offs of our LMs, we evaluate the end-to-end performance of our system against the aggregates of human forecasts. On average, the system nears the crowd aggregate of competitive forecasters, an…

  3. Close Same method, another problem
    Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Match Human Crowd Accuracy
    Philipp Schoenegger, Indre Tuminauskaite, Peter S. Park et al. · 2024 · Computers and Society · cited 131 times · AI summary available

    Wisdom of the Silicon Crowd aggregates LLM predictions to forecast events, rather than testing individual model signals on stock returns.

    Abstract

    Human forecasting accuracy in practice relies on the 'wisdom of the crowd' effect, in which predictions about future events are significantly improved by aggregating across a crowd of individual forecasters. Past work on the forecasting ability of large language models (LLMs) suggests that frontier LLMs, as individual forecasters, underperform compared to the gold standard of a human crowd forecasting tournament aggregate. In Study 1, we expand this research by using an LLM ensemble approach consisting of a crowd of twelve LLMs. We compare the aggregated LLM predictions on 31 binary questions to that of a crowd of 925 human forecasters from a three-month forecasting tournament. Our main anal…

  4. Close Groundwork for this paper
    The Memorization Problem: Can We Trust LLMs'Economic Forecasts?
    Alejandro Lopez-Lira, Yue-Hua Tang, Ming Zhu · 2025 · cited 50 times · Not summarized yet

    The Memorization Problem shows that LLMs recall financial data and reconstruct masked entities, motivating this paper’s masking and look-ahead controls.

    Abstract

    Large language models (LLMs) cannot be trusted for economic forecasts during periods covered by their training data. Counterfactual forecasting ability is non-identified when the model has seen the realized values: any observed output is consistent with both genuine skill and memorization. Any evidence of memorization represents only a lower bound on encoded knowledge. We demonstrate LLMs have memorized economic and financial data, recalling exact values before their knowledge cutoff. Instructions to respect historical boundaries fail to prevent recall-level accuracy, and masking fails as LLMs reconstruct entities and dates from minimal context. Post-cutoff, we observe no recall. Memorizatio…

  5. Close Groundwork for this paper
    Information, Trading, and Volatility: Evidence from Firm-Specific News
    J. Boudoukh, Ronen Feldman, Shimon Kogan et al. · 2018 · The Review of financial studies · cited 152 times · Not summarized yet

    Information, Trading, and Volatility measures firm-specific news information in stock returns, providing a basis for this paper’s study of news-based signals.

    Abstract

    What moves stock prices? Prior literature concludes that the revelation of private information through trading, and not public news, is the primary driver. We revisit the question by using textual analysis to identify fundamental information in news. We find that this information accounts for 49.6% of overnight idiosyncratic volatility (vs. 12.4% during trading hours), with a considerable fraction due to days with multiple news types. We use our measure of public information arrival to reinvestigate two important contributions in the literature related to individual R2s of stock returns on aggregate factors. (JEL G14)

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

Updated on 9 November 2026.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.