Decision Model Factory: Building Specialized Sovereign Decision Models

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Close Broader context Not relevant

   Foundation Models for Decision Making: Problems, Methods, and Opportunities

   Sherry Yang, Ofir Nachum, Yilun Du et al. · 2023 · Artificial Intelligence · preprint, not peer reviewed · cited 262 times · AI summary available

   Foundation Models for Decision Making surveys emerging methods for training foundation models to interact with agents and perform long-term reasoning.

   Abstract

   Foundation models pretrained on diverse data at scale have demonstrated extraordinary capabilities in a wide range of vision and language tasks. When such models are deployed in real world environments, they inevitably interface with other entities and agents. For example, language models are often used to interact with human beings through dialogue, and visual perception models are used to autonomously navigate neighborhood streets. In response to these developments, new paradigms are emerging for training foundation models to interact with other agents and perform long-term reasoning. These paradigms leverage the existence of ever-larger datasets curated for multimodal, multitask, and gene…

2. Related Same question Cited by this paper Not relevant

   LLM-as-Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them

   Yin-Heng Li, Justin Wagle · 2026 · cited 19 times · Not summarized yet

   LLM-as-Jev investigates when general-purpose language models need fine-tuning to produce categorical probability distributions for decision-making.

   Abstract

   Jev-style decision models have gained surging traction by making decision-making dramatically faster and more efficient, returning categorical probability distributions over predefined options without generating free-form text. However, because Jev's architecture and training methodologies remain proprietary, how to replicate these capabilities effectively remains an open question. In this work, we investigate the extent to which general-purpose LLMs already possess this capability out of the box, and when fine-tuning is actually necessary. We present LLM-as-Jev, an architecture-preserving framework offering both a training-free inference recipe and a fine-tuning objective with KL-divergence…

3. Related* Broader context Not relevant

   Large Language Model as Autonomous Decision Maker

   Yining Ye, Xin Cong, Yujia Qin et al. · 2023 · arXiv.org · preprint, not peer reviewed · cited 11 times · Not summarized yet

   Large Language Model as Autonomous Decision Maker concerns language models used for decision-making, but its abstract provides no further details.

4. Related Broader context Not relevant

   Unified Deployment-Aware Evaluation of Open Reasoning Language Models

   M. Manik, Wang Ge · 2026 · cited 3 times · Not summarized yet

   Unified Deployment-Aware Evaluation compares open reasoning models across benchmarks, prompting strategies, accuracy, and deployment measures.

   Abstract

   Open reasoning language models are often compared under mixed sample sizes, partially standardized prompts, and accuracy-centered summaries, which makes practical model selection difficult to interpret. We present a unified evaluation of seven open reasoning language model configurations across four benchmarks: ARC-Challenge, GSM8K, MATH levels 1 to 3, and TruthfulQA MC1. We test zero-shot, chain-of-thought (CoT), and few-shot CoT prompting on the same 238-example subset for every model--dataset--strategy condition, yielding a complete 7 x 4 x 3 design with 84 conditions and 19,992 evaluated examples. Beyond accuracy, we report Wilson confidence intervals, latency, peak video random access m…

5. Related Same question Cited by this paper Not relevant

   Releasing the Inner Jev: A Simple Recipe for Open Decision Models

   Jonathan Geuter, Andrew Cheng, David Alvarez-Melis · 2026 · cited 6 times · Not summarized yet

   Releasing the Inner Jev tests open models’ option-probability readout and supervised fine-tuning as a recipe for decision models.

   Abstract

   LLM-based decision-making systems, also called System One models, have recently gained popularity, most notably through Jev (Almeida, 2026). Given a typed input state, a question, and a set of options, they return a probability for each option in a single forward pass, without text generation. We show that existing open instruction-tuned language models can achieve comparable performance \textit{without any additional training}, by reading out answers from letters (A, B, C, etc.) in the model's output logits at the last token position. Furthermore, we show how a simple supervised fine-tuning recipe can improve their performance further. We evaluate four open instruction-tuned language models…

6. Related Another approach to the problem Not relevant

   DeLLMa: Decision Making Under Uncertainty with Large Language Models

   Ollie Liu, Deqing Fu, Dani Yogatama et al. · 2024 · Artificial Intelligence · cited 33 times · AI summary available

   DeLLMa improves decisions under uncertainty through multi-step reasoning, rather than building specialized models and optimizing decision-specific inference.

   Abstract

   The potential of large language models (LLMs) as decision support tools is increasingly being explored in fields such as business, engineering, and medicine, which often face challenging tasks of decision-making under uncertainty. In this paper, we show that directly prompting LLMs on these types of decision-making problems can yield poor results, especially as the problem complexity increases. To aid in these tasks, we propose DeLLMa (Decision-making Large Language Model assistant), a framework designed to enhance decision-making accuracy in uncertain environments. DeLLMa involves a multi-step reasoning procedure that integrates recent best practices in scaling inference-time reasoning, dra…

7. Related Broader context Not relevant

   Demand-Driven Context: A Methodology for Building Enterprise Knowledge Bases Through Agent Failure

   Raj Navakoti, Saideep Navakoti · 2026 · Artificial Intelligence · preprint, not peer reviewed · AI summary available

   Demand-Driven Context builds enterprise knowledge bases from agent failures, addressing domain-specific agent performance through context rather than model specialization.

   Abstract

   Large language model agents demonstrate expert-level reasoning, yet consistently fail on enterprise-specific tasks due to missing domain knowledge -- terminology, operational procedures, system interdependencies, and institutional decisions that exist largely as tribal knowledge. Current approaches fall into two categories: top-down knowledge engineering, which documents domain knowledge before agents use it, and bottom-up automation, where agents learn from task experience. Both have fundamental limitations: top-down efforts produce bloated, untested knowledge bases; bottom-up approaches cannot acquire knowledge that exists only in human heads. We present Demand-Driven Context (DDC), a prob…

8. Related Broader context Not relevant

   Sakana Fugu Technical Report

   Yujin Tang, Edoardo Cetin, Jinglue Xu et al. · 2026 · Machine Learning · preprint, not peer reviewed · cited 4 times · AI summary available

   Sakana Fugu trains orchestrator models to coordinate specialized language-model agents through adaptive scaffolds.

   Abstract

   The capabilities of frontier Large Language Models (LLMs) continue to advance, with different providers increasingly specializing in distinct domains. This raises a natural next objective: how to combine the individual specializations of various LLMs into a collectively intelligent system. To this end, we report the development of Sakana Fugu, a family of orchestrator models that harness and amplify the capabilities of an LLM agent team. Fugu models are themselves language models trained to understand user queries and dynamically devise agentic scaffolds to solve them. Through these adaptive scaffolds, Fugu accesses performance beyond any individual LLM agent, achieving state-of-the-art resu…

Prior work it builds on

1. Related Groundwork for this paper Not relevant

   Language Models are Few-Shot Learners

   Tom B. Brown, Benjamin Mann, Nick Ryder et al. · 2020 · Neural Information Processing Systems · cited 66,023 times · AI summary available

   Language Models are Few-Shot Learners establishes task-agnostic few-shot capabilities of scaled language models, which the paper cites.

   Abstract

   Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Spec…

2. Related Broader context Not relevant

   PHOTON: Hierarchical Autoregressive Modeling for Lightspeed and Memory-Efficient Language Generation

   Yuma Ichikawa, N. Takagi, Takumi Nakagawa et al. · 2025 · Annual Meeting of the Association for Computational Linguistics · cited 1 time · Not summarized yet

   PHOTON targets faster, memory-efficient language generation through hierarchical context scanning, while this paper speeds up decision-specific inference.

   Abstract

   Transformers operate as horizontal token-by-token scanners; at each generation step, attending to an ever-growing sequence of token-level states. This access pattern increases prefill latency and makes long-context decoding more memory-bound, as KV-cache reads and writes dominate inference time over arithmetic operations. We propose Parallel Hierarchical Operation for TOp-down Networks (PHOTON), a hierarchical autoregressive model that replaces horizontal scanning with vertical, multi-resolution context scanning. PHOTON maintains a hierarchy of latent streams: a bottom-up encoder compresses tokens into low-rate contextual states, while lightweight top-down decoders reconstruct fine-grained t…

3. Related Groundwork for this paper Not relevant

   Distilling the Knowledge in a Neural Network

   Geoffrey E. Hinton, O. Vinyals, Jeffrey Dean · 2015 · arXiv.org · preprint, not peer reviewed · cited 27,520 times · AI summary available

   Distilling the Knowledge in a Neural Network develops model compression through distillation, a technique the paper cites and optionally uses.

   Abstract

   A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions. Unfortunately, making predictions using a whole ensemble of models is cumbersome and may be too computationally expensive to allow deployment to a large number of users, especially if the individual models are large neural nets. Caruana and his collaborators have shown that it is possible to compress the knowledge in an ensemble into a single model which is much easier to deploy and we develop this approach further using a different compression technique. We achieve some surprising results on MNIST and we show that we …

4. Related Groundwork for this paper Not relevant

   Lost in the Middle: How Language Models Use Long Contexts

   Nelson F. Liu, Kevin Lin, John Hewitt et al. · 2023 · Transactions of the Association for Computational Linguistics · cited 5,819 times · AI summary available

   Lost in the Middle analyzes failures to use long contexts robustly, relevant to the paper’s long-context passage-ranking evaluation.

   Abstract

   While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly de…

5. Related Groundwork for this paper Not relevant

   Document Ranking with a Pretrained Sequence-to-Sequence Model

   Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep et al. · 2020 · Findings · cited 938 times · Not summarized yet

   Document Ranking with a Pretrained Sequence-to-Sequence Model uses target-token logits as relevance probabilities for passage ranking, a task the paper evaluates.

   Abstract

   This work proposes the use of a pretrained sequence-to-sequence model for document ranking. Our approach is fundamentally different from a commonly adopted classification-based formulation based on encoder-only pretrained transformer architectures such as BERT. We show how a sequence-to-sequence model can be trained to generate relevance labels as “target tokens”, and how the underlying logits of these target tokens can be interpreted as relevance probabilities for ranking. Experimental results on the MS MARCO passage ranking task show that our ranking approach is superior to strong encoder-only models. On three other document retrieval test collections, we demonstrate a zero-shot transfer-b…