# Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models

**Authors:** Avanika Narayan, Dan Biderman, Sabri Eyuboglu, Avner May, Scott Linderman, James Zou, Christopher Re  
**arXiv:** [2502.15964](https://arxiv.org/abs/2502.15964)  
**Date:** 2025-02-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.15964v1/

## Key points

- Study focuses on local-remote system collaboration for real-world tasks in finance, medicine, and science
- Objective is to reduce cloud inference costs while maintaining high performance quality
- Basic collaboration protocol results in 30.4x reduction in remote costs but only achieves 87% of frontier model's performance
- Enhanced protocol MinionS breaks down tasks into smaller subtasks executed locally, leading to 5.7x cost reduction and recovering 97.9% of remote model's performance
- Techniques in MinionS inspired by orchestration for long-contexts, decomposition techniques, and test-time sampling strategies
- Emphasis on optimizing task handling by leveraging both local and remote capabilities effectively
- Study highlights key design choices influencing balance between cost efficiency and performance in local-remote systems

## Summary

This study delves into the intricacies of a local-remote system where a small on-device language model (LM) collaborates with a cloud-hosted LM to tackle real-world tasks involving financial, medical, and scientific reasoning over extensive documents. The primary objective is to reduce cloud inference costs while maintaining high performance quality. Initially, a basic collaboration protocol is explored where the local and remote models engage in simple communication. This approach results in a significant 30.4x reduction in remote costs but only achieves 87% of the frontier model's performance due to limitations such as the local model's difficulty in following multi-step instructions and reasoning over lengthy contexts. Building upon these observations, an enhanced protocol called MinionS is introduced. In MinionS, the remote model breaks down tasks into smaller subtasks over shorter document chunks that are executed locally in parallel by the on-device model. This innovative approach leads to an average cost reduction of 5.7x while recovering an impressive 97.9% of the performance of the remote model alone. The techniques employed in MinionS draw inspiration from existing literature on orchestration for long-contexts, decomposition techniques, and test-time sampling and verification strategies. These methods aim to optimize task handling by leveraging both local and remote capabilities effectively. Furthermore, the study highlights key design choices that influence the balance between cost efficiency and performance in local-remote systems. By focusing on reducing cloud inference costs without compromising task quality, this research contributes valuable insights into optimizing collaborative setups for efficient real-world task execution across diverse domains like finance, medicine, and science.