Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Not relevant

   Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs

   Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 687 times · Not summarized yet

   Abstract

   We introduce Phi-4-Mini and Phi-4-Multimodal, compact yet highly capable language and multimodal models. Phi-4-Mini is a 3.8-billion-parameter language model trained on high-quality web and synthetic data, significantly outperforming recent open-source models of similar size and matching the performance of models twice its size on math and coding tasks requiring complex reasoning. This achievement is driven by a carefully curated synthetic data recipe emphasizing high-quality math and coding datasets. Compared to its predecessor, Phi-3.5-Mini, Phi-4-Mini features an expanded vocabulary size of 200K tokens to better support multilingual applications, as well as group query attention for more …

2. Very close Builds on this paper Not relevant

   Phi-4 Technical Report

   Marah Abdin, Jyoti Aneja, Harkirat Behl et al. · 2024 · Computation and Language · preprint, not peer reviewed · cited 880 times · AI summary available

   Phi-4 extends the Phi family’s data-focused training approach, retaining much of phi-3’s architecture while emphasizing synthetic data and STEM performance.

   Abstract

   We present phi-4, a 14-billion parameter language model developed with a training recipe that is centrally focused on data quality. Unlike most language models, where pre-training is based primarily on organic data sources such as web content or code, phi-4 strategically incorporates synthetic data throughout the training process. While previous models in the Phi family largely distill the capabilities of a teacher model (specifically GPT-4), phi-4 substantially surpasses its teacher model on STEM-focused QA capabilities, giving evidence that our data-generation and post-training techniques go beyond distillation. Despite minimal changes to the phi-3 architecture, phi-4 achieves strong perfo…

3. Close Cited by this paper Not relevant

   Textbooks Are All You Need

   Suriya Gunasekar, Yi Zhang, J. Aneja et al. · 2023 · arXiv.org · preprint, not peer reviewed · cited 691 times · Not summarized yet

   Abstract

   We introduce phi-1, a new large language model for code, with significantly smaller size than competing models: phi-1 is a Transformer-based model with 1.3B parameters, trained for 4 days on 8 A100s, using a selection of ``textbook quality"data from the web (6B tokens) and synthetically generated textbooks and exercises with GPT-3.5 (1B tokens). Despite this small scale, phi-1 attains pass@1 accuracy 50.6% on HumanEval and 55.5% on MBPP. It also displays surprising emergent properties compared to phi-1-base, our model before our finetuning stage on a dataset of coding exercises, and phi-1-small, a smaller model with 350M parameters trained with the same pipeline as phi-1 that still achieves …

4. Close Groundwork for this paper Cited by this paper A method it uses Not relevant

   Textbooks Are All You Need II: phi-1.5 technical report

   Yuanzhi Li, Sébastien Bubeck, Ronen Eldan et al. · 2023 · Computation and Language · preprint, not peer reviewed · cited 708 times · AI summary available

   Phi-1.5 investigates small language models trained with generated textbook-quality data, an approach that informs the Phi family’s focus on curated and synthetic training data.

   Abstract

   We continue the investigation into the power of smaller Transformer-based language models as initiated by \textbf{TinyStories} -- a 10 million parameter model that can produce coherent English -- and the follow-up work on \textbf{phi-1}, a 1.3 billion parameter model with Python coding performance close to the state-of-the-art. The latter work proposed to use existing Large Language Models (LLMs) to generate ``textbook quality" data as a way to enhance the learning process compared to traditional web data. We follow the ``Textbooks Are All You Need" approach, focusing this time on common sense reasoning in natural language, and create a new 1.3 billion parameter model named \textbf{phi-1.5},…

5. Close Broader context Not relevant

   Nanbeige4-3B Technical Report: Exploring the Frontier of Small Language Models

   Chen Yang, Guangyue Peng, Jiaying Zhu et al. · 2025 · Computation and Language · preprint, not peer reviewed · cited 6 times · AI summary available

   Nanbeige4-3B also targets high-performing small language models, using large-scale pretraining and refined instruction data.

   Abstract

   We present Nanbeige4-3B, a family of small-scale but high-performing language models. Pretrained on 23T high-quality tokens and finetuned on over 30 million diverse instructions, we extend the boundary of the scaling law for small language models. In pre-training, we design a Fine-Grained Warmup-Stable-Decay (FG-WSD) training scheduler, which progressively refines data mixtures across stages to boost model performance. In post-training, to improve the quality of the SFT data, we design a joint mechanism that integrates deliberative generation refinement and chain-of-thought reconstruction, yielding substantial gains on complex tasks. Following SFT, we employ our flagship reasoning model to d…

6. Close Cites this paper Uses this paper in depth Not relevant

   A comparison of local LLMs in paragraph decomposition context

   Nannicha Phraemetta, Nattawadee Wuttivoradit, Chayada Muangboonsri et al. · 2026 · International Conference on Advances in Information Technology · Not summarized yet

   Abstract

   This study presents a comparative evaluation of three locally deployable large language models—LLaMA 3.1 8B, Mistral 7B, and Phi-3.5 Mini—on paragraph decomposition tasks using real-world student feedback. Unlike prior work that primarily targets large cloud-hosted models, this study operates entirely on local infrastructure using 4-bit quantization in a Google Colab environment. A standardized instruction-based prompting strategy with a single in-context example was applied across all models using a single experimental configuration with greedy decoding and a repetition penalty of ρ = 1.05. Outputs were manually evaluated against seven error categories and two caution labels covering struct…

7. Close Not relevant

   BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model

   Nolan Dey, Daria Soboleva, Faisal Al-Khateeb et al. · 2023 · arXiv.org · preprint, not peer reviewed · cited 24 times · Not summarized yet

   Abstract

   We introduce the Bittensor Language Model, called"BTLM-3B-8K", a new state-of-the-art 3 billion parameter open-source language model. BTLM-3B-8K was trained on 627B tokens from the SlimPajama dataset with a mixture of 2,048 and 8,192 context lengths. BTLM-3B-8K outperforms all existing 3B parameter models by 2-5.5% across downstream tasks. BTLM-3B-8K is even competitive with some 7B parameter models. Additionally, BTLM-3B-8K provides excellent long context performance, outperforming MPT-7B-8K and XGen-7B-8K on tasks up to 8,192 context length. We trained the model on a cleaned and deduplicated SlimPajama dataset; aggressively tuned the \textmu P hyperparameters and schedule; used ALiBi posit…

8. Close Not relevant

   Hints-In-Browser: Benchmarking Language Models for Programming Feedback Generation

   Nachiket Kotalwar, Alkis Gotovos, Adish Singla · 2024 · Machine Learning · cited 22 times · AI summary available

   Abstract

   Generative AI and large language models hold great promise in enhancing programming education by generating individualized feedback and hints for learners. Recent works have primarily focused on improving the quality of generated feedback to achieve human tutors' quality. While quality is an important performance criterion, it is not the only criterion to optimize for real-world educational deployments. In this paper, we benchmark language models for programming feedback generation across several performance criteria, including quality, cost, time, and data privacy. The key idea is to leverage recent advances in the new paradigm of in-browser inference that allow running these models directl…

Prior work it builds on

1. Close Broader context A method it uses Not relevant

   LLaMA: Open and Efficient Foundation Language Models

   Hugo Touvron, Thibaut Lavril, Gautier Izacard et al. · 2023 · Computation and Language · preprint, not peer reviewed · cited 22,457 times · AI summary available

   LLaMA shows strong benchmark performance from models trained on trillions of publicly available tokens, addressing efficient language-model training at larger scales.

   Abstract

   We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

2. Related Not relevant

   Program Synthesis with Large Language Models

   Jacob Austin, Augustus Odena, Maxwell Nye et al. · 2021 · Programming Languages · preprint, not peer reviewed · cited 4,982 times · AI summary available

   Abstract

   This paper explores the limits of the current generation of large language models for program synthesis in general purpose programming languages. We evaluate a collection of such models (with between 244M and 137B parameters) on two new benchmarks, MBPP and MathQA-Python, in both the few-shot and fine-tuning regimes. Our benchmarks are designed to measure the ability of these models to synthesize short Python programs from natural language descriptions. The Mostly Basic Programming Problems (MBPP) dataset contains 974 programming tasks, designed to be solvable by entry-level programmers. The MathQA-Python dataset, a Python version of the MathQA benchmark, contains 23914 problems that evaluat…

3. Related Not relevant

   Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

   Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao et al. · 2022 · arXiv.org · preprint, not peer reviewed · cited 2,714 times · Not summarized yet

   Abstract

   Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabilities are as yet poorly characterized. In order to inform future research, prepare for disruptive new model capabilities, and ameliorate socially harmful effects, it is vital that we understand the present and near-future capabilities and limitations of language models. To address this challenge, we introduce the Beyond the Imitation Game benchmark (BIG-bench). BIG-bench currently consists of 204 tasks, contributed by 450 authors across 132 institutions. Task topics are diverse, drawing problems from linguistics, c…

4. Related Used in depth by this paper Not relevant

   How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites

   Zhe Chen, Wei-Yun Wang, Hao Tian et al. · 2024 · Science China Information Sciences · cited 1,328 times · Not summarized yet

   Abstract

   In this paper, we introduce InternVL 1.5, an open-source multimodal large language model (MLLM) to bridge the capability gap between open-source and proprietary commercial models in multimodal understanding. We introduce three simple improvements. (1) Strong vision encoder: we explored a continuous learning strategy for the large-scale vision foundation model — InternViT-6B, boosting its visual understanding capabilities, and making it can be transferred and reused in different LLMs. (2) Dynamic high-resolution: we divide images into tiles ranging from 1 to 40 of 448×448 pixels according to the aspect ratio and resolution of the input images, which supports up to 4K resolution input. (3) Hig…

5. Related Used in depth by this paper Not relevant

   Improved Baselines with Visual Instruction Tuning

   Haotian Liu, Chunyuan Li, Yu-Heng Li et al. · 2023 · Computer Vision and Pattern Recognition · cited 6,142 times · AI summary available

   Abstract

   Large multimodal models (LMM) have recently shown encouraging progress with visual instruction tuning. In this paper, we present the first systematic study to investigate the design choices of LMMs in a controlled setting under the LLaVA framework. We show that the fully-connected vision-language connector in LLaVA is surprisingly power-ful and data-efficient. With simple modifications to LLa VA, namely, using CLIP- ViT-L-336px with an MLP projection and adding academic-task-oriented VQA data with response formatting prompts, we establish stronger baselines that achieve state-of-the-art across 11 benchmarks. Our final 13B checkpoint uses merely 1.2M publicly available data, and finishes full…

Papers that cite it

1. Close Uses this paper in depth Not relevant

   Educational Robotics With On-Device AI: An Interactive System Based on the Phi-3 Mini Small Language Model Running on Conventional Hardware

   Vitor Amadeu Souza · 2026 · Anais do LIII Seminário Integrado de Software e Hardware (SEMISH 2026) · Not summarized yet

   Abstract

   This work presents an educational robotics system integrating speech recognition, a Small Language Model (SLM), and speech synthesis on GPU-free hardware. It uses Google Speech Recognition, Phi-3 Mini via Ollama with 4-bit quantization, pyttsx3, and Pygame. Experiments on an Intel Core i5-3570 with 16 GB RAM evaluated five interactions. The average latency was 30.16 s, with speech recognition (6.80 s), SLM processing (8.06 s), and synthesis (15.30 s). Speech synthesis was the main bottleneck (50.73%), followed by SLM (26.72%) and recognition (22.55%). Results demonstrate feasibility on low-cost hardware while highlighting the need for latency optimization for more natural interaction.

2. Close Not relevant

   Optimizing the Phi-2 Small Language Model for Real-time Chatbot Applications Using Parameter-Efficient Fine-Tuning (PEFT) with QLoRA Quantization

   P. Nguyen, Ashfaq Ali Shafin, Khandaker Mamun Ahmed · 2026 · Not summarized yet

   Abstract

   This study explores the optimization of the Phi-2 Small Language Models (SLMs) for real-time chatbot applications through Parameter-Efficient Fine-Tuning (PEFT) and Quantized Low-Rank Adaptation (QLoRA). QLoRA specifically refers to the integration of PEFT with LoRA alongside a 4-bit quantization process, aimed at enhancing computational efficiency. These models, initially designed for high performance with minimal computational overhead, are further refined to address the constraints of mobile and edge computing environments. By integrating PEFT with QLoRA, the research aims to reduce memory usage significantly while maintaining, or potentially improving, the accuracy of model responses in …

3. Related Not relevant

   A Comparative Analysis of Instruction Fine-Tuning Large Language Models for Financial Text Classification

   Sorouralsadat Fatemi, Yuheng Hu, Maryam Mousavi · 2024 · ACM Transactions on Management Information Systems · cited 27 times · Not summarized yet

   Abstract

   Large Language Models (LLMs) have demonstrated impressive capabilities across diverse Natural Language Processing (NLP) tasks, including language understanding, reasoning, and generation. However, general-domain LLMs often struggle with financial tasks due to the technical and specialized nature of financial texts. This study investigates the efficacy of instruction fine-tuning smaller-scale LLMs, including Mistral-7B, Llama3-8B, and Phi3-mini, to enhance their performance in financial text classification tasks. We fine-tuned both instruction-tuned and base models across four financial classification tasks, achieving significant improvements in task-specific performance. Furthermore, we eval…

4. Related Not relevant

   CallScreenBench: Benchmarking Small Language Models as Phone Secretaries

   Jia-Qi Gan, Hao Tang, Jamey Z. Liang et al. · 2026 · cited 3 times · Not summarized yet

   Abstract

   Language models small enough to run on a handset, quantized to a few bits, are increasingly capable of acting on their user's behalf -- which makes on-device task automation newly plausible. One such task is answering the phone. A phone secretary takes an unknown inbound call on its owner's behalf, and unlike the agents most benchmarks evaluate, it has no cooperative caller-assigned task to complete: the caller holds the goal and may be an adversary, while the secretary must begin deciding how to respond without an oracle. What matters is not task success but whether the owner would endorse how their proxy handled the call. We evaluate only the text-domain conversational decision layer; spee…

5. Related Not relevant

   Performance Profiling of Small Language Models for Domain-Specific Recommender Systems: A Case Study in Personalized Meal Planning

   A. K, Prerna Pareek · 2026 · International Journal For Multidisciplinary Research · Not summarized yet

   Abstract

   Background: The integration of large language models (LLMs) in clinical nutrition recommender systems is limited by computational complexity, privacy concerns with patient data and a lack of mechanistic explanation. Small language models (SLMs) with fewer than 4 billion parameters are available for edge deployment, but their use in clinical reasoning in domain specific nutrition problems is not yet explored especially in India. Objective: This study evaluates and compares the clinical reasoning performance of four SLMs (Phi-3.5-mini-instruct, Phi-4-mini-instruct, Qwen2.5-3B-Instruct and Llama-3.2-3B-Instruct) in a Retrieval-Augmented Generation (RAG) framework for customized Indian clinical …