# Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens

**Authors:** Wei-Lin Chen, Liqian Peng, Tian Tan, Chao Zhao, Blake JianHang Chen, Ziqian Lin, Alec Go, Yu Meng  
**arXiv:** [2602.13517](https://arxiv.org/abs/2602.13517)  
**Date:** 2026-02-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2602.13517v1/

## Key points

- Authors Wei-Lin Chen, Liqian Peng, Tian Tan, Chao Zhao, Blake JianHang Chen, Ziqian Lin, Alec Go, and Yu Meng study large language models (LLMs) and their reasoning capabilities.
- They introduce a novel approach to quantify inference-time effort through identifying deep-thinking tokens in LLMs.
- Deep-thinking tokens represent points in the generated sequence where internal predictions undergo significant revisions in deeper model layers before converging.
- The deep-thinking ratio shows a robust and consistently positive correlation with accuracy across various reasoning-focused models and benchmarks.
- This metric outperforms traditional length-based and confidence-based baselines.
- The authors propose Think@n - a test-time scaling strategy that prioritizes samples with high deep-thinking ratios to enhance LLM reasoning capabilities while reducing inference costs.

## Summary

In their study titled "Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens," authors Wei-Lin Chen, Liqian Peng, Tian Tan, Chao Zhao, Blake JianHang Chen, Ziqian Lin, Alec Go, and Yu Meng delve into the realm of large language models (LLMs) and their reasoning capabilities. The authors address the issue of unreliable raw token counts as indicators of reasoning quality in LLMs by introducing a novel approach to quantify inference-time effort through identifying deep-thinking tokens. These tokens represent points in the generated sequence where internal predictions undergo significant revisions in deeper model layers before converging. Through extensive experimentation across four challenging mathematical and scientific benchmarks using various reasoning-focused models, the authors establish that the deep-thinking ratio exhibits a robust and consistently positive correlation with accuracy. This metric outperforms traditional length-based and confidence-based baselines. Building upon this insight, they propose Think@n - a test-time scaling strategy that prioritizes samples with high deep-thinking ratios. By leveraging this strategy, they demonstrate its effectiveness in enhancing LLM reasoning capabilities while reducing inference costs. Overall, this study provides valuable insights into improving LLM performance by focusing on deep-thinking tokens rather than solely relying on generation length as a measure of quality.