# Predicting the Performance of Black-box LLMs through Self-Queries

**Authors:** Dylan Sam, Marc Finzi, J. Zico Kolter  
**arXiv:** [2501.01558](https://arxiv.org/abs/2501.01558)  
**Date:** 2025-01-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2501.01558v1/

## Key points

- The reliance on large language models (LLMs) in AI systems is increasing.
- Predicting when LLMs may make mistakes is crucial.
- Accessing internal representations of LLMs is challenging with black-box access through an API.
- A novel approach using follow-up prompts and analyzing response probabilities can extract features of LLMs in a black-box manner.
- Training a linear model on these low-dimensional representations yields reliable and generalizable predictors of model performance at the instance level.
- These predictors can determine if a specific generation correctly answers a question and outperform white-box linear predictors.
- Extracted features can help evaluate nuanced aspects of LLMs, such as distinguishing between clean versions and those influenced by adversarial prompts or bugs in generated code.
- The features are effective in distinguishing between different model architectures and sizes, enabling detection of misrepresented models provided through an API.
- By predicting model behavior through self-queries and leveraging small amounts of labeled data, this approach offers promising results comparable to examining activations directly.

## Summary

In the realm of AI systems, the reliance on large language models (LLMs) is steadily increasing. It has become imperative to predict when these models may make mistakes. Previous research has delved into interpreting model behavior using internal representations. However, accessing these representations becomes challenging when only provided with black-box access through an API. This paper introduces a novel approach to extracting features of LLMs in a black-box manner by utilizing follow-up prompts and analyzing the probabilities of different responses as representations to train reliable predictors of model behavior. The study demonstrates that training a linear model on these low-dimensional representations yields dependable and generalizable predictors of model performance at the instance level. These predictors can determine if a specific generation correctly answers a question. Surprisingly, they often outperform white-box linear predictors that operate over a model's hidden state or the full distribution over its vocabulary. Furthermore, the extracted features can be utilized to evaluate more nuanced aspects of a language model's state. This includes distinguishing between clean versions of LLMs and those influenced by adversarial system prompts that lead to incorrect question-answering tasks or introduce bugs into generated code. Additionally, the extracted features prove effective in distinguishing between different model architectures and sizes, enabling the detection of misrepresented models provided through an API. By predicting model behavior and performance through self-queries and leveraging small amounts of labeled data from downstream tasks, this work offers promising results that can sometimes match the performance achieved by examining activations directly.