# Large Language Models are Built-in Autoregressive Search Engines

**Authors:** Noah Ziems, Wenhao Yu, Zhihan Zhang, Meng Jiang  
**arXiv:** [2305.09612](https://arxiv.org/abs/2305.09612)  
**Date:** 2023-05-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.09612v1/

## Key points

- Large language models (LLMs) can generate Web URLs for document retrieval based on query-URL pairs as demonstrations.
- LLMs can act as built-in search engines without explicit training for mapping questions to document identifiers.
- LLMs achieve nearly 90% accuracy in generating URLs that lead to documents with correct answers to open-domain questions.
- The method consistently outperforms existing retrieval approaches on three open-domain question answering benchmarks, in both zero-shot and few-shot settings.
- Future research directions include fine-tuning prompts for individual questions and using clustering to select diverse demonstrations for improved retrieval performance.
- Limitations include the need for retraining when updating knowledge, potential hallucination errors, and slow web requests and document processing.
- A case study comparing LLM-URL approach with Contriever and BM25 retrievers shows superior performance in retrieving answer-containing documents.
- LLMs offer effective URL generation for document retrieval but face challenges related to knowledge updating, hallucination errors, and practical usability.

## Summary

This paper investigates the use of large language models (LLMs) for document retrieval by generating URLs prompted by human instructions. The authors find that by providing a few (query, URL) pairs as in-context demonstrations, LLMs can generate Web URLs where nearly 90% of the corresponding documents contain correct answers to open-domain questions. This suggests that LLMs can act as built-in search engines without explicit training for mapping questions to document identifiers. Experimental results demonstrate that this method consistently outperforms existing retrieval approaches on three open-domain question answering benchmarks, both in zero-shot and few-shot settings. The authors also explore future directions for research. They discuss the potential for further tuning the few-shot demonstrations for specific retrieval tasks and improving retrieval performance by fine-tuning prompts for individual questions. They mention prior work on using clustering to select diverse demonstrations which has shown improvements in retrieval and downstream QA performance. However, there are limitations to their approach. The current method is limited in its ability to update knowledge state and adapt to new domains; adding new knowledge would likely require retraining the large language model. Additionally, large generation models may suffer from hallucination errors leading to incorrect predictions. The approach also involves slow web requests and document processing making it cumbersome to use in practice. To illustrate the advantages of their LLM-URL approach over existing methods a case study is presented comparing it with Contriever and BM25 retrievers in a closed-book setting; LLM-URL performs significantly better by retrieving documents containing the answer while Contriever and BM25 fail to do so. In conclusion, this paper demonstrates that LLMs can effectively generate URLs for document retrieval when provided with query-URL pairs as demonstrations; showing superior retrieval performance compared to existing approaches on various benchmarks. However, there are still challenges regarding knowledge updating, hallucination errors and practical usability that need further exploration.