# PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance

**Authors:** Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, Jimin Huang  
**arXiv:** [2306.05443](https://arxiv.org/abs/2306.05443)  
**Date:** 2023-06-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.05443v1/

## Key points

- Introduction of PIXIU framework for advancing financial AI development
- Addressing the lack of publicly available financial tailored large language models (LLMs), instruction tuning datasets, and evaluation benchmarks
- Proposal of FinMA, the first financial LLM based on fine-tuning LLaMA with instruction data
- Construction of a large-scale multi-task instruction dataset covering various financial tasks, document types, and data modalities
- Introduction of an evaluation benchmark consisting of five financial NLP tasks and one financial prediction task
- Detailed analysis of FinMA and existing LLMs using the proposed benchmark to identify strengths and weaknesses in handling critical financial tasks
- Open-sourcing the model, datasets, benchmark, and experimental results to facilitate future research in financial AI.

## Summary

This paper introduces PIXIU, a comprehensive framework for advancing the development of financial artificial intelligence (AI). The authors address the lack of publicly available financial tailored large language models (LLMs), instruction tuning datasets, and evaluation benchmarks. They propose the first financial LLM called FinMA, which is based on fine-tuning LLaMA with instruction data. To support the fine-tuning process, they construct a large-scale multi-task instruction dataset that covers various financial tasks, document types, and data modalities. Additionally, they introduce an evaluation benchmark consisting of five financial NLP tasks and one financial prediction task. The authors conduct a detailed analysis of FinMA and several existing LLMs using the proposed benchmark. This analysis reveals the strengths and weaknesses of these models in handling critical financial tasks. The model, datasets, benchmark, and experimental results are open-sourced to facilitate future research in financial AI. Overall, this paper presents an important contribution to the field by providing a comprehensive framework for developing and evaluating financial LLMs. The availability of tailored models, instruction data, and evaluation benchmarks will greatly benefit the open-source development of financial AI.