# Learning to Program with Natural Language

**Authors:** Yiduo Guo, Yaobo Liang, Chenfei Wu, Wenshan Wu, Dongyan Zhao, Nan Duan  
**arXiv:** [2304.10464](https://arxiv.org/abs/2304.10464)  
**Date:** 2023-04-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.10464v1/

## Key points

- Large Language Models (LLMs) are good at natural language tasks but struggle with complex tasks that require combining basic capabilities.
- Natural language can be used as a new programming language to teach LLMs how to complete complex tasks.
- The inference process is divided into two steps: finding a natural language program for the test sample and then using it to generate a specific solution.
- Two methods for finding natural language programs are introduced: Self-Program (SP) and Learning to Program (LP).
- LP method learns the natural language program in text form based on the training dataset, which can guide LLMs in solving complex tasks more effectively than SP methods alone.
- The authors demonstrate the effectiveness of their approach through experiments on high school math and competition mathematics problems datasets.
- Natural Language Programs offer generalizability and human understandability advantages as they can guide LLMs to solve all questions similar to the test sample while being readable and editable by humans.
- Leveraging natural language as a new programming language can enhance LLMs' capabilities in completing complex tasks systematically.
- The authors release their code at https://github.com/microsoft/NaturalLanguageProgram for further research into this field of study.

## Summary

Large Language Models (LLMs) have shown impressive performance in various natural language tasks, such as generating passages and solving grade school math problems. However, they still struggle with complex tasks that require a procedure for combining basic capabilities. To address this issue, the authors propose using natural language as a new programming language to teach LLMs how to complete complex tasks. They suggest making the inference process into two steps: finding a natural language program for the test sample and then using it to generate a specific solution. These programs guide LLMs by analyzing all possible cases, decomposing the task into sub-tasks, and employing functions sequentially to complete it. The authors introduce two methods for finding natural language programs: Self-Program (SP) and Learning to Program (LP). The SP method involves asking LLMs to generate a program consisting of general solutions but may result in programs with factual errors or incompleteness. In contrast, LP learns the natural language program in text form based on the training dataset, which can guide LLMs in solving complex tasks more effectively. The LP method is inspired by traditional deep learning processes but differs in that it learns program p as text instead of vector weight. The authors demonstrate the effectiveness of their approach through experiments on high school math and competition mathematics problems datasets. Their LP method outperforms direct zero-shot test performance by 18.3% when testing ChatGPT on ten tasks from the AMPS dataset. Natural Language Programs offer generalizability and human understandability advantages as they can guide LLMs to solve all questions similar to the test sample while being readable and editable by humans. In conclusion, leveraging natural language as a new programming language can enhance LLMs' capabilities in completing complex tasks systematically. The proposed LP method provides an effective way of learning natural language programs from training datasets that can guide LLMs towards solving complex tasks more accurately than SP methods alone. The authors release their code at https://github.com/microsoft/NaturalLanguageProgram for further research into this field of study.