# Metadata Caching in Presto: Towards Fast Data Processing

**Authors:** Beinan Wang, Chunxu Tang, Rongrong Zhong, Bin Fan, Yi Wang, Jasmine Wang, Shouwei Chen, Bowen Ding, Lu Zhang  
**arXiv:** [2211.10889](https://arxiv.org/abs/2211.10889)  
**Date:** 2022-11-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.10889v1/

## Key points

- Presto is an open-source distributed SQL query engine for OLAP
- Presto has gained significant popularity in large-scale data analytics
- Parsing column-oriented data files in Presto consumes a considerable amount of CPU resources
- The authors propose a metadata caching layer as a solution to this issue
- The caching layer is built on top of the Alluxio SDK cache and integrated into each Presto worker node
- Two methods of caching are offered: caching decompressed metadata bytes and caching deserialized metadata objects
- Experiments using the TPC-DS benchmark show that both methods can reduce CPU consumption during queries
- The first method reduces CPU consumption by 10%-20% when the cache is warm
- The second method achieves even greater reductions, reducing CPU usage by 20%-40%
- This approach improves data processing efficiency in Presto and enables handling larger volumes of analytical data effectively.

## Summary

Presto is an open-source distributed SQL query engine that aims to provide "SQL on everything" for OLAP (Online Analytical Processing). Since its release in 2013, Presto has gained significant popularity in the field of large-scale data analytics and has been adopted by numerous enterprises. However, during the development and operation of Presto, it was observed that parsing column-oriented data files in Presto worker nodes consumed a considerable amount of CPU resources. This issue hindered companies from scaling up their analytical data volumes. To address this problem, the authors propose a solution in the form of a metadata caching layer. This caching layer is built on top of the Alluxio SDK cache and is integrated into each Presto worker node. Its purpose is to cache intermediate results during file parsing. The metadata cache offers two methods of caching: caching decompressed metadata bytes from raw data files and caching deserialized metadata objects. In order to evaluate the effectiveness of their approach, the authors conducted experiments using the TPC-DS benchmark on Presto. The results demonstrate that when the cache is warm (i.e., already populated with relevant data), the first method can reduce CPU consumption during queries by 10%-20%. On the other hand, utilizing the second method can minimize CPU usage by an even greater extent, achieving reductions of 20%-40%. Overall, this paper presents a valuable contribution to improving data processing efficiency in Presto through the implementation of a metadata caching layer. By reducing CPU consumption during file parsing, this approach enables companies to handle larger volumes of analytical data effectively.