Résultats de recherche pour « Metadata Caching »

Cloud storage is a hot topic in current research. Different from previous work, we emphasize the importance of metadata cache in the study of cloud storage. Because the efficiency of distributed file system has much effect on cloud storage The metadata operation accounts for more than 50% of the total file operation. So the strategy of efficient metadata management is important. There are three parts in this paper. We start with a brief introduction of cloud storage. Then a metadata caching algorithm for cloud storage is proposed. An additional discussion of its performance is also provided. The prototype which incorporates the proposed metadata caching algorithm is realized on Luster to evaluate its performance. Comparing experimental results from this study conclude that the metadata caching subsystem can improve the performance of cloud storage.

Presto is an open-source distributed SQL query engine for OLAP, aiming for “SQL on everything”. Since open-sourced in 2013, Presto has been consistently gaining popularity in large-scale data analytics and attracting adoption from a wide range of enterprises. From the development and operation of Presto, we witnessed a significant amount of CPU consumption on parsing column-oriented data files in Presto worker nodes. This blocks some companies, including Meta, from increasing analytical data volumes.In this paper, we present a metadata caching layer, built on top of the Alluxio SDK cache and incorporated in each Presto worker node, to cache the intermediate results in file parsing. The metadata cache provides two caching methods: caching the decompressed metadata bytes from raw data files and caching the deserialized metadata objects. Our evaluation of the TPC-DS benchmark on Presto demonstrates that when the cache is warm, the first method can reduce the query’s CPU consumption by 10%-20%, whereas the second method can minimize the CPU usage by 20%-40%.

Fast and scalable metadata management across multiple metadata servers is crucial for distributed file systems to handle numerous files and directories. Client-side caching of frequently accessed metadata can mitigate server loads, but it incurs significant overhead and complexity in maintaining cache consistency when the number of clients increases. We propose Fletch, an in-switch file-system metadata caching framework that leverages programmable switches to serve file-system metadata requests from multiple clients directly in the switch data plane. Unlike prior in-switch key-value caching systems, Fletch addresses file-system-specific path dependencies under stringent switch resource constraints. We implement Fletch atop HDFS and evaluate it on a Tofino-switch testbed using real-world filesystem metadata workloads. Fletch achieves up to 181.6% higher throughput than vanilla HDFS and complements client-side caching with throughput gains of up to 139.6% on 128 simulated metadata servers.

Nowadays, Linux file systems have to manage millions of tiny files for different applications, and face with higher metadata operations. So how to provide such high metadata performance with such enormous number of files and large scale directories is a big challenge for Linux file system. We viewed that metadata lookup operations dominate metadata workload and incur low metadata performance. In this paper, we present a metadata cache to accelerate metadata access for Linux file system. Through this optimization, the Linux file system (such as EXT2, EXT4, BTRFS, etc.) can gain improvement in read rates as well as write rates.

This paper presents an analysis of a performance bottleneck in enterprise file servers using Linux and proposes a modification to this operation system for avoiding the bottleneck. The analysis shows that metadata cache deallocation of current Linux causes large latency in file-request processing when the operational throughput of a file server becomes large. To eliminate the latency caused by metadata cache deallocation, a new method, called “split reclaim,” which divides metadata cache deallocation from conventional cache deallocation, is proposed. It is experimentally shown that the split-reclaim method reduces the worst response time by more than 95% and achieves three times higher throughput under a metadata-intensive workload. The split-reclaim method also reduces latency caused by cache deallocation under a general file-server workload by more than 99%. These results indicate that the split-reclaim method can eliminate metadata cache deallocation latency and make possible the use of commodity servers as enterprise file servers.

Caching strategies and the resulting cache coherence control technique have become the key techniques for the system development.Based on the study of the cache technology of the home and abroad distributed file systems, and combined with the design requirements and characteristics for our multi-user parallel file system Cappella, this paper proposes a new meta-data cache management strategy.And this strategy not only improves the performance of metadata services, reducing the user access latency, but also controls metadata cache consistency very well.

Cache replacement is central to data system performance, yet metadata caches exhibit access patterns that differ substantially from data caches. In particular, metadata blocks often exhibit dense bursts of correlated references (e.g., multiple lookups that touch tuples that co-reside on the same metadata page). Treating such bursts as repeated independent hits can misclassify cold metadata as hot, polluting the cache and degrading hit ratio. We present Clock2Q+, a cache eviction algorithm deployed in the metadata cache of VMware vSAN, a production-grade distributed storage system. Clock2Q+ is the latest stage of nearly a decade of iterative evolution of vSAN's caching layer, shaped by real-world deployment, customer-driven feature requests, and an emphasis on simplicity as a first-class requirement for long-term maintainability. Clock2Q+ has a three-queue structure similar to S3-FIFO, and introduces a correlation window at the head of the Small FIFO in which accesses do not set the reference bit, preventing short-term correlated bursts from triggering premature promotion; accesses beyond this window set the bit so genuinely hot blocks are promoted efficiently. Clock2Q+ achieves the highest median and mean miss-ratio improvement among the state-of-the-art policies we evaluate on both metadata and data workloads, with the largest per-trace gap over S3-FIFO reaching a 28.5% relative miss-ratio reduction on metadata traces. To support production deployment, Clock2Q+ is engineered for low per-hit overhead and multi-core scalability, with allocation-free, array-based structures, fine-grained synchronization, dirty-page-aware eviction and promotion, bounded per-eviction work, and online cache resizing.

In the ATLAS event store, files are sometimes 'an inconvenient truth.' From the point of view of the ATLAS distributed data management system, files are too small—datasets are the units of interest. From the point of view of the ATLAS event store architecture, files are simply a physical clustering optimization: the units of interest are event collections—sets of events that satisfy common conditions or selection predicates—and such collections may or may not have been accumulated into files that contain those events and no others. It is nonetheless important to maintain file-level metadata, and to cache metadata in event data files. When such metadata may or may not be present in files, or when values may have been updated after files are written and replicated, a clear and transparent model for metadata retrieval from the file itself or from remote databases is required. In this paper we describe how ATLAS reconciles its file and non-file paradigms, the machinery for associating metadata with files and event collections, and the infrastructure for metadata propagation from input to output for provenance record management and related purposes.

Hybrid storage systems integrate slower HDDs with faster SSDs to improve overall performance. To this end, various research efforts have investigated how to use SSDs as cache for HDDs. One of them proposes to cache only metadata blocks because all IO requests are accompanied with the requests to the metadata blocks before issuing the requests for the data blocks. However, our evaluation of the method could not confirm the claimed improvement. It motivated us to modify the method to integrate the caching functionality. The implementation of the proposed method in the Linux systems required to address several challenging issues. The evaluation results showed that the proposed method improved the performance by at least 90% over the original scheme. It implicitly means that caching the data blocks has more significant impact on performance than the metadata caching.

In SAN systems with a multilevel caching hierarchy, the storage cache lacks visibility into the page cache of the compute node, resulting in redundant caching of the same blocks and inefficient use of limited cache space. Prior efforts have explored exclusive caching, but they often incur network overhead by transferring evicted pages for cache admission and require intrusive modifications to components on compute nodes. In this paper, we present BNEX, a metadata-driven exclusive caching system for NVMe-oF SANs that significantly reduces network overhead by avoiding full-page transfers. BNEX non-intrusively traces page cache evictions using eBPF and transmits only lightweight block-level metadata. Leveraging metadata of evicted pages, the storage cache prefetches nearby blocks in fixed-size block groups and selectively retains them without compromising exclusivity. Our evaluation shows that BNEX improves throughput by up to 1.15× and cache hit ratio by up to 8× compared to inclusive SAN caching systems.