JUCS - Journal of Universal Computer Science 26(1): 89-106, doi: 10.3897/jucs.2020.006
Scalable Distributed Metadata Server Based on Nonblocking Transactions
Kohei Hiraga‡, Osamu Tatebe§, Hideyuki Kawashima‡‡ Keio University, Kanagawa, Japan§ University of Tsukuba, Ibaraki, Japan
Corresponding author: Kohei Hiraga ( hiraga@keio.jp ) © Kohei Hiraga, Osamu Tatebe, Hideyuki Kawashima. This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY-ND 4.0). This license allows reusers to copy and distribute the material in any medium or format in unadapted form only, and only so long as attribution is given to the creator. The license allows for commercial use. Citation:
Hiraga K, Tatebe O, Kawashima H (2020) Scalable Distributed Metadata Server Based on Nonblocking Transactions. JUCS - Journal of Universal Computer Science 26(1): 89-106. https://doi.org/10.3897/jucs.2020.006 |  |
Abstract
Metadata performance scalability is critically important in high-performance computing when accessing many small files from millions of clients. This paper proposes a design of a scalable distributed metadata server, PPMDS, for parallel file systems using multiple key-value servers. In PPMDS, hierarchical namespace of a file system is efficiently managed by multiple servers. Multiple entries can be atomically updated using a nonblocking distributed transaction based on an algorithm of dynamic software transactional memory. This paper also proposes optimizations to further improve the metadata performance by introducing a server-side transaction processing, multiple readers, and a shared lock mode, which reduce the number of remote procedure calls and prevent unnecessary blocking. Performance evaluation shows the scalable performance up to 3 servers, and achieves 62,000 operations per second, which is 2.58x performance improvement compared to a single metadata performance.
Keywords
distributed metadata server, nonblocking distributed transaction, parallel file system