Please use this identifier to cite or link to this item:
Title: OctopuSFS: A distributed file system with tiered storage management
Authors: Kakoulli, Elena 
Herodotou, Herodotos 
Keywords: Cluster computing;Data handling;Digital storage;Distributed computer systems;Distributed database systems;Fault tolerance;File organization;Molluscs;Multiobjective optimization;Multiprocessing systems;Storage management
Category: Electrical Engineering - Electronic Engineering - Information Engineering
Field: Engineering and Technology
Issue Date: May-2017
Publisher: Association for Computing Machinery
Source: Proceedings of the 2017 ACM International Conference on Management of Data, Pages 65-78
Abstract: The ever-growing data storage and I/O demands of modern large-scale data analytics are challenging the current distributed storage systems. A promising trend is to exploit the recent improvements in memory, storage media, and networks for sustaining high performance and low cost. While past work explores using memory or SSDs as local storage or combine local with network-attached storage in cluster computing, this work focuses on managing multiple storage tiers in a distributed setting. We present OctopusFS, a novel distributed file system that is aware of heterogeneous storage media (e.g., memory, SSDs, HDDs, NAS) with different capacities and performance characteristics. The system offers a variety of pluggable policies for automating data management across the storage tiers and cluster nodes. The policies employ multi-objective optimization techniques for making intelligent data management decisions based on the requirements of fault tolerance, data and load balancing, and throughput maximization. At the same time, the storage media are explicitly exposed to users and applications, allowing them to choose the distribution and placement of replicas in the cluster based on their own performance and fault tolerance requirements. Our extensive evaluation shows the immediate benefits of using OctopusFS with data-intensive processing systems, such as Hadoop and Spark, in terms of both increased performance and better cluster utilization.
ISBN: 9781450341974
Rights: © 2017 Copyright held by the owner/author(s).
Type: Book Chapter
Appears in Collections:Κεφάλαια βιβλίων/Book chapters

Show full item record

Page view(s) 5

Last Week
Last month
checked on Nov 15, 2018

Google ScholarTM


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.