Please use this identifier to cite or link to this item:
|Title:||OctopuSFS: A distributed file system with tiered storage management||Authors:||Kakoulli, Elena
|Keywords:||Cluster computing;Data handling;Digital storage;Distributed computer systems;Distributed database systems;Fault tolerance;File organization;Molluscs;Multiobjective optimization;Multiprocessing systems;Storage management||Category:||Electrical Engineering - Electronic Engineering - Information Engineering||Field:||Engineering and Technology||Issue Date:||May-2017||Publisher:||Association for Computing Machinery||Source:||Proceedings of the 2017 ACM International Conference on Management of Data, Pages 65-78||metadata.dc.doi:||http://dx.doi.org/10.1145/3035918.3064023||Abstract:||The ever-growing data storage and I/O demands of modern large-scale data analytics are challenging the current distributed storage systems. A promising trend is to exploit the recent improvements in memory, storage media, and networks for sustaining high performance and low cost. While past work explores using memory or SSDs as local storage or combine local with network-attached storage in cluster computing, this work focuses on managing multiple storage tiers in a distributed setting. We present OctopusFS, a novel distributed file system that is aware of heterogeneous storage media (e.g., memory, SSDs, HDDs, NAS) with different capacities and performance characteristics. The system offers a variety of pluggable policies for automating data management across the storage tiers and cluster nodes. The policies employ multi-objective optimization techniques for making intelligent data management decisions based on the requirements of fault tolerance, data and load balancing, and throughput maximization. At the same time, the storage media are explicitly exposed to users and applications, allowing them to choose the distribution and placement of replicas in the cluster based on their own performance and fault tolerance requirements. Our extensive evaluation shows the immediate benefits of using OctopusFS with data-intensive processing systems, such as Hadoop and Spark, in terms of both increased performance and better cluster utilization.||URI:||http://ktisis.cut.ac.cy/handle/10488/10295||ISBN:||9781450341974||Rights:||© 2017 Copyright held by the owner/author(s).||Type:||Book Chapter|
|Appears in Collections:||Κεφάλαια βιβλίων/Book chapters|
Show full item record
Page view(s) 5148
checked on Nov 15, 2018
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.