Please use this identifier to cite or link to this item:
|Title:||Towards a distributed multi-tier file system for cluster computing||Authors:||Herodotou, Herodotos||Keywords:||Cache memory;Cluster computing;Data handling||Category:||Electrical Engineering - Electronic Engineering - Information Engineering||Field:||Engineering and Technology||Issue Date:||20-Jun-2016||Publisher:||Institute of Electrical and Electronics Engineers Inc.||Source:||32nd IEEE International Conference on Data Engineering Workshops, 2016, Helsinki, Finland||DOI:||10.1109/ICDEW.2016.7495633||Conference:||IEEE International Conference on Data Engineering Workshops||Abstract:||Distributed storage systems running on clusters of commodity hardware are challenged by the ever-growing data storage and I/O demands of modern large-scale data analytics. A promising trend is to exploit the recent improvements in memory, storage media, and network technologies for sustaining high performance at low cost. While recent work explores using memory and SSDs as a cache for local storage or combining local with network-attached storage, no work has ever looked at all layers together in a distributed setting. We present a novel design for a distributed file system that is aware of heterogeneous storage media (e.g., memory, SSDs, HDDs, NAS) with different capacities and performance characteristics. The storage media are explicitly exposed to users and applications, allowing them to choose the distribution and placement of replicas in the cluster based on their own performance and fault tolerance requirements. At the same time, the system offers a variety of pluggable policies for automating data management for increased performance and better cluster utilization. We analyze the new trends and challenges that led to our application- and data-centric design choices, and discuss how those choices inspire new research opportunities for data-intensive processing systems.||ISBN:||978-150902108-6||Rights:||© 2016 IEEE.||Type:||Conference Papers|
|Appears in Collections:||Δημοσιεύσεις σε συνέδρια/Conference papers|
Show full item record
Page view(s) 585
checked on Aug 21, 2019
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.