Closed AccessA Distributed File System with Storage-Media Awareness
Date Issued
March 10, 2015
Author(s)
Abstract
Improvements in memory, storage devices, and network technologies are constantly exploited by distributed systems in order to meet the increasing data storage and I/O demands of modern large-scale data analytics. Some systems use memory and SSDs as a cache for local storage while others combine local with network-attached storage to increase performance. However, no work has ever looked at all layers together in a distributed setting. We present a novel design for a distributed file system that is aware of storage media (e.g., memory, SSDs, HDDs, NAS) with different capacities and performance characteristics. The storage media are explicitly exposed to users, allowing them to choose the distribution and placement of replicas in the cluster based on their own performance and fault tolerance requirements. Meanwhile, the system offers a variety of pluggable policies for automating data management with the dual goal of increased performance and better cluster utilization. These two features combined inspire new research opportunities for data-intensive processing systems.
Affiliation
File(s)![Thumbnail Image]()
Name
Herodotou Herodotos.pdf
Size
171.27 KB
Format
Adobe PDF
Checksum (MD5)
5710b57fa9d2c020ba7972f70802d4ce
Scopus Subject Areas
Hardware and Architecture
•
Computer Networks and Communications