Please use this identifier to cite or link to this item: https://hdl.handle.net/20.500.14279/32659
DC FieldValueLanguage
dc.contributor.authorHerodotou, Herodotos-
dc.contributor.authorKakoulli, Elena-
dc.date.accessioned2024-06-20T08:59:39Z-
dc.date.available2024-06-20T08:59:39Z-
dc.date.issued2023-11-13-
dc.identifier.citationACM Transactions on Database Systems, 2023, vol. 48, iss. 4, pp. 1-40en_US
dc.identifier.issn03625915-
dc.identifier.urihttps://hdl.handle.net/20.500.14279/32659-
dc.description.abstractThe use of storage tiering is becoming popular in data-intensive compute clusters due to the recent advancements in storage technologies. The Hadoop Distributed File System, for example, now supports storing data in memory, SSDs, and HDDs, while OctopusFS and hatS offer fine-grained storage tiering solutions. However, current big data platforms (such as Hadoop and Spark) are not exploiting the presence of storage tiers and the opportunities they present for performance optimizations. Specifically, schedulers and prefetchers will make decisions only based on data locality information and completely ignore the fact that local data are now stored on a variety of storage media with different performance characteristics. This article presents Trident, a scheduling and prefetching framework that is designed to make task assignment, resource scheduling, and prefetching decisions based on both locality and storage tier information. Trident formulates task scheduling as a minimum cost maximum matching problem in a bipartite graph and utilizes two novel pruning algorithms for bounding the size of the graph, while still guaranteeing optimality. In addition, Trident extends YARN's resource request model and proposes a new storage-tier-aware resource scheduling algorithm. Finally, Trident includes a cost-based data prefetching approach that coordinates with the schedulers for optimizing prefetching operations. Trident is implemented in both Spark and Hadoop and evaluated extensively using a realistic workload derived from Facebook traces as well as an industry-validated benchmark, demonstrating significant benefits in terms of application performance and cluster efficiency.en_US
dc.formatpdfen_US
dc.language.isoenen_US
dc.relation.ispartofACM Transactions on Database Systemsen_US
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc-nd/4.0/*
dc.titleCost-based Data Prefetching and Scheduling in Big Data Platforms over Tiered Storage Systemsen_US
dc.typeArticleen_US
dc.collaborationCyprus University of Technologyen_US
dc.subject.categoryElectrical Engineering - Electronic Engineering - Information Engineeringen_US
dc.journalsOpen Accessen_US
dc.countryCyprusen_US
dc.subject.fieldEngineering and Technologyen_US
dc.publicationPeer Revieweden_US
dc.identifier.doi10.1145/3625389en_US
dc.identifier.scopus2-s2.0-85183076629-
dc.identifier.urlhttps://api.elsevier.com/content/abstract/scopus_id/85183076629-
dc.relation.issue4en_US
dc.relation.volume48en_US
cut.common.academicyear2023-2024en_US
dc.identifier.spage1en_US
dc.identifier.epage40en_US
item.grantfulltextopen-
item.openairecristypehttp://purl.org/coar/resource_type/c_6501-
item.fulltextWith Fulltext-
item.languageiso639-1en-
item.cerifentitytypePublications-
item.openairetypearticle-
crisitem.author.deptDepartment of Electrical Engineering, Computer Engineering and Informatics-
crisitem.author.deptDepartment of Electrical Engineering, Computer Engineering and Informatics-
crisitem.author.facultyFaculty of Engineering and Technology-
crisitem.author.facultyFaculty of Engineering and Technology-
crisitem.author.orcid0000-0002-8717-1691-
crisitem.author.orcid0000-0003-1489-807X-
crisitem.author.parentorgFaculty of Engineering and Technology-
crisitem.author.parentorgFaculty of Engineering and Technology-
Appears in Collections:Άρθρα/Articles
Files in This Item:
File Description SizeFormat
Herodotou et al.-2023.pdfopen access4.02 MBAdobe PDFView/Open
CORE Recommender
Show simple item record

Page view(s)

111
Last Week
1
Last month
6
checked on Dec 22, 2024

Download(s)

104
checked on Dec 22, 2024

Google ScholarTM

Check

Altmetric


This item is licensed under a Creative Commons License Creative Commons