Please use this identifier to cite or link to this item: https://hdl.handle.net/20.500.14279/19324
Title: A Survey on Automatic Parameter Tuning for Big Data Processing Systems
Authors: Herodotou, Herodotos 
Chen, Yuxing 
Lu, Jiaheng 
Major Field of Science: Natural Sciences
Field Category: Computer and Information Sciences
Keywords: MapReduce;Parameter tuning;Self-tuning;Spark;Storm;Stream
Issue Date: Jun-2020
Source: ACM Computing Surveys, 2020, vol. 53, no 2, articl. no. 3381027
Volume: 53
Issue: 2
Journal: ACM Computing Surveys 
Abstract: Big data processing systems (e.g., Hadoop, Spark, Storm) contain a vast number of configuration parameters controlling parallelism, I/O behavior, memory settings, and compression. Improper parameter settings can cause significant performance degradation and stability issues. However, regular users and even expert administrators grapple with understanding and tuning them to achieve good performance. We investigate existing approaches on parameter tuning for both batch and stream data processing systems and classify them into six categories: rule-based, cost modeling, simulation-based, experiment-driven, machine learning, and adaptive tuning. We summarize the pros and cons of each approach and raise some open research problems for automatic parameter tuning.
URI: https://hdl.handle.net/20.500.14279/19324
ISSN: 15577341
DOI: 10.1145/3381027
Rights: © owner/author(s).
Attribution-NonCommercial-NoDerivatives 4.0 International
Type: Article
Affiliation : Cyprus University of Technology 
University of Helsinki 
Publication Type: Peer Reviewed
Appears in Collections:Άρθρα/Articles

Files in This Item:
File Description SizeFormat
3381027.pdfFulltext900.91 kBAdobe PDFView/Open
CORE Recommender
Show full item record

SCOPUSTM   
Citations

54
checked on Nov 6, 2023

WEB OF SCIENCETM
Citations

40
Last Week
0
Last month
4
checked on Oct 26, 2023

Page view(s)

305
Last Week
1
Last month
2
checked on Nov 21, 2024

Download(s)

193
checked on Nov 21, 2024

Google ScholarTM

Check

Altmetric


This item is licensed under a Creative Commons License Creative Commons