Please use this identifier to cite or link to this item:
https://hdl.handle.net/20.500.14279/19324
Title: | A Survey on Automatic Parameter Tuning for Big Data Processing Systems | Authors: | Herodotou, Herodotos Chen, Yuxing Lu, Jiaheng |
Major Field of Science: | Natural Sciences | Field Category: | Computer and Information Sciences | Keywords: | MapReduce;Parameter tuning;Self-tuning;Spark;Storm;Stream | Issue Date: | Jun-2020 | Source: | ACM Computing Surveys, 2020, vol. 53, no 2, articl. no. 3381027 | Volume: | 53 | Issue: | 2 | Journal: | ACM Computing Surveys | Abstract: | Big data processing systems (e.g., Hadoop, Spark, Storm) contain a vast number of configuration parameters controlling parallelism, I/O behavior, memory settings, and compression. Improper parameter settings can cause significant performance degradation and stability issues. However, regular users and even expert administrators grapple with understanding and tuning them to achieve good performance. We investigate existing approaches on parameter tuning for both batch and stream data processing systems and classify them into six categories: rule-based, cost modeling, simulation-based, experiment-driven, machine learning, and adaptive tuning. We summarize the pros and cons of each approach and raise some open research problems for automatic parameter tuning. | URI: | https://hdl.handle.net/20.500.14279/19324 | ISSN: | 15577341 | DOI: | 10.1145/3381027 | Rights: | © owner/author(s). Attribution-NonCommercial-NoDerivatives 4.0 International |
Type: | Article | Affiliation : | Cyprus University of Technology University of Helsinki |
Publication Type: | Peer Reviewed |
Appears in Collections: | Άρθρα/Articles |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
3381027.pdf | Fulltext | 900.91 kB | Adobe PDF | View/Open |
CORE Recommender
SCOPUSTM
Citations
54
checked on Nov 6, 2023
WEB OF SCIENCETM
Citations
40
Last Week
0
0
Last month
4
4
checked on Oct 26, 2023
Page view(s)
305
Last Week
1
1
Last month
2
2
checked on Nov 21, 2024
Download(s)
193
checked on Nov 21, 2024
Google ScholarTM
Check
Altmetric
This item is licensed under a Creative Commons License