Please use this identifier to cite or link to this item:
Title: Investigating Automatic Parameter Tuning for SQL-on-Hadoop Systems
Authors: Lucas Filho, Edson Ramiro 
Cunha de Almeida, Eduardo 
Scherzinger, Stefanie 
Herodotou, Herodotos 
Major Field of Science: Natural Sciences
Field Category: Computer and Information Sciences
Keywords: SQL-on-Hadoop;Parameter tuning;Self-tuning
Issue Date: Jul-2021
Source: Big Data Research, 2021, vol. 25, articl. no. 100204
Volume: 25
Journal: Big Data Research 
Abstract: SQL-on-Hadoop engines such as Hive provide a declarative interface for processing large-scale data over computing frameworks such as Hadoop. The underlying frameworks contain a large number of configuration parameters that can significantly impact performance, but which are hard to tune. The problem of automatic parameter tuning has become a lively research area and several sophisticated tuning advisors have been proposed for Hadoop. In this paper, we conduct an experimental study to explore the impact of Hadoop parameter tuning on Hive. We reveal that the performance of Hive queries does not necessarily improve when using Hadoop-focused tuning advisors out-of-the-box, at least when following the current approach of applying the same tuning setup uniformly for evaluating the entire query. After extending the Hive query processing engine, we propose an alternative tuning approach and experimentally show how current Hadoop tuning advisors can now provide good and robust performance for Hive queries, as well as improved cluster resource utilization. We share our observations with the community and hope to create an awareness for this problem as well as to initiate new research on automatic parameter tuning for SQL-on-Hadoop systems.
ISSN: 2214-5796
DOI: 10.1016/j.bdr.2021.100204
Rights: © Elsevier
Attribution-NonCommercial-NoDerivatives 4.0 International
Type: Article
Affiliation : University of Passau 
Federal University of Paraná 
Cyprus University of Technology 
Appears in Collections:Άρθρα/Articles

CORE Recommender
Show full item record


checked on Sep 21, 2021

Page view(s)

Last Week
Last month
checked on Sep 22, 2021

Google ScholarTM



This item is licensed under a Creative Commons License Creative Commons