Please use this identifier to cite or link to this item: https://hdl.handle.net/20.500.14279/13846
Title: No one (cluster) size fits all: Automatic cluster sizing for data-intensive analytics
Authors: Herodotou, Herodotos 
Babu, Shivnath
Dong, Fei 
Major Field of Science: Engineering and Technology
Field Category: Electrical Engineering - Electronic Engineering - Information Engineering
Keywords: Cloud computing;Cluster provisioning;MapReduce
Issue Date: 30-Nov-2011
Source: 2nd ACM Symposium on Cloud Computing, SOCC 2011; Cascais; Portugal; 26 October 2011 through 28 October 2011
Conference: ACM Symposium on Cloud Computing 
Abstract: Infrastructure-as-a-Service (IaaS) cloud platforms have brought two unprecedented changes to cluster provisioning practices. First, any (nonexpert) user can provision a cluster of any size on the cloud within minutes to run her data-processing jobs. The user can terminate the cluster once her jobs complete, and she needs to pay only for the resources used and duration of use. Second, cloud platforms enable users to bypass the traditional middleman-the system administrator-in the cluster-provisioning process. These changes give tremendous power to the user, but place a major burden on her shoulders. The user is now faced regularly with complex cluster sizing problems that involve finding the cluster size, the type of resources to use in the cluster from the large number of choices offered by current IaaS cloud platforms, and the job configurations that best meet the performance needs of her workload. In this paper, we introduce the Elastisizer, a system to which users can express cluster sizing problems as queries in a declarative fashion. The Elastisizer provides reliable answers to these queries using an automated technique that uses a mix of job profiling, estimation using black-box and white-box models, and simulation. We have prototyped the Elastisizer for the Hadoop MapReduce framework, and present a comprehensive evaluation that shows the benefits of the Elastisizer in common scenarios where cluster sizing problems arise. Copyright 2011 ACM.
ISBN: 9781450309769
DOI: 10.1145/2038916.2038934
Rights: © ACM
Type: Conference Papers
Affiliation : Duke University 
Publication Type: Peer Reviewed
Appears in Collections:Δημοσιεύσεις σε συνέδρια /Conference papers or poster or presentation

CORE Recommender
Show full item record

SCOPUSTM   
Citations 10

184
checked on Mar 14, 2024

Page view(s) 10

261
Last Week
0
Last month
3
checked on Dec 25, 2024

Google ScholarTM

Check

Altmetric


Items in KTISIS are protected by copyright, with all rights reserved, unless otherwise indicated.