Cost-based optimization of configuration parameters and cluster sizing for hadoop
Title
Cost-based optimization of configuration parameters and cluster sizing for hadoop
Date Issued
2013-03-15
Inventor(s)
Major Field of Science
Engineering and Technology
Field Category
ENGINEERING AND TECHNOLOGY
Date Issued
March 15, 2013
Type
Patents
Application
13/843,347
Affiliation
Abstract
Cost-based optimization of configuration parameters and cluster sizing for distributed data processing systems are disclosed. According to an aspect, a method includes receiving at least one job profile of a MapReduce job. The method also includes using the at least one job profile to predict execution of the MapReduce job within a plurality of different predetermined settings of a distributed data processing system. Further, the method includes determining one of the predetermined settings that optimizes performance of the MapReduce job. The method may also include automatically adjusting the distributed data processing system to the determined predetermined setting.