Please use this identifier to cite or link to this item: https://ktisis.cut.ac.cy/handle/10488/13849
DC FieldValueLanguage
dc.contributor.authorBabu, Shivnath-
dc.contributor.authorHerodotou, Herodotos-
dc.date.accessioned2019-05-31T07:37:11Z-
dc.date.available2019-05-31T07:37:11Z-
dc.date.issued2011-08-
dc.identifier.citationProceedings of the VLDB Endowment, 2011,vol. 4, no 11, pp. 1111-1122en_US
dc.identifier.issn2150-8097-
dc.description.abstractMapReduce has emerged as a viable competitor to database systems in big data analytics. MapReduce programs are being written for a wide variety of application domains including business data processing, text analysis, natural language processing, Web graph and social network analysis, and computational science. However, MapReduce systems lack a feature that has been key to the historical success of database systems, namely, cost-based optimization. A major challenge here is that, to the MapReduce system, a program consists of black-box map and reduce functions written in some programming language like C++, Java, Python, or Ruby. We introduce, to our knowledge, the first Cost-based Optimizer for simple to arbitrarily complex MapReduce programs. We focus on the optimization opportunities presented by the large space of configuration parameters for these programs. We also introduce a Profiler to collect detailed statistical information from unmodified MapReduce programs, and a What-if Engine for fine-grained cost estimation. All components have been prototyped for the popular Hadoop MapReduce system. The effectiveness of each component is demonstrated through a comprehensive evaluation using representative MapReduce programs from various application domains. © 2011 VLDB Endowment.en_US
dc.formatpdfen_US
dc.language.isoenen_US
dc.relation.ispartofProceedings of the VLDB Endowmenten_US
dc.rights© ACMen_US
dc.subjectHadoopen_US
dc.subjectDistributed File Systemen_US
dc.subjectMapreduceen_US
dc.titleProfiling, what-if analysis, and costbased optimization of mapreduce programsen_US
dc.typeArticleen_US
dc.collaborationDuke Universityen_US
dc.subject.categoryElectrical Engineering - Electronic Engineering - Information Engineeringen_US
dc.journalsSubscriptionen_US
dc.countryUnited Statesen_US
dc.subject.fieldEngineering and Technologyen_US
dc.publicationPeer Revieweden_US
dc.identifier.scopus2-s2.0-82155174846en
dc.identifier.urlhttps://api.elsevier.com/content/abstract/scopus_id/82155174846en
dc.contributor.orcid#NODATA#en
dc.contributor.orcid#NODATA#en
dc.relation.issue11en_US
dc.relation.volume4en_US
cut.common.academicyear2011-2012en_US
dc.identifier.spage1111en_US
dc.identifier.epage1122en_US
item.cerifentitytypePublications-
item.fulltextNo Fulltext-
item.grantfulltextnone-
item.languageiso639-1en-
item.openairecristypehttp://purl.org/coar/resource_type/c_6501-
item.openairetypearticle-
crisitem.author.deptDepartment of Electrical Engineering, Computer Engineering and Informatics-
crisitem.author.facultyFaculty of Engineering and Technology-
crisitem.author.orcid0000-0002-8717-1691-
crisitem.author.parentorgFaculty of Engineering and Technology-
Appears in Collections:Άρθρα/Articles
CORE Recommender
Show simple item record

SCOPUSTM   
Citations

250
checked on Jul 25, 2021

Page view(s)

54
Last Week
0
Last month
6
checked on Jul 27, 2021

Google ScholarTM

Check


Items in KTISIS are protected by copyright, with all rights reserved, unless otherwise indicated.