Παρακαλώ χρησιμοποιήστε αυτό το αναγνωριστικό για να παραπέμψετε ή να δημιουργήσετε σύνδεσμο προς αυτό το τεκμήριο:
https://hdl.handle.net/20.500.14279/13849
Τίτλος: | Profiling, what-if analysis, and costbased optimization of mapreduce programs | Συγγραφείς: | Babu, Shivnath Herodotou, Herodotos |
Major Field of Science: | Engineering and Technology | Field Category: | Electrical Engineering - Electronic Engineering - Information Engineering | Λέξεις-κλειδιά: | Hadoop;Distributed File System;Mapreduce | Ημερομηνία Έκδοσης: | Αυγ-2011 | Πηγή: | Proceedings of the VLDB Endowment, 2011,vol. 4, no 11, pp. 1111-1122 | Volume: | 4 | Issue: | 11 | Start page: | 1111 | End page: | 1122 | Περιοδικό: | Proceedings of the VLDB Endowment | Περίληψη: | MapReduce has emerged as a viable competitor to database systems in big data analytics. MapReduce programs are being written for a wide variety of application domains including business data processing, text analysis, natural language processing, Web graph and social network analysis, and computational science. However, MapReduce systems lack a feature that has been key to the historical success of database systems, namely, cost-based optimization. A major challenge here is that, to the MapReduce system, a program consists of black-box map and reduce functions written in some programming language like C++, Java, Python, or Ruby. We introduce, to our knowledge, the first Cost-based Optimizer for simple to arbitrarily complex MapReduce programs. We focus on the optimization opportunities presented by the large space of configuration parameters for these programs. We also introduce a Profiler to collect detailed statistical information from unmodified MapReduce programs, and a What-if Engine for fine-grained cost estimation. All components have been prototyped for the popular Hadoop MapReduce system. The effectiveness of each component is demonstrated through a comprehensive evaluation using representative MapReduce programs from various application domains. © 2011 VLDB Endowment. | ISSN: | 21508097 | Rights: | © ACM | Type: | Article | Affiliation: | Duke University | Publication Type: | Peer Reviewed |
Εμφανίζεται στις συλλογές: | Άρθρα/Articles |
CORE Recommender
SCOPUSTM
Citations
278
checked on 6 Νοε 2023
Page view(s)
331
Last Week
4
4
Last month
28
28
checked on 14 Μαρ 2025
Google ScholarTM
Check
Όλα τα τεκμήρια του δικτυακού τόπου προστατεύονται από πνευματικά δικαιώματα