Παρακαλώ χρησιμοποιήστε αυτό το αναγνωριστικό για να παραπέμψετε ή να δημιουργήσετε σύνδεσμο προς αυτό το τεκμήριο:
https://hdl.handle.net/20.500.14279/29866
Τίτλος: | On combining system and machine learning performance tuning for distributed data stream applications | Συγγραφείς: | Odysseos, Lambros Herodotou, Herodotos |
Major Field of Science: | Engineering and Technology | Field Category: | Electrical Engineering - Electronic Engineering - Information Engineering | Λέξεις-κλειδιά: | Hyper-parameter tuning;Machine learning;Stream processing;System parameter tuning | Ημερομηνία Έκδοσης: | 17-Μαΐ-2023 | Πηγή: | Distributed and Parallel Databases, 2023 | Περίληψη: | The growing need to identify patterns in data and automate decisions based on them in near-real time, has stimulated the development of new machine learning (ML) applications processing continuous data streams. However, the deployment of ML applications over distributed stream processing engines (DSPEs) such as Apache Spark Streaming is a complex procedure that requires extensive tuning along two dimensions. First, DSPEs have a plethora of system configuration parameters, like degree of parallelism, memory buffer sizes, etc., that have a direct impact on application throughput and/or latency, and need to be optimized. Second, ML models have their own set of hyperparameters that require tuning as they can affect the overall prediction accuracy of the trained model significantly. These two forms of tuning have been studied extensively in the literature but only in isolation from each other. This manuscript presents a comprehensive experimental study that combines system configuration and hyperparameter tuning of ML applications over DSPEs. The experimental results reveal unexpected and complex interactions between the choices of system configurations and hyperparameters, and their impact on both application and model performance. These insights motivate the need for new combined system and ML model tuning approaches, and open up new research directions in the field of self-managing distributed stream processing systems. | URI: | https://hdl.handle.net/20.500.14279/29866 | ISSN: | 09268782 | DOI: | 10.1007/s10619-023-07434-0 | Rights: | © The Author(s) Attribution-NonCommercial-NoDerivatives 4.0 International |
Type: | Article | Affiliation: | Cyprus University of Technology | Publication Type: | Peer Reviewed |
Εμφανίζεται στις συλλογές: | Άρθρα/Articles |
Αρχεία σε αυτό το τεκμήριο:
Αρχείο | Περιγραφή | Μέγεθος | Μορφότυπος | |
---|---|---|---|---|
herodotou 1.pdf | Full text | 3.29 MB | Adobe PDF | Δείτε/ Ανοίξτε |
CORE Recommender
Page view(s)
202
Last Week
0
0
Last month
27
27
checked on 14 Μαρ 2025
Download(s)
168
checked on 14 Μαρ 2025
Google ScholarTM
Check
Altmetric
Αυτό το τεκμήριο προστατεύεται από άδεια Άδεια Creative Commons