Please use this identifier to cite or link to this item:
https://hdl.handle.net/20.500.14279/29866
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Odysseos, Lambros | - |
dc.contributor.author | Herodotou, Herodotos | - |
dc.date.accessioned | 2023-07-14T09:04:36Z | - |
dc.date.available | 2023-07-14T09:04:36Z | - |
dc.date.issued | 2023-05-17 | - |
dc.identifier.citation | Distributed and Parallel Databases, 2023 | en_US |
dc.identifier.issn | 09268782 | - |
dc.identifier.uri | https://hdl.handle.net/20.500.14279/29866 | - |
dc.description.abstract | The growing need to identify patterns in data and automate decisions based on them in near-real time, has stimulated the development of new machine learning (ML) applications processing continuous data streams. However, the deployment of ML applications over distributed stream processing engines (DSPEs) such as Apache Spark Streaming is a complex procedure that requires extensive tuning along two dimensions. First, DSPEs have a plethora of system configuration parameters, like degree of parallelism, memory buffer sizes, etc., that have a direct impact on application throughput and/or latency, and need to be optimized. Second, ML models have their own set of hyperparameters that require tuning as they can affect the overall prediction accuracy of the trained model significantly. These two forms of tuning have been studied extensively in the literature but only in isolation from each other. This manuscript presents a comprehensive experimental study that combines system configuration and hyperparameter tuning of ML applications over DSPEs. The experimental results reveal unexpected and complex interactions between the choices of system configurations and hyperparameters, and their impact on both application and model performance. These insights motivate the need for new combined system and ML model tuning approaches, and open up new research directions in the field of self-managing distributed stream processing systems. | en_US |
dc.format | en_US | |
dc.language.iso | en | en_US |
dc.rights | © The Author(s) | en_US |
dc.rights | Attribution-NonCommercial-NoDerivatives 4.0 International | * |
dc.rights.uri | http://creativecommons.org/licenses/by-nc-nd/4.0/ | * |
dc.subject | Hyper-parameter tuning | en_US |
dc.subject | Machine learning | en_US |
dc.subject | Stream processing | en_US |
dc.subject | System parameter tuning | en_US |
dc.title | On combining system and machine learning performance tuning for distributed data stream applications | en_US |
dc.type | Article | en_US |
dc.collaboration | Cyprus University of Technology | en_US |
dc.subject.category | Electrical Engineering - Electronic Engineering - Information Engineering | en_US |
dc.journals | Open Access | en_US |
dc.country | Cyprus | en_US |
dc.subject.field | Engineering and Technology | en_US |
dc.publication | Peer Reviewed | en_US |
dc.identifier.doi | 10.1007/s10619-023-07434-0 | en_US |
dc.identifier.scopus | 2-s2.0-85159710158 | - |
dc.identifier.url | https://api.elsevier.com/content/abstract/scopus_id/85159710158 | - |
cut.common.academicyear | 2022-2023 | en_US |
item.fulltext | With Fulltext | - |
item.languageiso639-1 | en | - |
item.grantfulltext | open | - |
item.openairecristype | http://purl.org/coar/resource_type/c_6501 | - |
item.cerifentitytype | Publications | - |
item.openairetype | article | - |
crisitem.author.dept | Department of Electrical Engineering, Computer Engineering and Informatics | - |
crisitem.author.faculty | Faculty of Engineering and Technology | - |
crisitem.author.orcid | 0000-0002-8717-1691 | - |
crisitem.author.parentorg | Faculty of Engineering and Technology | - |
Appears in Collections: | Άρθρα/Articles |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
herodotou 1.pdf | Full text | 3.29 MB | Adobe PDF | View/Open |
CORE Recommender
Page view(s)
202
Last Week
0
0
Last month
27
27
checked on Mar 14, 2025
Download(s)
168
checked on Mar 14, 2025
Google ScholarTM
Check
Altmetric
This item is licensed under a Creative Commons License