Please use this identifier to cite or link to this item:
https://hdl.handle.net/20.500.14279/29868
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Panousis, Konstantinos P. | - |
dc.contributor.author | Antoniadis, Anastasios | - |
dc.contributor.author | Chatzis, Sotirios P. | - |
dc.date.accessioned | 2023-07-14T09:22:22Z | - |
dc.date.available | 2023-07-14T09:22:22Z | - |
dc.date.issued | 2022-06-30 | - |
dc.identifier.citation | Proceedings of the 36th AAAI Conference on Artificial Intelligence (Virtual, Online), 2022, pp. 7931 - 7940 | en_US |
dc.identifier.isbn | 1577358767 | - |
dc.identifier.isbn | 978-157735876-3 | - |
dc.identifier.uri | https://hdl.handle.net/20.500.14279/29868 | - |
dc.description.abstract | This work aims to address the long-established problem of learning diversified representations. To this end, we combine information-theoretic arguments with stochastic competition-based activations, namely Stochastic Local Winner-Takes-All (LWTA) units. In this context, we ditch the conventional deep architectures commonly used in Representation Learning, that rely on non-linear activations; instead, we replace them with sets of locally and stochastically competing linear units. In this setting, each network layer yields sparse outputs, determined by the outcome of the competition between units that are organized into blocks of competitors. We adopt stochastic arguments for the competition mechanism, which perform posterior sampling to determine the winner of each block. We further endow the considered networks with the ability to infer the sub-part of the network that is essential for modeling the data at hand; we impose appropriate stick-breaking priors to this end. To further enrich the information of the emerging representations, we resort to information-theoretic principles, namely the Information Competing Process (ICP). Then, all the components are tied together under the stochastic Variational Bayes framework for inference. We perform a thorough experimental investigation for our approach using benchmark datasets on image classification. As we experimentally show, the resulting networks yield significant discriminative representation learning abilities. In addition, the introduced paradigm allows for a principled investigation mechanism of the emerging intermediate network representations. | en_US |
dc.language.iso | en | en_US |
dc.rights | Copyright © Elsevier B.V | en_US |
dc.rights.uri | http://creativecommons.org/licenses/by-nc-nd/4.0/ | - |
dc.subject | Artificial intelligence | en_US |
dc.subject | Classification (of information) | en_US |
dc.subject | Information theory | en_US |
dc.subject | Network layers | en_US |
dc.subject | Stochastic systems | en_US |
dc.title | Competing Mutual Information Constraints with Stochastic Competition-Based Activations for Learning Diversified Representations | en_US |
dc.type | Article | en_US |
dc.collaboration | Cyprus University of Technology | en_US |
dc.subject.category | Electrical Engineering - Electronic Engineering - Information Engineering | en_US |
dc.country | Cyprus | en_US |
dc.subject.field | Engineering and Technology | en_US |
dc.relation.conference | 36th AAAI Conference on Artificial Intelligence (AAAI) | en_US |
dc.relation.volume | 36 | en_US |
cut.common.academicyear | 2022-2023 | en_US |
dc.identifier.spage | 7931 | en_US |
dc.identifier.epage | 7940 | en_US |
item.fulltext | No Fulltext | - |
item.cerifentitytype | Publications | - |
item.grantfulltext | none | - |
item.openairecristype | http://purl.org/coar/resource_type/c_6501 | - |
item.openairetype | article | - |
item.languageiso639-1 | en | - |
crisitem.author.dept | Department of Electrical Engineering, Computer Engineering and Informatics | - |
crisitem.author.faculty | Faculty of Engineering and Technology | - |
crisitem.author.orcid | 0000-0002-4956-4013 | - |
crisitem.author.parentorg | Faculty of Engineering and Technology | - |
Appears in Collections: | Δημοσιεύσεις σε συνέδρια /Conference papers or poster or presentation |
CORE Recommender
This item is licensed under a Creative Commons License