Local Competition and Stochasticity for Adversarial Robustness in Deep Learning
Date Issued
January 4, 2021
Abstract
This work addresses adversarial robustness in deep learning by considering
deep networks with stochastic local winner-takes-all (LWTA) activations. This
type of network units result in sparse representations from each model layer,
as the units are organized in blocks where only one unit generates a non-zero
output. The main operating principle of the introduced units lies on stochastic
arguments, as the network performs posterior sampling over competing units to
select the winner. We combine these LWTA arguments with tools from the field of
Bayesian non-parametrics, specifically the stick-breaking construction of the
Indian Buffet Process, to allow for inferring the sub-part of each layer that
is essential for modeling the data at hand. Then, inference is performed by
means of stochastic variational Bayes. We perform a thorough experimental
evaluation of our model using benchmark datasets. As we show, our method
achieves high robustness to adversarial perturbations, with state-of-the-art
performance in powerful adversarial attack schemes.
deep networks with stochastic local winner-takes-all (LWTA) activations. This
type of network units result in sparse representations from each model layer,
as the units are organized in blocks where only one unit generates a non-zero
output. The main operating principle of the introduced units lies on stochastic
arguments, as the network performs posterior sampling over competing units to
select the winner. We combine these LWTA arguments with tools from the field of
Bayesian non-parametrics, specifically the stick-breaking construction of the
Indian Buffet Process, to allow for inferring the sub-part of each layer that
is essential for modeling the data at hand. Then, inference is performed by
means of stochastic variational Bayes. We perform a thorough experimental
evaluation of our model using benchmark datasets. As we show, our method
achieves high robustness to adversarial perturbations, with state-of-the-art
performance in powerful adversarial attack schemes.
Funding(s)
File(s)![Thumbnail Image]()
Name
Local_competition.pdf
Size
612.4 KB
Format
Adobe PDF
Checksum (MD5)
213b6f1810b5bb8adc65f865a081e049