Stochastic Sensitivity Regularized Autoencoder for Robust Feature Learning

We present a new regularized autoencoder for robust feature learning. The regularization, implying stochastic sensitivity, is defined as the sum of entries of the absolute covariance matrix of the output perturbation at each layer of the autoencoder. The advantages of the stochastic sensitivity regularization are two-fold. Firstly, we show that the classical Frobenius norm regularization effectively enforces the network to be insensitive to input perturbation and that the Frobenius norm regularization is a special case of the proposed stochastic sensitivity regularization which enables the proposed method to train an autoencoder for robust feature learning. Secondly, we also show that the stochastic sensitivity regularization attempts to drive the network to learn a set of decorrelated feature maps which removes redundant information and thus improves generalization capabilities. These two properties enable the autoencoder to learn a set of robust and diverse feature maps. Finally, the efficacy and the robustness of the proposed regularization method are confirmed a nd quantified by comparing it against existing regularized auto encoders over a range of tasks.