On Connections Between Regularizations for Improving DNN Robustness
This paper analyzes regularization terms proposed recently for improving the adversarial robustness of deep neural networks (DNNs), from a theoretical point of view. Specifically, we study possible connections between several effective methods, including input-gradient regularization, Jacobian regul...
| Publié dans: | IEEE transactions on pattern analysis and machine intelligence. - 1979. - 43(2021), 12 vom: 30. Dez., Seite 4469-4476 |
|---|---|
| Auteur principal: | |
| Autres auteurs: | , , |
| Format: | Article en ligne |
| Langue: | English |
| Publié: |
2021
|
| Accès à la collection: | IEEE transactions on pattern analysis and machine intelligence |
| Sujets: | Journal Article Research Support, Non-U.S. Gov't |
| Résumé: | This paper analyzes regularization terms proposed recently for improving the adversarial robustness of deep neural networks (DNNs), from a theoretical point of view. Specifically, we study possible connections between several effective methods, including input-gradient regularization, Jacobian regularization, curvature regularization, and a cross-Lipschitz functional. We investigate them on DNNs with general rectified linear activations, which constitute one of the most prevalent families of models for image classification and a host of other machine learning applications. We shed light on essential ingredients of these regularizations and re-interpret their functionality. Through the lens of our study, more principled and efficient regularizations can possibly be invented in the near future |
|---|---|
| Description: | Date Completed 10.12.2021 Date Revised 14.12.2021 published: Print-Electronic Citation Status MEDLINE |
| ISSN: | 1939-3539 |
| DOI: | 10.1109/TPAMI.2020.3006917 |