2021 IEEE International Conference on Image Processing
19-22 September 2021 • Anchorage, Alaska, USA
Imaging Without Borders
| Paper ID | MLR-APPL-IP-5.4 | ||
| Paper Title | ADVERSARIAL TRAINING WITH STOCHASTIC WEIGHT AVERAGE | ||
| Authors | Joong-Won Hwang, Youngwan Lee, Sungchan Oh, Yuseok Bae, Electronics and Telecommunications Research Institute, Republic of Korea | ||
| Session | MLR-APPL-IP-5: Machine learning for image processing 5 | ||
| Location | Area E | ||
| Session Time: | Tuesday, 21 September, 13:30 - 15:00 | ||
| Presentation Time: | Tuesday, 21 September, 13:30 - 15:00 | ||
| Presentation | Poster | ||
| Topic | Applications of Machine Learning: Machine learning for image processing | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | Although adversarial training is the most reliable method to train robust deep neural networks so far, adversarially trained networks still show large gap between their accuracies on clean images and those on adversarial images. In conventional classification problem, one can gain higher accuracy by ensembling multiple networks. However, in adversarial training, there are obstacles to adopt such ensemble method. First, as inner maximization is expensive, training multiple networks adversarially becomes overburden. Moreover, the naive ensemble faces dilemma on choosing target model to generate adversarial examples with. Training adversarial examples of the members causes covariate shift, while training those of ensemble diminishes the benefit of ensembling. With these insights, we adopt stochastic weight average methods and improve it by considering overfitting nature of adversarial training. Our method take the benefit of ensemble while avoiding the described problems. Experiments on CIFAR10 and CIFAR100 shows our method improves the robustness effectively. | ||