“Adversarial Robustness via Label Smoothing” is a research paper by Jiang et al. that explores the use of label smoothing as a method to improve the robustness of neural networks against adversarial attacks. Adversarial attacks are a type of attack on a machine learning model in which an attacker adds small perturbations to the input data to mislead the model’s prediction.
In traditional machine learning, it is common practice to use a one-hot encoding scheme for the labels of the training data, in which each label is represented by a vector with a 1 in the position corresponding to the label and 0’s elsewhere. However, this approach has been shown to be vulnerable to adversarial attacks because it provides a large margin for the attacker to manipulate the input data.
In this paper, the authors propose to use label smoothing to improve the robustness of neural networks against adversarial attacks. Label smoothing is a regularization technique that involves replacing the one-hot encoding scheme with a smoothed probability distribution over the labels. This distribution is generated by adding a small amount of uniform noise to the one-hot encoding, resulting in a probability distribution that assigns a small probability to the incorrect labels.
The authors demonstrate the effectiveness of label smoothing in improving the robustness of neural networks against different types of adversarial attacks, such as the Fast Gradient Sign Method (FGSM) and the Projected Gradient Descent (PGD) attack. They show that by using label smoothing, the adversarial accuracy of the neural network is improved, while maintaining high accuracy on the clean test data.
The authors also explore the effect of different smoothing factors on the performance of the model and show that the performance of the model is relatively robust to the choice of smoothing factor. They also demonstrate that label smoothing can be applied to different types of neural networks, including convolutional neural networks and residual networks.
In summary, “Adversarial Robustness via Label Smoothing” proposes a regularization technique that involves replacing the one-hot encoding scheme with a smoothed probability distribution over the labels to improve the robustness of neural networks against adversarial attacks. The authors demonstrate the effectiveness of label smoothing in improving the adversarial accuracy of the neural network while maintaining high accuracy on clean test data. The paper provides insights into the effect of different smoothing factors and demonstrates the applicability of label smoothing to different types of neural networks.