Skip to content

Fix inconsistencies in fgsm_tutorial #2418

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from

Conversation

QasimKhan5x
Copy link
Contributor

@QasimKhan5x QasimKhan5x commented Jun 3, 2023

Fixes #1032

Description

  1. The architecture of the model is changed. The tutorial says that it uses the same MNIST model from pytorch/examples/mnist. However, the dimensions of the convolutional layers were different. Therefore, the exact model is copy-pasted as is.
  2. The model in the MNIST example was trained using a transform in the data loader that normalized the data using mean=0.1307 and std=0.3081. However, no normalization was being applied in this example. Thus, the same normalization is applied to the test_loader in this example.
  3. The state_dict of the MNIST was being loaded on the CPU even though use_cuda=True and device is being dynamically defined. Therefore, map_location=device instead of cpu
  4. A denormalization function is added to unnormalize the data before applying FGSM on it, since FGSM takes unnormalized input
  5. The test function is modified. After computing the gradient of the normalized data, the data is unnormalized. FGSM is then applied on unnormalized data. Finally, the perturbed data is again normalized before applying the model on it.

I also checked the code without step 5, i.e., not performing normalization and denormalization inside the test function. The observation was that the accuracy stayed the same between 98-99%, although it was supposed to fall. Applying step 5 rectified this issue.

Checklist

  • The issue that is being fixed is referred in the description (see above "Fixes #ISSUE_NUMBER")
  • Only one issue is addressed in this pull request
  • Labels from the issue that this PR is fixing are added to this pull request
  • No unnecessary issues are included into this pull request.

Model architecture was not the same as that of the one in Basic MNIST Example, so it has been changed to be the exact same
The model is trained on normalized data, so it is unfair to use unnormalized data in this example.
The MNIST model is trained with normalized data but no normalization was applied in this tutorial. Thus, a denorm function is created, which is called to denorm the data before performing FGSM. The perturbed data is again normalized before feeding it to the model.
@netlify
Copy link

netlify bot commented Jun 3, 2023

Deploy Preview for pytorch-tutorials-preview ready!

Name Link
🔨 Latest commit a9ff79d
🔍 Latest deploy log https://app.netlify.com/sites/pytorch-tutorials-preview/deploys/647b909585384f00089dfb9b
😎 Deploy Preview https://deploy-preview-2418--pytorch-tutorials-preview.netlify.app/beginner/fgsm_tutorial
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docathon-h1-2023 A label for the docathon in H1 2023 medium
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Adversarial example generation by FGSM: different normalization of training vs test images?
2 participants