Fixed reparametrization for shear X/Y in autoaugment ops

[vfdev-5]

Description:

Context

In this PR we are fixing parametrization bug in affine transformation matrix that we have with shear ops when applied to autoaugmentations. In torchvision we are constructing affine transformation matrix for shear X as [1, tan(sx), 0, 0, 1, 0]

vision/torchvision/transforms/functional.py

Lines 976 to 982 in 21790df

	b = -math.cos(rot - sy) * math.tan(sx) / math.cos(sy) - math.sin(rot)
	c = math.sin(rot - sy) / math.cos(sy)
	d = -math.sin(rot - sy) * math.tan(sx) / math.cos(sy) + math.cos(rot)
	# Inverted rotation matrix with scale and shear
	# det([[a, b], [c, d]]) == 1, since det(rotation) = 1 and det(shear) = 1
	matrix = [d, -b, 0.0, -c, a, 0.0]

and official autoaugment implementations are using [1, sx, 0, 0, 1, 0], for example:

https://github.com/tensorflow/models/blob/dd02069717128186b88afa8d857ce57d17957f03/research/autoaugment/augmentation_transforms.py#L290

The difference between two is very small in range of [-0.3, 0.3]:

import torch

print([(v.tan() - v).item() for v in torch.linspace(-0.3, 0.3, 10)])
> [-0.009336262941360474, -0.004328876733779907, -0.0015605539083480835, -0.00033467262983322144, -1.2349337339401245e-05, 1.2349337339401245e-05, 0.00033467262983322144, 0.0015605539083480835, 0.004328876733779907, 0.009336262941360474]

In details, 1) autoaugment policy provides shear magnitude in radians, 2) as F.affine works on shear in degrees and applies math.tan(math.radians(shear_degree)) we have to modify f(magnitude) in a form such that we have the equality:

math.tan(math.radians( f(magnitude) )) == magnitude
# let's take f as math.degrees(math.atan( . ))
= math.tan(math.radians( math.degrees(math.atan(magnitude)) ))
= math.tan( math.atan(magnitude) )
= magnitude

I do not expect a major impact of that when training with autoaugment.
Right now with the fix we'll have exactly the same results on Pillow images.

What was done

1. Fixed shear X/Y reparametrization mismatch between torchvision and one of the official autoaugment implementations. (In TF there are two ways of implementing autoaugmentations: a) using PIL as for CIFAR10 and b) using TF/keras image ops for ImageNet. The difference is in padding. For imagenet reflect padding is used. See Added center as top-left for shear X/Y ops for autoaugment #5285)

2. Added a test to check torchvision's implementation vs ref implementations using PIL

We are testing 4 cases:

• PIL image input, nearest interpolation mode => exact match between PIL Image output by torchvision and expected PIL Image
• PIL image input, bilinear interpolation mode => exact match
• Tensor image, nearest interpolation mode => exact match
• Tensor image, bilinear interpolation mode => almost match
  • bilinear modes produce large non-zero pixel diffs for a certain amount of pixels, others are zeros.
  • we wont do anything for this case as it can be acceptable to have a small difference between between PIL and torch.tensor backends.

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit fc3f95f (more details on the Dr. CI page):

---

✅ None of the CI failures appear to be your fault 💚

---

• 1/1 broken upstream at merge base 6032775 since Feb 04

---

🚧 1 ongoing upstream failure:

These were probably caused by upstream breakages that are not fixed yet.

• binary_win_wheel_py3.9_cu115 since Feb 04 (22f8dc4)
  • 🔁 rerun

---

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[datumbox]

LGTM, with a minor nit.

I assume you will merge after fixing the issue surfaced on the tests on a separate PR. Right?