tutorials/CIFAR_Captum_Robustness.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Captum Robustness with Image Classification\n",
    "\n",
    "\n",
    "This tutorial demonstrates how to apply robustness tooling in captum.robust including adversarial attacks (FGSM and PGD) as well as robustness metrics to measure attack effectiveness and generate counterfactual examples.\n",
    "\n",
    "In this tutorial, we use a simple image classification model trained on the CIFAR-10 dataset. Be sure to install the torchvision and matplotlib packages before you start."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms\n",
    "import torchvision.transforms.functional as TF\n",
    "\n",
    "from captum.robust import FGSM\n",
    "from captum.robust import PGD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The cell below loads the test data, transforms the data to a tensor, defines necessary normalization and defines the label classes. For the purpose of this tutorial, we will only need the test dataset, because we will use a pretrained model in our `models` folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "transform_tensor = transforms.ToTensor()\n",
    "normalize = transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\n",
    "unnormalize = lambda x: 0.5*x + 0.5\n",
    "\n",
    "testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n",
    "                                       download=True, transform=transform_tensor)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=5,\n",
    "                                         shuffle=False, num_workers=2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The classification model under attack is defined below. It is the same as the CIFAR model from https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
    "        self.pool1 = nn.MaxPool2d(2, 2)\n",
    "        self.pool2 = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "        self.relu1 = nn.ReLU()\n",
    "        self.relu2 = nn.ReLU()\n",
    "        self.relu3 = nn.ReLU()\n",
    "        self.relu4 = nn.ReLU()\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool1(self.relu1(self.conv1(x)))\n",
    "        x = self.pool2(self.relu2(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = self.relu3(self.fc1(x))\n",
    "        x = self.relu4(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "\n",
    "net = Net()\n",
    "softmax = nn.Softmax(dim=1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define a few helper methods to obtain predictions for a given image and visualize image and results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def image_show(img, pred):\n",
    "    npimg = img.squeeze().permute(1, 2, 0).detach().numpy()\n",
    "    plt.imshow(npimg)\n",
    "    plt.title(\"prediction: %s\" % pred)\n",
    "    plt.show()\n",
    "    \n",
    "def get_prediction(model, input, normalize_im=False):\n",
    "    if normalize_im:\n",
    "        input = normalize(input)\n",
    "    output = model(input)\n",
    "    _, pred = torch.max(output, dim=1)\n",
    "    return classes[pred], softmax(output)[:,pred]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To save training time, we will load the pretrained weights from `models/cifar_torchvision.pt` instead of training the model from scratch. You may also train your own CIFAR model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<All keys matched successfully>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net.load_state_dict(torch.load('models/cifar_torchvision.pt'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are ready for the central part of this tutorial - generating adversarial examples with FGSM and PGD from Captum and then applying robustness metrics to better understand model vulnerabilities and the decision boundary. \n",
    "\n",
    "We will pick one image from the test set to use as an example and generate a perturbed image (adversarial example) that is very similar to the original image but causes the model to make a false prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "image, label = testset[11]\n",
    "image = image.unsqueeze(0)\n",
    "\n",
    "image.requires_grad = True\n",
    "# Get original prediction\n",
    "pred, score  = get_prediction(net, image, normalize_im=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first utilize the Fast Gradient Sign Method (FGSM) to construct an adversarial example. FGSM utilizes the sign of the gradient to perturb the input.\n",
    "\n",
    "We now construct the FGSM object, providing the desired lower and upper bound of perturbed inputs. In this case, since we are working with normalized inputs, the values should be in the range -1 to 1. We then call perturb on the FGSM object, providing epsilon (magnitude of the perturbation) as well as the label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Construct FGSM attacker\n",
    "fgsm = FGSM(net, lower_bound=-1, upper_bound=1)\n",
    "perturbed_image_fgsm = fgsm.perturb(normalize(image), epsilon=0.16, target=label) \n",
    "new_pred_fgsm, score_fgsm = get_prediction(net, perturbed_image_fgsm)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now visualize the original and perturbed images, and see that the perturbed image is predicted incorrectly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQQAAAEICAYAAAC5yopxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAolklEQVR4nO2deZhc1XXgf6eqel/Uau0ItCKz2ggilhiGYLwBXzLYE9uficfBMyTEMyZ2tvni2ONAPE7iyRc740zyOcGxA04cL/ESk/kcLyEQYYMxAgtJoAUE2oWkltTqvbuWM3+818+l5p7bJXWrCiXn9331VdU977533n23zrvvnjrniqriOI4DkGu0Ao7jvHJwg+A4ToYbBMdxMtwgOI6T4QbBcZwMNwiO42ScVQZBRHaJyBvSzx8Skb86zf08IyI3zKZujaC6PRxnNjirDEI1qvoHqvpL020nIveJyMem1L1EVR8+Y8r95NgPi8i0OtYDEVkhIg+JyIiIbIsZEhHpEZH7ReRw+rpnivy1IvIjERkUkU0icl2VTETkwyKyR0QGRORLItJdJX9GRIaqXiUR+ccq+VoReTLV80kRWVsle4+IlKfUvyGVtYjIZ0Vkd6rXj0Xk5innr1PqfqRKfo+IFKfIV9XSfiLyOhHZLCL9InJURL4hIkur5O8QkUfTug8H2ltFZLjquH9VJRMR+ZiI7BeRE2mfuqRK/rCIjFXV3V4le9eU8xlJj/VT1rVHVRvyAgqnUWcX8IZTrHMf8LEGnePDwC/NZhucbnsAjwGfBNqAnwf6gQXGtn8N/D3QDqwAdgL/JZX1An3A24E88J+B48DcVH47sA04D+gEvgncbxxHgBeAX0y/NwO7gV8HWoD3p9+bU/l7gO8b++oA7kn1zQE/CwwCK1L5CkCtNk/r/u3ptB+wCDgn/dwC/BHwQFXdNwDvAH4XeDiwbwXON477DuAAsCpt7z8Enqq1j03Z13vSaynmNrP8A9gF/A7wbNpJ/hpoTWU3APuA3wZeAv4mvXAfTJU8CnwF6K3a37vTDnEU+DBVP4CpFxC4Dng0vVB705O/EygCE8AQ8I9Tf0jpBfw/aaMfSD+3TNH5N4HDwEHSH0YNbfH7QBkYS4/9Z1UX/33Ac8CLoY469SIDvwxsJengzwJXBM7jwnR/7wzo8ipgHOiqKnsEeK+hex9wZdX3DwGPpJ9/FnhmyvY7gDvSz18F/keV7LVpG7QHjvMzadt0pN/fBOynqsMCe4CbpjMIxnlsAn5+pgbhVNov7U9/CDwbkP0Sp24Qfhv4StX3S4Axq69M0x4PAXfHtjkTjwzvAt4MrE4b8n9WyRaT3GGWk/xY3w+8haRjnENiRP4cQEQuBj5NYhTOAeYB54YOKCLLgH8C/i+wAFgLbFTVe4EvAH+kqp2q+nOB6h8GrknrXAZcFdB5DrAUuAP4cxGZmx73F0RkU0gnVf0wSae5Kz32XVXitwBXAxeH6k45t7eTdNZfBLqB/0hiIKu3uQL4LvCrqvqlwG4uAV5Q1cGqsqfTcvPQUz5fWvVZAttaciH5kawJHON24KuqOlyl5yZNe2/Kpil6Xi4ifSKyQ0Q+IiKFoPIii0j63zNTRLtFZJ+I/LWIzJ8i+zkROZY+1vy3qvJp209ElolIPzAK/BbJKOFUWC8iL4nI10VkRVX5l4DzReRVItJE0mbfnlL3D9M2+YEYc2Mishy4Hvh8VItarW2NFmgXVVYTuAXYqT+5206QjhjSsq3A66u+LyG5oxdIhldfqpJ1pPVfNkIgGZV8w9DpPqY8MnDynXUncEuV7M3AriqdRzn57n0YuKbG9niYKdab5G5wY9X3FURGCMB3gA9E2vv3SEYxr4vo8W7gh1PKfh+4z9j+b4GvA13A+WkbjaeyeSSjsNuAyQ5aAf6y6i64Iz2vOcAD6fn99JRjtAMDwA1VZR+pvuZp2ReAe9LPq4CVJCPLV5OMln4noH8T8M+TOqVlncC6tG8tIhnJfKdKfjHJjSdPMqo5CNx2qu1HcsP77VAfwR4hXE/yuNQD/BmwZbI/pOWfStuwRDIKXFlV9+r0OrWk12IQWB04xkdCx576OhMjhL1Vn3eTNPIkR1R1rOr7cuAb6WRMP4mBKJM+k1XvS5O7yEl3xirOI+m0p8M5qZ6WzkdVtVT1fYSkc82EvdNvkjHdub0XeFRVH4psM0Qyuqimm6TzhHg/iSF8jmQO4IskRgdVPQrcCvwGcAi4ieTHty+t+7l0+4dJ7s6Tek3KJ/lPwDHgX2vVU1VfUNUXVbWiqpuBjwJvq95YRHIkj6MTQDYqU9UhVd2gqiVVPZTK3jQ54amqz6rqAVUtq+qjJD/CyX3X3H6qegy4H/imNXoJ1FmvqhOq2g98gMToXZSK7wauJOkHrSQ3gH8Rkfa07uOqOqiq46p6P/ADkhvxVH4x1SvKmTAI51V9XkbyXD6JTtl2L3CzqvZUvVpVdT+Jhc72lTbAPOOYe0keUUJMPeZUDpAYJkvnmWAdu7p8crjcXlW2uOpz7NwgMQjLRORPIts8A6wSka6qsst4+XA6UU71mKq+S1UXq+olJP3kR1Xyf1XVK1W1l+TuecGkPP2x3q2qK1T13PQY+9NXNbcDn9f09lWl52tEpPqR4zWWniTtmG2b1vssyQ3l51W1aNSbrAsvf/wJ7fuU2o9kFLKQlxuRWqk+9mXAl1V1X2rM7gPmYj9untQmACJyLclN7qvTH3mGjwlThiW7gM0kz/q9JM/Qf6A/GX7vm7L9r5PcSZan3xcAt6afLyGxzNeRDJv+mGTIFHpkWEZird9BcjHmAWtT2ceBvwvoObmfj5FMRi4A5gPfJ33EMHTO6tbQHl+aPP+qspdNIJHcPf87yXD1v5I8Nk0+MrydxCj8VHqhz69qr10kM9g9wJPAxyO6/DBtw1bgrcS9DKvTNswDN5NMMl5SJb+cZFjeTTIJ+4MqWW9aX0g67Rbgzin7Pze9lqunlE96GT5AMgS+i5O9DDcDi9LPF6b7vruq/l+k59kZOKerSQxXLj23LwMPVclvJfmhCck80n7g9lraj2S0M7nvBSST49WegHxa773A+vRzU1U/X5tu05m25/Yq+d0kfXJRuv93k9xEetLXm9P9FUjm74aBC6ac+70kxnf6PnsGDMKkl6GfZIjSHvlx5UiGnttJftA7qfoBkdxF9lCbl+E/AI+TPJfunbyYJJNZG1N9/iFgEFqBPyUZkRxMP7dGdK6u+y6mzLhP2fanSZ6njwN/GjEIN5M8G/YDnyAZRld7Gd6bttEQyY/g8oAuvSQTXf/L0GUFifEdTff1hiltN1T1fdLVNZK23Zun7OuLwIn09WVgYZXsVen+R0h+zL8R0OV3SL0WAdnlJMZtFHhq8lxT2R+TPKYMk7grP8pPfjjL07ad9OpMvt6Vym9L23g4vc6fBxZPOaejaZ1twPtPof1+tWrfL5HcCJZXyd+T6lb9ui+V3Zjub5hkfuofgDVVdVtJJtoPkvTtp/iJ12UB8ATJb6efxGi9cYrerans9aH2nvqStNKsICK7SDryP8/aTh3HqRtn7T8VHceZfdwgOI6TMauPDI7jnN34CMFxnIya/jgxW8yfP19XrFg+/YZTqOcYRiIH01nW5GR3+9RjRerNqhbOqTA6MmrKSuWSKevstP/LFusHp8quXbvp6+s77R3OyCCIyE0k/+jKA3+lqh+Pbb9ixXI2bHg8KCtXymY967GmcpqnnYv82qIGwdBDT1ePnD1Aiz3KxTqQWObiTFiRWTaes/nDmI6odhVbuvXpzabsaP8xU3bNddeasubm5mD56fSBK9ddY9aphdN+ZBCRPIl/9GaSP6DclgYkOY5zljKTOYSrgOc1+X/5BMmfMW6dHbUcx2kEMzEISzk5SGdfWnYSInKniGwQkQ1HjvTN4HCO45xpZmIQQg8xL3voUdV7VXWdqq5bsGBq+LnjOK8kZmIQ9nFyZOO5zF6UoOM4DWAmXoYngDUispIkMuydwC/EKihQMrwJ5ZLtstmxdXuwfP7CBWad+YsWmrJiccKUbfzhj0zZ2GjY5XTNz1xv1ik02U1cjJxzQWxbXZGKLTPKczHPRGTKvRKdj4/Mgp+GW0NjdWIeiNhs/CkLQEp2++7eusOU7XzhBVN2xTVXm7J8wegjp+lpmgmnbRBUtSQid5Fk9MkDn1NVKz7ccZyzgBn9D0FVvwV8a5Z0cRynwfhflx3HyXCD4DhOhhsEx3Ey3CA4jpNR12hHiDuqLPoOvhQsb8nb6i9cbLsd9+550ZRtf+pJU5Y3jnfi0kuD5QCdc+fY+2tvNWUDh+1Ameb2cDAMQMHYZyXSvvnTDK2Mub4mxoyowEjgUFN7hymLuT8lIquYLkTbtVhoypuy3l77eh7ea/fHoRPHTVl72znB8miukjMUAuwjBMdxMtwgOI6T4QbBcZwMNwiO42S4QXAcJ6POXgalouHZ3VisRj4XnvU93nfErLN3mz0Nu/6BfzRlJ/bbAZvzl4Zng5967AdmnfY5XabsyuuvM2WPPvQvpuz8Cy4wZResfXWwvKh2ijpyduOPDQ+bMiv1F8DO554Plo+PjAXLAa58rZ1mbLxoL9PYHPE2HTg0dY3ZhL6j1rrBsOoCeynNQ0cOmrIju+01eZ/83ndM2Y1ve2ewXIx+D5DPh+/lM82i7iMEx3Ey3CA4jpPhBsFxnAw3CI7jZLhBcBwnww2C4zgZdQ9ushxcGgl6KY2NB8sfefAhs05Xk61Da3HIPtZwvyl7YcdAsPz4XtsV1dxtB+xc8GrbfajDER3HRkxZcTzsnhuPuKMiXke2Pr3RlHV3dpuy8aGwjiODthuzPB6+zgAV47wAxgu2e25oYDCsx7FwOcDBnXtM2TM/+qEpmy+2a3f/Uxvsfa5aGSxftuYis053d0+wfKZrN/sIwXGcDDcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhORl3djsMjYzzxdHhxpwN79pv1igfCEWvHX3jOrNO9cK4pmzev05TlO+08h4cGwpF6I4O2izCWC/Chrz5gykb325Gcx186bMo2b9gULC9Flzuzl5Q7sMeO4GuPLFPXZbgkh8fsY+3fYS+TJiXbpdfU3W7KhofCbs7hPjvHobWyGkB32cgVCSzstSNbi5Hl4bZ899vB8kP77Mjb62/5uWC5GtHEtTIjgyAiu4BBoAyUVHXdjLRxHKehzMYI4XWq2jcL+3Ecp8H4HILjOBkzNQgKfFdEnhSRO0MbiMidIrJBRDb0H7ef2xzHaTwzNQjXquoVwM3A+0Tk+qkbqOq9qrpOVdf1zLUn+hzHaTwzMgiqeiB9Pwx8A7hqNpRyHKcxnPakooh0ADlVHUw/vwn4aKzOicEh/umhR4OyvkO2m+1VzeFItwvn25GEXS12BFxp3HZ9dTS1mLK5LWHX195RO0pvrGTb3BNPbzZlMe9R65DtNm0fDutSrkzYOyzZrrTO9jZT1lyxz1tHw4+HQ4N22w8YLkKA8UiU5Nzzlpiy+YsWBMt37QwngQVojbTV8sW9puyE0fYALXk7/DbfH3Yj79EtZp2JG98YLJ9pktWZeBkWAd9I1/crAH+nqmGHquM4ZwWnbRBU9QXgslnUxXGcBuNuR8dxMtwgOI6T4QbBcZwMNwiO42TUNdpxYnyc3c/vCsqODfSb9ZYtCLvZFuZt1+Jws31qMm++KctHXF8tErafi3ptV1THXPtYHV09pkya7ajLpg7b7bj03HOD5a2ttju1VLTdbPlIGwu2b7Q4Ht7nhWpndK2YKXhBJ2x3WrFgywoadgXu2xaJrBywQ3PGI7fQEzk76rK72XY7dhTCCWllwr4uE0PhCFstR9bwrAEfITiOk+EGwXGcDDcIjuNkuEFwHCfDDYLjOBl19TIUx4sc2hPOj/ji7l1mvfPOXxwsX77AzmHX2mUvM9bUu8iud649u9/WFg6mKrfOMeuo4ZkAKEfiUPI5u97EuL2UW64pPJvd3GbPgHf12F6SQmSZNHthPltUjsyCR/ZGQe2uWszbDXniSDhX57wu21NTGj5kynJlOzirqcXuOzFvTaEY3md+8JhZZ+ePw0vKjY/YXrJa8BGC4zgZbhAcx8lwg+A4ToYbBMdxMtwgOI6T4QbBcZyM+rodJybYu2dPUDY+Zuf127g37DpatvJqs87atbaseYGdg2+sGM7fCDAwEQ6UKY3ZufQqZTsAqFSyj1WIuKm0Yru+SpXw8U6cGDDrHDiwzZQVI+0xYbQHwJw5PcHyjg47D+bRI3ZezeK47VosF2yH5by2sKx/cNCsI5FAqi4jvydAa6XflOUm7HtvqdUIfIq4OB975JFg+ZAR9FQrPkJwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3Ey3CA4jpNRV7djRSuMG3niNOKCW7Li0mB57twLzTrDLXYEYmnUPtZwJKdiqRyuN7fXPlbMNVeciLgPsV1fldg6b7mwu/K73/6OWeX768MuLICu7h5TNmbkTQS49rXXBssvvvhis86jPwhH8AEMj9vXrBLJ03j9la8Oli9ZFs49CVBot/NPtrfZP5njGnEjF+1r1jwSdhWOlez2PWZ1ncoZzqkoIp8TkcMisqWqrFdEviciz6Xvvqyz4/wboJZHhvuAm6aUfRB4UFXXAA+m3x3HOcuZ1iCo6npgaqaGW4H708/3A2+ZXbUcx2kEpzuHsEhVDwKo6kERWWhtKCJ3AncC5At2bnrHcRrPGfcyqOq9qrpOVdfl83Wdw3Qc5xQ5XYNwSESWAKTvh2dPJcdxGsXp3rIfAG4HPp6+f7OmWgqUwm6R1rY2s9pl634qWD6n206kOjZiJyJt6rKTs0rOdmGVJ8K6799/wK4TSSra1Wkn+sxHkrPmI4lPDx0KJwhdv369WefqK68yZStXrTZlR4/ZSUAXLw4nsl24yF7a7robbzBlBSN5LEDZiPAEKOTD7VguLbDrLHuVKavkbD0KJTtit7x/tykbPRTuP2XDzQ1w5NiJYHnJSNhaK7W4Hb8IPAZcICL7ROQOEkPwRhF5Dnhj+t1xnLOcaUcIqnqbIXr9LOviOE6D8b8uO46T4QbBcZwMNwiO42S4QXAcJ6Ou/xRSrVAshqP/5i9aatYrNIWjz8aMyEmIu/RUI4sqiu127DNcen2H7bUArehDgPPXnG/KCh32Woz5nB2N99QTG4PloyN2Wy1aYkf+PRKJQNy0cZMpu/nmm4PlI0W77fcdivydJXLJIiLKRpLbYqQPtDTb7Tuvw+5Xne32z2luhx0Ru8+IGm0es69ZodAcFkT6by34CMFxnAw3CI7jZLhBcBwnww2C4zgZbhAcx8lwg+A4TkbdExRUjMi0BQvs6LOikWCz0GmvE9jcbLhlIjoAaGQtRjUi7trn2Skl25vsJm6KyFRs15eobcf7j4ej4Ew3FTA4YK8H+OILL5qyE5FoRzHauKXF1iMXWc+yr++4KRsethPjjo2PBcubI9G1CyPXc/5C2304v8d2SY6P2td6+4lwhOKCSJLVtnnhaNJcxM1dCz5CcBwnww2C4zgZbhAcx8lwg+A4ToYbBMdxMurqZZBcjtbW1qBs5aqVZr1cPhywEUk7SCWypFXOyLOXCO2gl2VtYVm3DJp1NOLRGFV7xnoYO+9jPhLOs3jJkmD5tu3bzTqx8KDhQfvcKiW73kQxPEM+2B/2ggD0HxswZdu2PW/KikU79+D4WNjLkGu2cyMOL7Cvy/md9iz+sNj5DEfK9vFKGva8jE30m3WOHQsH1BUjeRhrwUcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3Ey6ut2BHLGUmnz5/eY9RYtDi//NTRiu6kQe38ltV2SFY3kaTwWXo6rbfcGs05RbNfc6Fo7oGuiObLMm9p587ZtD7vniiXb/Zlvtu8LFex6pUj3OdAXzo/Y0mO7U49EApiO9/ebskpk+TKphNtfIq7KwWb7mh09buvRNxDpj932Enb51nAgW2HCvs5HBsNLFZYiwXm1UMtSbp8TkcMisqWq7B4R2S8iG9PXLTPSwnGcVwS1PDLcB9wUKP8TVV2bvr41u2o5jtMIpjUIqroesAPfHcf5N8NMJhXvEpFN6SOFmVFCRO4UkQ0isqFcntlS1Y7jnFlO1yB8GlgNrAUOAp+wNlTVe1V1naquy+frnqDJcZxT4LQMgqoeUtWyqlaAzwBXza5ajuM0gtO6ZYvIElU9mH59K7Altn01Vs63jsgSWXPmdAfLh0ZtN0+lbLsWY8tdxVbCEsMlWRqzp1jyzXbuPiYiEZkVWxGNuDJHRsPuqIkJ282WL9iReL0LbdfoyICtfxNh/YdG7PyHQ6N2bsfubjt/ZouR6xKgzViWbWQivKQgQGeXfawDg/Zjr0bc2TJx1JQNjY4Gy5dE+kDeuJyRrlET0xoEEfkicAMwX0T2AXcDN4jIWpK42V3Ar8xMDcdxXglMaxBU9bZA8WfPgC6O4zQY/+uy4zgZbhAcx8lwg+A4ToYbBMdxMur6T6HWthYuunh5UNYciTAraTiCKx/JsqoV2z3UJHaizEjeUCZaw8k32xatNuuU87ZLrCyRZeMiUYYa8Y22tISPF1vhq60tnPgW4A2vv8GU7Vmxz5R1doTdrXv2HjDr7H5xrylriSSCLTXb+g9ouN7EsJ08dqDVdoEfyB00ZZGV6Ghrs4VrOsJJVguRiNfujnD/zjf5Um6O48wSbhAcx8lwg+A4ToYbBMdxMtwgOI6T4QbBcZyMurodm5tyLF3UHpR1N9tutqZSODJNIj7C8UgSzZjbrlSy3ZUnOs4Jlh9daSfQbIqEn2nejqprztnuynze3md3dziJ6cJI1GJvr5nfhjmd4UhTgHIkWrNSCbf/DWuuNuusOS+SiHTEjhZsbZtnykrtvcHyCexkuoVI9OeCrnD/BeiVcNQiQHPEG7ivFN7nU9//Z7POaC58Ly/HwnVrwEcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhORl29DLm80N4TDkSZM8eeve3qCgfKHO231R8v2zPxE6P2DHMx4p0QCc8+l8TWY8yUQC6yJFtupN+UFSIBU73zwjPu3d22t2Dx4sWmrKPNPtb8BeFgL4C+o33B8pUrwsFtAFdeeoEpG3lusynLz7H1ONgU9ro892J4WT6AcmRpuEohHIgEMDAaudoTtgdi1Ai0WrJ4oVmn2Bbu382P2NerFnyE4DhOhhsEx3Ey3CA4jpPhBsFxnAw3CI7jZLhBcBwno5aVm84DPg8sBirAvar6KRHpBb4MrCBZvekdqno8tq98SzNd5y8LysaNJbcAjh8Mu7COD9p58U707TJlHZEcfO0ddsBRU1tYx0KL7YrK5ewmzhsBKoCxEFqCRqQthi6HDx8x66xf/6gpW7F8kSkrNNtt1d4ebuMTW58w6/zDD+wVAfcM2a7iof5+U3bsRHh5uIHIknKVyCrlGrmHdkfcn3M77SX9VhsuxMtuutKsU+gNX+fWv7d/R7VQywihBPymql4EXAO8T0QuBj4IPKiqa4AH0++O45zFTGsQVPWgqj6Vfh4EtgJLgVuB+9PN7gfecoZ0dBynTpzSHIKIrAAuBx4HFk2uAJ2+23+rchznrKBmgyAincDXgF9TVXsd9pfXu1NENojIhuHB8FLljuO8MqjJIEjyJ/6vAV9Q1a+nxYdEZEkqXwIcDtVV1XtVdZ2qruuIZJtxHKfxTGsQRERIln/fqqqfrBI9ANyefr4d+Obsq+c4Tj2pJdrxWuDdwGYR2ZiWfQj4OPAVEbkD2AO8fbodqQjaFI7GKquddG7CyJ340n57WbB//dZ3TFlTJMqw0GQ3Sa417OrpMPIYAvTM6TFlvb3hfH8Aixfb7r6Ya3TjExuC5YOG+w2gs8V2VQ3Msd1l5SF76bV5S8Pu5Zbjtmf66PPPmrJjeTvfYrkYzrkJ0JoPX8/OeZH8jbE+EHMVR2SFyNJ8BQm7OSeMXKIABZ1ZVKO53+k2UNXvY7vFXz+76jiO00j8n4qO42S4QXAcJ8MNguM4GW4QHMfJcIPgOE5GXZOsCtAk4eW/8nl7WbDeuWG33tionbhyzjzbpVccsusNDdruORkPR9zt3LHTrNPWZrvtmgwXLEA+Z7tG8wVbNmjoX67YdY4cOmTKFre9zpSt6rKjPAf7w0uvjUdcvlakJkAztltamuw/vFUqYZd1RW03IBFZuWjLSsXI0nZlO3nv6PJwlOSRiDuV/nBC1/FIpGYt+AjBcZwMNwiO42S4QXAcJ8MNguM4GW4QHMfJcIPgOE5Gnd2OFXIaTpJy6NgLdr3xcDReT6edLLVn7lxTNhhZi1GN9RsBuowow/7+WL4Y2+aWI+tPDg3ZSUBz2PXMpK55223XH0lc89VvP2bKeiK3EzUSji5fbF+X40VbxxPDdpRksWy7AicMV2BZ7TYsFiNrNFbsYzVFXMUXXXi+KVv+6tXB8r6JfrNOyegD5Yh+teAjBMdxMtwgOI6T4QbBcZwMNwiO42S4QXAcJ6OuXobWfBMXdJ4TFkZm3Nt7wjPWYwfs2fHWSy41Zc9u22HKynMjwTdWrr0d28w6pZIdbJLkr7Vktq3OR2RIeKZ+Trcd7NXcYgdgHT16wpQdEXt5tdyR/mB5S8UO8nnb295qyv7iL+8zZfv2HzRlFeOe19phB0StucD2CLS32Z6t7c/aOSGPHg8HewEcOxZe0qTJviwUjGY0YgdrxkcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3EypnU7ish5wOeBxUAFuFdVPyUi9wC/DBxJN/2Qqn4rtq/WXBMXdIeXKBsbs/PHjY6Eg00KE3bAyzmLwkuJATxjewlZuXqFKetoCQc+/fCRR8w65UjgTS4ScJQzF8sCMVyLAE3N4UCwQsG2/StX2W1FZAmyzc9uMmWtRu7E8WH7Om959jlTNjwSWzncdlmr4ebMqe3+7Gyx27c8Ybta1VhyEODHP3ralA0ZLtpf/8CvmHU6OsI/3QdavmfWqYVa/odQAn5TVZ8SkS7gSRGZPOqfqOofz0gDx3FeMdSytuNB4GD6eVBEtgJLz7RijuPUn1OaQxCRFcDlwONp0V0isklEPicidqC74zhnBTUbBBHpBL4G/JqqDgCfBlYDa0lGEJ8w6t0pIhtEZMOxE7FEIo7jNJqaDIKINJEYgy+o6tcBVPWQqpZVtQJ8BrgqVFdV71XVdaq6rndO92zp7TjOGWBagyBJBM5nga2q+smq8iVVm70V2DL76jmOU09q8TJcC7wb2CwiG9OyDwG3ichaEp/PLsD2kaRUUEaN6L/ByLJsx/sN96LtHWLrFtsl9uKO7aasqWAvJ9ZjjHDKJTvEzHYeQi4S7UgkolEj+QCtXXZ1hfNBAoyM2MvXrVy5wpQdPGBHUL6070CwfHvksXHLc8+bspiLViP3NSuidGzYdh8+8dhTpqwcWZJtTo89An7djdeastdcFo7MXbT8PLNO3kqd2WT331qoxcvwfcL9OvqfA8dxzj78n4qO42S4QXAcJ8MNguM4GW4QHMfJcIPgOE5GXZOsliswOB52Aw2M2a600Uo4yrCtzXbzLOi0l+O6fLUdinHg4C5TtuOZsMusXD69zJbFYiRyzm4OchHXqBWb2NnZadbZtetFUzY6ZkcZdvf0mLKjR/qC5e1z7CSlixaHI2EBBgZsd+VLLx0yZaNjVj+IuCoj/uxyJEnsuecuMWV3/PLtpqy1LRyhWoq4OEtG0GjMJV0LPkJwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3Ey3CA4jpNRV7cjQMWI/xMrfAvIGbL2Nnt9vnOW2i6g3l47udPyY8dM2eHD4fX5+lcsNuuMRqI4JyIJO4eHbXffaNF2LRUrRrij2q7RCy+62JQdHxg0ZfsP2u6+iXI4qvXG615r1lm+zHYHHz9mr404MmInbn3iiSeD5bt37TPrKHZbtbTZLt9LL73QlJUn7H4wMDocLC/kbPdnwYiG1UhS31rwEYLjOBluEBzHyXCD4DhOhhsEx3Ey3CA4jpPhBsFxnIy6uh0FaJawy6yUtxOO5lrDrp7Ogh3BJxU7qq5UshOOLp5vR1CWV4XdYhPjtksploB1IhLtODpqR2uODNvH6+8PJ0zdtPOwWWdoqM2UTYzbbqyB/hOmTCrh6/zjDXYC02c3bjZl7R22jh2d9vVsbQp38eZCJFGt2ue8JpL4dGGPrcfB3XYC2fb2cL32VrsPkzN+upWwu7dWfITgOE6GGwTHcTLcIDiOk+EGwXGcDDcIjuNkTOtlEJFWYD3Qkm7/VVW9W0R6gS8DK0iWcnuHqhprriXkqNBGeIa8qSkyO9pk2K2IJ6FiLBkHUIzISkU7oKRUDjdXsTWyplwEa5mxRI+I/uN2rr2RoXCgTEe7HQi2cYcd6FMct4OsFnWEcwECZjCVDtueCdvnAuVBu61Gm2xZc0u477xmtZ2/sbvD9l4tWjjPlBWKtvenqWR7Sdok7EVrtTwJQIvxm4itDlgLtYwQxoEbVfUykqXfbxKRa4APAg+q6hrgwfS74zhnMdMaBE2YdG43pS8FbgXuT8vvB95yJhR0HKd+1DSHICL5dOXnw8D3VPVxYJGqHgRI3xeeMS0dx6kLNRkEVS2r6lrgXOAqEQmvXx1ARO4UkQ0isuH4gL3suOM4jeeUvAyq2g88DNwEHBKRJQDpe/C/sap6r6quU9V1c7vtyRrHcRrPtAZBRBaISE/6uQ14A7ANeACYXI7mduCbZ0hHx3HqRC3BTUuA+0UkT2JAvqKq/09EHgO+IiJ3AHuAt0+7JwErhqnQEl6uDUimMEMYATQAsRCPQs62g9pk61Ey8gQWi3YzipH7DkAjQTS02OdWabUDpuZ0hUdh8yPusksvWmHKBiOPeWPjtqPQWt6uVIosT1a0zyvmoi0U7DbO58Oy5mY7N2I+0gdaW21Xa0uLLbMCmABaDVlzZH+W/gXjfGtlWoOgqpuAywPlR4HXz+jojuO8ovB/KjqOk+EGwXGcDDcIjuNkuEFwHCfDDYLjOBmiaru3Zv1gIkeA3enX+UBf3Q5u43qcjOtxMmebHstVdcHpHqSuBuGkA4tsUNV1DTm46+F6uB5B/JHBcZwMNwiO42Q00iDc28BjV+N6nIzrcTL/rvRo2ByC4zivPPyRwXGcDDcIjuNkNMQgiMhNIrJdRJ4XkYYlZxWRXSKyWUQ2isiGOh73cyJyWES2VJX1isj3ROS59H1ug/S4R0T2p22yUURuqYMe54nIQyKyVUSeEZEPpOV1bZOIHnVtExFpFZEficjTqR6/l5af+fZQ1bq+gDywE1gFNANPAxfXW49Ul13A/AYc93rgCmBLVdkfAR9MP38Q+N8N0uMe4Lfq3B5LgCvSz13ADuDierdJRI+6tgnJusid6ecm4HHgmnq0RyNGCFcBz6vqC6o6AXyJJIPzvxtUdT1wbEpx3bNYG3rUHVU9qKpPpZ8Hga3AUurcJhE96oomNCTTeSMMwlJgb9X3fTSg0VMU+K6IPCkidzZIh0leSVms7xKRTekjxRl/dKlGRFaQJORpaGbvKXpAndukUZnOG2EQQrmwGuX7vFZVrwBuBt4nItc3SI9XEp8GVpMsynMQ+ES9DiwincDXgF9T1YF6HbcGPereJjqDTOczoREGYR9wXtX3c4EDDdADVT2Qvh8GvkHyONMoaspifaZR1UNpZ6wAn6FObSIiTSQ/wi+o6tfT4rq3SUiPRrVJeux+TjHT+UxohEF4AlgjIitFpBl4J0kG57oiIh0i0jX5GXgTsCVe64zyishiPdnhUt5KHdpEkgyqnwW2quonq0R1bRNLj3q3SUMznddr5nTKLOotJDO4O4EPN0iHVSQejqeBZ+qpB/BFkqFnkWTEdAcwj2SNzOfS994G6fE3wGZgU9oBl9RBj+tIHhs3ARvT1y31bpOIHnVtE+A1wI/T420BfjctP+Pt4X9ddhwnw/+p6DhOhhsEx3Ey3CA4jpPhBsFxnAw3CI7jZLhBcBwnww2C4zgZ/x9pJMz6xIk8qwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQQAAAEICAYAAAC5yopxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAArBklEQVR4nO2deZwdV3Xnv6f3RWrtbrVa+75aC+0FC2zFGC9kiHFCSIAwJmPiTD5xAoTkEwLD4CRMwuSThCSTDGCDsQPGQACzBbCNx8SWF1ktW5K1WWtrbam1dEst9fr6nfmjqstPz/eUntXSa4mc7+fTn37v/t6tunXrvvNu1alzrqgqjuM4ACXD3QDHcS4d3CA4jpPgBsFxnAQ3CI7jJLhBcBwnwQ2C4zgJl5VBEJEWEbkpfv0JEfnSeW5ns4isupBtK3C/94rI14q9X8cplMvKIOSiqn+lqh861+dE5EER+Uxe3UWq+vOL1rhhQETeJyJ7ReSMiHxPRMamfPYpETkqIqdEZIOI3J6n/4GI7In1ZhF5S2AbY+NtrM4rXyYi60SkK/6/LEerFJHPicghEWkXkf8rIuWBbc8RkZ5c4yki00VEReR0zt+ncvTRIvKQiLTFf/fmbbNFRLpz6j6eo4mIfFJE9sXH/A0Rqctr9wOxdlhE/ihv2+8UkU3xdp8TkYU52mIReUxEjonI6x76EZF74j7uFZEHA/p7RGSriHSKyBYReVeevkJEno73fUREPlzIMZuo6rD8AWXnUacFuOkN1nkQ+MxwHWdeW+4FvnYRtrsI6ASuB0YAXwe+kfL5Kwf7H7gmrtuQ8/4M8CZAgN8DjgKledu4H3gaWJ1TVgHsBT4KVAJ/GL+viPVPA88AY4EJwAvAnwfa93j8ua/llE0H1Bo3wFeAfwNq4s/uAn67kLED3AlsA6bE/fd94KEc/a/j9owBFgCHgVtjbQ5wCngLUAb8GbAzp3/nAXcBt0dft9ft+1eBdwGfBx7M0xqBPuC2+Fz8MtAFXBHr44E24P1xf48EFgzp+3KBB2ZL3CFbgPb4JFXF2irgAPCncYd+lWiG8vH45B0HvgWMzdneB+IBdRz4ZO4Bkvflik/Ic0AHsB/4IHA30B936mngh/kdFXfkPwCH4r9/ACrz2vyxuONbcwdZAf0xA/gPoi/cE8A/57X5V4DNcZt/nncyVwAvx3X/DfgmhmED/gr4es77WfExjyygjVcDPcDV8fvfAF7M0WuJvogNOWVvBp4HfpuzDcLNwEFAcsr28dqXpxn49RztfcD+vPb8ZjwO8s/vdNINwjHgqpz3nwCeKdAgfBv4k5z318V9UhO/PwjcnKP/JbHBBe4B/j1HKwG6gbfl7WM2AYOQo3+G1xuEa4C2vLKjwJtzzvtXz/F9fEMG4WJcMrwfuIVoUM4F/keONpHo12Ea0Zf1D4ms4w3AJCIj8i8A8bTr80RGYRIwDpgc2qGITAV+Avwfol+eZcB6Vb0PeBj4G1UdoarvDFT/JHBtXGcp0Rckv82jiKz1XcC/iMiYeL/vE5GNKX3xdWAdkSX/S6JfosE2zwUeAT4St/nHwA9FpEJEKoBHiWY3Y+PP3ZGyn0XAhsE3qrqLyCDMtSqIyI9EpAdYQ2SMmmPpJ0CpiFwjIqXAfwPWExlx4rJ/Ifoi5E+BFwEbNR6NMRvjcoh+5SS3GcBkERkVb7sO+AsiA2yxV0QOiMhXRGR8/mHlvV6cpz8cX+Y8LiJL8z6bX7cSmBOf60nk9G/8Ou2YQvs+H5qBrSLyKyJSGl8u9BL1KUTj9kR8mdImIj+Mvwu5WMcc5o1YjwJ+bVqA/57z/h3ALn3t17aPeMYQl20lx5ICDUS/6GXA/yRn2kv0S9VHYIZANCt51GjTg+T9snL2DGEX8I4c7RagJafN3eT8KhHNFK4toC+mAhmgNqfs6zlt/hTwrbxfloPxPq/n9b+0q/OPI0d7Mrffc37VVp2jjeVE09GP5pQJ0a9rf9z+/F/ejwKfj19/kLNnCJ8i71KFyCDfm/Mr+CyRAZxIZIyS2Qfwj8Cf5p/f+P0IoCkeG/VEv+qP5ehfA75LNG2eHZ/X3hx9JVBNdEnxZ0QGbnSsfQjYTjQLGQX8IG7Xm4kuI5Szx+3bc8bIfKJLrFVEl0yfArLAn+X1wxueIcTldxHNbjNElwu/nKNtJ5pdXgVUAf8EPFvIMVt/F2OGsD/n9V4i6zrIUVXtyXk/DXhURDpEpIPIQAwQnfBJudtS1TNElw4hphANgPNhUtxOq83HVTWT876LaHAWst32uN252w7uV1WzRMfbGGsHVc/6pc3t13xOA3V5ZXVElxsmqtqvqj8BbhGRX4mLP0Q0K1hENMB/C/iRiEwSkUlEs7pPnmc7/hfRZdB6osu77xEZnrb45uNNwOeMtp5W1WZVzajqEaIZys05N//+kMh47yC6B/AI0eXeYP1nVbVbVbtU9a+JvkhvjeUH4s//nOgS7qm4/EB8TIPH8bpjUtVtRDO/fya6pBxPdMl8gCEikUftb3jN2NwAfCnnRm030Q/h2vh79efAdYMzrnMcc5CLYRCm5LyeSnRdPkj+FHM/cJuqjs75q1LVg0Sdm2xLRGqILhtC7Ce6RAmRv898DhEZJqvN50srMEZEavO2HdyviAjR8Q4ee2NcNkhuv+azmehyZ3BbM4mmvNsLbGsZr/XfUqJ7LdtVNauqP43bcx3R5VQDsEVEDhP9ol8d33kvjdtxZV67r4zLiQfnParaqKoziQz8OlUdIBr004F98bb/GPg1EXnJaPPgeZV42ydU9f2qOlFVFxGN7RdTjllz6mZV9dOqOl1VJ8ftPUhklNvj48+dbi8dPKa4/rdVdbGqjiO6cToNWJuy70JZBjwdG8Ksqq4lmlXdFOsbOXt8n9UnATRFiz9x4S8ZXiG61h9LdGf2r/S16feBvM9/lMgqT4vfTwBuj18vIrLObyGyjn9LNG0KXTJMJbLY7yEa3OOAZbH2WXJuuOW0c3A7nyH6tZpAZN2TqbnR5qRuAf3xQtzuivg4TuW0eR7RVPNtRFP3PwZ2x5+tILoZ9wfx8dxOdLlkXTIsirf9VqJLq69heBmIpri3EU0ly4lmAH3Aili/k8iQzCQaPG8nmhXNJzIyE3P+Pkw0QCfGdQe9DB+OP3sPZ3sZBmc/QnT9u5/4Zh3RtDZ3239LdFkwIdavifusJD6/3wSeyjmuWXF5aXx8x4BFOeNjZdy+KuBPiG7OjYv1sXF9ARYCm4C7c7b9WaKbw2PifmglvlEa62+K9zshblfuDV6J97mQ+NKD+KZ1rJfFZX9NdKO9itc8FDfExzE4lpcTGdHBPruR6L7bsvhcfo74Ruq5jtkcsxfBIAx6GTqAh3jtTu0qXv/lKgH+CHiV6Au9i9iA5AzOfRTmZXgr0eA8FQ+0O+PyOURT1A7gewGDMHjt1Rr//RN5npEUY/J+YHNKf8wkMoqnCXsZ7oj76mQ84BblaE1xu08TeRm+C3wqZV/vi/vqDNGUOddb8wXgC/HrBXE/dcZ9sha4I28A/0W8rU6iy7gPGPv8IDn3EHIG7Tqi6exLwPIc7fq4/7ric/7+lOPJP7/vBfbEx9cK/CuxIYr19xDNurrifrslz2BujOseJ7rn0pSjz43b00VkwP4ory2VRJcVp4AjAX113FcngC9y9n2j6USGIPevJe848/V7c/R7iNyYnUQ/GB/L2/fvEc1m2oEfAlMKOWbrT+LKFwQRaQE+pKo/u2AbdQAQkTVEX+qvDHdbnF9cLtsnFX/REZEbRGSiiJSJyJ1E1+I/He52Ob/YlA13AxyTeUQP6IwgupR6t6q2Dm+TnF90Luglg+M4lzd+yeA4TkJRLxnGjRurU6aE3ekXeqZS/roYutfop8Kul7bR/r7zqJSG3Y5+a1/nJOxmLk/zPl+UURA+n/39do3ylEb2Z0yJ8gvefvuE9p86aWqnBwZMbcwY+1m2TCZ83JpyXGK0cX9LC8ePHUt/1iCFIXWliNxK9HBKKfAlVf1s2uenTJnCE0+E74sNZO3OtIxFNuWwG+ttrTXlGZ8Guxoc2hcunxQuPjf5j52/Ruth+8HEs5/7ydMMg1CfNkSuSNHOm/A5O3zErjExpZFH2ux69Re8/Y2mcvixn5ja6o4Tpvbu31hpam1Hwz8MOsH+kRSjjTc3NZl1CuG8LxlyglxuI3ro4r25ceCO41x+DOUewtXATlXdrap9wDeInqhzHOcyZSgGoZGzA24OEJhricjdcUaY5uPHrdgkx3EuBYZiEEIXfK+76FHV+1S1SVWbxo2zYpMcx7kUGIpBOMDZEXiTuTBRgo7jDBND8TKsJcooM4MouOI3iQJs7J2VKRMmWN4E26/UevDVYPnx4xPMOpJy6zmzf7ep7ThpR8z2vNQdLF9yw/VmnVSfWIqntaHRFg+1Zk1tUsO0YPkhDA8JMCnF23HQVKCRFltsDXsM7JbbdQDqG2ztYEpH2v6CFA60mNLelIjyvhP2uIqCNcMMZI0xcjiYIAyAhonh8qF6YM+7vqpmROQe4DEit+MDqrr5HNUcx7mEGZJBUdUfE+UCdBznFwB/dNlxnAQ3CI7jJLhBcBwnwQ2C4zgJl0yClAMHbbdSe+vhYPnkyXbz+1Iidvbv22Pva9c6UyvdviRYvmCxHUlVlhLltq/GlOjO2NusaLCjJPcTdn1ZQU8Ardj90ZiWkiXFFUhD2EXbt9t2ER6cWWtqjamuxbRI2bAbdn9K1v6yyVYCbxjbeq2pte2329/fb/gJAakPjxEND/sIS0uJJi0EnyE4jpPgBsFxnAQ3CI7jJLhBcBwnwQ2C4zgJw+BlCIe3pGQFo7SkNFjefuyoWae+1L7zfPTRH5raCwftgM3xjeG7z23Pf8msUzPqalO76vrrTK39qf9narPnzTO1bGX4lParnaKuYWJK53edMaX9+21vR6nsDJZrpidYDtCInWbswCH79nlFqT2MD41YEyyfkJKboztrB9odOWq7XY7utT0XX3viMVO7cdGvBcsl5ef6YPgrMVQng88QHMd5DTcIjuMkuEFwHCfBDYLjOAluEBzHSXCD4DhOQlHdjv39cNAIymicZLsJj7f0BsufefIps87IlOXVqupPm1pme4ep7d6+LVjevn+UWaeibrWpzVtiuw+3nLHb2P1yl6nNXV4dLG9MXSpvtqns2vCSqR0cYecQnDIl3MauTtuNyYHweQaY3Gs71PaXGT444HS2M1i+70S4HGDccTv/5OYXv2JqR7fNNTV2f9Xe5qdmBMunZhaYderqwud5qCsi+gzBcZwENwiO4yS4QXAcJ8ENguM4CW4QHMdJcIPgOE5CUd2OA9rN6b7w4k5P3W8vGtZ/6ECwvP3wDrNO3RV2JGFbmx1xN35ElamNPRWO1NzaabsIsyn5/p7a/ANT6z5oR3JOHNlmat9pDvt158yfb9aR1c+a2qHHfmRqNSnL1HVeFV667EyPHUl48OgXTE0ydrRm+TV2csozp8PL1PUcazfr/HzH46Y2c3w4VySAjLHdsP0ZO6J04PGfBstPL7QjbxdXh13FFWqP7UIYkkEQkRagExgAMqraNKTWOI4zrFyIGcIvqeqxC7Adx3GGGb+H4DhOwlANggKPi8g6Ebk79AERuVtEmkWkuf3EqSHuznGci8lQDcJKVV0B3Ab8vohcn/8BVb1PVZtUtWnM2Loh7s5xnIvJkAyCqh6K/7cBjwJ2AkHHcS55zvumoojUAiWq2hm/vhn4i7Q6JzvP8JOnngtqx56z3WwDFeFIt7or7KWzrqi0I+CO99qur6YrK03tubXhEMrT3XaUXk/GtrknN7xiahr2cAIw9rS9PNzY1jHB8rY625VGxnaljXhTOKoOoCI709S2dIfP87hOu+9PVdtJW3s7w+5DgDGH7MSn4+vD+2vZZUehjqTP1PbuGGtqpaX2OKgstcNvS4+E3cj7dJNZZ1TD5GB5TzZl4BTAULwM9cCjEqVLLgO+rqphh6rjOJcF520QVHU3sPQCtsVxnGHG3Y6O4yS4QXAcJ8ENguM4CW4QHMdJKGq0Y19vL3t3tgS1Zw92mPU+NCHsZpvaYLsWz1TYkZBdb51gahuM6DiAyqvXBcvrd9r3VmvHjDe1pqtGm9q2XXbUZbbWdjuOWh52Ry2vshN2Zmbbbrbt2+0+FmOdToBFvYuD5QNqR/1lsbX5fXbUaP8SW9uxJewKPNq+3ayzsNEeH+0n95rapEVvMrWDu8NrXQJkjW+h9Nnnpe90OMJWh+h29BmC4zgJbhAcx0lwg+A4ToIbBMdxEtwgOI6TUFQvQ6avjyP7wvkRSze0mPX2vXNisHxa70izzp6Rdqj1iLH1plY1eaGpVVdfFSwfmGQv5aZi29y66abE6F67Xl9v2NsBUFIezqm3q9r2TCybsNLUVqZ4crameAUsaWDAzo0oW7ea2hm1h+rs6Xa+yEO14Vyd4372M7POwe12UFF5lR3stXOH7RkaUWEHbpX1h4PcSjuXmHV2vfxCsLy3K2WpvALwGYLjOAluEBzHSXCD4DhOghsEx3ES3CA4jpPgBsFxnIQiBzf1sX/fvqDWO8PO67d+f9h1NLUnvFwYwJtvsrVdxxpMrWeynSr+VF84UCkzw86llx2wg03aT4ZzRQJ0nLKDbzRr5yWcM29esHzbFtul98gjj5haf7/dxr4++7hHXTs6WN5Uay/udfyFNaa2sdcOYGptP2Fq41rC/s+Ozk6zjqQEUo008nsCLM92mFpFX9h1DpCpGh0s37rJzrnZUbEnWH46Yx9XIfgMwXGcBDcIjuMkuEFwHCfBDYLjOAluEBzHSXCD4DhOQlHdjllVeo08cZqx3TkN08P5+Uqus6Pczuy2IxCnldv7an7ejhbLDLwULB+z8lqzTt+G9aa2Zaqdv3HhwBxTy85PyZtXEo5O3P9E2HULcP/T95nayLrRpjajd5aprRwRjqDsXGi7Kr/8bDiCD+BMr33OWo/bbsfrR4SH+I1T327WKauxlxWsqba/Mu1qt3FH/2ZTW5oN53BsmmP37+Mv7QoLXRc5p6KIPCAibSKyKadsrIg8ISI74v/hBQUdx7msKOSS4UHg1ryyjwNPquoc4Mn4veM4lznnNAiq+jSQPye7HXgofv0Q8K4L2yzHcYaD872HUK+qrQCq2ioiV1gfFJG7gbsBKirs7DuO4ww/F93LoKr3qWqTqjaVlbtTw3EuZc73G3pERBoA4v9tF65JjuMMF+d7yfAD4E7gs/H/7xdUS4FMOMlmVbWdvHJpU3iJrFF1diLVntouUysfaSdnXfiynTj05b5w2+sOHjLrDIwZZ2pTU5Y1256SnHVBuX3pdeTIkWD5F5/+olnng1ddbWozfvm/mNrqE7a7r3FiOJHt8Xp7abu33LjK1MrKy01trhHhCVC2PdyPmzLrzTpLbrGTzmZL7CXZyl6xI3YHDjaaWndveFm2gQHbjXn0SNg12t9vR8IWQiFux0eA54F5InJARO4iMgRvF5EdwNvj947jXOacc4agqu81pLdd4LY4jjPM+F0+x3ES3CA4jpPgBsFxnAQ3CI7jJBQ12lG1gv7+mUFtfL2d3HTHzspg+ai6cOQkwMj99lqG2mQn0URsV+Axw6UnKZGaVvQhwOyUegtqw65WgNItu03t4Y3hxJzdXXZf1TfYkX/PpEQgbly/0dRO3nZbsHzKxi1mnQNHUh5nSTllnRvsczZgJLndqLb78LTaY2dcrflQLiNO7TC1slo7+vaw4b6t6LHPWVlZeK1ISRm/heAzBMdxEtwgOI6T4AbBcZwENwiO4yS4QXAcJ8ENguM4CUV1O0IP2Ww42eSxCSvMWtNnht1zO0przTqNS8NuGYBs1k5EqSlrMaoRcVcz7rBZp6Z8iantPHrc1MrFdi1eqYtMrWFieJuWmwrgxVNrTe2xx/eb2shqO9pRNofPc+XiBWadktJwMl2ACRPaTe1Ac6upvdz7crB8aUp0bW+PnWh3/BW2+3D8mKtM7Vh3uB0Ar+4JRyhOyNhux+qacKRvSYqbuxB8huA4ToIbBMdxEtwgOI6T4AbBcZwENwiO4yQU1csgJTVUVS0ParfNfIdZb0JpOKBkXImdyzCbDec/BCgpTbGDJXYUzS0rwlpdz2SzzpbsalMbMT8c6AVwZo+d93EzdoDQxIaGYPmy5eF+j7CP+czkTlPbd8Sut7A/fId8csMks05PxvbWbNpiByP177aDrPb09ATLa5ZeadY5s9Pe1+z99l385nHhwDKArgE7J2RGwx6gnr4Os87hE+FAu/60QLsC8BmC4zgJbhAcx0lwg+A4ToIbBMdxEtwgOI6T4AbBcZyE4rodgZKScM638eNHm/XqJ4aXrVq7zl466+1X2Nt7pdR2Sc6dn5KncXdNsLzlp/ebdcrFds1Nf+/1pvZctsPUSlOWgNu166fB8v6MHQhWWmH/LmQJBykBZLDdrYeOhfMjHm23A7qOHrMDmNqf7zC17GzblTlrc9hF29u8zqzTeZ0d+HT8Sjug61Vj2TgA6uwl7EZUhQPZyvoOmHWOdobHcCYlcK8QClnK7QERaRORTTll94rIQRFZH//ZDxE4jnPZUMglw4PArYHyz6nqsvjvxxe2WY7jDAfnNAiq+jRgz5Mcx/mFYSg3Fe8RkY3xJcUY60MicreINItIc7/xOKvjOJcG52sQPg/MApYBrcDfWR9U1ftUtUlVm8rL7aw9juMMP+dlEFT1iKoOqGoWuB+4+sI2y3Gc4eC83I4i0qCqg4ns7gA2pX1+kB7gVSPn275aO3dfdlTYvbVgob38W3as7VqcV5ay3FXWzvkn+nywPNNj32IprbBdWPTZbVw0b6Gp7dpmRzuu654WLG/ts/MOziizI/HGXjHB1KadCrthAcoXhNt/ep2dr3DtXju346m6k6a2bKIduVh9JrxM3bq+9Wad7sO2i/bQjDmmNmqi/TWQlCXgOo5ND5dnD5p1Sq2gxpQl7wrhnAZBRB4BVgHjReQA8GlglYgsi3ffAvzu0JrhOM6lwDkNgqq+N1D85YvQFsdxhhl/dNlxnAQ3CI7jJLhBcBwnwQ2C4zgJRY12rKrOsmBhV1Cr2GX7SzJnwhFc29tSljS7drSplUulvS+1n6bs2xNexqukfpZZZ6DUdukNiB2ZtmWrHWVYYUSMAlRWhhOElmzfZ9apXmG7OG+atMrUaqfbiWBH1IbdrY9tshPj7t3TYmqVC+abWnO77ZIcOBx20fadsZPHNi63l2Rbt36bqW0v3WFqK6pTxlztrmD5TRUjzDqj68Puz22tHWadQvAZguM4CW4QHMdJcIPgOE6CGwTHcRLcIDiOk+AGwXGchKK6HevKa7mp/k1BrWnpPLNedV04UuypQ3aU4fqN9hp3FUttt10mkzG16U3hZJ67t/yeWSctyWpdlR1Vt7Sk19S2L07Z5vFwJPoNN0w366xceZ2pbdpgR2TW1Ow1tWw23P+r3nKNWWfOFDsRaek0OzlrVfU4U8tMCmt92O7lJWV29OSEq+0IzyMy1tQqSu0o2pGZ8HqcD9//M7PO9CXh3CID21MieQvAZwiO4yS4QXAcJ8ENguM4CW4QHMdJcIPgOE5CUb0MJaVCzeiqoDZqlH339sCBcKBM+VK7+b077byDfS/ZwTz9/bZ3YpOEA4e0xL7r32PfXKYrZUm2knAMGADT1tkBUyvHhZdQy9TZxzWx3V5CbdY8e18n6u278cdWHwuW33hlOOcjQNVi29O0bse3TG3x1YtNrXXlm4PlP/hxeMk7gO394fMM0FbWZGoV3fa4WtBnLzv4XPORYHnDRDsP45SZYY9XRYV9vgrBZwiO4yS4QXAcJ8ENguM4CW4QHMdJcIPgOE6CGwTHcRIKWblpCvCvwEQgC9ynqv8oImOBbwLTiVZveo+q2v4roK8yy4HZ4aW8eivsnHP7Wp8Jlk+sDweFALyw3bZ1tRU/MbWaWjvgqNzIi1dWaS9iW7LN7uLSRXYbxfZgoWk5FQ+G2/IfbUfNOtU1z5na9K56UyurtHP+1dSE3csnt9rLtX3vhL0U2r61G0zt++vtIKsTJ08Hy9esazbrZOfaAW7zn3vZ1OpGhXNuAnx7f4upzaoOu62X3mrndpy/bEawvKrG/h4VQiEzhAzwMVVdAFwL/L6ILAQ+DjypqnOAJ+P3juNcxpzTIKhqq6q+FL/uBLYCjcDtwEPxxx4C3nWR2ug4TpF4Q/cQRGQ6sBxYA9QPrgAd/7cfq3Ic57KgYIMgIiOA7wAfUVV7HfbX17tbRJpFpLn7lP34puM4w09BBkFEyomMwcOq+t24+IiINMR6AxB8iF5V71PVJlVtqq4LxyQ4jnNpcE6DICJCtPz7VlX9+xzpB8Cd8es7ge9f+OY5jlNMCol2XAl8AHhFRNbHZZ8APgt8S0TuAvYBv36uDVX1VjN/ZzhCbqCt1KzXlwm7ZQ4fTFkWbNtjplau9nJcZeV2l5RUhV16tXX2kmajR402tZ4OOwffxIm2u2/vPts12tkZdqdVn5xi1jlZaeemXNNnz+rm/vujpjaucWqwfHe7vdzZ8Z0HTO3EeNvFPNBl55/cYyyvNmlcSv7GxiWmNrMkxVWconW02uOqTMLLss2aM9OswxYjwrbHrlII5zQIqroasBzfbxva7h3HuZTwJxUdx0lwg+A4ToIbBMdxEtwgOI6T4AbBcZyEoiZZ7QV2SnhpsLml9pJhY8eE3XpPPPkfZp1R42yX3sbTy01tSqcdjSe94eW/dm3fY9apNiIkAcrL7YSYpa/aEY3by2xt8uRwdN9A1g6fPHrEDlJdNueXTK1zpB3lOaoj/CR7r9oJTCtTokYrsN3S23ba0Y7V2XDy3mxp1qzDls2mNGDngeWVjfYYzg7YSW6n3xoeI0c32u5UasL+xd6UpQgLwWcIjuMkuEFwHCfBDYLjOAluEBzHSXCD4DhOghsEx3ESiup2FLKUaHjRwp+f2G3X6w0n7Bw9wnYfbmOXqVWJfdj75DpTG2lEGe6utKMnF6XY3IGBV01tbWM4GS1AyS57LcnGAePYSu1jfr7TXkhy597nTW10ys+JHgq7zKZNPGzWaT/Uampjp841tfET7GRdff1hV+CA2mt/9vfbIYOP7bfdleVttjv4V+94p6lNWzIrWP7Mtg6zToZw+89k7XFTCD5DcBwnwQ2C4zgJbhAcx0lwg+A4ToIbBMdxEorqZagsGWDeiJNhcaDDrFcz+tpg+YxD9t3xqu7FprZl3HZTGxiTskxar9Fdx2wvQyYl2ERktq3tspc1W5xmxyV83Ifq7GCvUbs7Te34VuN8AUcXhu+OA5Ts/GmwvDI7yazz7nffYWpf+OKDpjaibrSpZY2+qqo9btaZM88+L/Oqwx4vgFeztvfn+HOrTe3EnLCXpHyaWYUl5QuC5c9VvGhXKgCfITiOk+AGwXGcBDcIjuMkuEFwHCfBDYLjOAluEBzHSTin21FEpgD/CkwEssB9qvqPInIv8DvA0fijn1DVH6dta0zpSN5Vtyqo9fSsN+t1d4VzGX6lz84FOGlVeCkxgKydipGuvummVjsnnA/whdVbzTo6y3Y7lpTartESc7EskJT8guUV4fx8S3bYtn/WnFtMbesCO5gns2WjqVUtCedw7Gg7YtbZtMUejlOn2T64E+0dpqbZcC7DV7fYOQ6ryu3+HciExyLAq9vsZeraeuyl49b+eziA7KM3/65Zp7Y23Ffl+m2zTiEU8hxCBviYqr4kIiOBdSLyRKx9TlX/dkgtcBznkqGQtR1bgdb4daeIbAUaL3bDHMcpPm/oHoKITAeWA2viontEZKOIPCAiYy504xzHKS4FGwQRGQF8B/iIqp4CPg/MApYRzSD+zqh3t4g0i0jz8c6OITfYcZyLR0EGQUTKiYzBw6r6XQBVPaKqA6qaBe4Hrg7VVdX7VLVJVZvGjRx9gZrtOM7F4JwGQUQE+DKwVVX/Pqe8IedjdwB2NI7jOJcFhXgZVgIfAF4RkfVx2SeA94rIMkCBFsD2kcRkVek2ov86u7vNeu0dJ8KC7R1i6ybbJfbMdjuX4e0L7VyMow9dEyyfO9vO96dqR8CViO1aTMvFqNjbtDY5cmQ4HyTAuq7w8m8AM05MN7Xe0XYE5VMHngyWZyvtvhp42XaZlZTaJ3vLTvt8zp81P1g+Y6odqXm6wx6LAylLsi3EXspt0W+tNLXyirD7uX7aFLOOlSKzNGV5wEIoxMuwGoJO8dRnDhzHufzwJxUdx0lwg+A4ToIbBMdxEtwgOI6T4AbBcZyEoiZZPaPwYm/YL3aqJ+weApiSbQ6WV1fbS3hNGPGyqb1vlh2Kcah1jaltbw+7nAYGbHcT2JGQ/bYHK8WxCCVUmJoVmzhixAizTkuL7TDq7qk3teXL7aX0yp9eGiyvqdpj1lnyS6tMbc2aU6ZWIXaUYdZcss2ONE2LJt28x37c5j3vnGNqd/3Onaa2pyW8jGFmwHadZ3rDSVY1JdFrIfgMwXGcBDcIjuMkuEFwHCfBDYLjOAluEBzHSXCD4DhOQlHdjlX0MI/wOohnSu2kqM1GaNebVuw162T6bjS1nrE9ptb1rO3e6jsTdnPKGDvZa3e37YqaNcuOuGvuWmdqp/tt11L7gXCS1f6Jtrts/oJwQlSA9lOTTe1gq50wddbccFTrtKnXmXXqG643tbdeb6+N2NV1m6mdPh1e/3NvywGzzpbddl9VpHxlShfba1Nu6kuJoKwPR/OWldiu0SUSHh+VFWfMOoXgMwTHcRLcIDiOk+AGwXGcBDcIjuMkuEFwHCfBDYLjOAlFdTsKVVRIOKoxU2q7Za6uCkfO6ej19r6yo0wtk7ETji6eO93UBmaGwxP7eu2ov4GMHQk5a7btdpwywY5o7Dpj91VHQzgx58Yf2Qtanp6ywtT6usNuO4BT+0+amhhRdz1nXjLrtOyw18Gsqa02tdF1Kee6d0OwfNceO+qyIiXWdM7Um03thtH2uBq9N7wuKEBNTdh9W1Nl92/mVeOr25UWeXtufIbgOE6CGwTHcRLcIDiOk+AGwXGcBDcIjuMknNPLICJVwNNAZfz5b6vqp0VkLPBNYDrRUm7vUVU7ygfIlvVxZsL+oJYxlngDGH/0aFgYWWXW2Zyx7yL3v2Lva2ytvc3MwJLw9qr6zDrhzHcR0mAv5dY+ZZKp9ffONrVpp8PBLbVL7ACgXrEDfdp6bS/DutpwLkAANHy3u/6MfQe/j2OmNtAZDooDeGmn3Y8VleHfvCunrTLr1DXZ+SdXHR9namX9tvdnZ8b2ktRtC3uvqqvtr2flleHj6rV3UxCFzBB6gRtVdSnR0u+3isi1wMeBJ1V1DvBk/N5xnMuYcxoEjRhcDbQ8/lPgduChuPwh4F0Xo4GO4xSPgu4hiEhpvPJzG/CEqq4B6lW1FSD+b+dEdxznsqAgg6CqA6q6DJgMXC0iduaGPETkbhFpFpHm9lP2suOO4ww/b8jLoKodwM+BW4EjItIAEP9vM+rcp6pNqto0ps6+WeM4zvBzToMgIhNEZHT8uhq4CdgG/AAYXI7mTuD7F6mNjuMUiUKCmxqAh0SklMiAfEtVfyQizwPfEpG7gH3Ar59zSwKlhoeorLLcrFZVcWVY2GW7sCpm2ctglS217aCq3Y7MQE2wvL/f7sYD8qq9ry3WwmswKmXptXlVdaY2MDJc77/eYLvLunumm9qLa9aaWlOvnQPRWt4uM8devy7TbwfmyLZrTG3FXPt8li4OaxW7wgFzAKVX2mNgeVU4ZyXA7t22G3bSBDuPZ1VX+HxWVNr7qhgVbn9p6SNmnUI4p0FQ1Y3A68L5VPU48LYh7d1xnEsKf1LRcZwENwiO4yS4QXAcJ8ENguM4CW4QHMdJEFXbdXfBdyZyFBhcf208pIS3FQ9vx9l4O87mcmvHNFWdcL47KapBOGvHIs2q2jQsO/d2eDu8HUH8ksFxnAQ3CI7jJAynQbhvGPedi7fjbLwdZ/Ofqh3Ddg/BcZxLD79kcBwnwQ2C4zgJw2IQRORWEXlVRHaKyLAlZxWRFhF5RUTWi0hzEff7gIi0icimnLKxIvKEiOyI/48ZpnbcKyIH4z5ZLyLvKEI7pojIUyKyVUQ2i8iH4/Ki9klKO4raJyJSJSIvisiGuB1/Hpdf/P5Q1aL+AaXALmAmUAFsABYWux1xW1qA8cOw3+uBFcCmnLK/AT4ev/448L+HqR33An9c5P5oAFbEr0cC24GFxe6TlHYUtU8AAUbEr8uBNcC1xeiP4ZghXA3sVNXdqtoHfIMog/N/GlT1aeBEXnHRs1gb7Sg6qtqqqi/FrzuBrUAjRe6TlHYUFY0Ylkznw2EQGoHc1VoOMAydHqPA4yKyTkTuHqY2DHIpZbG+R0Q2xpcUF/3SJRcRmU6UkGdYM3vntQOK3CfDlel8OAxCKInacPk+V6rqCuA24PdFxM4J9p+HzwOziBblaQX+rlg7FpERwHeAj6jqqWLtt4B2FL1PdAiZzofCcBiEA8CUnPeTgUPD0A5U9VD8vw14lOhyZrgoKIv1xUZVj8SDMQvcT5H6RETKib6ED6vqd+PiovdJqB3D1Sfxvjt4g5nOh8JwGIS1wBwRmSEiFcBvEmVwLioiUisiIwdfAzcDm9JrXVQuiSzWgwMu5g6K0CciIsCXga2q+vc5UlH7xGpHsftkWDOdF+vOad5d1HcQ3cHdBXxymNowk8jDsQHYXMx2AI8QTT37iWZMdwHjiNbI3BH/HztM7fgq8AqwMR6ADUVox1uILhs3Auvjv3cUu09S2lHUPgGuBF6O97cJ+J9x+UXvD3902XGcBH9S0XGcBDcIjuMkuEFwHCfBDYLjOAluEBzHSXCD4DhOghsEx3ES/j/AHqsvyh/hDAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "image_show(image, pred+ \" \" + str(score.item()))\n",
    "image_show(unnormalize(perturbed_image_fgsm), new_pred_fgsm + \" \" + str(score_fgsm.item()))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now perform a similar attack using Projected Gradient Descent (PGD). PGD is an iterated version of FGSM, making multiple steps based on gradient sign, bounded by a fixed L2 or Linf norm.\n",
    "\n",
    "In this example, we use cross-entropy loss rather than the default log-loss, and also target this attack to predict the ship class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pgd = PGD(net, torch.nn.CrossEntropyLoss(reduction='none'), lower_bound=-1, upper_bound=1)  # construct the PGD attacker\n",
    "\n",
    "perturbed_image_pgd = pgd.perturb(inputs=image, radius=0.13, step_size=0.02, \n",
    "                                  step_num=7, target=torch.tensor([8]), targeted=True) \n",
    "new_pred_pgd, score_pgd = get_prediction(net, perturbed_image_pgd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQQAAAEICAYAAAC5yopxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAolklEQVR4nO2deZhc1XXgf6eqel/Uau0ItCKz2ggilhiGYLwBXzLYE9uficfBMyTEMyZ2tvni2ONAPE7iyRc740zyOcGxA04cL/ESk/kcLyEQYYMxAgtJoAUE2oWkltTqvbuWM3+818+l5p7bJXWrCiXn9331VdU977533n23zrvvnjrniqriOI4DkGu0Ao7jvHJwg+A4ToYbBMdxMtwgOI6T4QbBcZwMNwiO42ScVQZBRHaJyBvSzx8Skb86zf08IyI3zKZujaC6PRxnNjirDEI1qvoHqvpL020nIveJyMem1L1EVR8+Y8r95NgPi8i0OtYDEVkhIg+JyIiIbIsZEhHpEZH7ReRw+rpnivy1IvIjERkUkU0icl2VTETkwyKyR0QGRORLItJdJX9GRIaqXiUR+ccq+VoReTLV80kRWVsle4+IlKfUvyGVtYjIZ0Vkd6rXj0Xk5innr1PqfqRKfo+IFKfIV9XSfiLyOhHZLCL9InJURL4hIkur5O8QkUfTug8H2ltFZLjquH9VJRMR+ZiI7BeRE2mfuqRK/rCIjFXV3V4le9eU8xlJj/VT1rVHVRvyAgqnUWcX8IZTrHMf8LEGnePDwC/NZhucbnsAjwGfBNqAnwf6gQXGtn8N/D3QDqwAdgL/JZX1An3A24E88J+B48DcVH47sA04D+gEvgncbxxHgBeAX0y/NwO7gV8HWoD3p9+bU/l7gO8b++oA7kn1zQE/CwwCK1L5CkCtNk/r/u3ptB+wCDgn/dwC/BHwQFXdNwDvAH4XeDiwbwXON477DuAAsCpt7z8Enqq1j03Z13vSaynmNrP8A9gF/A7wbNpJ/hpoTWU3APuA3wZeAv4mvXAfTJU8CnwF6K3a37vTDnEU+DBVP4CpFxC4Dng0vVB705O/EygCE8AQ8I9Tf0jpBfw/aaMfSD+3TNH5N4HDwEHSH0YNbfH7QBkYS4/9Z1UX/33Ac8CLoY469SIDvwxsJengzwJXBM7jwnR/7wzo8ipgHOiqKnsEeK+hex9wZdX3DwGPpJ9/FnhmyvY7gDvSz18F/keV7LVpG7QHjvMzadt0pN/fBOynqsMCe4CbpjMIxnlsAn5+pgbhVNov7U9/CDwbkP0Sp24Qfhv4StX3S4Axq69M0x4PAXfHtjkTjwzvAt4MrE4b8n9WyRaT3GGWk/xY3w+8haRjnENiRP4cQEQuBj5NYhTOAeYB54YOKCLLgH8C/i+wAFgLbFTVe4EvAH+kqp2q+nOB6h8GrknrXAZcFdB5DrAUuAP4cxGZmx73F0RkU0gnVf0wSae5Kz32XVXitwBXAxeH6k45t7eTdNZfBLqB/0hiIKu3uQL4LvCrqvqlwG4uAV5Q1cGqsqfTcvPQUz5fWvVZAttaciH5kawJHON24KuqOlyl5yZNe2/Kpil6Xi4ifSKyQ0Q+IiKFoPIii0j63zNTRLtFZJ+I/LWIzJ8i+zkROZY+1vy3qvJp209ElolIPzAK/BbJKOFUWC8iL4nI10VkRVX5l4DzReRVItJE0mbfnlL3D9M2+YEYc2Mishy4Hvh8VItarW2NFmgXVVYTuAXYqT+5206QjhjSsq3A66u+LyG5oxdIhldfqpJ1pPVfNkIgGZV8w9DpPqY8MnDynXUncEuV7M3AriqdRzn57n0YuKbG9niYKdab5G5wY9X3FURGCMB3gA9E2vv3SEYxr4vo8W7gh1PKfh+4z9j+b4GvA13A+WkbjaeyeSSjsNuAyQ5aAf6y6i64Iz2vOcAD6fn99JRjtAMDwA1VZR+pvuZp2ReAe9LPq4CVJCPLV5OMln4noH8T8M+TOqVlncC6tG8tIhnJfKdKfjHJjSdPMqo5CNx2qu1HcsP77VAfwR4hXE/yuNQD/BmwZbI/pOWfStuwRDIKXFlV9+r0OrWk12IQWB04xkdCx576OhMjhL1Vn3eTNPIkR1R1rOr7cuAb6WRMP4mBKJM+k1XvS5O7yEl3xirOI+m0p8M5qZ6WzkdVtVT1fYSkc82EvdNvkjHdub0XeFRVH4psM0Qyuqimm6TzhHg/iSF8jmQO4IskRgdVPQrcCvwGcAi4ieTHty+t+7l0+4dJ7s6Tek3KJ/lPwDHgX2vVU1VfUNUXVbWiqpuBjwJvq95YRHIkj6MTQDYqU9UhVd2gqiVVPZTK3jQ54amqz6rqAVUtq+qjJD/CyX3X3H6qegy4H/imNXoJ1FmvqhOq2g98gMToXZSK7wauJOkHrSQ3gH8Rkfa07uOqOqiq46p6P/ADkhvxVH4x1SvKmTAI51V9XkbyXD6JTtl2L3CzqvZUvVpVdT+Jhc72lTbAPOOYe0keUUJMPeZUDpAYJkvnmWAdu7p8crjcXlW2uOpz7NwgMQjLRORPIts8A6wSka6qsst4+XA6UU71mKq+S1UXq+olJP3kR1Xyf1XVK1W1l+TuecGkPP2x3q2qK1T13PQY+9NXNbcDn9f09lWl52tEpPqR4zWWniTtmG2b1vssyQ3l51W1aNSbrAsvf/wJ7fuU2o9kFLKQlxuRWqk+9mXAl1V1X2rM7gPmYj9untQmACJyLclN7qvTH3mGjwlThiW7gM0kz/q9JM/Qf6A/GX7vm7L9r5PcSZan3xcAt6afLyGxzNeRDJv+mGTIFHpkWEZird9BcjHmAWtT2ceBvwvoObmfj5FMRi4A5gPfJ33EMHTO6tbQHl+aPP+qspdNIJHcPf87yXD1v5I8Nk0+MrydxCj8VHqhz69qr10kM9g9wJPAxyO6/DBtw1bgrcS9DKvTNswDN5NMMl5SJb+cZFjeTTIJ+4MqWW9aX0g67Rbgzin7Pze9lqunlE96GT5AMgS+i5O9DDcDi9LPF6b7vruq/l+k59kZOKerSQxXLj23LwMPVclvJfmhCck80n7g9lraj2S0M7nvBSST49WegHxa773A+vRzU1U/X5tu05m25/Yq+d0kfXJRuv93k9xEetLXm9P9FUjm74aBC6ac+70kxnf6PnsGDMKkl6GfZIjSHvlx5UiGnttJftA7qfoBkdxF9lCbl+E/AI+TPJfunbyYJJNZG1N9/iFgEFqBPyUZkRxMP7dGdK6u+y6mzLhP2fanSZ6njwN/GjEIN5M8G/YDnyAZRld7Gd6bttEQyY/g8oAuvSQTXf/L0GUFifEdTff1hiltN1T1fdLVNZK23Zun7OuLwIn09WVgYZXsVen+R0h+zL8R0OV3SL0WAdnlJMZtFHhq8lxT2R+TPKYMk7grP8pPfjjL07ad9OpMvt6Vym9L23g4vc6fBxZPOaejaZ1twPtPof1+tWrfL5HcCJZXyd+T6lb9ui+V3Zjub5hkfuofgDVVdVtJJtoPkvTtp/iJ12UB8ATJb6efxGi9cYrerans9aH2nvqStNKsICK7SDryP8/aTh3HqRtn7T8VHceZfdwgOI6TMauPDI7jnN34CMFxnIya/jgxW8yfP19XrFg+/YZTqOcYRiIH01nW5GR3+9RjRerNqhbOqTA6MmrKSuWSKevstP/LFusHp8quXbvp6+s77R3OyCCIyE0k/+jKA3+lqh+Pbb9ixXI2bHg8KCtXymY967GmcpqnnYv82qIGwdBDT1ePnD1Aiz3KxTqQWObiTFiRWTaes/nDmI6odhVbuvXpzabsaP8xU3bNddeasubm5mD56fSBK9ddY9aphdN+ZBCRPIl/9GaSP6DclgYkOY5zljKTOYSrgOc1+X/5BMmfMW6dHbUcx2kEMzEISzk5SGdfWnYSInKniGwQkQ1HjvTN4HCO45xpZmIQQg8xL3voUdV7VXWdqq5bsGBq+LnjOK8kZmIQ9nFyZOO5zF6UoOM4DWAmXoYngDUispIkMuydwC/EKihQMrwJ5ZLtstmxdXuwfP7CBWad+YsWmrJiccKUbfzhj0zZ2GjY5XTNz1xv1ik02U1cjJxzQWxbXZGKLTPKczHPRGTKvRKdj4/Mgp+GW0NjdWIeiNhs/CkLQEp2++7eusOU7XzhBVN2xTVXm7J8wegjp+lpmgmnbRBUtSQid5Fk9MkDn1NVKz7ccZyzgBn9D0FVvwV8a5Z0cRynwfhflx3HyXCD4DhOhhsEx3Ey3CA4jpNR12hHiDuqLPoOvhQsb8nb6i9cbLsd9+550ZRtf+pJU5Y3jnfi0kuD5QCdc+fY+2tvNWUDh+1Ameb2cDAMQMHYZyXSvvnTDK2Mub4mxoyowEjgUFN7hymLuT8lIquYLkTbtVhoypuy3l77eh7ea/fHoRPHTVl72znB8miukjMUAuwjBMdxMtwgOI6T4QbBcZwMNwiO42S4QXAcJ6POXgalouHZ3VisRj4XnvU93nfErLN3mz0Nu/6BfzRlJ/bbAZvzl4Zng5967AdmnfY5XabsyuuvM2WPPvQvpuz8Cy4wZResfXWwvKh2ijpyduOPDQ+bMiv1F8DO554Plo+PjAXLAa58rZ1mbLxoL9PYHPE2HTg0dY3ZhL6j1rrBsOoCeynNQ0cOmrIju+01eZ/83ndM2Y1ve2ewXIx+D5DPh+/lM82i7iMEx3Ey3CA4jpPhBsFxnAw3CI7jZLhBcBwnww2C4zgZdQ9ushxcGgl6KY2NB8sfefAhs05Xk61Da3HIPtZwvyl7YcdAsPz4XtsV1dxtB+xc8GrbfajDER3HRkxZcTzsnhuPuKMiXke2Pr3RlHV3dpuy8aGwjiODthuzPB6+zgAV47wAxgu2e25oYDCsx7FwOcDBnXtM2TM/+qEpmy+2a3f/Uxvsfa5aGSxftuYis053d0+wfKZrN/sIwXGcDDcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhORl3djsMjYzzxdHhxpwN79pv1igfCEWvHX3jOrNO9cK4pmzev05TlO+08h4cGwpF6I4O2izCWC/Chrz5gykb325Gcx186bMo2b9gULC9Flzuzl5Q7sMeO4GuPLFPXZbgkh8fsY+3fYS+TJiXbpdfU3W7KhofCbs7hPjvHobWyGkB32cgVCSzstSNbi5Hl4bZ899vB8kP77Mjb62/5uWC5GtHEtTIjgyAiu4BBoAyUVHXdjLRxHKehzMYI4XWq2jcL+3Ecp8H4HILjOBkzNQgKfFdEnhSRO0MbiMidIrJBRDb0H7ef2xzHaTwzNQjXquoVwM3A+0Tk+qkbqOq9qrpOVdf1zLUn+hzHaTwzMgiqeiB9Pwx8A7hqNpRyHKcxnPakooh0ADlVHUw/vwn4aKzOicEh/umhR4OyvkO2m+1VzeFItwvn25GEXS12BFxp3HZ9dTS1mLK5LWHX195RO0pvrGTb3BNPbzZlMe9R65DtNm0fDutSrkzYOyzZrrTO9jZT1lyxz1tHw4+HQ4N22w8YLkKA8UiU5Nzzlpiy+YsWBMt37QwngQVojbTV8sW9puyE0fYALXk7/DbfH3Yj79EtZp2JG98YLJ9pktWZeBkWAd9I1/crAH+nqmGHquM4ZwWnbRBU9QXgslnUxXGcBuNuR8dxMtwgOI6T4QbBcZwMNwiO42TUNdpxYnyc3c/vCsqODfSb9ZYtCLvZFuZt1+Jws31qMm++KctHXF8tErafi3ptV1THXPtYHV09pkya7ajLpg7b7bj03HOD5a2ttju1VLTdbPlIGwu2b7Q4Ht7nhWpndK2YKXhBJ2x3WrFgywoadgXu2xaJrBywQ3PGI7fQEzk76rK72XY7dhTCCWllwr4uE0PhCFstR9bwrAEfITiOk+EGwXGcDDcIjuNkuEFwHCfDDYLjOBl19TIUx4sc2hPOj/ji7l1mvfPOXxwsX77AzmHX2mUvM9bUu8iud649u9/WFg6mKrfOMeuo4ZkAKEfiUPI5u97EuL2UW64pPJvd3GbPgHf12F6SQmSZNHthPltUjsyCR/ZGQe2uWszbDXniSDhX57wu21NTGj5kynJlOzirqcXuOzFvTaEY3md+8JhZZ+ePw0vKjY/YXrJa8BGC4zgZbhAcx8lwg+A4ToYbBMdxMtwgOI6T4QbBcZyM+rodJybYu2dPUDY+Zuf127g37DpatvJqs87atbaseYGdg2+sGM7fCDAwEQ6UKY3ZufQqZTsAqFSyj1WIuKm0Yru+SpXw8U6cGDDrHDiwzZQVI+0xYbQHwJw5PcHyjg47D+bRI3ZezeK47VosF2yH5by2sKx/cNCsI5FAqi4jvydAa6XflOUm7HtvqdUIfIq4OB975JFg+ZAR9FQrPkJwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3Ey3CA4jpNRV7djRSuMG3niNOKCW7Li0mB57twLzTrDLXYEYmnUPtZwJKdiqRyuN7fXPlbMNVeciLgPsV1fldg6b7mwu/K73/6OWeX768MuLICu7h5TNmbkTQS49rXXBssvvvhis86jPwhH8AEMj9vXrBLJ03j9la8Oli9ZFs49CVBot/NPtrfZP5njGnEjF+1r1jwSdhWOlez2PWZ1ncoZzqkoIp8TkcMisqWqrFdEviciz6Xvvqyz4/wboJZHhvuAm6aUfRB4UFXXAA+m3x3HOcuZ1iCo6npgaqaGW4H708/3A2+ZXbUcx2kEpzuHsEhVDwKo6kERWWhtKCJ3AncC5At2bnrHcRrPGfcyqOq9qrpOVdfl83Wdw3Qc5xQ5XYNwSESWAKTvh2dPJcdxGsXp3rIfAG4HPp6+f7OmWgqUwm6R1rY2s9pl634qWD6n206kOjZiJyJt6rKTs0rOdmGVJ8K6799/wK4TSSra1Wkn+sxHkrPmI4lPDx0KJwhdv369WefqK68yZStXrTZlR4/ZSUAXLw4nsl24yF7a7robbzBlBSN5LEDZiPAEKOTD7VguLbDrLHuVKavkbD0KJTtit7x/tykbPRTuP2XDzQ1w5NiJYHnJSNhaK7W4Hb8IPAZcICL7ROQOEkPwRhF5Dnhj+t1xnLOcaUcIqnqbIXr9LOviOE6D8b8uO46T4QbBcZwMNwiO42S4QXAcJ6Ou/xRSrVAshqP/5i9aatYrNIWjz8aMyEmIu/RUI4sqiu127DNcen2H7bUArehDgPPXnG/KCh32Woz5nB2N99QTG4PloyN2Wy1aYkf+PRKJQNy0cZMpu/nmm4PlI0W77fcdivydJXLJIiLKRpLbYqQPtDTb7Tuvw+5Xne32z2luhx0Ru8+IGm0es69ZodAcFkT6by34CMFxnAw3CI7jZLhBcBwnww2C4zgZbhAcx8lwg+A4TkbdExRUjMi0BQvs6LOikWCz0GmvE9jcbLhlIjoAaGQtRjUi7trn2Skl25vsJm6KyFRs15eobcf7j4ej4Ew3FTA4YK8H+OILL5qyE5FoRzHauKXF1iMXWc+yr++4KRsethPjjo2PBcubI9G1CyPXc/5C2304v8d2SY6P2td6+4lwhOKCSJLVtnnhaNJcxM1dCz5CcBwnww2C4zgZbhAcx8lwg+A4ToYbBMdxMurqZZBcjtbW1qBs5aqVZr1cPhywEUk7SCWypFXOyLOXCO2gl2VtYVm3DJp1NOLRGFV7xnoYO+9jPhLOs3jJkmD5tu3bzTqx8KDhQfvcKiW73kQxPEM+2B/2ggD0HxswZdu2PW/KikU79+D4WNjLkGu2cyMOL7Cvy/md9iz+sNj5DEfK9vFKGva8jE30m3WOHQsH1BUjeRhrwUcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3Ey6ut2BHLGUmnz5/eY9RYtDi//NTRiu6kQe38ltV2SFY3kaTwWXo6rbfcGs05RbNfc6Fo7oGuiObLMm9p587ZtD7vniiXb/Zlvtu8LFex6pUj3OdAXzo/Y0mO7U49EApiO9/ebskpk+TKphNtfIq7KwWb7mh09buvRNxDpj932Enb51nAgW2HCvs5HBsNLFZYiwXm1UMtSbp8TkcMisqWq7B4R2S8iG9PXLTPSwnGcVwS1PDLcB9wUKP8TVV2bvr41u2o5jtMIpjUIqroesAPfHcf5N8NMJhXvEpFN6SOFmVFCRO4UkQ0isqFcntlS1Y7jnFlO1yB8GlgNrAUOAp+wNlTVe1V1naquy+frnqDJcZxT4LQMgqoeUtWyqlaAzwBXza5ajuM0gtO6ZYvIElU9mH59K7Altn01Vs63jsgSWXPmdAfLh0ZtN0+lbLsWY8tdxVbCEsMlWRqzp1jyzXbuPiYiEZkVWxGNuDJHRsPuqIkJ282WL9iReL0LbdfoyICtfxNh/YdG7PyHQ6N2bsfubjt/ZouR6xKgzViWbWQivKQgQGeXfawDg/Zjr0bc2TJx1JQNjY4Gy5dE+kDeuJyRrlET0xoEEfkicAMwX0T2AXcDN4jIWpK42V3Ar8xMDcdxXglMaxBU9bZA8WfPgC6O4zQY/+uy4zgZbhAcx8lwg+A4ToYbBMdxMur6T6HWthYuunh5UNYciTAraTiCKx/JsqoV2z3UJHaizEjeUCZaw8k32xatNuuU87ZLrCyRZeMiUYYa8Y22tISPF1vhq60tnPgW4A2vv8GU7Vmxz5R1doTdrXv2HjDr7H5xrylriSSCLTXb+g9ouN7EsJ08dqDVdoEfyB00ZZGV6Ghrs4VrOsJJVguRiNfujnD/zjf5Um6O48wSbhAcx8lwg+A4ToYbBMdxMtwgOI6T4QbBcZyMurodm5tyLF3UHpR1N9tutqZSODJNIj7C8UgSzZjbrlSy3ZUnOs4Jlh9daSfQbIqEn2nejqprztnuynze3md3dziJ6cJI1GJvr5nfhjmd4UhTgHIkWrNSCbf/DWuuNuusOS+SiHTEjhZsbZtnykrtvcHyCexkuoVI9OeCrnD/BeiVcNQiQHPEG7ivFN7nU9//Z7POaC58Ly/HwnVrwEcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhORl29DLm80N4TDkSZM8eeve3qCgfKHO231R8v2zPxE6P2DHMx4p0QCc8+l8TWY8yUQC6yJFtupN+UFSIBU73zwjPu3d22t2Dx4sWmrKPNPtb8BeFgL4C+o33B8pUrwsFtAFdeeoEpG3lusynLz7H1ONgU9ro892J4WT6AcmRpuEohHIgEMDAaudoTtgdi1Ai0WrJ4oVmn2Bbu382P2NerFnyE4DhOhhsEx3Ey3CA4jpPhBsFxnAw3CI7jZLhBcBwno5aVm84DPg8sBirAvar6KRHpBb4MrCBZvekdqno8tq98SzNd5y8LysaNJbcAjh8Mu7COD9p58U707TJlHZEcfO0ddsBRU1tYx0KL7YrK5ewmzhsBKoCxEFqCRqQthi6HDx8x66xf/6gpW7F8kSkrNNtt1d4ebuMTW58w6/zDD+wVAfcM2a7iof5+U3bsRHh5uIHIknKVyCrlGrmHdkfcn3M77SX9VhsuxMtuutKsU+gNX+fWv7d/R7VQywihBPymql4EXAO8T0QuBj4IPKiqa4AH0++O45zFTGsQVPWgqj6Vfh4EtgJLgVuB+9PN7gfecoZ0dBynTpzSHIKIrAAuBx4HFk2uAJ2+23+rchznrKBmgyAincDXgF9TVXsd9pfXu1NENojIhuHB8FLljuO8MqjJIEjyJ/6vAV9Q1a+nxYdEZEkqXwIcDtVV1XtVdZ2qruuIZJtxHKfxTGsQRERIln/fqqqfrBI9ANyefr4d+Obsq+c4Tj2pJdrxWuDdwGYR2ZiWfQj4OPAVEbkD2AO8fbodqQjaFI7GKquddG7CyJ340n57WbB//dZ3TFlTJMqw0GQ3Sa417OrpMPIYAvTM6TFlvb3hfH8Aixfb7r6Ya3TjExuC5YOG+w2gs8V2VQ3Msd1l5SF76bV5S8Pu5Zbjtmf66PPPmrJjeTvfYrkYzrkJ0JoPX8/OeZH8jbE+EHMVR2SFyNJ8BQm7OSeMXKIABZ1ZVKO53+k2UNXvY7vFXz+76jiO00j8n4qO42S4QXAcJ8MNguM4GW4QHMfJcIPgOE5GXZOsCtAk4eW/8nl7WbDeuWG33tionbhyzjzbpVccsusNDdruORkPR9zt3LHTrNPWZrvtmgwXLEA+Z7tG8wVbNmjoX67YdY4cOmTKFre9zpSt6rKjPAf7w0uvjUdcvlakJkAztltamuw/vFUqYZd1RW03IBFZuWjLSsXI0nZlO3nv6PJwlOSRiDuV/nBC1/FIpGYt+AjBcZwMNwiO42S4QXAcJ8MNguM4GW4QHMfJcIPgOE5Gnd2OFXIaTpJy6NgLdr3xcDReT6edLLVn7lxTNhhZi1GN9RsBuowow/7+WL4Y2+aWI+tPDg3ZSUBz2PXMpK55223XH0lc89VvP2bKeiK3EzUSji5fbF+X40VbxxPDdpRksWy7AicMV2BZ7TYsFiNrNFbsYzVFXMUXXXi+KVv+6tXB8r6JfrNOyegD5Yh+teAjBMdxMtwgOI6T4QbBcZwMNwiO42S4QXAcJ6OuXobWfBMXdJ4TFkZm3Nt7wjPWYwfs2fHWSy41Zc9u22HKynMjwTdWrr0d28w6pZIdbJLkr7Vktq3OR2RIeKZ+Trcd7NXcYgdgHT16wpQdEXt5tdyR/mB5S8UO8nnb295qyv7iL+8zZfv2HzRlFeOe19phB0StucD2CLS32Z6t7c/aOSGPHg8HewEcOxZe0qTJviwUjGY0YgdrxkcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3EypnU7ish5wOeBxUAFuFdVPyUi9wC/DBxJN/2Qqn4rtq/WXBMXdIeXKBsbs/PHjY6Eg00KE3bAyzmLwkuJATxjewlZuXqFKetoCQc+/fCRR8w65UjgTS4ScJQzF8sCMVyLAE3N4UCwQsG2/StX2W1FZAmyzc9uMmWtRu7E8WH7Om959jlTNjwSWzncdlmr4ebMqe3+7Gyx27c8Ybta1VhyEODHP3ralA0ZLtpf/8CvmHU6OsI/3QdavmfWqYVa/odQAn5TVZ8SkS7gSRGZPOqfqOofz0gDx3FeMdSytuNB4GD6eVBEtgJLz7RijuPUn1OaQxCRFcDlwONp0V0isklEPicidqC74zhnBTUbBBHpBL4G/JqqDgCfBlYDa0lGEJ8w6t0pIhtEZMOxE7FEIo7jNJqaDIKINJEYgy+o6tcBVPWQqpZVtQJ8BrgqVFdV71XVdaq6rndO92zp7TjOGWBagyBJBM5nga2q+smq8iVVm70V2DL76jmOU09q8TJcC7wb2CwiG9OyDwG3ichaEp/PLsD2kaRUUEaN6L/ByLJsx/sN96LtHWLrFtsl9uKO7aasqWAvJ9ZjjHDKJTvEzHYeQi4S7UgkolEj+QCtXXZ1hfNBAoyM2MvXrVy5wpQdPGBHUL6070CwfHvksXHLc8+bspiLViP3NSuidGzYdh8+8dhTpqwcWZJtTo89An7djdeastdcFo7MXbT8PLNO3kqd2WT331qoxcvwfcL9OvqfA8dxzj78n4qO42S4QXAcJ8MNguM4GW4QHMfJcIPgOE5GXZOsliswOB52Aw2M2a600Uo4yrCtzXbzLOi0l+O6fLUdinHg4C5TtuOZsMusXD69zJbFYiRyzm4OchHXqBWb2NnZadbZtetFUzY6ZkcZdvf0mLKjR/qC5e1z7CSlixaHI2EBBgZsd+VLLx0yZaNjVj+IuCoj/uxyJEnsuecuMWV3/PLtpqy1LRyhWoq4OEtG0GjMJV0LPkJwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3Ey3CA4jpNRV7cjQMWI/xMrfAvIGbL2Nnt9vnOW2i6g3l47udPyY8dM2eHD4fX5+lcsNuuMRqI4JyIJO4eHbXffaNF2LRUrRrij2q7RCy+62JQdHxg0ZfsP2u6+iXI4qvXG615r1lm+zHYHHz9mr404MmInbn3iiSeD5bt37TPrKHZbtbTZLt9LL73QlJUn7H4wMDocLC/kbPdnwYiG1UhS31rwEYLjOBluEBzHyXCD4DhOhhsEx3Ey3CA4jpPhBsFxnIy6uh0FaJawy6yUtxOO5lrDrp7Ogh3BJxU7qq5UshOOLp5vR1CWV4XdYhPjtksploB1IhLtODpqR2uODNvH6+8PJ0zdtPOwWWdoqM2UTYzbbqyB/hOmTCrh6/zjDXYC02c3bjZl7R22jh2d9vVsbQp38eZCJFGt2ue8JpL4dGGPrcfB3XYC2fb2cL32VrsPkzN+upWwu7dWfITgOE6GGwTHcTLcIDiOk+EGwXGcDDcIjuNkTOtlEJFWYD3Qkm7/VVW9W0R6gS8DK0iWcnuHqhprriXkqNBGeIa8qSkyO9pk2K2IJ6FiLBkHUIzISkU7oKRUDjdXsTWyplwEa5mxRI+I/uN2rr2RoXCgTEe7HQi2cYcd6FMct4OsFnWEcwECZjCVDtueCdvnAuVBu61Gm2xZc0u477xmtZ2/sbvD9l4tWjjPlBWKtvenqWR7Sdok7EVrtTwJQIvxm4itDlgLtYwQxoEbVfUykqXfbxKRa4APAg+q6hrgwfS74zhnMdMaBE2YdG43pS8FbgXuT8vvB95yJhR0HKd+1DSHICL5dOXnw8D3VPVxYJGqHgRI3xeeMS0dx6kLNRkEVS2r6lrgXOAqEQmvXx1ARO4UkQ0isuH4gL3suOM4jeeUvAyq2g88DNwEHBKRJQDpe/C/sap6r6quU9V1c7vtyRrHcRrPtAZBRBaISE/6uQ14A7ANeACYXI7mduCbZ0hHx3HqRC3BTUuA+0UkT2JAvqKq/09EHgO+IiJ3AHuAt0+7JwErhqnQEl6uDUimMEMYATQAsRCPQs62g9pk61Ey8gQWi3YzipH7DkAjQTS02OdWabUDpuZ0hUdh8yPusksvWmHKBiOPeWPjtqPQWt6uVIosT1a0zyvmoi0U7DbO58Oy5mY7N2I+0gdaW21Xa0uLLbMCmABaDVlzZH+W/gXjfGtlWoOgqpuAywPlR4HXz+jojuO8ovB/KjqOk+EGwXGcDDcIjuNkuEFwHCfDDYLjOBmiaru3Zv1gIkeA3enX+UBf3Q5u43qcjOtxMmebHstVdcHpHqSuBuGkA4tsUNV1DTm46+F6uB5B/JHBcZwMNwiO42Q00iDc28BjV+N6nIzrcTL/rvRo2ByC4zivPPyRwXGcDDcIjuNkNMQgiMhNIrJdRJ4XkYYlZxWRXSKyWUQ2isiGOh73cyJyWES2VJX1isj3ROS59H1ug/S4R0T2p22yUURuqYMe54nIQyKyVUSeEZEPpOV1bZOIHnVtExFpFZEficjTqR6/l5af+fZQ1bq+gDywE1gFNANPAxfXW49Ul13A/AYc93rgCmBLVdkfAR9MP38Q+N8N0uMe4Lfq3B5LgCvSz13ADuDierdJRI+6tgnJusid6ecm4HHgmnq0RyNGCFcBz6vqC6o6AXyJJIPzvxtUdT1wbEpx3bNYG3rUHVU9qKpPpZ8Hga3AUurcJhE96oomNCTTeSMMwlJgb9X3fTSg0VMU+K6IPCkidzZIh0leSVms7xKRTekjxRl/dKlGRFaQJORpaGbvKXpAndukUZnOG2EQQrmwGuX7vFZVrwBuBt4nItc3SI9XEp8GVpMsynMQ+ES9DiwincDXgF9T1YF6HbcGPereJjqDTOczoREGYR9wXtX3c4EDDdADVT2Qvh8GvkHyONMoaspifaZR1UNpZ6wAn6FObSIiTSQ/wi+o6tfT4rq3SUiPRrVJeux+TjHT+UxohEF4AlgjIitFpBl4J0kG57oiIh0i0jX5GXgTsCVe64zyishiPdnhUt5KHdpEkgyqnwW2quonq0R1bRNLj3q3SUMznddr5nTKLOotJDO4O4EPN0iHVSQejqeBZ+qpB/BFkqFnkWTEdAcwj2SNzOfS994G6fE3wGZgU9oBl9RBj+tIHhs3ARvT1y31bpOIHnVtE+A1wI/T420BfjctP+Pt4X9ddhwnw/+p6DhOhhsEx3Ey3CA4jpPhBsFxnAw3CI7jZLhBcBwnww2C4zgZ/x9pJMz6xIk8qwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQIAAAEICAYAAAC01Po2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAoYklEQVR4nO2de5TlV1XnP/s+6t56dlV1Vb+7U+mkE0gYEpg2BkFEQQhxKbAcGAkLg4NGZ2SpI44izpJkBhQV8THDsCYIEpGnAsKMCGI0YpRXg52kQ0jS6XT6Xd31ft9b9949f/xOmdvl2acq1V23OsP+rFWr7j37nt9v/87vd/fv/M737nNEVXEc5zub3EY74DjOxuOBwHEcDwSO43ggcBwHDwSO4+CBwHEcnqaBQESOishLwuu3isgfrXE7D4rIiy6mb09x/28QkXsT9r8SkVtb6ZPzHYqqPu3+gKPAS55inQ8Cb99o35f59Abg3nXadj/waWAWeAK4JfHZEvB7wClgHPhfQLHJ/kzgb4FJ4DDwqmX1fzKUzwCfB3Y02XqBu4Cz4e/2ZXWvB/4hbPsE8OvL7LcE/2eBvwD6l9lfAnwz2I8Dr2my/UCwTQFHgNuabAK8HTgZ9n0PcO1q22+FY74dWAy2pb+9TfYh4O+AOeDbzddy8OvXgGPB748BPU32dwGPAtOh7o8v88s85uT1cgl8GQprqOOBYOVtfxT4ONAFvCBc7Ncan31b+DL2A4PAV4A7ls4P8Ajwi0A+XGizwFXB/n3hC34t0Aa8F/j7pm3/MfBnQEf4AjwG/EST/VvAO8K2rwBOAz8SbNeGC/6F4Tg+Anysqe41Yd8vD35uBq4ItmI45p8OX67vCl/I64L9NWSBb2/Y928C31xN+63imG8H/jRxbr4MvBtoB34UmAAGg+3W8AXfHfb9GeCuprp3AM8g681/N1ng/p7VHHPyelmni/Ao8KvhJI+Hi6EcbC8ii/y/ApwBPhQO6i3hIhkFPkFT5AdeTxaVR8mi5VFCIFje6OGk/VNo3ONkX7bbyCJ0NTTM/2nyc2k7JeD3w8VxKrwuLfP5zeECOE3TxbzKL/wRsov6ceB1zYGALMqPB9vLm+rdA/xk02f/Efgf4WR/G3ixsb/OcKxXNZV9CHin8fkDwKub3t8CHA+vnxXaTJrsfw389/D6XcB7mmw7AOXJL+QI8F1N9rcC/9D0fg64pun9nwG/Gl7/BvCRJtsV4bi6w/uPLPkROaatwY+OprKvA68Nr38F+EST7VpgYTXtt4pjvh0jEABXAZWlYwhl/wD8THj958B/abJ9D7DQfBzLtvdZ4M2rOebU33qOEbwOeBnZybsK+K9Ntm1kd5/LyL6kPwe8kizS7iD7UrwHQESuIYu4rw+2zcCu2A5FZA/wV2RflkGybudBVb0T+DDw26rapao/HKn+a8CNoc51wA0RnzcBO4E3Au8Rkb6w31tE5H7Dp07gD8m+4N1kJ/Zg00e+G3gYGAB+G3i/iEhsW+GzR8Jn3wZ8SkT6I5+7Cqir6iNNZfeRXexRN8Nf8/tdIrJpWXmz/VmJujTZidibbb8P/LiIFEXkauB5wN8E27XBbwBU9THCFzQU3QggIg+IyGkR+dOl9lDVYbK7+k+ISF5Enkd2vS2NyXwMuFJErhKRItmd+PPBtlL7reaYf1hExsI41H9sKr8WOKKq009h2yVgH8sQkXayu/6Dqzxmm3XsEfxM0/ubgcea7q5VQg8hlD1E090N2E52By8Av8753cGlaP2vegRkvZBPGz59kGWPBpzfI3gMuLnJ9jLgaJPP8zQ9xpD1DG5cRVt0kvVOfhRoX2Z7A3C46X0HWUTfFt7fw/k9glOcf2f+GvD6yD6/FzizrOyngHsMH99O1tsYJAt4Xw1+bCfrbh4Bfjm8fmlo/y+Eui8mu+s/m6yr+7+BBk/eef8U+BTQDVwZ2rmy7I53GKiFfd7RZLu7+ToKZSeBF4XX1XAOryLrRn8S+HDTZ38YGA7brgE/1WRrA/4g7LNG1hu7fDXtt4pjvobsppUPx3e6yfZ64CvLtv0O4IPh9U+SPYoNkd14Pht8fF7kvN1FFrxkNce8UT2C402vnwgNs8Q5VV1oen8Z8GkRmRCRCbLAUCfr6uxo3paqzpI9IsTYTXahrYUdwU/L51FVrTW9nyO7+JIEf/898DPAaRH5SxF5RtNHzjR9di68tLZ7UsPZNnxcYgboWVbWQ/ZoEuMdwD+T9VT+iWxQbhE4q6qLZL21Hwq+vpns0e1E8Plust7JJ4M/R8N+ToRt/xxZEH2U7Hn3o0u2cPf+PPDfgDLZ+XuZiPynVR7HPPDHqvqIqs6QPUrcHLb9DLJn/B8n+9JfC/yyiPxQqPs2srvp7rDvO4C/FZGOlfa70jGr6rdU9ZSq1lX1n8gCzr9b5TF9ILTRPWR3+r8L5SeaK4jI75D1QF6zdE2s4phN1jMQ7G56vYfsbraELvvscbKuc2/TX1lVT5JF03/ZVjhRm419Hid7FImxfJ/LOUUWkCyf14yqfkFVf5DsDvtt4H1r3NTOZY8Nlo+PAAURae5OXkfoQkb8m1fVN6nqTlXdSxZov6Gq9WC/X1W/T1U3q+rLyAbYvtZU/z2quk9Vt5B9OQrAoWAbU9XXqeo2Vb2W7JpbqruXrAv+J6paU9UTZF32m4P9weA3ACKyl6ybvNRlvx/7vD4LeDi0fUNVHwb+kmxgcak9Pq6qJ8K+Pwj0kd3NV2y/1DHHmpgnu/sPAntFpDu27eDr21R1SFV3hfKT4W+pHe4Ix/FSVZ16Csdss5puw1P9I4uQD5A9y/eTDYb8hj7ZzT6x7PP/mSwCXhbeDwKvCK+vJYuiLyCLcu8i6/LEHg32kEXW1/DkKPL1wfZOmgaemvxc2s7bye6Gg2TP4PcSHiUMn/+l7gptsRX4EbJHhBzZnWepi/kGlqkGZBfNlRp/NKgBP0/WRX81mUS02djvx8juLJ3A80mrBjvJehZC9tx9nOwiW7I/m+yu2QH8Elk3emkgtUx2AUpo/3uWznWwXxHOQ57sghzhydH3HrLHpltC22wjG1F/R9O5nyLrqneSPWY0Pyb+h+DL3uDbJ4APNe13hkzlkPD+MKGrTHZHvzecnxxZl30W6F2p/VZxzK8gCypCNtZ0Eri1yf4Vsuu4DLyK81WD/uCrkAWlQ5wve/4qWe9qe+Q8Jo85eZ2uYyBYUg0myJ5lOhJfqhyZPPUw2Rf5sWUNeyuZrroa1eB7yZ5xp8gu6FtD+T6yru8E8BeRQFAmG9Q7Hf7+kGVKRyKIvA540GiL7cDfhwtpIlw016wxEPwj8D/Dth6h6csa2W8/WRd/NrTdLU22PeGC2RPevzAcz1w4B69btq3fIRvAnSEbjL2yydZLdmeeJXt0+E0g32RfkunmQvu/bNm2f4BsZHsy1H8f54963xL8nyV7tFj+O4I7gHPh70NA37J9H+LJbvtvAbmm8/2ecK6nyLT3m1bZfisd80fJrtUZsh7gzy3zeSic2/nQ3s2/I7gqlM2RPXb8YuT6qHD+bxTeuppjTv1JqHxREZGjZBfw36z0WWd1iMgbyNr0BRvti/P/H0/Lnxg7jnNx8UDgOM76PBo4jvP0wnsEjuNQaOXOBgYGdOiyoaitYf2olkS0SnVmEttLUa/ZNjUcySXCaa6R2F7Cf8mnKiZ2uJbjTviRbOLUviz3W3zraRh+5FK+J2yLVfu81Ou2rdj21L9q+YStTj1afuzYMUZHR5/yVXBBgUBEbiL71VQe+CNVfWfq80OXDXHgyweitpk2u575871KYmepI0u08ORYvIEB6qV4xWLZ3l73nG1bTHzLil0zpq2xaP+gUUpx/xuJg84vmCaqxgUH0FZONKTl/oq/xby4zMwabqS+ZYnzefqEsUFgfMxuyO1D1m/gwIr5PYmgOdWYiJZ/3/d/v10pwZrjs4jkyXTYl5P98OG1IUHIcZynGRfSUbuBLGHmiKpWyX6J9YqL45bjOK3kQgLBTs5PLDoRys5DRG4TkQMicuDcyLkL2J3jOOvFhQSC2IDEv3rqVdU7VXW/qu4fHBi8gN05jrNeXEggOMH5GYa7uEjZeo7jtJYLUQ2+DuwTkcvJsqt+jCxBxEQbdRYXxqO2rnyfWe+RJ+JD79vbOuyd9SRG3XP2SHitZuuH1apRr3OL7UfRNqVkx3rOHl6fS9SrGupA3m4OpGrb2g2lBLIMKhPD/YQoumKeuFkvoXrkDPcXEqrBbOIJ9lx9zLSNN+LXNsCWmq0a9MXmmFqB8lRvtDyXFB1t1hwIVLUmIm8CvkAmyH1AVaP57o7jXNpc0O8IVPVzwOcuki+O42wQ/hNjx3E8EDiO44HAcRw8EDiOQ4uzD0VyFHNxya8yO2HW616M61szYsexckKnmpuYMm3VmUnbZsiHvQU7S6Xe1m3aJot2klhKZkvkZ6GGFFhKJPvMW5Ock02cZzJpy7CLm+IyVoetsLHQbuuAjXnbk5La+psajVVIJazlbD01N2+3fqFgX49zM7Y4OlmIXwebEnrqvCG2pq6bFN4jcBzHA4HjOB4IHMfBA4HjOHggcByHFqsGGfHRzlpilrWZSjwRqK9kTxs1OWKPQI+dsZMk2/OLpk2qccXjWPUBs05t0E5IKg/YtscePWvarujbbtoKbXF5oJyYi7GQsBUTt4rJhq0aLIzGs5yqE/a49rYtduLZlDX8D+TF9sO6sIYT6lCxZGeK9SSytyZP2ZLI6OwR09a+x1qu02ZTNX5c+TXe2r1H4DiOBwLHcTwQOI6DBwLHcfBA4DgOHggcx6HV8qHYeyzM2zGptxaXAo/fd9KsU0vMFdjTYWecTEycMW1zxpJW+VF7Z0dHbTny8v09pk3P2RLn5EDJtPVq3Jdzs7ZGKNJp2hpnhk1btduWaBc64olYC9O25LvtMrs9+rrtufgai/axTS7EE4jarOwsoDpmn7PR4SdMW1/DzhIaO22fz3P3b4pvb9BOppoydMJaQklN4T0Cx3E8EDiO44HAcRw8EDiOgwcCx3HwQOA4Di2WDxdrdU6OxyfIOzNsZ9t1alyWmWobNeuU8/ahVeftVMeOhASkhjSH2HXaxu2MtPHj95u23lxcUgIoPH7ctD0xcThanq/YkmlnwZapKp22XCaJOQsLtbj/C+O2H8fOPGT7kZDF8mX7XNc74tLiuWlbcqwn9rW5ZGdBNhq2rNuWt+XKkfGHo+WVwlazztW7r4yWp5bRS3FBgUBEjgLTQB2oqer+C9me4zgbw8XoEXy/qo5chO04jrNB+BiB4zgXHAgU+GsR+YaI3Bb7gIjcJiIHROTA6Kj9TO84zsZxoYHg+ar6XODlwM+KyAuXf0BV71TV/aq6f/Nme414x3E2jgsKBKp6Kvw/C3wauOFiOOU4TmtZ82ChZClrOVWdDq9fCvy3VJ2FSpWHDelret5+bGgvxzPZSjlbBhSxbVNiT6DZ3t1r2tqq8WW3xhdtvak2YPsxNmov41Wu27b5BVuCK2l8ibX2PnvptVze3p7Wd5k2Sci3tdl4lmGhzc50HJmypdbqrJ0R2LnJXs+toy0uYxYatnzYSHwrxnL2tZObsyXCfCItcOds3FatJh6l+40JT9eYfXghqsFW4NPhC1cAPqKqn7+A7TmOs0GsORCo6hHguovoi+M4G4TLh47jeCBwHMcDgeM4eCBwHIcWZx/WanXODU9EbeMSz0oE2FeMSz1tXbaUo3k7I7Cjw5bSSvNTpm3RkNl6e+yMtLaGPbFpaUtcFgVoH7Nj9OXdO0xbz9C2aHmx3/Zj7JgtpU0s2hPEDvT1mrbG7riO1bVoy6mKfVzYp4Xxun2u60YqYYfYx9xZsaXKuRlbnys37Ouq3mZvc7Yat1WwJ4ctFY5Gy3NiS8EpvEfgOI4HAsdxPBA4joMHAsdx8EDgOA4tVg0ajRrzc/FEitHpCbPe5aUt8e112CPQtao94qo9O01bcdNAwjYZ92PKHpGvzNgj2v3GslUAnVfYyTn1M/botBAfgc6fsBNiesv2SHOlzVY2ajW7/QcMIaXYaR9zPbGsWWGn3Y6dY7YCcGpyJlpeWrCVHqna21tQWzWYGLC/TjJiH9tuQzXIVU+bdYYfjZcvVuzznMJ7BI7jeCBwHMcDgeM4eCBwHAcPBI7j4IHAcRxaLB/qYoPGcFzW6xJb7hudPBMtL3Q+y6zT073HtOU7bfmtctaWjioT8Tn1CuXEMmk9dhOPV2xprjFpS3PFxMR0I+NxWay8yZY4F7HnR6zlEpLYYxOmbXzbYLS8upiQKotxqQ+gPjtv2qYScl+J+P7mF2yZrZi6P6qdHFc9Z9vaGgmpuxCXFivVHrPOoen4vubrdiJeCu8ROI7jgcBxHA8EjuPggcBxHDwQOI6DBwLHcWh59mGd2YWxqK2jZM/p1tUZzz4sFGx5pVKz5aFyw5bfFrbZUhSz8eW/em1liHlsP6Zr8WXBAGa1ZNouU1siyhfiEtzwqL2c2NEjj5u2cmeHaZt89JRpG5qMZ3gW9+w165w8ec60TU/YmXi5hGS2a0dcRi4Xes06hZotpy6W+kxb22a7Xi6xjlr7RPx6LHXY9+nJWjwTVhLZkSlW7BGIyAdE5KyIHGoq6xeRL4rIo+G/3TqO41zyrObR4IPATcvK3gLcrar7gLvDe8dxnqasGAhU9UvA8v78K4C7wuu7gFdeXLccx2klax0s3KqqpwHC//hDPCAit4nIARE5MDNr/4TUcZyNY91VA1W9U1X3q+r+rk57HXvHcTaOtQaCYRHZDhD+n714LjmO02rWKh9+FrgVeGf4/5nVVFKUxUZcnuvRdrPejt1b49tbsOPYbCIJS+YT2YK2okd9Pp7JVt1ib2/m5Ihp68jZ9RbydjZmYa+dPXnuZDwr7Rtf/UezzpbnXGfatm7aZNqqxqSbAHNd8SzI67bb2+spJiZzfbb59MmZs3a9jr54hqc27MlEZ0v2eclP2/vq6bCl5+mH7Otgrhq/j7YVbDmS04bUmsjETLEa+fCjwJeBq0XkhIi8kSwA/KCIPAr8YHjvOM7TlBV7BKr6WsP04ovsi+M4G4T/xNhxHA8EjuN4IHAcBw8EjuPQ6uxDlAXi8katx5ZsijlDWqzamVa1Hls/rCzYE2i2F+1tymJ8As2Z0ZNmnZlxe3ube+PZjACbdtlZf6J2/J45Fv/15sS87cfV3XbO2Lmj9k9Evn3Izlq88sqro+UjU7YkNjlpS2yPPGGa6C/abTU1G5dh5ybslNFK2ZZuO9rsjFfdZLdx+1bbx9FqPJMwP2lr2bmctXZjIhU2gfcIHMfxQOA4jgcCx3HwQOA4Dh4IHMfBA4HjOLRYPgShnotPylns2GHWmpyNZ5DlSvYEn1vydoZeLWdnnhWLdkbd6HTcVpyxpc/dg52mrR3b/xx2NqYk1tEbnn80Wj6Yt9c+rI3YEuHxGVsa3ZS4fLrL8f3VxW77mTG7HWer9sSm1Xk7o3HmbLytyj2W/Ab5qn1/zBfs7L6Oit0e27ba61z+/YG4j53z9oSzQ1c/M1qeT3wnUniPwHEcDwSO43ggcBwHDwSO4+CBwHEcWqwa5HJ5OtvjSRvbN9nJHGIkquS32CPQZ7umbD+K9ihzbeSEaesuxEdxd4s9kqzTth+18qBpq9qD/DBsj663GwlEp+fsJKwFY4QfYPGoPQX94qKtsCzOGQkzam/v5Ly9rNnYOVs12NRj38/OVeJLzm1u2HXK3XZ79O621ZzFUqIdF+NLwAE0NH591+r2dTVfjSfANRLL4aXwHoHjOB4IHMfxQOA4Dh4IHMfBA4HjOHggcByHFsuHIkqpLT5fYGmTnZTR1hlPIFKx5aui2ElHjZwtsXQO2slPW8fjySHHD99r1qknppDbftU+0zaaqLe5z5aVKo/EZbb+hi3b7e6xE6Pua8Tn0wOYq8alOYCuwdFo+dZ5W0brSSTMPLFgzyPYu9WeK1BH4201XrTbo1y25eX2+cRXJrHc2FgpLvcB5Izl/vKJRLHZSvzab6gtLadYzZJnHxCRsyJyqKnsdhE5KSIHw9/Na9q74ziXBKt5NPggcFOk/PdU9frw97mL65bjOK1kxUCgql8Cxlrgi+M4G8SFDBa+SUTuD48O5sT4InKbiBwQkQMzM/ZzmeM4G8daA8F7gSuA64HTwO9aH1TVO1V1v6ru7+rqWuPuHMdZT9YUCFR1WFXrqtoA3gfccHHdchynlaxJPhSR7aq6lCr2KuBQ6vP/Ug9BanGJKN9pZ3XVS/HMxPEZO7Ove8TOTJTttsSyKyHbFYnLVI1aQhpKSEC5TbYf3XO2FCXt9mmbbsSl0UoukSmYs30sd9pSWrUyYdqG5PJo+UJimbonzg6btvayvWTYpi7bVt4Rb6uxCXs+wJ4eu62mq4nHW01koR6zl3qr1qaj5Xv77CXxZhbj53mN6uHKgUBEPgq8CBgQkRPA24AXicj1gAJHgZ9e2+4dx7kUWDEQqOprI8XvXwdfHMfZIPwnxo7jeCBwHMcDgeM4eCBwHIdWZx/mc5T6DalnLjGZZFdcctrVsOWaRmHEtM2O2Flztc0Dpm14OC4TSle/Waco20xbA1vrWWi3fSyJfdr6uvZEyx+fuM+sUy8aE40C13/vvzFtxx94wrTV2uPZpEdO25OQTk5MmLbtfeaPV6kctpdsO1mJXwdbC3b75kbs7NQT37KlxcRpId9hL7E2cHn82KqNxAbHjetbbXk2hfcIHMfxQOA4jgcCx3HwQOA4Dh4IHMfBA4HjOLR87cMC7e2GPJe3sw/ztbib+YqdNddRtNMIN7XHJ1AFqM8eN21njOyyXO+QWeeKDjsLsjpry4dt7fZknYh93JMSXz+w1GnPBTG7aE8c27fJrjexyc6omz5zKlq+q+sKs07/kC19bdlpT0bbXbD9H5rcEi0/lspc7bS/FtdfZk+wWhq0z3U+Z/s/Mxe3DT/wLbPOpMavgbom0mcTeI/AcRwPBI7jeCBwHAcPBI7j4IHAcRxavuRZg2JbPMGlZOf6UJqNj6DPiz1aXB60D61as5f4yomdcLJty5Fo+cJZO57aKTYwXz1j2iqJqfEGZmwfO4rxBKhtV15m1tm3Y8i09RTsY9uUWO1ibnYi7sdW+0SXB+w5+uY74yoEQKHDnrNw+rL4Ns8dvN+sk7oGdMBWeiaNpDQAOux21Im4gtGWWIrOasVC3lYuUniPwHEcDwSO43ggcBwHDwSO4+CBwHEcPBA4jsPqVjraDfwJsA1oAHeq6h+ISD/wcWCIbLWj16iqvY4U0NZeYvczh6K2BWwJqFSPb/bYrJ04su3rdhJT21Y7aadtsx0bC+Xt0fKeyxPLq4mdPNQtdvJTQxNLlKm9zc6F+PJwk1O2tHX2scdNW3XOnmsvp7bG2dseP5+1k/Y8hwcTy5DN1u3zUq3HlwwDGD4Rl2jHGnYbbtthy9KnTtlfmULJTpqaqtlLx+3a3Bst39EXT5gC6MjFz0uhsH7yYQ14s6o+E7gR+FkRuQZ4C3C3qu4D7g7vHcd5GrJiIFDV06r6zfB6GngI2Am8ArgrfOwu4JXr5KPjOOvMUxojEJEh4DnAV4GtSysih/92P8ZxnEuaVQcCEekCPgn8gqraD+f/ut5tInJARA5MjCWHEBzH2SBWFQhEpEgWBD6sqp8KxcMisj3YtwPRVSZU9U5V3a+q+3v77UUqHMfZOFYMBCIiZMugP6Sq724yfRa4Nby+FfjMxXfPcZxWsJrsw+cDrwceEJGDoeytwDuBT4jIG4FjwKtX2tBiDYbH4rGnd48911oxH5/Tbeaw/YTyN49+wbS1Pzpk2sifME1SiPvRX7Iz0holu4mv3G7PY9fV12v7UbRtdWOuwFzFXuLrTCHR9jlb4szVbImzg7hcNlexUxYnK/b5rIh9z5oanrBt7XEfd/fYvdP2Hvu4KhVbWsy32e24MGm3f1Xj189oT2Leze54+2pxbfLhioFAVe8FLI9evKa9Oo5zSeG/LHQcxwOB4zgeCBzHwQOB4zh4IHAch1ZPXqpKrhLP0JK8LcF1l+PLTNVzdhwrt9vZh+U2O6NuYiwxcaVMRMsnR+3st66e+GSiALpgT21aPGj7UUlIRJVGfBmyctGeNLQ6E59QFmDbNlviHNyy097mXHx5uI4u+7ja52xbIxffHkBHhy2z9QzEszGrVTtTcG7B3leuZmdxjs/ZflQrdjtWFuPHnT+VWPauHL/261W7DVN4j8BxHA8EjuN4IHAcBw8EjuPggcBxHDwQOI5Di+VDVMGYlHP2nD1pyaHZuFxWSqxR19W+x7RtrthrDg6W7YlNK4NxW3XWlpSm52xbbc6WsCbEllPnpmx5q6+zK76vhDw7Xp0wbQcO2xOU9p+ITkEBwAJxyXdwk33JTeTi0ifAVM4+19VEO9aPxrMn27vs9qgY0ifA9Lxdr5hYq/DyyzebNqnH5ey5nL0vycUl6wYuHzqOs0Y8EDiO44HAcRwPBI7j4IHAcRxarBoUcnkG2+LJF41EwkZR4vPEVXfYc8tdvtsewR3/pj2yWi/Zo+TTtXi9WtFO2mkU7JHfc0V7dHpzwU6aKrbb8wjmjXyk4qId84tle/6++Qm73qFpe7S+0BZvK+mzj2vr3itMW/Xxg6YNe5OcGpmMlud746oGQFtiLslnqF1vumJ/nUaOHjFtg8+4OlreYQtY9NXjxoI5q2Aa7xE4juOBwHEcDwSO4+CBwHEcPBA4joMHAsdxWIV8KCK7gT8BtgEN4E5V/QMRuR34KWBp4r23qurnUtvqkBzPLcWTYo4ZchNAZSYus22esud06xuKLwkFMDpky2V9asuO1UpcthsbtpezKuZtqa8tMTdeLZFks5BY/qvRFtcP5xJS5cCA3Vblkj2vYkFtP3K5uAw7MWdLrXLKllonFu15Gmdq9pyRi7Px9i8bUjDAtk57fsdGY4tpGx95yLSdOWu31Wz38Wj58/btN+vQGI0Wyxrlw9X8jqAGvFlVvyki3cA3ROSLwfZ7qvquNe3ZcZxLhtWsfXgaOB1eT4vIQ4A9fa3jOE87ntIYgYgMAc8BvhqK3iQi94vIB0TE7m87jnNJs+pAICJdwCeBX1DVKeC9wBXA9WQ9ht816t0mIgdE5MDIuL0ktuM4G8eqAoGIFMmCwIdV9VMAqjqsqnVVbQDvA26I1VXVO1V1v6ruH+izF/twHGfjWDEQiIgA7wceUtV3N5Vvb/rYq4BDF989x3FawWpUg+cDrwceEJGDoeytwGtF5HpAgaPAT6+0IUVZbMTlnJkJu97cQlwmHE+sCPX414ZN27FjI6ZtYKstv5RL8WzH+YadsbhYtiWxWj2xvFrBlvvIJ6TFhbjM1t5pZ81NJjI/ezdvNW2FcTv7c6Eaz447dcLOwjt91J4Dsa/PTsVbLNp+0Bu/183X7ZTFoydtyffEYwdMW3timb09/3afadvSvTtansoKlcH4Um6SWM4vxWpUg3shKk4mfzPgOM7TB/9loeM4Hggcx/FA4DgOHggcx8EDgeM4tHjy0orWOarxTLFKw876qxlZbsWCLYl19tjy27ZtHaZtevKYaZtYjC/J1dZhS4Sas6WtYtm2jS3Yp6Y9b0tEdeKaqoo9IWd9ys4IPGdkfgL0iy2X1YvxJew6Ntl1pGCfl0oii7OjYMu3Y4YymlP7x23VSiK70z7VdG3ZYdqe9103mjZrXtnGiD05LDUjG1N98lLHcdaIBwLHcTwQOI7jgcBxHDwQOI6DBwLHcWixfFhE2UJcBpov2PJWlzFZpxTsjLT2cnySVIB2bA2osMf2Y3YhPmFkrmpnzVkTrwKM1W3JdEt53rTVFmyJaNHI0uuq25N/Fsq2H6NztmyXiy9JCUA+F5fnhvbvMuts6kxkVS7G5UiAmRlbRj7zxOFo+VjVlubOnLHlyKKxdifANfv2mraCkUELMGNoku3t9nmuTxrXcMNuwxTeI3AcxwOB4zgeCBzHwQOB4zh4IHAcBw8EjuPQYvlQpEhbPj4Z5lDNlqnqbfHsuHrOXpevksgSm9xsr1WY77ez42rEJcnirC3NzbXZ0tzuui0pVRLtwax92uoLcelr5LQtsSWWAaSjYN8r5hP+12bjJ+DwVw6adSoJWbdYto+53GlnY44ZKuHswoRZp1S3s0K3PsOehHQxb8t9I7NTpi1nrFdYT7R9Yy6+PW0kJnJN4D0Cx3E8EDiO44HAcRw8EDiOgwcCx3FYhWogImXgS0ApfP7PVfVtItIPfBwYIlvy7DWqameGAPVcjanO+HJjjUTCSVs5PvRbHrdHi4tt9ghuIpeDWs5ukpF4zhGNsh1Pe3OJkd9E0lFnzVYvFhOrWi0YI9CdA/H5FgGOH7XVF3L2zhax/Z+aiasesw3bj3xCNWhbsFWPmYqtsGzNx5O+tvTacxbqdvsC0elEplViCb56w05yyhsnVDvstir1x5PLcoX1m7OwAvyAql5HtgT6TSJyI/AW4G5V3QfcHd47jvM0ZMVAoBlL4nQx/CnwCuCuUH4X8Mr1cNBxnPVnVWMEIpIPKyGfBb6oql8FtqrqaYDwf8u6eek4zrqyqkCgqnVVvR7YBdwgIs9a7Q5E5DYROSAiB8bGk0MIjuNsEE9JNVDVCeAe4CZgWES2A4T/0Wl6VPVOVd2vqvv7++z13h3H2ThWDAQiMigiveF1O/AS4NvAZ4Fbw8duBT6zTj46jrPOrCbpaDtwl4jkyQLHJ1T1/4rIl4FPiMgbgWPAq1fakABFjAQdSSTuEE+kmN9sSzmLiYQYlXjiE0B90a5XHrCkGTuJqSq2nJMrJJKHpu35GGtiz2fYUY3Ljgmlkj1be03b7Jx9bIOzdlsN9cTPTVt5t1knn5DYGol71pZcYm4/48DzxVRyme1HIWfXm5lJfJ3KiTkju+O23Lx90krGsndifb9WYMVAoKr3A8+JlI8CL17TXh3HuaTwXxY6juOBwHEcDwSO4+CBwHEcPBA4jgOIamJyv4u9M5FzwBPh7QAQT0VsLe7H+bgf5/N08+MyVR18qhtvaSA4b8ciB1R1/4bs3P1wP9yP8/BHA8dxPBA4jrOxgeDODdx3M+7H+bgf5/Md4ceGjRE4jnPp4I8GjuN4IHAcZ4MCgYjcJCIPi8hhEdmwSU9F5KiIPCAiB0XkQAv3+wEROSsih5rK+kXkiyLyaPi/7rO4GH7cLiInQ5scFJGbW+DHbhH5OxF5SEQeFJGfD+UtbZOEHy1tExEpi8jXROS+4McdoXz92kNVW/oH5IHHgL1AG3AfcE2r/Qi+HAUGNmC/LwSeCxxqKvtt4C3h9VuA39ogP24HfqnF7bEdeG543Q08AlzT6jZJ+NHSNiGbuqMrvC4CXwVuXM/22IgewQ3AYVU9oqpV4GNkMyJ/x6CqXwLGlhW3fFZow4+Wo6qnVfWb4fU08BCwkxa3ScKPlqIZLZ05fCMCwU7geNP7E2xAYwcU+GsR+YaI3LZBPixxKc0K/SYRuT88OrR0okkRGSKbCGdDZ8pe5ge0uE1aPXP4RgSC2LxSG6VhPl9Vnwu8HPhZEXnhBvlxKfFe4AqyxWxOA7/bqh2LSBfwSeAXVHWqVftdhR8tbxO9gJnD18JGBIITQPPEdbuAUxvgB6p6Kvw/C3ya7LFlo1jVrNDrjaoOh4uwAbyPFrWJiBTJvnwfVtVPheKWt0nMj41qk7DvCZ7izOFrYSMCwdeBfSJyuYi0AT9GNiNySxGRThHpXnoNvBQ4lK61rlwSs0IvXWiBV9GCNhERAd4PPKSq724ytbRNLD9a3SYbMnN4q0ZCl42K3kw2IvsY8Gsb5MNeMsXiPuDBVvoBfJSsi7lI1kN6I7CZbA3JR8P//g3y40PAA8D94cLb3gI/XkD2eHg/cDD83dzqNkn40dI2AZ4N/HPY3yHg10P5urWH/8TYcRz/ZaHjOB4IHMfBA4HjOHggcBwHDwSO4+CBwHEcPBA4jgP8Pz9uUjKHcOg7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "image_show(image, pred+ \" \" + str(score.item()))\n",
    "image_show(unnormalize(perturbed_image_pgd.detach()), new_pred_pgd + \" \" + str(score_pgd.item()))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As seen above, the perturbed input is classified as a ship, confirming the targetted attack was successful. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Robustness Metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Attack Comparisons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to adversarial attacks, we have developed an AttackComparator, which allows quantifying model performance against any set of perturbations or attacks, including custom transformations.\n",
    "\n",
    "In this section, we will use the AttackComparator to measure how this model performs against the FGSM / PGD attacks described above as well as torchvision transforms. Note that the attack comparator can be used with any perturbation or attack functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will first define the desired metric function, which calculates the desired metrics we would like to evaluate and compare for different attacks. The metric function takes the model output as well as any other arguments necessary, such as the target label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import collections\n",
    "\n",
    "ModelResult = collections.namedtuple('ModelResults', 'accuracy logit softmax')\n",
    "\n",
    "def metric(model_out, target):\n",
    "    if isinstance(target, int):\n",
    "        target = torch.tensor([target])\n",
    "    reshaped_target = target.reshape(len(target), 1)\n",
    "    logit = torch.gather(model_out, 1, reshaped_target).detach()\n",
    "    _, pred = torch.max(model_out, dim=1)\n",
    "    acc = (pred == target).float()\n",
    "    softmax_score =torch.gather(softmax(model_out), 1, reshaped_target).detach()\n",
    "    return ModelResult(accuracy=acc, logit=logit, softmax=softmax_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now import and initialize AttackComparator with the model, metric, and preprocessing functions. We then add the desired perturbations we want to evaluate, including FGSM, random rotations, and Gaussian blur."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from captum.robust import AttackComparator\n",
    "\n",
    "comparator = AttackComparator(forward_func=net, metric=metric, preproc_fn=normalize)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "comparator.add_attack(transforms.RandomRotation(degrees=30), \"Random Rotation\", num_attempts=100)\n",
    "comparator.add_attack(transforms.GaussianBlur(kernel_size=3), \"Gaussian Blur\", num_attempts=1)\n",
    "comparator.add_attack(fgsm, \"FGSM\", attack_kwargs={\"epsilon\": 0.15}, \n",
    "                      apply_before_preproc=False, additional_attack_arg_names=[\"target\"], num_attempts=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now run the comparison for the truck image we looked at previously."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Original': ModelResults(accuracy=tensor([1.]), logit=tensor([[13.0536]]), softmax=tensor([[0.9997]])),\n",
       " 'Random Rotation': {'mean': ModelResults(accuracy=tensor([0.9900]), logit=tensor([[8.2371]]), softmax=tensor([[0.8830]])),\n",
       "  'max': ModelResults(accuracy=tensor([1.]), logit=tensor([[13.3560]]), softmax=tensor([[0.9998]])),\n",
       "  'min': ModelResults(accuracy=tensor([0.]), logit=tensor([[1.0243]]), softmax=tensor([[0.1925]]))},\n",
       " 'Gaussian Blur': {'mean': ModelResults(accuracy=tensor([1.]), logit=tensor([[12.7143]]), softmax=tensor([[0.9996]]))},\n",
       " 'FGSM': {'mean': ModelResults(accuracy=tensor([0.]), logit=tensor([[-0.0665]]), softmax=tensor([[0.0424]]))}}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comparator.evaluate(image, target=label) # perturbations_per_eval can be set to > 1 for improved performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From these results, we see that random rotations generally lead to a correct prediction, but the worst-case rotation still led to a misclassified result as well as the FGSM attack."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The comparator also allows us to aggregate results over a series of batches. We start by resetting the stored metrics from this example, and evaluate a series of batches from the test dataset. Once complete, we can look at the summary returned by the Attack Comparator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "comparator.reset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_batches = 100\n",
    "for i, (batch, batch_label) in enumerate(testloader):\n",
    "    if i > n_batches:\n",
    "        break\n",
    "    comparator.evaluate(batch, target=batch_label, perturbations_per_eval=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Original': {'mean': ModelResults(accuracy=tensor(0.5723), logit=tensor([3.7854]), softmax=tensor([0.4603]))},\n",
       " 'Random Rotation Mean Attempt': {'mean': ModelResults(accuracy=tensor(0.4861), logit=tensor([2.8618]), softmax=tensor([0.3841]))},\n",
       " 'Random Rotation Max Attempt': {'mean': ModelResults(accuracy=tensor(0.8752), logit=tensor([4.8613]), softmax=tensor([0.6419]))},\n",
       " 'Random Rotation Min Attempt': {'mean': ModelResults(accuracy=tensor(0.0752), logit=tensor([0.6963]), softmax=tensor([0.1161]))},\n",
       " 'Gaussian Blur': {'mean': ModelResults(accuracy=tensor(0.5208), logit=tensor([3.0006]), softmax=tensor([0.3945]))},\n",
       " 'FGSM': {'mean': ModelResults(accuracy=tensor(0.0297), logit=tensor([-0.8150]), softmax=tensor([0.0403]))}}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comparator.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Minimal Perturbation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Next, we would like to find the minimal perturbation that causes misclassification, e.g. what is the minimum blurring of an image that causes it to be misclassified? What does this counterfactual image look like?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to do this, we first implement a Gaussian blur function parametrized by the kernel size, using the corresponding torchvision transform."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gaussian_blur(image, kernel_size):\n",
    "    blur = transforms.GaussianBlur(kernel_size)\n",
    "    return blur(image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now import and construct the MinimalPerturbation object, providing the model, attack, and parameter to test. We provide the range of kernel sizes between 1 and 31, setting the step to 2 to include only odd numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from captum.robust import MinParamPerturbation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# By default MinimalPerturbation compares the argmax index of the model output with target to determine\n",
    "# correctness. If another definition of correctness is preferred, a custom correctness function can be provided.\n",
    "min_pert = MinParamPerturbation(forward_func=net, attack=gaussian_blur, arg_name=\"kernel_size\", mode=\"linear\",\n",
    "                               arg_min=1, arg_max=31, arg_step=2,\n",
    "                               preproc_fn=normalize, apply_before_preproc=True, num_attempts=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling evaluate with the given image returns the perturbed image as well as minimum kernel size needed for the model to misclassify the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum Kernel Size for Misclassification:  7\n"
     ]
    }
   ],
   "source": [
    "# None is returned if no choice of argument leads to an incorrect prediction\n",
    "alt_im, kernel_size = min_pert.evaluate(image, target=label)\n",
    "print(\"Minimum Kernel Size for Misclassification: \", kernel_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that a kernel size of 5 was the minimum necessary to misclassify this image. Let's look at the perturbed image and corresponding prediction, and how this compares with the original."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAEICAYAAACQ6CLfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAlB0lEQVR4nO2de7RlVXWnv3le91kFFIjyKEUM2mB3KO1qgsEHHYlB0kaMiYIOB+kmKTsdhxpJt6idiA7TrQ7f7aNTKkIigiaCqEEDISK+2rawEdASFUcBBQUFBUXduq/zmv3HWiWnjnvOe+6te88t3PMb44xzzl5n7T332mvux/qdOZeoKkEQlI/KahsQBMHqEM4fBCUlnD8ISko4fxCUlHD+ICgp4fxBUFIe884vIttE5Iz8+c0i8oklrueHInL6ctoWBAc1qvqYfgHbgDMWWecS4B2rbfsKtMU64CpgGrgTeIXz23OA24FHgJ3ApcDanvLjgGuAh4H7gA8DtZ7ylwFbgSngR8DZPWUXAS1gb8/r+AIbngdo77EABHgLcBewB7iiz65LgGbfuqu57AjgW8AuYDfwHeC0vm0eD3w52/0g8O4Cu04A5oBP9y0fBz6a6z0C3NhX/kzgxmzT/cDr+vrpbI/N1y5in9cBn83bfRC4rK9c8zHft+5PDNRfVrvD9jVebQl1Suf8ubNUCpZfnjvJJPDs3EGfbqxjPXBE/jyZO9SHesqvye00CjwBuBV4bS47JjvgC7MtvwvMAEfm8ov6Hadg+3XgZuD/9Dn/ecCPs32TwNXApYMcu2zr00h3tAKcDTy0r18BDeAO4A3ARP79rxes51rgGwXO/+nsmI8DqsC/7Sk7gnQSfSUwAqwBThyknw6wzx/NNq0FDgH+GXhfT7kCv7bYfrTit/35tvxNIvIjEXlYRD4lIqO57HQR2S4ibxSR+4BPiUhFRC4UkTtEZJeIfE5E1vWs71Uicmcue0vfti4SkU/3fH+2iHxbRHaLyN0i8kcisol0gP6biOwVkS/12Lnv8WFERD4gIvfm1wdEZKTP5gtEZKeI7BCR/7iI9liX2+De3B5fyMsPE5Evi8gDefmXReTYnno3iMhfi8i3SI52fN96J4CXAn+pqntV9ZvAF4FXFdmhqner6oM9izrAr/V8fzLwOVWdU9X7gK8CT89lxwK7VfUrmvhH0pXnKYO2A3ABqUP/uG/5i4BPZvv2Au8CXi4i4wutMNt6u6p2Sc7fAQ4jXTkB/gi4V1Xfp6rT+fe39K5DRM4h3TVc37f8acDvAZtU9QFV7ajqTT0/eQPwT6p6marOq+qUqm4dpCEG2OcnA19Q1T2q+gjp7u7pxroGZljP/K8EfofUOZ4K/PeesieQDs6TgE3Aa0ln7OcBR5NuOz8CICInAR8jdeijgcNJHfGXEJEnAl8B/hfpTL0BuFlVN5Oucu9W1UlVfVFB9bcAp+Y6JwOnFNh8COkKeD7wERE5LG/3FSKyX4fq4+9It49PB44E3p+XV4BP5XZ4IukW8cN9dV9FaqM1pNv6Xp4KdFT1Jz3LfoDTSfLJ8RHSLfBLgQ/0FH8QOEdExkXkGNJV/qu5bAuwVUR+T0SqInI2MA/07veLROShPJbyp33bfRLwn4C3F5mVX73fR0i34vv4L3ndN4nISwv26xbSbfsXSbfAO3PRqcA2EfmKiDyYT6j/pqfe2mzTBQV2/Qapzd+W697at+1TgYfyxWaniHwp98FeLssn92tF5ORF7PNHgP+QLxCHkY7VV/rWfaOI3CciV4rIcQX2/zJDuEXdBvznnu9nAXfkz6eTbh9He8q3As/v+X4U6fmxBvwVcEVP2USuf0b/7SbwJuAqw6ZL6Lt1pOe2jHRreFZP2e8A23psnmX/59+dwKkDtMVRQBc4bIDfbgAe7vl+A/B25/fPAe7rW/YnwA0DbOuY3HZP7Vl2InAT0CbdVl4CSE/5+aTnyzbpTuR3e8pOIp2cq8BvAjuAc3vKrwZeXnQsgD8GfkIacziE5MAKPCuXP5N00q/lvjRF33N9/t0ocC5wXs+ya3NfeiHpEeC/Aj8HGrn8g8Ab+/tS/v7mbMdFue7z8v6fmMt/Qrpj+Hd52x8CvtVT/zRgjHTifxNpHOXQAff5aNKtfje/rttncy5/brbpUNIF4zYGeIQe1pX/7p7Pd+ad2ccDqjrX8/1JwFX5Vn036WTQAR6f6/1iXao6TRrcKWI9yYmXwtHsf2Xtt3mXqrZ7vs+QntUWYj3wkKo+3F+Qr7B/kx9p9pAGjg4VkWrPz+7ur9fDXtIzYS9rSc7hoqr3kK7qV2RbKsA/AVeSTrBHkG6f35XLzwDeTToR7nOET4jIhry+H6nqvZpujb9Ncqo/yHVfBKxR1c8a5lxMGru4Afgh8LW8fHte9/dVdZeqtlX1GtJd3O8X7NOcql4OXNhzlZ0FvqnpcaUJvId0Ijkx234Gj96J9TNLOnG8Q1Wbqvr1bNsLesqvUtXv5f78NuA3ReSQbM+3VHVWVWdU9X+SThTPGWSfgb8nnRzWkI7pHaTxh337emO2aTfwOtJjwonGfvyCYTn/+p7PTwTu7fneH1Z4N/BCVT205zWaO+iO3nXlZ6LDjW3ejf0MulAo472kk5Bl81K5G1gnIocWlF1AGqz6DVVdSzqbw/63g57dPwFqItJ7e3wyqTMNQo1H22sdqZ0/rOn5dRfpkeSsXL6BNNK9RVW7qvo94Lsk5ylCe/bj+cDGfIt6H/By4PUicjVAXt9bVfU4VT02239Pfi207iLqPDo+cgt2G55OuvLele36C+ClIvL9nroe/eve99my7Rd2D7DPJwN/o2mcYi/wv3n0WLjrdlno1uBAX6Tb6VtJz+brSKOo/0MfvYXe3vf7PyedAZ+Uvz8OeHH+/HTSFe7ZpCvOe0i3nUW3/U8kXfVeRurYhwMbctk7gc8U2LlvPe8Avp23fQTwTfKtqWHzL+oO0B7/CHyGdCWtA8/Ny99Neo4b5VHJTnl0pPoG4I8XWPcVpCvIBOk20xvtf2VuIyGd6L4OXNlT/nPgwtx2h2Z7LstlzyNJTvva8xmkO7AX5O8vzvsnpPGSe8i336Sr1xN6Xp8lXW3X5fJ1pJOQkB4fbiMNsu2z6w9Id1kV0lV3Cjg9l53a0zfGgDfm8qNz+dNId2lnkB5J/px0FW2Qbsd77XoP8A/A43LdOvAz4C9zm5yW1/2vcvlvkcanNuTfvh/4Rk9fPC1vZ5T0uPEAcPiA+/w10tjVWH59lPxIQfKJDXl/JknjNrcD9QX74pCc/00kLXg3SU8edxypQho5vT037h3kk0UuP4+kh+4iDcxto8D58/fnkK5Ie0hX3X0d8ASSzLSbNIpK33r2PbPtyK8PkcclDJt7674S+KHTHutyG9yfO8uVefnRJAffS7qKv5rFO/864Aukkfe76NH5cwfcCzwxf/9r0m3ldH7fvK8z5vINeZsPkxz978lSXi5/DckZpkgnigt6yi7Px2cvaTT/tY7Nl7D/M/9T87GfIT1uvaHv998gndT2kAY0z+kpe15eNkWS+L5OPrn2/Ob3s9178v5ZJ8f9+lKPo30nt9mPgJf0lf8p6UT3MPAlYH1PvVtyvV0kJWHjIvb5yXl9u/J+fRU4oeekc3te9858/E8YxDclr2DFEJFtpE77zyu6oSAIFsVj/u+9QRAsjXD+ICgpK37bHwTBwUlc+YOgpNSGubHDDz9c169fb5Qu/g7Eu2vRbndp9ZwykWLp1FoOIBX7/Fpxyrx1etjVlra+pWO143DtsI5nt2P3j26345TZ9bwuLBV7vyvVauHyqrEc7P5x55138eCDDw7UyAfk/CJyJunfW1XSf6jf6f1+/fr1/Mu/XFtc6Did1eCtVtOs05qfd8rsep1O2yyzGrzRGDHrjI6PmWUjY3ZZrV43y7yTRsXoZN5JyDvRLPVEuTTnd06iSzwZtlvFx3N2716zzvSU/afI2ZkZs8w7MXjHenLNmsLlE2uLlwM0Go3C5c961nMKlxex5Nv+/LfTj5D+J30ScG4OvAmC4DHAgTzznwL8TFV/rul/0leQ/tkVBMFjgANx/mPYP9Bke162HyKySUS2iMiWXbusGJwgCIbNgTh/0UPYLz3oqepmVd2oqhsPP9yKwQmCYNgciPNvZ/9ovWNZnsi3IAiGwIGM9n8POEFEnkwKZjgHeIVfRe0RUWfkuN0uHrFtNVtmnaYz2t+cs8u6xrbAkWsc2z25xpdyzCKoeoet+HzuneV1yaP9npxavNwd6xfbSs9GD0u263RsOa/j9IFOy+5z3mh/p2YfM0th6no2GttajGC+ZOdX1baIvIaU9KEKXKyqg8aOB0GwyhyQzq8pk8o1y2RLEARDJP7eGwQlJZw/CEpKOH8QlJRw/iAoKUON6tOumjKbJ5O05ucKl89OT5t15pyy5qwdnGHJimAH1DRGR806rbli2wFaTTvb94izzroR1AFQrRcf0kp1aYE9XvRbx4l+s7Q+T86rVBzp0wlM8mg1i4O4vP7hBf3MTttlbiSpEzBWqxXvW92RBy0JU71j0kdc+YOgpITzB0FJCecPgpISzh8EJSWcPwhKylBH+zvdDnv27Cks84IY5meKR2Zn9zxi1pk1tuOtD6DtBG5YudbqI/bI/MykPaI/bqRvAhgdt6ejb4zaacMsJcALIsLJL+cFuXjHTLvGaL8zal+tOanLvKAfswTaxmj/9JTdP2YesfvV3F47xZc32t920rmJFrejOHvWmCvuc912jPYHQbAA4fxBUFLC+YOgpITzB0FJCecPgpISzh8EJWWoUl+71eaBnQ8Ulnm585pGMMX8nt1mnXlHymm6Up+Tw88Ijqk1HKlvYsIs8wJIRp16I2OLlwEtmRLAUdF8qc8ps3L/iRO8U63aUp8b2ONofU0rKMzpH56EbPVF8INqWhP2Meu2i+Vlr33rI8XH2Tte/cSVPwhKSjh/EJSUcP4gKCnh/EFQUsL5g6CkhPMHQUkZqtTXbLW4Z3vxdH5ejrPubLE0p47s0jHqAHRbznRdTqSaNU1W05s2zJFeWo6sODdXHI0GMDJu5wVsGBGG1botsVW8HH7d5Y7q86Q+pzsuMc9gc644X6Mb9TllS31tp1/hTF/WmZ81yyxJr9Oy+0CtUSz1tQ3ZsHAdA/+yABHZBkwBHaCtqhsPZH1BEAyP5bjy/3tVfXAZ1hMEwRCJZ/4gKCkH6vwKXCsiN4nIpqIfiMgmEdkiIlumnOesIAiGy4He9p+mqveKyJHAdSLyY1W9sfcHqroZ2Axw/PHHL2b68CAIVpADuvKr6r35fSdwFXDKchgVBMHKs+Qrv4hMABVVncqfXwC83avTnG9y5513F5ZVjCSGAPVOsXzR6NhSSN2RqKpeGJgjKSnFZV0jgg1AvYSKRnJJgG7FmarJCcNrG2pTrW2vr+Ik8MQ5LupIbFZUH2KvTyqLlw4BOk7S1fmZYjl4zokIbe61H087hnQIgBPVp45sZ8nc7aYtSVfrxRGQXlv0cyC3/Y8HrsrzvNWAz6jqVw9gfUEQDJElO7+q/hw4eRltCYJgiITUFwQlJZw/CEpKOH8QlJRw/iAoKUOO6mty9/bthWV1R2KbrBWXHdKwzZ8ctcvqRkQUQMWQUADUiDrzZDkrEjAZYtvRdep1HKlPDEXMk8oqjlTpToRnSJ9umTp1bOXQjdxrOfJWc75YLms5MponsXWcMvESeHpNZTSyF9VXsfpiJPAMgmAhwvmDoKSE8wdBSQnnD4KSEs4fBCVl6NN17dxZnPRnpOaMbk8W56Ubra0x60zU7Sm0KhOTZlnNmAYpFRojrGLnpVNnmilrxBagUnNUB6etukaOPGewH3FG2T3EUR3MvIDe3GAOHUd26Dj71jVUBytIC3Bt9KcNsxuy6+SobM8V5/freqP9Rvt6U4b90joG/mUQBL9ShPMHQUkJ5w+CkhLOHwQlJZw/CEpKOH8QlJShSn2dbpep6eIcaK1Gw6w3Nlosv7Wqjhw2OmGWyYQtEVYcqU+NAB5vuit1yrypq3Akpa6Tc69jBOl0WrYE5E5R5iDOvtUMOdJTyrz1db0cfo7U1zaub+r0HXH6YsWR82g7efqcXIgdIxin6wQRWXS7g+u2ceUPgpISzh8EJSWcPwhKSjh/EJSUcP4gKCnh/EFQUoYq9alC29Bl2o5c0zEkNm3YkXs6Mr6ksq6Xw8+KpFpiCryKo3t1nKizrjUnF9AyIsHmZm3ZqOlJSs4OVKu2VNkw8iTWnPb12kO9iDlPxjQkQnHyLkp9zC7zohJbTpmXF7A9V7hcnenc1JD0rOVFLHjlF5GLRWSniNzWs2ydiFwnIj/N74cNvMUgCA4KBrntvwQ4s2/ZhcD1qnoCcH3+HgTBY4gFnV9VbwQe6lv8YuDS/PlS4OzlNSsIgpVmqQN+j1fVHQD5/UjrhyKySUS2iMgW63k0CILhs+Kj/aq6WVU3qurGet3+z3QQBMNlqc5/v4gcBZDfdy6fSUEQDIOlSn1fBM4D3pnfrx6kkohQMa7+VeeuoGrIRtKwJRk8GdCJ6Oo6Uo4VMeVFUqmjlVXU25ZZRKttT081O1OcDHLv1F6zztxscZ1kh21/zUkyOjpefGwsCRCgWnMSoXpSsNMedIrLqs766hV7v+rG1HEAFUeeVWe6sY4WS33tlp30U42EoOo1VB+DSH2XA98BniYi20XkfJLT/7aI/BT47fw9CILHEAte+VX1XKPo+ctsSxAEQyT+3hsEJSWcPwhKSjh/EJSUcP4gKClDjeqTSoXRsWIJaGTUlubqo8V1ao5s5M11h5co0klm2TUkpa4hJ4Ef1adOAk8ncI/5eXt709PFst2ePVNOneKkquC3hyf1jRkRaY0RZw5FZ31dpyU9SQxj7rpG1e4DY3X7uIw6kYx18dzJLmt1im1pedGKVtLP5ZT6giD41SScPwhKSjh/EJSUcP4gKCnh/EFQUsL5g6CkDFXqq1YqTEwUz6E3YszHBzA6ViwP1RpOMsiqc17z5lvr2AlHpFUcfVVx5mjDi/Sq2s3fNeQfgFbTlrbm54sTRc46CTxnZmypz5PRqo401zaiAetNW74SY34/gI4TXdhsOVKrIfWN1O1tTY7ZfbHdcGRAT2VzJhS0ktrOO3qvlbQ0pL4gCBYknD8ISko4fxCUlHD+ICgp4fxBUFKGOtpfqVaYmCyeKqvRsHP4WUpAzRkdrlTs0XJxRvurTpCOWNMqGSpAqmTb0ek62Yy79qiyN11Xu108Ctw2AkG8OgAtp543Ao8x9ZY3gm3VAWg6AUZeoFPXGO0fbTiBNi07N2R71FY4ms6ltNK0bewY6o0b2NMy8kkOPtgfV/4gKCvh/EFQUsL5g6CkhPMHQUkJ5w+CkhLOHwQlZbhSX6XCxESxjOJN4jk6UlxWq9vnLkfpQ7AllIrakkylWxzAI107aEad86sbg9F1gn4ctcyaOsxVgBw50g1McnbAkhb9aatsK5qOHDk/bwdWqSHrdluOTOzsV8WRZ7tOXsCqM6WYWvKs0yBW2y8irmeg6bouFpGdInJbz7KLROQeEbk5v84afJNBEBwMDHLbfwlwZsHy96vqhvy6ZnnNCoJgpVnQ+VX1RuChIdgSBMEQOZABv9eIyC35seAw60cisklEtojIllknaUQQBMNlqc7/MeApwAZgB/Be64equllVN6rqxrHx4v/1B0EwfJbk/Kp6v6p2NA2lfhw4ZXnNCoJgpVmS1CciR6nqjvz1JcBt3u/3UalUmDSkPi9Cb2SkOJKq5kyd5KpXnh7iRPwJxWXiCGlWnYXqLUqz6cGKZqw7+Q7Hus5UaXW7ntfEVoSeFwko3lRpYtfrOpewrhZb6UnBVt4/gI4XHel2K3ud1rHuOkaqGn3fPSj7s6Dzi8jlwOnAESKyHXgrcLqIbCDJx9uAVw++ySAIDgYWdH5VPbdg8SdXwJYgCIZI/L03CEpKOH8QlJRw/iAoKeH8QVBShhzVJ4yPF0dFVZ2pqxrGFElOvkczmgug403X5SlsUmyHVLxmtI3sGOsDUG/nHNmrakimo850aHVHZnUlR08uM+q1nKm1Wk6Sy5qzrZrTHh0jilCc9vVkYitqEqDj2OEljRVD0hNPZrWmo5PBr+dx5Q+CkhLOHwQlJZw/CEpKOH8QlJRw/iAoKeH8QVBShi71jY0VyxfiSBSWKtPt2BFW856c59SrtOyyqpFQsWJFWOFLSh3n3NsyotHAVyMrxva8uRArI07iSUtSAqpe1JkRxdacs5Odzs/Zcx7O2U3MnHMJaxmRgu40g27yV08mXposWqkXu6FU3QyvxXU84/u3O/AvgyD4lSKcPwhKSjh/EJSUcP4gKCnh/EFQUoY62i8i1EeKN+kNlHaN4Ixm0x457rTsnGltJ7hEnWmVKoaC4A2wVp08g86APi1nULnZtgvbTplFrWbbaE2VBtCoecnzjNF+se2rdu22p203Vscxo2MoNN5UY1Z/A/Ca14vF8mK/akbUUtXJu2gFH8VofxAECxLOHwQlJZw/CEpKOH8QlJRw/iAoKeH8QVBSBpmxZz3wt8ATSNEEm1X1gyKyDvgscBxp1p6XqerD/rqgYsgaVq41gHazONhmdsaW+man7bK5WTuApOPIgJYeaU2RBf40ZF7Qj5MqjrZT2DUiVqretpz8fl5+vIbYUpQli4oTVKVO27fn7ePZdI7nvNF3Wk4berKzp6R1vESDo7acauVdrNadYDczh9/ySn1t4AJVPRE4FfgzETkJuBC4XlVPAK7P34MgeIywoPOr6g5V/X7+PAVsBY4BXgxcmn92KXD2CtkYBMEKsKhnfhE5DngG8F3g8ftm6s3vRy67dUEQrBgDO7+ITAKfB16vqnsWUW+TiGwRkS179+5dio1BEKwAAzm/iNRJjn+Zql6ZF98vIkfl8qOAnUV1VXWzqm5U1Y2Tk5PLYXMQBMvAgs4vIkKaknurqr6vp+iLwHn583nA1ctvXhAEK8UgUX2nAa8CbhWRm/OyNwPvBD4nIucDdwF/uOCaxJlNyIi+AmgbkXazMzNmnalH7EeMqT122bwjKVnRXuLIK1ZOPfClPk9v6npalJEszpuSa3xs1CzrTI7bZWOORGjsWsvJ0zc7PW2WTe+1j/X0jL3OmfnivtNykvipkyWx5mh9HWNaOQCp2NGRNSOHYs3rO9amBlf6FnZ+Vf2ms8rnD76pIAgOJuIffkFQUsL5g6CkhPMHQUkJ5w+CkhLOHwQlZagJPAGqlWIZpetEj6HFkVmddtOsMj83a5bNOJLSjCMpNZvF2+sYU0KBnyjSw6+3eGmxXrcj8CbGx8yy+Wlb6psdtyXChhF15kVNTs/Yx2yPc1ymHKlv1ojqa3vzdTlyWc2Zvqwz6kQ5Vu0owvqYccy6jnRoGLmY3hZX/iAoKeH8QVBSwvmDoKSE8wdBSQnnD4KSEs4fBCVlqFJfBaVhSHriSH01Q8CoGkkiAcRL0Ni2pbmWIecBzBpS1NycHQnYbtnrcyVCR4ryogGr1eJD2mjYUWVzs7bENu+UTTuJP615/NTZ57l5u632OnLe9Kzd/vMtQyZ2pFQzOSbQcKIjK848hCNOAk8rea16mqMZHjs4ceUPgpISzh8EJSWcPwhKSjh/EJSUcP4gKClDHe0XEUaNfHdVJzda0ygbcUdl7TIvOMPLx2fl8Gs5ef9mnamkPGWh66gV1og+QK1RHFziTYfmBRF1HGVk3lE5atXidhRnWy1jZB5g1lECPJWg2S5epxfXU6s5U2t5OfK6TmCP069q1eJ6jaqnphTXkUWoAHHlD4KSEs4fBCUlnD8ISko4fxCUlHD+ICgp4fxBUFIWlPpEZD3wt8ATgC6wWVU/KCIXAX8CPJB/+mZVvcZbVwUYMc43VSewp1ktll7GHElmrGHv2tiILcnMOvXmjO15Mo44wUddJ8jFk/qs/G0A2i2235XzHBmw6chvno2WcutdbbxAp2bTzv3XcvICWvJh1zkuqN0/6o6E7KjVNGp2YNV4oziH4sTohFlnZLR4fdVFSH2D6Pxt4AJV/b6IrAFuEpHrctn7VfU9A28tCIKDhkHm6tsB7Mifp0RkK3DMShsWBMHKsqhnfhE5DngG8N286DUicouIXCwihy23cUEQrBwDO7+ITAKfB16vqnuAjwFPATaQ7gzea9TbJCJbRGTLHmdq7CAIhstAzi8idZLjX6aqVwKo6v2q2lHVLvBx4JSiuqq6WVU3qurGtWsnl8vuIAgOkAWdX1KkyyeBrar6vp7lR/X87CXAbctvXhAEK8Ugo/2nAa8CbhWRm/OyNwPnisgG0gxB24BXL7QiAerG+cbL4dcwctaN1B2pr27v2viIXTZvSCgALWM6pk7TrkPHlqEqzuRKnvxWWUJUX93JPSeGlAp+9FvLifgTY9+8oDh15LeOJ5k6Mma3W2yjJ1OqkyPRlfOMtgeYGLOnRFszUSzpHTKxxqwzMlIc8Vet2Meyn0FG+79J8TFzNf0gCA5u4h9+QVBSwvmDoKSE8wdBSQnnD4KSEs4fBCVlqAk8QagYUUdeYsSGIW2N1R1pxZDlAFrjdmJEdabXkm6xXFPDlrxGa/aOzc/bNrbbTtSZF7lltJUYSSIB1FmfL6PZZVYUoZcg1SvzJKxa1YmONGS7jiOzWslHAUYdCXlyzO5XayfHzbJDDKlvzbhdZ8SYfq3qyJT9xJU/CEpKOH8QlJRw/iAoKeH8QVBSwvmDoKSE8wdBSRnuXH3YCQalYksvI0bizAknAk/bo2ZZ1YkQazh2jDeKbV87Zstoc7N2NNe8M1dfq2XLV07AHx0tlqlaxnKAlpfA04ncM6bBA0CN7XmSrpcIlZp9XNrOSuvG5c2L6ht1EryunbD71SGT9rE+xKm3xpCeJxw76obk6Mml/cSVPwhKSjh/EJSUcP4gKCnh/EFQUsL5g6CkhPMHQUkZrtQndsSUF1kmxvx5onYUVc1JCDrizLc2MWJHj81OFG9vbs6Ovmq6cp6d3NObI6/pyIBzzeKymXl7fdNzth0zc7b9TScyzpqHsOpkwPQi0jwFq+vkrGxUrUSitu2jo3a/OnTN0uS8yTFblh4z9EhPdq5JsVTpJcLtJ678QVBSwvmDoKSE8wdBSQnnD4KSEs4fBCVlwdF+ERkFbgRG8u//QVXfKiLrgM8Cx5Gm63qZqj7srguhboz2eqP9VcPMqjvFl23HqDPa33Sm8mo2i0eBvVF7t8yJjGk17Xqz83bZ9Ezx6Pye6TmzjjuFljHdFYCoE3xkDM9XnK05s4b5QT9V+3hqrTg4xlMdxse8IBy7zMsbOWpFGAE1Y1S/4rSvMdjPIgb7B7ryzwO/paonk6bjPlNETgUuBK5X1ROA6/P3IAgeIyzo/JrYm7/W80uBFwOX5uWXAmevhIFBEKwMAz3zi0g1z9C7E7hOVb8LPF5VdwDk9yNXzMogCJadgZxfVTuqugE4FjhFRP71oBsQkU0iskVEtuzeM7VEM4MgWG4WNdqvqruBG4AzgftF5CiA/L7TqLNZVTeq6sZD19rzjQdBMFwWdH4ReZyIHJo/jwFnAD8Gvgicl392HnD1CtkYBMEKMEhgz1HApSJSJZ0sPqeqXxaR7wCfE5HzgbuAPxxkg5Zk4ykUYskyjjYkNTtHmzi7XXG0EktR8qZ3qjuyYrtt29905CsvT1unU2z/nBMo1DByJALUHTvajlxG15D6PMXOna7LEySda5hRzztmjbrdHg3neHrr9My3gowEJ1mjmZNxcK1vQedX1VuAZxQs3wU8f+AtBUFwUBH/8AuCkhLOHwQlJZw/CEpKOH8QlJRw/iAoKaK6iDCgA92YyAPAnfnrEcCDQ9u4TdixP2HH/jzW7HiSqj5ukBUO1fn327DIFlXduCobDzvCjrAjbvuDoKyE8wdBSVlN59+8itvuJezYn7Bjf35l7Vi1Z/4gCFaXuO0PgpISzh8EJWVVnF9EzhSR20XkZyKyaok/RWSbiNwqIjeLyJYhbvdiEdkpIrf1LFsnIteJyE/z+2GrZMdFInJPbpObReSsIdixXkS+JiJbReSHIvK6vHyobeLYMdQ2EZFREfm/IvKDbMfb8vLlbQ9VHeoLqAJ3AMcDDeAHwEnDtiPbsg04YhW2+1zgmcBtPcveDVyYP18IvGuV7LgI+Isht8dRwDPz5zXAT4CTht0mjh1DbRNSNvXJ/LkOfBc4dbnbYzWu/KcAP1PVn6tqE7iClAm4NKjqjcBDfYuHng3ZsGPoqOoOVf1+/jwFbAWOYcht4tgxVDSx4hmzV8P5jwHu7vm+nVVo4IwC14rITSKyaZVs2MfBlA35NSJyS34sWPHHj15E5DhS8phVzRDdZwcMuU2GkTF7NZy/KP/QaumNp6nqM4EXAn8mIs9dJTsOJj4GPIU0QcsO4L3D2rCITAKfB16vqnuGtd0B7Bh6m+gBZMwelNVw/u3A+p7vxwL3roIdqOq9+X0ncBXpkWS1GCgb8kqjqvfnjtcFPs6Q2kRE6iSHu0xVr8yLh94mRXasVpvkbe9mkRmzB2U1nP97wAki8mQRaQDnkDIBDxURmRCRNfs+Ay8AbvNrrSgHRTbkfZ0r8xKG0CaSMpJ+Etiqqu/rKRpqm1h2DLtNhpYxe1gjmH2jmWeRRlLvAN6ySjYcT1IafgD8cJh2AJeTbh9bpDuh84HDSXMe/jS/r1slO/4OuBW4JXe2o4Zgx7NJj363ADfn11nDbhPHjqG2CfDrwP/L27sN+Ku8fFnbI/7eGwQlJf7hFwQlJZw/CEpKOH8QlJRw/iAoKeH8QVBSwvmDoKSE8wdBSfn/GzAXM26eFM8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQQAAAEICAYAAAC5yopxAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAolklEQVR4nO2deZhc1XXgf6eqel/Uau0ItCKz2ggilhiGYLwBXzLYE9uficfBMyTEMyZ2tvni2ONAPE7iyRc740zyOcGxA04cL/ESk/kcLyEQYYMxAgtJoAUE2oWkltTqvbuWM3+818+l5p7bJXWrCiXn9331VdU977533n23zrvvnjrniqriOI4DkGu0Ao7jvHJwg+A4ToYbBMdxMtwgOI6T4QbBcZwMNwiO42ScVQZBRHaJyBvSzx8Skb86zf08IyI3zKZujaC6PRxnNjirDEI1qvoHqvpL020nIveJyMem1L1EVR8+Y8r95NgPi8i0OtYDEVkhIg+JyIiIbIsZEhHpEZH7ReRw+rpnivy1IvIjERkUkU0icl2VTETkwyKyR0QGRORLItJdJX9GRIaqXiUR+ccq+VoReTLV80kRWVsle4+IlKfUvyGVtYjIZ0Vkd6rXj0Xk5innr1PqfqRKfo+IFKfIV9XSfiLyOhHZLCL9InJURL4hIkur5O8QkUfTug8H2ltFZLjquH9VJRMR+ZiI7BeRE2mfuqRK/rCIjFXV3V4le9eU8xlJj/VT1rVHVRvyAgqnUWcX8IZTrHMf8LEGnePDwC/NZhucbnsAjwGfBNqAnwf6gQXGtn8N/D3QDqwAdgL/JZX1An3A24E88J+B48DcVH47sA04D+gEvgncbxxHgBeAX0y/NwO7gV8HWoD3p9+bU/l7gO8b++oA7kn1zQE/CwwCK1L5CkCtNk/r/u3ptB+wCDgn/dwC/BHwQFXdNwDvAH4XeDiwbwXON477DuAAsCpt7z8Enqq1j03Z13vSaynmNrP8A9gF/A7wbNpJ/hpoTWU3APuA3wZeAv4mvXAfTJU8CnwF6K3a37vTDnEU+DBVP4CpFxC4Dng0vVB705O/EygCE8AQ8I9Tf0jpBfw/aaMfSD+3TNH5N4HDwEHSH0YNbfH7QBkYS4/9Z1UX/33Ac8CLoY469SIDvwxsJengzwJXBM7jwnR/7wzo8ipgHOiqKnsEeK+hex9wZdX3DwGPpJ9/FnhmyvY7gDvSz18F/keV7LVpG7QHjvMzadt0pN/fBOynqsMCe4CbpjMIxnlsAn5+pgbhVNov7U9/CDwbkP0Sp24Qfhv4StX3S4Axq69M0x4PAXfHtjkTjwzvAt4MrE4b8n9WyRaT3GGWk/xY3w+8haRjnENiRP4cQEQuBj5NYhTOAeYB54YOKCLLgH8C/i+wAFgLbFTVe4EvAH+kqp2q+nOB6h8GrknrXAZcFdB5DrAUuAP4cxGZmx73F0RkU0gnVf0wSae5Kz32XVXitwBXAxeH6k45t7eTdNZfBLqB/0hiIKu3uQL4LvCrqvqlwG4uAV5Q1cGqsqfTcvPQUz5fWvVZAttaciH5kawJHON24KuqOlyl5yZNe2/Kpil6Xi4ifSKyQ0Q+IiKFoPIii0j63zNTRLtFZJ+I/LWIzJ8i+zkROZY+1vy3qvJp209ElolIPzAK/BbJKOFUWC8iL4nI10VkRVX5l4DzReRVItJE0mbfnlL3D9M2+YEYc2Mishy4Hvh8VItarW2NFmgXVVYTuAXYqT+5206QjhjSsq3A66u+LyG5oxdIhldfqpJ1pPVfNkIgGZV8w9DpPqY8MnDynXUncEuV7M3AriqdRzn57n0YuKbG9niYKdab5G5wY9X3FURGCMB3gA9E2vv3SEYxr4vo8W7gh1PKfh+4z9j+b4GvA13A+WkbjaeyeSSjsNuAyQ5aAf6y6i64Iz2vOcAD6fn99JRjtAMDwA1VZR+pvuZp2ReAe9LPq4CVJCPLV5OMln4noH8T8M+TOqVlncC6tG8tIhnJfKdKfjHJjSdPMqo5CNx2qu1HcsP77VAfwR4hXE/yuNQD/BmwZbI/pOWfStuwRDIKXFlV9+r0OrWk12IQWB04xkdCx576OhMjhL1Vn3eTNPIkR1R1rOr7cuAb6WRMP4mBKJM+k1XvS5O7yEl3xirOI+m0p8M5qZ6WzkdVtVT1fYSkc82EvdNvkjHdub0XeFRVH4psM0Qyuqimm6TzhHg/iSF8jmQO4IskRgdVPQrcCvwGcAi4ieTHty+t+7l0+4dJ7s6Tek3KJ/lPwDHgX2vVU1VfUNUXVbWiqpuBjwJvq95YRHIkj6MTQDYqU9UhVd2gqiVVPZTK3jQ54amqz6rqAVUtq+qjJD/CyX3X3H6qegy4H/imNXoJ1FmvqhOq2g98gMToXZSK7wauJOkHrSQ3gH8Rkfa07uOqOqiq46p6P/ADkhvxVH4x1SvKmTAI51V9XkbyXD6JTtl2L3CzqvZUvVpVdT+Jhc72lTbAPOOYe0keUUJMPeZUDpAYJkvnmWAdu7p8crjcXlW2uOpz7NwgMQjLRORPIts8A6wSka6qsst4+XA6UU71mKq+S1UXq+olJP3kR1Xyf1XVK1W1l+TuecGkPP2x3q2qK1T13PQY+9NXNbcDn9f09lWl52tEpPqR4zWWniTtmG2b1vssyQ3l51W1aNSbrAsvf/wJ7fuU2o9kFLKQlxuRWqk+9mXAl1V1X2rM7gPmYj9untQmACJyLclN7qvTH3mGjwlThiW7gM0kz/q9JM/Qf6A/GX7vm7L9r5PcSZan3xcAt6afLyGxzNeRDJv+mGTIFHpkWEZird9BcjHmAWtT2ceBvwvoObmfj5FMRi4A5gPfJ33EMHTO6tbQHl+aPP+qspdNIJHcPf87yXD1v5I8Nk0+MrydxCj8VHqhz69qr10kM9g9wJPAxyO6/DBtw1bgrcS9DKvTNswDN5NMMl5SJb+cZFjeTTIJ+4MqWW9aX0g67Rbgzin7Pze9lqunlE96GT5AMgS+i5O9DDcDi9LPF6b7vruq/l+k59kZOKerSQxXLj23LwMPVclvJfmhCck80n7g9lraj2S0M7nvBSST49WegHxa773A+vRzU1U/X5tu05m25/Yq+d0kfXJRuv93k9xEetLXm9P9FUjm74aBC6ac+70kxnf6PnsGDMKkl6GfZIjSHvlx5UiGnttJftA7qfoBkdxF9lCbl+E/AI+TPJfunbyYJJNZG1N9/iFgEFqBPyUZkRxMP7dGdK6u+y6mzLhP2fanSZ6njwN/GjEIN5M8G/YDnyAZRld7Gd6bttEQyY/g8oAuvSQTXf/L0GUFifEdTff1hiltN1T1fdLVNZK23Zun7OuLwIn09WVgYZXsVen+R0h+zL8R0OV3SL0WAdnlJMZtFHhq8lxT2R+TPKYMk7grP8pPfjjL07ad9OpMvt6Vym9L23g4vc6fBxZPOaejaZ1twPtPof1+tWrfL5HcCJZXyd+T6lb9ui+V3Zjub5hkfuofgDVVdVtJJtoPkvTtp/iJ12UB8ATJb6efxGi9cYrerans9aH2nvqStNKsICK7SDryP8/aTh3HqRtn7T8VHceZfdwgOI6TMauPDI7jnN34CMFxnIya/jgxW8yfP19XrFg+/YZTqOcYRiIH01nW5GR3+9RjRerNqhbOqTA6MmrKSuWSKevstP/LFusHp8quXbvp6+s77R3OyCCIyE0k/+jKA3+lqh+Pbb9ixXI2bHg8KCtXymY967GmcpqnnYv82qIGwdBDT1ePnD1Aiz3KxTqQWObiTFiRWTaes/nDmI6odhVbuvXpzabsaP8xU3bNddeasubm5mD56fSBK9ddY9aphdN+ZBCRPIl/9GaSP6DclgYkOY5zljKTOYSrgOc1+X/5BMmfMW6dHbUcx2kEMzEISzk5SGdfWnYSInKniGwQkQ1HjvTN4HCO45xpZmIQQg8xL3voUdV7VXWdqq5bsGBq+LnjOK8kZmIQ9nFyZOO5zF6UoOM4DWAmXoYngDUispIkMuydwC/EKihQMrwJ5ZLtstmxdXuwfP7CBWad+YsWmrJiccKUbfzhj0zZ2GjY5XTNz1xv1ik02U1cjJxzQWxbXZGKLTPKczHPRGTKvRKdj4/Mgp+GW0NjdWIeiNhs/CkLQEp2++7eusOU7XzhBVN2xTVXm7J8wegjp+lpmgmnbRBUtSQid5Fk9MkDn1NVKz7ccZyzgBn9D0FVvwV8a5Z0cRynwfhflx3HyXCD4DhOhhsEx3Ey3CA4jpNR12hHiDuqLPoOvhQsb8nb6i9cbLsd9+550ZRtf+pJU5Y3jnfi0kuD5QCdc+fY+2tvNWUDh+1Ameb2cDAMQMHYZyXSvvnTDK2Mub4mxoyowEjgUFN7hymLuT8lIquYLkTbtVhoypuy3l77eh7ea/fHoRPHTVl72znB8miukjMUAuwjBMdxMtwgOI6T4QbBcZwMNwiO42S4QXAcJ6POXgalouHZ3VisRj4XnvU93nfErLN3mz0Nu/6BfzRlJ/bbAZvzl4Zng5967AdmnfY5XabsyuuvM2WPPvQvpuz8Cy4wZResfXWwvKh2ijpyduOPDQ+bMiv1F8DO554Plo+PjAXLAa58rZ1mbLxoL9PYHPE2HTg0dY3ZhL6j1rrBsOoCeynNQ0cOmrIju+01eZ/83ndM2Y1ve2ewXIx+D5DPh+/lM82i7iMEx3Ey3CA4jpPhBsFxnAw3CI7jZLhBcBwnww2C4zgZdQ9ushxcGgl6KY2NB8sfefAhs05Xk61Da3HIPtZwvyl7YcdAsPz4XtsV1dxtB+xc8GrbfajDER3HRkxZcTzsnhuPuKMiXke2Pr3RlHV3dpuy8aGwjiODthuzPB6+zgAV47wAxgu2e25oYDCsx7FwOcDBnXtM2TM/+qEpmy+2a3f/Uxvsfa5aGSxftuYis053d0+wfKZrN/sIwXGcDDcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhORl3djsMjYzzxdHhxpwN79pv1igfCEWvHX3jOrNO9cK4pmzev05TlO+08h4cGwpF6I4O2izCWC/Chrz5gykb325Gcx186bMo2b9gULC9Flzuzl5Q7sMeO4GuPLFPXZbgkh8fsY+3fYS+TJiXbpdfU3W7KhofCbs7hPjvHobWyGkB32cgVCSzstSNbi5Hl4bZ899vB8kP77Mjb62/5uWC5GtHEtTIjgyAiu4BBoAyUVHXdjLRxHKehzMYI4XWq2jcL+3Ecp8H4HILjOBkzNQgKfFdEnhSRO0MbiMidIrJBRDb0H7ef2xzHaTwzNQjXquoVwM3A+0Tk+qkbqOq9qrpOVdf1zLUn+hzHaTwzMgiqeiB9Pwx8A7hqNpRyHKcxnPakooh0ADlVHUw/vwn4aKzOicEh/umhR4OyvkO2m+1VzeFItwvn25GEXS12BFxp3HZ9dTS1mLK5LWHX195RO0pvrGTb3BNPbzZlMe9R65DtNm0fDutSrkzYOyzZrrTO9jZT1lyxz1tHw4+HQ4N22w8YLkKA8UiU5Nzzlpiy+YsWBMt37QwngQVojbTV8sW9puyE0fYALXk7/DbfH3Yj79EtZp2JG98YLJ9pktWZeBkWAd9I1/crAH+nqmGHquM4ZwWnbRBU9QXgslnUxXGcBuNuR8dxMtwgOI6T4QbBcZwMNwiO42TUNdpxYnyc3c/vCsqODfSb9ZYtCLvZFuZt1+Jws31qMm++KctHXF8tErafi3ptV1THXPtYHV09pkya7ajLpg7b7bj03HOD5a2ttju1VLTdbPlIGwu2b7Q4Ht7nhWpndK2YKXhBJ2x3WrFgywoadgXu2xaJrBywQ3PGI7fQEzk76rK72XY7dhTCCWllwr4uE0PhCFstR9bwrAEfITiOk+EGwXGcDDcIjuNkuEFwHCfDDYLjOBl19TIUx4sc2hPOj/ji7l1mvfPOXxwsX77AzmHX2mUvM9bUu8iud649u9/WFg6mKrfOMeuo4ZkAKEfiUPI5u97EuL2UW64pPJvd3GbPgHf12F6SQmSZNHthPltUjsyCR/ZGQe2uWszbDXniSDhX57wu21NTGj5kynJlOzirqcXuOzFvTaEY3md+8JhZZ+ePw0vKjY/YXrJa8BGC4zgZbhAcx8lwg+A4ToYbBMdxMtwgOI6T4QbBcZyM+rodJybYu2dPUDY+Zuf127g37DpatvJqs87atbaseYGdg2+sGM7fCDAwEQ6UKY3ZufQqZTsAqFSyj1WIuKm0Yru+SpXw8U6cGDDrHDiwzZQVI+0xYbQHwJw5PcHyjg47D+bRI3ZezeK47VosF2yH5by2sKx/cNCsI5FAqi4jvydAa6XflOUm7HtvqdUIfIq4OB975JFg+ZAR9FQrPkJwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3Ey3CA4jpNRV7djRSuMG3niNOKCW7Li0mB57twLzTrDLXYEYmnUPtZwJKdiqRyuN7fXPlbMNVeciLgPsV1fldg6b7mwu/K73/6OWeX768MuLICu7h5TNmbkTQS49rXXBssvvvhis86jPwhH8AEMj9vXrBLJ03j9la8Oli9ZFs49CVBot/NPtrfZP5njGnEjF+1r1jwSdhWOlez2PWZ1ncoZzqkoIp8TkcMisqWqrFdEviciz6Xvvqyz4/wboJZHhvuAm6aUfRB4UFXXAA+m3x3HOcuZ1iCo6npgaqaGW4H708/3A2+ZXbUcx2kEpzuHsEhVDwKo6kERWWhtKCJ3AncC5At2bnrHcRrPGfcyqOq9qrpOVdfl83Wdw3Qc5xQ5XYNwSESWAKTvh2dPJcdxGsXp3rIfAG4HPp6+f7OmWgqUwm6R1rY2s9pl634qWD6n206kOjZiJyJt6rKTs0rOdmGVJ8K6799/wK4TSSra1Wkn+sxHkrPmI4lPDx0KJwhdv369WefqK68yZStXrTZlR4/ZSUAXLw4nsl24yF7a7robbzBlBSN5LEDZiPAEKOTD7VguLbDrLHuVKavkbD0KJTtit7x/tykbPRTuP2XDzQ1w5NiJYHnJSNhaK7W4Hb8IPAZcICL7ROQOEkPwRhF5Dnhj+t1xnLOcaUcIqnqbIXr9LOviOE6D8b8uO46T4QbBcZwMNwiO42S4QXAcJ6Ou/xRSrVAshqP/5i9aatYrNIWjz8aMyEmIu/RUI4sqiu127DNcen2H7bUArehDgPPXnG/KCh32Woz5nB2N99QTG4PloyN2Wy1aYkf+PRKJQNy0cZMpu/nmm4PlI0W77fcdivydJXLJIiLKRpLbYqQPtDTb7Tuvw+5Xne32z2luhx0Ru8+IGm0es69ZodAcFkT6by34CMFxnAw3CI7jZLhBcBwnww2C4zgZbhAcx8lwg+A4TkbdExRUjMi0BQvs6LOikWCz0GmvE9jcbLhlIjoAaGQtRjUi7trn2Skl25vsJm6KyFRs15eobcf7j4ej4Ew3FTA4YK8H+OILL5qyE5FoRzHauKXF1iMXWc+yr++4KRsethPjjo2PBcubI9G1CyPXc/5C2304v8d2SY6P2td6+4lwhOKCSJLVtnnhaNJcxM1dCz5CcBwnww2C4zgZbhAcx8lwg+A4ToYbBMdxMurqZZBcjtbW1qBs5aqVZr1cPhywEUk7SCWypFXOyLOXCO2gl2VtYVm3DJp1NOLRGFV7xnoYO+9jPhLOs3jJkmD5tu3bzTqx8KDhQfvcKiW73kQxPEM+2B/2ggD0HxswZdu2PW/KikU79+D4WNjLkGu2cyMOL7Cvy/md9iz+sNj5DEfK9vFKGva8jE30m3WOHQsH1BUjeRhrwUcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3Ey6ut2BHLGUmnz5/eY9RYtDi//NTRiu6kQe38ltV2SFY3kaTwWXo6rbfcGs05RbNfc6Fo7oGuiObLMm9p587ZtD7vniiXb/Zlvtu8LFex6pUj3OdAXzo/Y0mO7U49EApiO9/ebskpk+TKphNtfIq7KwWb7mh09buvRNxDpj932Enb51nAgW2HCvs5HBsNLFZYiwXm1UMtSbp8TkcMisqWq7B4R2S8iG9PXLTPSwnGcVwS1PDLcB9wUKP8TVV2bvr41u2o5jtMIpjUIqroesAPfHcf5N8NMJhXvEpFN6SOFmVFCRO4UkQ0isqFcntlS1Y7jnFlO1yB8GlgNrAUOAp+wNlTVe1V1naquy+frnqDJcZxT4LQMgqoeUtWyqlaAzwBXza5ajuM0gtO6ZYvIElU9mH59K7Altn01Vs63jsgSWXPmdAfLh0ZtN0+lbLsWY8tdxVbCEsMlWRqzp1jyzXbuPiYiEZkVWxGNuDJHRsPuqIkJ282WL9iReL0LbdfoyICtfxNh/YdG7PyHQ6N2bsfubjt/ZouR6xKgzViWbWQivKQgQGeXfawDg/Zjr0bc2TJx1JQNjY4Gy5dE+kDeuJyRrlET0xoEEfkicAMwX0T2AXcDN4jIWpK42V3Ar8xMDcdxXglMaxBU9bZA8WfPgC6O4zQY/+uy4zgZbhAcx8lwg+A4ToYbBMdxMur6T6HWthYuunh5UNYciTAraTiCKx/JsqoV2z3UJHaizEjeUCZaw8k32xatNuuU87ZLrCyRZeMiUYYa8Y22tISPF1vhq60tnPgW4A2vv8GU7Vmxz5R1doTdrXv2HjDr7H5xrylriSSCLTXb+g9ouN7EsJ08dqDVdoEfyB00ZZGV6Ghrs4VrOsJJVguRiNfujnD/zjf5Um6O48wSbhAcx8lwg+A4ToYbBMdxMtwgOI6T4QbBcZyMurodm5tyLF3UHpR1N9tutqZSODJNIj7C8UgSzZjbrlSy3ZUnOs4Jlh9daSfQbIqEn2nejqprztnuynze3md3dziJ6cJI1GJvr5nfhjmd4UhTgHIkWrNSCbf/DWuuNuusOS+SiHTEjhZsbZtnykrtvcHyCexkuoVI9OeCrnD/BeiVcNQiQHPEG7ivFN7nU9//Z7POaC58Ly/HwnVrwEcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhORl29DLm80N4TDkSZM8eeve3qCgfKHO231R8v2zPxE6P2DHMx4p0QCc8+l8TWY8yUQC6yJFtupN+UFSIBU73zwjPu3d22t2Dx4sWmrKPNPtb8BeFgL4C+o33B8pUrwsFtAFdeeoEpG3lusynLz7H1ONgU9ro892J4WT6AcmRpuEohHIgEMDAaudoTtgdi1Ai0WrJ4oVmn2Bbu382P2NerFnyE4DhOhhsEx3Ey3CA4jpPhBsFxnAw3CI7jZLhBcBwno5aVm84DPg8sBirAvar6KRHpBb4MrCBZvekdqno8tq98SzNd5y8LysaNJbcAjh8Mu7COD9p58U707TJlHZEcfO0ddsBRU1tYx0KL7YrK5ewmzhsBKoCxEFqCRqQthi6HDx8x66xf/6gpW7F8kSkrNNtt1d4ebuMTW58w6/zDD+wVAfcM2a7iof5+U3bsRHh5uIHIknKVyCrlGrmHdkfcn3M77SX9VhsuxMtuutKsU+gNX+fWv7d/R7VQywihBPymql4EXAO8T0QuBj4IPKiqa4AH0++O45zFTGsQVPWgqj6Vfh4EtgJLgVuB+9PN7gfecoZ0dBynTpzSHIKIrAAuBx4HFk2uAJ2+23+rchznrKBmgyAincDXgF9TVXsd9pfXu1NENojIhuHB8FLljuO8MqjJIEjyJ/6vAV9Q1a+nxYdEZEkqXwIcDtVV1XtVdZ2qruuIZJtxHKfxTGsQRERIln/fqqqfrBI9ANyefr4d+Obsq+c4Tj2pJdrxWuDdwGYR2ZiWfQj4OPAVEbkD2AO8fbodqQjaFI7GKquddG7CyJ340n57WbB//dZ3TFlTJMqw0GQ3Sa417OrpMPIYAvTM6TFlvb3hfH8Aixfb7r6Ya3TjExuC5YOG+w2gs8V2VQ3Msd1l5SF76bV5S8Pu5Zbjtmf66PPPmrJjeTvfYrkYzrkJ0JoPX8/OeZH8jbE+EHMVR2SFyNJ8BQm7OSeMXKIABZ1ZVKO53+k2UNXvY7vFXz+76jiO00j8n4qO42S4QXAcJ8MNguM4GW4QHMfJcIPgOE5GXZOsCtAk4eW/8nl7WbDeuWG33tionbhyzjzbpVccsusNDdruORkPR9zt3LHTrNPWZrvtmgwXLEA+Z7tG8wVbNmjoX67YdY4cOmTKFre9zpSt6rKjPAf7w0uvjUdcvlakJkAztltamuw/vFUqYZd1RW03IBFZuWjLSsXI0nZlO3nv6PJwlOSRiDuV/nBC1/FIpGYt+AjBcZwMNwiO42S4QXAcJ8MNguM4GW4QHMfJcIPgOE5Gnd2OFXIaTpJy6NgLdr3xcDReT6edLLVn7lxTNhhZi1GN9RsBuowow/7+WL4Y2+aWI+tPDg3ZSUBz2PXMpK55223XH0lc89VvP2bKeiK3EzUSji5fbF+X40VbxxPDdpRksWy7AicMV2BZ7TYsFiNrNFbsYzVFXMUXXXi+KVv+6tXB8r6JfrNOyegD5Yh+teAjBMdxMtwgOI6T4QbBcZwMNwiO42S4QXAcJ6OuXobWfBMXdJ4TFkZm3Nt7wjPWYwfs2fHWSy41Zc9u22HKynMjwTdWrr0d28w6pZIdbJLkr7Vktq3OR2RIeKZ+Trcd7NXcYgdgHT16wpQdEXt5tdyR/mB5S8UO8nnb295qyv7iL+8zZfv2HzRlFeOe19phB0StucD2CLS32Z6t7c/aOSGPHg8HewEcOxZe0qTJviwUjGY0YgdrxkcIjuNkuEFwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3EypnU7ish5wOeBxUAFuFdVPyUi9wC/DBxJN/2Qqn4rtq/WXBMXdIeXKBsbs/PHjY6Eg00KE3bAyzmLwkuJATxjewlZuXqFKetoCQc+/fCRR8w65UjgTS4ScJQzF8sCMVyLAE3N4UCwQsG2/StX2W1FZAmyzc9uMmWtRu7E8WH7Om959jlTNjwSWzncdlmr4ebMqe3+7Gyx27c8Ybta1VhyEODHP3ralA0ZLtpf/8CvmHU6OsI/3QdavmfWqYVa/odQAn5TVZ8SkS7gSRGZPOqfqOofz0gDx3FeMdSytuNB4GD6eVBEtgJLz7RijuPUn1OaQxCRFcDlwONp0V0isklEPicidqC74zhnBTUbBBHpBL4G/JqqDgCfBlYDa0lGEJ8w6t0pIhtEZMOxE7FEIo7jNJqaDIKINJEYgy+o6tcBVPWQqpZVtQJ8BrgqVFdV71XVdaq6rndO92zp7TjOGWBagyBJBM5nga2q+smq8iVVm70V2DL76jmOU09q8TJcC7wb2CwiG9OyDwG3ichaEp/PLsD2kaRUUEaN6L/ByLJsx/sN96LtHWLrFtsl9uKO7aasqWAvJ9ZjjHDKJTvEzHYeQi4S7UgkolEj+QCtXXZ1hfNBAoyM2MvXrVy5wpQdPGBHUL6070CwfHvksXHLc8+bspiLViP3NSuidGzYdh8+8dhTpqwcWZJtTo89An7djdeastdcFo7MXbT8PLNO3kqd2WT331qoxcvwfcL9OvqfA8dxzj78n4qO42S4QXAcJ8MNguM4GW4QHMfJcIPgOE5GXZOsliswOB52Aw2M2a600Uo4yrCtzXbzLOi0l+O6fLUdinHg4C5TtuOZsMusXD69zJbFYiRyzm4OchHXqBWb2NnZadbZtetFUzY6ZkcZdvf0mLKjR/qC5e1z7CSlixaHI2EBBgZsd+VLLx0yZaNjVj+IuCoj/uxyJEnsuecuMWV3/PLtpqy1LRyhWoq4OEtG0GjMJV0LPkJwHCfDDYLjOBluEBzHyXCD4DhOhhsEx3Ey3CA4jpNRV7cjQMWI/xMrfAvIGbL2Nnt9vnOW2i6g3l47udPyY8dM2eHD4fX5+lcsNuuMRqI4JyIJO4eHbXffaNF2LRUrRrij2q7RCy+62JQdHxg0ZfsP2u6+iXI4qvXG615r1lm+zHYHHz9mr404MmInbn3iiSeD5bt37TPrKHZbtbTZLt9LL73QlJUn7H4wMDocLC/kbPdnwYiG1UhS31rwEYLjOBluEBzHyXCD4DhOhhsEx3Ey3CA4jpPhBsFxnIy6uh0FaJawy6yUtxOO5lrDrp7Ogh3BJxU7qq5UshOOLp5vR1CWV4XdYhPjtksploB1IhLtODpqR2uODNvH6+8PJ0zdtPOwWWdoqM2UTYzbbqyB/hOmTCrh6/zjDXYC02c3bjZl7R22jh2d9vVsbQp38eZCJFGt2ue8JpL4dGGPrcfB3XYC2fb2cL32VrsPkzN+upWwu7dWfITgOE6GGwTHcTLcIDiOk+EGwXGcDDcIjuNkTOtlEJFWYD3Qkm7/VVW9W0R6gS8DK0iWcnuHqhprriXkqNBGeIa8qSkyO9pk2K2IJ6FiLBkHUIzISkU7oKRUDjdXsTWyplwEa5mxRI+I/uN2rr2RoXCgTEe7HQi2cYcd6FMct4OsFnWEcwECZjCVDtueCdvnAuVBu61Gm2xZc0u477xmtZ2/sbvD9l4tWjjPlBWKtvenqWR7Sdok7EVrtTwJQIvxm4itDlgLtYwQxoEbVfUykqXfbxKRa4APAg+q6hrgwfS74zhnMdMaBE2YdG43pS8FbgXuT8vvB95yJhR0HKd+1DSHICL5dOXnw8D3VPVxYJGqHgRI3xeeMS0dx6kLNRkEVS2r6lrgXOAqEQmvXx1ARO4UkQ0isuH4gL3suOM4jeeUvAyq2g88DNwEHBKRJQDpe/C/sap6r6quU9V1c7vtyRrHcRrPtAZBRBaISE/6uQ14A7ANeACYXI7mduCbZ0hHx3HqRC3BTUuA+0UkT2JAvqKq/09EHgO+IiJ3AHuAt0+7JwErhqnQEl6uDUimMEMYATQAsRCPQs62g9pk61Ey8gQWi3YzipH7DkAjQTS02OdWabUDpuZ0hUdh8yPusksvWmHKBiOPeWPjtqPQWt6uVIosT1a0zyvmoi0U7DbO58Oy5mY7N2I+0gdaW21Xa0uLLbMCmABaDVlzZH+W/gXjfGtlWoOgqpuAywPlR4HXz+jojuO8ovB/KjqOk+EGwXGcDDcIjuNkuEFwHCfDDYLjOBmiaru3Zv1gIkeA3enX+UBf3Q5u43qcjOtxMmebHstVdcHpHqSuBuGkA4tsUNV1DTm46+F6uB5B/JHBcZwMNwiO42Q00iDc28BjV+N6nIzrcTL/rvRo2ByC4zivPPyRwXGcDDcIjuNkNMQgiMhNIrJdRJ4XkYYlZxWRXSKyWUQ2isiGOh73cyJyWES2VJX1isj3ROS59H1ug/S4R0T2p22yUURuqYMe54nIQyKyVUSeEZEPpOV1bZOIHnVtExFpFZEficjTqR6/l5af+fZQ1bq+gDywE1gFNANPAxfXW49Ul13A/AYc93rgCmBLVdkfAR9MP38Q+N8N0uMe4Lfq3B5LgCvSz13ADuDierdJRI+6tgnJusid6ecm4HHgmnq0RyNGCFcBz6vqC6o6AXyJJIPzvxtUdT1wbEpx3bNYG3rUHVU9qKpPpZ8Hga3AUurcJhE96oomNCTTeSMMwlJgb9X3fTSg0VMU+K6IPCkidzZIh0leSVms7xKRTekjxRl/dKlGRFaQJORpaGbvKXpAndukUZnOG2EQQrmwGuX7vFZVrwBuBt4nItc3SI9XEp8GVpMsynMQ+ES9DiwincDXgF9T1YF6HbcGPereJjqDTOczoREGYR9wXtX3c4EDDdADVT2Qvh8GvkHyONMoaspifaZR1UNpZ6wAn6FObSIiTSQ/wi+o6tfT4rq3SUiPRrVJeux+TjHT+UxohEF4AlgjIitFpBl4J0kG57oiIh0i0jX5GXgTsCVe64zyishiPdnhUt5KHdpEkgyqnwW2quonq0R1bRNLj3q3SUMznddr5nTKLOotJDO4O4EPN0iHVSQejqeBZ+qpB/BFkqFnkWTEdAcwj2SNzOfS994G6fE3wGZgU9oBl9RBj+tIHhs3ARvT1y31bpOIHnVtE+A1wI/T420BfjctP+Pt4X9ddhwnw/+p6DhOhhsEx3Ey3CA4jpPhBsFxnAw3CI7jZLhBcBwnww2C4zgZ/x9pJMz6xIk8qwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Blurred Image\n",
    "new_pred_blur, score_blur = get_prediction(net, alt_im, normalize_im=True)\n",
    "image_show(alt_im, new_pred_blur + \" \" + str(score_blur.item()))\n",
    "\n",
    "# Original\n",
    "image_show(image, pred+ \" \" + str(score.item()))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the blurred image is now predicted as a car rather than a truck."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also utilze this perturbation metric in conjunction with attribution results to identify a counterfactual example with the minimum number of of pixels ablated (based on attribution ordering) to misclassify the image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first obtain attribution results using FeatureAblation, using a feature mask which groups 4 x 4 squares of pixels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[[ 0,  0,  0,  ...,  7,  7,  7],\n",
      "          [ 0,  0,  0,  ...,  7,  7,  7],\n",
      "          [ 0,  0,  0,  ...,  7,  7,  7],\n",
      "          ...,\n",
      "          [56, 56, 56,  ..., 63, 63, 63],\n",
      "          [56, 56, 56,  ..., 63, 63, 63],\n",
      "          [56, 56, 56,  ..., 63, 63, 63]]]])\n"
     ]
    }
   ],
   "source": [
    "feature_mask = torch.arange(64).reshape(8,8).repeat_interleave(repeats=4, dim=1).repeat_interleave(repeats=4, dim=0).reshape(1,1,32,32)\n",
    "print(feature_mask)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from captum.attr import FeatureAblation\n",
    "\n",
    "ablator = FeatureAblation(net)\n",
    "attr = ablator.attribute(normalize(image), target=label, feature_mask=feature_mask)\n",
    "# Choose single channel, all channels have same attribution scores\n",
    "pixel_attr = attr[:,0:1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now define a method which perturbs the image to dropout a given number of pixels based on feature ablation results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def pixel_dropout(image, dropout_pixels):\n",
    "    keep_pixels = image[0][0].numel() - int(dropout_pixels)\n",
    "    vals, _ = torch.kthvalue(pixel_attr.flatten(), keep_pixels)\n",
    "    return (pixel_attr < vals.item()) * image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create a minimal perturbation object to find the minimum value of dropout_pixels needed for misclassification. We can also use binary mode rather than linear, which performs binary search between the given min and max ranges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "min_pert_attr = MinParamPerturbation(forward_func=net, attack=pixel_dropout, arg_name=\"dropout_pixels\", mode=\"linear\",\n",
    "                                     arg_min=0, arg_max=1024, arg_step=16,\n",
    "                                     preproc_fn=normalize, apply_before_preproc=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum Pixels Dropped: 304\n"
     ]
    }
   ],
   "source": [
    "pixel_dropout_im, pixels_dropped = min_pert_attr.evaluate(image, target=label, perturbations_per_eval=10)\n",
    "print(\"Minimum Pixels Dropped:\", pixels_dropped)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now take a look at this counterfactual example and corresponding prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP0AAAEICAYAAACUHfLiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjfklEQVR4nO2deZhdVZXof+veW/OQpDJUQkaGEIgoEQNGoBFBFPnaRtvWFn02vg9Ndz95T1t9/VBbxW61aVtxeM8JFUFUEFEU+3NCRIOIQIwhBMIUzFxkrtSQGu9d749zCm7d7HWqqLp1q5Kzft9XX92719nnrLPvWWfvs9dZa4uq4jhOeshMtgKO41QWN3rHSRlu9I6TMtzoHSdluNE7Tspwo3eclHHMGL2IbBGRV8afPygiXx/jfh4RkfPLqZtxnFYRWSMinSLymYk+nuM8i6oeE3/AFuCVz7PODcDHJ0nfDwM/BKRCx1sC3A0cBh5LaivgPcDTQAewC/gskCuSrwDuAQ4BO4CPFMkE+BCwLa5/C9BcJH8E6Cr6GwR+UiRXoLtI/nVDx1/H2xbr1QLcHtffCryl5Py15NgfLtnnGcCaWLYbeHeR7GzgAaAT2ACcWyR7BfAw0A7sj3WYXySvAa6P2+MZ4L0lxx3TOcf7/UZ8rp3An4DXjHgtTMYFP4oLNDeGOkeb0X896dhjaYMRjncfcC1QB7whvkBnG9ueCEyPP7fEF9t7i+SPAp8AsvG2bcBfxbLLiW4qC4FG4MfAjcZxhOjm8ndFZQqcNMK5vDU2zlKjvxn4Xnzcc4luSi+IZUtKty/Z5yxgT7zvGqAJOLWoDfYBb4zP+b8BB4EZsbwVOC7+XAN8CrijaN//TnSTnAGcGhv+xeM9Z6ABuDo+twzwl7HxL0ncVwUv8i3AB+IL5iDwTaA2lp1P1GP8n7hBbopP4ipgM9Hd81agpWh/byO6w+0n6lm2EBt93BDfLtr2XOD3RBf6duDtwGpgAOgnurv+pEjPof3UAJ8j6u12xZ9rSnR+X3yxtAH/fZRtcUPJsV8Z63wb8G2iHuEdwHHAHcAB4CngnUX7qANujNtyE/DPwA7jeCcDfUBTUdk9wD+MQteZwK+ALxWVHQaWF33/PvCB+PNtwP8ukp0N9AL1gX2/PD7/htEaADANeAJYxZEG0A+cXLTtTcA1ozT6TwI3GbK/BB4pKXsCuCKwbQ2RkT9aVLYTeFXR938DbhnvORvbbgDekPSbVvqZ/q3Aq4l6h5OBfymSzSW6oy4mMsj/BbyO6MI4juji/iKAiCwHvkxk+McRXZgLQgcUkUXAz4D/C8wmGpquV9XrgO8An1LVRlV9baD6h4gaegVwOnBWQOdpwHzgCuCLIjIjPu5bRGRDSCdVfXvJsX8Viy4lMprpsfxmohvLccDfAJ8UkQvjbT9KdCGfAFxE1PtYvAB4WlU7i8oeisuDxPp3EPVwpwNfLRJ/Dvg7EakSkWXAy4huDBD13lK8KyJDWBo4zOXAbaraXVK+RkSeEZEfisiSEtkniX77Z0rKTwbyqvrECOe4VUR2iMg3RWRWUfkq4ICI/F5E9ojIT+JrJ3ROQ2WnPftFZJGItAM9wPuJenvi6+G4WJckvcZyzsMVEmklaodHkrardE//D0XfLwE263O9Zj9xzx+XbQIuLPo+j6h3zAEfYfidcuguf0RPTzS6uN3Q6QZKhtgM7+k3A5cUyV4NbCnSuYfhw8s9wKpRtsewY8c6ryn6vhDIM7x3/nfghvjz08Cri2TvwO7p3wb8oaTsE0P7GkHPpUQ909yisrOJRh6DRD3Px0r0eILohjSNaKSiwMtK9ltPNKI5v6T8PKCa6Mb3/4CNPNebrwTWx9fAEob39H8BPFOyr3cCv4k/N8b1c0TD8duAXxRt+wTRSPBMoBb4AnBvLJsZyy4DqohuVgXgq4H2aiEasa4q+h2V4df2RUPX0XjOueS4VUQ33iN0Kv2rdE+/vejzVqI74BB7VbW36Pti4HYRaY/voJuIjKA1rvfsvjTqKfYbx1xIZLxj4bhYT0vn/ao6WPT9MNHFNVaK2+c44IAO7523Eo0qhuTF2xd/LqULaC4payZ6/ktEVZ8k6jm+BCAiLcDPgX8lMo6FwKtF5H/EVa4nGqH8Jq53d1y+o2TXf0302PLbkuOtUdV+VW0H3g0cD5wqIplYh3eXtPmozlFVu1R1raoOqupu4ErgVSIyVKeHqHN4ML4OPwacLSLTVHU/0SjsvUQTfBcTGVjpOaGqB4geu34sIrlYryFdjtBrnOcMQLzdTUQd35XWdkNU2ugXFn1eRPScPISWbLudaCZyetFfraruJHp+fnZfIlJPdDcOsZ3ocSJE6TFL2UV087F0LjfF+uwCWkSkqeT4O+PPbQx/pClu21IeAU4o2dfpjDQMfI4cz7XhCUTD6G/FBrSDaIb+EgBVLajqR1V1iaouiI+xs0jvIS4HvqVxN5WAEg2lm4l6ve+JyDPAg7F8h4j8BVFPnROR4seIpHMcOu7QsH0Dw9t/mFxVf6uqZ6pqC9HIaRnRbH6IHDCHyGtxkOi3On2Ueg0dezTnjIgI0Qx+K9Gz/EDCfuO9V3Z4/zDRhdpCNJH0SX1uqLyjZPt/IuotFsffZwOXxp9fQHQHPZdoWPRpoqFmaHi/iOiu+qb4x5gJrIhl1wDfDeg5tJ+PE00Aziaa3f0d8ZDc0PnZuqNojxs4cnj/7ZJt7iEa7tUCLyLqZS6KZf9B1IvOIOr915fqU7KvP8TtVAu8nuTZ+3cAc+LPy4ku0Gvj781x3bcQdRpziTwDn9DnhrcnEl20y4mGqqtL9r8g/r1OLCl/AdH8SZZoxPQ54HGioavExxr6O5PIOOYD1XH9W4hGGQ3AOQyfvX8pkaFm4mvge8DdRce+gGjeaEV8vM8C9xTJXxyXN8d63Vsk++uifc8mmnReVyS/hmhEMwM4hegmcHGZzvkr8W/bOGpbrLDRD83etxMNgeoTDChDNJx6nMhoNxPfJGL55US+4NHM3v8FcD/RM+R24PK4fCmRsbQDPwoY/dCzXVv89wVKPA4JN4y3UjLjOwajXwD8F9EweDPD50QaiIZ07USPPv9CPEdiHG8J0U20J27TV5a0T1fR928S3WC643P6T4Y/k15A1OscIppc+lrRb3lyvP/DRI8j7w3o8gGKDKpkv4/Hx90D/AhYmnA+w55viW44P4rrb2O4n/4y4M+xrA34FkXzFPE2/0g0IjkI/ARYWCS7OT7fQ0Q3jDlFsv9ZtO9niG4+i4vkxX763Qx3f475nIlGoUrkHSl+/+CtSbYoceUJR0S2AO/Q52aqnTIiIv8IvFlVXz7ZujhTm2PmNdy0ISLzROQcEcnEbrP3Eb0J5jiJ5CZbAWfMVBP5zo8nGuLfQjzD7jhJVGx47zjO1MCH946TMio6vBeRig0rznjJGWOql6ShjujWf57HktI3O4uPlVCvrFo4z4eewz2mbDBvvj9DY6P9zlbSdfB82bJlK/v27Uvc4biMXkQuBj5P5GP8uqpeM579lZP7HrjXlGUSLCrR6I1HIR3jb5bJ2AOtpMeupItErFvCRNwpynyDLOfFPxKJ2hVs6aaHHjZl+9sPmLJV555jyqqrq4PlY7kGzly5yqwzxJiH9yKSJQqAeQ3RSxiXxYEwjuNMYcbzTH8W8JSqPq2q/USzx5eWRy3HcSaK8Rj9fIYHeezguWCQZxGR1SKyVkTWjuNYjuOUifE804ceKo54CNEobv06qOxEnuM4YcbT0+9geGTXAiY2As1xnDIwnp7+QWCpiBxPFKTwZqLIqymBOYsNDAz0m7L1f7CiJaG3J+yuWfXy88w6uSq7iQcGbRdPTuz7cUEKtswozyTN+CeMvwqJ89wJs8tjcBdoUp2kmf2kWe7nLQAZtNt366YnTNnmp582ZWeseqkpy+aMa2SMHpyRGLPRq+qgiFwJ/ILIZXe9qo42PttxnEliXH56Vf0p8NMy6eI4TgXw13AdJ2W40TtOynCjd5yU4UbvOCnjmE2ikUnwaGzf9mdT9vi6P5qybDbcXIdOOy1YDtA4Y5q9v/paU9axxw7eqK4PB2gA5Ix9FhJ8VNkxhvQluY36e41otIRglqr6BlOW5DqUBFnBdL/ZbrlcVdaUtbTYv+ee7bY5dR06aMrq644LlifmuhjHa27e0ztOynCjd5yU4UbvOCnDjd5xUoYbveOkjGN29n77Y4+ZsjV3/MSUHdppBwrOmh+eZV13n52aq35akyk787xzTdnv7/61KTtp2TJTtmzFC4PlA5o36yS5Onq7S1eRfg4rzRPA5iefCpb3He4NlgOcebadUqpvwF6irdrwqgDs2n3EGpMA7NtvrXcKJyyzlj6E3XvbTNnerfY6qX+88xem7IK/eXOwXDK2FyGbDffXo8lu7T2946QMN3rHSRlu9I6TMtzoHSdluNE7Tspwo3eclFFRl90ZL3kx990fdm9pQl6ye39xV7D8vnt/b9a54zs3m7LagS5TNtjdbsqefqIjWP72qz5s1pkIHr73blM20Bd2bfUluHKSgpM2PbTelDU3Npuyvq7DwfLDnbYLMN/XZ8oKxnkB9OVs11ZXR2dYjwPhcoC2zdtM2SMP/MGUzRLbLbpznZ0B/pETjg+WL1p6qlmnuXl6sHw069F6T+84KcON3nFShhu946QMN3rHSRlu9I6TMtzoHSdlVNRl1324lwcfCi+Cs2vbTrPewK5wpNTBp5806zTPmWHKZs5sNGXZRjtv3e4OO0Kskqz79W9N2cNrNwTLBxOXfrKX19q1zY4cq09YsqvJcOd199rH2vmEvWSUDNrusKrmelPW3RV2EXbvs3PWWatMATTnjdx/wJwWO6JyIMElvfGXPw+W795hR3yed8lrg+Wq9nGGGJfRi8gWoBPIA4OqunI8+3McZ+IpR0//ClXdV4b9OI5TAfyZ3nFSxniNXoFfisgfRWR1aAMRWS0ia0VkbftB+znKcZzKMF6jP0dVzwBeA7xLRI5YqF1Vr1PVlaq6cvoMe3LNcZzKMC6jV9Vd8f89wO3AWeVQynGciWPME3ki0gBkVLUz/vwq4F+T6hzq7OJnd4cj4/bt3mvWO7k6HGF1yix7GaSmGjvyarDPdhs1VNWYshk1CcklK8iOtt2mrL47HKmWL/TbOxy03VCN9XWmrLpgR8VpT/hRrqvTbvsOw70G0JcQnTdj4TxTNqt1drB8y+Zw4k6A2oS2Wjy3xZQdMtoeoCZbZcqy7XuC5dt0o1mn/4KLguWjSYw5ntn7VuD2eD2zHPBdVQ07HB3HmTKM2ehV9Wng9DLq4jhOBXCXneOkDDd6x0kZbvSOkzLc6B0nZVQ0yq6/r4+tT20Jyg50tJv1Fs0OR8XNydpuue5q+9Rk5ixTlk1wG9XI1LhHLjvv5aZs/oIFwfLaWtsVOThgu6iyCW0s2BFdA33hfZ6idhbOArZM+21X1EDOluU07Ebb8VhCRF+HHUrSl3AJHMrY0X7N1bbLriEXTiIq/fbv0t8VTu6q+ZHdylPjKnYcp2K40TtOynCjd5yU4UbvOCnDjd5xUkZFZ+8H+gbYvS2c7+7PW7eY9RaeNDdYvni2nZPsii/e8HxUGxc//5kdcqAJM/75hNiIbMau198Xnu0FyFSFZ4mr6+yZ5abpdhBJLmHJKBJm2y1RPmF2OWFv5NS+VAeydkMe2hvOvTizyc6TONhtBzRl8nbAUFWNnV8xyQuSGwjvM9t5wKyz+U/h5bX6DtvepyG8p3eclOFG7zgpw43ecVKGG73jpAw3esdJGW70jpMyKuuy6+9n+7ZtQVlfr52nbf32sNtl0fEvLYte4+VQr50brZC3g1IGB8O5/wByCS4eLdhuo8FC+HiHDnWYdXbtesyUDQzYOvb32+c9bdr0YHlDg53XcP9eO0/iQJ/tlsvnbGffzLqwrL2z06wjCcE9TUa+RoDaQrspy/Tb/etgrRGMk+AevO+ee4LlXUYgzjBdRtzCcZxjCjd6x0kZbvSOkzLc6B0nZbjRO07KcKN3nJRRUZddQQv0GXm/NMF9NW/JacHyzIJTyqLXeKlPyD+X5NYa6E9wvWG7jQpquwHJhF19v/z5L8wqv1sTdv8ANDVPN2W9Rh48gHPOPidYvnz5crPO7+8NR44BdPfZ10chIe/eeWe+MFg+b1E4lyBArt7+PevrbJM5qAku2AH7N6s+HHaz9Q7a7XvAunQKZciRJyLXi8geEdlYVNYiIneKyJPxf1+O1nGOEkYzvL8BuLik7CrgLlVdCtwVf3cc5yhgRKNX1TVAaTT/pcCN8ecbgdeVVy3HcSaKsT7Tt6pqG4CqtonIHGtDEVkNrAbI5uzc347jVIYJn71X1etUdaWqrsxmKzpv6DhOgLEa/W4RmQcQ/99TPpUcx5lIxtr13gFcDlwT///xqGopMBh2KdTW1ZnVTl/5kmD5tObmUR12otm5c5cpS0oE2dRoJ2fMJiTUzCYkq9y9O5zUcc2aNWadl555lik7/oQTTdn+A3bixrlzW4Plc1rtJcXOveB8U5YzEn4C5I3IQoBcNtyO+cHZdp1FJ5uyQsbWIzdoR4rmd241ZT27w9dPPm+7APceOBQsHzSSbBYzGpfdzcB9wDIR2SEiVxAZ+0Ui8iRwUfzdcZyjgBF7elW9zBBdWGZdHMepAP4aruOkDDd6x0kZbvSOkzLc6B0nZVT0bRnVAgMD4aizWa3zzXq5qnDUU68RsQdw03e/b8qSkjNaUYAAO7b8OVj+58efMOtYUW8AJy09yZTlGuy1597yt39rH28M/PquO8u6v7HygY9/2hYmrPuXICJvJCb9z6v/eXRKlYmbv/IlU7bDiFas7rWvxVyuOiyQpBUBI7ynd5yU4UbvOCnDjd5xUoYbveOkDDd6x0kZbvSOkzIqHuBeMCKiZs+2o54GjKSIuUbb9VZdbbg0EnQA0IS159SI9KqfaacIrK+ym7gqQaZiJ2c8VskkrN+3b99BU9bd3W3Kevt6x6VTucjV2xGhjx8KR8bNTkiMWTczHMWYSXARP7vNiFs4jnNM4UbvOCnDjd5xUoYbveOkDDd6x0kZFZ29l0yG2traoOz4E44362Wy4SCChDRyFBKW98kYedMioR2+saguLGuWTrOOJngKenSaKeumyZQdq7Qf6DBljz32lCkbGLBzyfX1To3Z+46ehCXMNOxp6u1vN+scOBDOhTiQkFdvCO/pHSdluNE7Tspwo3eclOFG7zgpw43ecVKGG73jpIzKuuyATCbsfps1a7pZr3VueCmkrsO2iwex9zeotjuvoHaQQ9OB8NJEdVvXmnUGxHYB9qywg4z6q+0lr45V9iYE1RxsbzdlhYSlnKSQlEGvcmxvt5cAy9aGg6ty/Xa+u72dh4PlgwkBY0OMZlmr60Vkj4hsLCq7WkR2isj6+O+SEY/kOM6UYDTD+xuAiwPln1XVFfHfT8urluM4E8WIRq+qawB7bOI4zlHFeCbyrhSRDfHw38wiISKrRWStiKzN50deRtdxnIllrEb/ZeBEYAXQBnzG2lBVr1PVlaq6MputeKIex3FKGJPRq+puVc2ragH4GnBWedVyHGeiGFPXKyLzVLUt/vp6YGPS9sVYObwaGmwX1bRp4fxiXT22y66Qt91ySUv/JK0KJIY7b7A3wR1TXWfvsD8hErAw8vJExxq3fvNzk63ChNG2b78p6+rpCZbPS7gGskYwXYKH+FlGNHoRuRk4H5glIjuAjwLni8gKomXEtgB/P/KhHMeZCoxo9Kp6WaD4GxOgi+M4FcBfw3WclOFG7zgpw43ecVKGG73jpIyKvi1TW1fDqcsXB2XV1bavYVDDkUPZhMyYWrDf/qsSe+mfwQSXR39tOJFlXeuJZp18NrwUFkBeEpbQYuRoKefo4fENtld7aUM4MWYuIdKyuSF8fWerfFkrx3FKcKN3nJThRu84KcON3nFShhu946QMN3rHSRkVddlVV2WY31oflDVX2y6qqsG+YLkk+Nf6EtY304RQusFB29V3qOG4YPn+48OJOwGqEsKeNNtgyqoztqvPOfq48q9WmbIdg2GbWPe7X5l1ejLh/jqfFCYa4z2946QMN3rHSRlu9I6TMtzoHSdluNE7Tsqo6Ox9JivUT68NyqZNC89gAjQ1hfPM7W+31e/L27Pm/T320lUDCbP+IuEZ9UGx9eg1JZBRe6Y1c7g9oaZztLHvkH0l9HR3BsvnzZ1j1hmoC1/f1feM7PXxnt5xUoYbveOkDDd6x0kZbvSOkzLc6B0nZbjRO07KGM0KNwuBbwFzgQJwnap+XkRagO8BS4hWuXmTqh5M2le2ppqmkxYFZX3VNWa9g237wuWdYVcHwIffX7lFd77w1a+askzGbuKsETQBMFUWtbr+xm+asly1HTBUZ/yctfu3mXVe+873j1qvo43b7nrAlJ1ouN9Ov/hMs06uJZxXr/b7th0NMZqefhB4n6qeCqwC3iUiy4GrgLtUdSlwV/zdcZwpzohGr6ptqrou/twJbALmA5cCN8ab3Qi8boJ0dBynjDyvZ3oRWQK8GLgfaB1auTb+b78+5DjOlGHURi8ijcAPgPeoqr1G9JH1VovIWhFZ2915eCw6Oo5TRkZl9BK9dP4D4Duq+sO4eLeIzIvl84A9obqqep2qrlTVlQ1N9vv1juNUhhGNXkSEaGnqTap6bZHoDuDy+PPlwI/Lr57jOOVmNFF25wBvAx4WkfVx2QeBa4BbReQKYBvwxpF2pCJoVTgKKK/2cjz9Ri68Z3buGumQFeFHt9xqyqZPm27KWlpaTNncua3jUalsdLTbT3L5ru2mbOb8sGu25mCiV/eYpTZnm1pOwnkZ+43ckAA5HXsOxRGNXlV/h+02vnDMR3YcZ1LwN/IcJ2W40TtOynCjd5yU4UbvOCnDjd5xUkZFE2MKUCX5oCybDZcDtMxoCpb39vSUQ61xs2HtelNWVxdO6glQZbgvAbKZqRFn17Nriyk7oSkc6QXQ2b4/WN6XkAz0WKbjUJcp61k8LVi+d8B22dEeTrTZl7eXZRvCe3rHSRlu9I6TMtzoHSdluNE7Tspwo3eclOFG7zgpo8IuuwIZDSfS2H3gabteXzjZ3/TG8Lp4laamxnbLJd1X8wnr7XV1dY9Do/Jx28/vM2XTE7oMnRZ2Qy2eO2O8Kh2VzJk93ZQtfuGJwfJ9/e1mnUHC106+UBhRF+/pHSdluNE7Tspwo3eclOFG7zgpw43ecVJGRWfva7NVLGs8LixMmMmunx6eCe7dNTVSau/ct2OyVZgw9u8/ZMr2Sr8py+xtD5bXFAbMOt/4wudM2Ve+eoMp276zzZQVjH6ttsHOzLx02UmmrL7O9hg9/uijpmz/wXAAEsCBA+ElI6oSnEI5oxmNeLZheE/vOCnDjd5xUoYbveOkDDd6x0kZbvSOkzLc6B0nZYzoshORhcC3gLlAAbhOVT8vIlcD7wT2xpt+UFV/mrSv2kwVy5rDyzX19tr5wHoOh/OB5frTuURSJXnR6S8wZQ8/usGU1Rq58Pq67d9546NPmrLuw0nuWdvdq4aLMKO267Cxxl5iLd9vuynVWH4N4E8PPGTKugz35j+9++/NOg0NYdO9o+ZOs84Qo/HTDwLvU9V1ItIE/FFEhvb8WVX99Cj24TjOFGE0a9m1AW3x504R2QTMn2jFHMeZGJ7XM72ILAFeDNwfF10pIhtE5HoRSWegtOMcZYza6EWkEfgB8B5V7QC+DJwIrCAaCXzGqLdaRNaKyNoDh+xljx3HqQyjMnoRqSIy+O+o6g8BVHW3quZVtQB8DTgrVFdVr1PVlaq6smVac7n0dhxnjIxo9CIiwDeATap6bVH5vKLNXg9sLL96juOUm9HM3p8DvA14WETWx2UfBC4TkRVE/pItgO1fiCmg9AyGl93pTFii6mC74ZqzPStOmWhtDUeAAbTtajFlz+zYFSx/POERb+OTT5myTNb+sTWh74r6rCPp7bZdbw/et86U5fO2q2/adHsk+4oLzjFlLzr9tGB56+KFZp2sYbnZKnupsSFGM3v/O6Jl6EpJ9Mk7jjM18TfyHCdluNE7Tspwo3eclOFG7zgpw43ecVJGRRNj5gvQ2Rd2oXT02hFKPYWqYHldnb/sM9E8uslO9tg8fbop2793X7C8fpqdWLJ1bjgCE6Cjw3b1PfPMblPW0xuO0Ezq7yTBF5xPSOy5YME8U3bFOy83ZbV14WXbBhPcg4NGsKKqbUdDeE/vOCnDjd5xUoYbveOkDDd6x0kZbvSOkzLc6B0nZVTUZQdQCMbugFhhQ0DGkNXX2euRHcucdcpyUzZQCLfvzLnGGoJAbaPt+jzY0WnKdrbZrrL+fDia8oJzzzbrfP2G60xZuZnVNNeUKfaCcDV1dhTbaaedYsry/XYUaUdPd7A8l7FdhzkJ99eaL5h1hvCe3nFShhu946QMN3rHSRlu9I6TMtzoHSdluNE7TsqoqMtOgGoJRwENZsOuJoBMbdhN0phrLIdaRx2vu/DFpqy9vStYvmHzHrNOV1edKevvs11AHe2HTJkUwr/zn9baSScrSXXOjkYTtc95aUKyyjnTG0xZ21Y76Wd9fbhefa0dkUjGMN1C2FU6rOqIWziOc0zhRu84KcON3nFShhu946QMN3rHSRkjzt6LSC2wBqiJt79NVT8qIi3A94AlRMtavUlVjfWnIjIUqCMceFBVlTDrWGXcmwoJs5vHMCcfv8iUHe4KB2801NvBSeuf2GHKBvoOm7LWhnBuNwA0HLSi3faMfyV50Yl2Pr7mBtsr1DpnpinLDdhBNVWDtoekTsLeqVprhh6oMWzCWMVrGKPp6fuAC1T1dKJlqS8WkVXAVcBdqroUuCv+7jjOFGdEo9eIIedvVfynwKXAjXH5jcDrJkJBx3HKy2jXp8/GK9buAe5U1fuBVlVtA4j/28ubOo4zZRiV0atqXlVXAAuAs0QkvLZuABFZLSJrRWTtwY7w22KO41SO5zV7r6rtwG+Ai4HdIjIPIP4ffM9TVa9T1ZWqunJGczpfm3WcqcSIRi8is0Vkevy5Dngl8BhwBzC0bMflwI8nSEfHccrIaAJu5gE3ikiW6CZxq6r+l4jcB9wqIlcA24A3jrgnASuuJlcTXroKiKYNQxhBHcc6LdPsnHbTmsKjqVkJrqbTTl1iyjoTHsl6+/pNWT4fdtkNDtpLNa279klTVm7e8OqXmbJslX0t1tbabsqaGltmBdUA1Bqy6oT9VVeH3Xy57MiD9xGNXlU3AEeEdanqfuDCEY/gOM6Uwt/Ic5yU4UbvOCnDjd5xUoYbveOkDDd6x0kZolo5t5eI7AW2xl9nAfsqdnAb12M4rsdwjjY9Fqvq7KQNKmr0ww4sslZVV07KwV0P1yPFevjw3nFShhu946SMyTT6yq1LnIzrMRzXYzjHnB6T9kzvOM7k4MN7x0kZbvSOkzImxehF5GIReVxEnhKRSUuoKSJbRORhEVkvImsreNzrRWSPiGwsKmsRkTtF5Mn4/4xJ0uNqEdkZt8l6EbmkAnosFJG7RWSTiDwiIu+OyyvaJgl6VLRNRKRWRB4QkYdiPT4Wl5enPVS1on9AFtgMnABUAw8ByyutR6zLFmDWJBz3POAMYGNR2aeAq+LPVwH/MUl6XA28v8LtMQ84I/7cBDwBLK90myToUdE2IVrrtTH+XAXcD6wqV3tMRk9/FvCUqj6tqv3ALUSZdVODqq4BDpQUVzy7sKFHxVHVNlVdF3/uBDYB86lwmyToUVE0YsIyUE+G0c8Hthd938EkNGyMAr8UkT+KyOpJ0mGIqZRd+EoR2RAP/yf8MaMYEVlClLRlUjMul+gBFW6TicxAPRlGH0qYNVl+w3NU9QzgNcC7ROS8SdJjKvFl4ESihU3agM9U6sAi0gj8AHiPqnZU6rij0KPibaLjyEA9EpNh9DuAhUXfFwC7JkEPVHVX/H8PcDvRo8dkMarswhONqu6OL7gC8DUq1CYiUkVkaN9R1R/GxRVvk5Aek9Um8bHbeZ4ZqEdiMoz+QWCpiBwvItXAm4ky61YUEWkQkaahz8CrgI3JtSaUKZFdeOiiink9FWgTERHgG8AmVb22SFTRNrH0qHSbTHgG6krNSJbMTl5CNDO6GfjQJOlwApHn4CHgkUrqAdxMNEwcIBr5XAHMJFoT8Mn4f8sk6XET8DCwIb7I5lVAj3OJHvE2AOvjv0sq3SYJelS0TYAXAX+Kj7cR+EhcXpb28NdwHSdl+Bt5jpMy3OgdJ2W40TtOynCjd5yU4UbvOCnDjd5xUoYbveOkjP8P7RmHj63F0jMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Feature Dropout Image\n",
    "new_pred_dropout, score_dropout = get_prediction(net, pixel_dropout_im, normalize_im=True)\n",
    "image_show(pixel_dropout_im, new_pred_dropout + \" \" + str(score_dropout.item()))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Removed portions include the front of the truck as well as some portions of the background, causing the model to misclassify the truck as a frog."
   ]
  }
 ],
 "metadata": {
  "bento_stylesheets": {
   "bento/extensions/flow/main.css": true,
   "bento/extensions/kernel_selector/main.css": true,
   "bento/extensions/kernel_ui/main.css": true,
   "bento/extensions/new_kernel/main.css": true,
   "bento/extensions/system_usage/main.css": true,
   "bento/extensions/theme/main.css": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}