notebooks/Transfer_Learning.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sLMSHWM-oiyf"
      },
      "source": [
        "# Transfer Learning with skorch"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2KJVmPqYoiyj"
      },
      "source": [
        "In this tutorial, you will learn how to train a neural network using transfer learning with the `skorch` API. Transfer learning uses a pretrained model to initialize a network. This tutorial converts the pure PyTorch approach described in [PyTorch's Transfer Learning Tutorial](https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html) to `skorch`.\n",
        "\n",
        "We will be using `torchvision` for this tutorial. Instructions on how to install `torchvision` for your platform can be found at https://pytorch.org.\n",
        "\n",
        "<table align=\"left\"><td>\n",
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/skorch-dev/skorch/blob/master/notebooks/Transfer_Learning.ipynb\">\n",
        "    <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>  \n",
        "</td><td>\n",
        "<a target=\"_blank\" href=\"https://github.com/skorch-dev/skorch/blob/master/notebooks/Transfer_Learning.ipynb\"><img width=32px src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a></td></table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uFUihT98oiyp"
      },
      "source": [
        "**Note**: If you are running this in [a colab notebook](https://colab.research.google.com/github/skorch-dev/skorch/blob/master/notebooks/Transfer_Learning.ipynb), we recommend you enable a free GPU by going:\n",
        "\n",
        "> **Runtime**   →   **Change runtime type**   →   **Hardware Accelerator: GPU**\n",
        "\n",
        "If you are running in colab, you should install the dependencies and download the dataset by running the following cell:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "i1U_Mi4gon2r"
      },
      "outputs": [],
      "source": [
        "import subprocess\n",
        "\n",
        "# Installation on Google Colab\n",
        "try:\n",
        "    import os\n",
        "    import google.colab\n",
        "    subprocess.run(['python', '-m', 'pip', 'install', 'skorch', 'torchvision'])\n",
        "    subprocess.run(['mkdir', '-p', 'datasets'])\n",
        "    subprocess.run(['wget', '-nc', '--no-check-certificate', 'https://download.pytorch.org/tutorial/hymenoptera_data.zip', '-P', 'datasets'])\n",
        "    subprocess.run(['unzip', '-u', 'datasets/hymenoptera_data.zip', '-d' 'datasets'])\n",
        "except ImportError:\n",
        "    pass"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "8EctsAZCoiyy"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from urllib import request\n",
        "from zipfile import ZipFile\n",
        "\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "import numpy as np\n",
        "from torchvision import datasets, models, transforms\n",
        "\n",
        "from skorch import NeuralNetClassifier\n",
        "from skorch.helper import predefined_split\n",
        "\n",
        "torch.manual_seed(360);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_vaY9ew5oiy1"
      },
      "source": [
        "## Preparations"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ybj2whoboiy3"
      },
      "source": [
        "Before we begin, lets download the data needed for this tutorial:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vXrGCvunoiy4",
        "outputId": "155c8008-d1e7-4c9f-bbfc-0abbe3d61cbd"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Data has been downloaded and extracted to datasets.\n"
          ]
        }
      ],
      "source": [
        "def download_and_extract_data(dataset_dir='datasets'):\n",
        "    data_zip = os.path.join(dataset_dir, 'hymenoptera_data.zip')\n",
        "    data_path = os.path.join(dataset_dir, 'hymenoptera_data')\n",
        "    url = \"https://download.pytorch.org/tutorial/hymenoptera_data.zip\"\n",
        "\n",
        "    if not os.path.exists(data_path):\n",
        "        if not os.path.exists(data_zip):\n",
        "            print(\"Starting to download data...\")\n",
        "            data = request.urlopen(url, timeout=15).read()\n",
        "            with open(data_zip, 'wb') as f:\n",
        "                f.write(data)\n",
        "\n",
        "        print(\"Starting to extract data...\")\n",
        "        with ZipFile(data_zip, 'r') as zip_f:\n",
        "            zip_f.extractall(dataset_dir)\n",
        "        \n",
        "    print(\"Data has been downloaded and extracted to {}.\".format(dataset_dir))\n",
        "    \n",
        "download_and_extract_data()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V0K5OAKPoiy7"
      },
      "source": [
        "## The Problem"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m52cP6kaoiy8"
      },
      "source": [
        "We are going to train a neural network to classify **ants** and **bees**. The dataset consist of 120 training images and 75 validiation images for each class. First we create the training and validiation datasets:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "y3lg-xiaoiy9"
      },
      "outputs": [],
      "source": [
        "data_dir = 'datasets/hymenoptera_data'\n",
        "train_transforms = transforms.Compose([\n",
        "    transforms.RandomResizedCrop(224),\n",
        "    transforms.RandomHorizontalFlip(),\n",
        "    transforms.ToTensor(),\n",
        "    transforms.Normalize([0.485, 0.456, 0.406], \n",
        "                         [0.229, 0.224, 0.225])\n",
        "])\n",
        "val_transforms = transforms.Compose([\n",
        "    transforms.Resize(256),\n",
        "    transforms.CenterCrop(224),\n",
        "    transforms.ToTensor(),\n",
        "    transforms.Normalize([0.485, 0.456, 0.406], \n",
        "                         [0.229, 0.224, 0.225])\n",
        "])\n",
        "\n",
        "train_ds = datasets.ImageFolder(\n",
        "    os.path.join(data_dir, 'train'), train_transforms)\n",
        "val_ds = datasets.ImageFolder(\n",
        "    os.path.join(data_dir, 'val'), val_transforms)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qfbGs4wfoiy_"
      },
      "source": [
        "The train dataset includes data augmentation techniques such as cropping to size 224 and horizontal flips.The train and validiation datasets are normalized with mean: `[0.485, 0.456, 0.406]`, and standard deviation: `[0.229, 0.224, 0.225]`. These values are the means and standard deviations of the ImageNet images. We used these values because the pretrained model was trained on ImageNet."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "APIEJMARoizA"
      },
      "source": [
        "## Loading pretrained model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TUNjvYdwoizA"
      },
      "source": [
        "We use a pretrained `ResNet18` neural network model with its final layer replaced with a fully connected layer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "cegvloOPoizB"
      },
      "outputs": [],
      "source": [
        "class PretrainedModel(nn.Module):\n",
        "    def __init__(self, output_features):\n",
        "        super().__init__()\n",
        "        model = models.resnet18(pretrained=True)\n",
        "        num_ftrs = model.fc.in_features\n",
        "        model.fc = nn.Linear(num_ftrs, output_features)\n",
        "        self.model = model\n",
        "        \n",
        "    def forward(self, x):\n",
        "        return self.model(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "al8ehonloizD"
      },
      "source": [
        "Since we are training a binary classifier, the output of the final fully connected layer has size 2."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vNmUTqT6oizD"
      },
      "source": [
        "## Using skorch's API"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j2hJ-CUUoizE"
      },
      "source": [
        "In this section, we will create a `skorch.NeuralNetClassifier` to solve our classification problem. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ob1FqKNloizF"
      },
      "source": [
        "### Callbacks"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aHVRYecRoizF"
      },
      "source": [
        "First, we create a `LRScheduler` callback which is a learning rate scheduler that uses `torch.optim.lr_scheduler.StepLR` to scale learning rates by `gamma=0.1` every 7 steps:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "ZJEUie2VoizG"
      },
      "outputs": [],
      "source": [
        "from skorch.callbacks import LRScheduler\n",
        "\n",
        "lrscheduler = LRScheduler(\n",
        "    policy='StepLR', step_size=7, gamma=0.1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bli6ngl5oizG"
      },
      "source": [
        "Next, we create a `Checkpoint` callback which saves the best model by by monitoring the validation accuracy. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "rGPmqqhWoizH"
      },
      "outputs": [],
      "source": [
        "from skorch.callbacks import Checkpoint\n",
        "\n",
        "checkpoint = Checkpoint(\n",
        "    f_params='best_model.pt', monitor='valid_acc_best')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XQS39O_GoizJ"
      },
      "source": [
        "Lastly, we create a `Freezer` to freeze all weights besides the final layer named `model.fc`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "kXOe41FBoizJ"
      },
      "outputs": [],
      "source": [
        "from skorch.callbacks import Freezer\n",
        "\n",
        "freezer = Freezer(lambda x: not x.startswith('model.fc'))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4_JqX5wEoizK"
      },
      "source": [
        "### skorch.NeuralNetClassifier"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2ahqnHCZoizL"
      },
      "source": [
        "With all the preparations out of the way, we can now define our `NeuralNetClassifier`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "xQCwRTaZoizL"
      },
      "outputs": [],
      "source": [
        "net = NeuralNetClassifier(\n",
        "    PretrainedModel, \n",
        "    criterion=nn.CrossEntropyLoss,\n",
        "    lr=0.001,\n",
        "    batch_size=4,\n",
        "    max_epochs=25,\n",
        "    module__output_features=2,\n",
        "    optimizer=optim.SGD,\n",
        "    optimizer__momentum=0.9,\n",
        "    iterator_train__shuffle=True,\n",
        "    iterator_train__num_workers=2,\n",
        "    iterator_valid__num_workers=2,\n",
        "    train_split=predefined_split(val_ds),\n",
        "    callbacks=[lrscheduler, checkpoint, freezer],\n",
        "    device='cuda' # comment to train on cpu\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZggANL3DoizL"
      },
      "source": [
        "That is quite a few parameters! Lets walk through each one:\n",
        "\n",
        "1. `model_ft`: Our `ResNet18` neural network\n",
        "2. `criterion=nn.CrossEntropyLoss`: loss function\n",
        "3. `lr`: Initial learning rate\n",
        "4. `batch_size`: Size of a batch\n",
        "5. `max_epochs`: Number of epochs to train\n",
        "6. `module__output_features`: Used by `__init__` in our `PretrainedModel` class to set the number of classes.\n",
        "7. `optimizer`: Our optimizer\n",
        "8. `optimizer__momentum`: The initial momentum\n",
        "9. `iterator_{train,valid}__{shuffle,num_workers}`: Parameters that are passed to the dataloader.\n",
        "10. `train_split`: A wrapper around `val_ds` to use our validation dataset.\n",
        "11. `callbacks`: Our callbacks \n",
        "12. `device`: Set to `cuda` to train on gpu.\n",
        "\n",
        "Now we are ready to train our neural network:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 641,
          "referenced_widgets": [
            "2e427ec51ada45dbb95b3a8d432aa83a",
            "0f274dff5dd34ab3baaf9158c7ce5de5",
            "01efc735f0d34e478dc2820eb212f108",
            "b088563fd5f24fd39516b4999fdfc003",
            "bd612f740e534feeaca87c8d44825569",
            "12e62a72458c49fabaf223dcc8c7e32b",
            "05c0e1e9e6a443c8b5443b7beb34afb2",
            "8e1c5fb23d074fa68f607f1ebcadb289",
            "5e9cecf6e979425292cf7703b8822183",
            "1c35ef4c3d72459f84c2354fe57c61a9",
            "bd92813035144bca9fa48f555f8d218a"
          ]
        },
        "id": "sYjoTUdyoizM",
        "outputId": "f88743c0-cf16-4037-fa84-a66502d0f2fb"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.8/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.8/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.\n",
            "  warnings.warn(msg)\n",
            "Downloading: \"https://download.pytorch.org/models/resnet18-f37072fd.pth\" to /root/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0.00/44.7M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "2e427ec51ada45dbb95b3a8d432aa83a"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  epoch    train_loss    valid_acc    valid_loss    cp      lr     dur\n",
            "-------  ------------  -----------  ------------  ----  ------  ------\n",
            "      1        \u001b[36m0.6488\u001b[0m       \u001b[32m0.9477\u001b[0m        \u001b[35m0.1860\u001b[0m     +  0.0010  9.3038\n",
            "      2        \u001b[36m0.4275\u001b[0m       0.9412        \u001b[35m0.1697\u001b[0m        0.0010  3.0520\n",
            "      3        0.4977       0.9346        0.1728        0.0010  3.1005\n",
            "      4        0.5072       0.9346        0.1766        0.0010  3.1506\n",
            "      5        0.5104       \u001b[32m0.9608\u001b[0m        \u001b[35m0.1548\u001b[0m     +  0.0010  3.4832\n",
            "      6        \u001b[36m0.3861\u001b[0m       0.9216        0.1879        0.0010  3.2256\n",
            "      7        0.4329       0.9346        0.1839        0.0010  3.0548\n",
            "      8        \u001b[36m0.3634\u001b[0m       0.9477        0.1604        0.0001  3.3032\n",
            "      9        \u001b[36m0.3625\u001b[0m       0.9477        0.1606        0.0001  3.0581\n",
            "     10        \u001b[36m0.3444\u001b[0m       0.9412        0.1796        0.0001  3.0689\n",
            "     11        \u001b[36m0.3334\u001b[0m       0.9346        0.1904        0.0001  3.1114\n",
            "     12        0.3719       0.9477        0.1637        0.0001  3.0940\n",
            "     13        0.4330       0.9412        0.1616        0.0001  3.0920\n",
            "     14        \u001b[36m0.2887\u001b[0m       0.9477        0.1632        0.0001  3.0811\n",
            "     15        0.2981       0.9477        0.1682        0.0000  3.1915\n",
            "     16        0.3129       0.9477        0.1665        0.0000  3.0351\n",
            "     17        0.3422       0.9412        0.1983        0.0000  3.0565\n",
            "     18        0.4063       0.9412        0.1629        0.0000  4.8236\n",
            "     19        0.3207       0.9412        0.1796        0.0000  3.6040\n",
            "     20        0.3319       0.9477        0.1549        0.0000  3.0968\n",
            "     21        \u001b[36m0.2872\u001b[0m       0.9412        0.1658        0.0000  3.1563\n",
            "     22        0.3190       0.9346        0.1716        0.0000  3.1373\n",
            "     23        0.3513       0.9477        0.1720        0.0000  3.1291\n",
            "     24        0.3579       0.9608        0.1587        0.0000  3.1853\n",
            "     25        0.3300       0.9542        0.1639        0.0000  3.0743\n"
          ]
        }
      ],
      "source": [
        "net.fit(train_ds, y=None);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bqhhXTKFoizN"
      },
      "source": [
        "The best model is stored at `best_model.pt`, with a validiation accuracy of roughly 0.96.\n",
        "\n",
        "Congrualations! You now know how to finetune a neural network using `skorch`. Feel free to explore the other tutorials to learn more about using `skorch`."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "base",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.13 (default, Mar 28 2022, 08:03:21) [MSC v.1916 64 bit (AMD64)]"
    },
    "vscode": {
      "interpreter": {
        "hash": "bd97b8bffa4d3737e84826bc3d37be3046061822757ce35137ab82ad4c5a2016"
      }
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "2e427ec51ada45dbb95b3a8d432aa83a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0f274dff5dd34ab3baaf9158c7ce5de5",
              "IPY_MODEL_01efc735f0d34e478dc2820eb212f108",
              "IPY_MODEL_b088563fd5f24fd39516b4999fdfc003"
            ],
            "layout": "IPY_MODEL_bd612f740e534feeaca87c8d44825569"
          }
        },
        "0f274dff5dd34ab3baaf9158c7ce5de5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_12e62a72458c49fabaf223dcc8c7e32b",
            "placeholder": "​",
            "style": "IPY_MODEL_05c0e1e9e6a443c8b5443b7beb34afb2",
            "value": "100%"
          }
        },
        "01efc735f0d34e478dc2820eb212f108": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8e1c5fb23d074fa68f607f1ebcadb289",
            "max": 46830571,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_5e9cecf6e979425292cf7703b8822183",
            "value": 46830571
          }
        },
        "b088563fd5f24fd39516b4999fdfc003": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1c35ef4c3d72459f84c2354fe57c61a9",
            "placeholder": "​",
            "style": "IPY_MODEL_bd92813035144bca9fa48f555f8d218a",
            "value": " 44.7M/44.7M [00:00&lt;00:00, 127MB/s]"
          }
        },
        "bd612f740e534feeaca87c8d44825569": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "12e62a72458c49fabaf223dcc8c7e32b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "05c0e1e9e6a443c8b5443b7beb34afb2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8e1c5fb23d074fa68f607f1ebcadb289": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5e9cecf6e979425292cf7703b8822183": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1c35ef4c3d72459f84c2354fe57c61a9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bd92813035144bca9fa48f555f8d218a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}