Tutorial 5: Clarifying notation with illustration

Author: phlippe

docs/tutorial_notebooks/tutorial5/Inception_ResNet_DenseNet.ipynb

@@ -1525,7 +1525,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The overall ResNet architecture consists of stacking multiple ResNet blocks, of which some are downsampling the input. When talking about ResNet blocks in the whole network, we usually group them by the same output shape. Hence, if we say the ResNet has `[3,3,3]` blocks, it means that we have 3 times a group of 3 ResNet blocks, where a subsampling is taking place in the fourth and seventh block. The same notation is used by many other implementations such as in the [torchvision library](https://pytorch.org/docs/stable/_modules/torchvision/models/resnet.html#resnet18) from PyTorch. Our code looks as follows:"
+    "The overall ResNet architecture consists of stacking multiple ResNet blocks, of which some are downsampling the input. When talking about ResNet blocks in the whole network, we usually group them by the same output shape. Hence, if we say the ResNet has `[3,3,3]` blocks, it means that we have 3 times a group of 3 ResNet blocks, where a subsampling is taking place in the fourth and seventh block. The ResNet with `[3,3,3]` blocks on CIFAR10 is visualized below.\n",
+    "\n",
+    "<center width=\"100%\"><img src=\"resnet_notation.svg\" width=\"500px\"></center>\n",
+    "\n",
+    "The three groups operate on the resolutions $32\\times32$, $16\\times16$ and $8\\times8$ respectively. The blocks in orange denote ResNet blocks with downsampling. The same notation is used by many other implementations such as in the [torchvision library](https://pytorch.org/docs/stable/_modules/torchvision/models/resnet.html#resnet18) from PyTorch. Thus, our code looks as follows:"
    ]
   },
   {
@@ -1540,7 +1544,7 @@
     "        \"\"\"\n",
     "        Inputs: \n",
     "            num_classes - Number of classification outputs (10 for CIFAR10)\n",
-    "            num_blocks - List with the number of ResNet blocks to use. The first block of each group uses downsampling, expect the first.\n",
+    "            num_blocks - List with the number of ResNet blocks to use. The first block of each group uses downsampling, except the first.\n",
     "            c_hidden - List with the hidden dimensionalities in the different blocks. Usually multiplied by 2 the deeper we go.\n",
     "            act_fn_name - Name of the activation function to use, looked up in \"act_fn_by_name\"\n",
     "            block_name - Name of the ResNet block, looked up in \"resnet_blocks_by_name\"\n",
@@ -1576,7 +1580,7 @@
     "        blocks = []\n",
     "        for block_idx, block_count in enumerate(self.hparams.num_blocks):\n",
     "            for bc in range(block_count):\n",
-    "                subsample = (bc == 0 and block_idx > 0) # Subsample the first block of each \"super-block\", except the very first one.\n",
+    "                subsample = (bc == 0 and block_idx > 0) # Subsample the first block of each group, except the very first one.\n",
     "                blocks.append(\n",
     "                    self.hparams.block_class(c_in=c_hidden[block_idx if not subsample else (block_idx-1)],\n",
     "                                             act_fn=self.hparams.act_fn,\n",
@@ -2273,7 +2277,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.3"
+   "version": "3.7.4"
   }
  },
  "nbformat": 4,