Code/Day 42.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是Python，TensorFlow和Keras教程系列的深度学习基础知识的第4部分。\n",
    "\n",
    "在这一部分，我们将讨论的是TensorBoard。TensorBoard是一个方便的应用程序，允许您在浏览器中查看模型或模型的各个方面。我们将TensorBoard与Keras一起使用的方式是通过Keras回调。实际上有很多Keras回调，你可以自己制作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.callbacks import TensorBoard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#创建TensorBoard回调对象\n",
    "NAME = \"Cats-vs-dogs-CNN\"\n",
    "\n",
    "tensorboard = TensorBoard(log_dir=\"logs/{}\".format(NAME))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最终，你会希望获得更多的自定义NAME，但现在这样做。因此，这将保存模型的训练数据logs/NAME，然后由TensorBoard读取。\n",
    "\n",
    "最后，我们可以通过将它添加到.fit方法中来将此回调添加到我们的模型中，例如：\n",
    "```python\n",
    "model.fit(X, y,\n",
    "          batch_size=32,\n",
    "          epochs=3,\n",
    "          validation_split=0.3,\n",
    "          callbacks=[tensorboard])\n",
    "```\n",
    "请注意，这callbacks是一个列表。您也可以将其他回调传递到此列表中。我们的模型还没有定义，所以现在让我们把它们放在一起："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 17462 samples, validate on 7484 samples\n",
      "Epoch 1/3\n",
      "17462/17462 [==============================] - 44s 3ms/step - loss: 0.6992 - acc: 0.5480 - val_loss: 0.6900 - val_acc: 0.5274\n",
      "Epoch 2/3\n",
      "17462/17462 [==============================] - 41s 2ms/step - loss: 0.6754 - acc: 0.5782 - val_loss: 0.6685 - val_acc: 0.5885\n",
      "Epoch 3/3\n",
      "17462/17462 [==============================] - 41s 2ms/step - loss: 0.6377 - acc: 0.6483 - val_loss: 0.6217 - val_acc: 0.6625\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7ff86d691c18>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.keras.datasets import cifar10\n",
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten\n",
    "from tensorflow.keras.layers import Conv2D, MaxPooling2D\n",
    "from tensorflow.keras.callbacks import TensorBoard\n",
    "# more info on callbakcs: https://keras.io/callbacks/ model saver is cool too.\n",
    "import pickle\n",
    "import time\n",
    "\n",
    "NAME = \"Cats-vs-dogs-CNN\"\n",
    "\n",
    "pickle_in = open(\"../datasets/X.pickle\",\"rb\")\n",
    "X = pickle.load(pickle_in)\n",
    "\n",
    "pickle_in = open(\"../datasets/y.pickle\",\"rb\")\n",
    "y = pickle.load(pickle_in)\n",
    "\n",
    "X = X/255.0\n",
    "\n",
    "model = Sequential()\n",
    "\n",
    "model.add(Conv2D(256, (3, 3), input_shape=X.shape[1:]))\n",
    "model.add(Activation('relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "\n",
    "model.add(Conv2D(256, (3, 3)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "\n",
    "model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors\n",
    "model.add(Dense(64))\n",
    "\n",
    "model.add(Dense(1))\n",
    "model.add(Activation('sigmoid'))\n",
    "\n",
    "tensorboard = TensorBoard(log_dir=\"logs/{}\".format(NAME))\n",
    "\n",
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer='adam',\n",
    "              metrics=['accuracy'],\n",
    "              )\n",
    "\n",
    "model.fit(X, y,\n",
    "          batch_size=32,\n",
    "          epochs=3,\n",
    "          validation_split=0.3,\n",
    "          callbacks=[tensorboard])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "运行此之后，您应该有一个名为的新目录logs。我们现在可以使用tensorboard从这个目录中可视化初始结果。打开控制台，切换到工作目录，然后键入：tensorboard --logdir=logs/。您应该看到一个通知：TensorBoard 1.10.0 at http://H-PC:6006 (Press CTRL+C to quit)“h-pc”是您机器的名称。打开浏览器并前往此地址。你应该看到类似的东西：\n",
    "<img src = \"https://pythonprogramming.net/static/images/machine-learning/tensorboard-basic.png\">\n",
    "现在我们可以看到我们的模型随着时间的推移。让我们改变模型中的一些东西。首先，我们从未在密集层中添加激活。另外，让我们尝试整体较小的模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 17462 samples, validate on 7484 samples\n",
      "Epoch 1/10\n",
      "17462/17462 [==============================] - 11s 604us/step - loss: 0.6033 - acc: 0.6652 - val_loss: 0.5298 - val_acc: 0.7320\n",
      "Epoch 2/10\n",
      "17462/17462 [==============================] - 11s 646us/step - loss: 0.4859 - acc: 0.7659 - val_loss: 0.4723 - val_acc: 0.7763\n",
      "Epoch 3/10\n",
      "17462/17462 [==============================] - 11s 641us/step - loss: 0.4270 - acc: 0.8045 - val_loss: 0.4603 - val_acc: 0.7803\n",
      "Epoch 4/10\n",
      "17462/17462 [==============================] - 12s 699us/step - loss: 0.3675 - acc: 0.8347 - val_loss: 0.4476 - val_acc: 0.7929\n",
      "Epoch 5/10\n",
      "17462/17462 [==============================] - 12s 707us/step - loss: 0.3012 - acc: 0.8694 - val_loss: 0.4854 - val_acc: 0.7797\n",
      "Epoch 6/10\n",
      "17462/17462 [==============================] - 12s 705us/step - loss: 0.2165 - acc: 0.9118 - val_loss: 0.5450 - val_acc: 0.7865\n",
      "Epoch 7/10\n",
      "17462/17462 [==============================] - 12s 712us/step - loss: 0.1332 - acc: 0.9510 - val_loss: 0.6512 - val_acc: 0.7821\n",
      "Epoch 8/10\n",
      "17462/17462 [==============================] - 12s 705us/step - loss: 0.0764 - acc: 0.9743 - val_loss: 0.7487 - val_acc: 0.7809\n",
      "Epoch 9/10\n",
      "17462/17462 [==============================] - 12s 713us/step - loss: 0.0389 - acc: 0.9887 - val_loss: 0.9041 - val_acc: 0.7743\n",
      "Epoch 10/10\n",
      "17462/17462 [==============================] - 12s 708us/step - loss: 0.0287 - acc: 0.9921 - val_loss: 1.0411 - val_acc: 0.7702\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7ff86073ec50>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten\n",
    "from tensorflow.keras.layers import Conv2D, MaxPooling2D\n",
    "from tensorflow.keras.callbacks import TensorBoard\n",
    "# more info on callbakcs: https://keras.io/callbacks/ model saver is cool too.\n",
    "import pickle\n",
    "import time\n",
    "\n",
    "NAME = \"Cats-vs-dogs-64x2-CNN\"\n",
    "\n",
    "pickle_in = open(\"../datasets/X.pickle\",\"rb\")\n",
    "X = pickle.load(pickle_in)\n",
    "\n",
    "pickle_in = open(\"../datasets/y.pickle\",\"rb\")\n",
    "y = pickle.load(pickle_in)\n",
    "\n",
    "X = X/255.0\n",
    "\n",
    "model = Sequential()\n",
    "\n",
    "model.add(Conv2D(64, (3, 3), input_shape=X.shape[1:]))\n",
    "model.add(Activation('relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "\n",
    "model.add(Conv2D(64, (3, 3)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "\n",
    "model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors\n",
    "model.add(Dense(64))\n",
    "model.add(Activation('relu'))\n",
    "\n",
    "model.add(Dense(1))\n",
    "model.add(Activation('sigmoid'))\n",
    "\n",
    "tensorboard = TensorBoard(log_dir=\"logs/{}\".format(NAME))\n",
    "\n",
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer='adam',\n",
    "              metrics=['accuracy'],\n",
    "              )\n",
    "\n",
    "model.fit(X, y,\n",
    "          batch_size=32,\n",
    "          epochs=10,\n",
    "          validation_split=0.3,\n",
    "          callbacks=[tensorboard])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "除此之外，我还改名为NAME = \"Cats-vs-dogs-64x2-CNN\"。不要忘记这样做，否则你会偶然附加到你以前的型号的日志，它看起来不太好。我们现在检查TensorBoard：\n",
    "<img src = \"https://pythonprogramming.net/static/images/machine-learning/second-model-tensorboard.png\">\n",
    "看起来更好！但是，您可能会立即注意到验证丢失的形状。损失是衡量错误的标准，看起来很明显，在我们的第四个时代之后，事情开始变得糟糕。\n",
    "\n",
    "有趣的是，我们的验证准确性仍然持续，但我想它最终会开始下降。更可能的是，第一件遭受的事情确实是你的验证损失。这应该提醒你，你几乎肯定会开始过度适应。这种情况发生的原因是该模型不断尝试减少样本损失。\n",
    "\n",
    "在某些时候，模型不是学习关于实际数据的一般事物，而是开始只记忆输入数据。如果你继续这样做，是的，样本中的“准确性”会上升，但你的样本，以及你试图为模型提供的任何新数据可能会表现得很差。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}