Time_Series_Forecasting_With_CNNs.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Time Series Analysis and Forecasting with CNNs"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convolutional Neural Network models, or CNNs for short, can be applied to time series forecasting. I will be presenting different type of scenarios that we usually come across during solving problems related with time series and the variety of CNN architectures that we can use to tackle them.\n",
    "\n",
    "Variety of time series related problems:\n",
    "- CNN Models for Univariate time series\n",
    "- CNN Models for Multivariate time series\n",
    "- CNN Models for Multistep time series\n",
    "- CNN Models for Multivariate and Multisteps time series\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Univariate CNN Models\n",
    "\n",
    "Although traditionally developed for two-dimensional image data, CNNs can be used to model univariate time series forecasting problems. Univariate time series are datasets comprised of a single series of observations with a temporal ordering and a model is required to learn from the series of past observations to predict the next value in the sequence.\n",
    "\n",
    "This section is divided into two parts:\n",
    "1. Data Preparation \n",
    "2. CNN Model\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"data/Alcohol_Sales.csv\", parse_dates=[\"DATE\"], index_col=\"DATE\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>S4248SM144NCEN</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DATE</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1992-01-01</th>\n",
       "      <td>3459</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1992-02-01</th>\n",
       "      <td>3458</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            S4248SM144NCEN\n",
       "DATE                      \n",
       "1992-01-01            3459\n",
       "1992-02-01            3458"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.rename({\"S4248SM144NCEN\":\"Sales\"}, axis = 1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sales</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DATE</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1992-01-01</th>\n",
       "      <td>3459</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1992-02-01</th>\n",
       "      <td>3458</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1992-03-01</th>\n",
       "      <td>4002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1992-04-01</th>\n",
       "      <td>4564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1992-05-01</th>\n",
       "      <td>4221</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Sales\n",
       "DATE             \n",
       "1992-01-01   3459\n",
       "1992-02-01   3458\n",
       "1992-03-01   4002\n",
       "1992-04-01   4564\n",
       "1992-05-01   4221"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sales</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DATE</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-09-01</th>\n",
       "      <td>12396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-10-01</th>\n",
       "      <td>13914</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-11-01</th>\n",
       "      <td>14174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-12-01</th>\n",
       "      <td>15504</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2019-01-01</th>\n",
       "      <td>10718</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Sales\n",
       "DATE             \n",
       "2018-09-01  12396\n",
       "2018-10-01  13914\n",
       "2018-11-01  14174\n",
       "2018-12-01  15504\n",
       "2019-01-01  10718"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sales    325\n",
       "dtype: int64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.count()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So here the goal is to moniter Sales data over time. We have to model the Sales series "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3459,  3458,  4002,  4564,  4221,  4529,  4466,  4137,  4126,\n",
       "        4259,  4240,  4936,  3031,  3261,  4160,  4377,  4307,  4696,\n",
       "        4458,  4457,  4364,  4236,  4500,  4974,  3075,  3377,  4443,\n",
       "        4261,  4460,  4985,  4324,  4719,  4374,  4248,  4784,  4971,\n",
       "        3370,  3484,  4269,  3994,  4715,  4974,  4223,  5000,  4235,\n",
       "        4554,  4851,  4826,  3699,  3983,  4262,  4619,  5219,  4836,\n",
       "        4941,  5062,  4365,  5012,  4850,  5097,  3758,  3825,  4454,\n",
       "        4635,  5210,  5057,  5231,  5034,  4970,  5342,  4831,  5965,\n",
       "        3796,  4019,  4898,  5090,  5237,  5447,  5435,  5107,  5515,\n",
       "        5583,  5346,  6286,  4032,  4435,  5479,  5483,  5587,  6176,\n",
       "        5621,  5889,  5828,  5849,  6180,  6771,  4243,  4952,  6008,\n",
       "        5353,  6435,  6673,  5636,  6630,  5887,  6322,  6520,  6678,\n",
       "        5082,  5216,  5893,  5894,  6799,  6667,  6374,  6840,  5575,\n",
       "        6545,  6789,  7180,  5117,  5442,  6337,  6525,  7216,  6761,\n",
       "        6958,  7070,  6148,  6924,  6716,  7975,  5326,  5609,  6414,\n",
       "        6741,  7144,  7133,  7568,  7266,  6634,  7626,  6843,  8540,\n",
       "        5629,  5898,  7045,  7094,  7333,  7918,  7289,  7396,  7259,\n",
       "        7268,  7731,  9058,  5557,  6237,  7723,  7262,  8241,  8757,\n",
       "        7352,  8496,  7741,  7710,  8247,  8902,  6066,  6590,  7923,\n",
       "        7335,  8843,  9327,  7792,  9156,  8037,  8640,  9128,  9545,\n",
       "        6627,  6743,  8195,  7828,  9570,  9484,  8608,  9543,  8123,\n",
       "        9649,  9390, 10065,  7093,  7483,  8365,  8895,  9794,  9977,\n",
       "        9553,  9375,  9225,  9948,  8758, 10839,  7266,  7578,  8688,\n",
       "        9162,  9369, 10167,  9507,  8923,  9272,  9075,  8949, 10843,\n",
       "        6558,  7481,  9475,  9424,  9351, 10552,  9077,  9273,  9420,\n",
       "        9413,  9866, 11455,  6901,  8014,  9832,  9281,  9967, 11344,\n",
       "        9106, 10469, 10085,  9612, 10328, 11483,  7486,  8641,  9709,\n",
       "        9423, 11342, 11274,  9845, 11163,  9532, 10754, 10953, 11922,\n",
       "        8395,  8888, 10110, 10493, 12218, 11385, 11186, 11462, 10494,\n",
       "       11540, 11138, 12709,  8557,  9059, 10055, 10977, 11792, 11904,\n",
       "       10965, 10981, 10828, 11817, 10470, 13310,  8400,  9062, 10722,\n",
       "       11107, 11508, 12904, 11869, 11224, 12022, 11983, 11506, 14183,\n",
       "        8648, 10321, 12107, 11420, 12238, 13681, 10950, 12700, 12272,\n",
       "       11905, 13016, 14421,  9043, 10452, 12481, 11491, 13545, 14730,\n",
       "       11416, 13402, 11907, 12711, 13261, 14265,  9564, 10415, 12683,\n",
       "       11919, 14138, 14583, 12640, 14257, 12396, 13914, 14174, 15504,\n",
       "       10718])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"Sales\"].values"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay so basically we need to model the above series and we need to transform a univariate series into a supervised learning problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.functional as F\n",
    "from numpy import array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "#function for data transformation\n",
    "\n",
    "def split_sequence(sequences, n_steps):\n",
    "\n",
    "    X, y = [], []\n",
    "    for i in range(len(sequences)):\n",
    "        end_ix = i + n_steps\n",
    "        if end_ix > len(sequences)-1:\n",
    "            break \n",
    "        seq_x, seq_y = sequences[i:end_ix], sequences[end_ix]\n",
    "        X.append(seq_x)\n",
    "        y.append(seq_y)\n",
    "    return torch.tensor(array(X), dtype=torch.long), torch.tensor(array(y), dtype=torch.long)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running the example splits the univariate series into X and y where each row in X is of shape `n_steps` and y is single output per X's rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create feature and target from the above function\n",
    "\n",
    "sequences = df.Sales.values\n",
    "\n",
    "X, y = split_sequence(sequences, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[3459, 3458, 4002],\n",
       "         [3458, 4002, 4564]]),\n",
       " tensor([4564, 4221]))"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X[:2], y[:2]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we know how to prepare a univariate series for modeling, let’s look at developing a CNN model that can learn the mapping of inputs to outputs."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CNN Model\n",
    "\n",
    "A one-dimensional CNN is a CNN model that has a convolutional hidden layer that operates over a 1D sequence. This is followed by perhaps a second convolutional layer in some cases, such as very long input sequences, and then a pooling layer whose job it is to distill the output of the convolutional layer to the most salient elements. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_features = 1\n",
    "X = X.reshape(X.shape[0], n_features, X.shape[1])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([322, 1, 3])"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The CNN does not actually view the data as having time steps, instead, it is treated as a sequence over which convolutional read operations can be performed, like a one-dimensional image"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Model Architecture:\n",
    "\n",
    "- A convolutional layer with 64 filter maps and a kernel size of 2. \n",
    "- Max pooling layer and a dense layer to interpret the input feature \n",
    "- An output layer is specified that predicts a single numerical value\n",
    "- Optimiser - Adam\n",
    "- Loss function - MSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CNNTimeSeriesModel(nn.Module):\n",
    "\n",
    "    def __init__(self, n_steps, n_features):\n",
    "\n",
    "        super().__init__()\n",
    "\n",
    "        self.cnn_layer = nn.Conv1d(n_steps, n_features, kernel_size=2)\n",
    "        self.mp_layer = nn.MaxPool1d(kernel_size=2)\n",
    "        self.ln_layer = nn.Linear()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "cnn_layer = nn.Conv1d(in_channels= 1, out_channels= 64, kernel_size=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 3459,  3458,  4002]],\n",
       "\n",
       "        [[ 3458,  4002,  4564]],\n",
       "\n",
       "        [[ 4002,  4564,  4221]],\n",
       "\n",
       "        [[ 4564,  4221,  4529]],\n",
       "\n",
       "        [[ 4221,  4529,  4466]],\n",
       "\n",
       "        [[ 4529,  4466,  4137]],\n",
       "\n",
       "        [[ 4466,  4137,  4126]],\n",
       "\n",
       "        [[ 4137,  4126,  4259]],\n",
       "\n",
       "        [[ 4126,  4259,  4240]],\n",
       "\n",
       "        [[ 4259,  4240,  4936]],\n",
       "\n",
       "        [[ 4240,  4936,  3031]],\n",
       "\n",
       "        [[ 4936,  3031,  3261]],\n",
       "\n",
       "        [[ 3031,  3261,  4160]],\n",
       "\n",
       "        [[ 3261,  4160,  4377]],\n",
       "\n",
       "        [[ 4160,  4377,  4307]],\n",
       "\n",
       "        [[ 4377,  4307,  4696]],\n",
       "\n",
       "        [[ 4307,  4696,  4458]],\n",
       "\n",
       "        [[ 4696,  4458,  4457]],\n",
       "\n",
       "        [[ 4458,  4457,  4364]],\n",
       "\n",
       "        [[ 4457,  4364,  4236]],\n",
       "\n",
       "        [[ 4364,  4236,  4500]],\n",
       "\n",
       "        [[ 4236,  4500,  4974]],\n",
       "\n",
       "        [[ 4500,  4974,  3075]],\n",
       "\n",
       "        [[ 4974,  3075,  3377]],\n",
       "\n",
       "        [[ 3075,  3377,  4443]],\n",
       "\n",
       "        [[ 3377,  4443,  4261]],\n",
       "\n",
       "        [[ 4443,  4261,  4460]],\n",
       "\n",
       "        [[ 4261,  4460,  4985]],\n",
       "\n",
       "        [[ 4460,  4985,  4324]],\n",
       "\n",
       "        [[ 4985,  4324,  4719]],\n",
       "\n",
       "        [[ 4324,  4719,  4374]],\n",
       "\n",
       "        [[ 4719,  4374,  4248]],\n",
       "\n",
       "        [[ 4374,  4248,  4784]],\n",
       "\n",
       "        [[ 4248,  4784,  4971]],\n",
       "\n",
       "        [[ 4784,  4971,  3370]],\n",
       "\n",
       "        [[ 4971,  3370,  3484]],\n",
       "\n",
       "        [[ 3370,  3484,  4269]],\n",
       "\n",
       "        [[ 3484,  4269,  3994]],\n",
       "\n",
       "        [[ 4269,  3994,  4715]],\n",
       "\n",
       "        [[ 3994,  4715,  4974]],\n",
       "\n",
       "        [[ 4715,  4974,  4223]],\n",
       "\n",
       "        [[ 4974,  4223,  5000]],\n",
       "\n",
       "        [[ 4223,  5000,  4235]],\n",
       "\n",
       "        [[ 5000,  4235,  4554]],\n",
       "\n",
       "        [[ 4235,  4554,  4851]],\n",
       "\n",
       "        [[ 4554,  4851,  4826]],\n",
       "\n",
       "        [[ 4851,  4826,  3699]],\n",
       "\n",
       "        [[ 4826,  3699,  3983]],\n",
       "\n",
       "        [[ 3699,  3983,  4262]],\n",
       "\n",
       "        [[ 3983,  4262,  4619]],\n",
       "\n",
       "        [[ 4262,  4619,  5219]],\n",
       "\n",
       "        [[ 4619,  5219,  4836]],\n",
       "\n",
       "        [[ 5219,  4836,  4941]],\n",
       "\n",
       "        [[ 4836,  4941,  5062]],\n",
       "\n",
       "        [[ 4941,  5062,  4365]],\n",
       "\n",
       "        [[ 5062,  4365,  5012]],\n",
       "\n",
       "        [[ 4365,  5012,  4850]],\n",
       "\n",
       "        [[ 5012,  4850,  5097]],\n",
       "\n",
       "        [[ 4850,  5097,  3758]],\n",
       "\n",
       "        [[ 5097,  3758,  3825]],\n",
       "\n",
       "        [[ 3758,  3825,  4454]],\n",
       "\n",
       "        [[ 3825,  4454,  4635]],\n",
       "\n",
       "        [[ 4454,  4635,  5210]],\n",
       "\n",
       "        [[ 4635,  5210,  5057]],\n",
       "\n",
       "        [[ 5210,  5057,  5231]],\n",
       "\n",
       "        [[ 5057,  5231,  5034]],\n",
       "\n",
       "        [[ 5231,  5034,  4970]],\n",
       "\n",
       "        [[ 5034,  4970,  5342]],\n",
       "\n",
       "        [[ 4970,  5342,  4831]],\n",
       "\n",
       "        [[ 5342,  4831,  5965]],\n",
       "\n",
       "        [[ 4831,  5965,  3796]],\n",
       "\n",
       "        [[ 5965,  3796,  4019]],\n",
       "\n",
       "        [[ 3796,  4019,  4898]],\n",
       "\n",
       "        [[ 4019,  4898,  5090]],\n",
       "\n",
       "        [[ 4898,  5090,  5237]],\n",
       "\n",
       "        [[ 5090,  5237,  5447]],\n",
       "\n",
       "        [[ 5237,  5447,  5435]],\n",
       "\n",
       "        [[ 5447,  5435,  5107]],\n",
       "\n",
       "        [[ 5435,  5107,  5515]],\n",
       "\n",
       "        [[ 5107,  5515,  5583]],\n",
       "\n",
       "        [[ 5515,  5583,  5346]],\n",
       "\n",
       "        [[ 5583,  5346,  6286]],\n",
       "\n",
       "        [[ 5346,  6286,  4032]],\n",
       "\n",
       "        [[ 6286,  4032,  4435]],\n",
       "\n",
       "        [[ 4032,  4435,  5479]],\n",
       "\n",
       "        [[ 4435,  5479,  5483]],\n",
       "\n",
       "        [[ 5479,  5483,  5587]],\n",
       "\n",
       "        [[ 5483,  5587,  6176]],\n",
       "\n",
       "        [[ 5587,  6176,  5621]],\n",
       "\n",
       "        [[ 6176,  5621,  5889]],\n",
       "\n",
       "        [[ 5621,  5889,  5828]],\n",
       "\n",
       "        [[ 5889,  5828,  5849]],\n",
       "\n",
       "        [[ 5828,  5849,  6180]],\n",
       "\n",
       "        [[ 5849,  6180,  6771]],\n",
       "\n",
       "        [[ 6180,  6771,  4243]],\n",
       "\n",
       "        [[ 6771,  4243,  4952]],\n",
       "\n",
       "        [[ 4243,  4952,  6008]],\n",
       "\n",
       "        [[ 4952,  6008,  5353]],\n",
       "\n",
       "        [[ 6008,  5353,  6435]],\n",
       "\n",
       "        [[ 5353,  6435,  6673]],\n",
       "\n",
       "        [[ 6435,  6673,  5636]],\n",
       "\n",
       "        [[ 6673,  5636,  6630]],\n",
       "\n",
       "        [[ 5636,  6630,  5887]],\n",
       "\n",
       "        [[ 6630,  5887,  6322]],\n",
       "\n",
       "        [[ 5887,  6322,  6520]],\n",
       "\n",
       "        [[ 6322,  6520,  6678]],\n",
       "\n",
       "        [[ 6520,  6678,  5082]],\n",
       "\n",
       "        [[ 6678,  5082,  5216]],\n",
       "\n",
       "        [[ 5082,  5216,  5893]],\n",
       "\n",
       "        [[ 5216,  5893,  5894]],\n",
       "\n",
       "        [[ 5893,  5894,  6799]],\n",
       "\n",
       "        [[ 5894,  6799,  6667]],\n",
       "\n",
       "        [[ 6799,  6667,  6374]],\n",
       "\n",
       "        [[ 6667,  6374,  6840]],\n",
       "\n",
       "        [[ 6374,  6840,  5575]],\n",
       "\n",
       "        [[ 6840,  5575,  6545]],\n",
       "\n",
       "        [[ 5575,  6545,  6789]],\n",
       "\n",
       "        [[ 6545,  6789,  7180]],\n",
       "\n",
       "        [[ 6789,  7180,  5117]],\n",
       "\n",
       "        [[ 7180,  5117,  5442]],\n",
       "\n",
       "        [[ 5117,  5442,  6337]],\n",
       "\n",
       "        [[ 5442,  6337,  6525]],\n",
       "\n",
       "        [[ 6337,  6525,  7216]],\n",
       "\n",
       "        [[ 6525,  7216,  6761]],\n",
       "\n",
       "        [[ 7216,  6761,  6958]],\n",
       "\n",
       "        [[ 6761,  6958,  7070]],\n",
       "\n",
       "        [[ 6958,  7070,  6148]],\n",
       "\n",
       "        [[ 7070,  6148,  6924]],\n",
       "\n",
       "        [[ 6148,  6924,  6716]],\n",
       "\n",
       "        [[ 6924,  6716,  7975]],\n",
       "\n",
       "        [[ 6716,  7975,  5326]],\n",
       "\n",
       "        [[ 7975,  5326,  5609]],\n",
       "\n",
       "        [[ 5326,  5609,  6414]],\n",
       "\n",
       "        [[ 5609,  6414,  6741]],\n",
       "\n",
       "        [[ 6414,  6741,  7144]],\n",
       "\n",
       "        [[ 6741,  7144,  7133]],\n",
       "\n",
       "        [[ 7144,  7133,  7568]],\n",
       "\n",
       "        [[ 7133,  7568,  7266]],\n",
       "\n",
       "        [[ 7568,  7266,  6634]],\n",
       "\n",
       "        [[ 7266,  6634,  7626]],\n",
       "\n",
       "        [[ 6634,  7626,  6843]],\n",
       "\n",
       "        [[ 7626,  6843,  8540]],\n",
       "\n",
       "        [[ 6843,  8540,  5629]],\n",
       "\n",
       "        [[ 8540,  5629,  5898]],\n",
       "\n",
       "        [[ 5629,  5898,  7045]],\n",
       "\n",
       "        [[ 5898,  7045,  7094]],\n",
       "\n",
       "        [[ 7045,  7094,  7333]],\n",
       "\n",
       "        [[ 7094,  7333,  7918]],\n",
       "\n",
       "        [[ 7333,  7918,  7289]],\n",
       "\n",
       "        [[ 7918,  7289,  7396]],\n",
       "\n",
       "        [[ 7289,  7396,  7259]],\n",
       "\n",
       "        [[ 7396,  7259,  7268]],\n",
       "\n",
       "        [[ 7259,  7268,  7731]],\n",
       "\n",
       "        [[ 7268,  7731,  9058]],\n",
       "\n",
       "        [[ 7731,  9058,  5557]],\n",
       "\n",
       "        [[ 9058,  5557,  6237]],\n",
       "\n",
       "        [[ 5557,  6237,  7723]],\n",
       "\n",
       "        [[ 6237,  7723,  7262]],\n",
       "\n",
       "        [[ 7723,  7262,  8241]],\n",
       "\n",
       "        [[ 7262,  8241,  8757]],\n",
       "\n",
       "        [[ 8241,  8757,  7352]],\n",
       "\n",
       "        [[ 8757,  7352,  8496]],\n",
       "\n",
       "        [[ 7352,  8496,  7741]],\n",
       "\n",
       "        [[ 8496,  7741,  7710]],\n",
       "\n",
       "        [[ 7741,  7710,  8247]],\n",
       "\n",
       "        [[ 7710,  8247,  8902]],\n",
       "\n",
       "        [[ 8247,  8902,  6066]],\n",
       "\n",
       "        [[ 8902,  6066,  6590]],\n",
       "\n",
       "        [[ 6066,  6590,  7923]],\n",
       "\n",
       "        [[ 6590,  7923,  7335]],\n",
       "\n",
       "        [[ 7923,  7335,  8843]],\n",
       "\n",
       "        [[ 7335,  8843,  9327]],\n",
       "\n",
       "        [[ 8843,  9327,  7792]],\n",
       "\n",
       "        [[ 9327,  7792,  9156]],\n",
       "\n",
       "        [[ 7792,  9156,  8037]],\n",
       "\n",
       "        [[ 9156,  8037,  8640]],\n",
       "\n",
       "        [[ 8037,  8640,  9128]],\n",
       "\n",
       "        [[ 8640,  9128,  9545]],\n",
       "\n",
       "        [[ 9128,  9545,  6627]],\n",
       "\n",
       "        [[ 9545,  6627,  6743]],\n",
       "\n",
       "        [[ 6627,  6743,  8195]],\n",
       "\n",
       "        [[ 6743,  8195,  7828]],\n",
       "\n",
       "        [[ 8195,  7828,  9570]],\n",
       "\n",
       "        [[ 7828,  9570,  9484]],\n",
       "\n",
       "        [[ 9570,  9484,  8608]],\n",
       "\n",
       "        [[ 9484,  8608,  9543]],\n",
       "\n",
       "        [[ 8608,  9543,  8123]],\n",
       "\n",
       "        [[ 9543,  8123,  9649]],\n",
       "\n",
       "        [[ 8123,  9649,  9390]],\n",
       "\n",
       "        [[ 9649,  9390, 10065]],\n",
       "\n",
       "        [[ 9390, 10065,  7093]],\n",
       "\n",
       "        [[10065,  7093,  7483]],\n",
       "\n",
       "        [[ 7093,  7483,  8365]],\n",
       "\n",
       "        [[ 7483,  8365,  8895]],\n",
       "\n",
       "        [[ 8365,  8895,  9794]],\n",
       "\n",
       "        [[ 8895,  9794,  9977]],\n",
       "\n",
       "        [[ 9794,  9977,  9553]],\n",
       "\n",
       "        [[ 9977,  9553,  9375]],\n",
       "\n",
       "        [[ 9553,  9375,  9225]],\n",
       "\n",
       "        [[ 9375,  9225,  9948]],\n",
       "\n",
       "        [[ 9225,  9948,  8758]],\n",
       "\n",
       "        [[ 9948,  8758, 10839]],\n",
       "\n",
       "        [[ 8758, 10839,  7266]],\n",
       "\n",
       "        [[10839,  7266,  7578]],\n",
       "\n",
       "        [[ 7266,  7578,  8688]],\n",
       "\n",
       "        [[ 7578,  8688,  9162]],\n",
       "\n",
       "        [[ 8688,  9162,  9369]],\n",
       "\n",
       "        [[ 9162,  9369, 10167]],\n",
       "\n",
       "        [[ 9369, 10167,  9507]],\n",
       "\n",
       "        [[10167,  9507,  8923]],\n",
       "\n",
       "        [[ 9507,  8923,  9272]],\n",
       "\n",
       "        [[ 8923,  9272,  9075]],\n",
       "\n",
       "        [[ 9272,  9075,  8949]],\n",
       "\n",
       "        [[ 9075,  8949, 10843]],\n",
       "\n",
       "        [[ 8949, 10843,  6558]],\n",
       "\n",
       "        [[10843,  6558,  7481]],\n",
       "\n",
       "        [[ 6558,  7481,  9475]],\n",
       "\n",
       "        [[ 7481,  9475,  9424]],\n",
       "\n",
       "        [[ 9475,  9424,  9351]],\n",
       "\n",
       "        [[ 9424,  9351, 10552]],\n",
       "\n",
       "        [[ 9351, 10552,  9077]],\n",
       "\n",
       "        [[10552,  9077,  9273]],\n",
       "\n",
       "        [[ 9077,  9273,  9420]],\n",
       "\n",
       "        [[ 9273,  9420,  9413]],\n",
       "\n",
       "        [[ 9420,  9413,  9866]],\n",
       "\n",
       "        [[ 9413,  9866, 11455]],\n",
       "\n",
       "        [[ 9866, 11455,  6901]],\n",
       "\n",
       "        [[11455,  6901,  8014]],\n",
       "\n",
       "        [[ 6901,  8014,  9832]],\n",
       "\n",
       "        [[ 8014,  9832,  9281]],\n",
       "\n",
       "        [[ 9832,  9281,  9967]],\n",
       "\n",
       "        [[ 9281,  9967, 11344]],\n",
       "\n",
       "        [[ 9967, 11344,  9106]],\n",
       "\n",
       "        [[11344,  9106, 10469]],\n",
       "\n",
       "        [[ 9106, 10469, 10085]],\n",
       "\n",
       "        [[10469, 10085,  9612]],\n",
       "\n",
       "        [[10085,  9612, 10328]],\n",
       "\n",
       "        [[ 9612, 10328, 11483]],\n",
       "\n",
       "        [[10328, 11483,  7486]],\n",
       "\n",
       "        [[11483,  7486,  8641]],\n",
       "\n",
       "        [[ 7486,  8641,  9709]],\n",
       "\n",
       "        [[ 8641,  9709,  9423]],\n",
       "\n",
       "        [[ 9709,  9423, 11342]],\n",
       "\n",
       "        [[ 9423, 11342, 11274]],\n",
       "\n",
       "        [[11342, 11274,  9845]],\n",
       "\n",
       "        [[11274,  9845, 11163]],\n",
       "\n",
       "        [[ 9845, 11163,  9532]],\n",
       "\n",
       "        [[11163,  9532, 10754]],\n",
       "\n",
       "        [[ 9532, 10754, 10953]],\n",
       "\n",
       "        [[10754, 10953, 11922]],\n",
       "\n",
       "        [[10953, 11922,  8395]],\n",
       "\n",
       "        [[11922,  8395,  8888]],\n",
       "\n",
       "        [[ 8395,  8888, 10110]],\n",
       "\n",
       "        [[ 8888, 10110, 10493]],\n",
       "\n",
       "        [[10110, 10493, 12218]],\n",
       "\n",
       "        [[10493, 12218, 11385]],\n",
       "\n",
       "        [[12218, 11385, 11186]],\n",
       "\n",
       "        [[11385, 11186, 11462]],\n",
       "\n",
       "        [[11186, 11462, 10494]],\n",
       "\n",
       "        [[11462, 10494, 11540]],\n",
       "\n",
       "        [[10494, 11540, 11138]],\n",
       "\n",
       "        [[11540, 11138, 12709]],\n",
       "\n",
       "        [[11138, 12709,  8557]],\n",
       "\n",
       "        [[12709,  8557,  9059]],\n",
       "\n",
       "        [[ 8557,  9059, 10055]],\n",
       "\n",
       "        [[ 9059, 10055, 10977]],\n",
       "\n",
       "        [[10055, 10977, 11792]],\n",
       "\n",
       "        [[10977, 11792, 11904]],\n",
       "\n",
       "        [[11792, 11904, 10965]],\n",
       "\n",
       "        [[11904, 10965, 10981]],\n",
       "\n",
       "        [[10965, 10981, 10828]],\n",
       "\n",
       "        [[10981, 10828, 11817]],\n",
       "\n",
       "        [[10828, 11817, 10470]],\n",
       "\n",
       "        [[11817, 10470, 13310]],\n",
       "\n",
       "        [[10470, 13310,  8400]],\n",
       "\n",
       "        [[13310,  8400,  9062]],\n",
       "\n",
       "        [[ 8400,  9062, 10722]],\n",
       "\n",
       "        [[ 9062, 10722, 11107]],\n",
       "\n",
       "        [[10722, 11107, 11508]],\n",
       "\n",
       "        [[11107, 11508, 12904]],\n",
       "\n",
       "        [[11508, 12904, 11869]],\n",
       "\n",
       "        [[12904, 11869, 11224]],\n",
       "\n",
       "        [[11869, 11224, 12022]],\n",
       "\n",
       "        [[11224, 12022, 11983]],\n",
       "\n",
       "        [[12022, 11983, 11506]],\n",
       "\n",
       "        [[11983, 11506, 14183]],\n",
       "\n",
       "        [[11506, 14183,  8648]],\n",
       "\n",
       "        [[14183,  8648, 10321]],\n",
       "\n",
       "        [[ 8648, 10321, 12107]],\n",
       "\n",
       "        [[10321, 12107, 11420]],\n",
       "\n",
       "        [[12107, 11420, 12238]],\n",
       "\n",
       "        [[11420, 12238, 13681]],\n",
       "\n",
       "        [[12238, 13681, 10950]],\n",
       "\n",
       "        [[13681, 10950, 12700]],\n",
       "\n",
       "        [[10950, 12700, 12272]],\n",
       "\n",
       "        [[12700, 12272, 11905]],\n",
       "\n",
       "        [[12272, 11905, 13016]],\n",
       "\n",
       "        [[11905, 13016, 14421]],\n",
       "\n",
       "        [[13016, 14421,  9043]],\n",
       "\n",
       "        [[14421,  9043, 10452]],\n",
       "\n",
       "        [[ 9043, 10452, 12481]],\n",
       "\n",
       "        [[10452, 12481, 11491]],\n",
       "\n",
       "        [[12481, 11491, 13545]],\n",
       "\n",
       "        [[11491, 13545, 14730]],\n",
       "\n",
       "        [[13545, 14730, 11416]],\n",
       "\n",
       "        [[14730, 11416, 13402]],\n",
       "\n",
       "        [[11416, 13402, 11907]],\n",
       "\n",
       "        [[13402, 11907, 12711]],\n",
       "\n",
       "        [[11907, 12711, 13261]],\n",
       "\n",
       "        [[12711, 13261, 14265]],\n",
       "\n",
       "        [[13261, 14265,  9564]],\n",
       "\n",
       "        [[14265,  9564, 10415]],\n",
       "\n",
       "        [[ 9564, 10415, 12683]],\n",
       "\n",
       "        [[10415, 12683, 11919]],\n",
       "\n",
       "        [[12683, 11919, 14138]],\n",
       "\n",
       "        [[11919, 14138, 14583]],\n",
       "\n",
       "        [[14138, 14583, 12640]],\n",
       "\n",
       "        [[14583, 12640, 14257]],\n",
       "\n",
       "        [[12640, 14257, 12396]],\n",
       "\n",
       "        [[14257, 12396, 13914]],\n",
       "\n",
       "        [[12396, 13914, 14174]],\n",
       "\n",
       "        [[13914, 14174, 15504]]])"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "expected scalar type Long but found Float",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[1;32m/Users/ajitkumarsingh/Desktop/Time-Series-Analysis-and-Forecasting-with-Python/Time_Series_Forecasting_With_CNNs.ipynb Cell 29\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> <a href='vscode-notebook-cell:/Users/ajitkumarsingh/Desktop/Time-Series-Analysis-and-Forecasting-with-Python/Time_Series_Forecasting_With_CNNs.ipynb#X42sZmlsZQ%3D%3D?line=0'>1</a>\u001b[0m cnn_layer(torch\u001b[39m.\u001b[39;49mtensor([[\u001b[39m12\u001b[39;49m, \u001b[39m23\u001b[39;49m]]))\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py:1501\u001b[0m, in \u001b[0;36mModule._call_impl\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   1496\u001b[0m \u001b[39m# If we don't have any hooks, we want to skip the rest of the logic in\u001b[39;00m\n\u001b[1;32m   1497\u001b[0m \u001b[39m# this function, and just call forward.\u001b[39;00m\n\u001b[1;32m   1498\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m (\u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_backward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_backward_pre_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_hooks \u001b[39mor\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_forward_pre_hooks\n\u001b[1;32m   1499\u001b[0m         \u001b[39mor\u001b[39;00m _global_backward_pre_hooks \u001b[39mor\u001b[39;00m _global_backward_hooks\n\u001b[1;32m   1500\u001b[0m         \u001b[39mor\u001b[39;00m _global_forward_hooks \u001b[39mor\u001b[39;00m _global_forward_pre_hooks):\n\u001b[0;32m-> 1501\u001b[0m     \u001b[39mreturn\u001b[39;00m forward_call(\u001b[39m*\u001b[39;49margs, \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m   1502\u001b[0m \u001b[39m# Do not call functions when jit is used\u001b[39;00m\n\u001b[1;32m   1503\u001b[0m full_backward_hooks, non_full_backward_hooks \u001b[39m=\u001b[39m [], []\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/torch/nn/modules/conv.py:313\u001b[0m, in \u001b[0;36mConv1d.forward\u001b[0;34m(self, input)\u001b[0m\n\u001b[1;32m    312\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mforward\u001b[39m(\u001b[39mself\u001b[39m, \u001b[39minput\u001b[39m: Tensor) \u001b[39m-\u001b[39m\u001b[39m>\u001b[39m Tensor:\n\u001b[0;32m--> 313\u001b[0m     \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_conv_forward(\u001b[39minput\u001b[39;49m, \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mweight, \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mbias)\n",
      "File \u001b[0;32m~/opt/anaconda3/lib/python3.9/site-packages/torch/nn/modules/conv.py:309\u001b[0m, in \u001b[0;36mConv1d._conv_forward\u001b[0;34m(self, input, weight, bias)\u001b[0m\n\u001b[1;32m    305\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mpadding_mode \u001b[39m!=\u001b[39m \u001b[39m'\u001b[39m\u001b[39mzeros\u001b[39m\u001b[39m'\u001b[39m:\n\u001b[1;32m    306\u001b[0m     \u001b[39mreturn\u001b[39;00m F\u001b[39m.\u001b[39mconv1d(F\u001b[39m.\u001b[39mpad(\u001b[39minput\u001b[39m, \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_reversed_padding_repeated_twice, mode\u001b[39m=\u001b[39m\u001b[39mself\u001b[39m\u001b[39m.\u001b[39mpadding_mode),\n\u001b[1;32m    307\u001b[0m                     weight, bias, \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mstride,\n\u001b[1;32m    308\u001b[0m                     _single(\u001b[39m0\u001b[39m), \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mdilation, \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mgroups)\n\u001b[0;32m--> 309\u001b[0m \u001b[39mreturn\u001b[39;00m F\u001b[39m.\u001b[39;49mconv1d(\u001b[39minput\u001b[39;49m, weight, bias, \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mstride,\n\u001b[1;32m    310\u001b[0m                 \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mpadding, \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mdilation, \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mgroups)\n",
      "\u001b[0;31mRuntimeError\u001b[0m: expected scalar type Long but found Float"
     ]
    }
   ],
   "source": [
    "cnn_layer(torch.tensor([[12, 23]]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = nn.Conv1d(16, 33, 3, stride=2)\n",
    "input = torch.randn(20, 16, 50)\n",
    "output = m(input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([20, 33, 24])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output.shape"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}