DS_quick_sample_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a href=\"https://www.bigdatauniversity.com\"><img src=\"pngdslogo.png\" width=\"400\" align=\"center\"></a>\n",
    "\n",
    "<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "In this notebook we try to practice all the classification algorithms that we learned in this course.\n",
    "\n",
    "We load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n",
    "\n",
    "Lets first load required libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: seaborn in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (0.11.1)\n",
      "Requirement already satisfied: scipy>=1.0 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from seaborn) (1.5.4)\n",
      "Requirement already satisfied: pandas>=0.23 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from seaborn) (1.3.2)\n",
      "Requirement already satisfied: numpy>=1.15 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from seaborn) (1.19.5)\n",
      "Requirement already satisfied: matplotlib>=2.2 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from seaborn) (3.4.2)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from matplotlib>=2.2->seaborn) (1.3.1)\n",
      "Requirement already satisfied: pyparsing>=2.2.1 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from matplotlib>=2.2->seaborn) (2.4.7)\n",
      "Requirement already satisfied: cycler>=0.10 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from matplotlib>=2.2->seaborn) (0.10.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from matplotlib>=2.2->seaborn) (2.8.2)\n",
      "Requirement already satisfied: pillow>=6.2.0 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from matplotlib>=2.2->seaborn) (8.3.1)\n",
      "Requirement already satisfied: six in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from cycler>=0.10->matplotlib>=2.2->seaborn) (1.15.0)\n",
      "Requirement already satisfied: pytz>=2017.3 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from pandas>=0.23->seaborn) (2020.1)\n",
      "Requirement already satisfied: plotly in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (4.14.3)\n",
      "Requirement already satisfied: retrying>=1.3.3 in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from plotly) (1.3.3)\n",
      "Requirement already satisfied: six in c:\\users\\mitraind\\anaconda3\\lib\\site-packages (from plotly) (1.15.0)\n"
     ]
    }
   ],
   "source": [
    "!pip install seaborn\n",
    "!pip install plotly\n",
    "!pip install slearn\n",
    "\n",
    "\"\"\"\n",
    "# notice: installing seaborn might takes a few minutes\n",
    "!conda install -c anaconda seaborn -y\n",
    "\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\mitraind\\Anaconda3\\lib\\site-packages\\numpy\\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:\n",
      "C:\\Users\\mitraind\\Anaconda3\\lib\\site-packages\\numpy\\.libs\\libopenblas.NOIJJG62EMASZI6NYURL6JBKM4EVBGM7.gfortran-win_amd64.dll\n",
      "C:\\Users\\mitraind\\Anaconda3\\lib\\site-packages\\numpy\\.libs\\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll\n",
      "  warnings.warn(\"loaded more than 1 DLL from .libs:\\n%s\" %\n"
     ]
    }
   ],
   "source": [
    "import itertools\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.ticker import NullFormatter\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn import preprocessing\n",
    "import io\n",
    "import warnings\n",
    "import requests.exceptions\n",
    "import requests\n",
    "import datetime\n",
    "from datetime import timedelta\n",
    "from datetime import datetime\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### About dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n",
    "\n",
    "| Field          | Description                                                                           |\n",
    "|----------------|---------------------------------------------------------------------------------------|\n",
    "| Loan_status    | Whether a loan is paid off on in collection                                           |\n",
    "| Principal      | Basic principal loan amount at the                                                    |\n",
    "| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n",
    "| Effective_date | When the loan got originated and took effects                                         |\n",
    "| Due_date       | Since it’s one-time payoff schedule, each loan has one single due date                |\n",
    "| Age            | Age of applicant                                                                      |\n",
    "| Education      | Education of applicant                                                                |\n",
    "| Gender         | The gender of applicant                                                               |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets download the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Data From CSV File  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>9/22/2016</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  loan_status  Principal  terms effective_date   due_date  age  \\\n",
       "0     PAIDOFF       1000     30       9/8/2016  10/7/2016   45   \n",
       "1     PAIDOFF       1000     30       9/8/2016  10/7/2016   33   \n",
       "2     PAIDOFF       1000     15       9/8/2016  9/22/2016   27   \n",
       "3     PAIDOFF       1000     30       9/9/2016  10/8/2016   28   \n",
       "4     PAIDOFF       1000     30       9/9/2016  10/8/2016   29   \n",
       "\n",
       "              education  Gender  \n",
       "0  High School or Below    male  \n",
       "1              Bechalor  female  \n",
       "2               college    male  \n",
       "3               college  female  \n",
       "4               college    male  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('loan_train_data.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(346, 8)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  loan_status  Principal  terms effective_date   due_date  age  \\\n",
       "0     PAIDOFF       1000     30     2016-09-08 2016-10-07   45   \n",
       "1     PAIDOFF       1000     30     2016-09-08 2016-10-07   33   \n",
       "2     PAIDOFF       1000     15     2016-09-08 2016-09-22   27   \n",
       "3     PAIDOFF       1000     30     2016-09-09 2016-10-08   28   \n",
       "4     PAIDOFF       1000     30     2016-09-09 2016-10-08   29   \n",
       "\n",
       "              education  Gender  \n",
       "0  High School or Below    male  \n",
       "1              Bechalor  female  \n",
       "2               college    male  \n",
       "3               college  female  \n",
       "4               college    male  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['due_date'] = pd.to_datetime(df['due_date'])\n",
    "df['effective_date'] = pd.to_datetime(df['effective_date'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Data visualization and pre-processing\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let’s see how many of each class is in our data set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PAIDOFF       260\n",
       "COLLECTION     86\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['loan_status'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "260 people have paid off the loan on time while 86 have gone into collection \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets plot some columns to underestand data better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbBklEQVR4nO3de5xVdb3/8ddbnBwRzQuTIoQzKpIg/HY6aWZ2EI3wBnEsxcyk4zmkqcUps9CyTj4yE0rzeAtPhI+OoFSKhnmL4BiWF8BRwAveJpwEBOyRkkCAn98fe824Gfcwl71nZs3e7+fjsR57re9el89i9pfP/n7X2uuriMDMzCxtduruAMzMzPJxgjIzs1RygjIzs1RygjIzs1RygjIzs1RygjIzs1RyguokkvaVNFPSy5IWS/qzpHFF2vcISXOLsa+uIGmBpNrujsO6RynVBUlVkh6T9KSkYzvxOBs6a989iRNUJ5AkYA7wcEQcGBFHAOOBAd0Uz87dcVyzEqwLxwPPRcSHI+KPxYjJWuYE1TlGAv+MiJsbCyLiLxHx3wCSekmaIukJSU9L+lJSPiJpbfxa0nOSbksqOJJGJ2ULgX9t3K+k3SRNT/b1pKSxSfkESb+S9FvgwUJORtIMSTdJmp98C/6X5JjPSpqRs95NkhZJWi7pv1rY16jkG/SSJL4+hcRmqVcydUFSBrgaOElSnaRdW/o8S6qXdGXy3iJJh0t6QNJLks5L1ukjaV6y7dLGePMc9xs5/z5561XJighPRZ6ArwDX7OD9icC3k/ldgEVADTAC+DvZb5c7AX8GPg5UAq8CgwABs4G5yfZXAp9P5vcEVgC7AROABmDvFmL4I1CXZzohz7ozgNuTY48F3gSGJTEuBjLJensnr72ABcDwZHkBUAv0BR4GdkvKvwlc3t1/L0+dN5VgXZgAXJ/Mt/h5BuqB85P5a4Cngd2BKuD1pHxnYI+cfb0IKFnekLyOAqYl57oTMBf4RHf/XbtqctdPF5B0A9nK9c+I+AjZD91wSZ9JVnk/2Qr3T+DxiGhItqsDqoENwCsR8UJS/r9kKzbJvsZIujhZrgQGJvMPRcQb+WKKiPb2n/82IkLSUmBNRCxNYlmexFgHnC5pItmK1w8YQrZiNvpoUvZI8mX4fWT/47EyUSJ1oVFrn+d7ktelQJ+IeAt4S9ImSXsC/wCulPQJ4B2gP7AvsDpnH6OS6clkuQ/Zf5+HOxhzj+IE1TmWA6c1LkTEBZL6kv12CNlvQxdFxAO5G0kaAWzOKdrGu3+jlh6aKOC0iHi+2b6OIlsB8m8k/ZHsN7rmLo6I3+cpb4zrnWYxvgPsLKkGuBj4SET8Len6q8wT60MRcWZLcVnJKcW6kHu8HX2ed1hngLPItqiOiIgtkurJX2d+GBE/20EcJcvXoDrHH4BKSefnlPXOmX8AOF9SBYCkQyTttoP9PQfUSDooWc6tEA8AF+X0z3+4LQFGxLERkckz7ahC7sgeZP8T+LukfYET86zzKHCMpIOTWHtLOqSDx7OeoZTrQqGf5/eT7e7bIuk44IA86zwA/FvOta3+kj7QjmP0aE5QnSCyncefBv5F0iuSHgduJdtHDfA/wDPAEknLgJ+xg9ZsRGwi241xb3Jh+C85b18BVABPJ/u6osin0yYR8RTZbojlwHTgkTzrrCXbhz9L0tNkK/iHujBM62KlXBeK8Hm+DaiVtIhsa+q5PMd4EJgJ/DnpXv81+Vt7JanxgpyZmVmquAVlZmap5ARlZmap5ARlZmap5ARlZmaplIoENXr06CD72wZPnkphKirXD08lNrVZKhLUunXrujsEs9Ry/bBylYoEZWZm1pwTlJmZpZITlJmZpZIfFmtmJWXLli00NDSwadOm7g6lrFVWVjJgwAAqKio6vA8nKDMrKQ0NDey+++5UV1eTPDfWulhEsH79ehoaGqipqenwftzFZ2YlZdOmTeyzzz5OTt1IEvvss0/BrVgnKCsbB/Trh6SCpwP69evuU7FWODl1v2L8DdzFZ2Vj5erVNOw/oOD9DHitoQjRmFlr3IIys5JWrJZze1rQvXr1IpPJcNhhh/HZz36Wt99+G4CtW7fSt29fJk+evN36I0aMYNGi7CDD1dXVDBs2jGHDhjFkyBC+/e1vs3nzuwPyLl++nJEjR3LIIYcwaNAgrrjiChqHTZoxYwZVVVVkMhkymQxf+MIXAJgwYQI1NTVN5dddd11R/m07m1tQZlbSitVybtSWFvSuu+5KXV0dAGeddRY333wzX/va13jwwQcZPHgws2fP5sorr2yxG2z+/Pn07duXDRs2MHHiRCZOnMitt97Kxo0bGTNmDDfddBOjRo3i7bff5rTTTuPGG2/kggsuAOCMM87g+uuvf88+p0yZwmc+85mOn3g3aLUFJWm6pNeTESoby74n6a+S6pLppJz3Jkt6UdLzkj7VWYGbmfUExx57LC+++CIAs2bN4qtf/SoDBw7k0UcfbXXbPn36cPPNNzNnzhzeeOMNZs6cyTHHHMOoUaMA6N27N9dffz1XXXVVp55Dd2lLF98MYHSe8msiIpNMvwOQNAQYDwxNtrlRUq9iBWtm1pNs3bqV++67j2HDhrFx40bmzZvHKaecwplnnsmsWbPatI899tiDmpoaXnjhBZYvX84RRxyx3fsHHXQQGzZs4M033wTgjjvuaOrK+8UvftG03je+8Y2m8qVLlxbvJDtRqwkqIh4G3mjj/sYCt0fE5oh4BXgROLKA+MzMepyNGzeSyWSora1l4MCBnHvuucydO5fjjjuO3r17c9ppp3HXXXexbdu2Nu2v8RpTRLTYLdhYfsYZZ1BXV0ddXR1f/OIXm96fMmVKU/mwYcMKPMOuUcg1qAslfQFYBHw9Iv4G9Ady260NSdl7SJoITAQYOHBgAWGYlR7Xj54t9xpUo1mzZvHII49QXV0NwPr165k/fz4nnHDCDvf11ltvUV9fzyGHHMLQoUN5+OGHt3v/5Zdfpk+fPuy+++7FPIVU6OhdfDcBBwEZYBXw46Q8X2rPO/5HREyLiNqIqK2qqupgGGalyfWjtLz55pssXLiQlStXUl9fT319PTfccEOr3XwbNmzgy1/+Mp/+9KfZa6+9OOuss1i4cCG///3vgWxL7Stf+QqXXHJJV5xGl+tQCyoi1jTOS7oFmJssNgAfzFl1APBah6MzMyvQwP32K+pv1wbut1+7t7nzzjsZOXIku+yyS1PZ2LFjueSSS7a7hbzRcccdR0TwzjvvMG7cOL7zne8A2ZbZ3XffzUUXXcQFF1zAtm3bOPvss7nwwgs7fkIppsa+zR2uJFUDcyPisGS5X0SsSub/EzgqIsZLGgrMJHvdaX9gHjAoInbY0VpbWxuNvwEw6yySivZD3VbqTVEfY+D60T7PPvsshx56aHeHYbT4t2hz/Wi1BSVpFjAC6CupAfguMEJShmz3XT3wJYCIWC5pNvAMsBW4oLXkZGZmlk+rCSoizsxT/PMdrP8D4AeFBGVmZuZHHZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZlZSdt/wMCiDrex/4DWn+yxevVqxo8fz0EHHcSQIUM46aSTWLFiRatDZeT7PVN1dTXr1q3brqz5sBqZTIZnnnkGgBUrVnDSSSdx8MEHc+ihh3L66adv93y+Pn36MHjw4KbhOBYsWMApp5zStO85c+YwfPhwPvShDzFs2DDmzJnT9N6ECRPo379/02+31q1b1/RkjM7g4TbMrKSt+uurHHX5/UXb32Pfz/fs7HdFBOPGjeOcc87h9ttvB6Curo41a9YwYcKEHQ6V0R75htXYtGkTJ598Mj/5yU849dRTgezQHVVVVU2PXhoxYgRTp06ltrYWgAULFjRt/9RTT3HxxRfz0EMPUVNTwyuvvMInP/lJDjzwQIYPHw5kx7qaPn06559/frtjbi+3oMzMimj+/PlUVFRw3nnnNZVlMhlWrFjR6UNlzJw5k6OPPropOUH2qRSHHXZYm7afOnUql156KTU1NQDU1NQwefJkpkyZ0rTOpEmTuOaaa9i6dWvR4m6JE5SZWREtW7bsPUNiAG0aKqM9crvtMpkMGzdubPHYbZUvxtraWpYvX960PHDgQD7+8Y/zy1/+ssPHaSt38ZmZdYG2DJXRHi2NnFuIfDHmK7v00ksZM2YMJ598clGP35xbUGZmRTR06FAWL16ct7z5MxWLPVRGS8duz/bNY1yyZAlDhgzZruzggw8mk8kwe/bsDh+rLZygzMyKaOTIkWzevJlbbrmlqeyJJ55g0KBBnT5Uxuc+9zn+9Kc/ce+99zaV3X///W0eQffiiy/mhz/8IfX19QDU19dz5ZVX8vWvf/0961522WVMnTq1KHG3xF18ZlbS+vX/YKt33rV3fzsiibvuuotJkyZx1VVXUVlZSXV1Nddee22rQ2XMmDFju9u6H300O/7r8OHD2WmnbHvi9NNPZ/jw4dxxxx0sXLiwad0bb7yRj33sY8ydO5dJkyYxadIkKioqGD58OD/96U/bdG6ZTIYf/ehHnHrqqWzZsoWKigquvvpqMpnMe9YdOnQohx9+OEuWLGnTvjuiTcNtdDYPJ2BdwcNtlAcPt5EehQ630WoXn6Tpkl6XtCynbIqk5yQ9LekuSXsm5dWSNkqqS6ab2xqImZlZrrZcg5oBNG8fPwQcFhHDgRXA5Jz3XoqITDKdh5mZWQe0mqAi4mHgjWZlD0ZE46+0HiU7tLuZWSqk4dJFuSvG36AYd/H9G3BfznKNpCcl/Z+kY1vaSNJESYskLVq7dm0RwjArHa4fHVdZWcn69eudpLpRRLB+/XoqKysL2k9Bd/FJuozs0O63JUWrgIERsV7SEcAcSUMj4j0/k46IacA0yF4ELiQOs1Lj+tFxAwYMoKGhASf27lVZWcmAAYV1rnU4QUk6BzgFOD6SryoRsRnYnMwvlvQScAjgW5DMrEtUVFQ0PUvOerYOdfFJGg18ExgTEW/nlFdJ6pXMHwgMAl4uRqBmZlZeWm1BSZoFjAD6SmoAvkv2rr1dgIeSZzQ9mtyx9wng+5K2AtuA8yLijbw7NjMz24FWE1REnJmn+OctrPsb4DeFBmVmZuZn8ZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSo5QZmZWSq1mqAkTZf0uqRlOWV7S3pI0gvJ6145702W9KKk5yV9qrMCNzOz0taWFtQMYHSzsm8B8yJiEDAvWUbSEGA8MDTZ5sbGEXbNzMzao9UEFREPA81HxR0L3JrM3wp8Oqf89ojYHBGvAC8CRxYnVDMzKycdvQa1b0SsAkheP5CU9wdezVmvISl7D0kTJS2StGjt2rUdDMOsNLl+mBX/JgnlKYt8K0bEtIiojYjaqqqqIodh1rO5fph1PEGtkdQPIHl9PSlvAD6Ys94A4LWOh2dmZuWqownqHuCcZP4c4O6c8vGSdpFUAwwCHi8sRDMzK0c7t7aCpFnACKCvpAbgu8BVwGxJ5wIrgc8CRMRySbOBZ4CtwAURsa2TYjczsxLWaoKKiDNbeOv4Ftb/AfCDQoIyMzPzkyTMzCyVnKDMzCyVnKDMzCyVnKDMzCyVnKDMzCyVnKDMzCyVnKDMzCyVnKDMzCyVnKDMzCyVnKDMzCyVnKDMzCyVnKDMzCyVnKDMzCyVWn2aeUskDQbuyCk6ELgc2BP4D6BxnOpLI+J3HT2OmZmVpw4nqIh4HsgASOoF/BW4C/gicE1ETC1GgGZmVp6K1cV3PPBSRPylSPszM7MyV6wENR6YlbN8oaSnJU2XtFe+DSRNlLRI0qK1a9fmW8WsbLl+mBUhQUl6HzAG+FVSdBNwENnuv1XAj/NtFxHTIqI2ImqrqqoKDcOspLh+mBWnBXUisCQi1gBExJqI2BYR7wC3AEcW4RhmZlZmipGgziSne09Sv5z3xgHLinAMMzMrMx2+iw9AUm/gk8CXcoqvlpQBAqhv9p6ZmVmbFJSgIuJtYJ9mZWcXFJGZmRl+koSZmaWUE5SZmaWSE5SZmaWSE5SZmaWSE5SZmaWSE5SZmaVSQbeZm/Uk6lXBgNcairIfM+t8TlBWNmLbFo66/P6C9/PY90cXIRoza427+MzMLJWcoMzMLJWcoMzMLJWcoMzMLJWcoMzMLJWcoMzMLJUKHQ+qHngL2AZsjYhaSXsDdwDVZMeDOj0i/lZYmGZmVm6K0YI6LiIyEVGbLH8LmBcRg4B5ybKVoQP69UNSwdMB/fq1fjAzKzmd8UPdscCIZP5WYAHwzU44jqXcytWradh/QMH7KcbTH8ys5ym0BRXAg5IWS5qYlO0bEasAktcP5NtQ0kRJiyQtWrt2bYFhmJUW1w+zwhPUMRFxOHAicIGkT7R1w4iYFhG1EVFbVVVVYBhmpcX1w6zABBURryWvrwN3AUcCayT1A0heXy80SDMzKz8dTlCSdpO0e+M8MApYBtwDnJOsdg5wd6FBmplZ+SnkJol9gbskNe5nZkTcL+kJYLakc4GVwGcLD9PMzMpNhxNURLwM/L885euB4wsJyszMzE+SMDOzVHKCMjOzVHKCMjOzVHKCMjOzVHKCMjOzVHKCMjOzVHKCMjOzVHKCMjOzVHKCMjOzVHKCMjOzVHKCMjMrc2kd/bozRtQ1M7MeJK2jX7sFZWZmqVTIeFAflDRf0rOSlkv6alL+PUl/lVSXTCcVL1wzMysXhXTxbQW+HhFLkoELF0t6KHnvmoiYWnh4ZmZWrgoZD2oVsCqZf0vSs0D/YgVmZmblrSjXoCRVAx8GHkuKLpT0tKTpkvZqYZuJkhZJWrR27dpihGFWMlw/zIqQoCT1AX4DTIqIN4GbgIOADNkW1o/zbRcR0yKiNiJqq6qqCg3DrKS4fpgVmKAkVZBNTrdFxJ0AEbEmIrZFxDvALcCRhYdpZmblppC7+AT8HHg2In6SU577S61xwLKOh2dmZuWqkLv4jgHOBpZKqkvKLgXOlJQBAqgHvlTAMczMrEwVchffQkB53vpdx8MxMzPL8pMkzMwslfwsPus06lVRlGdzqVdFEaIxs57GCco6TWzbwlGX31/wfh77/ugiRGNmPY27+MzMLJWcoMzMLJWcoMzMLJWcoMzMLJWcoMzMulhah1hPG9/FZ2bWxdI6xHrauAVlZmap5ARlZmap5C4+M7Myl9anvjhBmZmVubQ+9cVdfGZmlkqdlqAkjZb0vKQXJX2r0P35tkwzs/LSKV18knoBNwCfBBqAJyTdExHPdHSfvi3TzKy8dNY1qCOBFyPiZQBJtwNjgQ4nqLQ5oF8/Vq5eXfB+Bu63H39ZtaoIEZU2Kd/YmJZGrhutK9ZNCTv1qijpuqGIKP5Opc8AoyPi35Pls4GjIuLCnHUmAhOTxcHA80UPpO36Auu68fiF6Kmx99S4ofXY10VEQVeLU1Q/SvnvlGY9Nfa2xN3m+tFZLah8KX27TBgR04BpnXT8dpG0KCJquzuOjuipsffUuKFrYk9L/fDfqXv01NiLHXdn3STRAHwwZ3kA8FonHcvMzEpQZyWoJ4BBkmokvQ8YD9zTSccyM7MS1CldfBGxVdKFwANAL2B6RCzvjGMVSbd3pRSgp8beU+OGnh17e/Xkc3XsXa+ocXfKTRJmZmaF8pMkzMwslZygzMwslcomQUnqJelJSXOT5b0lPSTpheR1r5x1JyePaHpe0qe6L2qQtKekX0t6TtKzko7uCbFL+k9JyyUtkzRLUmVa45Y0XdLrkpbllLU7VklHSFqavHedesgvKF03uiV214+21I+IKIsJ+BowE5ibLF8NfCuZ/xbwo2R+CPAUsAtQA7wE9OrGuG8F/j2Zfx+wZ9pjB/oDrwC7JsuzgQlpjRv4BHA4sCynrN2xAo8DR5P9HeB9wInd9blp5/m7bnRt3K4fbawf3V45uugfeAAwDxiZUwmfB/ol8/2A55P5ycDknG0fAI7uprj3SD7Ialae6tiTCvgqsDfZO0XnAqPSHDdQ3awCtivWZJ3ncsrPBH7WHZ+bdp6360bXx+760cb6US5dfNcClwDv5JTtGxGrAJLXDyTljR+eRg1JWXc4EFgL/CLpgvkfSbuR8tgj4q/AVGAlsAr4e0Q8SMrjbqa9sfZP5puXp921uG50KdeP7cp3qOQTlKRTgNcjYnFbN8lT1l334u9Mtml9U0R8GPgH2eZ0S1IRe9IfPZZsE39/YDdJn9/RJnnK0vr7h5Zi7UnnALhu0E2xu35sV75DJZ+ggGOAMZLqgduBkZL+F1gjqR9A8vp6sn6aHtPUADRExGPJ8q/JVsq0x34C8EpErI2ILcCdwMdIf9y52htrQzLfvDzNXDe6h+tHG8+h5BNUREyOiAERUU32kUt/iIjPk3300jnJaucAdyfz9wDjJe0iqQYYRPbiXpeLiNXAq5IGJ0XHkx2yJO2xrwQ+Kql3cqfO8cCzpD/uXO2KNenmeEvSR5Nz/kLONqnkutFtnzHXj7bWj+64SNhdEzCCdy8E70P24vALyeveOetdRvbuk+fp5juxgAywCHgamAPs1RNiB/4LeA5YBvyS7F09qYwbmEX2WsAWst/0zu1IrEBtcr4vAdfT7AJ+mifXjS6P3fWjDfXDjzoyM7NUKvkuPjMz65mcoMzMLJWcoMzMLJWcoMzMLJWcoMzMLJWcoFJM0jZJdckTj38lqXcL6/2pg/uvlXRdAfFt6Oi2ZoVw3SgPvs08xSRtiIg+yfxtwOKI+EnO+70iYlsa4jPrSq4b5cEtqJ7jj8DBkkZImi9pJrAU3v22lry3QO+OkXNb45grkj4i6U+SnpL0uKTdk/UbxwD6nqRfSvpDMsbLfyTlfSTNk7QkGctlbPecvlmLXDdK1M7dHYC1TtLOwInA/UnRkcBhEfFKntU/DAwl+5yrR4BjJD0O3AGcERFPSNoD2Jhn2+HAR4HdgCcl3Uv2GVvjIuJNSX2BRyXdE256Wwq4bpQ2t6DSbVdJdWQf57IS+HlS/ngLFbDxvYaIeAeoIzuOy2BgVUQ8ARARb0bE1jzb3h0RGyNiHTCfbGUXcKWkp4Hfk31E/r7FODmzArhulAG3oNJtY0RkcguSXol/7GCbzTnz28j+jUXbHs/ffJ0AzgKqgCMiYouyT76ubMO+zDqT60YZcAuqPDwH7C/pIwBJH3u+LydjJVVK2ofsw0OfAN5PdsygLZKOAw7oqqDNuoDrRoq5BVUGIuKfks4A/lvSrmT72E/Is+rjwL3AQOCKiHgtuUPqt5IWke0Wea6LwjbrdK4b6ebbzA3I3qkEbIiIqd0di1mauG50H3fxmZlZKrkFZWZmqeQWlJmZpZITlJmZpZITlJmZpZITlJmZpZITlJmZpdL/B7A+/1urYJiLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "bins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZB0lEQVR4nO3de5QU5bnv8e9PmDgiGEFGGR1hRsULChl1djTBJIjKYXtDj5dojIF1POFo8MKKxqi5rJPtWoREl5psbyHRwEoCyt5RcJMVFQkcg1EjIl4QIx4d2bPlrolyBALynD+6ZjLAwPQM1dPVPb/PWrW66+3qt56X6Zen663qehURmJmZZc1exQ7AzMysLU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QKZN0kKTpkt6W9KKkZyWdn1LdIyTNSaOuriBpgaSGYsdhxVdO/UJSlaTnJb0k6QsF3M+GQtVdKpygUiRJwCzg6Yg4LCJOBC4BaooUT89i7NestTLsF6cBb0TE8RHxxzRisrY5QaVrJPD3iLi/uSAi3o2IfwWQ1EPSbZJekPSKpP+VlI9Ijjb+XdIbkn6TdGokjU7KFgL/vbleSftKejCp6yVJY5LycZL+TdJ/AE/uSWMkTZV0n6T5yTffLyX7XCZpaqvt7pO0SNJSST/YRV2jkm/Ni5P4eu9JbFZSyqZfSKoHfgycKWmJpH129dmW1ChpUvLaIkknSHpC0v+VdGWyTW9J85L3vtocbxv7/Varf582+1hZiggvKS3AtcCdu3l9PPDd5PnewCKgDhgB/I3cN8q9gGeBU4BK4D+BwYCAmcCc5P2TgK8mz/cH3gT2BcYBTUC/XcTwR2BJG8vpbWw7FXgo2fcY4ENgaBLji0B9sl2/5LEHsAAYlqwvABqA/sDTwL5J+beB7xf77+Wla5Yy7BfjgLuT57v8bAONwFXJ8zuBV4A+QBWwJinvCezXqq63ACXrG5LHUcCUpK17AXOALxb779oVi4eACkjSPeQ61N8j4p/IfdCGSbow2eTT5DrZ34E/R0RT8r4lQC2wAXgnIpYn5b8m15lJ6jpX0g3JeiUwMHk+NyLebyumiOjomPl/RERIehVYHRGvJrEsTWJcAlwsaTy5zlYNDCHXGZudnJQ9k3wB/hS5/2ysGyqTftGsvc/2Y8njq0DviPgI+EjSJkn7A/8PmCTpi8A24BDgIGBVqzpGJctLyXpvcv8+T3cy5pLhBJWupcAFzSsRMUFSf3LfCCH3DeiaiHii9ZskjQA2tyr6hH/8bXZ1s0QBF0TEX3ao6yRyH/q23yT9kdy3uB3dEBFPtVHeHNe2HWLcBvSUVAfcAPxTRHyQDP1VthHr3Ii4dFdxWVkrx37Ren+7+2zvtv8Al5E7ojoxIrZIaqTt/vPDiPjZbuIoSz4Hla4/AJWSrmpV1qvV8yeAqyRVAEg6UtK+u6nvDaBO0uHJeutO8ARwTasx+ePzCTAivhAR9W0su+uEu7MfuY7/N0kHAf/cxjbPAcMlHZHE2kvSkZ3cn5Wecu4Xe/rZ/jS54b4tkk4FBrWxzRPA/2h1busQSQd2YB8lywkqRZEbMD4P+JKkdyT9GZhGblwa4BfA68BiSa8BP2M3R7ERsYnc0MXvkpPB77Z6+VagAnglqevWlJuTl4h4mdzQw1LgQeCZNrZZS27cfoakV8h16qO7MEwronLuFyl8tn8DNEhaRO5o6o029vEkMB14Nhlq/3faPtorO80n48zMzDLFR1BmZpZJTlBmZpZJTlBmZpZJTlBmZpZJXZqgRo8eHeR+v+DFS3dYOsX9xEs3XNrUpQlq3bp1Xbk7s5LkfmKW4yE+MzPLJCcoMzPLJCcoMzPLJN8s1szK3pYtW2hqamLTpk3FDqVbq6yspKamhoqKiry2d4Iys7LX1NREnz59qK2tJbmPrHWxiGD9+vU0NTVRV1eX13s8xGdmZW/Tpk0ccMABTk5FJIkDDjigQ0exTlBFMKi6GkmpLIOqq4vdHLOS4ORUfB39G3iIrwhWrFpF08E1qdRV815TKvWYmWWNj6DMrNtJcxQj35GMHj16UF9fz3HHHcdFF13Exx9/DMDWrVvp378/N99883bbjxgxgkWLcpMO19bWMnToUIYOHcqQIUP47ne/y+bN/5igd+nSpYwcOZIjjzySwYMHc+utt9I8ldLUqVOpqqqivr6e+vp6vva1rwEwbtw46urqWsp/+tOfpvJvmyYfQZlZt5PmKAbkN5Kxzz77sGTJEgAuu+wy7r//fr75zW/y5JNPctRRRzFz5kwmTZq0y2Gw+fPn079/fzZs2MD48eMZP34806ZNY+PGjZx77rncd999jBo1io8//pgLLriAe++9lwkTJgDw5S9/mbvvvnunOm+77TYuvPDCzje8wHwEZWbWxb7whS/w1ltvATBjxgyuu+46Bg4cyHPPPdfue3v37s3999/PrFmzeP/995k+fTrDhw9n1KhRAPTq1Yu7776byZMnF7QNXcEJysysC23dupXf//73DB06lI0bNzJv3jzOPvtsLr30UmbMmJFXHfvttx91dXUsX76cpUuXcuKJJ273+uGHH86GDRv48MMPAXj44YdbhvJ++ctftmz3rW99q6X81VdfTa+RKfEQn5lZF9i4cSP19fVA7gjqiiuuYPbs2Zx66qn06tWLCy64gFtvvZU777yTHj16tFtf8zmmiNjlsGBzeakO8eWVoCQ1Ah8BnwBbI6JBUj/gYaAWaAQujogPChOmmVlpa30OqtmMGTN45plnqK2tBWD9+vXMnz+f008/fbd1ffTRRzQ2NnLkkUdy7LHH8vTTT2/3+ttvv03v3r3p06dPmk3och0Z4js1IuojoiFZvwmYFxGDgXnJupmZ5eHDDz9k4cKFrFixgsbGRhobG7nnnnvaHebbsGED3/jGNzjvvPPo27cvl112GQsXLuSpp54Cckdq1157LTfeeGNXNKOg9mSIbwwwInk+DVgAfHsP4zEzK7iBAwak+hvCgQMGdPg9jzzyCCNHjmTvvfduKRszZgw33njjdpeQNzv11FOJCLZt28b555/P9773PSB3ZDZ79myuueYaJkyYwCeffMLll1/O1Vdf3fkGZYSaxzF3u5H0DvABuZkPfxYRUyT9NSL2b7XNBxHRt433jgfGAwwcOPDEd999N63YS5akVH+om8/f0Ioi75/Nu58U1rJlyzjmmGOKHYaxy79Fm30l3yG+4RFxAvDPwARJX8w3mIiYEhENEdFQVVWV79vMuhX3E7Od5ZWgIuK95HEN8CjwWWC1pGqA5HFNoYI0M7Pup90EJWlfSX2anwOjgNeAx4CxyWZjgdmFCtLMzLqffC6SOAh4NLmevicwPSIel/QCMFPSFcAK4KLChWlmZt1NuwkqIt4GPtNG+XrgtEIEZWZm5lsdmZlZJjlBmVm3c3DNwFSn2zi4ZmC7+1y1ahWXXHIJhx9+OEOGDOHMM8/kzTffbHeqjLZ+z1RbW8u6deu2K9txWo36+npef/11AN58803OPPNMjjjiCI455hguvvji7e7P17t3b4466qiW6TgWLFjA2Wef3VL3rFmzGDZsGEcffTRDhw5l1qxZLa+NGzeOQw45pOW3W+vWrWu5M8ae8r348jSoupoVq1YVOwwzS8HK//pPTvr+46nV9/y/jN7t6xHB+eefz9ixY3nooYcAWLJkCatXr2bcuHG7nSqjI9q6596mTZs466yzuOOOOzjnnHOA3NQdVVVVLbdeGjFiBLfffjsNDbkbBS1YsKDl/S+//DI33HADc+fOpa6ujnfeeYczzjiDww47jGHDhgG5ua4efPBBrrrqqg7HvDtOUHnyLLhm1lnz58+noqKCK6+8sqWsvr6eBx54oM2pMkaMGNGpBNWW6dOn87nPfa4lOUHurhT5uv3227nllluoq6sDoK6ujptvvpnbbruNX/3qVwBMnDiRO++8k69//eupxNzMQ3xmZgX22muv7TQlBpDXVBkd0XrYrr6+no0bN+5y3/lqK8aGhgaWLl3asj5w4EBOOeWUloSVFh9BmZkVST5TZXTErqbV2BNtxdhW2S233MK5557LWWedldq+fQRlZlZgxx57LC+++GKb5YsWLdquLO2pMna17468f8cYFy9ezJAhQ7YrO+KII6ivr2fmzJmd3teOnKDMzAps5MiRbN68mZ///OctZS+88AKDBw8u+FQZX/nKV/jTn/7E7373u5ayxx9/PO8ZdG+44QZ++MMf0tjYCEBjYyOTJk3i+uuv32nb73znO9x+++2pxA0e4jOzbqj6kEPbvfKuo/XtjiQeffRRJk6cyOTJk6msrKS2tpa77rqr3akypk6dut1l3c899xwAw4YNY6+9cscYF198McOGDePhhx9m4cKFLdvee++9fP7zn2fOnDlMnDiRiRMnUlFRwbBhw/jJT36SV9vq6+v50Y9+xDnnnMOWLVuoqKjgxz/+ccvswK0de+yxnHDCCSxevDivutuT13QbaWloaIgdDxVLRdpTZHi6jW6h4ycRKO1+klWebiM7CjHdhpmZWZdygjIzs0xygjKzbsFD4cXX0b+BE5SZlb3KykrWr1/vJFVEEcH69euprKzM+z2+is/Myl5NTQ1NTU2sXbu22KF0a5WVldTU5H+BmBNUidubzv3ivC0DBwzg3ZUrU6nLLEsqKipa7iVnpcMJqsRtBt/E1szKUt7noCT1kPSSpDnJej9JcyUtTx77Fi5MMzPrbjpykcR1wLJW6zcB8yJiMDAvWTczM0tFXglKUg1wFvCLVsVjgGnJ82nAealGZmZm3Vq+R1B3ATcC21qVHRQRKwGSxwPbeqOk8ZIWSVrkK2jM2uZ+YrazdhOUpLOBNRHRqfu1R8SUiGiIiIaqqqrOVGFW9txPzHaWz1V8w4FzJZ0JVAL7Sfo1sFpSdUSslFQNrClkoGZm1r20ewQVETdHRE1E1AKXAH+IiK8CjwFjk83GArMLFqWZmXU7e3Kro8nAGZKWA2ck62ZmZqno0A91I2IBsCB5vh44Lf2QzMzMfLNYMzPLKCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpHYTlKRKSX+W9LKkpZJ+kJT3kzRX0vLksW/hwzUzs+4inyOozcDIiPgMUA+MlnQycBMwLyIGA/OSdTMzs1S0m6AiZ0OyWpEsAYwBpiXl04DzChGgmZl1T3mdg5LUQ9ISYA0wNyKeBw6KiJUAyeOBu3jveEmLJC1au3ZtSmGblRf3E7Od5ZWgIuKTiKgHaoDPSjou3x1ExJSIaIiIhqqqqk6GaVbe3E/Mdtahq/gi4q/AAmA0sFpSNUDyuCbt4MzMrPvK5yq+Kkn7J8/3AU4H3gAeA8Ymm40FZhcoRjMz64Z65rFNNTBNUg9yCW1mRMyR9CwwU9IVwArgogLGaWZm3Uy7CSoiXgGOb6N8PXBaIYIyMzPznSTMzCyTnKDMzCyTnKDMzCyTnKDMzCyTyjpBDaquRlIqi5mZda18LjMvWStWraLp4JpU6qp5rymVeszMLD9lfQRlZmalywnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqd0EJelQSfMlLZO0VNJ1SXk/SXMlLU8e+xY+XDMz6y7yOYLaClwfEccAJwMTJA0BbgLmRcRgYF6ybmZmlop2E1RErIyIxcnzj4BlwCHAGGBastk04LwCxWhmZt1Qh85BSaoFjgeeBw6KiJWQS2LAgbt4z3hJiyQtWrt27R6Ga1ae3E/MdpZ3gpLUG/gtMDEiPsz3fRExJSIaIqKhqqqqMzGalT33E7Od5ZWgJFWQS06/iYhHkuLVkqqT16uBNYUJ0czMuqN8ruIT8ACwLCLuaPXSY8DY5PlYYHb64VlX2ht2O+19R5ZB1dXFbo6Zlbh8pnwfDlwOvCppSVJ2CzAZmCnpCmAFcFFBIrQusxloOrgmlbpq3mtKpR4z677aTVARsRDQLl4+Ld1wsks9KlL7T1c9P5VeXT0qUqnHzCxr8jmCMiA+2cJJ3388lbqe/5fRqdZlZlaOfKsjMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpLK+k0SatycyM7OuVdYJKu3bE5mZWdfxEJ+ZmWWSE5SZmWWSE5SZmWVSWZ+D6g5SnafKc0tZhgyqrmbFqlWp1LXPXj3YuO2TVOoaOGAA765cmUpdtntOUCXOF4JYuVqxalWqMzx7tujS0+4Qn6QHJa2R9Fqrsn6S5kpanjz2LWyYZmbW3eRzDmoqsONX65uAeRExGJiXrJu12BuQlMoyqLq62M0xsyJod4gvIp6WVLtD8RhgRPJ8GrAA+HaagVlp2wweUjGzPdLZq/gOioiVAMnjgbvaUNJ4SYskLVq7dm0nd2dW3sqlnwyqrk7tyNms4BdJRMQUYApAQ0NDFHp/ZqWoXPpJ2hc2WPfW2SOo1ZKqAZLHNemFZGZm1vkE9RgwNnk+FpidTjhmZmY5+VxmPgN4FjhKUpOkK4DJwBmSlgNnJOtmZmapyecqvkt38dJpKcdiZmbWInP34vNVQGZmBhm81ZGvAjIzM8hggrLi8Y1nzSxLnKCshW88a2ZZkrlzUGZmZuAEZWZmGeUEZWZmmeQEZWZmmeQEZZnnuaUKy789tKzyVXyWeZ5bqrD820PLKicoKwj/psrM9pQTlBWEf1NlZnvK56DMzCyTfARlmZfmcOFePSpSO5k/cMAA3l25MpW6ykWqQ7s9P+Vh4g4YVF3NilWrUqkrK59tJyjLvLSHC31BQOGk/bfyMHH+yvFiFw/xmZlZJmXuCCrNIQIzMytdmUtQvvrLzMxgDxOUpNHAT4AewC8iYnIqUZkVSLn8PivNE+LWMWleaLNXzwq2bd2SSl3lqNMJSlIP4B7gDKAJeEHSYxHxelrBmaWtXI7Qy/GEeKnY5ot2usyeXCTxWeCtiHg7Iv4OPASMSScsMzPr7hQRnXujdCEwOiL+Z7J+OXBSRFy9w3bjgfHJ6lHAXzof7nb6A+tSqisL3J7s6mxb1kVEXodZ7id5c3uyLdW+sifnoNoahN0p20XEFGDKHuyn7Z1LiyKiIe16i8Xtya6uaIv7SX7cnmxLuz17MsTXBBzaar0GeG/PwjEzM8vZkwT1AjBYUp2kTwGXAI+lE5aZmXV3nR7ii4itkq4GniB3mfmDEbE0tcjal/pwSJG5PdlVym0p5djb4vZkW6rt6fRFEmZmZoXke/GZmVkmOUGZmVkmZT5BSTpU0nxJyyQtlXRdUt5P0lxJy5PHvsWONR+SKiX9WdLLSXt+kJSXZHuaSeoh6SVJc5L1km2PpEZJr0paImlRUpb59rivZJ/7ScdkPkEBW4HrI+IY4GRggqQhwE3AvIgYDMxL1kvBZmBkRHwGqAdGSzqZ0m1Ps+uAZa3WS709p0ZEfavfdJRCe9xXss/9pCMioqQWYDa5+//9BahOyqqBvxQ7tk60pRewGDiplNtD7jdw84CRwJykrJTb0wj036Gs5NrjvpKtxf2k40spHEG1kFQLHA88DxwUESsBkscDixhahySH+UuANcDciCjp9gB3ATcC21qVlXJ7AnhS0ovJLYigxNrjvpJJd+F+0iGZmw9qVyT1Bn4LTIyID9O63X0xRMQnQL2k/YFHJR1X5JA6TdLZwJqIeFHSiCKHk5bhEfGepAOBuZLeKHZAHeG+kj3uJ51TEkdQkirIdbjfRMQjSfFqSdXJ69XkvmGVlIj4K7AAGE3ptmc4cK6kRnJ3tB8p6deUbnuIiPeSxzXAo+Tu3F8S7XFfySz3k07IfIJS7uvfA8CyiLij1UuPAWOT52PJjbdnnqSq5NsgkvYBTgfeoETbExE3R0RNRNSSu93VHyLiq5RoeyTtK6lP83NgFPAaJdAe95Xscj/ppGKfaMvjRNwp5MY6XwGWJMuZwAHkTjguTx77FTvWPNszDHgpac9rwPeT8pJszw5tG8E/Tv6WZHuAw4CXk2Up8J1SaY/7Smks7if5L77VkZmZZVLmh/jMzKx7coIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIqA5JmJTdsXNp800ZJV0h6U9ICST+XdHdSXiXpt5JeSJbhxY3erOu4r5QW/1C3DEjqFxHvJ7eDeQH4b8AzwAnAR8AfgJcj4mpJ04F7I2KhpIHAE5GbP8is7LmvlJaSuZu57da1ks5Pnh8KXA78n4h4H0DSvwFHJq+fDgxpdYfr/ST1iYiPujJgsyJxXykhTlAlLrl1/+nA5yLiY0kLyE0atqtvensl227skgDNMsJ9pfT4HFTp+zTwQdLhjiY31Xcv4EuS+krqCVzQavsngaubVyTVd2WwZkXkvlJinKBK3+NAT0mvALcCzwH/BUwiN5vqU8DrwN+S7a8FGiS9Iul14MquD9msKNxXSowvkihTknpHxIbkW+GjwIMR8Wix4zLLGveV7PIRVPn635KWkJtH5x1gVlGjMcsu95WM8hGUmZllko+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk5ygzMwsk/4/w0FgvqziN4oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bins = np.linspace(df.age.min(), df.age.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'age', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Pre-processing:  Feature selection/extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZtklEQVR4nO3de3hU9b3v8fdHSI0I1htqJIVExQsIO2p6rFVbxMtDvYHbe9GCx25OrTeOpW61tj27nsdS8fHS7a3WqrQVlFpvpacqUtiKFStiFBGLbk0xFRSwrVJBQb/nj1lJAwQySdZkFjOf1/PMMzNr1vqt7wr58p3fbya/nyICMzOzrNmq2AGYmZm1xQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQXKzMwyyQUqZZJ2lTRF0huSnpf0jKSTUmp7mKTpabTVHSTNllRf7Dis+EopLyT1lfSspBckHV7A86wqVNtbCheoFEkS8BDwZETsEREHAWcA1UWKp2cxzmvWWgnmxZHAqxFxQEQ8lUZM1jYXqHQNBz6OiNuaN0TEnyPiPwEk9ZA0SdJzkl6S9L+S7cOS3sb9kl6VdE+S1EgakWybA/xrc7uStpV0Z9LWC5JGJtvHSvqVpN8Aj3flYiTdLelWSbOSd75fTs65SNLdrfa7VdI8SQsl/ccm2jomedc8P4mvd1disy1KyeSFpDrgGuBYSQ2SttnU77akRklXJ6/Nk3SgpMck/bekbyT79JY0Mzl2QXO8bZz3261+Pm3mWEmKCN9SugEXAddv5vVxwJXJ462BeUAtMAz4O7l3lFsBzwCHAZXAW8BAQMA0YHpy/NXAWcnj7YHFwLbAWKAJ2HETMTwFNLRxO6qNfe8G7k3OPRJ4HxiSxPg8UJfst2Ny3wOYDQxNns8G6oGdgSeBbZPt/w58r9j/Xr51z60E82IscFPyeJO/20AjcF7y+HrgJaAP0Bd4N9neE9iuVVuvA0qer0rujwFuT651K2A68KVi/7t2x81DQAUk6WZyCfVxRHye3C/aUEmnJLt8llySfQz8MSKakuMagBpgFfBmRLyWbP8luWQmaetESROS55VA/+TxjIh4r62YIqKjY+a/iYiQtAB4JyIWJLEsTGJsAE6TNI5cslUBg8glY7MvJNueTt4Af4bcfzZWhkokL5q197v9SHK/AOgdER8AH0haI2l74B/A1ZK+BHwK9AN2BZa1auOY5PZC8rw3uZ/Pk52MeYvhApWuhcDJzU8i4nxJO5N7Rwi5d0AXRsRjrQ+SNAz4qNWmT/jnv82mJksUcHJE/GmDtg4m90vf9kHSU+TexW1oQkQ80cb25rg+3SDGT4GekmqBCcDnI+KvydBfZRuxzoiIMzcVl5W0UsyL1ufb3O/2ZvMHGE2uR3VQRKyV1Ejb+fPDiPjJZuIoSf4MKl2/ByolnddqW69Wjx8DzpNUASBpb0nbbqa9V4FaSXsmz1snwWPAha3G5A/IJ8CIODwi6tq4bS4JN2c7con/d0m7Al9pY5+5wKGS9kpi7SVp706ez7Y8pZwXXf3d/iy54b61ko4ABrSxz2PA/2z12VY/Sbt04BxbLBeoFEVuwHgU8GVJb0r6IzCZ3Lg0wB3AK8B8SS8DP2EzvdiIWENu6OK3yYfBf2718lVABfBS0tZVKV9OXiLiRXJDDwuBO4Gn29hnOblx+6mSXiKX1Pt2Y5hWRKWcFyn8bt8D1EuaR6439Wob53gcmAI8kwy130/bvb2S0/xhnJmZWaa4B2VmZpnkAmVmZpnkAmVmZpnkAmVmZpnUrQVqxIgRQe7vF3zzrRxuneI88a0Mb23q1gK1YsWK7jyd2RbJeWKW4yE+MzPLJBcoMzPLJBcoMzPLJE8Wa2Ylb+3atTQ1NbFmzZpih1LWKisrqa6upqKiIq/9XaDMrOQ1NTXRp08fampqSOaRtW4WEaxcuZKmpiZqa2vzOsZDfGZW8tasWcNOO+3k4lREkthpp5061It1gbKyMqCqCkmp3AZUVRX7cqwDXJyKr6P/Bh7is7KyZNkymnavTqWt6rebUmnHzNrmHpSZlZ00e9L59qZ79OhBXV0d+++/P6eeeioffvghAOvWrWPnnXfm8ssvX2//YcOGMW9ebtHhmpoahgwZwpAhQxg0aBBXXnklH330zwV6Fy5cyPDhw9l7770ZOHAgV111Fc1LKd1999307duXuro66urq+NrXvgbA2LFjqa2tbdn+4x//OJWfbZry6kFJ+t/A18lNSbEAOIfcipj3ATVAI3BaRPy1IFGamaUozZ405Neb3mabbWhoaABg9OjR3HbbbVxyySU8/vjj7LPPPkybNo2rr756k8Ngs2bNYuedd2bVqlWMGzeOcePGMXnyZFavXs2JJ57IrbfeyjHHHMOHH37IySefzC233ML5558PwOmnn85NN920UZuTJk3ilFNO6fyFF1i7PShJ/YCLgPqI2B/oAZwBXAbMjIiBwMzkuZmZtePwww/n9ddfB2Dq1KlcfPHF9O/fn7lz57Z7bO/evbntttt46KGHeO+995gyZQqHHnooxxxzDAC9evXipptuYuLEiQW9hu6Q7xBfT2AbST3J9ZzeBkaSW7aZ5H5U6tGZmZWYdevW8bvf/Y4hQ4awevVqZs6cyfHHH8+ZZ57J1KlT82pju+22o7a2ltdee42FCxdy0EEHrff6nnvuyapVq3j//fcBuO+++1qG8u66666W/b797W+3bF+wYEF6F5mSdgtURPwFuBZYAiwF/h4RjwO7RsTSZJ+lwC5tHS9pnKR5kuYtX748vcjNSojzpPStXr2auro66uvr6d+/P+eeey7Tp0/niCOOoFevXpx88sk8+OCDfPLJJ3m11/wZU0Rscliwefvpp59OQ0MDDQ0NnHPOOS2vT5o0qWX7kCFDuniF6Wv3MyhJO5DrLdUCfwN+JemsfE8QEbcDtwPU19dvclp1s3LmPCl9rT+DajZ16lSefvppampqAFi5ciWzZs3iqKOO2mxbH3zwAY2Njey9994MHjyYJ598cr3X33jjDXr37k2fPn3SvIRul88Q31HAmxGxPCLWAg8AXwTekVQFkNy/W7gwzcxKy/vvv8+cOXNYsmQJjY2NNDY2cvPNN7c7zLdq1Sq++c1vMmrUKHbYYQdGjx7NnDlzeOKJJ4BcT+2iiy7i0ksv7Y7LKKh8vsW3BPiCpF7AauBIYB7wD2AMMDG5f7hQQZqZpan/brul+nds/XfbrcPHPPDAAwwfPpytt966ZdvIkSO59NJL1/sKebMjjjiCiODTTz/lpJNO4rvf/S6Q65k9/PDDXHjhhZx//vl88sknnH322VxwwQWdv6CMUPM45mZ3kv4DOB1YB7xA7ivnvYFpQH9yRezUiHhvc+3U19dH8/f6zYpBUqp/qNtO/nRq6gLnSfoWLVrEfvvtV+wwjE3+W7SZK3n9HVREfB/4/gabPyLXmzIzM0udZ5IwM7NMcoEyM7NMcoEyM7NMcoEyM7NMcoEyM7NMcoEys7Kze3X/VJfb2L26f7vnXLZsGWeccQZ77rkngwYN4thjj2Xx4sXtLpXR1t8z1dTUsGLFivW2bbisRl1dHa+88goAixcv5thjj2WvvfZiv/3247TTTltvfr7evXuzzz77tCzHMXv2bI4//viWth966CGGDh3Kvvvuy5AhQ3jooYdaXhs7diz9+vVr+dutFStWtMyM0VVesNDMys7Sv7zFwd97NLX2nv3BiM2+HhGcdNJJjBkzhnvvvReAhoYG3nnnHcaOHbvZpTI6oq1lNdasWcNxxx3HddddxwknnADklu7o27dvy9RLw4YN49prr6W+vh6A2bNntxz/4osvMmHCBGbMmEFtbS1vvvkmRx99NHvssQdDhw4Fcmtd3XnnnZx33nkdjnlz3IMyMyuwWbNmUVFRwTe+8Y2WbXV1dSxevLjgS2VMmTKFQw45pKU4QW5Wiv333z+v46+99lquuOIKamtrAaitreXyyy9n0qRJLfuMHz+e66+/nnXr1qUWN7hAmZkV3Msvv7zRkhhAXktldETrYbu6ujpWr169yXPnq60Y6+vrWbhwYcvz/v37c9hhh/GLX/yi0+dpi4f4zMyKJJ+lMjpiUyvndkVbMba17YorruDEE0/kuOOOS+3c7kGZmRXY4MGDef7559vcvuG8i2kvlbGpc3fk+A1jnD9/PoMGDVpv21577UVdXR3Tpk3r9Lk25AJlZlZgw4cP56OPPuKnP/1py7bnnnuOgQMHFnypjK9+9av84Q9/4Le//W3LtkcffTTvFXQnTJjAD3/4QxobGwFobGzk6quv5lvf+tZG+37nO9/h2muvTSVu8BCfmZWhqn6fa/ebdx1tb3Mk8eCDDzJ+/HgmTpxIZWUlNTU13HDDDe0ulXH33Xev97XuuXPnAjB06FC22irXxzjttNMYOnQo9913H3PmzGnZ95ZbbuGLX/wi06dPZ/z48YwfP56KigqGDh3KjTfemNe11dXV8aMf/YgTTjiBtWvXUlFRwTXXXENdXd1G+w4ePJgDDzyQ+fPn59V2e/JabiMtXkbAis3LbZQnL7eRHR1ZbsNDfGZmlkmZK1ADqqpS++vuAVVVxb4cMzPrpMx9BrVk2bJUh2DMzGDzX+m27tHRj5Qy14MyM0tbZWUlK1eu7PB/kJaeiGDlypVUVlbmfUzmelBmZmmrrq6mqamJ5cuXFzuUslZZWUl1df4jZC5QZlbyKioqWuaSsy2Hh/jMzCyTXKDMzCyTXKDMzCyTXKDMzCyTXKDMzCyT8ipQkraXdL+kVyUtknSIpB0lzZD0WnK/Q6GDNTOz8pFvD+pG4NGI2Bf4F2ARcBkwMyIGAjOT52ZmZqlot0BJ2g74EvAzgIj4OCL+BowEJie7TQZGFSZEMzMrR/n0oPYAlgN3SXpB0h2StgV2jYilAMn9Lm0dLGmcpHmS5vmvuM3a5jwx21g+BaoncCBwa0QcAPyDDgznRcTtEVEfEfV9+/btZJhmpc15YraxfApUE9AUEc8mz+8nV7DekVQFkNy/W5gQzcysHLVboCJiGfCWpH2STUcCrwCPAGOSbWOAhwsSoZmZlaV8J4u9ELhH0meAN4BzyBW3aZLOBZYApxYmRLP0qEdFauuEqUdFKu2YWdvyKlAR0QDUt/HSkalGY1Zg8claDv7eo6m09ewPRqTSjpm1zTNJmJlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJuVdoCT1kPSCpOnJ8x0lzZD0WnK/Q+HCNDOzctORHtTFwKJWzy8DZkbEQGBm8tzMzCwVeRUoSdXAccAdrTaPBCYnjycDo1KNzMzMylq+PagbgEuBT1tt2zUilgIk97u0daCkcZLmSZq3fPnyrsRqVrKcJ2Yba7dASToeeDcinu/MCSLi9oioj4j6vn37dqYJs5LnPDHbWM889jkUOFHSsUAlsJ2kXwLvSKqKiKWSqoB3CxmomZmVl3Z7UBFxeURUR0QNcAbw+4g4C3gEGJPsNgZ4uGBRmplZ2enK30FNBI6W9BpwdPLczMwsFfkM8bWIiNnA7OTxSuDI9EMyMzPzTBJmZpZRLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBFMKCqCkmp3AZUVRX7cszMCqJD60FZOpYsW0bT7tWptFX9dlMq7ZiZZY17UGZmlkkuUGZmlkkuUGZmlkkuUGZmlkkuUGZmlkkuUGZmlkkuUGZmlkkuUGZmlkkuUGZmlkntFihJn5M0S9IiSQslXZxs31HSDEmvJfc7FD5cMzMrF/n0oNYB34qI/YAvAOdLGgRcBsyMiIHAzOS5mZlZKtotUBGxNCLmJ48/ABYB/YCRwORkt8nAqALFaGZmZahDn0FJqgEOAJ4Fdo2IpZArYsAumzhmnKR5kuYtX768i+GalSbnidnG8i5QknoDvwbGR8T7+R4XEbdHRH1E1Pft27czMZqVPOeJ2cbyKlCSKsgVp3si4oFk8zuSqpLXq4B3CxOimZmVo3y+xSfgZ8CiiLiu1UuPAGOSx2OAh9MPz8zMylU+CxYeCpwNLJDUkGy7ApgITJN0LrAEOLUgEZqZWVlqt0BFxBxAm3j5yHTDMTOzYhtQVcWSZctSaav/brvx56VLO3Wsl3w3M7P1LFm2jKbdq1Npq/rtpk4f66mOLPMGVFUhKZVbqUjzZzKgqqrYl2PWJvegLPOy8m4uS/wzsXLgHpSZmWVSSfegtobUhnW68kGfdY16VPhdvlkZKukC9RF4GKQExCdrOfh7j6bS1rM/GJFKO2ZWeB7iMzOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTCrpqY7MzKzj0pz/Uj0qOn2sC5SZma0nK/NfeojPrMw1z/rvxQ8ta9yDMitznvXfsso9KDMzyyQXKCuI3av7pzZsZGblyUN8VhBL//JWJj5kNbMtV+YKVFa+3mhmxTWgqooly5al0lb/3Xbjz0uXptKWdZ/MFaisfL1xS9H8Daw0OIktS5YsW+Yvb5S5LhUoSSOAG4EewB0RMTGVqCxv/gaWmZWqTn9JQlIP4GbgK8Ag4ExJg9IKzMwsLVn9W68BVVWpxdWrR8+S+2JSV3pQ/wN4PSLeAJB0LzASeCWNwMzM0pLVkYa0hzGzeI1doYjo3IHSKcCIiPh68vxs4OCIuGCD/cYB45Kn+wB/aqfpnYEVnQpqy+FrLA3tXeOKiMjrg1DnSZt8jaUhn2tsM1e60oNqqx+4UbWLiNuB2/NuVJoXEfVdiCvzfI2lIc1rdJ5szNdYGrpyjV35Q90m4HOtnlcDb3ehPTMzsxZdKVDPAQMl1Ur6DHAG8Eg6YZmZWbnr9BBfRKyTdAHwGLmvmd8ZEQtTiCnvYY4tmK+xNBTzGv3zLQ2+xs3o9JckzMzMCsmTxZqZWSa5QJmZWSZlpkBJGiHpT5Jel3RZseNJm6TPSZolaZGkhZIuLnZMhSKph6QXJE0vdiyFIGl7SfdLejX59zykG89d0nkC5ZMrpZ4n0PVcycRnUMm0SYuBo8l9ff054MyIKJlZKSRVAVURMV9SH+B5YFQpXWMzSZcA9cB2EXF8seNJm6TJwFMRcUfyDdZeEfG3bjhvyecJlE+ulHqeQNdzJSs9qJZpkyLiY6B52qSSERFLI2J+8vgDYBHQr7hRpU9SNXAccEexYykESdsBXwJ+BhARH3dHcUqUfJ5AeeRKqecJpJMrWSlQ/YC3Wj1vosR+IVuTVAMcADxb5FAK4QbgUuDTIsdRKHsAy4G7kuGZOyRt203nLqs8gZLOlRso7TyBFHIlKwUqr2mTSoGk3sCvgfER8X6x40mTpOOBdyPi+WLHUkA9gQOBWyPiAOAfQHd9FlQ2eQKlmytlkieQQq5kpUCVxbRJkirIJdw9EfFAseMpgEOBEyU1kht+Gi7pl8UNKXVNQFNENL+jv59cEnbXuUs+T6Dkc6Uc8gRSyJWsFKiSnzZJuUVWfgYsiojrih1PIUTE5RFRHRE15P4Nfx8RZxU5rFRFxDLgLUn7JJuOpPuWmCn5PIHSz5VyyBNIJ1cyseR7AadNypJDgbOBBZIakm1XRMT/K15I1kkXAvckReIN4JzuOGmZ5Ak4V0pJl3IlE18zNzMz21BWhvjMzMzW4wJlZmaZ5AJlZmaZ5AJlZmaZ5AJlZmaZ5AKVIZL+j6QJKba3r6SGZJqRPdNqt1X7jZJ2Trtds81xnpQPF6jSNgp4OCIOiIj/LnYwZhk1CudJJrlAFZmk7yTr+zwB7JNs+zdJz0l6UdKvJfWS1EfSm8kUMEjaLnlnViGpTtJcSS9JelDSDpKOBcYDX0/W1rlF0onJsQ9KujN5fK6k/5s8PkvSH5N3kz9JlndA0jGSnpE0X9KvkjnSWl/DNpIelfRv3fVzs/LiPClPLlBFJOkgclOdHAD8K/D55KUHIuLzEfEv5JYaODdZdmA2uSn6SY77dUSsBX4O/HtEDAUWAN9P/ur+NuD6iDgCeBI4PDm2HzAoeXwY8JSk/YDTgUMjog74BBidDE1cCRwVEQcC84BLWl1Gb+A3wJSI+Gk6Pxmzf3KelC8XqOI6HHgwIj5MZmtunldtf0lPSVoAjAYGJ9vv4J9ThZxDbhr7zwLbR8R/Jdsnk1uDZUNPAYdLGkRuPqx3lFsY7hDgD+TmyToIeC6ZXuZIctPlf4Fckj6dbB8DDGjV7sPAXRHx887/GMw2y3lSpjIxF1+Za2uuqbvJrSD6oqSxwDCAiHhaUo2kLwM9IuLlJPHaP0nEXyTtAIwg9y5xR+A0YFVEfCBJwOSIuLz1cZJOAGZExJmbaPpp4CuSpoTnzbLCcZ6UIfegiutJ4KRkbLoPcEKyvQ+wNBlHH73BMT8HpgJ3AUTE34G/Smoeljgb+C/a9gy58fYnyb1TnJDcA8wETpG0C4CkHSUNAOYCh0raK9neS9Lerdr8HrASuKWD126WL+dJmXKBKqJkWev7gAZya980J8F3ya0gOgN4dYPD7gF2IJd8zcYAkyS9BNQBP9jEKZ8CekbE68B8cu8On0pieYXcGPrjSTszgKqIWA6MBaYm2+cC+27Q7nigUtI1+V25Wf6cJ+XLs5lvYSSdAoyMiLOLHYtZVjlPSoM/g9qCSPpP4CvAscWOxSyrnCelwz0oMzPLJH8GZWZmmeQCZWZmmeQCZWZmmeQCZWZmmeQCZWZmmfT/AcKH/fljK6RSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['dayofweek'] = df['effective_date'].dt.dayofweek\n",
    "bins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\n",
    "g.axes[-1].legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  loan_status  Principal  terms effective_date   due_date  age  \\\n",
       "0     PAIDOFF       1000     30     2016-09-08 2016-10-07   45   \n",
       "1     PAIDOFF       1000     30     2016-09-08 2016-10-07   33   \n",
       "2     PAIDOFF       1000     15     2016-09-08 2016-09-22   27   \n",
       "3     PAIDOFF       1000     30     2016-09-09 2016-10-08   28   \n",
       "4     PAIDOFF       1000     30     2016-09-09 2016-10-08   29   \n",
       "\n",
       "              education  Gender  dayofweek  weekend  \n",
       "0  High School or Below    male          3        0  \n",
       "1              Bechalor  female          3        0  \n",
       "2               college    male          3        0  \n",
       "3               college  female          4        1  \n",
       "4               college    male          4        1  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets look at gender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender  loan_status\n",
       "female  PAIDOFF        0.865385\n",
       "        COLLECTION     0.134615\n",
       "male    PAIDOFF        0.731293\n",
       "        COLLECTION     0.268707\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "86 % of female pay there loans while only 73 % of males pay there loan\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets convert male to 0 and female to 1:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  loan_status  Principal  terms effective_date   due_date  age  \\\n",
       "0     PAIDOFF       1000     30     2016-09-08 2016-10-07   45   \n",
       "1     PAIDOFF       1000     30     2016-09-08 2016-10-07   33   \n",
       "2     PAIDOFF       1000     15     2016-09-08 2016-09-22   27   \n",
       "3     PAIDOFF       1000     30     2016-09-09 2016-10-08   28   \n",
       "4     PAIDOFF       1000     30     2016-09-09 2016-10-08   29   \n",
       "\n",
       "              education  Gender  dayofweek  weekend  \n",
       "0  High School or Below       0          3        0  \n",
       "1              Bechalor       1          3        0  \n",
       "2               college       0          3        0  \n",
       "3               college       1          4        1  \n",
       "4               college       0          4        1  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## One Hot Encoding  \n",
    "#### How about education?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "education             loan_status\n",
       "Bechalor              PAIDOFF        0.750000\n",
       "                      COLLECTION     0.250000\n",
       "High School or Below  PAIDOFF        0.741722\n",
       "                      COLLECTION     0.258278\n",
       "Master or Above       COLLECTION     0.500000\n",
       "                      PAIDOFF        0.500000\n",
       "college               PAIDOFF        0.765101\n",
       "                      COLLECTION     0.234899\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Feature before One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>education</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>High School or Below</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>Bechalor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender             education\n",
       "0       1000     30   45       0  High School or Below\n",
       "1       1000     30   33       1              Bechalor\n",
       "2       1000     15   27       0               college\n",
       "3       1000     30   28       1               college\n",
       "4       1000     30   29       0               college"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['Principal','terms','age','Gender','education']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature = df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Feature selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets defind feature sets, X:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = Feature\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "What are our lables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['loan_status'].values\n",
    "y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Normalize Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n",
       "        -0.38170062,  1.13639374, -0.86968108],\n",
       "       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n",
       "         2.61985426, -0.87997669, -0.86968108],\n",
       "       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X= preprocessing.StandardScaler().fit(X).transform(X)\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Classification "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\n",
    "You should use the following algorithm:\n",
    "- K Nearest Neighbor(KNN)\n",
    "- Decision Tree\n",
    "- Support Vector Machine\n",
    "- Logistic Regression\n",
    "\n",
    "\n",
    "\n",
    "__ Notice:__ \n",
    "- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n",
    "- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n",
    "- You should include the code of the algorithm in the following cells."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# K Nearest Neighbor(KNN)\n",
    "Notice: You should find the best k to build the model with the best accuracy.  \n",
    "**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    " X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=101)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "error_rate = []\n",
    "for i in range(1,40):\n",
    "    knn = KNeighborsClassifier(n_neighbors=i)\n",
    "    knn.fit(X_train,y_train)\n",
    "    knn_pred = knn.predict(X_test)\n",
    "    error_rate.append(np.mean(knn_pred != y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Error Rate')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAGDCAYAAABuj7cYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABMpUlEQVR4nO3deXxU1f3/8dcnIUQIBG2CtIKgoK0Vihu1oHVfKtai1dYF97WIWKW2Lt1+drEbbqUqVKN1r1vdK9XSRb8tYMUNVNQmKIgruEIIQyCf3x93UoYwGSbJ3GUy7+fjMY/M3HvuPZ+5czP55Nx7zjF3R0RERESSoSzuAERERERkHSVnIiIiIgmi5ExEREQkQZSciYiIiCSIkjMRERGRBFFyJiIiIpIgSs5ERLoxM7vRzH4edxwikj8lZyKSk5m9bmZNZrYi43FVxDH808xWpeteZmb3mtln8tx2bzNbEnaMHWFmW5mZm1mP9Gszs9+Z2ctmNrBN2WPSn4G1Wd7DzN4zs0OijF1EwqfkTETy8TV375PxmJStUGuy0WZZeUcqylF+krv3AbYB+gCXdmS/SZVOun4P7A3s5e5vtilyH7ApsFeb5QcBDvwl5BBFJGJKzkSk08zsJDP7t5ldYWYfABenL6NNM7NHzKwR2MfMPp9u/frIzF40s3EZ+9igfK463f0j4H5gx4x9nGxmC8xsuZktNLNvpZdXATOALTJa/bYwszIzu9DMGszsfTO7y8w+1c57XJDZOpVusVpmZjub2SZmdmt6Hx+Z2VNmNqADh7AcuBEYBezt7u9meb+rgLuAE9qsOgG4zd3XmNndZvaOmX1sZk+Y2fB23stJZvavNsvczLZJP680s0vNbLGZvWtm082sVwfej4gUgJIzEemqLwELgc2BS9LLxqef9wWeBB4CHkuXORu4zcw+l7GPzPLrJQ9tmVkNcDhQn7H4PeAQoBo4GbjCzHZ290ZgLPBWRqvfW8C3gcMIWqO2AD4Erm6nyj8Cx2S8/gqwzN2fAU4E+gFbAjXABKApV/xt3AZsB+zr7u/nKHcT8I3WRMnM+gFfA25Or58BbEtwfJ9J77czfg18liDx3QYYCPy4k/sSkU5SciYi+bg/3TLU+jg9Y91b7v47d1/j7q2JyQPu/m93byH4Q98H+JW7r3b3vwMPs37C87/y6ZaibKaa2cfAMqCWIMkDwN3/7O4NHnicIBHcI8f7+RbwA3df4u4p4GKC5GeDy7LA7cA4M+udfj0+vQygmSAp28bd17r70+7+SY562zoQuCvdGtgud/838C7w9fSiI4FX3f259Pob3H15xnvZIZ3A5S19efV0YLK7f+Duy4FfAEd3ZD8i0nVKzkQkH4e5+6YZj+sy1r2RpXzmsi2AN9KJWqtFBK0yufbR1rfdvR8wEtgMGNS6wszGmtkcM/vAzD4CDiZI4NozBLivNdkEFgBrgQ0uSbp7fXr919IJ2jjWJWe3AI8Cd5jZW2b2GzOryOO9tDoE+H9mdkoeZW9m3aXN4wla0zCzcjP7VfoS7SfA6+kyud5/Nv2B3sDTGcflL+nlIhIhJWci0lW+kWVvAVuaWeb3zWDgzXbK567MfT7wc+DqdC/HSuBPBB0EBrj7psAjQGvvxmz7fgMY2ybh3CTLzfitWi9tHgq8lE7YcPdmd/+Ju28P7EaQbLW9NyyXWQSXJ39rZuM3UvZmYD8zGwOMZl2COD4d1/4El1i3Si+3tjsAGgkSsKCA2acz1i0juCQ7POOY9Et3whCRCCk5E5GwPUmQFJxvZhVmtjdBQnJHF/Z5E8H9VeOAnkAlsBRYY2ZjCS4XtnoXqGlzmW86cImZDQEws/5mdmiO+u5I7/NM1iVFmNk+ZvaFdA/TTwguc67tyBtJX4Y9HLjWzL6Ro9wigvvx/gj81d3fSa/qC6SA9wkSr1/kqO55YLiZ7WhmmxBcAm3dfwtwHcH9epun399AM/tKR96PiHSdkjMRycdDtv44Z/flu6G7ryZIosYStM5cA5zg7i93Npj0PqcCP0rfG/Vtgh6NHxK0JD2YUfZlgoRmYfpy3RbAb9NlHjOz5cAcgo4N7dX3NjCboHXszoxVnwbuIUjMFgCPA7cCpHs6Ts/z/fwVOAq40cy+lqPoTQSXZG/OWHYzwWXiN4GX0u+lvXpeBX4KzAT+y4adLy4g6GgxJ32JdCbwOUQkUuae99UEEREREQmZWs5EREREEkTJmYiIiEiCKDkTERERSRAlZyIiIiIJouRMREREJEGyTVVStGpra32rrbaKOwwRERGRjXr66aeXufsGs3B0q+Rsq622Yu7cuXGHISIiIrJRZrYo23Jd1hQRERFJECVnIiIiIgmi5ExEREQkQZSciYiIiCSIkjMRERGRBFFyJiIiIpIgSs5EREREEkTJWR4aGmDyxBQDqpsoL2thQHUTkyemaGiIOzIRERHpbpScbcSMGTB6ZCO96qYya/kIUt6TWctH0KtuKqNHNjJjRtwRioiISHdi7h53DAUzatQoL+QMAQ0NQWL24Mr9GcOcDdbPZjTjes9kzrwqhg0rWLUiIiJSAszsaXcf1Xa5Ws5yuOqyFKc3X5M1MQMYwxxOa57G1VekIo5MREREuislZzncfmsLpzZPz1nmtOZp3H7L2ogiEhERke5OyVkOy1ZUMoSsc5L+z2AWs2zFJhFFJCIiIt2dkrMcavukWMSQnGUWM5jaPqsiikhERES6OyVnOYw/rozrKybkLFNXcSbjjy+PKCIRERHp7pSc5TDpvEquq5jIbEZnXT+b0dRVnMlZkysjjkxERES6KyVnOQwbBjffU8W43jO5qGIKDQylmR40MJQLe0xhXO+Z3HyPhtEQERGRwlFythFjx8KceVWkzjib3avn08tS7Fg2nyVfP5s586oYOzbuCEVERKQ70SC0IiIiIjHQILQFtnx58BAREREppFCTMzM7yMxeMbN6M7swy/pDzWyemT1nZnPN7MsZ6zY1s3vM7GUzW2BmY8KMtSPefRc23xzq6uKORERERLqb0JIzMysHrgbGAtsDx5jZ9m2K/Q3Ywd13BE4BMtOd3wJ/cfftgB2ABWHF2lEDBsB228Gdd8YdiYiIiHQ3Ybac7QrUu/tCd18N3AEcmlnA3Vf4upveqgAHMLNqYE/g+nS51e7+UYixdthRR8GTT8Lrr8cdiYiIiHQnYSZnA4E3Ml4vSS9bj5l93cxeBv5M0HoGMBRYCvzBzJ41szozqwox1g478sjg5913xxuHiIiIdC9hJmeWZdkGXUPd/b70pcvDgJ+lF/cAdgamuftOQCOwwT1rAGZ2Rvp+tblLly4tSOD5GDoURo2Cu+6KrEoREREpAT1C3PcSYMuM14OAt9or7O5PmNkwM6tNb7vE3Z9Mr76HdpIzd78WuBaCoTQKEXi+rrgC+vWLskYRERHp7sJMzp4CtjWzrYE3gaOB8ZkFzGwboMHd3cx2BnoC76dfv2Fmn3P3V4D9gJdCjLVTvvzljZcRERER6YjQkjN3X2Nmk4BHgXLgBnd/0cwmpNdPB44ATjCzZqAJOCqjg8DZwG1m1hNYCJwcVqxd8e9/wyOPwCWXxB2JiIiIdAeaIaCLrrgCvvMdePVV2HbbSKsWERGRIqYZAkLyzW8GP9UxQERERApByVkXDRoEu++uAWlFRESkMJScFcCRR8L8+bAgMXMYiIiISLFSclYA3/gGDBkCb7yx8bIiIiIiuYQ5lEbJ2GILeO01sGzD7oqIiIh0gFrOCsQM1q6Fxsa4IxEREZFipuSsQJqaYPBg+PWv445EREREipmSswLp1Qu22y4YUqMbDR0nIiIiEVNyVkBHHgmvvBL03BQRERHpDCVnBXT44VBerjHPREREpPOUnBVQ//6w7766tCkiIiKdp6E0CuxHP4JUKu4oREREpFgpOSuwPfaIOwIREREpZrqsGYKXX4af/1yXNkVERKTjlJyFYM6c4PLmU0/FHYmIiIgUGyVnITj0UKioCDoGiIiIiHSEkrMQbLYZHHigem2KiIhIxyk5C8lRR8EbbwSXOEVERETypeQsJF/4AlRXpvjqvk2Ul7UwoLqJyRNTNDTEHZmIiIgkmZKzEMyYAQfs3sjElqk8tWoEKe/JrOUj6FU3ldEjG5kxI+4IRUREJKnMu9FNUaNGjfK5c+fGGkNDA4we2ciDK/dnDBte05zNaMb1nsmceVUMGxZDgCIiIpIIZva0u49qu1wtZwV21WUpTm++JmtiBjCGOZzWPI2rr9A0AiIiIrIhJWcFdvutLZzaPD1nmdOap3H7LWsjikhERESKiZKzAlu2opIhLMpZZjCLWbZik4giEhERkWKi5KzAavukWMSQnGUWM5jaPqsiikhERESKiZKzAht/XBnXV0zIWaau4kzGH18eUUQiIiJSTJScFdik8yq5rmIisxmddf1sRlNXcSZnTa6MODIREREpBkrOCmzYMLj5nirG9Z7JRRVTaGAozfSggaFcVDGFcb1ncvM9GkZDREREslNyFoKxY2HOvCpSZ5zN7tXz6WUpdq+eT+qMs5kzr4qxY+OOUERERJJKyVlIhg2Dy6+q5O2PelPWo4yTJ/bm8qsq1WImIiIiOSk5C5kZ1NTAsmVxRyIiIiLFQMlZBGpq4P33445CREREikGoyZmZHWRmr5hZvZldmGX9oWY2z8yeM7O5ZvblNuvLzexZM3s4zDjDpuRMRERE8hVacmZm5cDVwFhge+AYM9u+TbG/ATu4+47AKUBdm/XnAAvCijEqSs5EREQkX2G2nO0K1Lv7QndfDdwBHJpZwN1XuLunX1YBrc8xs0HAV9kwYSs648fDxIlxRyEiIiLFoEeI+x4IvJHxegnwpbaFzOzrwC+BzQmSsVZXAucDfcMLMRrf+EbcEYiIiEixCLPlzLIs8w0WuN/n7tsBhwE/AzCzQ4D33P3pjVZidkb6frW5S5cu7WLI4WhqgoULYc2auCMRERGRpAszOVsCbJnxehDwVnuF3f0JYJiZ1QK7A+PM7HWCy6H7mtmt7Wx3rbuPcvdR/fv3L1jwhXTHHcG4Z2+8sfGyIiIiUtrCTM6eArY1s63NrCdwNPBgZgEz28bMLP18Z6An8L67X+Tug9x9q/R2f3f340KMNVQ1NcFPdQoQERGRjQntnjN3X2Nmk4BHgXLgBnd/0cwmpNdPB44ATjCzZqAJOCqjg0C3oeRMRERE8hVmhwDc/RHgkTbLpmc8/zXw643s45/AP0MILzJKzkRERCRfmiEgAkrOREREJF9KziLwqU/BlVfCnnvGHYmIiIgkXaiXNSVQXg7nnBN3FCIiIlIM1HIWkYYGeOWVuKMQERGRpFPLWUSOPRaqq+Gxx+KORERERJJMLWcRqa1VhwARERHZOCVnEampgWXL4o5CREREkk7JWURqatRyJiIiIhun5CwitbXQ2AipVNyRiIiISJKpQ0BEDjsMtt0WgplERURERLJTchaR7bcPHiIiIiK56LJmRJYvh7/9Dd57L+5IREREJMmUnEXk9ddh//3hiSfijkRERESSTMlZRDT5uYiIiORDyVlElJyJiIhIPpScRaSyEqqqNBCtiIiI5KbkLEIaiFZEREQ2RkNpROjmm6F//7ijEBERkSRTchahvfaKOwIRERFJOl3WjNCzz8J998UdhYiIiCSZkrMIXX89nHpq3FGIiIhIkik5i1BNDXz0EaxdG3ckIiIiklRKziJUUwPu8OGHcUciIiIiSaXkLEIaiFZEREQ2RslZhGprg58aiFZERETao6E0IjR6NDz1FHz+83FHIiIiIkml5CxC/frBqFFxRyEiIiJJpsuaEVq7Furq4D//iTsSERERSSolZxEqK4Mzz9RAtCIiItI+JWcRMtPk5yIiIpKbkrOI1daqt6aIiIi0T8lZxNRyJiIiIrmEmpyZ2UFm9oqZ1ZvZhVnWH2pm88zsOTOba2ZfTi/f0sz+YWYLzOxFMzsnzDijpORMREREcgltKA0zKweuBg4AlgBPmdmD7v5SRrG/AQ+6u5vZSOAuYDtgDXCeuz9jZn2Bp83sr222LUpXXRV0DBARERHJJsw0YVeg3t0Xuvtq4A7g0MwC7r7C3T39sgrw9PK33f2Z9PPlwAJgYIixRmaLLeDTn447ChEREUmqMJOzgcAbGa+XkCXBMrOvm9nLwJ+BU7Ks3wrYCXgynDCj9eyz8JOfwMqVcUciIiIiSRRmcmZZlvkGC9zvc/ftgMOAn623A7M+wJ+Ac939k6yVmJ2Rvl9t7tKlS7sedcieew4uvhjefTfuSERERCSJwkzOlgBbZrweBLzVXmF3fwIYZma1AGZWQZCY3ebu9+bY7lp3H+Xuo/r371+YyENUUxP8VKcAERERySbM5OwpYFsz29rMegJHAw9mFjCzbczM0s93BnoC76eXXQ8scPfLQ4wxcrW1wU+NdSYiIiLZhNZb093XmNkk4FGgHLjB3V80swnp9dOBI4ATzKwZaAKOSvfc/DJwPDDfzJ5L7/L77v5IWPFGRS1nIiIikktoyRlAOpl6pM2y6RnPfw38Ost2/yL7PWtFT8mZiIiI5BJqciYbqqkJLmluumnckYiIiEgSKTmLWOvk5yIiIiLZaKz6GFx5JdTVxR2FiIiIJJGSsxjceWfwEBEREWlLyVkMNPm5iIiItEfJWQyUnImIiEh7lJzFoLZWg9CKiIhIdkrOYlBTA2vWQCoVdyQiIiKSNErOYnDBBbBqFVRWxh2JiIiIJI3GOYtBeXncEYiIiEhSqeUsBg0NcNJJ8PzzcUciIiIiSaPkLAaNjXDTTfDqq3FHIiIiIkmj5CwGtbXBTw2nISIiIm0pOYtB69yaSs5ERESkLSVnMaishKoqJWciIiKyISVnMdl6azCLOwoRERFJGg2lEZP58+OOQERERJJILWciIiIiCaLkLCa/+10w1pmIiIhIJiVnMXnlFXjwwbijEBERkaRRchaT2lr46CNYuzbuSERERCRJlJzFpKYG3OHDD+OORERERJJEyVlMNBCtiIiIZKPkLCYDB8J228Hq1XFHIiIiIkmicc5istdesGBB3FGIiIhI0qjlTERERCRBNpqcWeA4M/tx+vVgM9s1/NC6t1WrYM894cYb445EREREkiSflrNrgDHAMenXy4GrQ4uoRFRWwpw5wXhnIiIiIq3yuefsS+6+s5k9C+DuH5pZz5Dj6vbMgh6by5bFHYmIiIgkST4tZ81mVg44gJn1B1pCjapE1NRoKA0RERFZXz7J2VTgPmBzM7sE+Bfwy1CjKhG1tUrOREREZH0bvazp7reZ2dPAfoABh7m7BoEogFGjNEOAiIiIrC+f3pq3uPvL7n61u1/l7gvM7JZ8dm5mB5nZK2ZWb2YXZll/qJnNM7PnzGyumX053227g0svheuvjzsKERERSZJ8LmsOz3yRvv9sl41tlC53NTAW2B44xsy2b1Psb8AO7r4jcApQ14FtRURERLqddpMzM7vIzJYDI83sEzNbnn79HvBAHvveFah394Xuvhq4Azg0s4C7r3B3T7+sIt3pIJ9tu4PbboNttoFPPok7EhEREUmKdpMzd/+lu/cFprh7tbv3TT9q3P2iPPY9EHgj4/WS9LL1mNnXzexl4M8ErWd5b1vsmpuhoUHDaYiIiMg6G72s6e4XmdlmZrarme3Z+shj35Ztd1n2f5+7bwccBvysI9sCmNkZ6fvV5i5dujSPsJKjpib4qR6bIiIi0mqjvTXN7DTgHGAQ8BwwGpgN7LuRTZcAW2a8HgS81V5hd3/CzIaZWW1HtnX3a4FrAUaNGpU1gUsqJWciIiLSVj4dAs4Bvggscvd9gJ2AfJqongK2NbOt0zMKHA08mFnAzLYxM0s/3xnoCbyfz7bdgZIzERERaSuf6ZtWufsqM8PMKt39ZTP73MY2cvc1ZjYJeBQoB25w9xfNbEJ6/XTgCOAEM2sGmoCj0h0Esm7bubeYXJtvDgcfDP37xx2JiIiIJIWt6yzZTgGz+4CTgXMJLmV+CFS4+8GhR9dBo0aN8rlz58YdhoiIiMhGmdnT7j6q7fJ8Zgj4evrpxWb2D6AfMKPA8YmIiIgI+d1z9j/u/jiwCngknHBKz157wUknxR2FiIiIJEWuQWj3NbNXzWyFmd1qZtub2VyCSc+nRRdi97Z6Nbz5ZtxRiIiISFLkajm7DDgDqAHuAeYAt7j7Lu5+bxTBlYKaGvXWFBERkXVyJWfu7v9095S73w8sdfffRhRXyVByJiIiIplydQjY1MwOz3htma/VelYYNTWavklERETWyZWcPQ58rZ3XDig5K4Ddd4fly2HtWigvjzsaERERidtGxzkrJhrnTERERIpFe+OcdWgoDQmHO7S0xB2FiIiIJIGSs5j9619QUQGPPx53JCIiIpIEOZMzMyszs92iCqYU9esX3G+mTgEiIiICG0nO3L2FYLwzCUlNTfBTw2mIiIgI5HdZ8zEzO8LMLPRoSpCSMxEREcm00YnPge8AVcBaM2sCjGCA2upQIysRlZXQp48ua4qIiEhgoy1n7t7X3cvcvcLdq9OvlZgV0Lnnwh57xB1F6WpogMkTUwyobqK8rIUB1U1MnpiioSHuyEREpBTl1VvTzMaZ2aXpxyFhB1VqfvYzOPzwjZeTwpsxA0aPbKRX3VRmLR9Bynsya/kIetVNZfTIRmbMiDtCEREpNRsdhNbMfgV8EbgtvegY4Gl3vzDk2DqsWAehbW4OZgn41KfijqS0NDQEidmDK/dnDHM2WD+b0YzrPZM586oYNiyGAEVEpFvryiC0BwMHuPsN7n4DcFB6mRTIiSfCrrvGHUXpueqyFKc3X5M1MQMYwxxOa57G1VekIo5MRERKWb6D0G6a8bxfCHGUtJoa9daMw+23tnBq8/ScZU5rnsbtt6yNKCIREZH8emv+AnjWzP5B0FNzT+CiUKMqMTU18NFHsGYN9MjnE5GCWLaikiEsyllmMItZtmKTiCISERHJY4YAoAUYDdybfoxx9zsiiK1ktI519uGH8cZRamr7pFjEkJxlFjOY2j6rIopIREQkvxkCJrn72+7+oLs/4O7vRBRbydBAtPEYf1wZdRUTcpapqziT8ceXRxSRiIhIfvec/dXMvmtmW5rZp1ofoUdWQnbeGS65BDbdNO5ISsuk8yqZxkRmMzrr+tmMpq7iTM6aXBlxZCIiUsryucPplPTPszKWOTC08OGUpu22g+9/P+4oSk8qBY1exVfKZzKxbBqnN09jMItZzGDqKs6kruJMbr5Hw2iIiEi08rnn7EJ337rNQ4lZAbW0wKJF8MEHcUdSWhYuhCFD4K//rmL1GWeze/V8NiHFqMr5pM44mznzqhg7Nu4oRUSk1OQzCO0T7r5nRPF0SbEOQrt8OVRXw29+A9/7XtzRlBb1kBURkbh0ZRBa3XMWsj59oKJCHQKi8vzzMH06uGdPzFpaglkbRERE4pBPcnYKwf1mTwBPpx/F1zyVYGYaiDYqq1fDCSfAxRcHY8u19d//Qt++8Kc/RR2ZiIhIYKMXdNx96ygCKXW1tUrOovCzn8G8efDAA7DZZhuuHzw46Cjw4ovRxyYiIgI5Ws7M7PyM599ss+4XYQZVimpqYNmyuKPo3p56Cn75y6DlbNy47GUqK2GbbZSciYhIfHJd1jw643nb6ZoOCiGWkvad78B558UdRfe1di2cfDJ8+tPw29/mLjtiBLzwQjRxiYiItJXrsqa18zzba+mi9lpypDDKy+GKK6CsbOOD/Q4fDvfdB6tWwSaaVlNERCKWKznzdp5ney1dtHQp1NfD6NFBBwEpnFQquFx5wAH5lf/KV4LPYPVqJWciIhK9XJc1dzCzT8xsOTAy/bz19Rfy2bmZHWRmr5hZvZldmGX9sWY2L/2YZWY7ZKybbGYvmtkLZvZHM+vWfyZvvhl22y0Y80w6r6EBJk9MMaC6ifKyFgZUNzHk0yl+9KP897HbbkFvzurq0MIUERFpV7vJmbuXu3u1u/d19x7p562vKza2YzMrB64GxgLbA8eY2fZtir0G7OXuI4GfAdemtx0IfBsY5e4jgHLWvweu22md/FydAjpvxgwYPbKRXnVTmbV8BCnvyazlIzjuo6lcfWkjM2bkv6/33w9mbRAREYlaPuOcddauQL27L3T31cAdwKGZBdx9lrt/mH45BxiUsboH0MvMegC9gbdCjDV2rcmZhtPonIYGOOEbjTy4cn9+0Xw+w1hID9YyjIVcyvn8edX+nPCNRhoa8tvfXnvB2WeHG7OIiEg2YSZnA4E3Ml4vSS9rz6nADAB3fxO4FFgMvA187O6PZdvIzM4ws7lmNnfp0qUFCTwOtbXBTyVnnXPVZSlOb76GMczJun4McziteRpXX5HKa3/Dh2s4DRERiUeYyVm229qzdiQws30IkrML0q83I2hl2xrYAqgys+Oybevu17r7KHcf1b9//4IEHge1nHXN7be2cGrz9JxlTmuexu23rM1rf8OHBxOjNzYWIjoREZH8hZmcLQG2zHg9iCyXJs1sJFAHHOruranJ/sBr7r7U3ZuBe4HdQow1doMGwe23w5e/HHckxWnZikqGkPsmscEsZtmK/PqVjBgR/FywoKuRiYiIdEyYydlTwLZmtrWZ9SS4of/BzAJmNpgg8Tre3V/NWLUYGG1mvc3MgP2Abv1nsndvOOYYGDIk7kiKU22fFIvIffAWM5jaPqvy2t/w4cFPXdoUEZGohZacufsaYBLwKEFidZe7v2hmE8xsQrrYj4Ea4Boze87M5qa3fRK4B3gGmJ+O89qwYk2KWbPguefijqI4jT+ujOsrJuQsU1dxJuOPL89rf8OGwXXXwZ57FiI6ERGR/Jl79xlPdtSoUT537ty4w+i0z34Wdt4Z7rgj7kiKT0NDMIzGgyv3z9opYDajGdd7JnPmVTFsWAwBioiItGFmT7v7qLbLw7ysKR1UU6MOAZ01bBjcfE8V43rP5Hs2hQaG0kwPGhjKRRVTGNd7Jjff07HE7PXXg2mcREREoqTkLEFqazUIbVeMHQtz5lVx36Cz2bFsPr3KUuxePZ/UGWczZ14VY8d2bH+33w6HH65ZG0REJFq55taUiNXUwPPPxx1FcRs2DKyykoO/AXfeCcH4xZ3T2ingpZfgS18qSHgiIiIbpZazBNFlzcLYf//8JznPRT02RUQkDmo5S5BvfQsOPRTcwbIN4St5mTatMPvZemvYZBMlZyIiEi0lZwny2c8GD+m85mbo0aMwyW15OXz+80rOREQkWrqsmSDvvhvchF7EU4TG7g9/gKoqeGuDuSg658Yboa6uMPsSERHJh5KzBFmwAI49FubPjzuS4rVwIaxZAwMGFGZ/I0cGU2uJiIhERclZgmjy865raICttgouSRbC0qVw6aXw6qsbLysiIlIIuucsQVqTM4111nkLF1LQGQBWrIDvfQ/69dP9gCIiEg21nCWIWs66xj1oORs6tHD7HDIkuIfthRcKt08REZFclJwlSGUl9OmT7OSsoQEmT0wxoLqJ8rIWBlQ3MXliioaGuCODtWvh3HPhkEMKt8+yMth+e/XYLGZJPmdFRLJRcpYwTzwBF1wQdxTZzZgRTC7eq24qs5aPIOU9mbV8BL3qpjJ6ZCMzZsQbX48ecPHFdHiapo0ZPlzJWbFK+jkrIpKNuXvcMRTMqFGjfO7cuXGH0S01NAR/5B5cuT9jmLPB+tmMZlzvmcyZ17HJxQvpgw+gpSW4PFzIQXwvvRQuvDC4F3DTTQu3XwlXMZyzIlLazOxpdx/VdrlazhLm0Ufh1lvjjmJDV12W4vTma7L+kQMYwxxOa57G1VekIo5snenToX9/aGws7H4nTAj2qcSsuBTDOSsiko1azhJm/Hj4z3+gvj7uSNY3oLqJWctHMIyF7ZZpYCi7V8/nnY87P9l4V5x2Gjz8MLzzTizVS8IUwzkrIqVNLWdFIqmTny9bUckQFuUsM5jFLFuxSUQRbaihobDDaGT60Y/gmmvC2beEoxjOWRGRbJScJUxNDXz0UTDKfZLU9kmxiCE5yyxmMLV9VkUU0YYWLizsMBqZZs6Eu+8OZ98SjmI4Z0VEslFyljCtY519+GG8cbQ1/rgyrq+YkLNMXcWZjD++QEPzd1AqBW+8EV7LmXpsFp+kn7MiIu1RcpYwSR2IdtJ5lVxXMZHZjM66fjajqas4k7MmV0YcWaClBa69FsaNC2f/w4cHUzm99144+5fCS/o5KyLSHiVnCfO1r8HixbDNNnFHsr5hw+Dme6o4qMdMzmMKDQylmR40MJSLKqYwrvdMbr4nviEJevUKOgTsvHM4+x8xIvip1rPiMWwYHHdGFfszkwt6rH/Ofs+msD8zmXK1htEQkeRRcpYwffvCllsGA6omzYEHQo/qKmYMO5tde82nkhRj+swndcbZzJlXVfDBXzuioQGefTZoQQvD8OEwYEBwP6AUjyuugAdnVtH8rbPZvXo+vcpS7F49n4+OPxurquKmm8I7Z0REOiuBKUBpW7ECrrwSDjgAvvSluKNZ35o18Otfw2c/W0llJey9N9zxQG/23TfuyODqq+H3vw+OXxi22EJDdBST998PWqB32gn22w/226+Sy69qXRsMm7Hb3sGcqWvWQM+ecUUqIrIhJWcJ4x4M21BZmbzkrLIyuHQIQWtDY2Mw92QStE54XsiZAaR4nXUWPPRQkKC13sfZ1sknRxuTiEi+EvKnVVr16RP8F5+0DgFr1gQtU0uXBq/LypKTmEG4Y5y1uu462G23IIGW5Lr7brjzTrjoovYTs0z/+hccdxysXRt+bCIi+UjQn1eBoOWnpiaYxzFJ/vGPYBqjf/973bKf/jT4Axg393DHOGu1ahXMnq3Lm0n23nswcSKMGhXMh5qP11+H224L7k8TEUkCJWcJlMRZAu68M2jVO+igdcuefx7uvz+2kP7nnXegqSn8lrPhw4Of6rGZTO7BPxDLl8NNN+XfqebYY+Gww+CHP4SXXgo1RBGRvCg5S6Da2mQlZ83NcO+9cOihsEnGTDfDhsFrr8Xf223TTYMJ4w85JNx6lJwlmzuMHAm//CVsv33+25nB9OnBPx8nnZS82TlEpPSoQ0ACPfAA9E7QPMx/+1swY8FRR62/fOjQYGT+t96CQYPiiQ2CMc4OPDD8ejbfPGjVVHKWTGVlcPHFndt2wIBg7tSjjoJbblFnARGJl1rOEqi6OlnjnM2dG7ROtU2AWi8jNjREHtJ6Zs+Ghx8Ovx4zOOIIGJJ7ukaJmDucfjo89ljX9nPkkfDgg3DCCYWJS0Sks5ScJdDMmcFQAEnpFfjDHwY3TVe2meVmm21g222DG+XjdM01wfGKwu9/Dz/4QTR1SX5uvBHq6mDBgq7v62tfg/JyePddWL266/sTEekMJWcJ09AAl/w4xQ3XNNGjvIUB1U1MnpiKrXWqNUHs12/DdVtvDa++Cl/5SrQxtdU6xllU3Dt/n11DA0yemGJAdRPlZeF9vp2tpzPbxVnX5n2b+Pa3Uuy6K5x9du768vXOO8H9heedl+zPSkS6r1CTMzM7yMxeMbN6M9ugY7uZHWtm89KPWWa2Q8a6Tc3sHjN72cwWmNmYMGNNghkzYPTIRnb9z1ReYAQp78ms5SPoVTeV0SMbmTEj+phOPz35l3kWLgy/p2arZ58NLvF25hJa6+fbq24qs5aH9/l2tp7ObBd3XbNXjOBbzVNpmN/Io492/phl+vSnYYcd4IarGqm8LpmflYh0c+4eygMoBxqAoUBP4Hlg+zZldgM2Sz8fCzyZse4m4LT0857Aphurc5dddvFiVV/vXtt7hc9itHvQOLPeYxajvbb3Cq+vjy6mpib3vn3dTzml/TIXXuh+xBHRxdTW8uXBIbrkkmjqW7o0qO/SSzu2XVSfb2fr6cx2xVBXlMcwqfWISHIBcz1LPhNmy9muQL27L3T31cAdwKFtEsNZ7v5h+uUcYBCAmVUDewLXp8utdvePQow1dlddluL05msYw5ys68cwh9Oap3H1FanIYnr00WDMqCOPbL/Mhx/CP/8ZWUgbeO214GdULWe1tUHPvo722Izq8+1sPZ3Zrhjq6oykf1YiUgKyZWyFeADfAOoyXh8PXJWj/HdbywM7Av8BbgSeBeqAqna2OwOYC8wdPHhwCHltNDbvu9LrGZr1P+jWRz1DfUB1Y2QxHXOMe02N++rV7Zf59a+D8D76KLKw1rN6tftLL7l/8EF0de67r/uuu3Zsm6g+33zr6VO2rp5zznHvXZbfdlVljf65z7kffHD+dfXtEWzT+si3rtZjMWZMx7eJ4hhG9VlF+TsvItEihpazbFNQe9aCZvsApwIXpBf1AHYGprn7TkAjkHUyFne/1t1Hufuo/v37dz3qmCxbUckQFuUsM5jFLFuxSc4yhdLUFAwrcPjhUFHRfrnWFquFCyMJawMVFfD5z8Nmm0VX54gRwUjynvVszi6qzzffela2rKtnq62gqSW/7ZpaNmHHHWG77fKvq3FNsE3rI9+6Wo/FiBEd36YrkvZZRfU7LyLJEWZytgTYMuP1IOCttoXMbCRBy9ih7v5+xrZL3P3J9Ot7CJK1bqu2T4pFDMlZZjGDqe0TzbgVzc3B3ISnnJK7XNzJ2b33wh/+EG2dX/0qfPvbwQC8+Yrq8823nv7V6+o591zo3zf/7e64Ay67rGN13XEH/3vkW1frsbj22o5v0xVJ+6yi+p0XkeQIMzl7CtjWzLY2s57A0cCDmQXMbDBwL3C8u7/autzd3wHeMLPPpRftB3TrWe/GH1fG9RUTcpapqziT8ceXRxJPdXUwvtno0bnLDR0K++4LVVWRhLWB666Dq6+Ots4DD4RLLll/KquNGX9cGXURfL6dPY86s10x1NUZUdWVtN95EUmQbNc6C/UADgZeJei1+YP0sgnAhPTzOuBD4Ln0Y27GtjsS3Es2D7ifdK/OXA/11iyMFSvc//Qn95Urw6+rq7bd1v2b34y+3k8+cX/77fzL19e7V/dIbg9A9daMvq76eveaXsn4nReReNDOPWehJmdRP4o5OXN3f+SR4I/ChRVTvJ6hvpoeXs9Qv7Biitf2XuGPPBJNHHfeGZwZ//hH/tu0tIQWTrvWrHGvqAiG84jawIHuxx+ff/knngiOaXWPDT/f79oUr+lVuM/3G99w78UKP7+8Y+dRZ86/zp6zUdbVGe3V9R2meHWPwtV1ySXuVbbCL+ix4TkR5e+8iMRDyVmRqK93n3zWKh9Q3ehlrPXqHo0++axVkf73fMQR7p/+dJD85OO889y33z7cmLJ5/fXgDL7uuujrPuAA9513zq/s8uXuQ4cGj+efX/f5lpet9QHVhf18Z892LysLWhM7U0/m+Zfvdp3ZJuq6OiNbXV/cYZWD+z//Wbh6FixYv57qikbv03OVv/RS4eoQkWRSclaEjjnGfdCgaOv85BP3TTZxnzQp/21++EP38vLcQ26E4fHHgzP4b3+Ltl5393PPde/Vy33t2o2Xfe89969+NWg9a8/Che577OH+wgtdi+tf/3Lfc0/3jz/u2n4ku5Urg9aurl7yr693/93vsp8/Dz8cnNcPP9y1OkQk+dpLznpEc2ebdMaBB8Inn8DKldC7dzR1PvxwMJF5roFn2xo6FNauhcWLoxsMFmDPPeHjjzt2Y36hDB8eDDfy2msbf8/9+wfHNZeqqmDi7hNPhNmzcw9fksvuu8Pjj3duW9m4Xr3g+98Pnq9eDT17dnwfLS1w8snw/PNwxBHwmc+sv/6AA4Jpnfbbr+vxikhx0sTnCXbSScEf9agSMwhG+99ii+CPfL7iHE6jurpzfyC7asSI4GeumQI++QSOPx4W5R7KCoDNN4fp0+Hpp+FXv+p4PE88EQx9skqjLkSivh623x4eeaTj206dCv/3f/Db326YmEFwPh90UOcTdBEpfkrOisDq1dHVNX06PPUUlHXgzBg6NPjZ0BBOTO254gq4/PJo62w1YkQwhMcOO7Rf5jvfgdtvh7ffzm+fRxwBxxwDP/0pPPdc/rGsWBEk8nffDWvW5L+ddN6WWwYttqefHkxhlq9XXoGLLoJDDglaSdvzySdBub//veuxikjxUXKWcCecAHvsEV19ZkHLWUdssUVwmSbKS5oAt94Kjz0WbZ2t+vSBiRNhSDtjiD7yCFx/PXzvexsfKy7T734XzN/585/nv83558Prr8ONNwZxSfgqK+Gmm+Ddd+Gcc/Lbxj0Y1LlXr2BgXcs2h0paa5kbbihMvCJSXJScJdzAgfDMM8H9TWE78cRg4NmOKisL/ogccEDhY8pl4cLoE8JMixfDzJkbLv/ww6BFZfhw+MlPOrbPmppgwvlbbsmv/MyZMG1aMMp/lEm8wC67wA9+EHxWDzyw8fJmQdJ9443ZL2dmqqgIpk574IFofvdFJFmUnCXcbrsFl6rmzg23no8+gj/+sfP3LLnDBx8UNKScPvggiLn1kmocfve74PLU2rXrL//pT4MWlZtuClpYOmrkyKDlZMWK4N6m9qxdG7Tefe5zwYwFEr0f/CCYL3RjyXRzc/Bzn31g3Lj89n3kkcE58Je/dClEESlCSs4SbsyY4OesWeHWc//9wR+QjvTSzPTd7waX+LwDk4F3RWvngzhbzoYPD+bXbHuv3U9+Esz5ucsuXdv/IYcEf8jbS5jLy+G++4L72nr16lpd0jk9ewbJ0113tV+muRm+/OWOd/TYZ5/gEvedd3YtRhEpPkrOEq62FrbdNvzk7K67YKut4Itf7Nz2gwcH/+UvW1bQsNr1/vtBT804W86GDw9+tvbY/PjjIFmrrs6/dSSXiy4Khtf40Y82XLd06boYdt6563VJ5w0YEFzaf+edoBdmW7/8JfznP8HvcUf06BHccxrXvLUiEh8lZ0Xg/PPhqKMKu8+GBpg8McWA6ibKy1r454wmBmya6vRwGFH32PzKV4LLml/4QjT1tdXQADddm2ITmvjG4S0MqG5izC4pdtmlcD0mv/IVOOMMuPRSOPrwdZ/V5n2b+OxWKSZNKkw9UhinnBIk5d86ed1n1b9PE7+4OMUhhwS9cTvqssuCjiVxavtdMaC6ickTUxv9Xe/Mdp2tS6IX5ecbVV2JOv+yjUxbrI/uNkNAWFrnDbyo4jdez1BvptzrGeoX9PhNp+fze/HFYFTz224rfLxJ097xO4/feL+Kws6HeM89wdyL32XDuj61ieZeTJLp0917s8K/a+t/Vt/lN17bxblT3367cHF2RHvn+kUVub8rOrNdZ+uS6EX5+UZVV1znH5q+qXi1tLj/97/uDQ1d31d9fXACzmJ08PG3ecxitNf2XtHhuQpXrgx28dOfdj3GfEya5H7xxdHUlSms4xd3XdI1YX5WP/95MFXY8uWFjzuXzr6nzmync714RPn5RlVXnOefkrMitmaNe3W1+4QJXd/XuWeu8osqfpP1BGx9XFgxxSeftarD+/7lL3PPH1lIgwe7H3tsNHVlCvP4xVmXdE2Yn1XrHLJ//GMIgeeQz3u6oMcUn3DKKl+61H3p0iCBPPfMVX7hxrarWH+7b528yi/soXO9GHTmvDhnQn6/H2edvm6bfM+LCzLOi48/znObjPg++CDe71olZ0XugAPcR47s+n4277vS6xma8ySsZ6gPqG7semUhSaXczdx//OPo647y+HWHz6pUhPlZrVnj/pnPuH/96yEEnkO+76kXjf9bNGFC57bbBJ3rxaIzn2//PLfpV9G43uKOnhfjxuW/TWt8O+4Y73dte8mZBeu6h1GjRvncsAcEi8nFF8PPfhYMcFpd3fn9lJe1kPKe9GBtu2Wa6UGvshRr1nasv8gnnwQTgeea0qgQXn01GNvrppuC3mxRCvP4xVmXdE3Yn9U558Dvfw/vvde13/+OyPs9WYorpwbvacQI2G/fjm93ztktpNC5Xgw6c16c8+2ObwMdPy/+/GcYd0ie26Trqq2FY8fH911rZk+7+6i2y3WWF4nddoOWlqBLflfU9kmxiCE5yyxmMLV9Oj4a7W9/GwzIGfaI5nGOcRbm8YuzLumasD+rI48Mhml56KFObd4peb+nvquYNAkmTYK99+7cdrV9da4Xi059vp3YpjPnxVe/2oFt0nUdfXQyv2uVnBWJL30pmP6lq+OdjT+ujOsrJuQsU1dxJuOPL+/wvluTpddf70RgHdDSEvyHHkdyFubxi7Mu6ZqwP6sxY+C224I/PlHp7HvqzHY614vH+OPKqOsRzecb1bmUyPMv27XOYn1053vO3N1nzHB/552u7SPMXimzZwe7eeihrsWYZOqtKdl0x88q6T3sJB719e79KtRbs1BQhwBp1TqeywUVU7yeob6aHl7PUL+wYkqXxnN5773gjPrtbwsbb9K0Hr8LC3z84q5LuibszyqVcr/yyuCftKg88oh7Ta+Of1d05li0t8355TrXk+ahh9w/tUlhPt8oz6Uw4usqJWfdwLvvul9+uftrr3V9X/X17pPPWuUDqhu9vGytD6hu9MlnrerSfwYtLe59+7p/+9tdjy+XAw5wv/DCcOvYmDCOXxLqkq4J87NqaQmGkDn44K7vKx9PPx3U2dn31JntdK4n2zvvBH+H3KP9fKOqK47zr73kTL01i0hrL8XrroPTTuvavk44IehkMCH3ZfYOu/tu2GYb2Gmnwu63lTv06RNMa3TFFeHUIZJU3/seXHll0Gtzs83Cq+fJJ4PvhyuugG9/O7x68rF2LaxeDb16xRtHqXOHQw6BF14I/hZVVsYdUfeg3prdwLbbQk1N1zsFLFkCt9wSziTl3/xmeIkZwLvvwsqV8XQGEInbUUcFc7fed194dTQ1wYknwsCBwc84ffwxDBoEV18dbxwCf/gDPPIInHeeErMoKDkrImZBr62uJmd33x38PPLIrsfU1ptvBn84WloKv29YN7G6kjMpRbvsAkOHwl13hVfHj34Er7wSTLjer1949eSjX78gObvzznjjKHWLF8PkybDXXsEQFxI+JWdFZrfdgi/O99/v/D7uuisYj+yzny1YWP/z0ENw+OHw1luF3zesG+Ns6NBw9i+SZGbBuExr1gSX+wrtX/+Cyy8Pbnc44IDC778zjjwS5s5d97sv0XIPbqNZuxZuuAHKlDVEQoe5yIwZA+Xl8OKLndt+0SKYMye4PBKG1qSptYWr0GpqYOxY2GqrcPYvknQ//znMnBl8DxRaKhX8AzhlSuH33VmtLfytLf4SrZUrg1kpLr1U/xRHSR0Cikxzc3BzbFVV57Z/+WX44Q/h178O59JgQ0PQIeCGG+Dkkwu/fxEJrFgRdI4pNPeghS5JRo8OvveeeSbuSEpTa5qQtPOiO1CHgG6ioqLziRnAdtvBPfeEd8/W4MHBf/RhtZyFcSlHpNjceCP07x/02iyEf/wjmLu3uTmZf4B/+lP41a/WJQkSvpYWOPfcoGemWTLPi+5MyVkRevjh4NLemjUd227p0vCSplYVFUGCFtb9IUOGwPnnh7NvkWKxyy6wahXce2/X9/XJJ0Er9y23BMlZEh14YPBQghCd3/0umC/53/+OO5LSpOSsCC1fDn/5C8yf37HtbrghuOT45pvhxNXqj3+EX/yi8PttbAxi33TTwu9bpJiMGBHc//Obn6UYUN1EeVkLA6qbmDwxlfMfsIYGmDxx/W32GpNi8WK46Sbo3Tu699BRL70El11WuP1lOxadOX5hbBN3fCePT3HBBcG4ZiedlDtOCYeSsyK0227Bz44OqXHnncEE6gMHFj6mTF/6Ujg37KunpkjgL3+B999o5PC3pjJr+QhS3pNZy0fQq24qo0c2MmPGhtvMmAGjRzbSq279bfZ7aSp9yxv56KPI30aHzJwJ3/1ucN9sV7V3LDpz/Aq9TRLiq/3jVMpTjRx9tForY5Nt2oBifXT36ZtatbS4f+Yz7scem/82r74aTNZ12WXhxdWqvj6YX3P58sLu9/77g/fwn/8Udr8ixaTYJnYulDffdDdz/8lPurafpE+mnfT4pLBoZ/qmUFvOzOwgM3vFzOrN7MIs6481s3npxywz26HN+nIze9bMHg4zzmJjFrSedaTlrHXQym9+M5yYMj37LJxzDtTXF3a/rU3zajmTUnbVZSlOb76GMczJun4McziteRpXX5Hq0jZJs8UWsMceXR+QNqrj19ljnvT4JCLZMrZCPIByoAEYCvQEnge2b1NmN2Cz9POxwJNt1n8HuB14OJ86S6XlzN19+nT3Aw90b2rKr/zOO7vvtlu4MbV65pngn6977insfh97zH3SpKDlUKRUbd53pdczNGtrR+ujnqG+eZ9Gd3d/7z33T/XKb5sB1Y0xv7vcrroqCPeFFzq/j3yPX/8+jX7//UGLfb7Hr6Z3o69eHdRTW5X/Nq31PPRQ/vHVVnU8vtre6z7fmt7d45wodkQ98bmZjQEudvevpF9flE4Gf9lO+c2AF9x9YPr1IOAm4BLgO+5+yMbqLIVxzjrr/ffhnXdg+PDw6/r44+Cm/d/8JpioWUQKp7yshZT3pAftjyvTTA96WYo1LWX84x+w374trCaPbcpSrFmb3FuR33kHPvc5qKvr/FWAjhy/tR4cCyO/41dJig8/KqNfPyi3FlJ5btN6EatXL0ityi++TUjRQsfi24R176nMusc5UeziGOdsIPBGxusl6WXtORXIvGXxSuB8IOcsjWZ2hpnNNbO5S5cu7WSoxSvf4TRqaqJJzCCYD6+mpvDDdrz9tsY5E6ntk2IRQ3KWWcxgavqsAmDUKKipym+b2vQ2SfXpTwdDAnXl9oyOHL9nngkGvs33+NVUrfrfwMA1+dZTta6eWbNCji/j8+0u50R3FWZylq2PR9ZmOjPbhyA5uyD9+hDgPXd/emOVuPu17j7K3Uf179+/K/EWnRNPhH333Xi5k06C++8PO5r1DRtW2ORs7dpgjLMf/rBw+xQpRuOPK+P6igk5y9RVnMmxJwTzO/XtC8edkN82448PYU6oAuvZM/jZ0XEeW3Xk+O20E+y0U/7H7/iTyv83rdaxx+e/TWs9O+6Yf3zHndjx+I47cd3n253OiW4p27XOQjyAMcCjGa8vAi7KUm4kwb1pn81Y9kuClrbXgXeAlcCtG6uzlO45c3c/7zz3ykr3VavaL/PCC8HtA1ddFV1c7u5vvOG+YkXh9rdoUfA+rr22cPsUKUal3jNv9Wr3nXZy//73O7d90ntDJj0+KSzauecszOSsB7AQ2Jp1HQKGtykzGKgHdsuxn71Rh4Cs/vSn4BOcPbv9Mj/+sXtZmfvbb0cXVxj+/vfgvc6cGXckIvF75JHgD+uFFVO8nqG+mh5ez1C/sGKK1/Ze4Y88UphtkuqAA9y32aZznYM+/th9883d+1WEf/w6e8yjqqs7nRPFKvLkLKiTg4FX0y1jP0gvmwBMSD+vAz4Enks/NghSyVn73nor+ATbG7uspcX9c59z32efaONyd3/5ZffvfS+IsRDq6oL3+tprhdmfSLGrr3effNYqH1Dd6OVla31AdaNPPmtVzpaOzmyTRNddF3wfPPNMx7c97bTgH9a7747m+HX2mEdVV3c5J4pVe8lZaL0141CKvTW33jq44ffuuzdcN28e7LADTJsGE3LfWlBwjz8Oe+8Njz0GBxzQ9f19//swZQo0NUGPHl3fn4gUr/ffDzoHnHdeMCF6vmbMgIMPhgsu6Nh2ImGJo7emROC734Vx47Kva2wMBm084ohoY4J1A8UWqlPAV78Kl1+uxExEgt7g++8fDK6db/vChx/CaacFvdZ/8pNw4xPpKv2pK3JnndX+ujFj4Iknoosl08CBUFm5bj7Mrtp99+AhIgJBq9lbbwU9ufP5p23+fEilggneKyvDj0+kK5ScFTl3WLw4mNJp8OB1y5cuDb6wNtssnrjKyoJLroVqOZs9G7bbLr73IyLJsv/+HSu/556waBFUVYUTj0gh6bJmkVu7FrbfHi69dP3ll18OgwbBihXxxAXBpc1CjAv84YfBXKLXX9/1fYlI9/H22zB9eu5Lm8uWBd8d7krMpHgoOStyPXrArrsGLUut3IPJgffYg/+NVh2He+8tzGXV1ta3YcO6vi8R6T5mzoQzz4Qnn2y/zFlnBWXq66OLS6SrlJx1A7vtBs8+G3QAAHj6aXjtNTjqqHjjKtR9Ha33rSk5E5FM48YFMwbcdVf29XfdFTwuvhi23TbS0ES6RMlZNzBmTHB5s3UUkTvvhIoKOOywWMPi5ZfhmGPghRe6tp/WlrOtt+56TCLSffTrB2PHBglYS5tZmN99FyZOhC9+Ec4/P574RDpLyVk3MHp08HP27OCS5l13wYEHxn/z/Nq1cMcdwXhrXbFwIWy+eTBHoIhIpiOPhDffXP/WDgguZa5YEfTO1BA8UmyUnHUDH38Mh45NcfklTfQob2Hl+01sVpUq6MTjndHa0tWZ4TQaGmDyxBQDqpu44foW1ixvYvLE+N+TiCTL8OFQ1SPFIfs1UV7WwoDq4Lti3Di46ir4/OfjjlCk45ScFbkZM2D0yEa2nzmV2StGkPKezGkcwZb3TWX0yEZmzIgvtt694TOf6fhwGq3vqVfdVGYtD97Tf5pG0Ksu/vckIskxYwbsv1sjZ9tU5qaC74pZy4Pviu+d1cjAgXFHKNI5mr6piDU0BEnMgyv3ZwxzNlg/m9GM6z2TOfOqYruZfo89gjHPHn88v/LF8J5EJH76rpDuQNM3dUNXXZbi9OZrsn4xAYxhDqc1T+PqK1IRR7bOF74Am2ySf/lieE8iEj99V0h3ppazIjaguolZy0cwjPZv6mpgKLtXz+edj3tHGFnndcf3JCKFp+8K6Q7UctYNLVtRyRAW5SwzmMUsW9GBpquYdcf3JCKFp+8K6c6UnBWx2j4pFjEkZ5nFDKa2z6qIItpQQwN8+cvw97/nV74Y3pOIxE/fFdKdKTkrYuOPK+P6igk5y9RVnMn448sjimhDffvCv/+d/0C0xfCeRCR++q6Q7kzJWRGbdF4l11VMZDajs66fzWjqKs7krMkFmkepE/r3D+b3zHess2J4TyISP31XSHem5KyIDRsGN99TxbjeM7moYgoNDKWZHjQwlIsqpjCu90xuvifebuRmQZz5jnU2bBicfFYVh2yS3PckIvErhu8/kc5Sclbkxo6FOfOqSJ1xNrtXz6dXWYrdq+eTOuNs5syrYuzYuCOEoUPzT87q64NRvUeOTvZ7EpH4FcP3n0hnaCgNCd3llwfz3t19d+5ya9fCXnvBiy8G96hpdG8REenO2htKQ9PBSui+8538yl15ZdB54OablZiJiEjp0mVNSYT6evjBD+DQQ+G44+KORkREJD5KziR0b70FQ4YELWLt2Xpr+MUv4Pe/DzoRiIiIlCpd1pTQ9e8PS5bAf/+bff3q1dCzZ/6XP0VERLoztZxJ6CoqYPDg7D02580LenPOnh19XCIiIkmk5EwiMWzYhgPRrl4NJ54Izc2w7bbxxCUiIpI0Ss4kEtnGOrvkEnjuueA+s9raWMISERFJHN1zJpHYf38oLw/GMisvh2eeCZKz446Dww6LOzoREZHkUMuZhK6hAWb/M8W9tzXRs6KFAdVNTDw1RU0NTJ0ad3QiIiLJouRMQjVjBowe2UivuqnMWj6ClPdk1vIR7PPiVNYub2TOnLgjFBERSRZN3yShaWgIErMHV+7PGDbMwmYzmnG9ZzJnniYnFhGR0tPe9E1qOZPQXHVZitObr8mamAGMYQ6nNU/j6itSEUcmIiKSXKEmZ2Z2kJm9Ymb1ZnZhlvXHmtm89GOWme2QXr6lmf3DzBaY2Ytmdk6YcUo4br+1hVObp+csc1rzNG6/ZW1EEYmIiCRfaL01zawcuBo4AFgCPGVmD7r7SxnFXgP2cvcPzWwscC3wJWANcJ67P2NmfYGnzeyvbbaVhFu2opIhLMpZZjCLWbZik4giEhERSb4wW852BerdfaG7rwbuAA7NLODus9z9w/TLOcCg9PK33f2Z9PPlwAJgYIixSghq+6RYxJCcZRYzmNo+qyKKSEREJPnCTM4GAm9kvF5C7gTrVGBG24VmthWwE/Bkto3M7Awzm2tmc5cuXdr5aKXgxh9XxvUVE3KWqas4k/HHl0cUkYiISPKFmZxZlmVZu4aa2T4EydkFbZb3Af4EnOvun2Tb1t2vdfdR7j6qf//+XQxZCmnSeZVcVzGR2YzOun42o6mrOJOzJldGHJmIiEhyhZmcLQG2zHg9CHirbSEzGwnUAYe6+/sZyysIErPb3P3eEOOUkAwbBjffU8W43jO5qGIKDQylmR40MJSLKqYwrvdMbr5Hw2iIiIhkCjM5ewrY1sy2NrOewNHAg5kFzGwwcC9wvLu/mrHcgOuBBe5+eYgxSsjGjoU586pInXE2u1fPp1dZit2r55M642zmzKti7Ni4IxQREUmWUAehNbODgSuBcuAGd7/EzCYAuPt0M6sDjoD/delb4+6jzOzLwP8B84GW9Lrvu/sjuerTILQiIiJSLNobhFYzBIiIiIjEQDMEiIiIiBQBJWciIiIiCaLkTERERCRBlJyJiIiIJIiSMxEREZEEUXImIiIikiBKzkREREQSpFuNc2ZmS1k3oG0+aoFlIYVTTHQc1tGxWEfHYh0di4COwzo6FuvoWKzT0WMxxN03mBi8WyVnHWVmc7MN/lZqdBzW0bFYR8diHR2LgI7DOjoW6+hYrFOoY6HLmiIiIiIJouRMREREJEFKPTm7Nu4AEkLHYR0di3V0LNbRsQjoOKyjY7GOjsU6BTkWJX3PmYiIiEjSlHrLmYiIiEiilGRyZmYHmdkrZlZvZhfGHU+czOx1M5tvZs+Z2dy444mSmd1gZu+Z2QsZyz5lZn81s/+mf24WZ4xRaedYXGxmb6bPjefM7OA4Y4yCmW1pZv8wswVm9qKZnZNeXnLnRY5jUVLnhZltYmb/MbPn08fhJ+nlpXhOtHcsSuqcyGRm5Wb2rJk9nH5dkPOi5C5rmlk58CpwALAEeAo4xt1fijWwmJjZ68Aody+5MWrMbE9gBXCzu49IL/sN8IG7/yqduG/m7hfEGWcU2jkWFwMr3P3SOGOLkpl9BviMuz9jZn2Bp4HDgJMosfMix7E4khI6L8zMgCp3X2FmFcC/gHOAwym9c6K9Y3EQJXROZDKz7wCjgGp3P6RQf0NKseVsV6De3Re6+2rgDuDQmGOSGLj7E8AHbRYfCtyUfn4TwR+jbq+dY1Fy3P1td38m/Xw5sAAYSAmeFzmORUnxwIr0y4r0wynNc6K9Y1GSzGwQ8FWgLmNxQc6LUkzOBgJvZLxeQgl+4WRw4DEze9rMzog7mAQY4O5vQ/DHCdg85njiNsnM5qUve3b7yzaZzGwrYCfgSUr8vGhzLKDEzov0pavngPeAv7p7yZ4T7RwLKLFzIu1K4HygJWNZQc6LUkzOLMuyks38gd3dfWdgLHBW+vKWCMA0YBiwI/A2cFms0UTIzPoAfwLOdfdP4o4nTlmORcmdF+6+1t13BAYBu5rZiJhDik07x6LkzgkzOwR4z92fDmP/pZicLQG2zHg9CHgrplhi5+5vpX++B9xHcNm3lL2bvtem9Z6b92KOJzbu/m76i7gFuI4SOTfS99L8CbjN3e9NLy7J8yLbsSjV8wLA3T8C/klwj1VJnhOtMo9FiZ4TuwPj0vdt3wHsa2a3UqDzohSTs6eAbc1sazPrCRwNPBhzTLEws6r0jb6YWRVwIPBC7q26vQeBE9PPTwQeiDGWWLV+waR9nRI4N9I3PF8PLHD3yzNWldx50d6xKLXzwsz6m9mm6ee9gP2BlynNcyLrsSi1cwLA3S9y90HuvhVBHvF3dz+OAp0XPQoSZRFx9zVmNgl4FCgHbnD3F2MOKy4DgPuC72B6ALe7+1/iDSk6ZvZHYG+g1syWAP8P+BVwl5mdCiwGvhlfhNFp51jsbWY7Elz2fx34VlzxRWh34Hhgfvq+GoDvU5rnRXvH4pgSOy8+A9yU7ulfBtzl7g+b2WxK75xo71jcUmLnRC4F+a4ouaE0RERERJKsFC9rioiIiCSWkjMRERGRBFFyJiIiIpIgSs5EREREEkTJmYiIiEiCKDkTEcnCzFZkPD/YzP5rZoPjjElESkPJjXMmItIRZrYf8DvgQHdfHHc8ItL9KTkTEWmHme1BMB3Nwe7eEHc8IlIaNAitiEgWZtYMLAf2dvd5cccjIqVD95yJiGTXDMwCTo07EBEpLUrORESyawGOBL5oZt+POxgRKR2650xEpB3uvtLMDgH+z8zedffr445JRLo/JWciIjm4+wdmdhDwhJktc/cH4o5JRLo3dQgQERERSRDdcyYiIiKSIErORERERBJEyZmIiIhIgig5ExEREUkQJWciIiIiCaLkTERERCRBlJyJiIiIJIiSMxEREZEE+f/P+FCj9V58BwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,6))\n",
    "plt.plot(range(1,40),error_rate,color='blue', linestyle='dashed', marker='o',\n",
    "         markerfacecolor='red', markersize=10)\n",
    "plt.title('Error Rate vs. K Value')\n",
    "plt.xlabel('K')\n",
    "plt.ylabel('Error Rate')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=1, n_neighbors=7, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn = KNeighborsClassifier(n_neighbors=7)\n",
    "knn.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Best k value to build the model is 7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "            splitter='best')"
      ]
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dtree = DecisionTreeClassifier()\n",
    "dtree.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = SVC()\n",
    "model.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 150,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logmodel = LogisticRegression()\n",
    "logmodel.fit(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Evaluation using Test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import jaccard_similarity_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, download and load the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2019-04-03 20:10:37--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.193\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.193|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 3642 (3.6K) [text/csv]\n",
      "Saving to: ‘loan_test.csv’\n",
      "\n",
      "100%[======================================>] 3,642       --.-K/s   in 0s      \n",
      "\n",
      "2019-04-03 20:10:37 (652 MB/s) - ‘loan_test.csv’ saved [3642/3642]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Test set for evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>9/15/2016</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>9/11/2016</td>\n",
       "      <td>9/25/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30       9/8/2016   \n",
       "1           5             5     PAIDOFF        300      7       9/9/2016   \n",
       "2          21            21     PAIDOFF       1000     30      9/10/2016   \n",
       "3          24            24     PAIDOFF       1000     30      9/10/2016   \n",
       "4          35            35     PAIDOFF        800     15      9/11/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   50              Bechalor  female  \n",
       "1  9/15/2016   35       Master or Above    male  \n",
       "2  10/9/2016   43  High School or Below  female  \n",
       "3  10/9/2016   26               college    male  \n",
       "4  9/25/2016   29              Bechalor    male  "
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv('loan_test.csv')\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   50              Bechalor  female  \n",
       "1 2016-09-15   35       Master or Above    male  \n",
       "2 2016-10-09   43  High School or Below  female  \n",
       "3 2016-10-09   26               college    male  \n",
       "4 2016-09-25   29              Bechalor    male  "
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\n",
    "test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  \n",
       "0 2016-10-07   50              Bechalor  female          3  \n",
       "1 2016-09-15   35       Master or Above    male          4  \n",
       "2 2016-10-09   43  High School or Below  female          5  \n",
       "3 2016-10-09   26               college    male          5  \n",
       "4 2016-09-25   29              Bechalor    male          6  "
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   50              Bechalor  female          3        0  \n",
       "1 2016-09-15   35       Master or Above    male          4        1  \n",
       "2 2016-10-09   43  High School or Below  female          5        1  \n",
       "3 2016-10-09   26               college    male          5        1  \n",
       "4 2016-09-25   29              Bechalor    male          6        1  "
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 \n",
    "\n",
    "test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   50              Bechalor       1          3        0  \n",
       "1 2016-09-15   35       Master or Above       0          4        1  \n",
       "2 2016-10-09   43  High School or Below       1          5        1  \n",
       "3 2016-10-09   26               college       0          5        1  \n",
       "4 2016-09-25   29              Bechalor       0          6        1  "
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Lets convert male to 0 and female to 1:\n",
    "\n",
    "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>43</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   50       1        0         1                     0   \n",
       "1        300      7   35       0        1         0                     0   \n",
       "2       1000     30   43       1        1         0                     1   \n",
       "3       1000     30   26       0        1         0                     0   \n",
       "4        800     15   29       0        1         1                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        0  \n",
       "3        1  \n",
       "4        0  "
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature = test_df[['Principal','terms','age','Gender','weekend']]\n",
    "feature = pd.concat([feature,pd.get_dummies(test_df['education'])], axis=1)\n",
    "feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "feature.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature and Label selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [],
   "source": [
    "x1 = feature\n",
    "y1 = test_df['loan_status']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalize Data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [],
   "source": [
    "x1 = preprocessing.StandardScaler().fit(x1).transform(x1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## KNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [],
   "source": [
    "p1 = knn.predict(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [],
   "source": [
    "v1 = jaccard_similarity_score(y1,p1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [],
   "source": [
    "v2 = f1_score(y1,p1,average='micro')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [],
   "source": [
    "p2 = dtree.predict(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [],
   "source": [
    "v3 = jaccard_similarity_score(y1,p2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [],
   "source": [
    "v4 = f1_score(y1,p2,average='micro')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [],
   "source": [
    "p3 = model.predict(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [],
   "source": [
    "v5 = jaccard_similarity_score(y1,p3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [],
   "source": [
    "v6 = f1_score(y1,p3,average='micro')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [],
   "source": [
    "p4 = logmodel.predict(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [],
   "source": [
    "p5 = logmodel.predict_proba(x1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [],
   "source": [
    "v7 = jaccard_similarity_score(y1,p4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [],
   "source": [
    "v8 = f1_score(y1,p4,average='micro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [],
   "source": [
    "v9 = log_loss(y1,p5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Report\n",
    "You should be able to report the accuracy of the built model using different evaluation metrics:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Algorithm          | Jaccard | F1-score | LogLoss |\n",
    "|--------------------|---------|----------|---------|\n",
    "| KNN                | ?       | ?        | NA      |\n",
    "| Decision Tree      | ?       | ?        | NA      |\n",
    "| SVM                | ?       | ?        | NA      |\n",
    "| LogisticRegression | ?       | ?        | ?       |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = pd.DataFrame([[v1,v2,'NA'],[v3,v4,'NA'],[v5,v6,'NA'],[v7,v8,v9]],['KNN','Decision Tree','SVM','LogisticRegression'],[ 'F1-score', 'Jaccard', 'LogLoss'])\n",
    "res.columns.names = ['Algorithm']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>Algorithm</th>\n",
       "      <th>F1-score</th>\n",
       "      <th>Jaccard</th>\n",
       "      <th>LogLoss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>KNN</th>\n",
       "      <td>0.722222</td>\n",
       "      <td>0.722222</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Decision Tree</th>\n",
       "      <td>0.685185</td>\n",
       "      <td>0.685185</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SVM</th>\n",
       "      <td>0.722222</td>\n",
       "      <td>0.722222</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LogisticRegression</th>\n",
       "      <td>0.759259</td>\n",
       "      <td>0.759259</td>\n",
       "      <td>0.479193</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Algorithm           F1-score   Jaccard   LogLoss\n",
       "KNN                 0.722222  0.722222        NA\n",
       "Decision Tree       0.685185  0.685185        NA\n",
       "SVM                 0.722222  0.722222        NA\n",
       "LogisticRegression  0.759259  0.759259  0.479193"
      ]
     },
     "execution_count": 176,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<h2>Want to learn more?</h2>\n",
    "\n",
    "<a> Look into Kaggle and UC Irvine Data Sets.</a>\n",
    "\n",
    "<h3>Thanks for completing this lesson!</h3>\n",
    "\n",
    "<p><a>Indranil Mitra</a>, Data Scientist, Intel .</p>\n",
    "\n",
    "<hr>\n",
    "\n",
    "<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}