quanteconomics/markov_chain_ex01_py.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Finite Markov Chains: Examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Daisuke Oyama**\n",
    "\n",
    "*Faculty of Economics, University of Tokyo*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook demonstrates how to analyze finite-state Markov chains\n",
    "with the `MarkovChain` class.\n",
    "For basic concepts and properties on Markov chains, see\n",
    "\n",
    "* [the lecture on finite Markov chains](http://quant-econ.net/py/finite_markov.html)\n",
    "  in Quantitative Economics, and\n",
    "* [the documentation for `MarkovChain`](http://quanteconpy.readthedocs.org/en/stable/markov/core.html).\n",
    "\n",
    "For algorithmic issues in detecting reducibility and periodicity of a Markov chain,\n",
    "see, for example,\n",
    "\n",
    "* J. P. Jarvis and D. R. Shier,\n",
    "  \"[Graph-Theoretic Analysis of Finite Markov Chains](http://www.ces.clemson.edu/~shierd/Shier/markov.pdf),\"\n",
    "\n",
    "from which we draw some examples below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import division, print_function\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from quantecon.markov import MarkovChain"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Example 1: Reducible chain"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the Markov chain given by the following stochastic matrix,\n",
    "taken from Exercise 3 in Jarvis and Shier\n",
    "(where the actual values of non-zero probabilities are not important):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "P = np.zeros((6, 6))\n",
    "P[0, 0] = 1\n",
    "P[1, 4] = 1\n",
    "P[2, [2, 3, 4]] = 1/3\n",
    "P[3, [0, 5]] = 1/2\n",
    "P[4, [1, 4]] = 1/2\n",
    "P[5, [0, 3]] = 1/2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.          0.          0.          1.          0.        ]\n",
      " [ 0.          0.          0.33333333  0.33333333  0.33333333  0.        ]\n",
      " [ 0.5         0.          0.          0.          0.          0.5       ]\n",
      " [ 0.          0.5         0.          0.          0.5         0.        ]\n",
      " [ 0.5         0.          0.          0.5         0.          0.        ]]\n"
     ]
    }
   ],
   "source": [
    "print(P)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a MarkovChain instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "mc1 = MarkovChain(P)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classification of states"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This Markov chain is reducible:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc1.is_irreducible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc1.num_communication_classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Determine the communication classes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0]), array([1, 4]), array([3, 5]), array([2])]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc1.communication_classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Classify the states of this Markov chain:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0]), array([1, 4])]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc1.recurrent_classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obtain a list of the recurrent states:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1 4]\n"
     ]
    }
   ],
   "source": [
    "recurrent_states = np.concatenate(mc1.recurrent_classes)\n",
    "print(recurrent_states)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obtain a list of the transient states:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2 3 5]\n"
     ]
    }
   ],
   "source": [
    "transient_states = np.setdiff1d(np.arange(mc1.n), recurrent_states)\n",
    "print(transient_states)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A Markov chain is reducible (i.e., its directed graph is not strongly connected)\n",
    "if and only if by symmetric permulations of rows and columns,\n",
    "its transition probability matrix is written in the form (\"canonical form\")\n",
    "$$\n",
    "\\begin{pmatrix}\n",
    "U & 0 \\\\\n",
    "W & V\n",
    "\\end{pmatrix},\n",
    "$$\n",
    "where $U$ and $W$ are square matrices.\n",
    "\n",
    "Such a form for `mc1` is obtained by the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.          1.          0.          0.          0.        ]\n",
      " [ 0.          0.5         0.5         0.          0.          0.        ]\n",
      " [ 0.          0.          0.33333333  0.33333333  0.33333333  0.        ]\n",
      " [ 0.5         0.          0.          0.          0.          0.5       ]\n",
      " [ 0.5         0.          0.          0.          0.5         0.        ]]\n"
     ]
    }
   ],
   "source": [
    "permutation = np.concatenate([recurrent_states, transient_states])\n",
    "print(mc1.P[permutation, :][:, permutation])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This Markov chain is aperiodic\n",
    "(i.e., the least common multiple of the periods of the recurrent sub-chains is one):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc1.is_aperiodic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indeed, each of the sub-chains corresponding to the recurrent classes has period $1$,\n",
    "i.e., every recurrent state is aperiodic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Period of the sub-chain\n",
      "[[ 1.]]\n",
      " = 1\n",
      "Period of the sub-chain\n",
      "[[ 0.   1. ]\n",
      " [ 0.5  0.5]]\n",
      " = 1\n"
     ]
    }
   ],
   "source": [
    "for recurrent_class in mc1.recurrent_classes:\n",
    "    sub_matrix = P[recurrent_class, :][:, recurrent_class]\n",
    "    d = MarkovChain(sub_matrix).period\n",
    "    print('Period of the sub-chain\\n{0}\\n = {1}'.format(sub_matrix, d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stationary distributions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each recurrent class $C$, there is a unique stationary distribution $\\psi^C$\n",
    "such that $\\psi^C_i > 0$ for all $i \\in C$ and $\\psi^C_i = 0$ otherwise.\n",
    "`MarkovChain.stationary_distributions` returns\n",
    "these unique stationary distributions for the recurrent classes.\n",
    "Any stationary distribution is written as a convex combination of these distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.33333333  0.          0.          0.66666667  0.        ]]\n"
     ]
    }
   ],
   "source": [
    "print(mc1.stationary_distributions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are indeed stationary distributions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.33333333  0.          0.          0.66666667  0.        ]]\n"
     ]
    }
   ],
   "source": [
    "print(mc1.stationary_distributions.dot(mc1.P))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot these distributions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def draw_histogram(distribution, ax=None, figsize=None,\n",
    "                   title=None, xlabel=None, ylabel=None, ylim=(0, 1)):\n",
    "    \"\"\"\n",
    "    Plot the given distribution.\n",
    "    \n",
    "    \"\"\"\n",
    "    if ax is None:\n",
    "        fig, ax = plt.subplots(figsize=figsize)\n",
    "    n = len(distribution)\n",
    "    ax.bar(np.arange(n), distribution, align='center')\n",
    "    ax.set_xlim(-0.5, (n-1)+0.5)\n",
    "    ax.set_ylim(*ylim)\n",
    "    if title:\n",
    "        ax.set_title(title)\n",
    "    if xlabel:\n",
    "        ax.set_xlabel(xlabel)\n",
    "    if ylabel:\n",
    "        ax.set_ylabel(ylabel)\n",
    "    if ax is None:\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAE7CAYAAAA7EoKDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYbFV57/Hvz3MYRGSSKLM44ACOVwM4xaPk6hGnXM0T\ng6JBjZhryIQmXJIbPWoS9HqTeBN9jFGcFTSOOIGitiIgggKKAuGoJwwyyTwp4HnvH3s3FEVVV50+\n1ae6e38/z9NPV+299tpr7dr99lur1t6VqkKSJEla7u4x7QZIkiRJm4KJryRJkjrBxFeSJEmdYOIr\nSZKkTjDxlSRJUieY+EqSJKkTOpH4JnlJkhOm3Y4NleSQJCf1PL8hyZ4TqvvIJO9pH++ZZH2SiZwP\nSfZo25pJ1NdX95OSXNDW/7xJ19/uY12SAxai7i6a9Pk1xv5mktySZGbM8g9JcmOS25O8coGbpx7G\n5oF1G5uH78PYPEGLPTbPo/4PJLk5yUVzlVsUiW+SJyc5Jcm1Sa5K8u0kj2/X3SXAjFHX3V7Iqvpo\nVT1zIdq+KVXVvatq3Vxlkqwa9aK3dR1VVa+aRLvaYPT0nrovbNu6EDeJfhPwL239x21sZe0fypv7\nFlf7o9a459UiUcAfV9Wq2QVJdkjymTbBXZfkoDsKV/1nVW0NnISv+10Ym8djbAaMzVOxDGLzYUnO\nSPLLJO8ft6Ik72vjyQPvqLzqEOBZo7ZduWFtnrwk2wBfAF4NfALYAngK8KuNrXojt99oSe5RVeun\n3Y5+SVZU1a8nWGWx6Y73HsCP57PhAvR7Q/e/sqpu3wT7CUDvP7dNte9FpP98fCfwS+C+wGOBLyY5\nu6rmdS51gbF50zM2T4exeZPqPx8vAd4MPBO451gVJE8GHsjgN0Gjz/eqmuoP8HjgmiHrHg7cAtwO\n3ABc3S5/NnAmcB1wIfCGnm0uBNa35a8H9gcOAU7qKfNE4HTgWuC7wBN61s3QvHP9drv9CcB9etb/\nB3Bpu+03gb171n0AeBfwJeBG4HXAZcA9esq8ADhrSH/vAxzX9us0mpOht93rgQe2jw8EftS28WLg\ncGCr9nj9uqf/OwNrgE8CH27rfmW77MNtXXu2db+K5iT8OfDavn69uef5KuCi9vGH2/3d3O7zdT31\n3aMts0vbr6uAC4A/7KlrDc0/1Q+27T0HeNyQ4/OTnn1dD2w2Rt29/X5FX32HArfS/CO/Afhcu/xn\nwGuBs9vX+Vhgi57tngOcBVwDnAw8co7zez3wmrZtPxm1PbA78GngCuAXwL/29OXDPeX6j/EM8Hdt\nfTcBD5rHvtcN6jdwL+5+Xu00oK/3BP6xredamhHULQa09eU0/yCvb1/TQ3vq2JEm2bqmfU2/1bPu\nCJpz/XrgPODpQ475N3pf67b9vwIe3LPsg8BRc23X9R+MzcZmY7Ox+c46Jh6b+9a9GXj/GHFpJfB9\n4JH0/N0N+hsYWsemCKAjOnHv9iT6ALAa2L5v/R/QE2DaZU8F9mkfP5ImgD2/fX7/3heyXXbIbB3A\nDu0L9xKaqR6/D1w9u9/2JL0AeDCwZftCHdVX171o/rD/GTizLwhdSxus2xPrR8DqnjKfAf5iyLE4\ntv25J7BPeyL1nly9wfVS4Ent422Bx/Ycm4v66l1DE0Se1z7fEngDdw+uH233/QiaP+4D2vXvB940\n7MSiCUZP73k+W9/sH9O3gHcAmwOPbut+Wk/bbmlf+wD/AJw6x/nSv69Rdd+l3wPqu0vf2mXrgO8A\nOwHb0wSCV7frHgtcDvxm296XtW3afEh719P8g96uPR+Gbb8ZsIImsP1j+zpsATyxreeO12vIMZ5p\n2/1wmvN6sw3Zd8+xHdbvu51XA/r6TuDrNP/Q70GT2Gw+oK0HAg9oH/8WzT+Dx7TPj6JJUFa0P7Pn\n+ENpEqed2ud70BfwetrRn/g+Fripr8zhwHFzbdf1H4zNxmZjs7F5AWNz37q/Y7zE9y+Bf+7/uxv2\nNzDoZ+pzfKvqBuDJNEPW7wGuSPK5JPdti9xt2LqqvllVP2of/5AmID11WPk+zwbOr2Zu2fqqOpbm\nHcrsZPyiOfhrq+qXNO94H9Oz7w9U1U1VdRvwRuDRSe7dU/9nq+rUtuyvaN4tHwzNPEPgGcDH+huV\nZAXNiMPrq+qWtn8fnKM/twL7JNmmqq6rqjNH9P+Uauddtf0aVO6N7b7PoQk6B/Wsm9fHZUl2pxnF\nOaKqbq2qs4H30vxhzzqpqo6v5qz9CE2QnFTd/f0eWFXf86KZq3ZZVV0DfJ47z4FDgXdX1enV+BDN\nqMT+czT1qKq6tj0fhm3/BGBfmsD0l+3r8KuqOmVIG/sV8IGqOrc9r2/bgH33tn1Yv+fcfztv8+XA\nn1XVpW0bvlNVt96toVVfqqqftY+/BXyFJshCc17vDOxZVb+uqpPb5b+m+QexT5LNqpmr+NMRx2TW\n1jQjEb1uoEnsNISxuW20sdnYbGyGhYnNd9n9qALteXUo8Pp51H+HqSe+AFV1XlW9vKp2p3lHuwvw\n9mHlk+yX5BtJrkhyLc0ctPuMubtdaN6d9Pqvdvmsy3oe30Lzj5MkK5K8JcnaJNfRvAuD5iMAaF64\n/knmHwWem2Qr4PdoRgkuH9Cu36AZwu/dvr+dvV5I8+5sXXul5Fx/3NCMUIzSv+9dhhXcALvQfAx6\nU1/du/Y87z0eNwNbjnmV6Th1j9PvQQaeAzSjVq9Ncs3sD7AbTUAYpve4zrX97sB/1fznHg66wGGc\nfY8898ewI81o1U9GFUzyrCTfaS+WuobmPJ79+30bsBb4SpKfJDkCoKrWAn9OM1J0eZJjksx1zHvd\nCGzTt2xbmuRXczA2A8bmWcZmY/OkY/Nddj9GmbfTfArQe2eSDX7jtygS315VdT7Nu+lHzC4aUOxj\nwGeB3apqO+DfuLMvo941XEJzkvW6f7t8lBfTjD4cUFXbAg9olw898FV1CXAqzYjBwTRzmga5kma+\n3B49y/YYUpaqOqOqfocmKH+WZvQDBve/BiwfVK5/37PH5CaaOWqzdhqjrlk/B3ZI0vtHugfzD3ob\nWveo82Hku8y+chcCf19V2/f8bF1VHx9zH3NtfxGwRzvC1O9G5n4NhvVl3H2PMuo4/YLm4rEHz1Uo\nyRbAp4D/A9y3qranmXc5e9HHjVX1uqp6EM3f2uGzV6VX1TFV9RSav9cC3jpGuwH+E1iZpLdtj6aZ\ns6gxGZuNzRvA2Dy6L8bmDesHwNOBtyW5lOYcAzg1ye9vyI6mnvgmeWiSw5Ps2j7fneZjnFPbIpcD\nuyXZrGezrWkuurg1yb40QW/2oF1JM+/jQUN2+WXgIUkOSrIyyYuAh9FM2r6jWUO23Zrm44erk9yL\nZs7TXbozZLsP0Uz+fgTN5Pi7qeaK1k8Da5LcM8neNHPo7ibJZmnuf7ltu90NNB83QHO87tNekT1X\nuwYt+9/tvvehmS83+0d3FnBgku2T7ETz7q7X5Qw53lV1EXAKcFSSLZI8CngFzcdmG2VCdV9Oc3Xo\nKLPH6z3AHyXZN417JXl2X4Cfy1zbn0YzP/AtSbZKsmWSJ7bbnQX8VpLdk2wLHDlHG+ez71EGnVd3\naEdC3gf8U5Kd2xG4JyTZvK/o5u3PL4D1SZ5F8xFz04HkOUke3L6bv57mvP51mnvtPr0Nzr+iCeRj\nXQXejjp9GnhTe1yfDDyX4YmOMDbPMjZvOGPz0DbOZ9+jLNnY3Na7IsmWNJ+qrGjPl0FvMAD2Ah5F\nM3AxO9XjOTRvMMc29cSXJjDsB5yW5EaaoPoDmisYAb5GcxHCZUmuaJe9huaf2PXA33JnEKCqbgb+\nHjg5ydVJ9qPnXXVVXUVzoF5L8wK/DnhOVV3d06bqezz7/EM0H71dQjNadOocZXt9hubd7mdq+Fwm\ngMNoAvhlNCfq+wbUP+tg4GdpPtY7lOaCEKrqPOAY4Kdt/3ce0q7+ZUVzJfRa4ETgbVV1YrvuwzQT\n+9cBx9PM2+vd9iiawHxNksMHtPUgmkn0P6f5B/L6qvr6kHb0bzvKhtbd72hg77btA//xcdfz53s0\nV1i/g+bCmwu467y1Qdve+WSO7dsA9Vyad+YX0owy/F677kSa8/wHNFe9f35A3+Z8Pse+hx2j3n73\nn1eDRjVeB/ywbd9VNOdFeuqanTf6pzSjYFfTvH6f66njwcBXaeLCKcA7q+qbNHPIjqJJni6l+fhu\n0D+YWf3/aF5Dc1HKFTT/fP+oqs4dsU3XGZvvZGy++7ajGJuH7GsD9t2F2Py3NNNojqD527kF+JuB\nna76RVVd0f5c3rb9FyP+du/egKq5z70k76O56OCKqnrkkDL/QnPT4JuBQ+rOyfxqJVlLc2uQr48s\nLGne0nwT2BOA06tq5Lc8JdmL5h/CSuA11VxYsmQZszeMsVnaNDY0Ns+j/qOB3wUur6qHDC03RuL7\nFJo5LB8aFESTHAgcVlUHtu/g/19VjZrM3ylJXkhzBefQF0KSJsGYPT5js9Q9I7+5rapOytzfQf48\nmgseqKrTkmyX5H41+OrYzknzndQPA1465aZI6gBj9niMzVI3TeIri3flrrfluJjmNhydCqLDVM93\nUkvSImDMxtgsddUkEl8YfJPpuxZINmRSvCQtKlW1nC5+M2ZLWvYGxe1J3NXhEpqbO8/ajSH3Xay7\nf/Ucd16cOImfN0y4vhrY7k3x84Y3vGEq+7Uvy78f9mXDf5aZecfspfr62Q/7stR/lks/NmVfhplE\n4nsc7S0/0nxDzbXVsblikrSEGLMlddbIqQ5JjqH5rvUdk1xEM6y6GUBVvbuqvpTkwPaWMDfRfCe0\nJGkKjNmSNNw4d3U4aIwyh02mORtr1bQbMDGrVq2adhMmZrn0Zbn0A+zLcra0Yvbyef2WSz/AvixG\ny6UfMP2+jLyP78R2lFT/vppvvlvs8+cy51wRSctfEmp5Xdw20qCYLUlLxbC4vRi+sliSJElacCa+\nkiRJ6gQTX0mSJHWCia8kSZI6wcRXkiRJnWDiK0mSpE4w8ZUkSVInmPhKkiSpE0x8JUmS1AkmvpIk\nSeoEE19JkiR1gomvJEmSOsHEV5IkSZ1g4itJkqROMPGVJElSJ5j4SpIkqRNMfCVJktQJJr6SJEnq\nBBNfSZIkdYKJryRJkjrBxFeSJEmdYOIrSZKkTjDxlSRJUieY+EqSJKkTTHwlSZLUCSa+kiRJ6gQT\nX0mSJHWCia8kSZI6wcRXkiRJnWDiK0mSpE4w8ZUkSVInmPhKkiSpE0x8JUmS1AkmvpIkSeoEE19J\nkiR1wsjEN8nqJOcluSDJEQPWb5vk80nOSnJOkkMWpKWSpLEYtyVpsFTV8JXJCuB84LeBS4DTgYOq\n6tyeMn8N3LuqjkyyY1v+flV1e19d1b+vJMDw/S8OYa5jJGn5S0JVZdrtGMek4vagmC1JS8WwuD1q\nxHdfYG1Vrauq24Bjgef3lVkPbNM+3ga4qj/plSRtMsZtSRpiVOK7K3BRz/OL22W93gHsneTnwNnA\nn02ueZKkDWTclqQhVo5YP87nXKuB71fV05I8CPhqkkdX1Q39BdesWXPH41WrVm1AMyVp05mZmWFm\nZmbazZivicXt/pht3Ja0WI0bt0fN8d0fWFNVq9vnRwLrq+qtPWW+ABxVVSe3z78GHFFVZ/TV5Rxf\nSUvSEpvjO5G47RxfSUvZfOf4ngHslWTPJJsDLwKO6ytzIc1FFCS5H/BQ4Kcb32RJ0jwYtyVpiDmn\nOlTV7UkOA04AVgBHV9W5SV7drn838GbgA0l+AAT4q6q6eoHbLUkawLgtScPNOdVhojtyqoOkJWop\nTXWYFKc6SFrK5jvVQZIkSVoWTHwlSZLUCSa+kiRJ6gQTX0mSJHWCia8kSZI6wcRXkiRJnWDiK0mS\npE4w8ZUkSVInmPhKkiSpE0x8JUmS1AkmvpIkSeoEE19JkiR1gomvJEmSOsHEV5IkSZ1g4itJkqRO\nMPGVJElSJ5j4SpIkqRNMfCVJktQJJr6SJEnqBBNfSZIkdYKJryRJkjrBxFeSJEmdYOIrSZKkTjDx\nlSRJUieY+EqSJKkTVk67AZIkSctJkmk3YSxVNe0mbHImvpIkSRO32JPKpZGcT5pTHSRJktQJJr6S\nJEnqBBNfSZIkdYKJryRJkjrBxFeSJEmdYOIrSZKkTjDxlSRJUieY+EqSJKkTRia+SVYnOS/JBUmO\nGFJmVZIzk5yTZGbirZQkjcWYLUnDZa6vq0uyAjgf+G3gEuB04KCqOrenzHbAycAzq+riJDtW1S8G\n1FX9+2q+0m/xf7NJF7/ST9KdklBVi/5rjhY6Zksaj/nN9A2L26NGfPcF1lbVuqq6DTgWeH5fmRcD\nn6qqiwEGBVBJ0iZhzJakOYxKfHcFLup5fnG7rNdewA5JvpHkjCQvnWQDJUljM2ZL0hxWjlg/zhj4\nZsB/Aw4AtgJOTfKdqrqgv+CaNWvueLxq1aqxGylJm9LMzAwzMzPTbsZ8LGjMNm5LWqzGjduj5vju\nD6ypqtXt8yOB9VX11p4yRwD3rKo17fP3AsdX1Sf76nKOr6QlaQnN8V3QmC1pPOY30zffOb5nAHsl\n2TPJ5sCLgOP6ynwOeHKSFUm2AvYDfjyJRkuSNogxW5LmMOdUh6q6PclhwAnACuDoqjo3yavb9e+u\nqvOSHA/8AFgPvKeqDKKStIkZsyVpbnNOdZjojpzqIGmJWipTHSbJqQ7S/JnfTN98pzpIkiRJy4KJ\nryRJkjrBxFeSJEmdYOIrSZKkTjDxlSRJUieY+EqSJKkTTHwlSZLUCSa+kiRJ6gQTX0mSJHWCia8k\nSZI6wcRXkiRJnWDiK0mSpE4w8ZUkSVInmPhKkiSpE0x8JUmS1AkmvpIkSeoEE19JkiR1gomvJEmS\nOsHEV5IkSZ1g4itJkqROMPGVJElSJ5j4SpIkqRNMfCVJktQJJr6SJEnqBBNfSZIkdYKJryRJkjrB\nxFeSJEmdYOIrSZKkTjDxlSRJUieY+EqSJKkTTHwlSZLUCSa+kiRJ6gQTX0mSJHWCia8kSZI6wcRX\nkiRJnWDiK0mSpE4YmfgmWZ3kvCQXJDlijnK/meT2JC+YbBMlSeMyZkvScHMmvklWAO8AVgN7Awcl\nefiQcm8FjgeyAO2UJI1gzJakuY0a8d0XWFtV66rqNuBY4PkDyv0J8Engygm3T5I0PmO2JM1hVOK7\nK3BRz/OL22V3SLIrTWB9V7uoJtY6SdKGMGZL0hxWjlg/TkB8O/C/qqqShDk+NluzZs0dj1etWjVG\n1ZK06c3MzDAzMzPtZszHgsZs47akxWrcuJ2q4XEyyf7Amqpa3T4/ElhfVW/tKfNT7gycOwI3A6+q\nquP66qr+fTUxd7EPNoS5jpGk5S8JVbXo58IudMyWNB7zm+kbFrdHJb4rgfOBA4CfA98FDqqqc4eU\nfz/w+ar69IB1Jr6SlqQllPguaMyWNB7zm+kbFrfnnOpQVbcnOQw4AVgBHF1V5yZ5dbv+3QvSWknS\nBjNmS9Lc5hzxneiOHPGVtEQtlRHfSXLEV5o/85vpGxa3/eY2SZIkdYKJryRJkjrBxFeSJEmdYOIr\nSZKkTjDxlSRJUieY+EqSJKkTTHwlSZLUCSa+kiRJ6gQTX0mSJHWCia8kSZI6YeW0GyBJ0qbWfKXs\n4recv1JWmgYTX0lSRy32pHJpJOfSUuJUB0mSJHWCia8kSZI6wcRXkiRJnWDiK0mSpE4w8ZUkSVIn\nmPhKkiSpE0x8JUmS1AkmvpIkSeoEE19JkiR1gomvJEmSOsHEV5IkSZ1g4itJkqROMPGVJElSJ5j4\nSpIkqRNMfCVJktQJJr6SJEnqBBNfSZIkdYKJryRJkjrBxFeSJEmdYOIrSZKkTjDxlSRJUieY+EqS\nJKkTTHwlSZLUCSa+kiRJ6oSxEt8kq5Ocl+SCJEcMWP+SJGcn+UGSk5M8avJNlSSNw5gtSYONTHyT\nrADeAawG9gYOSvLwvmI/BX6rqh4FvBn490k3VJI0mjFbkoYbZ8R3X2BtVa2rqtuAY4Hn9xaoqlOr\n6rr26WnAbpNtpiRpTMZsSRpinMR3V+CinucXt8uGeSXwpY1plCRp3ozZkjTEyjHK1LiVJXka8Arg\nSYPWr1mz5o7Hq1atGrdaSdqkZmZmmJmZmXYz5mvBYrZxW9JiNW7cTtXcMTLJ/sCaqlrdPj8SWF9V\nb+0r9yjg08Dqqlo7oJ7q31cSNiBGT0kYdYwkLW9JqKpMux3jWMiYvZz4/0cLyfNr+obF7XGmOpwB\n7JVkzySbAy8CjuurfA+aAHrwoAAqSdpkjNmSNMTIqQ5VdXuSw4ATgBXA0VV1bpJXt+vfDbwe2B54\nV/Muh9uqat+Fa7YkaRBjtiQNN3Kqw8R25FQHSUvUUprqMClOdVgM/P+zVHl+Td/GTHWQJEmSljwT\nX0mSJHWCia8kSZI6wcRXkiRJnWDiK0mSpE4w8ZUkSVInmPhKkiSpE0x8JUmS1AkmvpIkSeoEE19J\nkiR1gomvJEmSOsHEV5IkSZ1g4itJkqROMPGVJElSJ5j4SpIkqRNMfCVJktQJJr6SJEnqBBNfSZIk\ndYKJryRJkjrBxFeSJEmdYOIrSZKkTjDxlSRJUieY+EqSJKkTTHwlSZLUCSa+kiRJ6gQTX0mSJHWC\nia8kSZI6wcRXkiRJnWDiK0mSpE4w8ZUkSVInmPhKkiSpE0x8JUmS1AkmvpIkSeoEE19JkiR1gomv\nJEmSOmFk4ptkdZLzklyQ5IghZf6lXX92ksdOvpnjmpneridsZmZm2k2YmOXSl+XSD7Avy91SitvL\n5/WbmXYDJmb5vCbLqS8z027AxEz7NZkz8U2yAngHsBrYGzgoycP7yhwIPLiq9gIOBd61QG0dw8z0\ndj1h0z4xJmm59GW59APsy3K21OL28nn9ZqbdgIlZPq/JcurLzLQbMDHTfk1GjfjuC6ytqnVVdRtw\nLPD8vjLPAz4IUFWnAdslud/EWypJGodxW5KGGJX47gpc1PP84nbZqDK7bXzTJEnzYNyWpCFSVcNX\nJi8EVlfVq9rnBwP7VdWf9JT5PPCWqjq5fX4i8FdV9f2+uobvSJIWuarKtNswjknFbWO2pKVuUNxe\nOWKbS4Dde57vTjMyMFeZ3dplI3cuSZq4icRtY7ak5WjUVIczgL2S7Jlkc+BFwHF9ZY4DXgaQZH/g\n2qq6fOItlSSNw7gtSUPMOeJbVbcnOQw4AVgBHF1V5yZ5dbv+3VX1pSQHJlkL3AS8fMFbLUkayLgt\nScPNOcdXkiRJWi6WzTe3jXPD9qUgyfuSXJ7kh9Nuy8ZIsnuSbyT5UZJzkvzptNs0X0m2THJakrPa\nvqyZdps2VpIVSc5sL3JaspKsS/KDti/fnXZ7ND5j9uKzXOK2MXvxWgwxe1mM+LY3bD8f+G2aCzRO\nBw6qqnOn2rB5SPIU4EbgQ1X1yGm3Z76S7ATsVFVnJdka+B7wO0vxNQFIslVV3ZxkJfBt4M/a+58u\nSUkOBx4H3Luqnjft9sxXkp8Bj6uqq6fdFo3PmL04Lae4bcxenBZDzF4uI77j3LB9Saiqk4Brpt2O\njVVVl1XVWe3jG4FzgV2m26r5q6qb24ebA5sB66fYnI2SZDfgQOC9wHK4cn859KFrjNmL0HKK28bs\nRW2qfVguie84N2zXlCTZE3gssJTfbd8jyVnA5cBXqur0abdpI/wz8Jcs4X8EPQo4MckZSV417cZo\nbMbsRW6px21j9qI19Zi9XBLfpT9fY5lqPy77JM3HTDdOuz3zVVXrq+oxNPc73S/JPtNu03wkeQ5w\nRVWdyfIYOXhSVT0WeBbwx+3Hzlr8jNmL2HKI28bsRWvqMXu5JL7j3LBdm1iSzYBPAR+pqs9Ouz2T\nUFXXAd8AVk+7LfP0ROB57TyrY4CnJ/nQlNs0b1V1afv7SuAzNB+ha/EzZi9Syy1uG7MXl8UQs5dL\n4jvODdu1CSUJcDTw46p6+7TbszGS7Jhku/bxPYH/TjP3bcmpqr+uqt2r6gHA7wNfr6qXTbtd85Fk\nqyT3bh/fC3gGsOSvrO8IY/YitFzitjF7cVosMXtZJL5VdTswe8P2HwMfX4pXoQIkOQY4BXhIkouS\nLNUbyz8JOBh4WnvbkjOTLNV33DsDX09yNvBdmvliX5pymyZlKX/kfD/gpHYe32nAF6rqK1Nuk8Zg\nzF60lkvcNmYvTosiZi+L25lJkiRJoyyLEV9JkiRpFBNfSZIkdYKJryRJkjrBxFeSJEmdYOIrSZKk\nTjDxlSRJUieY+GpRSfI3Sc5JcnZ7D8l9k/xZexPyUdv++TjlJEmTYczWUuN9fLVoJHkC8I/AU6vq\ntiQ7AFsCJwOPr6qrRmz/s3HKSZI2njFbS5EjvlpMdgJ+UVW3AVTV1cDvArsA30jyNYAk70pyejvK\nsKZd9qcDyj0jySlJvpfkE+1XJJLkLUl+1I5QvG2T91KSlgdjtpYcR3y1aLRB7tvAVsCJNF9j+q12\nVOBxbVAlyfZVdU2SFW25P6mqc3rLJdkR+BSwuqpuSXIEsDnwTuCUqnpYW9c2VXX9Ju+sJC1xxmwt\nRY74atGoqpuAxwGHAlcCH09ySLs6PUVflOR7wPeBfYC9B1S3f7v8lCRnAi8D9gCuA36Z5Ogk/wO4\nZSH6IknLnTFbS9HKaTdA6lVV64FvAt9M8kPgkNlVAEkeALyWZl7YdUneTzOnbJCvVtWL+xcm2Rc4\ngOYjucPax5KkDWTM1lLjiK8WjSQPSbJXz6LHAuuAG4Bt2mXbADcB1ye5H/CsnvK95U4DnpTkQW3d\n90qyV/vR3HZV9WXgcODRC9UfSVrOjNlaihzx1WKyNfCvSbYDbgcuoPkI7cXA8UkuqaoD2o/BzgMu\noplfNuvf+8odAhyTZIt2/d/QBNrPJdmS5qO4v9gUHZOkZciYrSXHi9skSZLUCU51kCRJUieY+EqS\nJKkTTHwlSZLUCSa+kiRJ6gQTX0mSJHWCia8kSZI6wcRXkiRJnWDiK0mSpE4w8ZUkSVInmPhKkiSp\nE0x8JUmS1AkmvpIkSeoEE19JkiR1gomvJEmSOsHEV5IkSZ1g4itJkqROMPGVJElSJ5j4SpIkqRNM\nfCVJktRZu2r5AAAGxElEQVQJJr6SJEnqBBNfSZIkdYKJryRJkjrBxFeSJEmdYOIraUlL8pIkJ0y7\nHRsqySFJTup5fkOSPSdU95FJ3tM+3jPJ+iQTifdJ9mjbmknUJ0mbkomvpAWV5MlJTklybZKrknw7\nyePbdXdJ/sao625JXFV9tKqeuRBt35Sq6t5VtW6uMklWJblojLqOqqpXTaJdSdYleXpP3Re2ba1J\n1C9Jm9LKaTdA0vKVZBvgC8CrgU8AWwBPAX61sVVv5PYbLck9qmr9tNvRL8mKqvr1BKssFsHxlqRJ\ncMRX0kJ6CFBV9fFq/LKqvlpVP0zycOBdwBPaj86vBkjy7CRnJrkuyYVJ3tBT37fa39cmuT7J/gOm\nDDwxyentCPN3kzyhZ91Mkje1o87XJzkhyX161v9Hkkvbbb+ZZO+edR9I8q4kX0pyI3B4kst6R5+T\nvCDJWYMORJL7JDmu7ddpwIP61q9P8sD28YFJftS28eIkhyfZCvgysEt7vK5PsnOSNUk+meTDSa4D\nDmmXfbivCa9MckmSnyd5bV+/3tzz/I5R5baOPYDPt/t8Xf+oe5Jd2n5dleSCJH/YU9eaJJ9I8sG2\nveckeVzP+iPa/l2f5LzekWVJWggmvpIW0vnAr9vkanWS7WdXVNW5wB8Bp7Yfne/QrroROLiqtgWe\nDfzPJM9v1z2l/b1tVW1TVd/p3VmSHYAvAm8HdgD+Cfhi736Bg4BDgPsCmwOv61n3ReDBwG8A3wc+\n2tefg4A3V9XWwL8CVwHP6Fn/UuCDQ47FO4GbgZ2AVwAvpxlNHeRo4NCq2gbYB/hGVd0MrAZ+3h6v\nbarq0rb884D/aI/ZR4fUu6rt2zOAI5Ic0C6vYe2oqpcCFwLPaff5fwcUO7YtszPwu8A/JHlaz/rn\nAscA2wLHAe8ASPJQ4I+Bx7f9fAawbsjxkKSJMPGVtGCq6gbgyTSJ1XuAK5J8Lsl92yJ3+wi9qr5Z\nVT9qH/+QJrF66rDyfZ4NnN/O+11fVccC59EkhrTteH9Vra2qX9JMv3hMz74/UFU3VdVtwBuBRye5\nd0/9n62qU9uyv6JJcg+GO5LuZwAf629UkhXAC4DXV9Utbf8+OEd/bgX2SbJNVV1XVWeO6P8pVXVc\n265fDin3xnbf5wDvp0ni72jikHrnlGR34InAEVV1a1WdDbwXeFlPsZOq6vh2TvBHgEe3y39NM/Vl\nnySbtXOHfzqfdkjSuEx8JS2oqjqvql5eVbsDjwB2oRmRHSjJfkm+keSKJNfSzA++z7DyfXahGX3s\n9V/t8lmX9Ty+Bdi63e+KJG9JsradMvCztsyOs10B+i8s+yjw3HYawu8B36qqywe06zdorqno3b6/\nnb1eCBwIrGunZ+w/R1mAi0esZ8C+dxlWcAPsAlxdVTf11b1rz/Pe43EzsGU7P3ot8OfAGuDyJMck\n2XkCbZKkoUx8JW0yVXU+zUjnI2YXDSj2MeCzwG5VtR3wb9wZq0bdSeAS4P59y+7fLh/lxTQjwwe0\nUwYe0C4fOhpaVZcAp9KM5h4M9M+rnXUlcDvNfNlZewwpS1WdUVW/Q5Mwf5ZmZBoG93/QVIVB5fr3\nPXtMbgK26lm30xh1zfo5sEOSrfvqHicRp6qOqaqn0LxGBbx1nO0kab5MfCUtmCQPbS/M2rV9vjvN\nR+yntkUuB3ZLslnPZlsD11TVrUn2pUlIZ5OvK4H19F0Y1uPLwEOSHJRkZZIXAQ+jubPEHc0asu3W\nNHebuDrJvYB/6O/OkO0+BBxBk8x/elCB9i4LnwbWJLlne9HcHwwqm2SzNPcm3rbd7gaaaQHQHK/7\npLlbxlztGrTsf7f73odmjvPH2+VnAQcm2T7JTjSjsL0uZ8jxrqqLgFOAo5JskeRRNPOXPzKofF8/\nH5Lk6Um2oDnuv+zppyQtCBNfSQvpBmA/4LT2TginAj8AZu8q8DXgR8BlSa5ol70GeFOS64G/5c4E\njfYCr78HTk5ydZL96BnxrKqrgOe09f+C5sK151TV1T1tqr7Hs88/RDMt4hLgnLatw8r2+gzNKOdn\n2vm1wxxGk1xfBryv/emvf9bBwM/aKReHAi9p+3cezYViP237v/OQdvUvK+CbwFrgROBtVXViu+7D\nwNk0F5YdTzOnunfbo2iS5muSHD6grQcBe9KM/n6aZh7z14e0o3fbLdq6rwQupZlSciSStIDiPcgl\naeMkWUtzF4avjywsSZoaR3wlaSMkeSGw3qRXkhY/v7lNkuYpyQzNHOKXTrkpkqQxONVBkiRJneBU\nB0mSJHWCia8kSZI6wcRXkiRJnWDiK0mSpE4w8ZUkSVIn/H9uAktm01TWxwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d9e75f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(12, 4))\n",
    "titles = ['Stationary distribution for the recurrent class {0}'.format(recurrent_class)\n",
    "          for recurrent_class in mc1.recurrent_classes]\n",
    "\n",
    "for ax, title, dist in zip(axes, titles, mc1.stationary_distributions):\n",
    "    draw_histogram(dist, ax=ax, title=title, xlabel='States')\n",
    "\n",
    "fig.suptitle('Stationary distributions', y=-0.05, fontsize=12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simulation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us simulate our Markov chain `mc1`.\n",
    "The `simualte` method generates a sample path\n",
    "of length given by the first argument, `ts_length`,\n",
    "with an initial state as specified by an optional argument `init`;\n",
    "if not specified, the initial state is randomly drawn. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A sample path from state `0`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc1.simulate(50, init=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As is clear from the transition matrix `P`,\n",
    "if it starts at state `0`, the chain stays there forever,\n",
    "i.e., `0` is an absorbing state, a state that constitutes a singleton recurrent class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start with state `1`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 4, 1, 4, 1, 4, 1, 4, 1, 4, 4, 1, 4, 4, 4, 1, 4, 1, 4, 1, 4, 1, 4,\n",
       "       1, 4, 4, 4, 1, 4, 4, 1, 4, 1, 4, 1, 4, 4, 4, 1, 4, 4, 1, 4, 1, 4, 4,\n",
       "       1, 4, 1, 4])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc1.simulate(50, init=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can observe that the chain stays in the recurrent class $\\{1,4\\}$\n",
    "and visits states `1` and `4` with certain frequencies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If `init` is not specified, the initial state is randomly chosen:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 3, 5, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc1.simulate(50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note on reproducibility**:\n",
    "The `simulate` method offers an option `random_state` to set a random seed\n",
    "to initialize the pseudo-random number generator.\n",
    "As you provide the same random seed value,\n",
    "`simulate` returns the same outcome."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example, the following will always give the same sequence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2, 4, 1, 4, 4, 4, 4, 1, 4, 1, 4, 4, 1, 4, 1, 4, 4, 4, 4, 4, 1, 4, 4,\n",
       "       1, 4, 1, 4, 1, 4, 4, 1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 1, 4, 1, 4, 1, 4,\n",
       "       4, 4, 4, 4])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc1.simulate(50, random_state=12345)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Time series averages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let us compute the frequency distribution along a sample path, given by\n",
    "$$\n",
    "\\frac{1}{t} \\sum_{\\tau=0}^{t-1} \\mathbf{1}\\{X_{\\tau} = s\\}\n",
    "\\quad (s \\in S).\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def time_series_dist(mc, t, init=None, random_state=None):\n",
    "    \"\"\"\n",
    "    Return the distribution of visits by a sample path of length t\n",
    "    of mc with an initial state init.\n",
    "    \n",
    "    \"\"\"\n",
    "    t_max = np.max(t)\n",
    "    dim = 1\n",
    "    try:\n",
    "        ts_size = len(t)  # t is an array\n",
    "        ts_array = t\n",
    "        dim = 2\n",
    "    except:  # t is an int\n",
    "        ts_size = 1\n",
    "        ts_array = [t]\n",
    "        \n",
    "    X = mc.simulate(ts_length=t_max, init=init, random_state=random_state)\n",
    "    dists = np.empty((ts_size, mc.n))\n",
    "    bins = np.arange(mc.n+1)\n",
    "    for i, length in enumerate(ts_array):\n",
    "        hist, bin_edges = np.histogram(X[:length], bins=bins)\n",
    "        dists[i, :] = hist / length\n",
    "    if dim == 1:\n",
    "        return dists[0]\n",
    "    else:\n",
    "        return dists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a frequency distribution along a sample path,  of length 100,\n",
    "from initial state `1`, which is a recurrent state:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.  ,  0.38,  0.  ,  0.  ,  0.62,  0.  ])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "time_series_dist(mc1, t=100, init=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Length 10,000:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.    ,  0.3347,  0.    ,  0.    ,  0.6653,  0.    ])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "time_series_dist(mc1, t=10**4, init=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The distribution becomes close to the stationary distribution `(0, 1/3, 0, 0, 2/3, 0)`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the frequency distributions for a couple of different time lengths:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def plot_time_series_dists(mc, init, ts, seed=None, figsize=(12,4)):\n",
    "    dists = time_series_dist(mc, t=ts, init=init, random_state=seed)\n",
    "\n",
    "    fig, axes = plt.subplots(1, len(ts), figsize=figsize)\n",
    "    titles = ['t={0}'.format(t) for t in ts]\n",
    "\n",
    "    for ax, title, dist in zip(axes, titles, dists):\n",
    "        draw_histogram(dist, ax=ax, title=title, xlabel='States')\n",
    "\n",
    "    fig.suptitle('Time series distributions with init={0}'.format(init),\n",
    "                 y=-0.05, fontsize=12)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAE7CAYAAAA7EoKDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8tGVd7/HP1wdI5SAaRQkKHjDBrWIZ4jZ1qaWPpvKq\nnS+D1ChL223LrZWEHXzS2mqHrZVmUKRmbqnUCtxuUdPlARSlAEXBIH0EVPCEiIAKPr/9x9wLx8U6\nzFrrnrXWPdfn/XrNi5m5r7mu6575cj+/ueeaWakqJEmSpFl3m62egCRJkrQZLHwlSZLUBAtfSZIk\nNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCdxMk2Z3kket43J4kX0tyXXc5dRrzkxZsIKun\nJrkkybeS/OwS25+T5HNJrk1yWpJ9+pmxNDKt46zZVd+24jib5E5J/qnL+u4kx290P4bKwndzFJB1\nPva+VbV/d3lGn5OSlrDerF4A/DLw710ft0jyGOAk4JHAYcDdgd/b2DSlW+n9OGt2NSVbcZx9JfB1\n4HuBnwFeleSodcxh8Cx8pyzJ64C7Amd2ZxN+fY1d+BppU2wkq1X1F1X1LkYH1sV+Fvjrqrq4qr4C\nvBA4sY85SzDV46zZVa+24jibZF/gJ4Hfqaobqups4F+Ap25sb4bJomrKquqpwOXA47uzCX+c5CtJ\nrlnm8rxFXby3++jiTUkO24JdUCN6yOpyjgIuHLv9EeDgJHfsex/UpikeZ82uerVFx9l7ATdX1WVj\n2y8E7tPHPg3NXls9gRZV1YETNn0Y8EFgX+D3gbckObqqvjW1yUlj1pDVlewHXDt2+6vdf/cHrumh\nf+lWNnicvX9V7cHsahNswnF2v7HbC67rtjXHM77bWFW9v6purqprgWcDhwP33tpZSWv2NeCAsdt3\n6P573RbMRfoOyxxnj+w2m10NxUpZXbxtYXuTObbw3RyLF6GPf4N48eU3l+kji/4rTUMfWV3sY8DR\nY7fvD1xdVZ4xU5+mcZw1u5qGzT7O/gewV5J7Ltp+0UZ2Yqhc6rA5rgbuAbwLoKr2W+0B3bct9wE+\nCtwO+APgSuDi6U1TWntWAZLsDexg9GZ6nyS3Bb5RVQX8LfCaJK8HrgJ+B3j1FOautk3jOGt2NQ2b\nepytquuTvBl4YZJfAH4QeALw4F73aiA847s5Xgz8drdQ/bkTPuZg4HRGa3b+E7gLo8Xwru/VNK0n\nqwDvAG4AjgVO7a4/FKCqzgL+EHg3sJtRnl/Q45wlmMJx1uxqSrbiOPvLjN7cfR74O+CXqqrJE2kZ\nvVFYoUHyN8CPA5+vqvsu0+bPgMcyehFOrKrz+56otFZmV0NkbjVUZldDMMkZ31cDO5fbmORxwD2r\n6gjgGcCrepqbtFFmV0NkbjVUZlfb3qqFb1W9j5V/tuWJwGu7tucCByY5uJ/pSetndjVE5lZDZXY1\nBH2s8T0EuGLs9pXAoT30K02b2dUQmVsNldnVluvrVx0W/8TWrRYOJ1l5MbE0oarq8yfdzK42hbnV\nUJldDdVS2e3jjO9nGH0TdsGh3X1LTWDiywte8II1tV/rZdr9z8I+bMf+e2Z2t6D/WdiHtfZvbrfn\n69Ja/+sZw+xuz9fF/le/LKePwvcM4GkASY4FvlJVV/fQrzRtZldDZG41VGZXW27VpQ5J3gA8HDgo\nyRWMfhdub4CqOqWq3prkcUkuA64Hfm6aE5YmZXY1ROZWQ2V2NQSrFr5VdfwEbZ7Vz3S+bW5uru8u\nN7X/zRjD/ldmdrdn/5sxxpD7n9XcbsYY9r+1Y8xqdof+utj/d1r1D1j0NlBSmzWWZlcSqt8vWkwy\nptnVhphbDZXZ1VAtl13/ZLEkSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4\nSpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIk\nqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkW\nvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIk\nSWqCha8kSZKaYOErSZKkJlj4SpIkqQmrFr5Jdia5JMmlSU5aYvsdkpyZ5IIkFyU5cSozldbI7Gqo\nzK6GyNxqCFJVy29MdgCfAH4U+AzwYeD4qrp4rM3zgf2r6uQkB3XtD66qmxf1VSuNJU0iCVWVCdqZ\nXW0bk+a2a9tLds2t+uAxV0O1XHZXO+N7DHBZVe2uqpuA04HjFrXZAxzQXT8A+NLiEEtbwOxqqMyu\nhsjcahBWK3wPAa4Yu31ld9+4VwBHJfkscCHw7P6mJ62b2dVQmV0NkbnVIOy1yvZJPmvYCfx7VT0i\nyT2AdyS5f1Vdt7jhrl27brk+NzfH3NzcGqaqFs3PzzM/P7+eh5pdbZkN5BZ6zK651Vp5zNVQTZrd\n1db4Hgvsqqqd3e2TgT1V9dKxNm8BXlxVZ3e3/xU4qarOW9SXa3a0YWtYb2Z2tW2scY1vL9k1t+qD\nx1wN1XrX+J4HHJHk8CT7AE8GzljU5nJGi9lJcjDwA8AnNz5laUPMrobK7GqIzK0GYcWlDlV1c5Jn\nAWcBO4DTquriJM/stp8CvAh4TZKPAAGeV1VfnvK8pRWZXQ2V2dUQmVsNxYpLHXodyI8u1IO1fGTc\n45hmVxtibjVUZldDtd6lDpIkSdJMsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8\nJUmS1AQLX0mSJDVhxb/cJkmSNi7p929A+AcepPWx8JUkaVP0Vaxu6h9Sk2aKSx0kSZLUBAtfSZIk\nNcHCV5IkSU1wja8kSZJupe8vZcLWfzHTwleSJEnL6LNQ3fovZrrUQZIkSU2w8JUkSVITLHwlSZLU\nBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtf\nSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIk\nNWHVwjfJziSXJLk0yUnLtJlLcn6Si5LM9z5LaR3MrobK7GqIzK2GIFW1/MZkB/AJ4EeBzwAfBo6v\nqovH2hwInA08pqquTHJQVX1xib5qpbGkSSShqjJBO7OrbWPS3HZte8muud1ekgB9vR5hs15bj7lt\n6ze3sB2yu9oZ32OAy6pqd1XdBJwOHLeozQnAm6rqSoClQixtAbOroTK7GiJzq0FYrfA9BLhi7PaV\n3X3jjgDulOTdSc5L8tQ+Jyitk9nVUJldDZG51SDstcr2Sc5H7w38IPAo4PbAB5J8sKouXdxw165d\nt1yfm5tjbm5u4omqP6OPLvo1rY8u5ufnmZ+fX89Dze4MGkp2N5Bb6DG75lZr5TFXQzVpdldb43ss\nsKuqdna3Twb2VNVLx9qcBNyuqnZ1t/8aeFtVvXFRX67Z2SZmcc3OEu3M7gwaanbXuMa3l+ya2+2l\ngTW+HnNn0FCPubD+Nb7nAUckOTzJPsCTgTMWtfkX4EeS7Ehye+BBwMf7mLS0AWZXQ2V2NUTmVoOw\n4lKHqro5ybOAs4AdwGlVdXGSZ3bbT6mqS5K8DfgIsAf4q6oyyNpSZldDZXY1ROZWQ7HiUodeB/Kj\ni21jFj+6mPKYZnebGGp2za1mfalDz2Oa3W1iqMdcWP9SB0mSJGkmWPhKkiSpCRa+kiRJaoKFryRJ\nkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg\n4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJ\nkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQm\nWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkpqwauGbZGeSS5Jc\nmuSkFdr9cJKbk/xkv1OU1sfsaqjMrobI3GoIVix8k+wAXgHsBI4Cjk9y5DLtXgq8DcgU5imtidnV\nUJldDZG51VCsdsb3GOCyqtpdVTcBpwPHLdHuV4A3Al/oeX7SepldDZXZ1RCZWw3CaoXvIcAVY7ev\n7O67RZJDGIX7Vd1d1dvspPUzuxoqs6shMrcahL1W2T5JKF8O/GZVVZKwwkcXu3btuuX63Nwcc3Nz\nE3S/dqNp9KfK/ze3yvz8PPPz8+t56OCy23duwexulQ3kFnrM7mYdczU7WjrmarZMmt2s9A9jkmOB\nXVW1s7t9MrCnql461uaTfDu8BwE3AL9YVWcs6qs26x/h0f9PfY2VmSse+n1+YDOfoyRU1aoV4hCz\nO+TXZbMM9TmaNLdd216yu5nHXK1uqP8uzfIxV6sb6jEXls/uaoXvXsAngEcBnwU+BBxfVRcv0/7V\nwJlV9eYltln4bhOzGOQl2g0uu0N+XTbLUJ+jNRa+vWTX4mF7Geq/S7N8zNXqhnrMheWzu+JSh6q6\nOcmzgLOAHcBpVXVxkmd220+ZymylDTK7GiqzqyEytxqKFc/49jqQZ3y3jVl8BzflMT3ju00M9Tma\n5dxqMkP9d8nstm2ox1xYPrv+5TZJkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJ\nTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDw\nlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTdhrqycg\nSZK0FZL02l9V9dqf+mfhK0mSGtZXsdpvEa3pcKmDJEmSmmDhK0mSpCa41EGStCGuk5Q0FBa+kqQe\nuE5S0vbnUgdJkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJ\nktSEiQrfJDuTXJLk0iQnLbH9Z5JcmOQjSc5Ocr/+pyqtjbnVUJldDZXZ1Xa3auGbZAfwCmAncBRw\nfJIjFzX7JPCwqrof8CLg1L4nKq2FudVQmV0NldnVEExyxvcY4LKq2l1VNwGnA8eNN6iqD1TVtd3N\nc4FD+52mtGbmVkNldjVUZlfb3iSF7yHAFWO3r+zuW87TgbduZFJSD8ythsrsaqjMrra9vSZoU5N2\nluQRwM8DD1lq+65du265Pjc3x9zc3KRdq1Hz8/PMz8+v56G95RbMrtZmA7kFj7naQmZXQzVpdlO1\nck6THAvsqqqd3e2TgT1V9dJF7e4HvBnYWVWXLdFPrTZWX5Kwhv//VuuNzZr3Zun3+YHNfI6SUFWZ\noF0vue3abEp2h/y6bJahPkeT5rZr6zF3xnILw32OzO6aepu57A71mAvLZ3eSpQ7nAUckOTzJPsCT\ngTMWdX5XRiF+ynLFg7TJzK2GyuxqqMyutr1VlzpU1c1JngWcBewATquqi5M8s9t+CvC7wB2BV43e\nHXBTVR0zvWlLKzO3Giqzq6EyuxqCVZc69DaQH11sG7P40cWUx3SpwzYx1OdolnPbjYXH3JUN9Tky\nu2vqbeayO9RjLmxsqYMkSZI0eBa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJ\nkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg\n4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJ\nkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQm\nWPhKkiSpCRa+kiRJaoKFryRJkpqwauGbZGeSS5JcmuSkZdr8Wbf9wiQP6GNi8/PzfXSz0ghT7n/6\n+zD052ja8ze76+x96vP3OVrNVmR3M173ob8uQ88tTHcfPOZuYASzu3LvPc5/xcI3yQ7gFcBO4Cjg\n+CRHLmrzOOCeVXUE8AzgVX1MbOgvEhjkVXuf7gHY7K63dwvf1Xufwexa+G59/0P+d8lj7gZHMLsr\n975ZhS9wDHBZVe2uqpuA04HjFrV5IvBagKo6FzgwycG9zVBaH7OroTK7GiJzq0FYrfA9BLhi7PaV\n3X2rtTl041OTNsTsaqjMrobI3GoYqmrZC/DfgL8au/0U4M8XtTkTeMjY7XcCP7hEX+XFSx+XlTJr\ndr1s18skue0zu1u9v15m5+Ix18tQL0tldS9W9hngLmO378LoHdpKbQ7t7vsOVZVVxpL6ZHY1VL1k\n19xqk3nM1SCsttThPOCIJIcn2Qd4MnDGojZnAE8DSHIs8JWqurr3mUprY3Y1VGZXQ2RuNQgrnvGt\nqpuTPAs4C9gBnFZVFyd5Zrf9lKp6a5LHJbkMuB74uanPWlqF2dVQmV0NkbnVUKRbTyNJkiTNtG35\nl9sm+RHsDfT9N0muTvLRPvsd6/8uSd6d5GNJLkryqz33f9sk5ya5oOt/V5/9j42zI8n5Sc6cUv+7\nk3ykG+ND0xhjs00zt13/ZneycczuGpndVfsffHZnMbdgvTDBGGZ3sUm/abxZF0YfkVwGHA7sDVwA\nHNlj/w8FHgB8dErz/z7g6O76fsAn+px/1+/tu//uBXwQeNAU9uO5wOuBM6b0PH0KuNNmZmual2nn\nthvD7E42htld2/6Y3cnGGHR2Zy233T5ZL0w2jtkdu2zHM76T/Aj2ulXV+4Br+upvif6vqqoLuutf\nAy4G7tzzGDd0V/dh9D/7nj77T3Io8Djgr4Fpfrt2lr65O9XcgtmdhNldF7M72RizkN1Zyi1YL0w6\njtkdsx0L30l+BHsQkhzO6N3iuT33e5skFwBXA2+vqg/32T/wMuA36Pl/jkUKeGeS85L84hTH2Swz\nk1swu6swu9uY2V3WrOUWZii708pt17fZHbMdC9+Z+LZdkv2ANwLP7t7J9aaq9lTV0Yx+A/FBSe7T\nV99JHg98vqrOZ7pnBx5SVQ8AHgv8jyQPneJYm2EmcgtmdwJmd5syuyuatdzCjGR3mrkFs7vYdix8\nJ/kR7G0tyd7Am4C/q6p/ntY4VXUt8G5gZ4/d/lfgiUk+BbwBeGSSv+2xfwCq6nPdf78A/BOjj6yG\nbPC5BbM7CbO7PZndlc1gbmEGsrtZuQWzu2A7Fr6T/Aj2tpUkwGnAx6vq5VPo/6AkB3bXbwf8GKN1\nQb2oqudX1V2q6m7ATwPvqqqn9dU/QJLbJ9m/u74v8GhgKt+a3USDzi2Y3UmY3e3J7K5sRnMLA8/u\ntHPbjWF2F9l2hW9V3Qws/Aj2x4G/r6reXqQkbwDOAe6V5Iokff+A9kMY/Y3yR3Q/vXF+kj7fXX0/\n8K4kFwIfYrRe56099r/YND5KOhh4X7fm6FzgLVX19imMs2mmnVswu+tgdidgdicy9OzOXG7BemFC\nZncR/4CFJEmSmrDtzvhKkiRJ02DhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+E5B\nkt9KclGSC7vf5TsmybO7H49e7bH/c5J2Ut/MrYbK7GqozO7m83d8e5bkwcCfAA+vqpuS3Am4LXA2\n8MCq+tIqj//UJO2kPplbDZXZ1VCZ3a3hGd/+fR/wxaq6CaCqvgz8FHBn4N1J/hUgyauSfLh7p7er\nu+9Xl2j36CTnJPm3JP/Q/ck+krwkyce6d4l/tOl7qVljbjVUZldDZXa3gGd8e9YF7f3A7YF3MvoT\niu/t3pn9UBdsktyxqq5JsqNr9ytVddF4uyQHAW8CdlbVjUlOAvYBXgmcU1X37vo6oKq+uuk7q5lh\nbjVUZldDZXa3hmd8e1ZV1wM/BDwD+ALw90lO7DZnrOmTk/wb8O/AfYCjluju2O7+c5KcDzwNuCtw\nLfD1JKcl+Qngxmnsi9phbjVUZldDZXa3xl5bPYFZVFV7gPcA70nyUeDEhU0ASe4G/BqjtTnXJnk1\no3U9S3lHVZ2w+M4kxwCPYvSxyLO669K6mVsNldnVUJndzecZ354luVeSI8buegCwG7gOOKC77wDg\neuCrSQ4GHjvWfrzducBDktyj63vfJEd0H48cWFX/D3gucP9p7Y/aYG41VGZXQ2V2t4ZnfPu3H/Dn\nSQ4EbgYuZfQxxgnA25J8pqoe1X0UcQlwBaM1PgtOXdTuROANSb6r2/5bjML+L0luy+jjkOdsxo5p\npplbDZXZ1VCZ3S3gl9skSZLUBJc6SJIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIk\nSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqC\nha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8k\nSZKaYOErzbgkFyV52FbPYy2SvDXJU6c8xq4kr+uu3zXJdUnSU9+vSvLb3fW5JFf00W/X30OTXNJX\nf31aLWtJ5pM8fcK+Jt7P7fycSNpe9trqCUjamCRfA6q7uS/wdeBb3e1nVNV/2ZKJbUBVPW4zhhkb\n73Jg/9UekORE4OlV9dAVO6767xue3bfH3APcs6o+2fX9PuDeffXfp/GsJdkF3KOqxt/AFGPP+yp9\nTbyfi9sm2Q38fFW9a5LHj0tyKvAw4Iiuj9eutQ9J25eFrzRwVbXfwvUkn2JUmK35H/ztYOGMa1VN\nVBxtR0luU1V7+u625/5mXbH+5+wC4HTgpUxYpEsaDpc6SDMuye4kj+yu70ryj0lel+SrST6S5Igk\nJye5OsnlSX5s7LF3SHJaks8muTLJi5IsedxIckyS85Jcm+SqJH8ytu3YJOckuSbJBUkePrZtPsnv\nJzkb+Bpw98UfiSf5+SQfT/LlJG9LctexbS/r5n5ttz/3WWZ+d0vynm6/3w4cNLbt8CR7FvYtyYlJ\n/rNr+8kkJyS5N/CXwIO7ZRFf7tq+plva8Nbu7PsjuvtetGj8k5N8IcmnkpywaP/H9/XEJO/rrr+3\nu/vCbswnLV46keTIro9ruqUGTxjb9pokr0zylm5fPpjk7mt57pI8IslHxm6/I8mHxm6/L8kTu+u7\nkzwqyU7gZODJ3bzPH+vy8CTv7+ZzVpLvXub1Wryfu5P8WpILk3wlyelJvmtx24yWr9wVOLMb+9eX\n6n85VfUX3RvHr6/lcZKGwcJXmn2Lz1o9Hvhb4I7A+cBZ3f13Bl4InDLW9jXAN4F7AA8AHg38wjLj\n/Cnwsqq6A3B34B8AkhwCvAV4YVXdEfh14E2LCp6ndP3uD3yasY/EkxzHqIj6CUbF6vuAN3TbHgM8\nFDiiG/dJwJeWmd//AT4MfDfwIuBnl3huSLJvty87q+oA4MHABVV1CfBM4ANVtX9V3WnsYccDL+rO\nvr+fW3+k/33duHfuxj01yRHdtmU//q+qhfWy9+vG/MdFc90bOBN4G/A9wK8Ar09yr7FmTwZ2MXq9\nLwP+oHvspM/dB4EjktypG+9+wPcn2TfJ7YAfYvSa3LIvVfU24H8Bp3fzfsDClIETgBOB7wX2YZSH\nSVQ3x8cAd+vmceKtGo2WVlwOPL4b+4+7/f1K9+ZgqcvzJpyDpIGz8JXa896qekdVfQt4I6OC6SXd\n7b9ndEbugCQHA48FnlNVN1bVF4CXAz+9TL/fZFQgHVRVN1TVud39TwHe2hVDVNU7gfOAH++2F/Ca\nqrq4qvZU1c2L+v0l4MVV9YluCcGLgaO7s77fZFQsH5nREoNPVNVViyfWtX0g8DtVdVO3JvRMlv84\nfA9w3yS3q6qrq+rjC10t0baAf66qD3T7941l2i6M/V7g/zIqSDfqWGDfqnpJVd1cVe9m9Cbj+LE2\nb66q87rX9/XA0d39NzHBc1dVNzJ6w/BwRkXuBcDZwI90419aVdcsMbdw6+eggL+pqsuq6uuM3hwd\nfatHLu/Pquqqbrwz1/LYqjqwqu64zOUP1zAHSQNm4Su15/Nj128Evji2pvbG7r/7AYcBewOfWzgz\nxuij/u9Zpt+nA/cCLk7yoSQLhe1hwJPGz7ABD2F0FnTBSr96cBjwp2OPXTgreeeu0HsF8Erg6iSn\nJFnqS2p3Bq7pirgFn15qsKq6nlFR+kvAZ7tlAj+wwvxWmz/LjP39qzxmEndeYuxPd/fDqNC8emzb\njYxeW7qP8yd57gDeA8wxOkP8nu7ycEZfAptf45zHi+tb5rMJj5UkC19Jy7oC+Abw3WNnxu5QVfdd\nqnF3Fu+EqvoeRl8MemOS2zP62Pl1i86w7b/oLNtKXyK6nNGvU4w/ft+q+mA37p9X1QOBoxgV3r+x\nRB+fA+7YzWfBYcuNW1Vvr6pHMyrOLwH+aoJ53qqbsetLjf3Z7vr1jH6NY8H4G4LVfBa4S/IdP8N2\nGPCZiSY42XMHo0L3EXy70F0ohB/eXV+y+0nmMCVLLWH5Wrfmd6nLb27FJCVtPgtfSUuqqs8Bbwf+\nd5L9k9wmyT2yzO+0JnlKkoWzwdcyKj6+Bfwd8IQkj06yI8ltuy8jHTL+8BWm8pfA85Mc1Y1zhyRP\n6q4/MMmDurWnN/CdP+U2vi+fZrS84veS7J3kRxitdV5qP743yXHdWt+bGBWmC31eDRzajbfS3Jf6\nmH9h7IcyWuaxsF73AuAnk9wuyT0ZnTkfdzWjNdZLOZfRfj+v63uu26/TV5jbwn5O9Nx1zgF+APhh\n4EPd0o/DgAcB713mMVcxWjazeA6b8QsVt3rOqmq/7g3XUpeX3DK50fN4W0b/Pu7T5dVf1ZBmhIWv\n1Jalvki10u2nMfoC0seBLzMq1pY7I/kY4KIk1wEvA366qr5RVVcCxwHPZ7TM4nLg1/jOAmjZs4NV\n9c+MziBehnUpAAABCUlEQVSfnuRa4KPdWAAHAKd2c9sNfBH4o2W6OoFRofZl4HeBxb/PujCH2wDP\nYXTW9EuMPt5f+F3efwU+BlyV5PNjj1vqORy/73PANYzO0L4OeGZV/Ue37WWM1ipfDbya0RuF8cfu\nAl7bLfX4qfG+q+qbwBMYrcX+AqOlC08d63ul13vi566qbgD+DfjY2Brsc4DdVfXFpR7Dtwv7LyU5\nb4nxl5vfUnNdbtvivha8GPjt7jl77gp9LOUdjN4IHMvo+bmBUQYkzYAM+OcyJUmSpIl5xleSJElN\nsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXh/wNH4pxuMPAx\nBwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10e1e10b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "init = 1\n",
    "ts = [5, 10, 50, 100]\n",
    "plot_time_series_dists(mc1, init, ts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start with state `2`,\n",
    "which is a transient state:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAE7CAYAAAA7EoKDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8bWVd7/HP1w2kcjeMAhS84AWPimWIx8itlm5N5VUn\nXwapUZZ0OpRHKwm7uNFK7XLsooewSM08YqkpeDigpssbyqW4iGwM0q2AijdEBS/g/p0/5lgyXazL\nXGuNudYa6/m8X6/5Ys45nvk8z5jruwe/Ocaz5kpVIUmSJG12d1rvCUiSJElrwcJXkiRJTbDwlSRJ\nUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfNdAkp1JHruC1+1K8vUkX+tur57G/KRZq8jq\nq5NcleQ7SX5hnu3PS/LZJDclOSPJHv3MWBqZ1nHW7Kpv63GcTXK3JP/SZX1nkuNWux9DZeG7NgrI\nCl/74Krau7s9p89JSfNYaVYvBX4N+Peuj+9K8gTgZOCxwKHAvYFTVzdN6Q56P86aXU3JehxnXwV8\nE/gB4OeB05IcsYI5DJ6F75QleT1wT+Ds7mzCby2zC39GWhOryWpV/e+qeg+jA+tcvwD8XVXtqKqv\nAC8GTuhjzhJM9ThrdtWr9TjOJtkT+Bng96vqlqr6EPB24Jmr25thsqiasqp6JvBp4Mnd2YQ/S/KV\nJDcucHvBnC7e3126eEuSQ9dhF9SIHrK6kCOAy8YeXw4cmGT/vvdBbZricdbsqlfrdJy9H3BbVV0z\ntv0y4EF97NPQ7LbeE2hRVe03YdMfBz4C7An8IfCOJEdW1XemNjlpzDKyupi9gJvGHn+1++/ewI09\n9C/dwSqPsw+tql2YXa2BNTjO7jX2eNbXum3N8YzvBlZVH6yq26rqJuC5wGHAA9Z3VtKyfR3YZ+zx\nvt1/v7YOc5G+xwLH2Qd2m82uhmKxrM7dNru9yRxb+K6NuYvQx3+DeO7tdxboI3P+K01DH1md62PA\nkWOPHwrcUFWeMVOfpnGcNbuahrU+zv4HsFuS+87ZfsVqdmKoXOqwNm4A7gO8B6Cq9lrqBd1vW+4B\nfBS4C/BHwHXAjulNU1p+VgGS7A5sYfRheo8kdwa+VVUF/APw2iRvAD4H/D7wminMXW2bxnHW7Goa\n1vQ4W1U3J3kr8OIkvwz8MPAU4JG97tVAeMZ3bbwU+L1uofrzJ3zNgcCZjNbs/CdwD0aL4V3fq2la\nSVYB3gXcAhwNvLq7fwxAVZ0H/AnwXmAnozy/qMc5SzCF46zZ1ZSsx3H21xh9uPs88I/Ar1ZVkyfS\nMvqgsEiD5O+BnwI+X1UPXqDNXwFPZPRDOKGqLul7otJymV0NkbnVUJldDcEkZ3xfA2xbaGOSJwH3\nrarDgecAp/U0N2m1zK6GyNxqqMyuNrwlC9+q+gCLf23LU4HXdW0vAPZLcmA/05NWzuxqiMythsrs\nagj6WON7MHDt2OPrgEN66FeaNrOrITK3Giqzq3XX17c6zP2KrTssHE6y+GJiaUJV1edXupldrQlz\nq6Eyuxqq+bLbxxnf6xn9JuysQ7rn5pvAxLcXvehFy2q/3Nu0+98M+7AR+++Z2V2H/jfDPiy3f3O7\nMX8urfW/kjHM7sb8udj/0reF9FH4ngU8CyDJ0cBXquqGHvqVps3saojMrYbK7GrdLbnUIckbgUcD\nByS5ltH3wu0OUFWnV9U5SZ6U5BrgZuAXpzlhaVJmV0NkbjVUZldDsGThW1XHTdDmpH6mc7utW7f2\n3eWa9r8WY9j/4szuxux/LcYYcv+bNbdrMYb9r+8YmzW7Q/+52P/3WvIPWPQ2UFJrNZY2ryRUv79o\nMcmYZlerYm41VOuZ3aT/Yf030Y6FstvXtzpIkiT1rM9CdU3rd21QffxymyRJkrThWfhKkiSpCS51\nkKRNru+1kq6TlDRUFr6S1IS+ilXXSUoaLpc6SJIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIk\nqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkW\nvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIk\nSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWrC\nkoVvkm1JrkpydZKT59m+b5Kzk1ya5IokJ0xlptIymV0NldnVEJlbDUGqauGNyRbg48BPANcDFwHH\nVdWOsTYvBPauqlOSHNC1P7CqbpvTVy02ljSJJFRVJmhndrVhTJrbrm0v2R3PbRKgrwwH/z20Yz2P\nuf3mFsxuWxbK7lJnfI8CrqmqnVV1K3AmcOycNruAfbr7+wBfmhtiaR2YXQ2V2dUQmVsNwlKF78HA\ntWOPr+ueG/dK4IgknwEuA57b3/SkFTO7GiqzqyEytxqE3ZbYPsk1gW3Av1fVY5LcB3hXkodW1dfm\nNty+fft372/dupWtW7cuY6pq0czMDDMzMyt5ae/ZPfXUU1cyj4Un6CW3TWsVuYUeszt+zIUZYOtK\n56RGbKRjbnePUW63rmROasik2V1qje/RwPaq2tY9PgXYVVUvH2vzDuClVfWh7vG/AidX1cVz+nKd\npFZtGevNes+u6yS1Ustc49tLdl3jqz5snmMumN22rHSN78XA4UkOS7IH8HTgrDltPs1oMTtJDgTu\nD3xi9VOWVsXsaqjMrobI3GoQFl3qUFW3JTkJOA/YApxRVTuSnNhtPx14CfDaJJcDAV5QVV+e8ryl\nRZldDZXZ1RCZWw3Foksdeh3IpQ7qwXIuGfc4pksdtCrrmdvZ8c2uVmLzHHPB7LZlpUsdJEmSpE3B\nwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleS\nJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElN\nsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCV\nJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElS\nE5YsfJNsS3JVkquTnLxAm61JLklyRZKZ3mcprYDZ1VCZXQ2RudUQpKoW3phsAT4O/ARwPXARcFxV\n7Rhrsx/wIeAJVXVdkgOq6ovz9FWLjSVNIglVlQna9Z7dJEBfGQ7+e2jHpLnt2vaS3fFjrtnVSm2e\nYy6Y3bYslN2lzvgeBVxTVTur6lbgTODYOW2OB95SVdcBzBdiaR2YXQ2V2dUQmVsNwlKF78HAtWOP\nr+ueG3c4cLck701ycZJn9jlBaYXMrobK7GqIzK0GYbcltk9yTWB34IeBxwF3BT6c5CNVdfXchtu3\nb+fUU09d/iyX4KWLzWtmZoaZmZmVvLT37Hb3gK3dTZrfKnILPWb39twCzGButRSPuRqqSbO71Brf\no4HtVbWte3wKsKuqXj7W5mTgLlW1vXv8d8C5VfXmOX25Zkertoz1Zhs8u+a2Jctc49tLdl3jqz5s\nnmMumN22rHSN78XA4UkOS7IH8HTgrDlt3g78WJItSe4KPAK4so9JS6tgdjVUZldDZG41CIsudaiq\n25KcBJwHbAHOqKodSU7stp9eVVclORe4HNgF/G1VGWStK7OroTK7GiJzq6FYdKlDrwN56UI9WM4l\n4x7HdKmDVmU9czs7vtnVSmyeYy6Y3basdKmDJEmStClY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mS\npCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY\n+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqS\nJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJ\nFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmrBk4ZtkW5Krklyd5ORF2v1o\nktuS/Ey/U5RWxuxqqMyuhsjcaggWLXyTbAFeCWwDjgCOS/LABdq9HDgXyBTmKS2L2dVQmV0NkbnV\nUCx1xvco4Jqq2llVtwJnAsfO0+7XgTcDX+h5ftJKmV0NldnVEJlbDcJShe/BwLVjj6/rnvuuJAcz\nCvdp3VPV2+yklTO7GiqzqyEytxqE3ZbYPkko/wL4naqqJGGRSxfbt2+fvQds7W7SwmZmZpiZmVnJ\nS82u1s0qcgs9Zvf23ALMYG61FI+5GqpJs5uqhbOa5Ghge1Vt6x6fAuyqqpePtfkEt4f3AOAW4Feq\n6qw5fVVVMcp6nx/ywmL7oM0lCVW15LqwjZ9dc9uSSXPbte0lu7O5nR3f7GolNs8xF8xuWxbK7lKF\n727Ax4HHAZ8BLgSOq6odC7R/DXB2Vb11nm0GWau2jIPwBs+uuW3JMgvfXrJr4as+bJ5jLpjdtiyU\n3UWXOlTVbUlOAs4DtgBnVNWOJCd220+fymylVTK7GiqzqyEytxqKRc/49jqQn+DUg+WcOetxTM/4\nalXWM7ez45tdrcTmOeaC2W3LQtn1L7dJkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+\nkiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJ\naoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKF\nryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJ\nkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmTFT4JtmW5KokVyc5eZ7tP5/ksiSX\nJ/lQkof0P1VpecythsrsaqjMrja6VNXiDZItwMeBnwCuBy4CjquqHWNtHglcWVU3JdkGbK+qo+f0\nU1VFEmDxMZe5Cyy1D9o8klBVmaBdL7nt2k0hu+a2JZPmtmvb6zF3dnyzq5VYz+xaL2g1FsruJGd8\njwKuqaqdVXUrcCZw7HiDqvpwVd3UPbwAOGS1E5ZWydxqqMyuhsrsasObpPA9GLh27PF13XMLeTZw\nzmomJfXA3GqozK6Gyuxqw9ttgjYTXxdI8hjgl4BHzbd9+/bts/eArd1teEaXX/oz99JL3/3PN8ZQ\nzMzMMDMzs5KX9pZb2DzZ1dpYRW5hKsdcgBnMrZay8bK7HY+5msSk2Z1kje/RjNbgbOsenwLsqqqX\nz2n3EOCtwLaqumaefjbNmp1pr5fbDO/RtCxjjW8vue3auMZXq7LMdZK9HnNnxze7Won1zK7/L9Rq\nrGaN78XA4UkOS7IH8HTgrDmd35NRiJ+xUPEgrTFzq6Eyuxoqs6sNb8mlDlV1W5KTgPOALcAZVbUj\nyYnd9tOBPwD2B07rLtPfWlVHTW/a0uLMrYbK7GqozK6GYMmlDr0NtIkuXbjUYf0s57Jbj2O61EGr\nsp65nR3f7GolNs8xF8xuW1az1EGSJEkaPAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIk\nNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHC\nV5IkSU2w8JUkSVITLHwlSZLUhN3WewKS1k6SXvurqjXtX5Kk1bDwlZrTVzG5UJE77f4lSVoZlzpI\nkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSp\nCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmLFn4JtmW5Kok\nVyc5eYE2f9VtvyzJw/qZ2kw/3SzU+8x0++9GGXT/036Ppt2/2V3xCFPuf/pjmN2VmFl9F0uNsML3\nLUnvt2mPsZbvz0YZw2Puxh3D/m+3aOGbZAvwSmAbcARwXJIHzmnzJOC+VXU48BzgtH6mNtNPNwv1\nbuG7dO8DCvJcZndVI0y5/+mPYXZXYmb1XSw1wqret5rg9qIJ2610jNX2v7ghF1geczf2GPZ/u6XO\n+B4FXFNVO6vqVuBM4Ng5bZ4KvA6gqi4A9ktyYG8zlFbG7GqozK6GyNzOY9IrBKeeeupUr1RM2v9K\nr1YMyVKF78HAtWOPr+ueW6rNIaufmrQqZldDZXY1ROZ2QX1drZh2/yu/WjEoVbXgDfhvwN+OPX4G\n8Ndz2pwNPGrs8buBH56nr0necW/elrwtllmz622j3ibJbZ/ZXe/99bZ5bh5zvQ31Nl9Wd2Nx1wP3\nGHt8D0af0BZrc0j33Peoqs1//lwbidnVUPWSXXOrNeYxV4Ow1FKHi4HDkxyWZA/g6cBZc9qcBTwL\nIMnRwFeq6obeZyotj9nVUJldDZG51SAsesa3qm5LchJwHrAFOKOqdiQ5sdt+elWdk+RJSa4BbgZ+\nceqzlpZgdjVUZldDZG41FOnW00iSJEmb2ob8y22TfAn2Kvr++yQ3JPlon/2O9X+PJO9N8rEkVyT5\njZ77v3OSC5Jc2vW/vc/+x8bZkuSSJGdPqf+dSS7vxrhwGmOstWnmtuvf7E42jtldJrO7ZP+Dz+5m\nzC1YL0wwhtmda9LfNF6rG6NLJNcAhwG7A5cCD+yx/2OAhwEfndL8fxA4sru/F/DxPuff9XvX7r+7\nAR8BHjGF/Xg+8AbgrCm9T58E7raW2Zrmbdq57cYwu5ONYXaXtz9md7IxBp3dzZbbbp+sFyYbx+yO\n3TbiGd9JvgR7xarqA8CNffU3T/+fq6pLu/tfB3YAB/U8xi3d3T0Y/WPf1Wf/SQ4BngT8HTDN367d\nTL+5O9XcgtmdhNldEbM72RibIbubKbdgvTDpOGZ3zEYsfCf5EuxBSHIYo0+LF/Tc752SXArcALyz\nqi7qs3/gFcBv0/M/jjkKeHeSi5P8yhTHWSubJrdgdpdgdjcws7ugzZZb2ETZnVZuu77N7piNWPhu\nit+2S7IX8Gbgud0nud5U1a6qOpLRdyA+IsmD+uo7yZOBz1fVJUz37MCjquphwBOB/5HkmCmOtRY2\nRW7B7E7A7G5QZndRmy23sEmyO83cgtmdayMWvpN8CfaGlmR34C3AP1bV26Y1TlXdBLwX2NZjt/8V\neGqSTwJvBB6b5B967B+Aqvps998vAP/C6JLVkA0+t2B2J2F2Nyazu7hNmFvYBNldq9yC2Z21EQvf\nSb4Ee8NKEuAM4Mqq+osp9H9Akv26+3cBfpLRuqBeVNULq+oeVXUv4OeA91TVs/rqHyDJXZPs3d3f\nE3g8MJXfml1Dg84tmN1JmN2NyewubpPmFgae3WnnthvD7M6x4QrfqroNmP0S7CuBN1VVbz+kJG8E\nzgful+TaJH1/gfajGP2N8sd0X71xSZI+P139EPCeJJcBFzJar3NOj/3PNY1LSQcCH+jWHF0AvKOq\n3jmFcdbMtHMLZncFzO4EzO5Ehp7dTZdbsF6YkNmdwz9gIUmSpCZsuDO+kiRJ0jRY+EqSJKkJFr6S\nJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr5TkOR3k1yR5LLue/mOSvLc7sujl3rt/5ykndQ3c6uh\nMrsaKrO79vwe354leSTw58Cjq+rWJHcD7gx8CHh4VX1pidd/cpJ2Up/MrYbK7GqozO768Ixv/34Q\n+GJV3QpQVV8GfhY4CHhvkn8FSHJakou6T3rbu+d+Y552j09yfpJ/S/JP3Z/sI8nLknys+5T4p2u+\nl9pszK2GyuxqqMzuOvCMb8+6oH0QuCvwbkZ/QvH93SezH+mCTZL9q+rGJFu6dr9eVVeMt0tyAPAW\nYFtVfSPJycAewKuA86vqAV1f+1TVV9d8Z7VpmFsNldnVUJnd9eEZ355V1c3AjwDPAb4AvCnJCd3m\njDV9epJ/A/4deBBwxDzdHd09f36SS4BnAfcEbgK+meSMJD8NfGMa+6J2mFsNldnVUJnd9bHbek9g\nM6qqXcD7gPcl+ShwwuwmgCT3An6T0dqcm5K8htG6nvm8q6qOn/tkkqOAxzG6LHJSd19aMXOroTK7\nGiqzu/Y849uzJPdLcvjYUw8DdgJfA/bpntsHuBn4apIDgSeOtR9vdwHwqCT36freM8nh3eWR/arq\n/wHPBx46rf1RG8ythsrsaqjM7vrwjG//9gL+Osl+wG3A1YwuYxwPnJvk+qp6XHcp4irgWkZrfGa9\nek67E4A3Jvm+bvvvMgr725PcmdHlkOetxY5pUzO3Giqzq6Eyu+vAX26TJElSE1zqIEmSpCZY+EqS\nJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJ\nFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6S\nJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvtMkluSLJj6/3PJYjyTlJnjnlMbYneX13\n/55JvpYkPfV9WpLf6+5vTXJtH/12/R2T5Kq++uvTUllLMpPk2RP2NfF+buT3RNLGstt6T0DS6iT5\nOlDdwz2BbwLf6R4/p6r+y7pMbBWq6klrMczYeJ8G9l7qBUlOAJ5dVccs2nHVf1/17G4fcxdw36r6\nRNf3B4AH9NV/n8azlmQ7cJ+qGv8AU4y970v0NfF+zm2bZCfwS1X1nkleP/a6+wF/CjwS2AJcBPxG\nVf3HcvqRtHF5xlcauKraq6r2rqq9gU8BT559XFVvXO/5LUc66z2P1UgyjePqoN+TdVCs7D3bF3gb\ncD/gQOBC4O09zkvSOrPwlTa5JDuTPLa7vz3JPyd5fZKvJrk8yeFJTklyQ5JPJ/nJsdfum+SMJJ9J\ncl2SlyxU2CU5KsnFSW5K8rkkfz627egk5ye5McmlSR49tm0myR8m+RDwdeDecy+JJ/mlJFcm+XKS\nc5Pcc2zbK7q539Ttz4MWmN+9kryv2+93AgeMbTssya7ZfUtyQpL/7Np+IsnxSR4A/A3wyG5ZxJe7\ntq/tljac0519f0z33EvmjH9Kki8k+WSS4+fs//i+npDkA93993dPX9aN+bS5SyeSPLDr48ZuqcFT\nxra9Nsmrkryj25ePJLn3ct67JI9JcvnY43cluXDs8QeSPLW7vzPJ45JsA04Bnt7N+5KxLg9L8sFu\nPucl+f4Ffl5z93Nnkt9MclmSryQ5M8n3zW2b0fKVewJnd2P/1nz9z6eqLqqq11TVV6rqNuAvgPsn\n2X/SPiRtbBa+0uY399Lyk4F/APYHLgHO654/CHgxcPpY29cC3wbuAzwMeDzwywuM85fAK6pqX+De\nwD8BJDkYeAfw4qraH/gt4C1zCp5ndP3OnrX+7iXxJMcyKqJ+mlGx+gHgjd22JwDHAId34z4N+NIC\n8/s/jC5dfz/wEuAX5nlvSLJnty/bqmofRpe9L62qq4ATgQ93Z9PvNvay44CXVNVewAe54yX9H+zG\nPagb99VJDu+2LXj5v6pm18s+pBvzn+fMdXfgbOBc4O7ArwNvyOiS/aynA9sZ/byvAf6oe+2k791H\ngMOT3K0b7yHADyXZM8ldgB9h9DP57r5U1bnAHwNndvN+2OyUgeOBE4AfAPZglIdJVDfHJwD36uZx\nwh0ajZZWfJrbr3z8Wbe/X+k+HMx3e8ECY/448NmqunHCOUra4Cx8pfa8v6reVVXfAd7MqGB6Wff4\nTYzOyO2T5EDgicDzquobVfUFRmfAfm6Bfr/NqEA6oKpuqaoLuuefAZzTFUNU1buBi4Gf6rYX8Nqq\n2lFVu7ozbeN+FXhpVX28qnYBLwWO7M76fptRsfzAJHfq2nxu7sS6tg8Hfr+qbu3WhJ7NwpfDdwEP\nTnKXqrqhqq6c7WqetgW8rao+3O3ftxZoOzv2+4H/y6ggXa2jgT2r6mVVdVtVvZfRh4zjxtq8taou\n7n6+bwCO7J6/lQneu6r6BqMPDI9mVOReCnwI+LFu/KsXKAzDHd+DAv6+qq6pqm8y+nB05B1eubC/\nqqrPdeOdvZzXVtV+VbX/Arc/ucPkk0OAVwLPX8b8JG1wFr5Sez4/dv8bwBerqsYeA+wFHArsDnx2\n9swYo0v9d1+g32czWhu5I8mFSWYL20OBp42fYQMexegs6KzFvvXgUOAvx147e1byoK7QeyXwKuCG\nJKcnme+X1A4CbuyKuFmfmm+wqrqZUVH6q8BnumUC919kfkvNnwXG/qElXjOJg+YZ+1Pd8zAqNG8Y\n2/YNRj9bul/8muS9A3gfsJXRGeL3dbdHMzojOrPMOY8X19+dzxq8dmJJ7g68E3hVVb1pGmNIWh8W\nvpIWci3wLeD7x86M7VtVD56vcXcW7/iqujvwcuDNSe7K6LLz6+ecYdt7zlm2xX7T/9OMvp1i/PV7\nVtVHunH/uqoeDhzBqPD+7Xn6+CywfzefWYcuNG5VvbOqHs+oOL8K+NsJ5nmHbsbuzzf2Z7r7NzP6\nNo5Z4x8IlvIZ4B7J9/xC4KHA9RNNcLL3DkaF7mO4vdCdLYQf3d2ft/tJ5jAl8y1h+Xq35ne+2++M\ntdufUdH7tqp66VpOWtL0WfhKmldVfZZRAfC/kuyd5E5J7pMFvqc1yTO6M2UANzEqPr4D/CPwlCSP\nT7IlyZ27X0Y6ePzli0zlb4AXJjmiG2ffJE/r7j88ySO6tae38L1f5Ta+L59itLzi1CS7J/kxRmud\n59uPH0hybLfW91ZGhelsnzcAh3TjLTb3+S7zz459DKNlHrPrdS8FfibJXZLcl9GZ83E3MFpjPZ8L\nGO33C7oSnqGzAAABjUlEQVS+t3b7deYic5vdz4neu875wP2BHwUu7JZ+HAo8Anj/Aq/5HKNlM3Pn\nsBbfUHGH92z820/mub0MIMk+jNa8f7CqXrgG85S0xix8pbbM94tUiz1+FqNfQLoS+DKjYm2hM5JP\nAK5I8jXgFcDPVdW3quo64FjghYyWWXwa+E2+twBa8OxgVb2N0RnkM5PcBHy0GwtgH+DV3dx2Al9k\n9D2s8zmeUaH2ZeAPgNctsN93Ap7H6Kzplxhd3p/9Xt5/BT4GfC7J58deN997OP7cZ4EbGZ2hfT1w\n4th3w76C0VrlG4DXMPqgMP7a7cDruqUePzved1V9G3gKo7XYX2C0dOGZY30v9vOe+L2rqluAfwM+\nNrYG+3xgZ1V9cb7XcHth/6UkF88z/kLzm2+uC22b29eslwK/171ny1mj+9OM1oL/4tjZ4K92630l\nbQK5fWmfJEmStHl5xleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS\n1AQLX0mSJDXh/wPZf0llxApKgAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10e37f0f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "init = 2\n",
    "ts = [5, 10, 50, 100]\n",
    "plot_time_series_dists(mc1, init, ts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the above cell several times;\n",
    "you will observe that the limit distribution differs across sample paths.\n",
    "Sometimes the state is absorbed into the recurrent class $\\{0\\}$,\n",
    "while other times it is absorbed into the recurrent class $\\{1,4\\}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Some sample path with init=2\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAE7CAYAAAA7EoKDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYbHdd5/H3h5tEIDsGoySQsIQlDJtiCINIAwoXBPLo\nyIOJgFEUHAdlQCUGlzSgAi6DCwwGjYDIEBQQEoYhgNDsBKJJIJBgIlxIWMIWwg4J9zt/1Omk0uml\nuvtUV5/+vV/PU8+tqnPO7/c7VZ8+91vn/Lo6VYUkSZK0091k1gOQJEmStoKFryRJkppg4StJkqQm\nWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhugSR7kjxoA9vtTfL1JF/rbi+exvikRZvI6ouT\nXJLke0l+YZnlT03y2SRXJzkjyX79jFgamdZx1uyqb7M4zia5RZJ/6bK+J8mJm92PobLw3RoFZIPb\n3q2qDuxuT+xzUNIyNprVC4BfA/69a+M6SR4KnAI8CDgKuB3wzM0NU7qR3o+zZldTMovj7AuBbwM/\nAPw88KIkx25gDINn4TtlSV4O3AY4uzub8FvrbML3SFtiM1mtqv9dVW9jdGBd6heAv6uqi6vqK8Cz\ngJP7GLMEUz3Oml31ahbH2ST7Az8D/H5VfbOq3gO8Hnjc5vZmmCyqpqyqHgd8CnhEdzbhz5J8JclV\nK9yevqSJd3aXLl6T5KgZ7IIa0UNWV3IscOHY4w8Bhyc5tO99UJumeJw1u+rVjI6zdwSurarLxpZf\nCNy1j30amn1mPYAWVdUhE67648D7gf2BPwTekOSeVfW9qQ1OGrOOrK7mAODqscdf7f49ELiqh/al\nG9nkcfYeVbUXs6stsAXH2QPGHi/6WresOZ7x3caq6t1VdW1VXQ08BTgauPNsRyWt29eBg8YeH9z9\n+7UZjEW6gRWOs3fpFptdDcVqWV26bHF5kzm28N0aSyehj/8G8dLb76zQRpb8K01DH1ld6iPAPcce\n3wO4sqo8Y6Y+TeM4a3Y1DVt9nP0PYJ8kd1iy/KLN7MRQOdVha1wJ3B54G0BVHbDWBt1vW+4HfBi4\nGfBHwBXAxdMbprT+rAIk2RfYxejD9H5Jbgp8p6oK+AfgpUleAXwO+H3gJVMYu9o2jeOs2dU0bOlx\ntqq+keS1wLOS/DLww8Ajgfv2ulcD4RnfrfEc4Pe6iepPm3Cbw4EzGc3Z+U/g1owmwzu/V9O0kawC\nvAX4JnA88OLu/v0Bquoc4E+AtwN7GOX5tB7HLMEUjrNmV1Myi+PsrzH6cPd54B+BX62qJk+kZfRB\nYZUVkr8Hfgr4fFXdbYV1/gp4GKM34eSqOr/vgUrrZXY1ROZWQ2V2NQSTnPF9CbB7pYVJHg7coaqO\nAZ4IvKinsUmbZXY1ROZWQ2V2te2tWfhW1btY/WtbHgW8rFv3XOCQJIf3Mzxp48yuhsjcaqjMroag\njzm+RwCXjz2+Ajiyh3alaTO7GiJzq6Eyu5q5vr7VYelXbN1o4nCS1ScTSxOqqj6/0s3sakuYWw2V\n2dVQLZfdPs74fprRb8IuOrJ7brkBTHw77bTT1rX+em/Tbn8n7MN2bL9nZncG7e+EfVhv++Z2e74v\nrbW/kT7M7vZ8X2x/7dtK+ih8zwIeD5DkeOArVXVlD+1K02Z2NUTmVkNldjVza051SPJK4AHAYUku\nZ/S9cPsCVNXpVfXGJA9PchnwDeAXpzlgaVJmV0NkbjVUZldDsGbhW1UnTrDOk/sZzvXm5ub6bnJL\n29+KPmx/dWZ3e7a/FX0Muf2dmtut6MP2Z9vHTs3u0N8X27+hNf+ARW8dJbVVfWnnSkL1+4sWk/Rp\ndrUp5lZDZXY1VCtl1z9ZLEmSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCb09SeLJUmSepP0\n/2USfluELHwlSdI21WehuqXfyqZtyqkOkiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKF\nryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkprgnyyWpB0u6fdPtVb1+WdkJWnrWPhKUhP6\nKlb7LaIlaSs51UGSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQL\nX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mS\nJDVhzcI3ye4klyS5NMkpyyw/OMnZSS5IclGSk6cyUmmdzK6GyuxqiMythiBVtfLCZBfwMeAngE8D\nHwROrKqLx9Z5BnBgVZ2a5LBu/cOr6tolbdVqfUmTSEJVZYL1zK62jUlz263bS3bHc5sE6CvDwZ+H\ndszymNtvbsHstmWl7K51xvc44LKq2lNV1wBnAicsWWcvcFB3/yDgS0tDLM2A2dVQmV0NkbnVIKxV\n+B4BXD72+IruuXEvAI5N8hngQuAp/Q1P2jCzq6Eyuxoic6tB2GeN5ZNcE9gN/HtVPTDJ7YG3JLlH\nVX1t6Yrz8/PX3Z+bm2Nubm4dQ1WLFhYWWFhY2MimZlczs4ncQo/ZHc8tLABzGx2TGrH9jrnzjHI7\nt5ExqSGTZnetOb7HA/NVtbt7fCqwt6qeN7bOG4DnVNV7usf/CpxSVectact5ktq0dcw3M7vaNtY5\nx7eX7DrHV32Y5THXOb7ajI3O8T0POCbJ0Un2Ax4DnLVknU8xmsxOksOBOwEf3/yQpU0xuxoqs6sh\nMrcahFWnOlTVtUmeDJwD7ALOqKqLkzypW3468GzgpUk+BAR4elV9ecrjllZldjVUZldDZG41FKtO\ndei1Iy8XqwfruWTcY59mV5sy69w61UEbNcvsOtVBm7HRqQ6SJEnSjmDhK0mSpCZY+EqSJKkJFr6S\nJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElq\ngoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWv\nJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmS\nmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJaxa+SXYnuSTJpUlOWWGd\nuSTnJ7koyULvo5Q2wOxqqMyuhsjcaghSVSsvTHYBHwN+Avg08EHgxKq6eGydQ4D3AA+tqiuSHFZV\nX1ymrVqtL2kSSaiqTLCe2dW2MWluu3V7ye54bpMAfWU4+PPQjlkec/vNLZjdtqyU3bXO+B4HXFZV\ne6rqGuBM4IQl65wEvKaqrgBYLsTSDJhdDZXZ1RCZWw3CWoXvEcDlY4+v6J4bdwxwiyRvT3Jeksf1\nOUBpg8yuhsrsaojMrQZhnzWWT3JNYF/gh4EHAzcH3pfk/VV16dIV5+fnr7s/NzfH3NzcxANVmxYW\nFlhYWNjIpmZXM7OJ3EKP2R3PLSwAcxsdkxqx/Y6584xyO7eRMakhk2Z3rTm+xwPzVbW7e3wqsLeq\nnje2zinAzapqvnv8d8CbqurVS9pynqQ2bR3zzcyuto11zvHtJbvO8VUfZnnMdY6vNmOjc3zPA45J\ncnSS/YDHAGctWef1wI8l2ZXk5sB9gI/2MWhpE8yuhsrsaojMrQZh1akOVXVtkicD5wC7gDOq6uIk\nT+qWn15VlyR5E/AhYC/wt1VlkDVTZldDZXY1ROZWQ7HqVIdeO/JysXqwnkvGPfZpdrUps86tUx20\nUbPMrlMdtBkbneogSZIk7QgWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKa\nYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOEr\nSZKkJlj4SpIkqQkWvpIkSWrCPrMegDQUSXptr6p6bU+SJK3Owldal76K1X6LaEmStDanOkiSJKkJ\nFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6S\nJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCasWfgm2Z3kkiSXJjll\nlfV+NMm1SX6m3yFKG2N2NVRmV0NkbjUEqxa+SXYBLwB2A8cCJya5ywrrPQ94E5ApjFNaF7OroTK7\nGiJzq6FY64zvccBlVbWnqq4BzgROWGa9XwdeDXyh5/FJG2V2NVRmV0NkbjUIaxW+RwCXjz2+onvu\nOkmOYBTuF3VPVW+jkzbO7GqozK6GyNxqEPZZY/kkofwL4HeqqpKEVS5dzM/PX3d/bm6Oubm5CZpv\nz+hl7FfV1h5f+t6H0047bb2bTCm788Bcd5OWt7CwwMLCwkY37y2748dcWMDcai2byK7HXM3UpNnN\nagVRkuOB+ara3T0+FdhbVc8bW+fjXB/ew4BvAr9SVWctaau2uvgaqtHxoM/XKjMqfPvq8/rxJ6Gq\n1qyqp5Hdae2Tdr5Jc9ut20t2x4+5ZlcbtXOOuWB227JSdtcqfPcBPgY8GPgM8AHgxKq6eIX1XwKc\nXVWvXWaZhe+EdsIP+zYofHvPrsWDNmqdhW8v2bXwVR92zjEXzG5bVsruqlMdquraJE8GzgF2AWdU\n1cVJntQtP30qo5U2yexqqMyuhsjcaihWPePba0ee8Z3YTviUO+szvn3yjK82a5a5Xezf7Gojds4x\nF8xuW1bKrn+5TZIkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtf\nSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIk\nNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHC\nV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5Ik\nSU2w8JUkSVITLHwlSZLUBAtfSZIkNWGiwjfJ7iSXJLk0ySnLLP/5JBcm+VCS9yS5e/9DldbH3Gqo\nzK6Gyuxqu0tVrb5Csgv4GPATwKeBDwInVtXFY+vcF/hoVV2dZDcwX1XHL2mn1upLI0mAPl+rsNWv\nfb/7cP34k1BVmaD/XnLbrVdVNbV90s43aW67dXs/5ppdbdQss7sT/i/U7KyU3UnO+B4HXFZVe6rq\nGuBM4ITxFarqfVV1dffwXODIzQ5Y2iRzq6Eyuxoqs6ttb5LC9wjg8rHHV3TPreQJwBs3MyipB+ZW\nQ2V2NVRmV9vePhOsM/F1gSQPBH4JuN9yy+fn56+7Pzc3x9zc3KRNbyujyy/98dLLahaAG2ZnQr3l\n9ob9zwNz3U1LDf1no+/xA5x22mnr3WQqx9zRz9LceseixiwsLLCwsLDRzaeQ3Xk85moSk2Z3kjm+\nxzOag7O7e3wqsLeqnrdkvbsDrwV2V9Vly7SzY+b4Tnu+3E6Y17QN5vj2kttuHef4Tmjor9G0fvbW\nOU+y92Pu0N8Xzc4ss7sT/i/U7Gxmju95wDFJjk6yH/AY4Kwljd+GUYgfu1LxIG0xc6uhMrsaKrOr\nbW/NqQ5VdW2SJwPnALuAM6rq4iRP6pafDvwBcCjwou5S4TVVddz0hi2tztxqqMyuhsrsagjWnOrQ\nW0dOdVipNac6rN3auqc69MmpDpMb+mu0HaY69NazUx3Ug51zzAWz25bNTHWQJEmSBs/CV5IkSU2w\n8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUk\nSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNWGfWQ9A0tZJ0mt7VdVr\ne5IkTZOFr9ScvorVfotoSZKmzakOkiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJ\nkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQm7DPrAUiSNGtJem2vqnptby1DH7+0\nVSx8JUkCoK9ir98idHJDH780fU51kCRJUhMsfCVJktQEC19JkiQ1Yc3CN8nuJJckuTTJKSus81fd\n8guT3KuPgS0sLPTRzMza73oZdPvTf42m2/6ssuv7Mvs+hv4azSa7C5tvYq0eNvi+JOn9tsE92OB2\nE7Y+8P+XPOZufR/b5WdjM6/RVo9/1cI3yS7gBcBu4FjgxCR3WbLOw4E7VNUxwBOBF21058dZ+M6+\n/SEXD7PMru/L7PsY8ms0u+wubL6JtXrY1PtSE9xOm3C9jVrYxLYTtD7g/5c85s6yj9n/bGz+Ndq6\n8a91xvc44LKq2lNV1wBnAicsWedRwMsAqupc4JAkh0/UuzQ9ZldDZXY1ROZ2GZOepXzmM585xSsV\nGrdW4XsEcPnY4yu659Za58jND03aFLOroTK7GiJzu6K+zsiqF1W14g34b8Dfjj1+LPDXS9Y5G7jf\n2OO3Aj+8TFuTvPPevK15Wy2zZtfbdr1Nkts+szvr/fW2c24ec70N9bZcVtf6AxafBm499vjWjD6h\nrbbOkd1zN1BVnqPXVjK7GqpesmtutcU85moQ1prqcB5wTJKjk+wHPAY4a8k6ZwGPB0hyPPCVqrqy\n95FK62N2NVRmV0NkbjUIq57xraprkzwZOAfYBZxRVRcneVK3/PSqemOShye5DPgG8ItTH7W0BrOr\noTK7GiJzq6FIN59GkiRJ2tG25V9um+RLsDfR9t8nuTLJh/tsd6z9Wyd5e5KPJLkoyW/03P5Nk5yb\n5IKu/fk+2x/rZ1eS85OcPaX29yT5UNfHB6bRx1abZm679s3uZP2Y3XUyu2u2P/js7sTcgvXCBH2Y\n3aUm/U3jrboxukRyGXA0sC9wAXCXHtu/P3Av4MNTGv8PAvfs7h8AfKzP8Xft3rz7dx/g/cB9prAf\nTwNeAZw1pdfpE8AttjJb07xNO7ddH2Z3sj7M7vr2x+xO1segs7vTctvtk/XCZP2Y3bHbdjzjO8mX\nYG9YVb0LuKqv9pZp/3NVdUF3/+vAxcCteu7jm93d/Rj9sO/ts/0kRwIPB/4OmOZv1+6k39ydam7B\n7E7C7G6I2Z2sj52Q3Z2UW7BemLQfsztmOxa+k3wJ9iAkOZrRp8Vze273JkkuAK4E3lxVH+yzfeD5\nwG/T8w/HEgW8Ncl5SX5liv1slR2TWzC7azC725jZXdFOyy3soOxOK7dd22Z3zHYsfHfEb9slOQB4\nNfCU7pNcb6pqb1Xdk9F3IN4nyV37ajvJI4DPV9X5TPfswP2q6l7Aw4D/keT+U+xrK+yI3ILZnYDZ\n3abM7qp2Wm5hh2R3mrkFs7vUdix8J/kS7G0tyb7Aa4B/rKrXTaufqroaeDuwu8dm/yvwqCSfAF4J\nPCjJP/TYPgBV9dnu3y8A/8LoktWQDT63YHYnYXa3J7O7uh2YW9gB2d2q3ILZXbQdC99JvgR720oS\n4Azgo1X1F1No/7Akh3T3bwb8JKN5Qb2oqmdU1a2r6rbAzwFvq6rH99U+QJKbJzmwu78/8BBgKr81\nu4UGnVswu5Mwu9uT2V3dDs0tDDy7085t14fZXWLbFb5VdS2w+CXYHwVeVVW9vUlJXgm8F7hjksuT\n9P0F2vdj9DfKH9h99cb5Sfr8dPVDwNuSXAh8gNF8nTf22P5S07iUdDjwrm7O0bnAG6rqzVPoZ8tM\nO7dgdjfA7E7A7E5k6NndcbkF64UJmd0l/AMWkiRJasK2O+MrSZIkTYOFryRJkppg4StJkqQmWPhK\nkiSpCRa+kiRJaoKFryRJkppg4TsFSX43yUVJLuy+l++4JE/pvjx6rW3/5yTrSX0ztxoqs6uhMrtb\nz+/x7VmS+wJ/Djygqq5JcgvgpsB7gHtX1ZfW2P4Tk6wn9cncaqjMrobK7M6GZ3z794PAF6vqGoCq\n+jLws8CtgLcn+VeAJC9K8sHuk95899xvLLPeQ5K8N8m/Jfmn7k/2keS5ST7SfUr80y3fS+005lZD\nZXY1VGZ3Bjzj27MuaO8Gbg68ldGfUHxn98nsR7pgk+TQqroqya5uvV+vqovG10tyGPAaYHdVfSvJ\nKcB+wAuB91bVnbu2Dqqqr275zmrHMLcaKrOroTK7s+EZ355V1TeAHwGeCHwBeFWSk7vFGVv1MUn+\nDfh34K7Ascs0d3z3/HuTnA88HrgNcDXw7SRnJPlp4FvT2Be1w9xqqMyuhsrszsY+sx7ATlRVe4F3\nAO9I8mHg5MVFAEluC/wmo7k5Vyd5CaN5Pct5S1WdtPTJJMcBD2Z0WeTJ3X1pw8ythsrsaqjM7tbz\njG/PktwxyTFjT90L2AN8DTioe+4g4BvAV5McDjxsbP3x9c4F7pfk9l3b+yc5prs8ckhV/T/gacA9\nprU/aoO51VCZXQ2V2Z0Nz/j27wDgr5McAlwLXMroMsZJwJuSfLqqHtxdirgEuJzRHJ9FL16y3snA\nK5N8X7f8dxmF/fVJbsrocshTt2LHtKOZWw2V2dVQmd0Z8JfbJEmS1ASnOkiSJKkJFr6SJElqgoWv\nJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmS\nmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDh\nK0mSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK+1wSS5K8uOzHsd6JHljksdNuY/5JC/v7t8mydeS\npKe2X5Tk97r7c0ku76Pdrr37J7mkr/b6tFbWkiwkecKEbU28n9v5NZG0vewz6wFI2pwkXweqe7g/\n8G3ge93jJ1bVf5nJwDahqh6+Fd2M9fcp4MC1NkhyMvCEqrr/qg1X/fdNj+76PvcCd6iqj3dtvwu4\nc1/t92k8a0nmgdtX1fgHmGLsdV+jrYn3c+m6SfYAv1RVb5tk+7Ht7gj8KXBfYBfwQeA3quo/1tOO\npO3LM77SwFXVAVV1YFUdCHwSeMTi46p65azHtx7pzHocm5FkGsfVQb8mM1Bs7DU7GHgdcEfgcOAD\nwOt7HJekGbPwlXa4JHuSPKi7P5/kn5O8PMlXk3woyTFJTk1yZZJPJfnJsW0PTnJGks8kuSLJs1cq\n7JIcl+S8JFcn+VySPx9bdnyS9ya5KskFSR4wtmwhyR8meQ/wdeB2Sy+JJ/mlJB9N8uUkb0pym7Fl\nz+/GfnW3P3ddYXy3TfKObr/fDBw2tuzoJHsX9y3JyUn+s1v340lOSnJn4G+A+3bTIr7crfvSbmrD\nG7uz7w/snnv2kv5PTfKFJJ9IctKS/R/f15OTvKu7/87u6Qu7Ph+9dOpEkrt0bVzVTTV45NiylyZ5\nYZI3dPvy/iS3W89rl+SBST409vgtST4w9vhdSR7V3d+T5MFJdgOnAo/pxn3+WJNHJ3l3N55zknz/\nCu/X0v3ck+Q3k1yY5CtJzkzyfUvXzWj6ym2As7u+f2u59pdTVR+sqpdU1Veq6lrgL4A7JTl00jYk\nbW8WvtLOt/TS8iOAfwAOBc4HzumevxXwLOD0sXVfCnwXuD1wL+AhwC+v0M9fAs+vqoOB2wH/BJDk\nCOANwLOq6lDgt4DXLCl4Htu1u3jW+rpL4klOYFRE/TSjYvVdwCu7ZQ8F7g8c0/X7aOBLK4zv/zC6\ndP39wLOBX1jmtSHJ/t2+7K6qgxhd9r6gqi4BngS8rzubfouxzU4Enl1VBwDv5saX9H+w6/dWXb8v\nTnJMt2zFy/9VtThf9u5dn/+8ZKz7AmcDbwJuCfw68IqMLtkvegwwz+j9vgz4o27bSV+79wPHJLlF\n19/dgR9Ksn+SmwE/wug9uW5fqupNwB8DZ3bjvtfikIGTgJOBHwD2Y5SHSVQ3xocCt+3GcfKNVhpN\nrfgU11/5+LNuf7/SfThY7vb0Ffr8ceCzVXXVhGOUtM1Z+ErteWdVvaWqvge8mlHB9Nzu8asYnZE7\nKMnhwMOAp1bVt6rqC4zOgP3cCu1+l1GBdFhVfbOqzu2efyzwxq4YoqreCpwH/FS3vICXVtXFVbW3\nO9M27leB51TVx6pqL/Ac4J7dWd/vMiqW75LkJt06n1s6sG7dewO/X1XXdHNCz2bly+F7gbsluVlV\nXVlVH11sapl1C3hdVb2v27/vrLDuYt/vBP4vo4J0s44H9q+q51bVtVX1dkYfMk4cW+e1VXVe9/6+\nArhn9/w1TPDaVdW3GH1geACjIvcC4D3Aj3X9X7pCYRhu/BoU8PdVdVlVfZvRh6N73mjLlf1VVX2u\n6+/s9WxbVYdU1aEr3P7kRoNPjgReADxtHeOTtM1Z+Ert+fzY/W8BX6yqGnsMcABwFLAv8NnFM2OM\nLvXfcoV2n8BobuTFST6QZLGwPQp49PgZNuB+jM6CLlrtWw+OAv5ybNvFs5K36gq9FwAvBK5McnqS\n5X5J7VbAVV0Rt+iTy3VWVd9gVJT+KvCZbprAnVYZ31rjZ4W+f2iNbSZxq2X6/mT3PIwKzSvHln2L\n0XtL94tfk7x2AO8A5hidIX5Hd3sAozOiC+sc83hxfd14tmDbiSW5JfBm4IVV9app9CFpNix8Ja3k\ncuA7wPePnRk7uKruttzK3Vm8k6rqlsDzgFcnuTmjy84vX3KG7cAlZ9lW+03/TzH6dorx7fevqvd3\n/f51Vd0bOJZR4f3by7TxWeDQbjyLjlqp36p6c1U9hFFxfgnwtxOM80bNjN1fru/PdPe/wejbOBaN\nfyBYy2eAWyc3+IXAo4BPTzTAyV47GBW6D+T6QnexEH5Ad3/Z5icZw5QsN4Xl692c3+VuvzO23qGM\nit7XVdVztnLQkqbPwlfSsqrqs4wKgP+V5MAkN0ly+6zwPa1JHtudKQO4mlHx8T3gH4FHJnlIkl1J\nbtr9MtI9UyxHAAAB+0lEQVQR45uvMpS/AZ6R5Niun4OTPLq7f+8k9+nmnn6TG36V2/i+fJLR9Ipn\nJtk3yY8xmuu83H78QJITurm+1zAqTBfbvBI4sutvtbEvd5l/se/7M5rmsThf9wLgZ5LcLMkdGJ05\nH3cloznWyzmX0X4/vWt7rtuvM1cZ2+J+TvTadd4L3An4UeAD3dSPo4D7AO9cYZvPMZo2s3QMW/EN\nFTd6zca//WSZ23MBkhzEaM77u6vqGVswTklbzMJXastyv0i12uPHM/oFpI8CX2ZUrK10RvKhwEVJ\nvgY8H/i5qvpOVV0BnAA8g9E0i08Bv8kNC6AVzw5W1esYnUE+M8nVwIe7vgAOAl7cjW0P8EVG38O6\nnJMYFWpfBv4AeNkK+30T4KmMzpp+idHl/cXv5f1X4CPA55J8fmy75V7D8ec+C1zF6Azty4EnjX03\n7PMZzVW+EngJow8K49vOAy/rpnr87HjbVfVd4JGM5mJ/gdHUhceNtb3a+z3xa1dV3wT+DfjI2Bzs\n9wJ7quqLy23D9YX9l5Kct0z/K41vubGutGxpW4ueA/xe95qtZ47uTzOaC/6LY2eDv9rN95W0A+T6\nqX2SJEnSzuUZX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8\nJUmS1IT/D5ilPK/S98Y1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10e5e8ef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Another sample path with init=2\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAE7CAYAAAA7EoKDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYbHV95/H3xwtEZREMhkRQcMEFxy0xiGPQVhO9GpUn\nmfgYiBoSE8lkSBxNIsEk41WTUbOMWXQMJkSNccREjYLjiBptd0ESFlEwEL0KLrghLriA9zt/1Gko\nml6qu09196nf+/U89dyqOqd+v9+p+nD41jm/Op2qQpIkSZp1t9jqAUiSJEmbwcJXkiRJTbDwlSRJ\nUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfDdBkt1JHr6O1+1J8s0k3+huL5/G+KQFG8jq\ny5NcmuT7SX5xieXPSPL5JNckOT3JPv2MWBqZ1n7W7KpvW7GfTXLbJP/cZX13kuM3uh1DZeG7OQrI\nOl9776rav7s9rc9BSUtYb1YvAH4d+LeujRskeRRwCvBw4HDgzsBzNzZM6WZ638+aXU3JVuxnXwp8\nB/gh4BeAlyU5ah1jGDwL3ylL8mrgjsBZ3dGE315jE35G2hQbyWpV/e+qehejHetivwj8bVVdUlVf\nA54HnNjHmCWY6n7W7KpXW7GfTbIv8LPAH1TVtVX1AeDNwJM3tjXDZFE1ZVX1ZOAzwGO7owl/muRr\nSa5e5vasRU28tzt18YYkh2/BJqgRPWR1OUcBF449vgg4JMlBfW+D2jTF/azZVa+2aD97N+D6qrp8\nbPmFwL362Kah2WurB9CiqjpwwlUfAnwY2Bf4Q+AtSe5XVd+f2uCkMWvI6kr2A64Ze/z17t/9gat7\naF+6mQ3uZ+9bVXswu9oEm7Cf3W/s8YJvdMua4xHfbayq3l9V11fVNcDTgSOAe2ztqKQ1+yZwwNjj\n23T/fmMLxiLdxDL72Xt2i82uhmKlrC5etrC8yRxb+G6OxZPQx39BvPj2u8u0kUX/StPQR1YX+xhw\nv7HH9wWuqiqPmKlP09jPml1Nw2bvZ/8d2CvJXRctv3gjGzFUTnXYHFcBdwHeBVBV+632gu7XlvsA\nHwVuBfwRcCVwyfSGKa09qwBJ9gZ2MPoyvU+SWwLfraoC/h54ZZLXAF8A/gB4xRTGrrZNYz9rdjUN\nm7qfrapvJXkj8LwkvwL8KPA44EG9btVAeMR3c7wA+P1uovozJ3zNIcAZjObs/AdwB0aT4Z3fq2la\nT1YB3gFcCxwDvLy7fyxAVZ0N/DHwbmA3ozw/p8cxSzCF/azZ1ZRsxX721xl9ufsi8A/Ar1VVkwfS\nMvqisMIKyd8BPw18saruvcw6fwk8mtGHcGJVnd/3QKW1MrsaInOroTK7GoJJjvi+Ati53MIkjwHu\nWlVHAk8DXtbT2KSNMrsaInOroTK72vZWLXyr6n2sfNmWxwOv6tY9BzgwySH9DE9aP7OrITK3Giqz\nqyHoY47vocAVY4+vBA7roV1p2syuhsjcaqjMrrZcX1d1WHyJrZtNHE6y8mRiaUJV1ecl3cyuNoW5\n1VCZXQ3VUtnt44jvZxn9EnbBYd1zSw1g4ttznvOcNa2/1tu025+FbdiO7ffM7G5B+7OwDWtt39xu\nz8+ltfbX04fZ3Z6fi+2vfltOH4XvmcBTAJIcA3ytqq7qoV1p2syuhsjcaqjMrrbcqlMdkrwWeChw\ncJIrGF0Xbm+Aqjqtqt6a5DFJLge+BfzSNAcsTcrsaojMrYbK7GoIVi18q+r4CdY5uZ/h3Ghubq7v\nJje1/c3ow/ZXZna3Z/ub0ceQ25/V3G5GH7a/tX3ManaH/rnY/k2t+gcseusoqc3qS7MrCdXvDy0m\n6dPsakPMrYbK7Gqolsuuf7JYkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMs\nfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJ\nktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQE\nC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19J\nkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktSEVQvfJDuTXJrksiSnLLH8NknOSnJBkouTnDiVkUprZHY1\nVGZXQ2RuNQSpquUXJjuATwA/CXwW+AhwfFVdMrbOs4H9q+rUJAd36x9SVdcvaqtW6kuaRBKqKhOs\nZ3a1bUya227dXrJrbtUH97kaquWyu9oR36OBy6tqd1VdB5wBHLdonT3AAd39A4CvLA6xtAXMrobK\n7GqIzK0GYbXC91DgirHHV3bPjXsJcFSSzwEXAk/vb3jSupldDZXZ1RCZWw3CXqssn+Rcw07g36rq\nYUnuArwjyX2r6huLV9y1a9cN9+fm5pibm1vDUNWi+fl55ufn1/NSs6sts4HcQo/ZNbdaK/e5GqpJ\ns7vaHN9jgF1VtbN7fCqwp6peNLbOW4AXVNUHusf/ApxSVectass5O9qwNcw3M7vaNtY4x7eX7Jpb\n9cF9roZqvXN8zwOOTHJEkn2AJwJnLlrnM4wms5PkEODuwCc3PmRpQ8yuhsrsaojMrQZhxakOVXV9\nkpOBs4EdwOlVdUmSk7rlpwHPB16Z5CIgwLOq6qtTHre0IrOroTK7GiJzq6FYcapDrx156kI9WMsp\n4x77NLvaEHOroTK7Gqr1TnWQJEmSZoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg\n4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJ\nkqQmWPhKkiSpCRa+kiRJasJeWz0ASZpUkt7brKre25QkbU8WvpIGps9Ctf9CWpK0fTnVQZIkSU2w\n8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBK/jK0nSlPX9x1f8wyvS+lj4\nSpK0KfoqVv3DK9J6OdVBkiRJTbDwlSRJUhMsfCVJktQE5/hKkiTpZvr+USZs/Q8zLXwlSZK0jD4L\n1a3/YaZTHSRJktQEC19JkiQ1YdXCN8nOJJcmuSzJKcusM5fk/CQXJ5nvfZTSOphdDZXZ1RCZWw1B\nVppknGQH8AngJ4HPAh8Bjq+qS8bWORD4APCoqroyycFV9eUl2qqtntCs4UtCVa06ScjszqbRDy36\nnW+2GZ/tpLnt1u0lu+Z2e+k3u5uTW3Cf27qh7nNh+eyudsT3aODyqtpdVdcBZwDHLVrnBOANVXUl\nwFIhlraA2dVQmV0NkbnVIKxW+B4KXDH2+MruuXFHArdN8u4k5yV5cp8DlNbJ7GqozK6GyNxqEFa7\nnNkkx6P3Bn4UeARwa+BDST5cVZctXnHXrl033J+bm2Nubm7igao/Q7ou3/z8PPPz8+t5qdnVltlA\nbqHH7JpbrZX7XA3VpNldbY7vMcCuqtrZPT4V2FNVLxpb5xTgVlW1q3v8t8Dbqur1i9pyzs42MYtz\ndpZYz+zOoKFmd41zfHvJrrndXhqY4+s+dwYNdZ8L65/jex5wZJIjkuwDPBE4c9E6bwZ+IsmOJLcG\nHgh8vI9BSxtgdjVUZldDZG41CCtOdaiq65OcDJwN7ABOr6pLkpzULT+tqi5N8jbgImAP8DdVZZC1\npcyuhsrsaojMrYZixakOvXbkqYttYxZPXUy5T7O7TQw1u+ZWsz7Voec+ze42MdR9Lqx/qoMkSZI0\nEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8\nJUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS\n1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQL\nX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mS\nJDXBwleSJElNWLXwTbIzyaVJLktyygrr/XiS65P8bL9DlNbH7GqozK6GyNxqCFYsfJPsAF4C7ASO\nAo5Pcs9l1nsR8DYgUxintCZmV0NldjVE5lZDsdoR36OBy6tqd1VdB5wBHLfEer8BvB74Us/jk9bL\n7GqozK6GyNxqEFYrfA8Frhh7fGX33A2SHMoo3C/rnqreRietn9nVUJldDZG51SDstcrySUL558Dv\nVlUlCSucuti1a9cN9+fm5pibm5ugebVsfn6e+fn59bzU7GrLbCC30GN2za3Wyn2uhmrS7KZq+awm\nOQbYVVU7u8enAnuq6kVj63ySG8N7MHAt8KtVdeaitmqlvrR5RvubPj+LsFmfbRKqatV5YWZ3Ng01\nu5Pmtlu3l+ya2+2l3+y6z9XmGOo+F5bP7mqF717AJ4BHAJ8DzgWOr6pLlln/FcBZVfXGJZYZ5G1i\nFoO8xHpmdwYNNbtrLHx7ya653V4aKHzd586goe5zYfnsrjjVoaquT3IycDawAzi9qi5JclK3/LSp\njFbaILOroTK7GiJzq6FY8Yhvrx35DW7bmMVvcFPu0+xuE0PNrrnVrB/x7blPs7tNDHWfC8tn17/c\nJkmSpCZY+EqSJKkJFr6SJElqgoWvJEmSmmDhK0mSpCZY+EqSJKkJq/3JYkmSpJk0ulxXf7wM2/Zn\n4StJkhrW3/WVtf051UGSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElN8KoOkqQN8ZJQkobC\nwleS1AMvCSVp+3OqgyRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSp\nCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+\nkiRJaoKFryRJkppg4StJkqQmWPhKkiSpCRa+kiRJaoKFryRJkpowUeGbZGeSS5NcluSUJZb/QpIL\nk1yU5ANJ7tP/UKW1MbcaKrOroTK72u5WLXyT7ABeAuwEjgKOT3LPRat9EnhIVd0HeD7w8r4HKq2F\nudVQmV0NldnVEExyxPdo4PKq2l1V1wFnAMeNr1BVH6qqa7qH5wCH9TtMac3MrYbK7GqozK62vUkK\n30OBK8YeX9k9t5ynAm/dyKCkHphbDZXZ1VCZXW17e02wTk3aWJKHAb8MPHip5bt27brh/tzcHHNz\nc5M2vSZJem2v6qZvwbTb143m5+eZn59fz0t7yy1sTnb7zhWYra2ygdzCAPe5mh1mV0M1aXaz2v8Y\nkxwD7Kqqnd3jU4E9VfWiRevdB3gjsLOqLl+indqs/wmPCoi++soyhe/02p+2fscPm7kNSaiqVSvE\nvnLbrbMp2R3y57JZhvoeTZrbbl33uTOWWxjue2R219TazGV3qPtcWD67k0x1OA84MskRSfYBngic\nuajxOzIK8ZOWKx6kTWZuNVRmV0NldrXtrTrVoaquT3IycDawAzi9qi5JclK3/DTgfwAHAS/rTtde\nV1VHT2/Y0srMrYbK7GqozK6GYNWpDr11NEOnLoZ+amQWT11MuU+nOmwTQ32PZjm3XV8MeZ+4GYb6\nHpndNbU2c9kd6j4XNjbVQZIkSRo8C19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJX\nkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJTbDwlSRJUhMsfCVJktQEC19JkiQ1wcJXkiRJ\nTbDwlSRJUhMsfCVJktSEvbZ6AJI2T5Je26uqXtuTJGmaLHyl5vRVrPZbREuSNG1OdZAkSVITLHwl\nSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLU\nBAtfSZIkNcHCV5IkSU2w8JUkSVITLHwlSZLUBAtfSZIkNcHCV5IkSU1YtfBNsjPJpUkuS3LKMuv8\nZbf8wiT372Ng8/PzfTSzUg9Tbn/6fQz9PZr2+M3udm1/+O/RLGZ3+p8JrPdzSdL7bV2jH3huYbrb\n4D53Az1MeRuG/h71Of4VC98kO4CXADuBo4Djk9xz0TqPAe5aVUcCTwNe1sfAhv4hbUYfQ3+PprwD\nNrvbtv3hv0ezmN3tXPiO1AS350y43voMPbcwvW1wn7vBHix8V259swpf4Gjg8qraXVXXAWcAxy1a\n5/HAqwCq6hzgwCSH9DZCaX3MrobK7GqIzO0SJj0D8dznPndqZyp0U6sVvocCV4w9vrJ7brV1Dtv4\n0KQNMbsaKrOrITK3y+rrbIV6UVXL3oD/AvzN2OMnAX+1aJ2zgAePPX4n8KNLtDXJJ+/N26q3lTJr\ndr1t19skue0zu1u9vd5m5+Y+19tQb0tldS9W9lngDmOP78DoG9pK6xzWPXcTVeUxem0ms6uh6iW7\n5labzH2uBmG1qQ7nAUcmOSLJPsATgTMXrXMm8BSAJMcAX6uqq3ofqbQ2ZldDZXY1ROZWg7DiEd+q\nuj7JycDZwA7g9Kq6JMlJ3fLTquqtSR6T5HLgW8AvTX3U0irMrobK7GqIzK2GIt18GkmSJGmmbcu/\n3DbJRbA30PbfJbkqyUf7bHes/TskeXeSjyW5OMlv9tz+LZOck+SCrv1dfbY/1s+OJOcnOWtK7e9O\nclHXx7nT6GOzTTO3Xftmd7J+zO4amd1V2x98dmcxt2C9MEEfZnexSX9pvFk3RqdILgeOAPYGLgDu\n2WP7xwL3Bz46pfH/MHC/7v5+wCf6HH/X7q27f/cCPgw8cArb8UzgNcCZU3qfPgXcdjOzNc3btHPb\n9WF2J+vD7K5te8zuZH0MOruzlttum6wXJuvH7I7dtuMR30kugr1uVfU+4Oq+2lui/S9U1QXd/W8C\nlwC377mPa7u7+zD6j31Pn+0nOQx4DPC3wDR/XTtLv9ydam7B7E7C7K6L2Z2sj1nI7izlFqwXJu3H\n7I7ZjoXvJBfBHoQkRzD6tnhOz+3eIskFwFXA26vqI322D7wY+B16/o9jkQLemeS8JL86xX42y8zk\nFszuKszuNmZ2lzVruYUZyu60ctu1bXbHbMfCdyZ+bZdkP+D1wNO7b3K9qao9VXU/RtdAfGCSe/XV\ndpLHAl+sqvOZ7tGBB1fV/YFHA/8tybFT7GszzERuwexOwOxuU2Z3RbOWW5iR7E4zt2B2F9uOhe8k\nF8He1pLsDbwB+IeqetO0+qmqa4B3Azt7bPY/A49P8ingtcDDk/x9j+0DUFWf7/79EvDPjE5ZDdng\ncwtmdxJmd3syuyubwdzCDGR3s3ILZnfBdix8J7kI9raVJMDpwMer6s+n0P7BSQ7s7t8K+ClG84J6\nUVXPrqo7VNWdgJ8H3lVVT+mrfYAkt06yf3d/X+CRwFR+NbuJBp1bMLuTMLvbk9ld2YzmFgae3Wnn\ntuvD7C6y7QrfqroeWLgI9seB11VVbx9SktcCHwTuluSKJH1fQPvBjP5G+cO6S2+cn6TPb1c/Arwr\nyYXAuYzm67y1x/YXm8appEOA93Vzjs4B3lJVb59CP5tm2rkFs7sOZncCZnciQ8/uzOUWrBcmZHYX\n8Q9YSJIkqQnb7oivJEmSNA0WvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQkWvpIkSWqChe8UJPm9\nJBcnubC7Lt/RSZ7eXTx6tdf+90nWk/pmbjVUZldDZXY3n9fx7VmSBwF/Bjy0qq5LclvglsAHgAdU\n1VdWef2nJllP6pO51VCZXQ2V2d0aHvHt3w8DX66q6wCq6qvAzwG3B96d5F8AkrwsyUe6b3q7uud+\nc4n1Hpnkg0n+Nck/dn+yjyQvTPKx7lvin2z6VmrWmFsNldnVUJndLeAR3551QXs/cGvgnYz+hOJ7\nu29mP9YFmyQHVdXVSXZ06/1GVV08vl6Sg4E3ADur6ttJTgH2AV4KfLCq7tG1dUBVfX3TN1Yzw9xq\nqMyuhsrsbg2P+Pasqr4F/BjwNOBLwOuSnNgtztiqT0zyr8C/AfcCjlqiuWO65z+Y5HzgKcAdgWuA\n7yQ5PcnPAN+exraoHeZWQ2V2NVRmd2vstdUDmEVVtQd4D/CeJB8FTlxYBJDkTsBvMZqbc02SVzCa\n17OUd1TVCYufTHI08AhGp0VO7u5L62ZuNVRmV0NldjefR3x7luRuSY4ce+r+wG7gG8AB3XMHAN8C\nvp7kEODRY+uPr3cO8OAkd+na3jfJkd3pkQOr6v8BzwTuO63tURvMrYbK7GqozO7W8Ihv//YD/irJ\ngcD1wGWMTmOcALwtyWer6hHdqYhLgSsYzfFZ8PJF650IvDbJD3TLf49R2N+c5JaMToc8YzM2TDPN\n3GqozK6GyuxuAX/cJkmSpCY41UGSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElS\nEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8\nJUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS1AQLX0mSJDXBwleSJElNsPCVJElSEyx8JUmS\n1AQLX2nGJbk4yUO2ehxrkeStSZ485T52JXl1d/+OSb6RJD21/bIkv9/dn0tyRR/tdu0dm+TSvtrr\n02pZSzKf5KkTtjXxdm7n90TS9rLXVg9A0sYk+SZQ3cN9ge8A3+8eP62q/tOWDGwDquoxm9HNWH+f\nAfZf7QVJTgSeWlXHrthw1X/d8Ohu7HMPcNeq+mTX9vuAe/TVfp/Gs5ZkF3CXqhr/AlOMve+rtDXx\ndi5eN8lu4Jer6l2TvH7sdXcD/gR4ELAD+Ajwm1X172tpR9L25RFfaeCqar+q2r+q9gc+DTx24XFV\nvXarx7cW6Wz1ODYiyTT2q4N+T7ZAsb737DbAm4C7AYcA5wJv7nFckraYha8045LsTvLw7v6uJP+U\n5NVJvp7koiRHJjk1yVVJPpPkp8Zee5skpyf5XJIrkzx/ucIuydFJzktyTZIvJPmzsWXHJPlgkquT\nXJDkoWPL5pP8YZIPAN8E7rz4lHiSX07y8SRfTfK2JHccW/bibuzXdNtzr2XGd6ck7+m2++3AwWPL\njkiyZ2HbkpyY5D+6dT+Z5IQk9wD+GnhQNy3iq926r+ymNry1O/r+sO655y/q/9QkX0ryqSQnLNr+\n8W09Mcn7uvvv7Z6+sOvzCYunTiS5Z9fG1d1Ug8eNLXtlkpcmeUu3LR9Ocue1vHdJHpbkorHH70hy\n7tjj9yV5fHd/d5JHJNkJnAo8sRv3+WNNHpHk/d14zk7yg8t8Xou3c3eS30pyYZKvJTkjyQ8sXjej\n6St3BM7q+v7tpdpfSlV9pKpeUVVfq6rrgT8H7p7koEnbkLS9WfhKs2/xqeXHAn8PHAScD5zdPX97\n4HnAaWPrvhL4HnAX4P7AI4FfWaafvwBeXFW3Ae4M/CNAkkOBtwDPq6qDgN8G3rCo4HlS1+7CUesb\nToknOY5REfUzjIrV9wGv7ZY9CjgWOLLr9wnAV5YZ3/9hdOr6B4HnA7+4xHtDkn27bdlZVQcwOu19\nQVVdCpwEfKg7mn7bsZcdDzy/qvYD3s/NT+n/cNfv7bt+X57kyG7Zsqf/q2phvux9uj7/adFY9wbO\nAt4G3A74DeA1GZ2yX/BEYBejz/ty4I+610763n0YODLJbbv+7gP8SJJ9k9wK+DFGn8kN21JVbwP+\nJ3BGN+77LwwZOAE4EfghYB9GeZhEdWN8FHCnbhwn3myl0dSKz3DjmY8/7bb3a92Xg6Vuz1qmz4cA\nn6+qqycco6RtzsJXas97q+odVfV94PWMCqYXdo9fx+iI3AFJDgEeDTyjqr5dVV9idATs55dp93uM\nCqSDq+raqjqne/5JwFu7YoiqeidwHvDT3fICXllVl1TVnu5I27hfA15QVZ+oqj3AC4D7dUd9v8eo\nWL5nklt063xh8cC6dR8A/EFVXdfNCT2L5U+H7wHuneRWVXVVVX18oakl1i3gTVX1oW77vrvMugt9\nvxf4v4wK0o06Bti3ql5YVddX1bsZfck4fmydN1bVed3n+xrgft3z1zHBe1dV32b0heGhjIrcC4AP\nAD/R9X/ZMoVhuPl7UMDfVdXlVfUdRl+O7nezVy7vL6vqC11/Z63ltVV1YFUdtMztj282+OQw4CXA\nM9cwPknbnIWv1J4vjt3/NvDlqqqxxwD7AYcDewOfXzgyxuhU/+2WafepjOZGXpLk3CQLhe3hwBPG\nj7ABD2Z0FHTBSlc9OBz4i7HXLhyVvH1X6L0EeClwVZLTkiz1I7XbA1d3RdyCTy/VWVV9i1FR+mvA\n57ppAndfYXyrjZ9l+v6RVV4zidsv0fenu+dhVGheNbbs24w+W7offk3y3gG8B5hjdIT4Pd3toYyO\niM6vcczjxfUN49mE104sye2AtwMvrarXTaMPSVvDwlfScq4Avgv84NiRsdtU1b2XWrk7indCVd0O\neBHw+iS3ZnTa+dWLjrDtv+go20q/9P8Mo6tTjL9+36r6cNfvX1XVA4CjGBXev7NEG58HDurGs+Dw\n5fqtqrdX1SMZFeeXAn8zwThv1szY/aX6/lx3/1uMrsaxYPwLwWo+B9whuckPAg8HPjvRACd772BU\n6D6MGwvdhUL4od39JZufZAxTstQUlm92c36Xuv3u2HoHMSp631RVL9jMQUuaPgtfSUuqqs8zKgD+\nV5L9k9wiyV2yzHVakzypO1IGcA2j4uP7wD8Aj0vyyCQ7ktyy+zHSoeMvX2Eofw08O8lRXT+3SfKE\n7v4Dkjywm3t6LTe9lNv4tnya0fSK5ybZO8lPMJrrvNR2/FCS47q5vtcxKkwX2rwKOKzrb6WxL3Wa\nf6HvYxlN81iYr3sB8LNJbpXkroyOnI+7itEc66Wcw2i7n9W1Pddt1xkrjG1hOyd67zofBO4O/Dhw\nbjf143DbqryCAAABcUlEQVTggcB7l3nNFxhNm1k8hs24QsXN3rPxq58scXshQJIDGM15f39VPXsT\nxilpk1n4Sm1Z6odUKz1+CqMfIH0c+CqjYm25I5KPAi5O8g3gxcDPV9V3q+pK4Djg2YymWXwG+C1u\nWgAte3Swqt7E6AjyGUmuAT7a9QVwAPDybmy7gS8zug7rUk5gVKh9FfgfwKuW2e5bAM9gdNT0K4xO\n7y9cl/dfgI8BX0jyxbHXLfUejj/3eeBqRkdoXw2cNHZt2Bczmqt8FfAKRl8Uxl+7C3hVN9Xj58bb\nrqrvAY9jNBf7S4ymLjx5rO2VPu+J37uquhb4V+BjY3OwPwjsrqovL/Uabizsv5LkvCX6X258S411\nuWWL21rwAuD3u/dsLXN0f4bRXPBfGjsa/PVuvq+kGZAbp/ZJkiRJs8sjvpIkSWqCha8kSZKaYOEr\nSZKkJlj4SpIkqQkWvpIkSWqCha8kSZKaYOErSZKkJlj4SpIkqQn/HxYx1WbFtCXkAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10e875c88>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "init = 2\n",
    "ts = [5, 10, 50, 100]\n",
    "seeds = [222, 2222]\n",
    "descriptions = ['{0} sample path with init={1}'.format(adjective, init) for adjective\n",
    "                in ['Some'] + ['Another'] + ['Yet another']*(len(seeds)-1)]\n",
    "\n",
    "for seed, description in zip(seeds, descriptions):\n",
    "    print(description)\n",
    "    plot_time_series_dists(mc1, init, ts, seed=seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In fact,\n",
    "for almost every sample path of a finite Markov chain $\\{X_t\\}$,\n",
    "for some recurrent class $C$ we have\n",
    "$$\n",
    "\\frac{1}{t} \\sum_{\\tau=0}^{t-1} \\mathbf{1}\\{X_{\\tau} = s\\} \\to \\psi^C[s]\n",
    "\\quad \\text{as $t \\to \\infty$}\n",
    "$$\n",
    "for all states $s$,\n",
    "where $\\psi^C$ is the stationary distribution associated with the recurrent class $C$.\n",
    "\n",
    "If the initial state $s_0$ is a recurrent state,\n",
    "then the recurrent class $C$ above is the one that contains $s_0$,\n",
    "while if it is a transient state,\n",
    "then the recurrent class to which the convergence occurs depends on the sample path."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us simulate with the remaining states, `3`, `4`, and `5`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAD3CAYAAAD7T1f4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmcLXV55/HPV5a4sAhiUFmjwbgv0bniEGMrCV6JShLj\nKAQdEieSzGgyiVGjMfESTNQso0YTgoaIoiNJJMmAIeBGC24sRlD0YiSKbLIJIpt6kWf+qGotmj7d\npy+nz9L1eb9e9bp9qn6nfk9V3+ecp37nd6pTVUiSJEnr3T0mHYAkSZI0Dha+kiRJ6gULX0mSJPWC\nha8kSZJ6wcJXkiRJvWDhK0mSpF6w8B2TJKcmeeEy249J8toh9zWf5MWji05Sl/kqzRZzVsOy8L0b\nklyS5MBh2lbVwVV1Qvu8I5KctWj7b1TV64fsutplNbHum+SOJEP/ztvje/pq+hmFJGckuSbJjUnO\nT/Kccceg9cd8XVtJntrGfPSkYtD6Ys6ujbbfW5Pc1C6njTuGSdp20gHMuFUnxxTIKtrWKtuPym8C\nF1XVliQbgI8k2a+qrp5ALFo/zNc1kmQ74K3AZ5i9c6zpZc6ujQKeVVUfm0DfE+eI74i0V5ifSPJn\nSa5P8tUkGzvb55O8OMnDgL8BntxeaV3fbj9+YaQkyS5JPtiOel6f5JQkewwZx4Yk57WjpVcl+fN2\n05ntv99q+31Skock+ViS65Jcm+S9SXZu93MCsDdwStv+d9v1+yf5VJIb2tHYp47i/HVV1Reqaktn\n1XbAXqPuR/1lvo7cy4HTgC8zoeJb65s5O3K9zVML39HaAFwE3A/4U+C4zrYCqqouAo4EPl1VO1bV\nrt3t7c9pn7t3u9wGvH3IGN4KvLmqdgYeDPxju/4p7b87t/2e3T7+Y+CBwMNpistNNIG+ELiU5qpw\nx6r68/aF4YPAH1XVLsDvAicl2W2pQNoXlhsGLCcvdxDtc2+jGUE6o6rOG/L4pWGZrx1bm69J9gF+\nBTiaHr+ZaizM2Y678x4LvK8t/E9P8pghj31dsPAdra9X1XFVVcB7gAcm+dEl2g16cwhAVV1fVf9c\nVd+pqpuBPwGGver7HrBfkt2q6tZO8t2lz6r6z6r6aFVtqarrgDev0M/hwKlVdVr7/I8A5wEHL9W4\nqp5VVbsMWJadt1tVzwJ2aPf94RWOWdoa5uud97+1+fqXwGur6hZm86NpzQ5z9s7739qcPQzYp13O\nAE5fGInuAwvf0bpq4YequrX9cYfV7iTJvZMcm2YC+o3Ax4GdkwwzmvJi4KHA5iTnJPm5ZfrZPcmJ\nSS5v+zmB5kp6kH2A53WvKoEDgAcMfXCrUFXfb18ADkry7LXoQ71mvt5NbV7uUFULo17BUV+tHXN2\nBKrq01X13aq6rareCHyLH45Yr3t+uW0yBo2ILKx/OU1ibaiqa5I8Dvh3mjeUZUdTqupimqs5kjwX\n+ECSXQc870+A7wOPqqpvJfl54G3LxHkpcEJVvWS5GBYk+TfgpwZsPrOqBr5gLLIdzUdK0iSYr4Pz\n9enAE5N8o328M/D9JI+qql8Ypl9pDZizq3uP7dWnNI74TsbVwJ5pvgm9oDtSsgPNnKMb24R63RL7\nWPLKNMnhSe7fPryR5j/0HcC17b8P6TTfAbgF+HY7t+gVS8TZbf9e4NlJDkqyTZJ7JpnLgC8FVNUz\n27lLSy1LJmSSn0jyzCT3SrJdksNprkQ/vlR7aQzM18FvoH8A7Ac8FngccDLwDpo5v9KkmLOD32P3\nSnJAku3b/b+CZhT6k0u1X48sfEdnqbltg66iPgp8EbgqyTVLPP8twL2A64BPAf+2in0/A7gwyU00\n84le0H6kcSvNJPtPpvkW6wbgKOAnaZL3FOCkRft9A/Da9iOX36mqy4FDgNcA19Bcnb6c0f4/Cs2L\n0NVtHy8D/ltVnT/CPiTzdQSq6uaquqZdrqYpJm6pqm+Nqg+pZc6Oxo7AXwPXA5cDBwHPrKobRtjH\nVEszR3yZBsnfAT8HXFNVjx7Q5i+BZwK3AkdU1edGHaiklZmv0uwwX6XxG+Yq4l3AxkEbkxwM/HhV\n7Qe8BDhmRLFJWj3zVZod5qs0ZisWvlV1FrDcEPhzgHe3bc8G7ptk99GEJ2k1zFdpdpiv0viNYt7I\nHsBlnceXA3uOYL+SRs98lWaH+SqN2KgmTC/+9mOvbo0hzRjzVZod5qs0QqO4j+8VNH+Gb8Ge7bo7\nSWKySotU1bhv9m++SltpWvMVzFlpKUvl7ChGfE8GXgSQZH/gW+1tbZYKYKuX173udXfr+aNYjMEY\nRtn/hIwlX6fhdzQNMUy6f2MYXQwTMnS+AhOJdZp+R+shhkn3v55iGGTFEd8k76f529K7JbmM5h6r\n27X/4Y+tqlOTHJzkYpobNXvjcmlCzFdpdqxtvq62AN7ULqvhX6fW7Fmx8K2qQ4do89LVdDrcn8O+\nq6OOOmrVz1mu6pfWm7XIV0lrw3yVxm+Cf7mtVrmcsRXPGa25ubmR79MYZjOGSfc/C6bhHE06hkn3\nbwzTFcN0m5t0AFPxO5p0DJPuvw8xrPiX20bWUVILfTUjvuPoN474amolocb/ZZmhdPNV0nTnK/ge\nKy02KGcnOOIrSZIkjY+FryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqSJKkXLHwlSZLUCxa+\nkiRJ6gULX0mSJPWCha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS1AsWvpIk\nSeoFC19JkiT1goWvJEmSesHCV5IkSb1g4StJkqResPCVJElSL1j4SpIkqRcsfCVJktQLKxa+STYm\nuSjJV5K8aontOyc5Jcn5SS5McsSaRCppKOasNDvMV2m8UlWDNybbAF8Gfga4AjgXOLSqNnfavAbY\nsapenWS3tv3uVXX7on3VQl9JgMH9jk5Y7vikSUpCVWXE+xxJznbzVdJ052vbzvdYqWNQzq404rsB\nuLiqLqmqLcCJwCGL2twB7NT+vBPwzcUJKWlszFlpdpiv0pitVPjuAVzWeXx5u67r7cAjklwJXAD8\n1ujCk7RK5qw0O8xXacy2XWH7MJ9hbAT+vaqeluQhwIeTPLaqblrccNOmTZ1H88DckGFKs29+fp75\n+fm17mZkOdvN17m5Oebm5kYZpzTVZi1fwfdY9duwObvSHN/9gU1VtbF9/Grgjqp6U6fNB4E3VNUn\n28cfBV5VVect2pfzj6SONZozOJKcdY6vdGfTnK/tet9jpY6tneN7HrBfkn2TbA88Hzh5UZtLaSbm\nk2R34CeAr979kCVtBXNWmh3mqzRmy051qKrbk7wUOB3YBjiuqjYnObLdfixwNHB8ks8DAV5ZVdev\ncdySlmDOSrPDfJXGb9mpDiPtyI9hpDtZi49OR8WpDtKdTXO+gu+x0mJbO9VBkiRJWhcsfCVJktQL\nFr6SJEnqBQtfSZIk9YKFryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqSJKkXLHwlSZLUCxa+\nkiRJ6gULX0mSJPWCha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS1AsWvpIk\nSeoFC19JkiT1wraTDkCSNDuSjK2vqhpbX5L6wcJXkrRK4yhIx1dgS+oPpzpIkiSpFyx8JUmS1Asr\nFr5JNia5KMlXkrxqQJu5JJ9LcmGS+ZFHKWlo5qw0O8xXabyy3JcHkmwDfBn4GeAK4Fzg0Kra3Glz\nX+CTwDOq6vIku1XVdUvsqxb6ar4cMZ45Yn45QtMqCVU10omMo8rZbr5KXX19/Z7mfG3b+R4rdQzK\n2ZVGfDcAF1fVJVW1BTgROGRRm8OAk6rqcoClElLS2Jiz0uwwX6UxW6nw3QO4rPP48nZd137ArknO\nSHJekheOMkBJq2LOSrPDfJXGbKXbmQ3zGcZ2wE8CBwL3Bj6d5DNV9ZW7G5ykVTNnpdlhvkpjtlLh\newWwV+fxXjRXpF2XAddV1W3AbUnOBB4L3CUpN23a1Hk0D8ytLlpphs3PzzM/P7/W3YwsZ7v5Ojc3\nx9zc3BqEK02nWctX8D1W/TZszq705bZtaSbeHwhcCZzDXSfePwx4O/AM4EeAs4HnV9WXFu3LifdS\nxxp9WWYkOeuX2zRIX1+/pzlf23a+x0odg3J22RHfqro9yUuB04FtgOOqanOSI9vtx1bVRUlOAz4P\n3AG8c3FCShoPc1aaHearNH7LjviOtCOvRqU7WYsRpFFxxFeD9PX1e5rzFXyPlRbb2tuZSZIkSeuC\nha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS1AsWvpIkSeoFC19JkiT1goWv\nJEmSesHCV5IkSb1g4StJkqResPCVJElSL1j4SpIkqRcsfCVJktQLFr6SJEnqBQtfSZIk9YKFryRJ\nknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqSJKkXLHwlSZLUCxa+kiRJ6gULX0mSJPXCioVvko1J\nLkrylSSvWqbdf0lye5JfHG2IklbDnJVmh/kqjdeyhW+SbYC3AxuBRwCHJnn4gHZvAk4DsgZxShqC\nOSvNDvNVGr+VRnw3ABdX1SVVtQU4EThkiXYvAz4AXDvi+CStjjkrzQ7zVRqzlQrfPYDLOo8vb9f9\nQJI9aBL1mHZVjSw6Satlzkqzw3yVxmzbFbYPk2BvAX6vqipJWOZjmE2bNnUezQNzQ+xeWh/m5+eZ\nn59f625GlrPdfJ2bm2Nubm4U8UkzYdbyFXyPVb8Nm7OpGpx3SfYHNlXVxvbxq4E7qupNnTZf5YeJ\nuBtwK/BrVXXyon3VQl9N7o7jojUsd3zSJCWhqkY6X29UOdvNV6mrr6/f05yvbTvfY6WOQTm7UuG7\nLfBl4EDgSuAc4NCq2jyg/buAU6rqn5bYZlJKHWv0RjqSnLXw1SB9ff2e5nxtt/keK3UMytllpzpU\n1e1JXgqcDmwDHFdVm5Mc2W4/dk2ilbRVzFlpdpiv0vgtO+I70o68GpXuZC1GkEbFEV8N0tfX72nO\nV/A9VlpsUM76l9skSZLUCxa+kiRJ6gULX0mSJPXCSvfxlaSJa+Ysjo/zFiVpfbLwlTQjxlWMTu33\nlyRJd5NTHSRJktQLFr6SJEnqBQtfSZIk9YKFryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqS\nJKkXLHwlSZLUCxa+kiRJ6gULX0mSJPWCha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSp\nFyx8JUmS1AsWvpIkSeoFC19JkiT1wlCFb5KNSS5K8pUkr1pi+y8nuSDJ55N8MsljRh+qpGGYr9Ls\nMF+l8Vqx8E2yDfB2YCPwCODQJA9f1OyrwE9X1WOAo4F3jDpQSSszX6XZYb5K4zfMiO8G4OKquqSq\ntgAnAod0G1TVp6vqxvbh2cCeow1T0pDMV2l2mK/SmA1T+O4BXNZ5fHm7bpAXA6fenaAkbTXzVZod\n5qs0ZtsO0aaG3VmSpwG/Chyw1RFJujvMV2l2mK/SmA1T+F4B7NV5vBfNVemdtBPu3wlsrKobltrR\npk2bOo/mgbkhw5Rm3/z8PPPz82vdzZrk69zcHHNzc6OMU5pqs5av4Hus+m3YnE3V8hecSbYFvgwc\nCFwJnAMcWlWbO232Bj4GHF5Vnxmwn1roKwmruNC9G8JKxydNShKqKiPe58jzdRqM7zUDfN1YXl9f\nv6c5X9t2vsdKHYNydsUR36q6PclLgdOBbYDjqmpzkiPb7ccCfwjsAhzTJBxbqmrDKA9A0srMV2l2\nmK/S+K044juyjrwale5kLUaQRsUR3+k59mnT19fvac5X8D1WWmxQzvqX2yRJktQLFr6SJEnqBQtf\nSZIk9YKFryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqSJKkXLHwlSZLUCxa+kiRJ6gULX0mS\nJPWCha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS1AsWvpIkSeoFC19JkiT1\ngoWvJEmSesHCV5IkSb1g4StJkqResPCVJElSL1j4SpIkqRcsfCVJktQLKxa+STYmuSjJV5K8akCb\nv2y3X5Dk8aMPE2B+bXa7mgjmjcEYpqP/5UxLzk7HOZqfbO9TcA6mIYZJ/x5gWs7DXU1Lvvo7mo4Y\nJt1/H2JYtvBNsg3wdmAj8Ajg0CQPX9TmYODHq2o/4CXAMWsT6vza7HY1Eazz/wzGMDv9DzJNOTsd\n52h+sr1PwTmYhhgm/XuAaTkPdzZN+ervaDpimHT/fYhh2xW2bwAurqpLAJKcCBwCbO60eQ7wboCq\nOjvJfZPsXlVXr0G8GrEkW/W8o446atXPqaqt6mutjfMcwJqfB3NWmh3mq9a9aaszVprqsAdwWefx\n5e26ldrsOUR8mhq1yuV1W/GcaTeOczCW82DOSrPDfFVPTFGdUVUDF+C5wDs7jw8H3raozSnAAZ3H\nHwF+col9bU2V4OKyrpfl8m9rFkaUs5M+Ly4u07hMa76asy4uSy9L5d1KUx2uAPbqPN6L5mpzuTZ7\ntuvupKq2bqxb0mqMJGfNV2ksfI+VxmylqQ7nAfsl2TfJ9sDzgZMXtTkZeBFAkv2Bbzn3SJoYc1aa\nHearNGbLjvhW1e1JXgqcDmwDHFdVm5Mc2W4/tqpOTXJwkouBW4BfWfOoJS3JnJVmh/kqjV+m9Zv2\nkiRJ0ijNxF9uG+YG32vc/98luTrJF8bddyeGvZKckeSLSS5M8ptj7v+eSc5Ocn7b/6Zx9r8olm2S\nfC7JKRPq/5Ikn29jOGcSMUwz83Xy+drGMBU5a75Ov77nrPl6l1jWdc5O/Yhve4PvLwM/QzOh/1zg\n0KravOwTRxvDU4CbgfdU1aPH1e+iGB4APKCqzk+yA/BZ4OfHfB7uXVW3JtkW+ATwW1V19rj678Tx\nO8ATgB2r6jkT6P9rwBOq6vpx9z3tzNcfxDDxfG3jmHjOmq/TzZw1X5eIY13n7CyM+P7gBt9VtQVY\nuMH32FTVWcAN4+xziRiuqqrz259vprnB+YPGHMOt7Y/bA9sBd4yzf4AkewIHA38LTPJbzH6Demnm\nK9ORr23fE81Z83Um9D5nzdcf6kPOzkLhO8wNvnslyb7A44Fxj9zcI8n5wNXAh6rq3HH233oz8Aom\n8ILQUcBHkpyX5NcmGMc0Ml8XmVS+tn1POmfN1+lnznb0PF+hBzk7C4XvdM/FGLP2Y5gP0HwEcvM4\n+66qO6rqcTT3kXxSkkeOs/8kzwKuqarPMdkr0QOq6vHAM4H/1X5Mp4b52jHJfIXJ5qz5OjPM2Vaf\n8xX6k7OzUPgOc4PvXkiyHXAS8N6q+pdJxVFVNwJnABvH3PV/BZ7Tzv95P/D0JO8ZcwxU1Tfaf68F\n/pnmo0I1zNfWtOQrTCxnzdfZYM5ivrZ6kbOzUPgOc4PvdS9JgOOAL1XVWybQ/25J7tv+fC/gZ2nm\nQY1NVb2mqvaqqh8DXgB8rKpeNM4Yktw7yY7tz/cBDgImdveAKWS+Mvl8bWOYaM6arzOj9zlrvjb6\nkrNTX/hW1e3Awg2+vwT8/QS+afl+4FPAQ5NclmQSNxA/gObvuD+tvcXH55KM82rwgcDHklwAnEMz\n/+jUMfa/lEl8RLc7cFY7D+ts4INV9aEJxDGVzNcfmHS+wvTlrPk6hcxZwHwdZF3m7NTfzkySJEka\nhakf8ZUkSZJGwcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS1AsWvjMqye8nuTDJBe09\nBzck+a32xtcrPfd/D9NO0uiYs9LsMF/XL+/jO4OSPBn4C+CpVbUlya7APYFPAk+sqm+u8PyvDdNO\n0miYs9LsMF/XN0d8Z9MDgOuqagtAVV0P/BLwIOCMJB8FSHJMknPbq9ZN7brfXKLdQUk+leSzSf6h\n/TOBJHljki+2V7x/NvajlNYPc1aaHebrOuaI7wxqk+YTwL2Bj9D8ickz26vMJ7RJSpJdquqGJNu0\n7V5WVRd22yXZDTgJ2FhVtyV5FbA98FfAp6rqYe2+dqqqb4/9YKV1wJyVZof5ur454juDquoW4AnA\nS4Brgb9PckS7OZ2mz0/yWeDfgUcCj1hid/u36z+V5HPAi4C9gRuB7yQ5LskvALetxbFIfWDOSrPD\nfF3ftp10ANo6VXUH8HHg40m+AByxsAkgyY8BL6eZZ3RjknfRzFFayoer6rDFK5NsAA6k+Yjnpe3P\nkraCOSvNDvN1/XLEdwYleWiS/TqrHg9cAtwE7NSu2wm4Bfh2kt2BZ3bad9udDRyQ5CHtvu+TZL/2\no577VtW/Ab8DPHatjkda78xZaXaYr+ubI76zaQfgbUnuC9wOfIXmI5nDgNOSXFFVB7Yfq1wEXEYz\nX2nBOxa1OwJ4f5Ifabf/Pk3i/r8k96T5aOe3x3Fg0jplzkqzw3xdx/xymyRJknrBqQ6SJEnqBQtf\nSZIk9YKFryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqSJKkXLHwlSZLUCxa+kiRJ6gULX6ln\nklyY5KcnHcdqJDk1yQvXuI9NSU5of947yU1JMqJ9H5Pkte3Pc0kuG8V+2/09JclFo9rfkH3unuTM\nJN9O8mfj7FuS7g4LX2mdSXJzW7TdlOSOJLd2Hh9aVY+qqjMnHedqVNXBVXXCWnfT6e/SqtqxVvib\n7kmOSHLWijuu+o2qev0ogmx/pw/u7PusqnrYKPa9Ci8BrqmqnarqFXd3Z8OexwHPfVqSM5J8K8nX\nlti+b7v9liSbkxy4aPthSb7e5s0/J9lla49D0vSz8JXWmaraoS3adgS+Djxr4XFVvX/S8a1GWpOO\n4+5Ishavs5M+J/sAm7fmiUm2HXEsNwN/CwwqwN8PfBbYFfh94ANJdmtjeSTwN8AvA7sDtwJ/PeL4\nJE0RC1+pZ5JckuTp7c+bkvxjkhPaj60/n2S/JK9OcnWSS5P8bOe5Oyc5LsmVSS5PcvSgwi7JhiTn\nJbkxyVVJ/qKzbf8kn0pyQ5Lzkzy1s20+yeuTfJKmqHlwu+7FnTa/muRLSa5PclqSvTvb3tzGfmN7\nPI8cEN+PJfl4e9wfAnbrbNu3HVm9R/v4iCT/2bb9ajtK+DCaounJ7Wj69W3b49upDacmuRl4Wrvu\n6EX9vzrJtUm+luSwRcffPdYfjIYmWRipv6Dt83mLp04keXi7jxvSTGt5dmfb8Un+KskH22P5THf0\neJhzl+R44EXAK9sYnp5k+yRvSXJFu7w5yfZt+7n2/8ork3wDOG7R/h4OHLP4PA6rqs6tqvcBS432\nPhR4PPC6qvpuVf0T8HnguW2TXwZOrqpPVNUtwB8Av5jkPquJQdLssPCV+mfxx/fPAt4D7AJ8Dji9\nXf8g4I+AYzttjwe+BzyEpqA4CPgfA/p5K/DmqtoZeDDwDwBJ9gA+CPxRVe0C/C5wUpL7dZ57eLvf\nhVHrWog7ySHAq4FfoClWz6IZ1SPJM4CnAPu1/T4P+OaA+P4vcC5wP+Bo4L8vcW5oi6C3Ahuraifg\nycD5VXURcCTw6XY0fdfO0w4Fjq6qHYBPdONvPaDt90Ftv+9Isl+7bXHbH6iqhbnZj2n7/MdFsW4H\nnAKcBtwfeBnwvrYAXPB8YBPN7/ti4I/b5w517qrqCOB9wJvaGD4GvBbYADy2XTa06xbs3va3d3vO\nuvvbDPw6i85jkt9ri/ellmGL40cCX22L2gUXtOsXtl/QieWrwHeB7vmStI5Y+Eo6s6o+XFXfBz5A\nUzC9sX3898C+SXZKsjvwTOC3q+q2qroWeAvwggH7/R6wX5LdqurWqjq7XX84cGpVnQZQVR8BzgN+\nrt1ewPFVtbmq7qiq2xft99eBN1TVl6vqDuANwOPaUd/v0RTLD09yj7bNVYsDa9s+EfiDqtpSVWfR\nFIyDphDcATw6yb2q6uqq+tLCrpZoW8C/VNWn2+P77oC2C32fCfwrTUF6d+0P3Keq3lhVt1fVGTQX\nGYd22vxTVZ3X/n7fBzyuXb+FIc5dR/d4DqO5kLmuqq4DjgK6X0a8g2bUdUtVfWeFfQHQHsMuA5Zd\nl9jHUnYAbly07qZ2PcB9ltj+bZrzIGkdsvCVdE3n59uA6zpf6rqt/XcHmnmd2wHfWBh5o/mo//4D\n9vtimpGzzUnOSbJQ2O4DPK87ggccQDMKumC5ux7sA7y189yFUckHtYXe24G/Aq5OcmySpYqYBwE3\nVNVtnXVfX6qzdrTw+TQF95XtNIGfWCa+leJnQN8PXOE5w3jQEn1/vV0PTVF+dWfbbbRFYDtyO8y5\nG9Rv9/xd2ukT4Nqq+t6Q+xqlm4GdFq3bmab4Xdi+8zLbJa0zFr6ShnUZzcfA9+uMvO1cVY9eqnFV\nXVxVh1XV/YE30Xyp6N40RdEJi0bwdqyqP+0+fZk4LgVesuj596mqz7T9vq2qngg8gqbwXupLT98A\ndmnjWbDPoH6r6kNVdRBNcX4R8M4h4rzLbjo/L9X3le3Pt9CMRC7oXhCs5Epgr+ROXwjcB7hiqACH\nO3eD+t2383hvfng8sPJ5WmqKyWvyw7uRLF6+PWRcX6SZI75DZ91j2/UL2x/b6fMhwPbAfwy5f0kz\nxsJX0lCq6hvAh4D/k2THJPdI8pAMuCdwksOTLIwG30hT3HwfeC/w7CQHJdkmyT3bL0Dt0X36MqH8\nDfCaJI9o+9k5yfPan5+Y5EntXNdbge+0fS4+lq/TTK84Ksl2SX6KZq7zUsfxo0kOaef6bqEpTBf2\neTWwZ9vfcrFnifULfT+FZprHwnzd82m+YHWvJD9OM3LedTXNHOulnE1z3K9s9z3XHteJy8S2cJxD\nnbsB+3k/8Noku6W5Y8IfAqu5/dxVLDqPVfUnnbuRLF5+MIqbxj1pPo1Ikh9Z+GJdVf0Hzfl8Xfv/\n7BeBRwEntU9/H83/xZ9qf79HAyctmhMsaR2x8JX6bakvUi33+EU0I2JfAq6nKdYGjUg+A7gwyU3A\nm4EXtN+svxw4BHgNzTSLS4GXc+diauAIYVX9C80I8olJbgS+0PYFzcfa72hjuwS4Dhj0BxYOA57U\ntv1D4N2rrXVoAAAA7UlEQVQDjvsewG/TjJp+k+YLYL/RbvsozajhVUmu6TxvqXPYXfcN4AaaUdET\ngCPbIg2ac/U9mgL3XTQXCt3nbgLe3U71+KXuvtvpBM+mmYt9Lc3UhRd29r3c73s1527xfl5PcyHx\n+XY5r123uI9BPsZdz+OwnkpTqP8rsBfN9I3TOttfQDOf+3qaL/I9t6q+CdDO1f51mgL4auBewP9c\nZf+SZkhq+fuzS5IkSeuCI76SJEnqBQtfSZIk9YKFryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y\n+EqSJKkXLHwlSZLUCxa+kiRJ6oX/D6qLDUDA14a4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10e378780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "inits = [3, 4, 5]\n",
    "t = 100\n",
    "\n",
    "fig, axes = plt.subplots(1, 3, figsize=(12, 3))\n",
    "\n",
    "for init, ax in zip(inits, axes):\n",
    "    draw_histogram(time_series_dist(mc1, t=t, init=init), ax=ax,\n",
    "                   title='Initial state = {0}'.format(init),\n",
    "                   xlabel='States')\n",
    "\n",
    "fig.suptitle('Time series distributions for t={0}'.format(t),\n",
    "             y=-0.05, fontsize=12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Cross sectional averages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let us repeat the simulation for many times (say, 10,000 times)\n",
    "and obtain the distribution of visits to each state at a given time period `T`.\n",
    "That is, we want to simulate the marginal distribution at time `T`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def cross_sectional_dist(mc, T, init=None, num_reps=10**4, random_state=None):\n",
    "    \"\"\"\n",
    "    Return the distribution of visits at time T by num_reps times of simulation\n",
    "    of mc with an initial state init.\n",
    "    \n",
    "    \"\"\"\n",
    "    T_max = np.max(T)\n",
    "    dim = 1\n",
    "    try:\n",
    "        Ts_size = len(T)  # T is an array\n",
    "        Ts_array = T\n",
    "        dim = 2\n",
    "    except:  # T is an int\n",
    "        Ts_size = 1\n",
    "        Ts_array = [T]\n",
    "    \n",
    "    x = mc.simulate(ts_length=T_max+1, init=init, num_reps=num_reps,\n",
    "                    random_state=random_state)[:, Ts_array]\n",
    "    dists = np.empty((x.shape[-1], mc.n))\n",
    "    bins = np.arange(mc.n+1)\n",
    "    for i in range(x.shape[-1]):\n",
    "        hist, bin_edges = np.histogram(x[:, i], bins=bins)\n",
    "        dists[i, :] = hist / num_reps\n",
    "    if dim == 1:\n",
    "        return dists[0]\n",
    "    else:\n",
    "        return dists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Start with state `1`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.    ,  0.3333,  0.    ,  0.    ,  0.6667,  0.    ])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "init = 1\n",
    "T = 10\n",
    "cross_sectional_dist(mc1, init=init, T=T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.    ,  0.3348,  0.    ,  0.    ,  0.6652,  0.    ])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T = 100\n",
    "cross_sectional_dist(mc1, init=init, T=T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The distribution is close to the stationary distribution `(0, 1/3, 0, 0, 2/3, 0)`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the simulated marginal distribution at `T` for some values of `T`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def plot_cross_sectional_dists(mc, init, Ts, num_reps=10**4, seed=None,\n",
    "                               figsize=(12,4)):\n",
    "    dists = cross_sectional_dist(mc, T=Ts, init=init, num_reps=num_reps,\n",
    "                                 random_state=seed)\n",
    "\n",
    "    fig, axes = plt.subplots(1, len(Ts), figsize=figsize)\n",
    "    titles = ['T={0}'.format(T) for T in Ts]\n",
    "\n",
    "    for ax, title, dist in zip(axes, titles, dists):\n",
    "        draw_histogram(dist, ax=ax, title=title, xlabel='States')\n",
    "\n",
    "    fig.suptitle('Cross sectional distributions with init={0}'.format(init),\n",
    "                 y=-0.05, fontsize=12)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAE7CAYAAAA7EoKDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu87HVd7/HXO7Z4R1TUAlTSMC+lUYaoqUvt5PZy5GSa\nQl5IUzpm2rFzIsqTO+3G4zxKM3uYFqknFbwr5oUyXV4gURJRFBCOohtIVEBAMAP35/wxvwXDYq01\ns/b+zVrzW9/X8/GYB3P5zff3nVnvPbzXb74zK1WFJEmStNX90GZPQJIkSdoIFl9JkiQ1weIrSZKk\nJlh8JUmS1ASLryRJkppg8ZUkSVITLL6SJElqgsV3hpJ8N8lV3WlXkmvGLh+xjnGeleT0JFck2Znk\nuCR7zXLualePuX1aknO63F6S5A1JbjvLuattPWb3qCQ/GLvvVUkePsu5qy09ZvUnkpyc5FtJdq1w\n+x2SvLvb3wXrGXurin/AYmMk+SrwnKr6yG7c99eBLwCnAXcGTgLeXlXH9TtL6cb2MLd3Bb5fVd9M\ncmvgtcClVfWivucpLbeH2T0KeHZVWXY1c3uY1XsBDwUuBd5TVT+07PYTurPPAQ4B3g88pKq+tGez\nHq5tmz0BTVZVfzN28eIkbwYeuVnzkaZRVTvHLgbYBdxzk6YjrVc2ewLSJFX1ZeDLSX5s+W3dAYcn\nAferqmuAU5K8F3gGcOzGznR+uNRhEyU5Msnlq5wuS3LgKnd9BHDWRs5VWrKe3Cb5uSTfAa5k9AL8\nyk2buJo3IbuXj2W3gEO6t4/PTfISl5dpI+1BPxh3L+C6qjp/7LozgfvNZtbD4BHfTVRVbwHesp77\nJHk28NPAs2cyKWmC9eS2qj4J7Jtkf+C5wNdmOTdpLevI7scZHSX7WpKfAN4KXAf82SznJy3ZnX6w\ngtswOugw7iqg6c9aeMR3QJL8N+BPgMdW1WWbPR9pWlV1MfAh4MTNnos0SVV9taq+1p0/C3gZ8OTN\nnZW0bt8F9ll23e0Yld9mWXw3UZJfWfap4fHTlcveMt4OvA54QlV9cfNmrdatJ7fL3AzX+GoT7UF2\nwTW/2kB7mNUlXwa2LVv/+wAaXyrpUodNVFVvBt48abskj+q2O7yqTp/5xKQ1rCO3RwKfqKqdSe4O\n/DHw4VnPT1rNOrL7WOCzVXVJknsDLwHeNuv5SUumzSpAklsAe3fnb97d//tVdXWSdwEvS/JrjJZJ\n/lfgwbOZ9TB4xHcYXsJoTc4Hx37je/9mT0qa4L7AqUm+C3wSOJvROl9p3j0KOLPL7vuBdzJaZibN\nlSQHAdcwOopbwPcYvdYueT5wS+CbwJuAX6+qs2nYxO/xTfL3wOOBb1bVT66yzauAxzJ68o+qqjP6\nnqi0XmZXQ2RuNVRmV0MwzRHf1wPbV7sxyeOAH6uqg4HnAa/paW7SnjK7GiJzq6Eyu5p7E4tvVX0C\nuHyNTZ4IvLHb9jRGX110l36mJ+0+s6shMrcaKrOrIehjje8BwPhfaLoQmObThtJmM7saInOroTK7\n2nR9favD8q95ucnC4SRrLyaWplRVfX6tkNnVhjC3Giqzq6FaKbt9HPG9CLjr2OUDu+tWmsDUp5e+\n9KXr2n69p1mPvxUewzyO3zOzuwnjb4XHsN7xze18/lxaG3939mF25/Pn4viTT6vpo/ieBDwTIMlh\nwHeq6pIexpVmzexqiMythsrsatNNXOqQ5ATgEcB+SXYCL2X0F5ioqtdW1QeSPC7J+cDVwK/OcsLS\ntMyuhsjcaqjMroZgYvGtqiOm2OYF/UznBgsLC30PuaHjb8Q+HH9tZnc+x9+IfQx5/K2a243Yh+Nv\n7j62anaH/nNx/Bub+AcsettRUhu1L21dSah+P2gxzT7NrvaIudVQmV0N1WrZ9U8WS5IkqQkWX0mS\nJDXB4itJkqQmWHwlSZLUBIuvJEmSmmDxlSRJUhMsvpIkSWqCxVeSJElNsPhKkiSpCRZfSZIkNcHi\nK0mSpCZYfCVJktQEi68kSZKaYPGVJElSEyy+kiRJaoLFV5IkSU2w+EqSJKkJFl9JkiQ1weIrSZKk\nJlh8JUmS1ASLryRJkppg8ZUkSVITLL6SJElqgsVXkiRJTbD4SpIkqQkWX0mSJDXB4itJkqQmWHwl\nSZLUBIuvJEmSmmDxlSRJUhMsvpIkSWqCxVeSJElNsPhKkiSpCRZfSZIkNcHiK0mSpCZMLL5Jtic5\nJ8l5SY5Z4fbbJXlfks8lOSvJUTOZqbROZldDZXY1ROZWQ5CqWv3GZC/gXODngYuAzwBHVNXZY9v8\nHnDbqjo2yX7d9nepquuWjVVr7UuaRhKqKlNsZ3Y1N6bNbbdtL9k1t+qDr7kaqtWyO+mI76HA+VV1\nQVVdC5wIHL5sm13APt35fYBLl4dY2gRmV0NldjVE5laDMKn4HgDsHLt8YXfduFcD901yMXAm8KL+\npiftNrOroTK7GiJzq0HYNuH2ad5r2A58tqoemeSewD8neUBVXbV8wx07dlx/fmFhgYWFhXVMVS1a\nXFxkcXFxd+5qdrVp9iC30GN2za3Wy9dcDdW02Z20xvcwYEdVbe8uHwvsqqrjxrb5R+BPq+qU7vK/\nAMdU1enLxnLNjvbYOtabmV3NjXWu8e0lu+ZWffA1V0O1u2t8TwcOTnJQkr2BpwInLdvm64wWs5Pk\nLsCPA1/Z8ylLe8TsaqjMrobI3GoQ1lzqUFXXJXkBcDKwF3B8VZ2d5Oju9tcCLwfekOTzQIDfqarL\nZjxvaU1mV0NldjVE5lZDseZSh1535FsX6sF63jLucZ9mV3vE3GqozK6GaneXOkiSJElbgsVXkiRJ\nTbD4SpIkqQkWX0mSJDXB4itJkqQmWHwlSZLUhEl/sliS5kbS/7cq+bVJktQOi6+kgemzqG7o15NK\nkjaZSx0kSZLUBIuvJEmSmmDxlSRJUhMsvpIkSWqCxVeSJElNsPhKkiSpCRZfSZIkNcHiK0mSpCZY\nfCVJktQEi68kSZKaYPGVJElSEyy+kiRJasK2zZ6AJElbXZJex6uqXseTWmHxlSRpQ/RVVvst0VJL\nXOogSZKkJlh8JUmS1ASLryRJkprgGl9JkiTdRN8fyoTN/2CmxVeSJEmr6LOobv4HM13qIEmSpCZY\nfCVJktQEi68kSZKaYPGVJElSEyy+kiRJaoLFV5IkSU2w+EqSJKkJFl9JkiQ1YWLxTbI9yTlJzkty\nzCrbLCQ5I8lZSRZ7n6W0G8yuhsrsaojMrYYga/3puCR7AecCPw9cBHwGOKKqzh7bZl/gFOAxVXVh\nkv2q6tsrjFWb/WfqNHxJqKqJf/rF7G5Noz+f2e9fEdqIn+20ue227SW75na+9Jvdjckt+JrbuqG+\n5sLq2Z10xPdQ4PyquqCqrgVOBA5fts2RwDur6kKAlUIsbQKzq6Eyuxoic6tBmFR8DwB2jl2+sLtu\n3MHAHZJ8NMnpSZ7R5wSl3WR2NVRmV0NkbjUI2ybcPs3x6JsBPw08GrgV8K9JPlVV5y3fcMeOHdef\nX1hYYGFhYeqJqk2Li4ssLi7uzl3NrjbNHuQWesyuudV6+ZqroZo2u5PW+B4G7Kiq7d3lY4FdVXXc\n2DbHALesqh3d5b8DPlRV71g2lmt2tMfWsd7M7G5BQ11vts41vr1k19zOlwbW+PqauwUN9TUXdn+N\n7+nAwUkOSrI38FTgpGXbvBf4uSR7JbkV8CDgS31MWtoDZldDZXY1ROZWg7DmUoequi7JC4CTgb2A\n46vq7CRHd7e/tqrOSfIh4PPALuBvq8oga1OZXQ2V2dUQmVsNxZpLHXrdkW9dqAfrecu4x32a3Tkx\n1LfdzK22+lKHnvdpdufEUF9zYfeXOkiSJElbgsVXkiRJTbD4SpIkqQkWX0mSJDXB4itJkqQmWHwl\nSZLUBIuvJEmSmmDxlSRJUhMsvpIkSWqCxVeSJElNsPhKkiSpCRZfSZIkNcHiK0mSpCZYfCVJktQE\ni68kSZKaYPGVJElSEyy+kiRJaoLFV5IkSU3YttkTmIUkvY5XVb2Ot9n6fn5g6z1HkiRp69mSxXek\nryLWf0mcD30W1a36HEmSpK3EpQ6SJElqgsVXkiRJTbD4SpIkqQkWX0mSJDXB4itJkqQmWHwlSZLU\nBIuvJEmSmmDxlSRJUhMsvpIkSWqCxVeSJElNsPhKkiSpCRZfSZIkNcHiK0mSpCZYfCVJktQEi68k\nSZKaYPGVJElSEyYW3yTbk5yT5Lwkx6yx3c8muS7Jk/qdorR7zK6GyuxqiMythmDN4ptkL+DVwHbg\nvsARSe6zynbHAR8CMoN5SutidjVUZldDZG41FJOO+B4KnF9VF1TVtcCJwOErbPebwDuAb/U8P2l3\nmV0NldnVEJlbDcKk4nsAsHPs8oXddddLcgCjcL+mu6p6m520+8yuhsrsaojMrQZh24TbpwnlK4Hf\nrapKEtZ462LHjh3Xn19YWGBhYWGK4dWyxcVFFhcXd+euZlebZg9yCz1m19xqvXzN1VBNm91UrZ7V\nJIcBO6pqe3f5WGBXVR03ts1XuCG8+wHXAM+tqpOWjVVr7atPo39Pfe0rbNS8N0q/zw9s5HOUhKqa\nuC5sqNnV2oaa3Wlz223bS3bN7XwZ6v+XfM1t21Bfc2H17E4qvtuAc4FHAxcDnwaOqKqzV9n+9cD7\nqupdK9xm8Z0TWzHIK2w3yOxqbUPN7jqLby/ZNbfzZaj/X/I1t21Dfc2F1bO75lKHqrouyQuAk4G9\ngOOr6uwkR3e3v3Yms5X2kNnVUJldDZG51VCsecS31x15xHdubMXf4Ga8T48+zImhZtfcaqj/XzK7\nbRvqay6snl3/cpskSZKaYPGVJElSEyy+kiRJaoLFV5IkSU2w+EqSJKkJFl9JkiQ1weIrSZKkJlh8\nJUmS1ASLryRJkppg8ZUkSVITtm32BCRJkjbD6E/y9sc/tTz/LL6SJKlhfZXVfku0ZsOlDpIkSWqC\nxVeSJElNcKmDJGmPuE5S0lBYfCVJPXCdpKT551IHSZIkNcHiK0mSpCZYfCVJktQEi68kSZKaYPGV\nJElSEyy+kiRJaoLFV5IkSU2w+EqSJKkJFl9JkiQ1weIrSZKkJlh8JUmS1ASLryRJkpqwbbMnIGkk\nSe9jVlXvY0qSNFQWX2mu9FlU+y/SkiQNmUsdJEmS1ASLryRJkppg8ZUkSVITLL6SJElqgsVXkiRJ\nTbD4SpIkqQlTFd8k25Ock+S8JMescPuvJDkzyeeTnJLk/v1PVVofc6uhMrsaKrOreTex+CbZC3g1\nsB24L3BEkvss2+wrwMOr6v7Ay4HX9T1RaT3MrYbK7GqozK6GYJojvocC51fVBVV1LXAicPj4BlX1\nr1V1RXfxNODAfqcprZu51VCZXQ2V2dXcm6b4HgDsHLt8YXfdap4DfGBPJiX1wNxqqMyuhsrsau5N\n8yeLp/4bqkkeCTwbeOhKt+/YseP68wsLCywsLEw7tBq1uLjI4uLi7ty1t9yC2dX67EFuwddcbSKz\nq6GaNrupWjunSQ4DdlTV9u7yscCuqjpu2Xb3B94FbK+q81cYpybtqy9JWMe/v0mjsVHz3ij9Pj+w\nkc9REqoqU2zXS267bTYku0P+uWyUoT5H0+a229bX3C2WWxjuc2R21zXalsvuUF9zYfXsTrPU4XTg\n4CQHJdkbeCpw0rLB78YoxE9frTxIG8zcaqjMrobK7GruTVzqUFXXJXkBcDKwF3B8VZ2d5Oju9tcC\nfwDcHnjN6LcDrq2qQ2c3bWlt5lZDZXY1VGZXQzBxqUNvO/Kti7mxFd+6mPE+XeowJ4b6HG3l3Hb7\nwtfctQ31OTK76xpty2V3qK+5sGdLHSRJkqTBs/hKkiSpCRZfSZIkNcHiK0mSpCZYfCVJktQEi68k\nSZKaYPGVJElSEyy+kiRJaoLFV5IkSU2w+EqSJKkJFl9JkiQ1weIrSZKkJlh8JUmS1ASLryRJkppg\n8ZUkSVITLL6SJElqgsVXkiRJTbD4SpIkqQkWX0mSJDXB4itJkqQmWHwlSZLUBIuvJEmSmmDxlSRJ\nUhMsvpIkSWqCxVeSJElNsPhKkiSpCRZfSZIkNcHiK0mSpCZYfCVJktQEi68kSZKaYPGVJElSEyy+\nkiRJaoLFV5IkSU2w+EqSJKkJFl9JkiQ1YWLxTbI9yTlJzktyzCrbvKq7/cwkh/QxscXFxT6GWWsP\nMx5/9o9h6M/RrOdvdndz9JnP3+doks3I7kb83If+cxl6bmG2j8HX3D3Yg9lde/Qe579m8U2yF/Bq\nYDtwX+CIJPdZts3jgB+rqoOB5wGv6WNiQ/8hgUGeOPpsX4DN7u6ObvGdPPoWzK7Fd/PHH/L/l3zN\n3cM9mN21R9+o4gscCpxfVRdU1bXAicDhy7Z5IvBGgKo6Ddg3yV16m6G0e8yuhsrsaojMrQZhUvE9\nANg5dvnC7rpJ2xy451OT9ojZ1VCZXQ2RudUwVNWqJ+CXgL8du/x04K+WbfM+4KFjlz8M/PQKY5Un\nT32c1sqs2fU0r6dpcttndjf78XraOidfcz0N9bRSVrextouAu45dviuj39DW2ubA7robqapM2JfU\nJ7Oroeolu+ZWG8zXXA3CpKUOpwMHJzkoyd7AU4GTlm1zEvBMgCSHAd+pqkt6n6m0PmZXQ2V2NUTm\nVoOw5hHfqrouyQuAk4G9gOOr6uwkR3e3v7aqPpDkcUnOB64GfnXms5YmMLsaKrOrITK3Gop062kk\nSZKkLW0u/3LbNF+CvQdj/32SS5J8oc9xx8a/a5KPJvlikrOSvLDn8W+R5LQkn+vG39Hn+GP72SvJ\nGUneN6PxL0jy+W4fn57FPjbaLHPbjW92p9uP2V0nsztx/MFndyvmFuwLU+zD7C437SeNN+rE6C2S\n84GDgJsBnwPu0+P4DwMOAb4wo/n/MPBT3fnbAOf2Of9u3Ft1/90GfAp40Awex4uBNwMnzeh5+ipw\nh43M1ixPs85ttw+zO90+zO76Ho/ZnW4fg87uVstt95jsC9Ptx+yOnebxiO80X4K926rqE8DlfY23\nwvjfqKrPdee/C5wN7N/zPq7pzu7N6B/7rj7HT3Ig8Djg74BZfrp2K31yd6a5BbM7DbO7W8zudPvY\nCtndSrkF+8K0+zG7Y+ax+E7zJdiDkOQgRr8tntbzuD+U5HPAJcA/VdVn+hwfeAXwv+j5H8cyBXw4\nyelJnjvD/WyULZNbMLsTmN05ZnZXtdVyC1sou7PKbTe22R0zj8V3S3zaLsltgHcAL+p+k+tNVe2q\nqp9i9B2ID0pyv77GTvIE4JtVdQazPTrw0Ko6BHgs8BtJHjbDfW2ELZFbMLtTMLtzyuyuaavlFrZI\ndmeZWzC7y81j8Z3mS7DnWpKbAe8E3lRV75nVfqrqCuCjwPYeh30I8MQkXwVOAB6V5P/2OD4AVfXv\n3X+/Bbyb0VtWQzb43ILZnYbZnU9md21bMLewBbK7UbkFs7tkHovvNF+CPbeSBDge+FJVvXIG4++X\nZN/u/C2B/8JoXVAvqur3ququVfWjwNOAj1TVM/saHyDJrZLctjt/a+AXgJl8anYDDTq3YHanYXbn\nk9ld2xbNLQw8u7PObbcPs7vM3BXfqroOWPoS7C8Bb62q3n5ISU4ATgXulWRnkr6/QPuhjP5G+SO7\nr944I0mfv139CPCRJGcCn2a0XucDPY6/3CzeSroL8IluzdFpwD9W1T/NYD8bZta5BbO7G8zuFMzu\nVIae3S2XW7AvTMnsLuMfsJAkSVIT5u6IryRJkjQLFl9JkiQ1weIrSZKkJlh8JUmS1ASLryRJkppg\n8ZUkSVITLL4zkOT3k5yV5Mzue/kOTfKi7sujJ933t6bZTuqbudVQmV0NldndeH6Pb8+SPBj4c+AR\nVXVtkjsAtwBOAR5YVZdOuP9Xp9lO6pO51VCZXQ2V2d0cHvHt3w8D366qawGq6jLgycD+wEeT/AtA\nktck+Uz3m96O7roXrrDdLyQ5Ncm/JXlb9yf7SPJnSb7Y/Zb4fzb8UWqrMbcaKrOroTK7m8Ajvj3r\ngvZJ4FbAhxn9CcWPd7+Z/UwXbJLcvqouT7JXt91vVtVZ49sl2Q94J7C9qr6X5Bhgb+CvgVOr6t7d\nWPtU1ZUb/mC1ZZhbDZXZ1VCZ3c3hEd+eVdXVwM8AzwO+Bbw1yVHdzRnb9KlJ/g34LHA/4L4rDHdY\nd/2pSc4AngncDbgC+I8kxyf5ReB7s3gsaoe51VCZXQ2V2d0c2zZ7AltRVe0CPgZ8LMkXgKOWbgJI\n8qPAbzNam3NFktczWtezkn+uqiOXX5nkUODRjN4WeUF3Xtpt5lZDZXY1VGZ343nEt2dJ7pXk4LGr\nDgEuAK4C9umu2we4GrgyyV2Ax45tP77dacBDk9yzG/vWSQ7u3h7Zt6o+CLwYeMCsHo/aYG41VGZX\nQ2V2N4dHfPt3G+CvkuwLXAecx+htjCOBDyW5qKoe3b0VcQ6wk9EanyWvW7bdUcAJSW7e3f77jML+\n3iS3YPR2yP/YiAemLc3caqjMrobK7G4CP9wmSZKkJrjUQZIkSU2w+EqSJKkJFl9JkiQ1weIrSZKk\nJlh8JUmS1ASLryRJkppg8ZUkSVITLL6SJElqgsVXkiRJTbD4SpIkqQkWX0mSJDXB4itJkqQmWHwl\nSZLUBIuvJEmSmmDxlSRJUhMsvpIkSWqCxVeSJElNsPhKkiSpCRZfSZIkNcHiK0mSpCZYfCVJktQE\ni68kSZKaYPGVJElSEyy+kuZKkrOSPHzG+zgoya4kU70GJnlDkpd35x+W5Jwe5/KBJM/ozh+V5BM9\njv0rSU7ua7w+JbkqyUFr3H5BkkdPOdbUj3OenxNJs2fxleZYkiOTnN6VhIu7kvTQzZ5XX8YL5ZKq\n+omq+vhmzWkV1Z2oqk9U1b0n3SHJjiT/MHHgqsdV1cTtptjfTcp8Vb25qh6zp2PPQlXdtqougJVz\nwNhzPsVYUz/O5dt2z9k9ppz29ZLcLMk7kny1G+MR6x1D0saz+EpzKsmLgVcAfwTcGbgr8NfAE1fZ\nfq+Nm12T0utgnT7HXBp6BmNudbv7nH0ceDrwDaYs6ZI2l8VXmkNJbgf8IfD8qnpPVX2vqn5QVe+v\nqmO6bXZ0R5z+IckVwLOS7J/kpCSXJjkvya+NjXlod/T4iiTfSPLn3fW3SPKmJN9OcnmSTye58yrz\nOibJhUmuTHJOkkd11yfJ7yY5vxvnrUluP3a/n0tyajf+15M8K8lzgSOB3+mOaL+32/b6t7iT3DzJ\nK5Nc1J1ekWTv7raFbi4vTnJJd0T8qLF9Pj7JGd3j/XqSl67j+T8kyWe7x3kicIux2xaS7FzrOUmy\nHTgWeGr32M7otl1M8kdJTgG+C9yju+45N959/irJd5KcvfQcL39uxjKwdLR46Sj5d7q5HJZlSyeS\nPCTJZ7qxP53kwWO3LSZ5WZJPdvc/Ockdu9umykiSX01y0tjl85K8bezyziT3787vSnLPJM9jhRx0\nDklyZjffE5PcfJWf1/LHuSvJ0Um+3M331Sttm2TpOTuz2/dTVhp/JVV1bVW9qqpOAX4w7f0kbS6L\nrzSfHsyobL17wnZPBN5eVbcD3gKcCHwd+BHgycCfJHlkt+1fAq/otr0H8Nbu+mcB+wAHAncAjga+\nt3xHSX4c+A3ggVW1D/ALwAXdzS/s5vLwbt+XMzo6TZK7Ax/o9r8f8FPA56rqb4E3A8d1b3sf3o01\n/hb37wOHAg/oTocCLxmb1l26ue8PPAf464x+aYBRsXx693gfD/z3JIczQVes3wO8Ebg98Hbgl1jh\niN5qz0lVfQj4E+DE7rEdMna3pwO/BtwW+Bo3fUv/QcD5wB2BlwLvSrLvCs8Ny84/rPvv7apqn6r6\n1LK53gF4P/BKRj/nvwDeP/4LCnAEcBSjdxj2Bv5nd/1UGQEWl+aRZH/gZsBh3eV7ALeuqs+Pz7+q\nXsfKOQjwFOAxwI8C9+/mNq3HAw/s7vfLSW6yFKKqltaS37/b99uT3K0ry6udnraOOUiaMxZfaT7d\nEfh2Ve2asN2pVbV0hO1OwEOAY6rqP6vqTODvgGd2t/8ncHCS/arqmqr69Nj1dwQOrpEzquqqFfb1\nA+DmwP2S3Kyqvl5VX+luOxp4SVVdXFXXMjpa/eSMll8cCfxzVb21O2p9WTe3JWu9zXwk8LKq+nZV\nfbsb9xljt1/b3f6Dqvogo7L74wBV9bGq+mJ3/guMfimYZh3mYcC2qvrLbtx3Ap9ZZdu1npOs8NgK\neENVnV1Vu6rquhXG/ObYvt8GnMuoxK0kq5xfyeOBc7s1rruq6kTgHG5YOlPA66vq/Kr6D+BtjH5J\ngSkzUlVfBa5KcgijX4JOBi7ufkF4BDcclZ70WJbm86qq+kZVXQ68b2w+0/izqrqyqnYCH532vt3P\n8PZrnE5cxxwkzRmLrzSfLgX2y+RvHbhw7Pz+wGVVdfXYdV8HDujOPwe4F3B291b1Upn6B0YF5cRu\nOcFxSbYt31FVnQ/8FrADuCTJCUl+pLv5IODdS0fFgC8B1zE6Insg8JXl401pf0ZHRccfz/5jly9d\n9svBNcBtAJI8KMlHk3wzyXcYlfM7TrnPi5Zd97WVNpzwnKxm54TbV9r3/ittuE77M3r+1hr7G2Pn\nv0f3XDJlRjofAxYYHfn9WHd6BKMi/LF1znm1+az3vtcAt17nviVtQRZfaT79K/B94BfX2Gb5294X\nA3dIMl4O7kZXjrsjeUdW1Z2A44B3JLllVV1XVS+rqvsxOmL8BG44SnzjHVadUFUPA+7e7fu47qav\nA9uXHRm7VVVdzKjo3XONx7CWixmV6vHHc/GE+yx5C6MlCwdW1b7A3zDda96/c8MvC0vuvtrGazwn\nqz22SY95pX0vPearuXGB++F1jHsRN30cd+emRfsm1pMRRuX2kYyK7yI3FOFHsHrxneaDYRvy4bFu\nqcNVa5yO2Ih5SJoNi680h6rqCuAPGK1ZPTzJrTL6+qTHJlkqVll2n53AqcCfZvShsPsDzwbeBJDk\n6Unu1G1+BaMisSvJI5P8ZLcs4SpGywdu8mGdJPfK6INbN2dUyv9jbLu/YbSe+G7dtndKsvQW+puB\nn0/ylCTHa/LuAAACKklEQVTbktwxyQO62y5htN54NScAL0myX5L9uudk2q/+ug1weVX9Z5JDGS2b\nmKY8nQpcl+SF3XP+JOBnV9pwwnPyDeCg5Cbf3DBpScKdx/b9FODejNZIA3wOeFr3PD6QG689/haw\ni9V/yfggcK8kR3T3f2o39j9Omtu0GeksFd9bdL/4fBLYzmht8Bmr3GdSDlad25RWWnYyvu/rn7Nu\nqcNt1zidcP2go39nSx98HD8vaU5ZfKU5VVV/AbyY0Ye5vsnoqOrzueEDbyt9z+kRjI6QXgy8C/iD\nqvpId9tjgLOSXMXoa9KeVlXfZ7Qc4e2MyvCXGB2lW6lc3hz4U0YF698ZfVDt2O62vwROAv4pyZWM\njlgf2j2OncDjgN9mtITjDEYfOAI4Hrhvt0TiXSvs84+A04HPd6fTu+uuf5pWuM+S5wMv6+bzv7nh\nw3xr3rdbo/wkRh+kuhT4ZeCdq9x3refk7d1/L01y+pRzLuBTwMHdmC8Hfqlb40r3OO7J6MODOxj9\nUrE072uAPwZOSXJZkgdx4+8fvpTRkdrfBr7N6INrT6iqy1aZ23i+ps0IVXUeo3L8ie7ylcD/A06p\nqtU+mDcpB8vnM+m25duN37582x3AG7t9P3mV8VdzLqNlFPszWgpy9dIvf5LmU278OiRJkiRtTR7x\nlSRJUhMsvpIkSWqCxVeSJElNsPhKkiSpCRZfSZIkNcHiK0mSpCZYfCVJktQEi68kSZKa8P8BJnGL\nLvQQ7R4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10dc79588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "init = 1\n",
    "Ts = [2, 3, 5, 10]\n",
    "plot_cross_sectional_dists(mc1, init, Ts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Starting with a transient state `2`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.5075,  0.1659,  0.    ,  0.0008,  0.3247,  0.0011])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "init = 2\n",
    "T = 10\n",
    "cross_sectional_dist(mc1, init=init, T=T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.5014,  0.1661,  0.    ,  0.    ,  0.3325,  0.    ])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "T = 100\n",
    "dist = cross_sectional_dist(mc1, init=init, T=T)\n",
    "dist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAEZCAYAAABYR6TIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGgJJREFUeJzt3X/UXFV97/H3hySASCBAFCQEqRoUKCC1jVSKPoCViC5Q\nW4vBH02lyl3eVCvee1OsSgRbpF2taLFIbcSCSrQqNbrQqDWDQBCI8kMk4SYXI/mBAUIMAYIk5Hv/\n2HuSwzzz60nmyWRPPq+1Zj0zs/ecs895znzOnn3OmVFEYGZmZdmj3w0wM7ORc3ibmRXI4W1mViCH\nt5lZgRzeZmYFcnibmRXI4V0oSfdIevUoz+MISVskdbWdSPqipIvz/ZMlLelhW66X9M58f4akG3s4\n7bdLmt+r6Vl7kg6XtEGS2tTZIulFXU7vAkmf73XdXd1AhLekcyQtyhvE6vxGP6nf7eqVaijWRcTv\nRsSP+9WmFiLfiIgbI+JlnV4gabakazpOOOKMiOhYr4v5DdshRcSXI+L0HZ32drSlJuncFmUn5+15\ng6THc5vrjx+TdNgI5vNnkhZKekLSgiblL5f001y+SNLxDeUflPSgpPWS5kjac+RLu01EPBAR4yNf\nZNJuPXQ5vUsi4j0jrTvSzkmVpDdIuknSurxuPi9p35FOZ0cUH96Szgc+BXwCeD4wGfgscGaL+mN2\nXut2Sy17U9s1sayX06xPehSmOVItr5DLO7/xETEeOCY/vX9+br+IWDmC+awF/hn4ZGNBDuJvAVcD\nE4D/AL4laVwuPx2YBZwKvBB4EfDxEcy7G/2+UnB7toX9gIuAFwBHAZOAf+xlozqKiGJvwP7ABuBP\n2tSZDXwduAZYD7wbOBSYR9qolwJ/Wak/FViU6/4a+Kf8/N7Al4BHgHXAbcDzW8xzFrASeAxYApxa\n72QAfwMsy9P5KnBA5XV/BCzM038A+HPgPcDTwG/zsn4r110OnJbv7wVcBqzKt08Be+ayodyW84E1\nwGpgRmWebwDuyMv7AHBhpewIYAuwR4vlPAH4WV7OucC1wMWV+a5ot06AaXm5ns7LdkeuWyPtjG8G\nngBenJ87N5fPAG4C/gX4DbC4vo4b101lG7gm338gL9OG3JYT8/RurNR/FXB7nvZtwB9WymqkN+1N\n+fXzgYNarJ8JwHeAh4BHgW8Dk3LZ3wGbgY25LZ9psw23/T+M4P3yl8CChudeB6xseO5XwOvy/a8A\nn6iUnQI82GL6H68vBzAu/+/+IT9+DvBUXif15RnTaj3k8vOA/0t6P1ze4T1e///Wp/2uvBwPAx/u\nYlvYALxyB9btm4G7d+T/M+J57syZ9bzx6c2/qd1Gnf9ZTwNn5sd7Az8GLgf2BI7Pb65TcvktwNvz\n/X2Aqfn+eaTA35sUwicA45vM76V5ozgkPz4ceFG+/wFSOB+aN+7PAV/JZS8khcHZeaM+EDg+l10F\nXNQwn1+ybadwUZ7uxHy7uV6fFKKb8noYA7w+v6n2z+WvAY7J948l7bDOangjDFu/ed39Ki/TGOBP\n8nquzndFF+vkQuDqhmnXSAF8FOnT4VhgAfDuXD4jL1N93n9GCtoJjeumMo9rKuv5WctEJbzzel8H\nvD3P+22k4D2g0ralwEvytrAAuKTFtncg6U29N7Av8DXgukr51mXqsJ03/T8A/5rb2ux2Z5PpNAvv\nDwLXNzw3D/hgvn8n8NZK2UG5LQc0mf4p5AAj7QCXAT/Jj09l2875WcvTbD3k8nmkHu5k0nv09Bbr\np/r/rU/7SlKn5jjSTuOlXW4Lf9Rmna4DXtWiDZeR38s761b6sMlBwCMRsaVDvYURMS/ffx5pw5oV\nEU9HxF3Av5P21JACaIqkiRHxZETcVnn+IGBKJHdExIYm83qGtNEcI2lcpPG9+3PZecBHImJ1RGwi\n9VT+NA/lnAP8ICK+GhHPRMSjuW117T7anUMKzUci4pE83XdWyjfl8mci4rvA46RAJSJuiIhf5Ps/\nJ/WgX9NmXnUnAmMj4tN5ut8g9VababdO1GTZAvhiRCyOiC0RsbnJNB+qzPtrwH2kTxHNqMX9Zt4A\n3BdpHHxLRMwlfVKoD8MFcFVELIuIp0iB/PJmE8r/w+si4qmIeBz4e4av2+0evomI90XEAS1uTdvU\nxL6kT11VjwHjW5Q/lv+OZ7ifkN47BwInA3OASZKeS1ruG9q0o9l6+GREPBYRK0gB32qZmr324xHx\n24i4G7iL1ElrrDvsdRFxU5t1ekBELBw2c+mPSfnxsTbL13Olh/daYGIXBxyq44OHAo9GxBOV5x4g\njVkBnAscCSyWdJukeiBcQ/qIPFfSKkmXShrbOKOIWAb8Namnu0bStZJekIuPAK7LBznWAfeSPjIe\nDBwG3N84vS4dSuoFV5fn0MrjtQ07uCdJb0okvVLSAkkPSfoNaQdzUJfzXNXw3K+aVeywTlpZ0aG8\n2bwPbVZxhA4lrb920/515f5G8rpsJGkfSVdKWi5pPSm89m8Yw+/3eO8GUu+2akJ+HtKOvlq+f+V1\nzxIRG0lDjq8BXk1a3oXASZXHrTRbD9X1vHWb7dKOvLZrkk4Evkwaul02GvNopfTwvoU0ZvrmNnW2\nngGRrQYObDgyfDg54HOP6pyIeB5wKfB1Sc+JiM0RcVFEHEPqub+Rbb31Z88w4tqIOJn0sSzydCCF\nwrSGPfk+EbGaFFYvbrMM7awm7Riqy7O6w2vqvgL8F3BYREwgDeV0s108yLYdXt0LW1Vus05aLVun\nZW427/oyPwE8t1J2yAimu4rhy/FChu8suvEhUkdgakTsTwq16ieNHQpuSZ+rnIHSePt5k5c0m98v\nSEMLVcfm5+vl1R7v8cCaiFjXolk3AKeRhhVvz4+nkY4ltTo7akd3YNv7+mGvazjLp9ntpErdE0gH\ne2dExILtbMN2Kzq8I2I96aPKZyWdlXs64yS9XlI9HNTwmhWk3sAlkvaSdBzpIOaXACS9Q9LzcvX1\npH/wFkmnSDo2D3FsIA1FPNPYJklHSjpV0l6kHctTlXqfA/5e0uG57vMk1T+Ofxl4raS3Shor6aDK\nKVtrSEf5W7kW+IikiZIm5nXS7Wl1+wLrIuJpSVNJQzDdvBkWApslvT+v87cAf9CsYod18mvgiCZn\nlHQaTnh+Zd5vBV4GXJ/L7gTeltfj75PG4+vL9DBpnLPVjvK7wJGSpufXn52n/Z0RtK1uX1LPfH0e\nSriwoXxNm3Z0FBH/I/IZKU1ux25trLSHpL1Jx1n2yNv9uFxcA57J63IvSe8nrZ8f5fKrgXMlHSXp\nAOCjpGMwrdxA6tT8Ig8N1khj7fdHxNoWr+lmPbRb59s79DRsW4jKWT4tbjcDSPpd4HvAzIi4vvnk\nR1fR4Q0QEf9MOpPiI6SDGg8A7wOuq1dheBhNJ/VUVwPfBD4WEfWN9XTgHkkbSGdtvC0ifksa2vhP\nUqDfS9oomwXkXsAlpA3jQdIBxAty2adJB2G+L+kx0ieHqXk5VgBnkHpra0lngNR7RHOAo/Nwyzeb\nzPMTpI+rd+fbovzc1tXU5DV17wMuyu35KOkMmKqmr81vzLeQDvatJR00/EaL17ZbJ/+Z/66VtKjL\nNgd5fDVP82LSx9Z6b/CjpDfkOtJQzZcr7X6SdIbDzZIelfRKnn1++lrSp6oPkc4I+l/AGyPi0RZt\na7Z91V1GOsviEdLO7rsNdT9NOubxqKTL2ixv4zxH6l2koYN/JY1FbyQd0Kv/H9+U66zLf99UP84Q\nEfOBfyCNOS8H/h/Dd0JVt7DtpABIZwJtZHive6Trod16bizrtO3U/9fVbWFd7rx063zS8OIXOnza\nGTWKaL9NSPoC6SDOQ9W9eUOdz5DOYniS9BHijl431MzMtumm530VacyqKUlnAC+JiCnAe4EretQ2\nMzNroWN4R8SNpI9TrZxJuiqLiLgVmCDp4N40z8zMmunFmPcknn1a10rSaW9mZjZKenXAstlFFmZm\nNkqGXWSyHVaRLl+tO4wm58RKcqCbmW2HiBh2OmQvet7zyBer5KuNfhMRa1o0YKfdLrzwwp06v519\nG+TlG+Rl8/KVf9vZy9dKx563pGtJV4ZNlLSCdI7nuBzGV0bE9ZLOkLSMdGXbX4ww/M3MbIQ6hndE\nTO+izszeNMfMzLpR/BWWrQwNDfW7CaNqkJdvkJcNvHyl21WWr+MVlj2bkRQ7a15mZoNCEjFKByzN\nzGwnc3ibmRXI4W1mViCHt5lZgRzeZmYFcnibmRXI4W1mViCHt5lZgRzeZmYFcnibmRXI4W1mViCH\nt5lZgRzeZmYFcnibmRXI4W1mViCHt5lZgRzeZmYFcnibmRXI4W1mViCHt5lZgRzeZmYFcnibmRXI\n4W1mViCHt5lZgRzeZmYFcnibmRXI4W1mViCHt5lZgRzeZmYFcnibmRXI4W1mViCHt5lZgRzeZmYF\ncnibmRWoY3hLmiZpiaSlkmY1Kd9f0rcl3SnpHkkzRqWlZma2lSKidaE0BrgPeC2wCrgdmB4Riyt1\nPgyMj4gLJE3M9Q+OiM0N04p28zIzs+EkERFqfL5Tz3sqsCwilkfEJmAucFZDnS3Afvn+fsDaxuA2\nM7Pe6hTek4AVlccr83NVlwNHS1oN3AV8oHfNMzOzZsZ2KO9mnGMa8LOIOEXSi4EfSDo+IjY0Vpw9\ne/bW+0NDQwwNDY2gqWZmg69Wq1Gr1TrW6zTmfSIwOyKm5ccXAFsi4tJKne8Al0TEzfnxfwOzImJR\nw7Q85m1mNkLbO+a9CJgi6QhJewJnA/Ma6jxAOqCJpIOBlwL373iTzcyslbbDJhGxWdJMYD4wBpgT\nEYslnZfLrwQuBr4o6W5AwP+JiEdHud1mZru1tsMmPZ2Rh03MzEZse4dNzMxsF+TwNjMrkMPbzKxA\nDm8zswI5vM3MCuTwNjMrkMPbzKxADm8zswI5vM3MCuTwNjMrkMPbzKxADm8zswI5vM3MCuTwNjMr\nkMPbzKxADm8zswI5vM3MCuTwNjMrkMPbzKxADm8zswI5vM3MCuTwNjMrkMPbzKxADm8zswI5vM3M\nCuTwNjMrkMPbzKxADm8zswI5vM3MCuTwNjMrkMPbzKxADm8zswI5vM3MCuTwNjMrkMPbzKxAHcNb\n0jRJSyQtlTSrRZ0hSXdIukdSreetNDOzZ1FEtC6UxgD3Aa8FVgG3A9MjYnGlzgTgZuD0iFgpaWJE\nPNJkWtFuXmZmNpwkIkKNz3fqeU8FlkXE8ojYBMwFzmqocw7wjYhYCdAsuM3MrLc6hfckYEXl8cr8\nXNUU4EBJCyQtkvTOXjbQzMyGG9uhvJtxjnHA7wGnAfsAt0j6SUQs3dHGmZlZc53CexUwufJ4Mqn3\nXbUCeCQiNgIbJf0YOB4YFt6zZ8/een9oaIihoaGRt9jMbIDVajVqtVrHep0OWI4lHbA8DVgN3Mbw\nA5YvAy4HTgf2Am4Fzo6Iexum5QOWZmYj1OqAZdued0RsljQTmA+MAeZExGJJ5+XyKyNiiaTvAXcD\nW4DPNwa3mZn1Vtued09n5J63mdmIbe+pgmZmtgtyeJuZFcjhbWZWIIe3mVmBHN5mZgVyeJuZFcjh\nbWZWIIe3mVmBHN5mZgVyeJuZFcjhbWZWIIe3mVmBHN5mZgVyeJuZFcjhbWZWIIe3mVmBHN5mZgVy\neJuZFajTr8ePOmnYr/vs8vxzbmbWb30P76SkMCxvZ2Nmg8fDJmZmBXJ4m5kVyOFtZlYgh7eZWYEc\n3mZmBXJ4m5kVyOFtZlYgh7eZWYEc3mZmBXJ4m5kVyOFtZlYgh7eZWYEc3mZmBXJ4m5kVyOFtZlYg\nh7eZWYE6hrekaZKWSFoqaVaben8gabOkt/S2iWZm1qhteEsaA1wOTAOOBqZLOqpFvUuB7+GfmjEz\nG3Wdet5TgWURsTwiNgFzgbOa1Psr4OvAwz1un5mZNdEpvCcBKyqPV+bntpI0iRToV+SnSvpBSjOz\nInUK726C+DLgbyL9pLrwsImZ2ajr9Ovxq4DJlceTSb3vqlcAcyUBTAReL2lTRMxrnNjs2bO33h8a\nGmJoaGjkLTYzG2C1Wo1ardaxnlKHuUWhNBa4DzgNWA3cBkyPiMUt6l8FfDsivtmkLJrNK4V+SSMt\not06MzPrJUlExLARjbY974jYLGkmMB8YA8yJiMWSzsvlV45Ka83MrK22Pe+ezsg9bzOzEWvV8/YV\nlmZmBXJ4m5kVyOFtZlYgh7eZWYEc3mZmBXJ4m5kVyOFtZlYgh7eZWYEc3mZmBXJ4m5kVyOFtZlag\nTl8Ja2YDKn+Nc1H8vULbOLzNdmslhWF5O5vR5GETM7MCObzNzArk8DYzK5DD28ysQA5vM7MCObzN\nzArk8DYzK5DD28ysQA5vM7MCObzNzArk8DYzK5DD28ysQA5vM7MCObzNzArk8DYzK5DD28ysQA5v\nM7MCObzNzArk8DYzK5DD28ysQA5vM7MCObzNzArk8DYzK1BX4S1pmqQlkpZKmtWk/O2S7pJ0t6Sb\nJR3X+6aamVldx/CWNAa4HJgGHA1Ml3RUQ7X7gVdHxHHAxcC/9bqhZma2TTc976nAsohYHhGbgLnA\nWdUKEXFLRKzPD28FDuttM83MrKqb8J4ErKg8Xpmfa+Vc4PodaZSZmbU3tos60e3EJJ0CvBs4qVn5\n7Nmzt94fGhpiaGio20mbme0WarUatVqtYz1FtM9mSScCsyNiWn58AbAlIi5tqHcc8E1gWkQsazKd\naDYvSYxg/7ALEJ3WmVkJ/N4rgyQiQo3PdzNssgiYIukISXsCZwPzGiZ+OCm439EsuM3MrLc6DptE\nxGZJM4H5wBhgTkQslnReLr8S+BhwAHBF2puzKSKmjl6zzcx2bx2HTXo2Iw+bmO1S/N4rw44Mm5iZ\n2S7G4W1mViCHt5lZgbo5z9u2Uz54W5zdcVzRrDQO71FXWhCWucMx29142MTMrEAObzOzAjm8zcwK\n5PA2MyuQw9vMrEAObzOzAjm8zcwK5PA2MyuQw9vMrEAObzOzAjm8zcwK5PA2MyuQw9vMrEAObzOz\nAjm8zcwK5PA2MyuQw9vMrEAObzOzAjm8zcwK5PA2MyuQw9vMrEAObzOzAjm8zcwK5PA2MyuQw9vM\nrEAObzOzAjm8zcwK5PA2MyuQw9vMrEAObzOzAnUMb0nTJC2RtFTSrBZ1PpPL75J0Qu+baWZmVW3D\nW9IY4HJgGnA0MF3SUQ11zgBeEhFTgPcCV4xSW0eo1u8GjLJavxswamq1Wr+bMKoGffkGeduEXef/\n16nnPRVYFhHLI2ITMBc4q6HOmcB/AETErcAESQf3vKUjVut3A0ZZrd8NGDW7yptjtAz68g3ytgm7\nzv+vU3hPAlZUHq/Mz3Wqc9iON83MzFrpFN7R5XS0na8zM7PtoIjWOSvpRGB2REzLjy8AtkTEpZU6\nnwNqETE3P14CvCYi1jRMy4FuZrYdIqKxg8zYDq9ZBEyRdASwGjgbmN5QZx4wE5ibw/43jcHdauZm\nZrZ92oZ3RGyWNBOYD4wB5kTEYknn5fIrI+J6SWdIWgY8AfzFqLfazGw313bYxMzMdk0Dd4VlNxcV\nlUrSFyStkfTzfrdlNEiaLGmBpF9IukfS+/vdpl6StLekWyXdmZdvdr/b1GuSxki6Q9K3+92W0SBp\nuaS78zLe1te2DFLPO19UdB/wWmAVcDswPSIW97VhPSLpZOBx4OqIOLbf7ek1SYcAh0TEnZL2BX4K\nvGlQ/n8AkvaJiCcljQVuAj6Qr48YCJLOB14BjI+IM/vdnl6T9EvgFRHxaL/bMmg9724uKipWRNwI\nrOt3O0ZLRPw6Iu7M9x8HFgOH9rdVvRURT+a7ewLjgC19bE5PSToMOAP4d4afPjxIdollG7Tw7uai\nIitAPsPpBGBgeqUAkvaQdCewBvh+RNze7zb10KeA/80A7ZCaCOCHkhZJek8/GzJo4T04Y0C7sTxk\n8nXSkMLj/W5PL0XEloh4Oekq5FdKOqbfbeoFSW8EHoqIO9hFeqaj5KSIOAF4PfA/81BmXwxaeK8C\nJlceTyb1vq0QksYB3wC+FBH/1e/2jJaIWA8sIH3p2yB4FXBmHhO+FjhV0tV9blPPRcSD+e/DwHWk\nodq+GLTw3npRkaQ9SRcVzetzm6xLkgTMAe6NiMv63Z5ekzRR0oR8/znAH5PG9YsXER+OiMkR8TvA\n24AfRcS7+t2uXpK0j6Tx+f5zgdcBfTvza6DCOyI2k672nA/cC3x1wM5UuBZYCBwpaYWkQbsg6iTg\nHcAp+VSsOyQNSs8U4AXAjyTdBdxGGvO+vs9tGi2DOIR5MHBjPmZxK/CdiPh+vxozUKcKmpntLgaq\n521mtrtweJuZFcjhbWZWIIe3mVmBHN5mZgVyeJuZFcjhbQND0t/mr1q9K58jPlXSB/IFMZ1e+9fd\n1DPbVfg8bxsIkv4Q+CfS76duknQgsDdwM/D7EbG2w+t/2U09s12Fe942KA4BHslfBUz+vuU/JX2l\n7AJJ/w0g6QpJt1d/DCH/6ENjvddJWijpp5K+li+HRtIn849F3CXpH3f6Uppl7nnbQMjhehOwD/BD\n0lcj/Ljxy/MlHRAR6/IPd/wQ+KuIuKdaT9JE0pdjTYuIjfkXmfYEPgssjIiX5WntFxGP7fSFNcM9\nbxsQEfEE6Rdc3gs8DHxV0oxcXP2K0rMl/RT4GXAMcHSTyZ2Yn18o6Q7gXcDhwHrgKUlzJL0Z2Dga\ny2LWjba/Hm9WkojYAtwA3JB/53NGvQhA0u8AHyKNba+XdBVpXLyZH0TEOY1PSpoKnEYakpmZ75vt\ndO5520CQdKSkKZWnTgCWAxuA/fJz+wFPAI9JOpj0hfp11Xq3AidJenGe9nMlTclDMxMi4rvA+cDx\no7U8Zp24522DYl/gX/L3ZW8GlpKGUM4BvidpVUSclodBlpB+Lu+myuv/raHeDOBaSXvl8r8lBfy3\nJO1NGor54M5YMLNmfMDSzKxAHjYxMyuQw9vMrEAObzOzAjm8zcwK5PA2MyuQw9vMrEAObzOzAjm8\nzcwK9P8BkTp/IuqMeloAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10fd4ba58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "draw_histogram(dist,\n",
    "               title='Cross sectional distribution at T={T} with init={init}'\n",
    "                     .format(T=T, init=init),\n",
    "               xlabel='States')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Observe that the distribution is close to a convex combination of\n",
    "the stationary distributions `(1, 0, 0, 0, 0, 0)` and `(0, 1/3, 0, 0, 2/3, 0)`,\n",
    "which is a stationary distribution itself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How the simulated marginal distribution evolves:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAE7CAYAAAA7EoKDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu4tHVd7/H3pwcQT4iKUoBKKqZYGmWIkrrUdj4etuxM\nQ8gDeaJtpG3bRZTF46ED175KK7tMi9SdCB7wgHkgTZcpJEoiigLybEUfIEEBAcWMx+e7/5h7wbBY\na82ste5ZM/e636/rmouZuX/z+/1m1ucZvnPfv3smVYUkSZK02f3ItCcgSZIkbQQLX0mSJPWCha8k\nSZJ6wcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JyjJd5Pc0Fx2Jblx6PZRq+jnuUnOTXJd\nkh1JTkqyZZJzV3+1mNtnJrmoye2VSd6c5M6TnLv6rcXsHpPkh0OPvSHJoyc5d/VLi1n9ySRnJvlW\nkl1LbL9bkvc04126mr43q/gDFhsjydeA51fVx9bw2F8HvgicA9wTOAN4Z1Wd1O4spVtbZ27vBfyg\nqq5KckfgDcDVVfXStucpLbbO7B4DPK+qLHY1cevM6gOAw4GrgfdW1Y8s2n5qc/X5wCHAB4BHVtWX\n1zfr7tpt2hPQaFX1t0M3r0hyCvDYac1HGkdV7Ri6GWAXcL8pTUdarUx7AtIoVfUV4CtJ7r94W7PD\n4WnAg6vqRuCsJO8Dng2csLEznR0udZiiJEcnuXaZyzVJDljmoY8BLtjIuUoLVpPbJD+f5DvA9Qze\ngF87tYmr90Zk99qh7BZwSHP4+OIkL3d5mTbSOuqDYQ8AdlbV9qH7zgcePJlZd4N7fKeoqt4GvG01\nj0nyPOBngOdNZFLSCKvJbVV9Ctg7yX7AC4GvT3Ju0kpWkd1/ZbCX7OtJfhJ4O7AT+LNJzk9asJb6\nYAl3YrDTYdgNQK/PtXCPb4ck+R/AnwBPrKprpj0faVxVdQXwYeC0ac9FGqWqvlZVX2+uXwC8Enj6\ndGclrdp3gb0W3XcXBsVvb1n4TlGSX1101vDw5fpFh4y3Am8EnlJVX5rerNV3q8ntIrvjGl9N0Tqy\nC6751QZaZ1YXfAXYbdH634fS86WSLnWYoqo6BThlVLskj2vaHVFV5058YtIKVpHbo4FPVtWOJPcB\n/hj46KTnJy1nFdl9IvC5qroyyQOBlwPvmPT8pAXjZhUgyZ7AHs312zWP/0FVfS/Ju4FXJnkBg2WS\n/x14xGRm3Q3u8e2GlzNYk/OhoU98H5j2pKQRDgbOTvJd4FPAhQzW+Uqz7nHA+U12PwCczmCZmTRT\nkhwI3MhgL24B32fwXrvgxcDtgauAtwK/XlUX0mMjv8c3yT8ATwauqqqfWqbNXwFPZPDiH1NV57U9\nUWm1zK66yNyqq8yuumCcPb5vArYutzHJk4D7V9VBwIuA17c0N2m9zK66yNyqq8yuZt7IwreqPglc\nu0KTpwJvadqew+Cri/ZtZ3rS2plddZG5VVeZXXVBG2t89weGf6HpMmCcsw2laTO76iJzq64yu5q6\ntr7VYfHXvNxm4XCSlRcTS2Oqqja/VsjsakOYW3WV2VVXLZXdNvb4Xg7ca+j2Ac19S01g7MuJJ564\nqvarvUy6/83wHGax/5aZ3Sn0vxmew2r7N7ez+XfpW/9rGcPszubfxf5HX5bTRuF7BvAcgCSHAd+p\nqitb6FeaNLOrLjK36iqzq6kbudQhyanAY4B9kuwATmTwC0xU1Ruq6oNJnpRkO/A94NcmOWFpXGZX\nXWRu1VVmV10wsvCtqqPGaHNcO9O5xdzcXNtdbmj/GzGG/a/M7M5m/xsxRpf736y53Ygx7H+6Y2zW\n7Hb972L/tzbyByxaGyipjRpLm1cSqt0TLcYZ0+xqXcytusrsqquWy64/WSxJkqResPCVJElSL1j4\nSpIkqRcsfCVJktQLFr6SJEnqBQtfSZIk9YKFryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqS\nJKkXLHwlSZLUCxa+kiRJ6gULX0mSJPWCha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSp\nFyx8JUmS1AsWvpIkSeoFC19JkiT1goWvJEmSesHCV5IkSb1g4StJkqResPCVJElSL1j4SpIkqRcs\nfCVJktQLFr6SJEnqBQtfSZIk9YKFryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqSJKkXRha+\nSbYmuSjJJUmOX2L7XZK8P8nnk1yQ5JiJzFRaJbOrrjK76iJzqy5IVS2/MdkCXAz8AnA58FngqKq6\ncKjN7wN3rqoTkuzTtN+3qnYu6qtWGksaRxKqKmO0M7uaGePmtmnbSnbNrdrge666arnsjtrjeyiw\nvaouraqbgNOAIxa12QXs1VzfC7h6cYilKTC76iqzqy4yt+qEUYXv/sCOoduXNfcNex1wcJIrgPOB\nl7Y3PWnNzK66yuyqi8ytOmG3EdvHOdawFfhcVT02yf2AjyR5aFXdsLjhtm3bbr4+NzfH3NzcKqaq\nPpqfn2d+fn4tDzW7mpp15BZazK651Wr5nquuGje7o9b4HgZsq6qtze0TgF1VddJQm38C/rSqzmpu\n/wtwfFWdu6gv1+xo3Vax3szsamasco1vK9k1t2qD77nqqrWu8T0XOCjJgUn2AI4EzljU5hsMFrOT\nZF/gJ4Cvrn/K0rqYXXWV2VUXmVt1wopLHapqZ5LjgDOBLcDJVXVhkmOb7W8AXgW8OckXgAC/W1XX\nTHje0orMrrrK7KqLzK26YsWlDq0O5KELtWA1h4xbHNPsal3MrbrK7Kqr1rrUQZIkSdoULHwlSZLU\nCxa+kiRJ6gULX0mSJPWCha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS1AsW\nvpIkSeoFC19JkiT1goWvJEmSesHCV5IkSb1g4StJkqResPCVJElSL1j4SpIkqRcsfCVJktQLFr6S\nJEnqBQtfSZIk9YKFryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqSJKkXLHwlSZLUCxa+kiRJ\n6gULX0mSJPWCha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS1AsWvpIkSeqF\nkYVvkq1JLkpySZLjl2kzl+S8JBckmW99ltIamF11ldlVF5lbdUGqavmNyRbgYuAXgMuBzwJHVdWF\nQ232Bs4CnlBVlyXZp6q+vURftdJY0jiSUFUZo53Z1cwYN7dN21aya27VBt9z1VXLZXfUHt9Dge1V\ndWlV3QScBhyxqM3RwOlVdRnAUiGWpsDsqqvMrrrI3KoTRhW++wM7hm5f1tw37CDgbkk+nuTcJM9u\nc4LSGplddZXZVReZW3XCbiO2j3OsYXfgZ4DHA3cA/i3Jp6vqksUNt23bdvP1ubk55ubmxp6o+ml+\nfp75+fm1PNTsamrWkVtoMbvmVqvle666atzsjlrjexiwraq2NrdPAHZV1UlDbY4Hbl9V25rbfw98\nuKretagv1+xo3Vax3szsamasco1vK9k1t2qD77nqqrWu8T0XOCjJgUn2AI4EzljU5n3AzyfZkuQO\nwMOBL7cxaWkdzK66yuyqi8ytOmHFpQ5VtTPJccCZwBbg5Kq6MMmxzfY3VNVFST4MfAHYBfxdVRlk\nTZXZVVeZXXWRuVVXrLjUodWBPHShFqzmkHGLY5pdrYu5VVeZXXXVWpc6SJIkSZuCha8kSZJ6wcJX\nkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS1AsWvpIkSeoFC19JkiT1goWvJEmSesHCV5Ik\nSb1g4StJkqResPCVJElSL1j4SpIkqRcsfCVJktQLFr6SJEnqBQtfSZIk9cJu056AJEnSYkla77Oq\nWu9T3WLhK0mSZlSbhWr7hbS6x6UOkiRJ6gULX0mSJPWCha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JUk\nSVIvWPhKkiSpFyx8JUmS1AsWvpIkSeoFC19JkiT1goWvJEmSemG3aU9AkjRZSVrtr6pa7U+SNoqF\nryT1QlvFartFtCRtJJc6SJIkqRcsfCVJktQLFr6SJEnqBQtfSZIk9cLIwjfJ1iQXJbkkyfErtPu5\nJDuTPK3dKUprY3bVVWZXXWRu1QUrFr5JtgCvA7YCBwNHJXnQMu1OAj6Mp/xqBphddZXZVReZW3XF\nqD2+hwLbq+rSqroJOA04Yol2vwm8C/hWy/OT1srsqqvMrrrI3KoTRhW++wM7hm5f1tx3syT7Mwj3\n65u7/GZzzQKzq64yu+oic6tOGPUDFuOE8rXA71VVZfDzQMseuti2bdvN1+fm5pibmxuje/XZ/Pw8\n8/Pza3mo2dXUrCO30GJ2h3ML88DcWueknpi999xtDHI7t5Y5qUfGzW5W+unJJIcB26pqa3P7BGBX\nVZ001Oar3BLefYAbgRdW1RmL+ip/5lLrlYSqGrkuzOxqloyb26ZtK9kdzu2gxmjvl9v899Af03zP\nbTe3YHb7Zbnsjip8dwMuBh4PXAF8Bjiqqi5cpv2bgPdX1buX2GbxoHVbxZuw2dXMWGXh20p2LXzV\nhmm+51r4aj2Wy+6KSx2qameS44AzgS3AyVV1YZJjm+1vmMhspXUyu+oqs6suMrfqihX3+LY6kHvN\n1ILV7DlrcUyzq3WZdm7d46u1mmZ23eOr9Vguu/5ymyRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y\n+EqSJKkXLHwlSZLUCxa+kiRJ6gULX0mSJPWCha8kSZJ6YcWfLJakWTL4Jad2+UtOktQfFr6SOqbd\nnzCVNkLbH9r8wCatjYWvJEkboq1i1Q9s2hib8Sibha8kSZKWsbmOsnlymyRJknrBwleSJEm9YOEr\nSZKkXnCNrzQmz8qWJKnbLHylVfGsbEmSusqlDpIkSeoFC19JkiT1goWvJEmSesHCV5IkSb3gyW09\ntBl/glCSJGkUC9/e2lw/QShJkjSKSx0kSZLUCxa+kiRJ6gULX0mSJPWCha8kSZJ6wcJXkiRJvWDh\nK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS1AsWvpIkSeoFC19JkiT1goWvJEmSemGswjfJ1iQXJbkk\nyfFLbP/VJOcn+UKSs5I8pP2pSqtjbtVVZlddZXY160YWvkm2AK8DtgIHA0cledCiZl8FHl1VDwFe\nBbyx7YlKq2Fu1VVmV11ldtUF4+zxPRTYXlWXVtVNwGnAEcMNqurfquq65uY5wAHtTlNaNXOrrjK7\n6iqzq5k3TuG7P7Bj6PZlzX3LeT7wwfVMSmqBuVVXmV11ldnVzNttjDY1bmdJHgs8Dzh8qe3btm27\n+frc3Bxzc3Pjdq2emp+fZ35+fi0PbS23MJzdbcBcc5GWto7cwoTec2Eec6tRZi+72/A9V+MYN7up\nWjmnSQ4DtlXV1ub2CcCuqjppUbuHAO8GtlbV9iX6qVFjaWMkYRXvT+P0yEb9bZNQVRmjXSu5bdpU\nVbX8um3ca7aZdDW74+a2adv6e67Znb6u/g2mmd2u/nvfTLr8N1guu+MsdTgXOCjJgUn2AI4EzljU\n+b0ZhPhZyxUP0gYzt+oqs6uuMruaeSOXOlTVziTHAWcCW4CTq+rCJMc2298A/BFwV+D1g08H3FRV\nh05u2tLKzK26yuyqq8yuumDkUofWBnKpw8zYjIcuJjymSx1mRFezO83cLoxvdqerq3+DzfOeC2Z3\n9br8N1jPUgdJkiSp8yx8JUmS1AvjfJ2ZtGrN2q3WeHhKktQ2/1/VPxte+LYdMrht0AzyrGhvPZuk\n2eV7rrrN/1f1yZT2+La7UHqyYxhkSRrN91xJs881vpIkSeoFC19JkiT1goWvJEmSesFvdZBmxEac\n+ClJUp9Z+EozZSNO/JQkqZ9c6iBJkqResPCVJElSL1j4SpIkqRdc4yupNf6ClyRplln4rsGk/+fu\n2f3qNn/BS5I0myx812zS/3P37H5JkqQ2ucZXkiRJvWDhK0mSpF6w8JUkSVIvWPhKkiSpFyx8JUmS\n1AsWvpIkSeoFC19JkiT1goWvJEmSesHCV5IkSb1g4StJkqResPCVJElSL+w27QlI0qxI0nqfVdV6\nn5KktbHwlaRbabNQbb+QliStnUsdJEmS1AsWvpIkSeoFlzpIPdL2GlbXr0qSusTCV+qdtopV169q\n8/BDodQPFr6SJAF+KJQ2P9f4SpIkqRdGFr5Jtia5KMklSY5fps1fNdvPT3JIO1Obb6ebqfW/EWPY\n/0rM7qz2vxFjdLv/6WR3fv1dTHmM+fnJ9j/LuUrS6mWNc/A9d60jTDi7Xf+30eb8Vyx8k2wBXgds\nBQ4GjkryoEVtngTcv6oOAl4EvL6dqc23083U+t+IMex/OWZ3lvvfiDG62//0sju//i6mPEbX/+e+\n/v5rjMuJY7RZPd9z1zmChe/KvW9U4QscCmyvqkur6ibgNOCIRW2eCrwFoKrOAfZOsm9rM5TWxuyq\nq8yuusjcqhNGFb77AzuGbl/W3DeqzQHrn5q0LmZXXWV2Fxn38PwrXvGKiR7K14rMrbqhqpa9AL8M\n/N3Q7WcBf72ozfuBw4dufxT4mSX6GucYjBcvIy8rZdbsepnVyzi5bTO7036+XjbPxfdcL129LJXV\nUV9ndjlwr6Hb92LwCW2lNgc0991KVfkRWxvJ7KqrWsmuudUG8z1XnTBqqcO5wEFJDkyyB3AkcMai\nNmcAzwFIchjwnaq6svWZSqtjdtVVZlddZG7VCSvu8a2qnUmOA84EtgAnV9WFSY5ttr+hqj6Y5ElJ\ntgPfA35t4rOWRjC76iqzqy4yt+qK+LOKkiRJ6oOZ/OW2cb4Eex19/0OSK5N8sc1+h/q/V5KPJ/lS\nkguSvKTl/vdMck6Szzf9b2uz/6FxtiQ5L8n7J9T/pUm+0IzxmUmMsdEmmdumf7M73jhmd5XM7sj+\nO5/dzZhbsF4YYwyzu9i4Zxpv1IXBIZLtwIHA7sDngQe12P+jgEOAL05o/j8K/HRz/U7AxW3Ov+n3\nDs1/dwM+DTx8As/jZcApwBkTep2+BtxtI7M1ycukc9uMYXbHG8Psru75mN3xxuh0djdbbpvnZL0w\n3jhmd+gyi3t8x/kS7DWrqk8C17bV3xL9f7OqPt9c/y5wIbBfy2Pc2Fzdg8E/9l1t9p/kAOBJwN8D\nkzy7djOduTvR3ILZHYfZXROzO94YmyG7mym3YL0w7jhmd8gsFr7jfAl2JyQ5kMGnxXNa7vdHknwe\nuBL456r6bJv9A68BfoeW/3EsUsBHk5yb5IUTHGejbJrcgtkdwezOMLO7rM2WW9hE2Z1Ubpu+ze6Q\nWSx8N8XZdknuBLwLeGnzSa41VbWrqn6awXcgPjzJg9vqO8lTgKuq6jwmu3fg8Ko6BHgi8BtJHjXB\nsTbCpsgtmN0xmN0ZZXZXtNlyC5sku5PMLZjdxWax8B3nS7BnWpLdgdOBt1bVeyc1TlVdB3wc2Npi\nt48Enprka8CpwOOS/N8W+wegqv6j+e+3gPcwOGTVZZ3PLZjdcZjd2WR2V7YJcwubILsblVswuwtm\nsfAd50uwZ1aSACcDX66q106g/32S7N1cvz3w3xisC2pFVf1+Vd2rqn4ceCbwsap6Tlv9AyS5Q5I7\nN9fvCPwiMJGzZjdQp3MLZnccZnc2md2VbdLcQsezO+ncNmOY3UVmrvCtqp3Awpdgfxl4e1W19kdK\ncipwNvCAJDuStP0F2ocz+I3yxzZfvXFekjY/Xf0Y8LEk5wOfYbBe54Mt9r/YJA4l7Qt8sllzdA7w\nT1X1zxMYZ8NMOrdgdtfA7I7B7I6l69nddLkF64Uxmd1F/AELSZIk9cLM7fGVJEmSJsHCV5IkSb1g\n4StJkqResPCVJElSL1j4SpIkqRcsfCVJktQLFr4TkOQPklyQ5Pzme/kOTfLS5sujRz32t8ZpJ7XN\n3KqrzK66yuxuPL/Ht2VJHgH8OfCYqropyd2APYGzgIdV1dUjHv+1cdpJbTK36iqzq64yu9PhHt/2\n/Sjw7aq6CaCqrgGeDuwHfDzJvwAkeX2Szzaf9LY1971kiXa/mOTsJP+e5B3NT/aR5M+SfKn5lPh/\nNvxZarMxt+oqs6uuMrtT4B7fljVB+xRwB+CjDH5C8V+bT2Y/2wSbJHetqmuTbGna/WZVXTDcLsk+\nwOnA1qr6fpLjgT2AvwHOrqoHNn3tVVXXb/iT1aZhbtVVZlddZXanwz2+Lauq7wE/C7wI+Bbw9iTH\nNJsz1PTIJP8OfA54MHDwEt0d1tx/dpLzgOcA9wauA/4zyclJfgn4/iSei/rD3KqrzK66yuxOx27T\nnsBmVFW7gE8An0jyReCYhU0ASX4c+G0Ga3OuS/ImBut6lvKRqjp68Z1JDgUez+CwyHHNdWnNzK26\nyuyqq8zuxnOPb8uSPCDJQUN3HQJcCtwA7NXctxfwPeD6JPsCTxxqP9zuHODwJPdr+r5jkoOawyN7\nV9WHgJcBD53U81E/mFt1ldlVV5nd6XCPb/vuBPx1kr2BncAlDA5jHA18OMnlVfX45lDERcAOBmt8\nFrxxUbtjgFOT3K7Z/gcMwv6+JHsyOBzyvzbiiWlTM7fqKrOrrjK7U+DJbZIkSeoFlzpIkiSpFyx8\nJUmS1AsWvpIkSeoFC19JkiT1goWvJEmSesHCV5IkSb1g4StJkqResPCVJElSL1j4SpIkqRcsfCVJ\nktQLFr6SJEnqBQtfSZIk9YKFryRJknrBwleSJEm9YOErSZKkXrDwlSRJUi9Y+EqSJKkXLHwlSZLU\nCxa+kiRJ6gULX0mSJPWCha8kSZJ6wcJXkiRJvWDhK0mSpF6w8JU0U5JckOTREx7jwCS7koz1Hpjk\nzUle1Vx/VJKLWpzLB5M8u7l+TJJPttj3ryY5s63+2pTkhiQHrrD90iSPH7OvsZ/nLL8mkibPwlea\nYUmOTnJuUyRc0RRJh097Xm0ZLigXVNVPVtW/TmtOy6jmQlV9sqoeOOoBSbYl+ceRHVc9qapGthtj\nvNsU81V1SlU9Yb19T0JV3bmqLoWlc8DQaz5GX2M/z8Vtm9fsvmNO+2ZJDkvykSRXJ7kqyTuS/Ohq\n+5G0sSx8pRmV5GXAa4BXA/cE7gX8DfDUZdpv2bjZ9VJa7azRZp8LXU+gz81uLa/Z3sDfAvdpLjcA\nb2pzUpLaZ+ErzaAkdwFeAby4qt5bVd+vqh9W1Qeq6vimzbYk70ryj0muA56bZL8kZzR7oS5J8oKh\nPg9t9h5fl+SbSf68uX/PJG9N8u0k1yb5TJJ7LjOv45NcluT6JBcleVxzf5L8XpLtTT9vT3LXocf9\nfJKzm/6/keS5SV4IHA38brNH+31N25sPcSe5XZLXJrm8ubwmyR7NtrlmLi9LcmWzR/yYoTGfnOS8\n5vl+I8mJq3j9D0nyueZ5ngbsObRtLsmOlV6TJFuBE4Ajm+d2XtN2Psmrk5wFfBe4b3Pf8289fP46\nyXeSXLjwGi9+bYYysLC3eGEv+XeauRyWRUsnkjwyyWebvj+T5BFD2+aTvDLJp5rHn5nk7s22sTKS\n5NeSnDF0+5Ik7xi6vSPJQ5rru5LcL8mLWCIHjUOSnN/M97Qkt1vm77X4ee5KcmySrzTzfd1SbZMs\nvGbnN2M/Y6n+l1JVH66q06vqu1X1fQYfSjfN0Rhps7LwlWbTIxgUW+8Z0e6pwDur6i7A24DTgG8A\nPwY8HfiTJI9t2v4l8Jqm7X2Btzf3PxfYCzgAuBtwLPD9xQMl+QngN4CHVdVewC8ClzabX9LM5dHN\n2NcyKARIch/gg834+wA/DXy+qv4OOAU4qTnsfUTT1/Ah7j8ADgUe2lwOBV4+NK19m7nvBzwf+JsM\nPjTAoLB8VvN8nwz8zyRHMEJTWL8XeAtwV+CdwC+zxGH35V6Tqvow8CfAac1zO2ToYc8CXgDcGfg6\ntz2k/3BgO3B34ETg3Un2XuK1YdH1RzX/vUtV7VVVn14017sBHwBey+Dv/BfAB4Y/oABHAccwOMKw\nB/C/m/vHyggwvzCPJPsBuwOHNbfvC9yxqr4wPP+qeiNL5yDAM4AnAD8OPKSZ27ieDDysedyvJLnN\nUoiqWlhL/pBm7HcmuXdTLC93eeYy4z0auGAV85M0BRa+0my6O/Dtqto1ot3ZVbWwh+0ewCOB46vq\nv6rqfODvgec02/8LOCjJPlV1Y1V9Zuj+uwMH1cB5VXXDEmP9ELgd8OAku1fVN6rqq822Y4GXV9UV\nVXUTg73VT89g+cXRwEeq6u3NXutrmrktWOkw89HAK6vq21X17abfZw9tv6nZ/sOq+hCDYvcnAKrq\nE1X1peb6Fxl8KHjMCmMtOAzYrar+sun3dOCzy7Rd6TXJEs+tgDdX1YVVtauqdi7R51VDY78DuJhB\nEbeULHN9KU8GLm7WuO6qqtOAi7hl6UwBb6qq7VX1n8A7GHxIgTEzUlVfA25IcgiDQvBM4IrmA8Jj\nuGWv9KjnsjCfv6qqb1bVtcD7h+Yzjj+rquuragfw8XEf2/wN77rC5bTbTHywF/sPgd9ZxfwkTYGF\nrzSbrgb2yehvHbhs6Pp+wDVV9b2h+74B7N9cfz7wAODC5lD1QjH1jwwKlNOa5QQnJdlt8UBVtR34\nLWAbcGWSU5P8WLP5QOA9C3vFgC8DOxnskT0A+Ori/sa0H4O9osPPZ7+h21cv+nBwI3AngCQPT/Lx\nDE48+g6D4vzuY455+aL7vr5UwxGvyXJ2jNi+1Nj7LdVwlfZj8Pqt1Pc3h65/n+a1ZMyMND4BzDHY\n8/uJ5vIYBoXwJ1Y55+Xms9rH3gjccZVjjyXJ/Rkc0XhJVZ01iTEktcfCV5pN/wb8APilFdosPux9\nBXC3JMPFwb1piuNmT97RVXUP4CTgXUluX1U7q+qVVfVgBnuMn8Ite4lvPWDVqVX1KAYn81TTDwwK\nqq2L9ozdoaquYFDo3W+F57CSKxgU1cPP54oRj1nwNgZLFg6oqoUTkcZ5z/sPbvmwsOA+yzVe4TVZ\n7rmNes5Ljb3wnL/HrQu44W8RGNXv5dz2edyH2xbat7GajDAobh/LoPCd55ZC+DEsX/iO8+0NY33D\nw3o1Sx2gcAZMAAACpklEQVRuWOFy1FDb+wAfYXDU4ZSNmJ+k9bHwlWZQVV0H/BGDNatHJLlDkt2T\nPDHJQmGVRY/ZAZwN/GkGJ4U9BHge8FaAJM9Kco+m+XUMColdSR6b5KeaZQk3MFg+8MPFc0rygAxO\n3Lodg6L8P4fa/S2D9cT3btreI8nCIfRTgF9I8owkuyW5e5KHNtuuZLDeeDmnAi9Psk+SfZrXZNyv\n/roTcG1V/VeSQxksmxineDob2JnkJc1r/jTg55ZqOOI1+SZwYHKbb24YtSThnkNjPwN4IIM9igCf\nB57ZvI4P49Zrj78F7GL5DxkfAh6Q5Kjm8Uc2ff/TqLmNm5HGQuG7Z/PB51PAVgZrg89b5jGjcrDs\n3Ma01LKT4bFvfs2apQ53XuFyKkCS/YGPAa9r1ilL6gALX2lGVdVfAC9jcDLXVQz2qr6YW054W+p7\nTo9isIf0CuDdwB9V1ceabU8ALkhyA4OvSXtmVf2AwXKEdzIohr/MYC/dUsXl7YA/ZVBg/QeDE9VO\naLb9JXAG8M9Jrmewx/rQ5nnsAJ4E/DaDJRznMTjhCOBk4OBmicS7lxjz1cC5wBeay7nNfTe/TEs8\nZsGLgVc28/lDbjmZb8XHNmuUn8bgRKqrgV8BTl/msSu9Ju9s/nt1knPHnHMBnwYOavp8FfDLzRpX\nmudxPwYnD25j8KFiYd43An8MnJXkmiQP59bfP3w1gz21vw18m8GJa0+pqmuWmdtwvsbNCFV1CYPi\n+JPN7euB/wecVVXLnZg3KgeL5zNq2+J2w9sXt90GvKUZ++nL9L+UFzA46W7b0N7g61fxeElTkFu/\nD0mSJEmbk3t8JUmS1AsWvpIkSeoFC19JkiT1goWvJEmSesHCV5IkSb1g4StJkqResPCVJElSL1j4\nSpIkqRf+P3s4aAGjgxZxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10fe9a630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "init = 2\n",
    "Ts = [2, 3, 5, 10]\n",
    "plot_cross_sectional_dists(mc1, init, Ts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since our Markov chain is aperiodic (i.e., every recurrent class is aperiodic),\n",
    "the marginal disribution at time $T$ converges as $T \\to \\infty$\n",
    "to some stationary distribution,\n",
    "and the limit distribution depends on the initial state,\n",
    "according to the probabilities that the state is absorbed into the recurrent classes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For initial states `3`, `4`, and `5`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAD3CAYAAAD7T1f4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu4bXVd7/H3xw2KxlVRlItShte8pW0xMpfi0Q2aWFYK\noVGWnMysoxWZljs1zdPFG6VkpHlJzFuCoeSF5QUUREFFNzxwjLgpyG3LzQT29/wxxpK5F3OuNdfe\nc83LGu/X84xnzTnHb47fd4y1vnN+x2/+5lipKiRJkqS17k6TDkCSJEkaBwtfSZIkdYKFryRJkjrB\nwleSJEmdYOErSZKkTrDwlSRJUidY+I5JkpOTPHeJ9W9N8oohtzWf5Pmji05SL/NVmi3mrIZl4bsd\nklyU5OBh2lbVoVX17vZ5RyX5/KL1v1NVrxmy62qXlcS6f5ItSYb+nbf796SV9DMKSU5NcmWSzUnO\nSfKMccegtcd8XV1JntDG/OpJxaC1xZxdHW2/NyW5vl0+Me4YJmmHSQcw41acHFMgK2hbK2w/Ki8G\nzquqW5KsBz6V5ICqumICsWjtMF9XSZIdgTcBX2L2jrGmlzm7Ogp4elV9ZgJ9T5wjviPSnmF+Iclf\nJ7kmybeTbOhZP5/k+UkeBLwNeFx7pnVNu/6dCyMlSfZI8rF21POaJCcl2WfIONYnOasdLf1ukr9p\nV32u/Xld2+9jk9w/yWeSXJXke0nek2S3djvvBu4LnNS2/8P28QOTnJ7k2nY09gmjOH69quobVXVL\nz0M7AvuNuh91l/k6ci8FPgGcz4SKb61t5uzIdTZPLXxHaz1wHnAP4P8Cx/esK6Cq6jzgaOCLVbVL\nVd29d317O+1z79suNwPHDhnDm4A3VNVuwE8AH2gff3z7c7e23zPa+38J3Ad4ME1xuZEm0OcCF9Oc\nFe5SVX/TvjB8DHhVVe0B/CHwoSR79gukfWG5dsBy4lI70T73ZpoRpFOr6qwh918alvnaY1vzNcn9\ngN8AXk2H30w1FuZsj+15jwXe2xb+pyR5+JD7viZY+I7Wf1fV8VVVwLuA+yS5V592g94cAlBV11TV\nR6rqB1V1A/BaYNizvh8CByTZs6pu6km+O/RZVf+vqj5dVbdU1VXAG5bp50jg5Kr6RPv8TwFnAYf2\na1xVT6+qPQYsS87braqnAzu32/7kMvssbQvzdevtb2u+vhl4RVXdyGx+NK3ZYc5uvf1tzdkjgPu1\ny6nAKQsj0V1g4Tta3124UVU3tTd3XulGktwtyXFpJqBvBj4L7JZkmNGU5wMPADYlOTPJ05boZ68k\nJyS5tO3n3TRn0oPcD/iV3rNK4CDg3kPv3ApU1W3tC8BTkvzCavShTjNft1OblztX1cKoV3DUV6vH\nnB2BqvpiVf1PVd1cVX8FXMftI9Zrnl9um4xBIyILj7+UJrHWV9WVSR4JfJXmDWXJ0ZSqupDmbI4k\nzwI+mOTuA573WuA24Keq6rokzwTeskScFwPvrqoXLBXDgiQfB35uwOrPVdXAF4xFdqT5SEmaBPN1\ncL4+CXhMku+093cDbkvyU1X1i8P0K60Cc3Zl77Gd+pTGEd/JuALYN803oRf0jpTsTDPnaHObUK/s\ns42+Z6ZJjkxyz/buZpo/6C3A99qf9+9pvjNwI/D9dm7RH/WJs7f9e4BfSPKUJOuS7JRkLgO+FFBV\nh7Rzl/otfRMyyQOTHJLkrkl2THIkzZnoZ/u1l8bAfB38BvpnwAHAI4BHAicC/0gz51eaFHN28Hvs\nfkkOSnLndvt/RDMKfVq/9muRhe/o9JvbNugs6tPAN4HvJrmyz/PfCNwVuAo4Hfj4Crb9VODcJNfT\nzCd6TvuRxk00k+xPS/Mt1vXAXwA/TZO8JwEfWrTd1wGvaD9yeUlVXQocBvwpcCXN2elLGe3fUWhe\nhK5o+/g94Fer6pwR9iGZryNQVTdU1ZXtcgVNMXFjVV03qj6kljk7GrsA/wBcA1wKPAU4pKquHWEf\nUy3NHPElGiT/DDwNuLKqHjagzZuBQ4CbgKOq6uxRByppeearNDvMV2n8hjmLeAewYdDKJIcCP1lV\nBwAvAN46otgkrZz5Ks0O81Uas2UL36r6PLDUEPgzgH9p254B7J5kr9GEJ2klzFdpdpiv0viNYt7I\nPsAlPfcvBfYdwXYljZ75Ks0O81UasVFNmF787cdOXRpDmjHmqzQ7zFdphEZxHd/LaP4N34J928e2\nksRklRapqnFf7N98lbbRtOYrmLNSP/1ydhQjvicCzwNIciBwXXtZm34BcPtVJGqFyyu34Tm39zmK\n5ZWvfOVIt2cMsxvDKPqfkBXn66z+jqYhhkn3bwyji2FChs5X8D12LcQw6f7XUgyDLDvim+R9NP9b\nes8kl7TZsWP7B39cVZ2c5NAkF9JcqNkLl0sTYr5Ks8N8lcZv2cK3qg4fos2LRhOOpO1hvkqzw3yV\nxm+G/nPb3KQDYG7OGIxhOvqfBdNwjCYdw6T7N4bpimG6zU06gKn4HU06hkn334UYlv3PbSPrKKmF\nvpIwni+mZsl5HtIkJaHG/2WZofTmq6TpzlfwPVZabFDOztCIryRJkrTtLHwlSZLUCRa+kiRJ6gQL\nX0mSJHWCha8kSZI6wcJXkiRJnWDhK0mSpE6w8JUkSVInWPhKkiSpEyx8JUmS1AkWvpIkSeoEC19J\nkiR1goWvJEmSOsHCV5IkSZ1g4StJkqROsPCVJElSJ1j4SpIkqRMsfCVJktQJFr6SJEnqBAtfSZIk\ndYKFryRJkjrBwleSJEmdYOErSZKkTli28E2yIcl5SS5Ickyf9bslOSnJOUnOTXLUqkQqaSjmrDQ7\nzFdpvFJVg1cm64DzgScDlwFfBg6vqk09bf4U2KWqXpZkz7b9XlV166Jt1UJfSYDB/Y5OWGr/pElK\nQlVlxNscSc725quk6c7Xtp3vsVKPQTm73IjveuDCqrqoqm4BTgAOW9RmC7Bre3tX4OrFCSlpbMxZ\naXaYr9KYLVf47gNc0nP/0vaxXscCD0lyOfA14PdHF56kFTJnpdlhvkpjtsMy64f5DGMD8NWqemKS\n+wOfTPKIqrp+ccONGzf23JsH5oYMU5p98/PzzM/Pr3Y3I8vZ3nydm5tjbm5ulHFKU23W8hV8j1W3\nDZuzy83xPRDYWFUb2vsvA7ZU1et72nwMeF1Vndbe/zRwTFWdtWhbzj+SeqzSnMGR5KxzfKWtTXO+\nto/7Hiv12NY5vmcBByTZP8mdgWcDJy5qczHNxHyS7AU8EPj29ocsaRuYs9LsMF+lMVtyqkNV3Zrk\nRcApwDrg+KralOTodv1xwKuBdyb5OhDgj6vqmlWOW1If5qw0O8xXafyWnOow0o78GEbaymp8dDoq\nTnWQtjbN+Qq+x0qLbetUB0mSJGlNsPCVJElSJ1j4SpIkqRMsfCVJktQJFr6SJEnqBAtfSZIkdYKF\nryRJkjrBwleSJEmdYOErSZKkTrDwlSRJUidY+EqSJKkTLHwlSZLUCRa+kiRJ6gQLX0mSJHWCha8k\nSZI6wcJXkiRJnWDhK0mSpE6w8JUkSVIn7DDpACRJsyPJ2PqqqrH1JakbLHwlSSs0joJ0fAW2pO5w\nqoMkSZI6wcJXkiRJnWDhK0mSpE6w8JUkSVInLFv4JtmQ5LwkFyQ5ZkCbuSRnJzk3yfzIo5Q0NHNW\nmh3mqzReWepyMUnWAecDTwYuA74MHF5Vm3ra7A6cBjy1qi5NsmdVXdVnW7XQV3M5nPF8K9jL4Wha\nJaGqRvrV9VHlbG++Sr26+vo9zfnatvM9VuoxKGeXG/FdD1xYVRdV1S3ACcBhi9ocAXyoqi4F6JeQ\nksbGnJVmh/kqjdlyhe8+wCU99y9tH+t1AHD3JKcmOSvJc0cZoKQVMWel2WG+SmO23D+wGOYzjB2B\nnwYOBu4GfDHJl6rqgu0NTtKKmbPS7DBfpTFbrvC9DNiv5/5+NGekvS4Brqqqm4Gbk3wOeARwh6Tc\nuHFjz715YG5l0UozbH5+nvn5+dXuZmQ525uvc3NzzM3NrUK40nSatXwF32PVbcPm7HJfbtuBZuL9\nwcDlwJncceL9g4BjgacCdwHOAJ5dVd9atC0n3ks9VunLMiPJWb/cpkG6+vo9zfnatvM9VuoxKGeX\nHPGtqluTvAg4BVgHHF9Vm5Ic3a4/rqrOS/IJ4OvAFuDtixNS0niYs9LsMF+l8VtyxHekHXk2Km1l\nNUaQRsURXw3S1dfvac5X8D1WWmxbL2cmSZIkrQkWvpIkSeoEC19JkiR1goWvJEmSOsHCV5IkSZ1g\n4StJkqROsPCVJElSJ1j4SpIkqRMsfCVJktQJFr6SJEnqBAtfSZIkdYKFryRJkjrBwleSJEmdYOEr\nSZKkTrDwlSRJUidY+EqSJKkTLHwlSZLUCRa+kiRJ6gQLX0mSJHWCha8kSZI6wcJXkiRJnWDhK0mS\npE6w8JUkSVInWPhKkiSpEyx8JUmS1AnLFr5JNiQ5L8kFSY5Zot3PJLk1yS+NNkRJK2HOSrPDfJXG\na8nCN8k64FhgA/AQ4PAkDx7Q7vXAJ4CsQpyShmDOSrPDfJXGb7kR3/XAhVV1UVXdApwAHNan3e8B\nHwS+N+L4JK2MOSvNDvNVGrPlCt99gEt67l/aPvYjSfahSdS3tg/VyKKTtFLmrDQ7zFdpzHZYZv0w\nCfZG4E+qqpKEJT6G2bhxY8+9eWBuiM1La8P8/Dzz8/Or3c3IcrY3X+fm5pibmxtFfNJMmLV8Bd9j\n1W3D5myqBuddkgOBjVW1ob3/MmBLVb2+p823uT0R9wRuAn67qk5ctK1a6KvJ3XGctIal9k+apCRU\n1Ujn640qZ3vzVerV1dfvac7Xtp3vsVKPQTm7XOG7A3A+cDBwOXAmcHhVbRrQ/h3ASVX14T7rTEqp\nxyq9kY4kZy18NUhXX7+nOV/bdb7HSj0G5eySUx2q6tYkLwJOAdYBx1fVpiRHt+uPW5VoJW0Tc1aa\nHearNH5LjviOtCPPRqWtrMYI0qg44qtBuvr6Pc35Cr7HSosNyln/c5skSZI6wcJXkiRJnWDhK0mS\npE6w8JUkSVInWPhKkiSpE5b7z22SNHHNt9THx2+qS9LaZOEraUaMqxid2itWSZK2k1MdJEmS1AkW\nvpIkSeoEC19JkiR1goWvJEmSOsHCV5IkSZ1g4StJkqROsPCVJElSJ1j4SpIkqRMsfCVJktQJFr6S\nJEnqBAtfSZIkdYKFryRJkjrBwleSJEmdYOErSZKkTrDwlSRJUidY+EqSJKkTLHwlSZLUCUMVvkk2\nJDkvyQVJjumz/teSfC3J15OcluThow9V0jDMV2l2mK/SeC1b+CZZBxwLbAAeAhye5MGLmn0b+Pmq\nejjwauAfRx2opOWZr9LsMF+l8RtmxHc9cGFVXVRVtwAnAIf1NqiqL1bV5vbuGcC+ow1T0pDMV2l2\nmK/SmA1T+O4DXNJz/9L2sUGeD5y8PUFJ2mbmqzQ7zFdpzHYYok0Nu7EkTwR+EzhomyOStD3MV2l2\nmK/SmA1T+F4G7Ndzfz+as9KttBPu3w5sqKpr+21o48aNPffmgbkhw5Rm3/z8PPPz86vdzark69zc\nHHNzc6OMU5pqs5av4Husum3YnE3V0iecSXYAzgcOBi4HzgQOr6pNPW3uC3wGOLKqvjRgO7XQVxJW\ncKK7HcJy+ydNShKqKiPe5sjzdRqM7zUDfN1YWldfv6c5X9t2vsdKPQbl7LIjvlV1a5IXAacA64Dj\nq2pTkqPb9ccBfw7sAby1SThuqar1o9wBScszX6XZYb5K47fsiO/IOvJsVNrKaowgjYojvtOz79Om\nq6/f05yv4HustNignPU/t0mSJKkTLHwlSZLUCRa+kiRJ6gQLX0mSJHWCha8kSZI6wcJXkiRJnWDh\nK0mSpE6w8JUkSVInWPhKkiSpEyx8JUmS1AkWvpIkSeoEC19JkiR1goWvJEmSOsHCV5IkSZ1g4StJ\nkqROsPCVJElSJ1j4SpIkqRMsfCVJktQJFr6SJEnqBAtfSZIkdYKFryRJkjrBwleSJEmdYOErSZKk\nTrDwlSRJUidY+EqSJKkTli18k2xIcl6SC5IcM6DNm9v1X0vyqNGHCTC/OptdSQTzxmAM09H/UqYl\nZ6fjGM1PtvcpOAbTEMOkfw8wLcfhjqYlX/0dTUcMk+6/CzEsWfgmWQccC2wAHgIcnuTBi9ocCvxk\nVR0AvAB46+qEOr86m11JBGv8j8EYZqf/QaYpZ6fjGM1PtvcpOAbTEMOkfw8wLcdha9OUr/6OpiOG\nSfffhRiWG/FdD1xYVRdV1S3ACcBhi9o8A/gXgKo6A9g9yV4jj1TSMMxZaXaYr9KYLVf47gNc0nP/\n0vax5drsu/2hSdoG5qw0O8xXadyqauACPAt4e8/9I4G3LGpzEnBQz/1PAT/dZ1vl4uKy9bJU/m3L\nwohydtLHxcVlGpdpzVdz1sWl/9Iv73ZgaZcB+/Xc34/mbHOpNvu2j22lqrJMX5K230hy1nyVxsL3\nWGnMlpvqcBZwQJL9k9wZeDZw4qI2JwLPA0hyIHBdVV0x8kglDcOclWaH+SqN2ZIjvlV1a5IXAacA\n64Djq2pTkqPb9cdV1clJDk1yIXAj8BurHrWkvsxZaXaYr9L4pZ0bJEmSJK1pM/Gf24a5wPcq9//P\nSa5I8o1x990Tw35JTk3yzSTnJnnxmPvfKckZSc5p+984zv4XxbIuydlJTppQ/xcl+Xobw5mTiGGa\nma+Tz9c2hqnIWfN1+nU9Z83XO8SypnN26kd82wt8nw88mWZC/5eBw6tq0xhjeDxwA/CuqnrYuPpd\nFMO9gXtX1TlJdga+AjxzzMfhblV1U5IdgC8Av99eV3KskrwEeDSwS1U9YwL9/xfw6Kq6Ztx9Tzvz\n9UcxTDxf2zgmnrPm63QzZ83XPnGs6ZydhRHfYS7wvaqq6vPAtePss08M362qc9rbNwCbgL3HHMNN\n7c07AzsCW8bZP0CSfYFDgX8CJvktZr9B3Z/5ynTka9v3RHPWfJ0Jnc9Z8/V2XcjZWSh8h7nAd6ck\n2R94FDDukZs7JTkHuAL4z6r68jj7b70B+CMm8ILQo4BPJTkryW9PMI5pZL4uMql8bfuedM6ar9PP\nnO3R8XyFDuTsLBS+0z0XY8zaj2E+SPMRyA3j7LuqtlTVI2muI/nYJA8dZ/9Jng5cWVVnM9kz0YOq\n6lHAIcDvth/TqWG+9phkvsJkc9Z8nRnmbKvL+QrdydlZKHyHucB3JyTZEfgQ8J6q+vdJxVFVm4FT\ngQ1j7vpngWe083/eBzwpybvGHANV9Z325/eAj9B8VKiG+dqalnyFieWs+TobzFnM11YncnYWCt9h\nLvC95iUJcDzwrap64wT63zPJ7u3tuwL/i2Ye1NhU1Z9W1X5V9ePAc4DPVNXzxhlDkrsl2aW9/WPA\nU4CJXT1gCpmvTD5f2xgmmrPm68zofM6ar42u5OzUF75VdSuwcIHvbwHvn8A3Ld8HnA48IMklSSZx\nAfGDaP6P+xPbS3ycnWScZ4P3AT6T5GvAmTTzj04eY//9TOIjur2Az7fzsM4APlZV/zmBOKaS+foj\nk85XmL6cNV+nkDkLmK+DrMmcnfrLmUmSJEmjMPUjvpIkSdIoWPhKkiSpEyx8JUmS1AkWvpIkSeoE\nC19JkiR1goWvJEmSOsHCd0YleXmSc5N8rb3m4Pokv99e+Hq55/7BMO0kjY45K80O83Xt8jq+MyjJ\n44C/BZ5QVbckuTuwE3Aa8JiqunqZ5//XMO0kjYY5K80O83Vtc8R3Nt0buKqqbgGoqmuAXwb2Bk5N\n8mmAJG9N8uX2rHVj+9iL+7R7SpLTk3wlyb+1/yaQJH+V5JvtGe9fj30vpbXDnJVmh/m6hjniO4Pa\npPkCcDfgUzT/YvJz7Vnmo9skJckeVXVtknVtu9+rqnN72yXZE/gQsKGqbk5yDHBn4O+B06vqQe22\ndq2q7499Z6U1wJyVZof5urY54juDqupG4NHAC4DvAe9PclS7Oj1Nn53kK8BXgYcCD+mzuQPbx09P\ncjbwPOC+wGbgB0mOT/KLwM2rsS9SF5iz0uwwX9e2HSYdgLZNVW0BPgt8Nsk3gKMWVgEk+XHgpTTz\njDYneQfNHKV+PllVRyx+MMl64GCaj3he1N6WtA3MWWl2mK9rlyO+MyjJA5Ic0PPQo4CLgOuBXdvH\ndgVuBL6fZC/gkJ72ve3OAA5Kcv922z+W5ID2o57dq+rjwEuAR6zW/khrnTkrzQ7zdW1zxHc27Qy8\nJcnuwK3ABTQfyRwBfCLJZVV1cPuxynnAJTTzlRb846J2RwHvS3KXdv3LaRL3o0l2ovlo5/+MY8ek\nNcqclWaH+bqG+eU2SZIkdYJTHSRJktQJFr6SJEnqBAtfSZIkdYKFryRJkjrBwleSJEmdYOErSZKk\nTrDwlSRJUidY+EqSJKkTLHwlSZLUCRa+kmZaknOT/Pwq97F/ki1JhnrNTPLOJK9ubz8+yXkjjOXk\nJM9tbx+V5PMj3PavJTllVNuTpGlj4SutIUmOSHJWkuuTXN4WSQdNOq5R6S0oF1TVT1XV5yYV0wDV\nLlTV56vqQcs9IcnGJO9edsNVh1bVsu2G6O8OxXxVvbeqnrq9296GWOaTPH/Ause3f8/XJ7mhjXnh\n/veT7LuCfn41yelJbkxyap/1j0zylXb9WUkesT37JWn6WPhKa0SSlwBvAF4D3AvYD/h74BkD2q8b\nX3SdlJFurDXKbS5sehW2uVI1cEVz4rBLVe0CPLR9eLf2sV2r6tIV9HM18HfAXy1ekeTOwEeBdwG7\nA/8CfDTJjivYvqQpZ+ErrQFJdgP+AnhhVf17Vd1cVbdV1X9U1TFtm41JPpjk3Uk2A7+eZO8kJya5\nOskFSX6rZ5vr21GvzUm+m+Rv28d3SvKeJFcluTbJmUnuNSCuY5Jc2o7MnZfkSe3jSfInSS5st/P+\nJHv0PO/n2pG5a5NcnOTXk/w2cATwx+1o30fbthclObi9fZckb0xyWbu8oS1oSDLXxvKSJFe0I+JH\n9fT5tCRnt/t7cZJXruD4PyrJV9v9PAHYqWfdXJJLljomSTYALwOe3e7b2W3b+SSvSXIacAPwE31G\nR5PkLUmuS7Jp4RgvPjY9fwMLo8ULo+TXtbEcmEVTJ5L8bJIvt9s+M8njetbNJ3lVki+0zz8lyT0G\nHJ/dk3wsyZVJrklyUpJ92nV/CTweOLbd9zcvdaiXWLesqvp0VX0Q+E6f1XPAuqp6U1XdUlVvaft7\nUp+2kmaUha+0NjyOptj6yDLtngF8oKp2A/4VOAG4GLgP8MvAa5M8sW37JuANbdufAN7fPv7rwK7A\nvsDdgaOBmxd3lOSBwO8Cj6mqXYGnABe1q1/cxvLzbd/X0oxOk+R+wMlt/3sCjwTOqaq3A+8FXt+O\n9h3WbutH0wqAlwPrgUe0y3rgFT1h7dXGvjfwfODv25MGaArLI9v9fRrwO0kOYxltYf3vNCOEewAf\nAJ5Fn1HMQcekqj4BvBY4od23R/U87Ujgt4BdgP9etL8AjwUuBO4BvBL4cJLd+xwbFt1+fPtzt3bk\n9EuLYr078B/AG2l+z38H/EfvCQpwOHAUzScMdwb+sM8hgua95njgvu1yM3AsQFW9HPg88Lvtvr94\nwDYGSvIP7UlSv+WcITfzUODrix77GrePMktaAyx8pbXhHsBVVbVlmXanV9WJ7e17Aj8LHFNVP6yq\nrwH/BDyvXf9D4IAke1bVTVV1Zs/j9wAOqMbZVXV9n75uA+4CPDTJjlV1cVV9u113NPCKqrq8qm6h\nGa3+5TTTL44APllV729Hra9pY1uw1KjfEcCrquqqqrqq3e5ze9bf0q6/rao+TlPsPhCgqj5bVd9s\nb3+D5qTgCUv0teBAYId2pPC2qvoQ8OUBbZc6JumzbwW8s6o2VdWWqrq1zzav7On734DzaQr3fjLg\ndj9PA85v5/1uqaoTgPO4fepMAe+oqgur6gfAv9GcpNxB+zv8SFX9oKpuoCnyFx/bbR7NraoXVtUe\nA5a+MfWxM7B50WPfpznhkLRGWPhKa8PVwJ5Z/qoDvfMh9wauqaobex67GNinvf184AHApvZj7oVi\n6t3AKcAJ7XSC1yfZYXFHVXUh8AfARuCKJO9Lcp929f7ARxZG5YBvAbfSjMjuC3x78faGtDfNqGjv\n/uzdc//qRScHN9EUPCR5bJJT24/jr6Mpzvt+dN+nz8sWPfbf/Rouc0wGuWSZ9f363rtfwxXam+b4\nLbXt7/bcvpn2WC6W5G5JjmunXmwGPgvslmw1Z3ngPN8xuZ7m04Beu9EUv5LWCAtfaW34IvA/wC8u\n0Wbxx96XA3dP0lus3Je2OG5H8o6oqnsCrwc+mOSuVXVrVb2qqh5KM2L8dG4fJd66w6r3VdXjgfu1\nfb++XXUxsGHRyNzdqupymkLv/kvsw1Iupymqe/fn8mWes+BfaaYs7FtVuwNvY7jXyO9w+8nCgvsN\narzEMRm0b8vtc7++F/b5RuDHetbdewXbvYw77sf9uGOhPYyX0pxErW+nkjyBrUe4t6voTfK23H6l\nh8XLN/o8pV9/3wQevuixh7ePS1ojLHylNaCqNgN/TjNn9bB2hG3HJIckWSissug5lwCnA69rvxT2\ncOA3gfcAJDkyyT3b5ptpioUtSZ6Y5GHttITraaYP3LY4piQPaL+4dReaovwHPe3eRjOf+L5t23sm\nWfgI/b3Ak5P8SpIdktwjt19W6gqa+caDvA94RZI9k+zZHpNhL/21M3BtVf0wyXqaaRPDFGSnA7cm\neXF7zH8J+Jl+DZc5Jt8F9l80CgrLTwG4V0/fvwI8iGaONMA5wHPa4/gYtp57/D1gC4NPMj4OPCDJ\n4e3zn91u+2MriG3BzjQjwpvbucOLvzh4xRJxLKuq/vfClR/6LA/7UbDJnZLsBOwI3Kn9u1+4asM8\ncFt7LO+S5MU0x+cz2xqXpOlj4SutEVX1d8BLaL7MdSXNqOoLuf0Lb4tHfKH5ctL+NCOEHwb+vKoW\n3uifCpyb5Hqay6Q9p6r+h2Y6wgdoiuFv0RQM/YrLuwCvoymwvkPzRbWXteveBJwI/GeS79OMWK9v\n9+MS4FBMmqTmAAABFUlEQVSaUcKrgbO5fSTueOAh7RSJD/fp8zXAWTRfUvp6e/s1vYepz3MWvBB4\nVRvPn3H7l/mWfG47R/mXaL7kdTXwq8CHBjx3qWPygfbn1UnOGjLmAr4EHNBu89XAs6rq2nb9n9EU\nlNfSTK94b0/cNwF/CZzWXmnhsWx9/eGraUbzXwpcRfPFtadX1TUDYuv397XgjcBd2+2cTlNU97Z9\nE80c72uSvHGJ/V3c50o9j2Z6yz/QfLnvZuA4+NHv8Zltm2vbn88cMK9a0oxK1aSnVUmSJEmrzxFf\nSZIkdYKFryRJkjrBwleSJEmdYOErSZKkTrDwlSRJUidY+EqSJKkTLHwlSZLUCRa+kiRJ6gQLX0mS\nJHXC/wekWH4iVysTFwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10de5cef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "inits = [3, 4, 5]\n",
    "T = 10\n",
    "\n",
    "fig, axes = plt.subplots(1, 3, figsize=(12, 3))\n",
    "\n",
    "for init, ax in zip(inits, axes):\n",
    "    draw_histogram(cross_sectional_dist(mc1, T=T, init=init), ax=ax,\n",
    "                   title='Initial state = {0}'.format(init),\n",
    "                   xlabel='States')\n",
    "\n",
    "fig.suptitle('Cross sectional distribution at T={0}'.format(T),\n",
    "             y=-0.05, fontsize=12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Powers of $P$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The marginal distributions at time $T$ are obtained by $P^T$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.set_printoptions(suppress=True)  # Suppress printing with floating point notation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P^10 =\n",
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.33398438  0.          0.          0.66601562  0.        ]\n",
      " [ 0.49807228  0.16679179  0.00001694  0.0007677   0.33319974  0.00115155]\n",
      " [ 0.99902344  0.          0.          0.00097656  0.          0.        ]\n",
      " [ 0.          0.33300781  0.          0.          0.66699219  0.        ]\n",
      " [ 0.99902344  0.          0.          0.          0.          0.00097656]]\n",
      "P^20 =\n",
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.33333397  0.          0.          0.66666603  0.        ]\n",
      " [ 0.49999809  0.16666679  0.          0.00000076  0.33333321  0.00000114]\n",
      " [ 0.99999905  0.          0.          0.00000095  0.          0.        ]\n",
      " [ 0.          0.33333302  0.          0.          0.66666698  0.        ]\n",
      " [ 0.99999905  0.          0.          0.          0.          0.00000095]]\n",
      "P^30 =\n",
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.33333333  0.          0.          0.66666667  0.        ]\n",
      " [ 0.5         0.16666667  0.          0.          0.33333333  0.        ]\n",
      " [ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.33333333  0.          0.          0.66666667  0.        ]\n",
      " [ 1.          0.          0.          0.          0.          0.        ]]\n"
     ]
    }
   ],
   "source": [
    "Ts = [10, 20, 30]\n",
    "for T in Ts:\n",
    "    print('P^{T} =\\n{P_T}'.format(T=T, P_T=np.linalg.matrix_power(mc1.P, T)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the canonical form:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Q =\n",
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.          1.          0.          0.          0.        ]\n",
      " [ 0.          0.5         0.5         0.          0.          0.        ]\n",
      " [ 0.          0.          0.33333333  0.33333333  0.33333333  0.        ]\n",
      " [ 0.5         0.          0.          0.          0.          0.5       ]\n",
      " [ 0.5         0.          0.          0.          0.5         0.        ]]\n",
      "Q^10 =\n",
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.33398438  0.66601562  0.          0.          0.        ]\n",
      " [ 0.          0.33300781  0.66699219  0.          0.          0.        ]\n",
      " [ 0.49807228  0.16679179  0.33319974  0.00001694  0.0007677   0.00115155]\n",
      " [ 0.99902344  0.          0.          0.          0.00097656  0.        ]\n",
      " [ 0.99902344  0.          0.          0.          0.          0.00097656]]\n",
      "Q^20 =\n",
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.33333397  0.66666603  0.          0.          0.        ]\n",
      " [ 0.          0.33333302  0.66666698  0.          0.          0.        ]\n",
      " [ 0.49999809  0.16666679  0.33333321  0.          0.00000076  0.00000114]\n",
      " [ 0.99999905  0.          0.          0.          0.00000095  0.        ]\n",
      " [ 0.99999905  0.          0.          0.          0.          0.00000095]]\n",
      "Q^30 =\n",
      "[[ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 0.          0.33333333  0.66666667  0.          0.          0.        ]\n",
      " [ 0.          0.33333333  0.66666667  0.          0.          0.        ]\n",
      " [ 0.5         0.16666667  0.33333333  0.          0.          0.        ]\n",
      " [ 1.          0.          0.          0.          0.          0.        ]\n",
      " [ 1.          0.          0.          0.          0.          0.        ]]\n"
     ]
    }
   ],
   "source": [
    "Q = mc1.P[permutation, :][:, permutation]\n",
    "print('Q =\\n{Q}'.format(Q=Q))\n",
    "for T in Ts:\n",
    "    print('Q^{T} =\\n{Q_T}'.format(T=T, Q_T=np.linalg.matrix_power(Q, T)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Observe that the first three rows, which correspond to the recurrent states,\n",
    "are close to the stationary distributions associated with the corresponding recurrent classes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2: Periodic chain"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the Markov chain given by the following stochastic matrix,\n",
    "taken from Exercise 9 (see also Exercise 11) in Jarvis and Shier\n",
    "(where the actual values of non-zero probabilities are not important):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "P = np.zeros((10, 10))\n",
    "P[0, 3] = 1\n",
    "P[1, [0, 4]] = 1/2\n",
    "P[2, 6] = 1\n",
    "P[3, [1, 2, 7]] = 1/3\n",
    "P[4, 3] = 1\n",
    "P[5, 4] = 1\n",
    "P[6, 3] = 1\n",
    "P[7, [6, 8]] = 1/2\n",
    "P[8, 9] = 1\n",
    "P[9, 5] = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.     0.     0.     1.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.5    0.     0.     0.     0.5    0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     1.     0.     0.     0.   ]\n",
      " [ 0.     0.333  0.333  0.     0.     0.     0.     0.333  0.     0.   ]\n",
      " [ 0.     0.     0.     1.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     1.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.5    0.     0.5    0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.     0.     0.     1.   ]\n",
      " [ 0.     0.     0.     0.     0.     1.     0.     0.     0.     0.   ]]\n"
     ]
    }
   ],
   "source": [
    "np.set_printoptions(precision=3)  # Reduce the number of digits printed\n",
    "print(P)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "mc2 = MarkovChain(P)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This Markov chain is irreducible:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc2.is_irreducible"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This Markov chain is periodic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc2.is_aperiodic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Its period, which we denote by $d$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n"
     ]
    }
   ],
   "source": [
    "d = mc2.period\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Identify the cyclic classes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0, 4, 6, 8]), array([3, 9]), array([1, 2, 5, 7])]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mc2.cyclic_classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cyclic normal form"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If a Markov chain is periodic with period $d \\geq 2$,\n",
    "then its transition probability matrix is written in the form (\"cyclic normal form\")\n",
    "$$\n",
    "\\begin{pmatrix}\n",
    "0       & P_0    & 0      & 0      & \\cdots & 0 \\\\\n",
    "0       & 0      & P_1    & 0      & \\cdots & 0 \\\\\n",
    "0       & 0      & 0      & P_2    & \\cdots & 0 \\\\\n",
    "\\vdots  & \\vdots & \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "0       & 0      & 0      & 0      & \\cdots & P_{d-2} \\\\\n",
    "P_{d-1} & 0      & 0      & 0      & \\cdots & 0\n",
    "\\end{pmatrix}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Represent our Markov chain in cyclic normal form:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     1.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.     0.     1.     0.   ]\n",
      " [ 0.5    0.5    0.     0.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     1.     0.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     1.     0.     0.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.5    0.5    0.     0.     0.     0.     0.     0.   ]]\n"
     ]
    }
   ],
   "source": [
    "permutation = np.concatenate(mc2.cyclic_classes)\n",
    "Q = mc2.P[permutation, :][:, permutation]\n",
    "print(Q)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Re-define the Markov chain with the above matrix `Q`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mc2 = MarkovChain(Q)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obtain the block components $P_0, \\cdots, P_{d-1}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P_0 =\n",
      "[[ 1.  0.]\n",
      " [ 1.  0.]\n",
      " [ 1.  0.]\n",
      " [ 0.  1.]]\n",
      "P_1 =\n",
      "[[ 0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     1.     0.   ]]\n",
      "P_2 =\n",
      "[[ 0.5  0.5  0.   0. ]\n",
      " [ 0.   0.   1.   0. ]\n",
      " [ 0.   1.   0.   0. ]\n",
      " [ 0.   0.   0.5  0.5]]\n"
     ]
    }
   ],
   "source": [
    "P_blocks = []\n",
    "\n",
    "for i in range(d):\n",
    "    P_blocks.append(mc2.P[mc2.cyclic_classes[i%d], :][:, mc2.cyclic_classes[(i+1)%d]])\n",
    "    print('P_{i} =\\n{P_block}'.format(i=i, P_block=P_blocks[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$P^d$ is block diagonal:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.167  0.167  0.5    0.167  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.167  0.167  0.5    0.167  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.167  0.167  0.5    0.167  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     1.     0.     0.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.833  0.167  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.167  0.167  0.5    0.167]]\n"
     ]
    }
   ],
   "source": [
    "P_power_d = np.linalg.matrix_power(mc2.P, d)\n",
    "print(P_power_d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0th diagonal block of P^d =\n",
      "[[ 0.167  0.167  0.5    0.167]\n",
      " [ 0.167  0.167  0.5    0.167]\n",
      " [ 0.167  0.167  0.5    0.167]\n",
      " [ 0.     1.     0.     0.   ]]\n",
      "1st diagonal block of P^d =\n",
      "[[ 0.833  0.167]\n",
      " [ 1.     0.   ]]\n",
      "2nd diagonal block of P^d =\n",
      "[[ 0.333  0.333  0.     0.333]\n",
      " [ 0.333  0.333  0.     0.333]\n",
      " [ 0.333  0.333  0.     0.333]\n",
      " [ 0.167  0.167  0.5    0.167]]\n"
     ]
    }
   ],
   "source": [
    "P_power_d_blocks = []\n",
    "ordinals = ['0th', '1st', '2nd']\n",
    "\n",
    "for i in range(d):\n",
    "    P_power_d_blocks.append(P_power_d[mc2.cyclic_classes[i], :][:, mc2.cyclic_classes[i]])\n",
    "    print('{ordinal} diagonal block of P^d =\\n{P_power_d_block}'\n",
    "          .format(ordinal=ordinals[i], P_power_d_block=P_power_d_blocks[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The $i$th diagonal block of $P^d$ equals $P_i P_{i+1} \\cdots P_{d-1} P_0 \\cdots P_{i-1}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P_0 P_1 P_2 =\n",
      "[[ 0.167  0.167  0.5    0.167]\n",
      " [ 0.167  0.167  0.5    0.167]\n",
      " [ 0.167  0.167  0.5    0.167]\n",
      " [ 0.     1.     0.     0.   ]]\n",
      "P_1 P_2 P_0 =\n",
      "[[ 0.833  0.167]\n",
      " [ 1.     0.   ]]\n",
      "P_2 P_0 P_1 =\n",
      "[[ 0.333  0.333  0.     0.333]\n",
      " [ 0.333  0.333  0.     0.333]\n",
      " [ 0.333  0.333  0.     0.333]\n",
      " [ 0.167  0.167  0.5    0.167]]\n"
     ]
    }
   ],
   "source": [
    "products = []\n",
    "\n",
    "for i in range(d):\n",
    "    R = np.eye(P_blocks[i].shape[0])\n",
    "    string = ''\n",
    "    for j in range(d):\n",
    "        R = R.dot(P_blocks[(i+j)%d])\n",
    "        string += 'P_{0} '.format((i+j)%d)\n",
    "    products.append(R)\n",
    "    print(string + '=\\n{R}'.format(R=R))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "True\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "for matrix0, matrix1 in zip(P_power_d_blocks, products):\n",
    "    print(np.array_equal(matrix0, matrix1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stationary distributions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Markov chain `mc2` has a unique stationary distribution,\n",
    "which we denote by $\\pi$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(mc2.stationary_distributions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.048  0.095  0.143  0.048  0.286  0.048  0.095  0.095  0.048  0.095]\n"
     ]
    }
   ],
   "source": [
    "pi = mc2.stationary_distributions[0]\n",
    "print(pi)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEZCAYAAACD/A7qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGTxJREFUeJzt3X+8XHV95/HXu4lRIfy0WCSAsRpWpOIi3RhF17Fm2Vu0\nhq19bBZBFnWV3W0s/ug+WLa7crXdWtplt9vSpVFTi4pGyiJNKxqhOojAQlKRXyY8SCU1CYKUSPgh\n1MS8949zbjJMZu5Mcn/M3O99Px+PPDLnfL/fM59zbvKeM985Z65sExER5fiZQRcQERGTK8EeEVGY\nBHtERGES7BERhUmwR0QUJsEeEVGYBHsMhKSzJa0ddB37S9J5km5qWX5C0sJJ2vZFkj5ZP14oabek\nSfk/Kun4ulZNxvZiuCXYoyNJr5d0i6THJD0q6VuSfrFue1a49bGtfULK9pW2/+VU1D6dbB9ie/N4\nfSQ1JG3pY1sft/3eyahL0mZJv9Sy7e/XtebGlVlg7qALiOEj6VDgr4HzgauA5wJvAP5xopue4PgJ\nk/QztncPuo52kubY/ukkbtIMwfGOwcgZe3RyAmDbX3TlGdvX275b0onA5cBr67f22wEkvUXSHZJ2\nSPq+pItbtvfN+u/HJD0uaUmHKY3XSVpXv0O4XdJrW9qakj5Wv2t4XNJaSS9oaf8LST+ox94o6RUt\nbX8u6XJJ10l6EviQpIda3z1I+lVJ3+l0ICS9QNKaer9uA17a1r5b0s/Xj8+QdG9d41ZJH5J0EPAV\n4Jj6eD0u6UWSRiVdLemzknYA59XrPttWwnskbZP0oKQPt+3Xb7cs73lXUG/jeOCv6uf8zfZ3TZKO\nqffrUUn3S/p3LdsalXSVpCvqeu+RdGqn4xPDKcEendwH/LQOjxFJR4w12N4A/Hvg1vqt/ZF105PA\nObYPA94C/AdJy+q2N9R/H2b7UNv/r/XJJB0JfBn4Q+BI4H8CX259XuAs4DzghcA84Ddb2r4MvAw4\nCvg2cGXb/pwF/Lbt+cAfA48Cp7e0vxO4osux+BPgx8DRwLuBd1GdDXeyCnif7UOBk4Bv2P4xMAI8\nWB+vQ23/oO7/NuAv6mN2ZZftNup9Ox24UNKb6/XuVoftdwLfB95aP+f/6NBtdd3nRcCvAb8r6U0t\n7b8CfAE4DFgDXNZln2MIJdhjH7afAF5PFRyfBH4o6S8lvbDuss9bfNs32r63fnw3VXC8sVv/Nm8B\n7qvn3XfbXg1spAo+6jo+bXuT7Weopof+actz/7ntp2zvBD4KvErSIS3bv9b2rXXff6QK8XNgz4vK\n6cDn24uSNAf4VeAjtp+u9++KcfbnJ8BJkg61vcP2HT32/xbba+q6nunS76P1c98DfJrqRWpPiV22\nOy5JxwGvAy60/RPbdwKfAs5t6XaT7a/Wc/KfA151IM8Vg5Fgj45sb7T9LtvHAb8AHEN1Rt2RpNdI\n+oakH0p6jGp+/gXd+rc5hurssdXf1+vHPNTy+Glgfv28cyT9nqRN9ZTGA3Wfnx3bFaD9g8srgV+p\np0n+NfBN2w93qOsoqs+hWse319nq7cAZwOZ6+mjJOH0BtvZop8NzH9Ot4344Bthu+6m2bS9oWW49\nHj8GnqdJukInpl5+UNGT7fuozlR/YWxVh26fB64FjrV9OPCn7P331etKjG3Ai9vWvbhe38s7qM7s\n31xPabykXt/1bNb2NuBWqrPxc4D2ee0xjwC7qOarxxzfpS+219s+k+oF4VqqdxbQef87TaV06tf+\n3GPH5CngoJa2o/vY1pgHgSMlzW/bdj8vNDEDJNhjH5L+Sf3B34J6+TiqKYBb6y4PA8dKek7LsPnA\nj2z/RNJiqsAdC5dHgN20ffDY4ivACZLOkjRX0nLg5VRX5uwpq8vY+VRX62yXdDDwu+2702XcZ4AL\nqV6srunUob5K5RpgVNLz6w9l/22nvpKeo+ra/MPqcU8AY1e5PAy8QNXVRuPV1Wndf62f+ySqzxi+\nWK//DnCGpCMkHQ18oG3cw3Q53ra3ALcAH5f0XEknU31+8LlO/WPmSbBHJ08ArwFuq68kuRW4Cxi7\nKuNvgHuBhyT9sF73H4GPSXoc+G/sDSDqDxD/O3CzpO2SXkPLGavtR4G31tv/B6oPRt9qe3tLTW57\nPLb8Gappm23APXWt3fq2+hLVWeqX6vntblZQvXg8BPxZ/ad9+2POAR6op4TeB5xd799Gqg8iv1fv\n/4u61NW+zsCNwCbgBuAPbN9Qt30WuBPYDHyV6jON1rEfp3pR+JGkD3Wo9SxgIdXZ+zVUnyN8vUsd\n7WNjyKnX/QqSRqjmVucAn7J9SVv7MuBjVGdku4AP2L65btsMPE515rLT9uLJ3oGIAyVpE9VVLF/v\n2TliBhk32OurAu4DllKdEa0DzqoveRvrc/DYhzCSXglcZfvEevkB4NS2M6+IgZP0duDjtk8YdC0R\nk63XnaeLgU1jt0xLWg0sA/YEe9sn6/Opztxb5e63GCqSmlRz+O8ccCkRU6JXsC/g2ZdbbaWae30W\nSWdSzem9kOpyrzEGbpD0U2Cl7U9OrNyIibPdGHQNEVOp14enfX1gYvvaevrlTOB3WppOs30K8MvA\nr0t6Q8cNRETEpOl1xr4NOK5l+TjGudbV9k2Sfl7Skba3j906bfsRSV+imtp51rcCSsqn7RERB8B2\nx6nuXmfs64FF9RcIzQOWU31vxB6SXipV3/Es6dXAPNvbJR00dlt3fX3x6cDdXYqb0J+LL754wtso\n5U+ORY5FjsXsOBbjGfeM3fYuSSuAtVSXO66yvUHS+XX7SqrbqM+VtJPqVu/l9fCjgWvqzJ8LXGn7\na+NWExERE9bz+9htf4XqzsDWdStbHv8+8Psdxn2Pli9qioiI6VHEnaeNRmPQJQyNHIu9ciz2yrHY\nazYci553nk55AZIHXUNExEwjCR/gh6cRETHDJNgjIgqTYI+IKEyCPSKiMAn2iIjCJNgjIgqTYI+I\nKEyCPSKiMAn2iIjCJNgjIgqTYI+IKEyCPSKiMAn2iIjCJNgjIgqTYI+IKEyCPSKiMAn2iIjCJNgj\nIgqTYI+IKEyCPSKiMAn2iIjC9Ax2SSOSNkq6X9KFHdqXSbpT0h2S1kk6rd+xEREx+WS7e6M0B7gP\nWApsA9YBZ9ne0NLnYNtP1Y9fCVxl+8R+xtZjPF4NERGxL0nYVqe2Xmfsi4FNtjfb3gmsBpa1dhgL\n9dp8YHe/YyMiYvL1CvYFwJaW5a31umeRdKakDcBfA+/en7ERETG55vZo72uOxPa1wLWS3gD8DvAv\n9qeI0dHRPY8bjQaNRmN/hkdEFK/ZbNJsNvvq22uOfQkwanukXr4I2G37knHG/B3wz4AT+hmbOfaI\niP03kTn29cAiSQslzQOWA2vaNv5SSaofvxqYZ3t7P2MjImLyjTsVY3uXpBXAWmAOsMr2Bknn1+0r\ngbcD50raCTxNFeBdx07drkREBPSYipmWAjIVExGx3yYyFRMRETNMgj0iojAJ9oiIwiTYIyIKk2CP\niChMrztPIwauvk1i2uQqrZjpEuwxQ0xX2E7vi0jEVMhUTEREYRLsERGFSbBHRBQmwR4RUZgEe0RE\nYRLsERGFSbBHRBQmwR4RUZgEe0REYRLsERGFSbBHRBQmwR4RUZgEe0REYRLsERGFSbBHRBQmwR4R\nUZiewS5pRNJGSfdLurBD+9mS7pR0l6SbJZ3c0ra5Xn+HpNsnu/iIiNjXuL9BSdIc4DJgKbANWCdp\nje0NLd2+B/xz2zskjQCfAJbUbQYatrdPfukREdFJrzP2xcAm25tt7wRWA8taO9i+1faOevE24Ni2\nbeR3jUVETKNewb4A2NKyvLVe1817gOtalg3cIGm9pPceWIkREbE/ev0y675/g7CkNwHvBk5rWX2a\n7R9IOgq4XtJG2ze1jx0dHd3zuNFo0Gg0+n3aiIhZodls0mw2++oru3t2S1oCjNoeqZcvAnbbvqSt\n38nANcCI7U1dtnUx8KTtS9vWe7waIiSxH+cYE3028u8xZgJJ2O441d1rKmY9sEjSQknzgOXAmraN\nH08V6ue0hrqkgyQdUj8+GDgduPvAdyMiIvox7lSM7V2SVgBrgTnAKtsbJJ1ft68EPgIcAVxenVmx\n0/Zi4GjgmnrdXOBK21+bsj2JiAigx1TMtBSQqZjoIVMxEfuayFRMRETMMAn2iIjCJNgjIgqTYI+I\nKEyCPSKiMAn2iIjCJNgjIgqTYI+IKEyCPSKiMAn2iIjCJNgjIgqTYI+IKEyCPSKiMAn2iIjCJNgj\nIgqTYI+IKEyCPSKiMAn2iIjCJNgjIgqTYI+IKEyCPSKiMAn2iIjCJNgjIgrTM9gljUjaKOl+SRd2\naD9b0p2S7pJ0s6ST+x0bERGTT7a7N0pzgPuApcA2YB1wlu0NLX1eC3zX9g5JI8Co7SX9jK3He7wa\nIiQB0/VvROTfY8wEkrCtTm29ztgXA5tsb7a9E1gNLGvtYPtW2zvqxduAY/sdGxERk69XsC8AtrQs\nb63XdfMe4LoDHBsREZNgbo/2vt+TSnoT8G7gtP0dOzo6uudxo9Gg0Wj0OzQiYlZoNps0m82++vaa\nY19CNWc+Ui9fBOy2fUlbv5OBa4AR25v2c2zm2GNcmWOP2NdE5tjXA4skLZQ0D1gOrGnb+PFUoX7O\nWKj3OzYiIibfuFMxtndJWgGsBeYAq2xvkHR+3b4S+AhwBHB5dWbFTtuLu42dwn2JiAh6TMVMSwGZ\niokeMhUTsa+JTMVERMQMk2CPiChMgj0iojAJ9oiIwiTYIyIKk2CPiChMgj0iojAJ9oiIwiTYIyIK\nk2CPiChMgj0iojAJ9oiIwiTYIyIKk2CPiChMgj0iojAJ9oiIwiTYIyIKk2CPiChMgj0iojAJ9oiI\nwiTYIyIKk2CPiChMgj0iojA9g13SiKSNku6XdGGH9pdLulXSM5I+3Na2WdJdku6QdPtkFh4REZ3N\nHa9R0hzgMmApsA1YJ2mN7Q0t3R4F3g+c2WETBhq2t09SvRER0UOvM/bFwCbbm23vBFYDy1o72H7E\n9npgZ5dtaOJlRkREv3oF+wJgS8vy1npdvwzcIGm9pPfub3EREbH/xp2KoQrmiTjN9g8kHQVcL2mj\n7ZvaO42Oju553Gg0aDQaE3zaiIiyNJtNms1mX31ld89uSUuAUdsj9fJFwG7bl3ToezHwpO1Lu2yr\nY7skj1dDhCQmfo7R97ORf48xE0jCdsep7l5TMeuBRZIWSpoHLAfWdHuetic9SNIh9eODgdOBu/er\n8oiI2G/jTsXY3iVpBbAWmAOssr1B0vl1+0pJRwPrgEOB3ZIuAF4BvBC4pjrbYi5wpe2vTd2uREQE\n9JiKmZYCMhUTPWQqJmJf403F9PrwNAaofrczLRJmEeVIsA+96Qjc3GoQUZJ8V0xERGES7BERhUmw\nR0QUJsEeEVGYBHtERGES7BERhUmwR0QUJsEeEVGYBHtERGES7BERhUmwR0QUJsEeEVGYBHtERGES\n7BERhUmwR0QUJsEeEVGYBHtERGES7BERhUmwR0QUJsEeEVGYBHtERGF6BrukEUkbJd0v6cIO7S+X\ndKukZyR9eH/GRkTE5JPt7o3SHOA+YCmwDVgHnGV7Q0ufo4AXA2cCP7J9ab9j634er4bZTBIwHcdG\nDPPPYPqOAwz7sYgYIwnb6tTW64x9MbDJ9mbbO4HVwLLWDrYfsb0e2Lm/YyMiYvL1CvYFwJaW5a31\nun5MZGxERByguT3aJ/KetO+xo6Ojex43Gg0ajcYEnjYiojzNZpNms9lX315z7EuAUdsj9fJFwG7b\nl3ToezHwZMsce19jM8feXebYK5ljj9jXRObY1wOLJC2UNA9YDqzp9jwTGBsREZNk3KkY27skrQDW\nAnOAVbY3SDq/bl8p6WiqK14OBXZLugB4he0nO42dyp2ZLNUZ4vTJGWL0a1j+bQ5LHcNgGI/FuFMx\n02EYp2KG5a1/pmIqw/LzGAbDciyGpY5hMKhjMZGpmIiImGES7BERhUmwR0QUJsEeEVGYBHtERGES\n7BERhUmwR0QUJsEeEVGYBHtERGES7BERhUmwR0QUJsEeEVGYBHtERGES7BERhUmwR0QUJsEeEVGY\nBHtERGES7BERhUmwR0QUJsEeEVGYBHtERGES7BERhUmwR0QUpmewSxqRtFHS/ZIu7NLnj+r2OyWd\n0rJ+s6S7JN0h6fbJLDwiIjqbO16jpDnAZcBSYBuwTtIa2xta+pwBvMz2IkmvAS4HltTNBhq2t09J\n9RERsY9eZ+yLgU22N9veCawGlrX1eRtwBYDt24DDJf1cS7smq9iIiOitV7AvALa0LG+t1/Xbx8AN\nktZLeu9ECo2IiP6MOxVDFcz96HZW/nrbD0o6Crhe0kbbN7V3Gh0d3fO40WjQaDT6fNqIiNmiCTw7\nL7uR3T27JS0BRm2P1MsXAbttX9LS50+Bpu3V9fJG4I22H27b1sXAk7YvbVvv1hqk6Zu56bbvVQ39\nvqZNlIagjl41TJ9OdQzXz2P6DP+xGGwdw/Dz2FvH9B8LSdjueBB6TcWsBxZJWihpHrAcWNPWZw1w\nbv1ES4DHbD8s6SBJh9TrDwZOB+7ur3hPw5/o33T8PGbKzyTHYrjk59HJuFMxtndJWgGsBeYAq2xv\nkHR+3b7S9nWSzpC0CXgKeFc9/GjgmvpVdS5wpe2vTdWOREREZdypmGkpoONUzDBMP8ymt7vDUEP3\nOoahhmGpYxhqGJY6hqGGQdYxkamYiIiYYRLsERGFSbBHRBQmwR4RUZgEe0REYRLsERGFSbBHRBQm\nwR4RUZgEe0REYRLsERGFSbBHRBQmwR4RUZgEe0REYRLsERGFSbBHRBQmwR4RUZgEe0REYRLsERGF\nSbBHRBQmwR4RUZgEe0REYRLsERGF6RnskkYkbZR0v6QLu/T5o7r9Tkmn7M/YiIiYXOMGu6Q5wGXA\nCPAK4CxJJ7b1OQN4me1FwPuAy/sdO3maU7PZGak56AKGSHPQBQyR5qALGCLNQRcw5XqdsS8GNtne\nbHsnsBpY1tbnbcAVALZvAw6XdHSfYydJc2o2OyM1B13AEGkOuoAh0hx0AUOkOegCplyvYF8AbGlZ\n3lqv66fPMX2MjYiISdYr2N3ndjTRQiIiYnLM7dG+DTiuZfk4qjPv8focW/d5Th9jAZDaXxcO5HXi\no/s9Yt/nnWgNB2by6yjxWBxoDTkWe+VY7FXisdirV7CvBxZJWgg8CCwHzmrrswZYAayWtAR4zPbD\nkh7tYyy2c7YfETGJxg1227skrQDWAnOAVbY3SDq/bl9p+zpJZ0jaBDwFvGu8sVO5MxERAbL7nUaP\niIiZYEbfeZoboPaSdJykb0i6V9I9kn5j0DUNkqQ5ku6Q9FeDrmWQJB0u6WpJGyR9t54unZUkfbD+\nv3G3pM9Leu6ga5oqMzbYp/cGqBlhJ/BB2ycBS4Bfn+XH4wLgu/R/ZVep/jdwne0TgZOBWTkdKmkB\n8H7gVNuvpJoe/jeDrWrqzNhgZ1pvgBp+th+y/Z368ZNU/4GPGWxVgyHpWOAM4FPM4ktxJR0GvMH2\nn0H1uZftHQMua5DmAgdJmgscRHVFX5FmcrD3c/PUrFRfiXQKcNtgKxmY/wX8J2D3oAsZsJcAj0j6\ntKRvS/qkpIMGXdQg2N4GXAp8n+oqvcds3zDYqqbOTA722f4WuyNJ84GrgQvqM/dZRdJbgR/avoNZ\nfLZemwu8Gvg/tl9NddXafx5sSYMh6Qiqrz9ZSPVOdr6kswda1BSaycHez81Ts4qk5wD/F/ic7WsH\nXc+AvA54m6QHgC8AvyTpMwOuaVC2Alttr6uXr6YK+tloKfCA7Udt7wKuofq3UqSZHOx7bp6SNI/q\nBqg1A65pYFTdjrYK+K7tPxx0PYNi+7/YPs72S6g+HPu67XMHXdcg2H4I2CLphHrVUuDeAZY0SH8P\nLJH0/Pr/ylKqD9eL1OvO06GVG6D2cRpwDnCXpDvqdRfZ/uoAaxoGs33K7v3AlfXJz99R30A429i+\nXdLVwLeBXfXfnxhsVVMnNyhFRBRmJk/FREREBwn2iIjCJNgjIgqTYI+IKEyCPSKiMAn2iIjCJNij\neJJ+q/661jvrr/JdLOkCSc/vY+wH+ukXMUxyHXsUTdJrqb786Y22d0o6EngecDPwi7Yf7TH+gX76\nRQyTnLFH6Y4G/qH+amdsbwd+jeqLoL4h6W8AJF0uaV19Zj9ar/uNDv1Ol3SLpL+VdJWkg+v1v1f/\nkpM7Jf3BtO9lRIucsUfR6uD9FtX3b98AfNH2N+sz8VProEfSEbZ/VP8ClxuA99u+p7WfpJ+l+pK1\nEdtP17+1ax7wJ8Attl9eb+tQ249P+85G1HLGHkWz/RRwKvA+4BHgi5LOq5tbv9Z3uaS/pfoOkZOo\nfitXuyX1+lvq7+M5Fzge2AE8I2mVpH8FPD0V+xLRrxn7JWAR/bK9G7gRuFHS3cB5Y00Akl4CfJhq\nLn2HpE9TzcN3cr3td7SvlLQYeDPVNM+K+nHEQOSMPYom6QRJi1pWnQJsBp4ADq3XHUr1Sygel/Rz\nwC+39G/tdxtwmqSX1ts+WNKierrncNtfAT4EvGqq9ieiHzljj9LNB/5Y0uFUX9d6P9W0zDuAr0ra\nZvvN9dTKRqpft/itlvGfaOt3HvCFlt9w/1tU4f+Xkp5HNb3zwenYsYhu8uFpRERhMhUTEVGYBHtE\nRGES7BERhUmwR0QUJsEeEVGYBHtERGES7BERhUmwR0QU5v8D00cQAgxTCmEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10de34240>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "draw_histogram(pi,\n",
    "               title='Stationary distribution', xlabel='States', ylim=(0, 0.35))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Obtain the stationary distributions $\\pi^0, \\ldots, \\pi^{d-1}$\n",
    "each associated with the diagonal blocks of $P^d$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pi^0 =\n",
      "[ 0.143  0.286  0.429  0.143]\n",
      "pi^1 =\n",
      "[ 0.857  0.143]\n",
      "pi^2 =\n",
      "[ 0.286  0.286  0.143  0.286]\n"
     ]
    }
   ],
   "source": [
    "pi_s = []\n",
    "\n",
    "for i in range(d):\n",
    "    pi_s.append(MarkovChain(P_power_d_blocks[i]).stationary_distributions[0])\n",
    "    print('pi^{i} =\\n{pi_i}'.format(i=i, pi_i=pi_s[i]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAD6CAYAAABH0UQmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X20bXVd7/H3x3MQRJ7lhh04hiaYYJpZhFnXnTD0RIpd\nbaT4FNZNxrhh5sONLJOjVmRd0wojNXw2sNQM71C4mWyfIPABQRAIFPRwUOQZIsxz5Hv/mHPDOou9\n9t5n77XXw57v1xhrnDXX/K05v3PO9d3nu37zN+dKVSFJkiStdQ8YdwCSJEnSKFj4SpIkqRMsfCVJ\nktQJFr6SJEnqBAtfSZIkdYKFryRJkjrBwleSJEmdsH7cAWj0kuwH/CbwXeCSqvrSmEOStIAkewNH\nAY+qqlPGHY+kwZIcAjwGeCzwsar68phDUg97fLvp14BzgfcBrxhzLJIWUVW3A18CHjjuWCQt6unA\nVuAvgFeNORb1sfDtpkcA366q7cB+4w5GkqS1oqreXFUXAhuBa8Ydj3Zk4dtNDwB+0D73N6slSRq+\nXwb+eNxBaEcWvt10JXBAkt2AO8YdjKQlybgDkLQ0SY4F/ho4cNyxaEcWvt30AeBo4EXAX445FkmL\nSLIn8GzgCUkeM+54JA2W5FnAHwIfAX51zOGoT6o8071WtVeC/wVwKLAN2IdmwP3nquqN44xN0v2Z\ns9L0WCBfP1tVfzbO2DSYhe8aluRoYJbmLg7vAX6rquzhlSaUOStND/N1Onkf3zWsqj4JkOQRVbU9\nyUHjjknSYOasND3M1+nkGN81LslRNKdeAH5inLFIWpw5K00P83X6WPiufb8G/HP7fI8kXhkuTTZz\nVpoe5uuUcYyvJEmSOmHRHt8k70xyQ5KvLtDmr5JcleTiJI8fboiSlsp8laaH+SqN3lKGOrwL2DRo\nZpJjgEdW1SHAS4DThhSbpJ1nvkrTw3yVRmzRwreqPgvcukCTY2lu40FVXQDsk+SA4YQnaWeYr9L0\nMF+l0RvGxW0HAlt6pq8DvKWHNJnMV2l6mK/SkA3rrg79VzF6xZw0ucxXaXqYr9IQDeMHLLYCG3um\nD+K+e9rdK4nJKvWpqlHf+sZ8lZZpUvMVzFlpPvPl7DB6fM8CXgSQ5Ejgtqq6YUAAiz5OPvnkJbUb\nx2OSY5v0+Izt/o8x6Uy+Tnp8xjZd8Y3JkvMVFs9Zj+vajG3S45u0/2MX7fFNcgbwZGD/JFuAk4Fd\n2iR7W1V9PMkxSa4G7gJevBNJLWmIzFdpepiv0ugtWvhW1XFLaHPicMKRtBLmqzQ9zFdp9CbuJ4tn\nZmbGHcJAkxwbTHZ8xrY2Tfq+m+T4jG35Jj2+STXp+22S45vk2GCy45u02Eb2k8VJalTrkqZBEmr0\nF8ssifkq7WiS8xXMWanfoJyduB5fSZIkaTVY+EqSJKkTLHwlSZLUCRa+kiRJ6gQLX0mSJHWCha8k\nSZI6wcJXkiRJnWDhK0mSpE6w8JUkSVInWPhKkiSpEyx8JUmS1AkWvpIkSeoEC19JkiR1goWvJEmS\nOsHCV5IkSZ1g4StJkqROsPCVJElSJ1j4SpIkqRPWjzsASVqLkqx4GVU1hEgkSXMsfCVp1aykcF15\n4SxJ2pFDHSRJktQJFr6SJEnqhEUL3ySbklyR5KokJ80zf+8kH0vylSSXJjl+VSKVtCTmrDQ9zFdp\ntLLQxRNJ1gFXAkcDW4EvAMdV1eU9bX4f2LOqXp1k/7b9AVW1vW9Z5YUa0n2SUFVDHcg5rJw1X1eu\nubhtZWN8PQaTY5LztW1nzko9BuXsYj2+RwBXV9W1VbUNOBN4Zl+be4C92ud7ATf3J6SkkTFnpelh\nvkojtljheyCwpWf6uva1XqcChyW5HrgYeNnwwpO0k8xZaXqYr9KILVb4LuW8ySbgy1W1AfgJ4K1J\n9lxxZJKWw5yVpof5Ko3YYvfx3Qps7JneSPONtNfxwCkAVfX1JNcAjwK+2L+wzZs33/t8ZmaGmZmZ\nnY1Xmlqzs7PMzs6u9mqGlrPmq7ps2vIVzFl121JzdrGL29bTDKQ/CrgeuJD7D7z/G+CGqnpdkgOA\nLwGPrapb+pblwHupxypdLDOUnDVfV86L29aWSc7Xtp05K/UYlLML9vhW1fYkJwLnAOuA06vq8iQn\ntPPfBrwBeHeSS2h+auh3+xNS0miYs9L0MF+l0Vuwx3eoK/LbqLSD1ehBGhbzdeXs8V1bJjlfwZyV\n+i33dmaSJEnSmmDhK0mSpE6w8JUkSVInWPhKkiSpEyx8JUmS1AkWvpIkSeoEC19JkiR1goWvJEmS\nOsHCV5IkSZ1g4StJkqROsPCVJElSJ1j4SpIkqRMsfCVJktQJFr6SJEnqBAtfSZIkdYKFryRJkjrB\nwleSJEmdYOErSZKkTrDwlSRJUidY+EqSJKkTLHwlSZLUCRa+kiRJ6gQLX0mSJHWCha8kSZI6wcJX\nkiRJnbBo4ZtkU5IrklyV5KQBbWaSXJTk0iSzQ49S0pKZs9L0MF+l0UpVDZ6ZrAOuBI4GtgJfAI6r\nqst72uwDfB54WlVdl2T/qrppnmXVQuuSuiYJVZUhL3MoOWu+rlwSYCX7MHgMJsck52vbzpyVegzK\n2cV6fI8Arq6qa6tqG3Am8My+Ns8DPlxV1wHMl5CSRsaclaaH+SqN2GKF74HAlp7p69rXeh0C7Jfk\n3CRfTPLCYQYoaaeYs9L0MF+lEVu/yPylnDfZBfhJ4Chgd+D8JP9WVVetNDhJO82claaH+SqN2GKF\n71ZgY8/0RppvpL22ADdV1d3A3Uk+AzwOuF9Sbt68+d7nMzMzzMzM7HzE0pSanZ1ldnZ2tVcztJw1\nX9Vl05avYM6q25aas4td3LaeZuD9UcD1wIXcf+D9jwGnAk8DdgUuAJ5TVV/rW5YD76Ueq3SxzFBy\n1nxdOS9uW1smOV/bduas1GNQzi7Y41tV25OcCJwDrANOr6rLk5zQzn9bVV2R5GzgEuAe4B39CSlp\nNMxZaXqYr9LoLdjjO9QV+W1U2sFq9CANi/m6cvb4ri2TnK9gzkr9lns7M0mSJGlNsPCVJElSJ1j4\nSpIkqRMsfCVJktQJFr6SJEnqBAtfSZIkdYKFryRJkjrBwleSJEmdYOErSZKkTrDwlSRJUidY+EqS\nJKkTLHwlSZLUCRa+kiRJ6gQLX0mSJHWCha8kSZI6wcJXkiRJnWDhK0mSpE6w8JUkSVInWPhKkiSp\nEyx8JUmS1AkWvpIkSeoEC19JkiR1goWvJEmSOsHCV5IkSZ2waOGbZFOSK5JcleSkBdr9dJLtSZ41\n3BBHJ8mKH9K4dSlnpWlnvkqjtWDhm2QdcCqwCTgMOC7Jowe0eyNwNjDl1V+t4CGNVzdzVppO5qs0\neov1+B4BXF1V11bVNuBM4JnztHsp8CHgxiHHJ2nnmLPS9DBfpRFbrPA9ENjSM31d+9q9khxIk6in\ntS/Z9SmNjzkrTQ/zVRqxxQrfpSTYW4Dfq6qiOQXjaRhpfMxZaXqYr9KIrV9k/lZgY8/0RppvpL2e\nAJzZXti1P/CLSbZV1Vn9C9u8efO9z2dmZpiZmdn5iKUpNTs7y+zs7GqvZmg5a76qy6YtX8GcVbct\nNWfTfIkcMDNZD1wJHAVcD1wIHFdVlw9o/y7gY1X1kXnm1ULrmgTNH5aVxBgmfRs1OZJQVUPtvRlW\nzk5Dvk46/56sLZOcr+08c1bqMShnF+zxrartSU4EzgHWAadX1eVJTmjnv21VopW0LOasND3MV2n0\nFuzxHeqKpuDbqD00GqXV6EEalmnI10nn35O1ZZLzFcxZqd+gnPWX2yRJktQJFr6SJEnqBAtfSZIk\ndYKFryRJkjrBwleSJEmdYOErSZKkTrDwlSRJUidY+EqSJKkTLHwlSZLUCRa+kiRJ6gQLX0mSJHWC\nha8kSZI6wcJXkiRJnWDhK0mSpE6w8JUkSVInrB93ACuVZEXvr6ohRSJJkjR8w6x1Vrqs/uUN22rH\nN/WFb2O5B2DlO1eSJGn1DbPWWUnhOoraafXic6iDJEmSOsHCV5IkSZ1g4StJkqROsPCVJElSJ1j4\nSpIkqRMsfCVJktQJFr6SJEnqBAtfSZIkdcKSCt8km5JckeSqJCfNM//5SS5OckmSzyd57PBDlbQU\n5qs0PcxXabQWLXyTrANOBTYBhwHHJXl0X7NvAP+9qh4LvAF4+7ADlbQ481WaHuarNHpL6fE9Ari6\nqq6tqm3AmcAzextU1flVdXs7eQFw0HDDlLRE5qs0PcxXacSWUvgeCGzpmb6ufW2Q3wA+vpKgJC2b\n+SpND/NVGrH1S2hTS11Ykl8Afh140rIjkrQS5qs0PcxXacSWUvhuBTb2TG+k+Va6g3bA/TuATVV1\n63wL2rx5873PZ2ZmmJmZ2YlQpek2OzvL7Ozsaq/GfJWGYNryFcxZdd1s+9gxF/qlauEvnEnWA1cC\nRwHXAxcCx1XV5T1tHgZ8CnhBVf3bgOXUYutajiTsxJfm/nfTG9PKlnX/5UkLSUJVZcjLnOh87RL/\nnqwtk5yvbTtzdg3rUq0zrPgG5eyiPb5VtT3JicA5wDrg9Kq6PMkJ7fy3Aa8F9gVOawJmW1UdsYKo\nJS2D+SpND/NVGr1Fe3yHtiJ7fKUdrEYP0rDYe7Ry/j1ZWyY5X8GcXeu6VOusdo+vv9wmSZKkTrDw\nlSRJUidY+EqSJKkTLHwlSZLUCRa+kiRJ6oSl/IDFULW3Y1mRablytUvbKml1rfTviX9LJK2Gaat1\nRl74NlZ2m4rp0qVtlbS6ln87I0laPdNT6zjUQZIkSZ1g4StJkqROsPCVJElSJ1j4SpIkqRMsfCVJ\nktQJFr6SJEnqBAtfSZIkdYKFryRJkjrBwleSJEmdYOErSZKkTrDwlSRJUidY+EqSJKkTLHwlSZLU\nCRa+kiRJ6gQLX0mSJHWCha8kSZI6wcJXkiRJnbBo4ZtkU5IrklyV5KQBbf6qnX9xkscPP0xJS2XO\nStPDfJVGa8HCN8k64FRgE3AYcFySR/e1OQZ4ZFUdArwEOG1lIc2u7O2ranbcASxodnZ23CEMZGyj\nMeqcnfR9N9nxzY47gIEme79NfnxLZb7uaJLjm+TYGrPjDmABs+MOYAeL9fgeAVxdVddW1TbgTOCZ\nfW2OBd4DUFUXAPskOWD5Ic0u/62rbnbcASxokhPT2EZmpDk76ftusuObHXcAA032fpv8+HaC+dpj\nkuOb5Ngas+MOYAGz4w5gB4sVvgcCW3qmr2tfW6zNQSsPTdIymLPS9DBfpRFbrPCtJS4ny3yfpOEy\nZ6XpYb5Ko1ZVAx/AkcDZPdOvBk7qa/O3wHN7pq8ADphnWeXDh48dHwvl33IeDClnx71ffPiYxMek\n5qs568PH/I/58m49C/sicEiSg4HrgecAx/W1OQs4ETgzyZHAbVV1Q/+Cqqr/G6uk4RtKzpqv0kj4\nf6w0YgsWvlW1PcmJwDnAOuD0qro8yQnt/LdV1ceTHJPkauAu4MWrHrWkeZmz0vQwX6XRS3uKRJIk\nSVrTJuaX25ZyE+9xSbIxyblJLktyaZLfHndM/ZKsS3JRko+NO5ZeSfZJ8qEklyf5WnuqbmIkeXl7\nTL+a5O+T7DrumKbFpOas+boyk5yz5uvyTWq+gjm7EpOcrzCZOTsRhe9SbuI9ZtuAl1fV4TQXI/zW\nhMUH8DLgazQDuifJXwIfr6pHA48FLh9zPPdKciDwUuAJVfXjNKcanzveqKbDhOes+boyE5mz5uvy\nTXi+gjm7EhOZrzC5OTsRhS9Lu4n32FTVd6rqK+3z/6D5YG0Yb1T3SXIQcAzwd9z/tjdjk2Rv4Oer\n6p3QjGerqtvHHFa/9cDuSdYDuwNbxxzPtJjYnDVfl28KctZ8XZ6JzVcwZ5drCvIVJjBnJ6XwXcpN\nvCdCe/Xt44ELxhvJDt4M/G/gnnEH0ufhwI1J3pXky0nekWT3cQc1p6q2Am8CvkVzRfVtVfXJ8UY1\nNaYiZ83XnTaxOWu+rshU5CuYsztpYvMVJjdnJ6XwnbRTB/NKsgfwIeBl7bfSsUvydOC7VXURE/RN\ntLUe+Engb6rqJ2muSP698YZ0nyT70vwc6ME0vQt7JHn+WIOaHhOfs+brskxszpqvKzLx+Qrm7DJM\nbL7C5ObspBS+W4GNPdMbab6RTowkuwAfBt5fVR8ddzw9fhY4Nsk1wBnAU5K8d8wxzbkOuK6qvtBO\nf4gmSSfF0cA1VXVzVW0HPkKzP7W4ic5Z83XZJjlnzdflm+h8BXN2mSY5X2FCc3ZSCt97b+Kd5IE0\nN/E+a8wx3StJgNOBr1XVW8YdT6+q+v2q2lhVD6cZNP6pqnrRuOOCZtwWsCXJoe1LRwOXjTGkft8E\njkzyoPYYH01z8YIWN7E5a74u34TnrPm6fBObr2DOLteE5ytMaM4u9sttIzHoJt5jDqvXk4AXAJck\nuah97dVVdfYYYxpk0k5pvRT4QPvH9utM0M3Xq+rCJB8Cvgxsb/99+3ijmg4TnrPm68pMZM6ar8s3\n4fkK5uxKTGS+wuTmrD9gIUmSpE6YlKEOkiRJ0qqy8JUkSVInWPhKkiSpEyx8JUmS1AkWvpIkSeoE\nC19JkiR1goXvlEryB0kuTXJxkouSHJHkZUketIT3/s5S2kkaHnNWmh7m69rlfXynUJInAm8CnlxV\n25LsB+wGfB74qaq6eZH3X7OUdpKGw5yVpof5urbZ4zudHgrcVFXbAKrqFuBXgA3AuUn+FSDJaUm+\n0H5r3dy+9tvztHtqkvOSfCnJPyR5cPv6nya5rP3G++cj30pp7TBnpelhvq5h9vhOoTZpPgfsDnwS\n+GBVfab9lvmENklJsm9V3ZpkXdvupVV1aW+7JPsDHwY2VdXdSU4CHgi8FTivqn6sXdZeVXXHyDdW\nWgPMWWl6mK9rmz2+U6iq7gKeALwEuBH4YJLj29npafqcJF+i+X3sw4HD5lncke3r57W/kf4i4GHA\n7cD3kpye5H8Ad6/GtkhdYM5K08N8XdvWjzsALU9V3QN8Gvh0kq8Cx8/NAkjycOCVNOOMbk/yLpox\nSvP5l6p6Xv+LSY4AjqI5xXNi+1zSMpiz0vQwX9cue3ynUJJDkxzS89LjgWuBO4G92tf2Au4C7khy\nAPCLPe17210APCnJj7bLfnCSQ9pTPftU1SeAVwCPW63tkdY6c1aaHubr2maP73TaA/jrJPsA24Gr\naE7JPA84O8nWqjqqPa1yBbCFZrzSnLf3tTseOCPJru38P6BJ3H9OshvNqZ2Xj2LDpDXKnJWmh/m6\nhnlxmyRJkjrBoQ6SJEnqBAtfSZIkdYKFryRJkjrBwleSJEmdYOErSZKkTrDwlSRJUidY+EqSJKkT\nLHwlSZLUCRa+kiRJ6gQLX0mSJHWCha80BZI8P8k5445jZyU5Pslne6bvTHLwkJb96iTvaJ8fnOSe\nJEP5m5bkYW2sGcbylrjOByX5WJLbknxwldaxOcn7VmnZq3asV8Nin5kk1yY5asC8mSRbVnP9klaH\nCScNQZKfS3JeW7TcnORzSX6qnbdDQbCEZd3vP8Sq+kBVPW01Yh+lqtqzqq5dqM1Si4qqOqWqfnMY\ncbVFzlN6lv2tNtYaxvKX6FeAHwL2q6rnrHRhA/bjyLZnKcd6whUj3F+SRmP9uAOQpl2SvYD/C5wA\n/AOwK/DzwH+tdNErfP+KJXlAVd0z7jj6JVlXVT8Y4iKL8e/vHwH+fTn7O8n6qtq+lKY7H5YkrR32\n+EordyhQVfXBanyvqv6lqr6a5NHAacAT21O/twAk+aUkFyW5Pcm3kpzcs7zPtP/eluSOJEfOcxr5\nZ5N8oe1hvjDJE3vmzSZ5fdvrfEeSc5I8pGf+Pyb5dvveTyc5rGfeu5OcluTjSf4DeEWS7/T2Pid5\nVpKvzLcjkjwkyVntdl0A/Gjf/HuSPKJ9fkySy9oYr0vyiiS7A58ANrT7644kP9yeov9QkvcluR04\nfsBp+99IsjXJ9Ule2bddb+iZvrc3tF3Gw4CPtet8VX+ve5IN7XbdnOSqJP+zZ1mbk/xDkve08V6a\n5Ak9809qt++OJFf09iz3tHkd8IfAc9oYXpzGa9re6Bva5e/Vtp+L79eTfBP4ZN/yHjzffqQp8B+4\nQKwbknw4yXeTfCPJS+c7zss41gt93knyoiTfTHJTzzYf1c7bNclb2uO6Ncmbkzyw5zjOfXZuaI/7\n8T3LXXC9S3BE+xm9Jck7k+w6YF88Ok3e3dru02f0zHtQkje123Rbks/Ot5wkz05yTZLDkuyW5P3t\n/rg1TY7/0E7GLmk+VeXDh48VPIA9gZuAdwObgH375v8a8Nm+154MHN4+/3HgO8Az2+kfAe4BHtDT\n/vi5ZQD7AbcCz6f58vpc4Ja59QKzwFXAI4HdgHOBU/qW9WBgF+DNwEU9894N3AY8sZ3eFbgM2NTT\n5p+Alw/YF2e2jwcBhwPXAZ/pmX8P8Ij2+beBJ7XP9wYe37NvtvQtdzPwfeDYdno34GTgfe30we2y\nP9Cu+zHAd4Gj2vnvAl7fs7yZ3nUA1wBP6ZmeW94D2unPAKcCDwQe1y77F3piu7s99gH+BDi/nfco\n4FvAQ9vph81t/zz77mTgvT3Tv94ex4Pb4/Xhufk98b273d5d51neoP04KNYHAF8CXkNzNvDhwNeB\npw7hWC/0eT8MuBP4WZrP5J+3x/op7fzXA+cB+7ePz88dy/Y4bmu3ax3wi8BdwN5LWO8Ox3ie7bsW\nuAQ4ENgX+Bzwhv7PTxvz1cDvtfvtF4A7gEPb+W8FPgX8cLuPj6T5HM2tfx3w4vZYz+2vE4CzaD7n\nAR4P7Dnuv3U+fKyFhz2+0gpV1Z3Az9H0pr0D+G6Sf+7pobnf6eWq+nRVXdY+/ypNAfHkQe37/BJw\nZTXjfu+pqjOBK4Bj5xYPvKuqrq6q79EMv/iJnnW/u6ruqqptwOuAxyXZs2f5H62q89u2/wW8B3gB\nQJL9gKcCf98fVJJ1wLOA11bV3e32vWeB7fk+cHiSvarq9qq6aJHtP6+qzmrj+t6Adq9r130pTbF7\nXG+IA5a7oCQbaYqyk6rq+1V1MfB3wIt6mn22qs6uqgLeT1McA/yA5svD4Ul2qWbs8DcGraovxucD\nb6qqa6vqLuDVwHOz48VQm9vtnW9YzaDtHRTrTwP7V9UfVdX2qrqm3c7nzrNPdupYL/J5/xXgrKo6\nr/1MvpYdx9Y+j6bQvamqbqL5zL6wZ/62dv4PquoTwH/QfOFYbL2LKeDUqtpaVbcCf8yOn6c5RwIP\nrqo/bffbuTRDn45rj9WLgZdV1bfbfP23qvp+z/tfDrwKeHLPZ+P7wEOAQ6pxUft3RtIKWfhKQ1BV\nV1TVi6tqI01v4wbgLYPaJ/mZJOe2p5Rvo+nhecig9n020PQi9vpm+/qc7/Q8vxvYo13vuiR/muTq\nNEMGrmnb7D+3KUD/BVEfAJ6RZhjCr9L06t0wT1z/jabHq/f9/XH2ejZwDHBte5r4yAXaQtOjuJj+\ndW8Y1HAnbABuaYvP3mUf2DPduz/+E9gtzfjoq4HfoemRvCHJGe2Qg6X4YZrj2rvO9cABPa8t584C\n88ZKc6ZhQ3tq/dYkt9IU2/OdYt+pY73I530DPce2qu4Gbu55+wbuvx96j+vNteO46P/kvs/7SvIM\nlvZ52sD9j8NcPj6Eptf26wus45XAW6vq+p7X3gecA5zZDu94YxKvyZGGwMJXGrKqupKm9+sxcy/N\n0+zvgY8CB1XVPsDfcl8+LnYl+VaaIqXXj7SvL+Z5ND3DR1XV3jSns2GB3tCq2gqcT9PD9wKa/5Tn\ncyOwneZ0/pyHDWhLVX2xqn6Zpoj6KE3PNMy//fNdYT9fu/51z+2Tu4Dde+Y9dAnLmnM9sF+SPfqW\nvZRCnKo6o6p+nuYYFfDGQU3nWe/Bfevczo6F60JxD9qPg2wBrqmqfXsee1XV0+dpu1PHmvk/73Of\nueuBg+YaJnkQOxan8+2H3iJxIQvl2VL0b998670e2JjscOu7uXy8CfgezbCjQZ4KvCbJs+ZeaHuO\nX19Vh9OcbXg6O55hkLRMFr7SCiV5VHtxzYHt9EaaU6Lnt01uAA5KskvP2/YAbq2q7yc5gqYgnStK\nbqQZ+7fDxUI9PgEcmuS4JOuTPAf4MZrTq/eGNeC9e9DcbeKW9gKoP+nfnAHvey9wEk0x/5H5GlRz\nl4WPAJvbC3oOoxnffD9Jdklzb+K92/fdSTMsAJr99ZC0F3ItENd8r72mXffhNGOZ5+6H+xXgmCT7\nJnkoTS9srxsYsL+ragvNGNNT2gutHksz/vb987Xv285DkzylvZjpv2iKoEF3o+jfnjOAl6e5kG0P\nmmN1Zi39rg9L3Y9zLgTuTPK77T5cl+QxaW/L12tnjnVrvs/7nA/TnFF4YnvR2ua+OM+gOa77J9mf\nZijEUu9FvFCeLSbAbyU5sB3i8wc0QyX6XUDTy/y77ed6hqZQPbMdTvJO4C/SXKS5rmc751xGM+b6\nrXMXxaW5aO/H2yEld9IM5xjmXUykzrLwlVbuTuBngAvS3AnhfJqLYubuKvCvNP+5fSfJd9vX/hfw\n+iR30FzNf+8PFlTVf9KMJ/x8ezX5z9DT41lVN9P8x/pKmh6lVwFPr6pbemKqvudz0++lOQ27Fbi0\njXVQ217/RNPj9U/t+NpBTqQpNr5D8x/+O+dZ/pwXANe0Qy5eQjOmlaq6gqbY+Ua7/XN3I5ivx7d/\n2Z+mudDok8CfV9Xc3Q7eB1xMc8HS2TQFTO97T6Eprm5N8op5Yj2OptfxepqC77VV9akBcfS+d9d2\n2TfSXMy3P83wgfn0L+edbdyfAb5BU1y9tK/9QDu5H+eK2afTjAf/Rhvz24G9mN/OHOuFPu+Xtdt1\nJs3+vZPm4sG5cct/BHyRJqcuaZ//0YD19Bu43iW8t2iG+fw/mqEKV8233na87jNoLqy7keYiyBdW\n1b+37V4FfBX4As0QjlO4r7CfW8YlNPv+HUmeRnNG4h+B24Gv0Vywuio/PCJ1TZovpJK0sCRXAy/p\nKfikoWt7t28FHllV31ysvSTtDHt8JS0qybOBeyx6tRqSPCPJ7u3wm/8DXGLRK2k1eJWopAUlmaUZ\nQ/zCRZpuiUzLAAAASElEQVRKy3UszTCc0AwJuN8t1CRpGBzqIEmSpE5wqIMkSZI6wcJXkiRJnWDh\nK0mSpE6w8JUkSVInWPhKkiSpEyx8JUmS1An/HwA0L6gxTXfwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10de78710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axes = plt.subplots(1, d, figsize=(12, 3))\n",
    "\n",
    "for i, ax in enumerate(axes):\n",
    "    pi_i_full_dim = np.zeros(mc2.n)\n",
    "    pi_i_full_dim[mc2.cyclic_classes[i]] = pi_s[i]\n",
    "    draw_histogram(pi_i_full_dim,\n",
    "                   ax=ax, title='$\\pi^{i}$'.format(i=i), xlabel='States')\n",
    "\n",
    "fig.suptitle('Stationary distributions for the diagonal blocks',\n",
    "             y=-0.05, fontsize=12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verify that $\\pi^{i+1} = \\pi^i P_i$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pi^0 P_0 =\n",
      "[ 0.857  0.143]\n",
      "pi^1 P_1 =\n",
      "[ 0.286  0.286  0.143  0.286]\n",
      "pi^2 P_2 =\n",
      "[ 0.143  0.286  0.429  0.143]\n"
     ]
    }
   ],
   "source": [
    "for i in range(d):\n",
    "    print('pi^{i} P_{i} =\\n{dot}'.format(i=i, dot=np.dot(pi_s[i], P_blocks[i])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verify that $\\pi = (\\pi^0 + \\cdots + \\pi^{d-1})/d$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.048  0.095  0.143  0.048  0.286  0.048  0.095  0.095  0.048  0.095]\n"
     ]
    }
   ],
   "source": [
    "# Right hand side of the above identity\n",
    "rhs = np.zeros(mc2.n)\n",
    "\n",
    "for i in range(d):\n",
    "    rhs[mc2.cyclic_classes[i]] = pi_s[i]\n",
    "rhs /= d\n",
    "print(rhs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.allclose(pi, rhs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Powers of $P$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the Markov chain in consideration is periodic,\n",
    "the marginal distribution does not converge, but changes periodically."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us compute the powers of the transition probability matrix (in cyclic normal form):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.set_printoptions(suppress=True)  # Suppress printing with floating point notation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print $P^1, P^2, \\ldots, P^d$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P^1 =\n",
      "[[ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     1.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.     0.     1.     0.   ]\n",
      " [ 0.5    0.5    0.     0.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     1.     0.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     1.     0.     0.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.5    0.5    0.     0.     0.     0.     0.     0.   ]]\n",
      "P^2 =\n",
      "[[ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.     0.     1.     0.   ]\n",
      " [ 0.167  0.167  0.5    0.167  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     1.     0.     0.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.5    0.5    0.     0.     0.     0.   ]]\n",
      "P^3 =\n",
      "[[ 0.167  0.167  0.5    0.167  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.167  0.167  0.5    0.167  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.167  0.167  0.5    0.167  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     1.     0.     0.     0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.833  0.167  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     1.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.333  0.333  0.     0.333]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.167  0.167  0.5    0.167]]\n"
     ]
    }
   ],
   "source": [
    "for i in range(1, d+1):\n",
    "    print('P^{i} =\\n{P_i}'.format(i=i, P_i=np.linalg.matrix_power(mc2.P, i)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print $P^{2d}$, $P^{4d}$, and $P^{6d}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P^6 =\n",
      "[[ 0.139  0.306  0.417  0.139  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.139  0.306  0.417  0.139  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.139  0.306  0.417  0.139  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.167  0.167  0.5    0.167  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.861  0.139  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.833  0.167  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.278  0.278  0.167  0.278]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.278  0.278  0.167  0.278]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.278  0.278  0.167  0.278]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.306  0.306  0.083  0.306]]\n",
      "P^12 =\n",
      "[[ 0.143  0.286  0.428  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.428  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.428  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.144  0.282  0.431  0.144  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.856  0.144  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.285  0.285  0.144  0.285]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.285  0.285  0.144  0.285]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.285  0.285  0.144  0.285]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.141  0.286]]\n",
      "P^18 =\n",
      "[[ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]]\n"
     ]
    }
   ],
   "source": [
    "for i in [k*d for k in [2, 4, 6]]:\n",
    "    print('P^{i} =\\n{P_i}'.format(i=i, P_i=np.linalg.matrix_power(mc2.P, i)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$P^{kd}$ converges as $k \\to \\infty$ to a matrix that contains $\\pi^0, \\ldots, \\pi^{d-1}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print $P^{kd+1}, \\ldots, P^{kd+d}$ with $k = 10$ for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "P^31 =\n",
      "[[ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]]\n",
      "P^32 =\n",
      "[[ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]]\n",
      "P^33 =\n",
      "[[ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.143  0.286  0.429  0.143  0.     0.     0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.857  0.143  0.     0.     0.     0.   ]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]\n",
      " [ 0.     0.     0.     0.     0.     0.     0.286  0.286  0.143  0.286]]\n"
     ]
    }
   ],
   "source": [
    "for i in range(10*d+1, 10*d+1+d):\n",
    "    print('P^{i} =\\n{P_i}'.format(i=i, P_i=np.linalg.matrix_power(mc2.P, i)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But $P^i$ itself does not converge."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simulation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the frequency distribution of visits to the states\n",
    "along a sample path starting at state `0`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.047  0.092  0.148  0.046  0.287  0.046  0.093  0.098  0.046  0.096]\n"
     ]
    }
   ],
   "source": [
    "init = 0\n",
    "dist = time_series_dist(mc2, init=init, t=10**4)\n",
    "print(dist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEZCAYAAACD/A7qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGxpJREFUeJzt3X20XXV95/H3xwRGCQ+iUhmejA+hokss6sQ4iB6VYuqo\nsdO6EKQOhVHaaaxF2zLYVq7VWYodxtrqILYZH9FoKcNEBwVpPYBIMWkRVBKHiAFCICJgEMSamM/8\nsfclOzfn6eY+nHN/9/NaKyvn7P377f3b+9z7Ob/zPfucK9tEREQ5HjPsAURExPRKsEdEFCbBHhFR\nmAR7RERhEuwREYVJsEdEFCbBPqIkfUfSS4Y9jsmQdLmk35rhfYxJ+nR9+yhJP5Gkadr2hZL+tL7d\nknTndGy33t4JkjZM1/ame/+SFkvaKWmgTGieq+lsG9NDuY59OCQ9BIyf/EXAz4Bf1PffYvtzQxnY\niJN0HvAM2wM/gUg6HTjT9gmT6NMCPm37yEkPsuq/k2qct+1N/5kmaRNwhu1/rO8vBm4DFtreOYP7\nbTG183o28MfAfsAlwO/a/vn0jbAMmbEPie39bR9g+wDgduDV4/fnWqirNuxxTMWgM9XJbnYGtjld\nzGiPbw+SXgmcA7wceArwNODdQx3UiEqwjyhJmyS9vL49JunvJH1a0oOSbpa0RNK5krZKukPSrzb6\nHiRplaQtkjZLek+34JK0VNI6Sdsk3SPpgsa6ZZK+IekBSd+S9NLGurak90q6DngIeFq97MxGmzMk\n3SLpfklfkXRUY90H67Fvq4/n2V3G91RJV9fHfSXwpMa63coHkk6X9P267W2STpX0TOCjwIvqss39\nddtP1CWCy+tXTy+rl71nwv7PlXSvpB9IOnXC8TeP9XRJ19a3r6kX31Tv8/UTSzuSjqm38UBddntN\nY90nJH1E0pfqY/knSU/rcn4+Kent9e3D6/PxX+r7T5d0X3370f2rKmUdBXyxHt8fNjZ5mqTb62N+\nZ6d9Nsb4nsa2N0t6e/2YblH1Kmm3tpL2A74MHFbv90FJh3bbRwf/Cfhb2+tt/xj4c+D03l3mpwT7\n6JpYI3s18CngYOBG4Ip6+WFUP+AXNdp+Avg58HTgOOAk4D932c+HgA/aPohqBvQFqEIC+BLw57YP\nBv4Q+HtJT2z0Pa3e7virDo+PW9IK4Fzg16nC+Frgc/W6VwInAEvq/b4euK/L+D4LrAWeCLyH6pd7\nj/qhpEX1sSy3fSDwIuBbtjcAZwHX16+GntDodgrwHtv7A19vjr92aL3fw+r9fkzSknrdxLaPsj3+\n3six9T7/bsJY9wG+CHwFOAR4K3CxpKMbzU4Gxqge743Af+t4dqANtOrbL6Uqp7ykcf+aiR3qMtYd\n7HqV+N8bq48HjgZeAbyrfmLseJjsfvxPBg6kOldnAh+RdFCzre2fAsuBLfV+D7R9T/0E/ECPf0fU\n23kWcFNjnzcDT5Z0cJcxzlsJ9rnjGttftf0LqtriIcD76/ufBxZLOlDSk4FfA862/Yjte4G/BN7Q\nZbs/B5ZIepLtn9q+oV5+GnC57a8A2L4KWAf8h3q9gU/Us6edtndM2O7vAO+z/b26Zvs+4FfqWfvP\nqZ4MjpH0mLrNPRMHVrd9AfBntrfbvpYqELuVEHYCz5H0ONtbbd8yvqkObQ1cZvv6+vj+tUvb8X1f\nA/xfqsCdqmXAItvvt73D9teonkRPabS51Pa6+vG9GPiVLtu6BnixJFE9WX6AKpyhCvarJzm2d9v+\nV9s3U4Xoc3u0bZ6r7VSTgF/Y/jLVq7hf7tB2j8fC9mdtH9zj3+a66f7AtkbXB+v/D5jE8c0LCfa5\n44eN248AP/Kud74fqf/fn6r2uA9w9/iMh6oUcUiX7Z5JNUNbL+mbksaD+ynA65szJ6rAaL507nXV\nyFOADzX6js/ID6uD7MPAR4Ctki6S1OmX8zDgAduPNJbd3mlnth+mCt3fAbbUZYxf7tR2wPHTZd//\ntk+fQRzWYd+318uhetLZ2lj3CNVjuwfb3wcepgr+E6ieILbUs/+XMPlgbz7B/pTqjf1B3DfhTdef\ndhvzFDxE9apg3Pgrgp9M837mvAR7ee4E/hV4YmPGc5Dt53RqbHuj7VNtHwKcD1xS10LvoLp6oTlz\nOsD2B5rde4zjDqqre5r9F9n+p3q/f237BVQvr48G/qjDNu4GDq7HM+4p3fZr+0rbJ1E9+WwA/maA\nce6xmcbtTvveUt9+mN1DbzK14i3AkfUsu7ntuyaxjaarqcpZ+9jeUt8/naqM860ufabjcri9Oa+d\nymhvrGvunf492CjFfJfdX7k8F9hq+4G9G365EuyFsX03cCXwPyQdIOkx9ZtoHa+Jl3SapPHZ/Daq\nX7xfAJ8BXiPpJEkLJD22fpPs8Gb3HkP5KPBOSc+q93OQpNfXt18g6YV1rfmn7H6pZ/NYbqcq/7xb\n0j6SXkz1XkOn4/glSSvqWvt2quAd3+ZW4Ih6f73Grg7Lx/d9AlUZarxe/i3gP0p6nKRnUL3yadpK\n9R5HJzdQHfcf19tu1ce1usfYerkaWMmuenq7vn9t41XdRL3G19RtLJ3OVa9tjLfdCjxR0qMzb9sX\nN64Im/jvwEYp5lPAmareeD4Y+DPg4wOOYV5JsM8Nnd6o63X/TcC+wC3A/VRh1G1G+UrgO5J+AnwQ\neENdY90MrADeSVUGugN4B7v/Mnedsdm+jOoVwGpJ24Bv1/uC6uX0x+qxbQJ+BPxFl02dCrywbvsu\n4JNdjvsxwNlUs977qMoSv1uv+weq2d49kn7Y6NfpHDaX3Q08QDXD/jRwlu3/V6/7INV7BVupwuUz\nE/qOAZ+sS1G/2dx2fd31a6jeC7mXqiz1W41tD/J4N11DVfYYD/brgMex5xunzW28D/jTenxv77GP\nbvudOMZe42se+waqN9FvU3W11MCvdGxfQfUewteofm6+D5w3aP/5pO8HlCQtp3rzbQHVpUbnT1i/\nguqqjJ3ADuAPbF9Xr9tE9QbHL4DttpdO9wFERMTuega7pAXA94ATqWZCa4FTbK9vtFlUv3GFpOcA\nX7B9TH3/B8Dzbd8/c4cQERFN/UoxS4GNtjfZ3k5VA1zRbDAe6rX9qWbuTXPq020REXNdv2A/nN0v\ny9pcL9uNpNdJWk91qdUZjVUGrlL1ycY3T3WwERHRX79gH+hyJtuX1eWX1wHvbaw63vZxVG8S/V59\nZUFERMyghX3W3wU0v4XtSKpZe0e2r5X0NElPsH1/fekdtu+V9L+pSjvXNvtIytdLRkTsBdsdS939\nZuzrqD5uvljSvlSf7FvTbFBfI6369vOAfW3fL2m/8U8T1tcWn0R1yVunwU3p33nnnTflbZTyL+ci\n5yLnYn6ci156ztht75C0kuoLpxYAq2yvl3RWvf4i4DeAN0naTvXR5/Hv0jgUuLTO/IXAxbav7Dma\niIiYsn6lGFx9oc+XJyy7qHH7A1QfGpjY7za6f3FRRETMkCI+edpqtYY9hJGRc7FLzsUuORe7zIdz\nMfQ/jSfJwx5DRMRcIwnv5ZunERExxyTYIyIKk2CPiChMgj0iojAJ9oiIwiTYIyIKk2CPiChMgj0i\nojAJ9oiIwiTYIyIKk2CPiChMgj0iojAJ9oiIwiTYIyIKk2CPiChMgj0iojAJ9oiIwiTYIyIKk2CP\niChMgj0iojAJ9oiIwvQNdknLJW2QdKukczqsXyHpJkk3Slor6fhB+0ZExPST7e4rpQXA94ATgbuA\ntcApttc32iyy/XB9+znAF2wfM0jfuo97jSEiIvYkCdvqtK7fjH0psNH2JtvbgdXAimaD8VCv7Q/s\nHLRvRERMv37BfjhwZ+P+5nrZbiS9TtJ64EvAGZPpGxER02thn/UD1UhsXwZcJukE4L3Ar05mEGNj\nY4/ebrVatFqtyXSPiCheu92m3W4P1LZfjX0ZMGZ7eX3/XGCn7fN79Pk+8O+Aowfpmxp7RMTkTaXG\nvg5YImmxpH2Bk4E1Ezb+dEmqbz8P2Nf2/YP0jYiI6dezFGN7h6SVwBXAAmCV7fWSzqrXXwT8BvAm\nSduBR6gCvGvfmTuUiIiAPqWYWRlASjEREZM2lVJMRETMMQn2iIjCJNgjIgqTYI+IKEy/DyhFDF19\nNe2syZv5Mdcl2GOOmK2wnd0nkYiZkFJMRERhEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdE\nFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERh+ga7\npOWSNki6VdI5Hda/UdJNkm6WdJ2kYxvrNtXLb5T0zekefERE7Knnn8aTtAD4MHAicBewVtIa2+sb\nzW4DXmJ7m6TlwMeAZfU6Ay3b90//0CMiopN+M/alwEbbm2xvB1YDK5oNbF9ve1t99wbgiAnbyB+R\njIiYRf2C/XDgzsb9zfWybs4ELm/cN3CVpHWS3rx3Q4yIiMnoWYphEn8aXtLLgDOA4xuLj7d9t6RD\ngK9K2mD72ol9x8bGHr3darVotVqD7jYiYl5ot9u02+2B2srunt2SlgFjtpfX988Fdto+f0K7Y4FL\ngeW2N3bZ1nnAQ7YvmLDcvcYQIYlJzDGmujfy8xhzgSRsdyx19yvFrAOWSFosaV/gZGDNhI0fRRXq\npzVDXdJ+kg6oby8CTgK+vfeHERERg+hZirG9Q9JK4ApgAbDK9npJZ9XrLwLeBRwMXFjNrNhueylw\nKHBpvWwhcLHtK2fsSCIiAuhTipmVAaQUE32kFBOxp6mUYiIiYo5JsEdEFCbBHhFRmAR7RERhEuwR\nEYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7\nRERhEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmL7BLmm5pA2SbpV0Tof1b5R0\nk6SbJV0n6dhB+0ZExPST7e4rpQXA94ATgbuAtcApttc32rwIuMX2NknLgTHbywbpW/d3rzFESAJm\n62dE5Ocx5gJJ2Fandf1m7EuBjbY32d4OrAZWNBvYvt72tvruDcARg/aNiIjp1y/YDwfubNzfXC/r\n5kzg8r3sGxER02Bhn/UDvyaV9DLgDOD4yfYdGxt79Har1aLVag3aNSJiXmi327Tb7YHa9quxL6Oq\nmS+v758L7LR9/oR2xwKXAsttb5xk39TYo6fU2CP2NJUa+zpgiaTFkvYFTgbWTNj4UVShftp4qA/a\nNyIipl/PUoztHZJWAlcAC4BVttdLOqtefxHwLuBg4MJqZsV220u79Z3BY4mICPqUYmZlACnFRB8p\nxUTsaSqlmIiImGMS7BERhUmwR0QUJsEeEVGYBHtERGES7BERhUmwR0QUJsEeEVGYBHtERGES7BER\nhUmwR0QUJsEeEVGYBHtERGES7BERhUmwR0QUJsEeEVGYBHtERGES7BERhUmwR0QUJsEeEVGYBHtE\nRGES7BERhUmwR0QUpm+wS1ouaYOkWyWd02H9MyVdL+lnkt4xYd0mSTdLulHSN6dz4BER0dnCXisl\nLQA+DJwI3AWslbTG9vpGs/uAtwKv67AJAy3b90/TeCMioo9+M/alwEbbm2xvB1YDK5oNbN9rex2w\nvcs2NPVhRkTEoPoF++HAnY37m+tlgzJwlaR1kt482cFFRMTk9SzFUAXzVBxv+25JhwBflbTB9rUT\nG42NjT16u9Vq0Wq1prjbiIiytNtt2u32QG1ld89uScuAMdvL6/vnAjttn9+h7XnAQ7Yv6LKtjusl\nudcY5jNp9qpYo/wYVOdhtsankT4XEeMkYbtjSPQrxawDlkhaLGlf4GRgTbf9TNjpfpIOqG8vAk4C\nvj2pkQdVoM30v4goSc9SjO0dklYCVwALgFW210s6q15/kaRDgbXAgcBOSW8DngX8EnBpPetcCFxs\n+8qZO5SIiIA+pZhZGUBKMV3NXglitMsPKcVE7GkqpZiIiJhjEuwREYVJsEdEFCbBHhFRmAR7RERh\nEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFR\nmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFKZvsEtaLmmDpFslndNh/TMlXS/p\nZ5LeMZm+EREx/WS7+0ppAfA94ETgLmAtcIrt9Y02hwBPAV4HPGD7gkH71u3cawzzmSRgNs6NGOXH\nYPbOA4z6uYgYJwnb6rSu34x9KbDR9ibb24HVwIpmA9v32l4HbJ9s34iImH79gv1w4M7G/c31skFM\npW9EROylhX3WT+U16cB9x8bGHr3darVotVpT2G1ERHna7Tbtdnugtv1q7MuAMdvL6/vnAjttn9+h\n7XnAQ40a+0B9U2PvLjX2Smrso6d6TGZPHpM99aqx95uxrwOWSFoMbAFOBk7ptp8p9I2IOWf2nmxH\n2Sg+yfUMdts7JK0ErgAWAKtsr5d0Vr3+IkmHUl3xciCwU9LbgGfZfqhT3ykdUUTESBqtJ7mepZjZ\nkFJMdynFVFKKGT15THYZ1rmYSilmXhrFl1YRkJ/NGEyCvavRemkVsUt+NqO3fFdMRERhEuwREYVJ\nsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERh\nEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERh+ga7pOWSNki6VdI5Xdr8\nVb3+JknHNZZvknSzpBslfXM6Bx4REZ0t7LVS0gLgw8CJwF3AWklrbK9vtHkV8AzbSyS9ELgQWFav\nNtCyff+MjD4iIvbQb8a+FNhoe5Pt7cBqYMWENq8FPglg+wbg8ZKe3Fiv6RpsRET01y/YDwfubNzf\nXC8btI2BqyStk/TmqQw0IiIG07MUQxXMg+g2K3+x7S2SDgG+KmmD7WsnNhobG3v0dqvVotVqDbjb\niIj5og3snpfdyO6e3ZKWAWO2l9f3zwV22j6/0eajQNv26vr+BuCltrdO2NZ5wEO2L5iw3L3GMAyS\nGPw5bcp7o9vxz944uo9hFIzK4zEKRuVcjMo4RsGwzoUkbHecVPebsa8DlkhaDGwBTgZOmdBmDbAS\nWF0/EfzY9lZJ+wELbP9E0iLgJODdfYet2SvJj/IPy6iYzccDRvsxybkYLXk8uusZ7LZ3SFoJXAEs\nAFbZXi/prHr9RbYvl/QqSRuBh4HfrrsfClxan/yFwMW2rxxsWLMzS41Bzd5sZPTlXIyWPB6d9CzF\nzMoAJpRiRqH8MCovM3MuRmcMozKOURjDqIxjFMYwzHH0KsXkk6cREYVJsEdEFCbBHhFRmAR7RERh\nEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFCbBHhFR\nmAR7RERhEuwREYVJsEdEFCbBHhFRmAR7RERhEuwREYVJsEdEFKZvsEtaLmmDpFslndOlzV/V62+S\ndNxk+kZExPTqGeySFgAfBpYDzwJOkXTMhDavAp5hewnwFuDCQftOn/bMbHZOag97ACOkPewBjJD2\nsAcwQtrDHsCM6zdjXwpstL3J9nZgNbBiQpvXAp8EsH0D8HhJhw7Yd5q0Z2azc1J72AMYIe1hD2CE\ntIc9gBHSHvYAZly/YD8cuLNxf3O9bJA2hw3QNyIiplm/YPeA29FUBxIREdNjYZ/1dwFHNu4fSTXz\n7tXmiLrNPgP0BUCa+LywN88T7550jz33O9Ux7J3pH0eJ52Jvx5BzsUvOxS4lnotd+gX7OmCJpMXA\nFuBk4JQJbdYAK4HVkpYBP7a9VdJ9A/TFdmb7ERHTqGew294haSVwBbAAWGV7vaSz6vUX2b5c0qsk\nbQQeBn67V9+ZPJiIiADZg5bRIyJiLpjTnzzNB6B2kXSkpK9J+q6k70j6/WGPaZgkLZB0o6QvDnss\nwyTp8ZIukbRe0i11uXReknR2/bvxbUmflfRvhj2mmTJng312PwA1J2wHzrb9bGAZ8Hvz/Hy8DbiF\nwa/sKtWHgMttHwMcC8zLcqikw4G3As+3/Ryq8vAbhjuqmTNng51Z/QDU6LN9j+1v1bcfovoFPmy4\noxoOSUcArwL+lnl8Ka6kg4ATbP8vqN73sr1tyMMapoXAfpIWAvtRXdFXpLkc7IN8eGpeqq9EOg64\nYbgjGZoPAn8E7Bz2QIbsqcC9kj4u6V8k/Y2k/YY9qGGwfRdwAXAH1VV6P7Z91XBHNXPmcrDP95fY\nHUnaH7gEeFs9c59XJL0a+KHtG5nHs/XaQuB5wP+0/Tyqq9b+63CHNBySDqb6+pPFVK9k95f0xqEO\nagbN5WAf5MNT84qkfYC/Bz5j+7Jhj2dI/j3wWkk/AD4HvFzSp4Y8pmHZDGy2vba+fwlV0M9HJwI/\nsH2f7R3ApVQ/K0Way8H+6IenJO1L9QGoNUMe09Co+jjaKuAW23857PEMi+132j7S9lOp3hz7R9tv\nGva4hsH2PcCdko6uF50IfHeIQxqm24Flkh5X/66cSPXmepH6ffJ0ZOUDUHs4HjgNuFnSjfWyc21/\nZYhjGgXzvWT3VuDievLzfeoPEM43tr8p6RLgX4Ad9f8fG+6oZk4+oBQRUZi5XIqJiIgOEuwREYVJ\nsEdEFCbBHhFRmAR7RERhEuwREYVJsEfxJP1J/XWtN9Vf5btU0tskPW6Avn8wSLuIUZLr2KNokl5E\n9eVPL7W9XdITgMcC1wEvsH1fn/4/GKRdxCjJjD1Kdyjwo/qrnbF9P/CbVF8E9TVJ/wAg6UJJa+uZ\n/Vi97Pc7tDtJ0jck/bOkL0haVC9/f/1HTm6S9BezfpQRDZmxR9Hq4P061fdvXwV83vY19Uz8+XXQ\nI+lg2w/Uf8DlKuCttr/TbCfpSVRfsrbc9iP1X+3aF/gI8A3bz6y3daDtB2f9YCNqmbFH0Ww/DDwf\neAtwL/B5SafXq5tf63uypH+m+g6RZ1P9Va6JltXLv1F/H8+bgKOAbcDPJK2S9OvAIzNxLBGDmrNf\nAhYxKNs7gauBqyV9Gzh9fBWApKcC76CqpW+T9HGqOnwnX7V96sSFkpYCr6Aq86ysb0cMRWbsUTRJ\nR0ta0lh0HLAJ+AlwYL3sQKo/QvGgpCcDv9Zo32x3A3C8pKfX214kaUld7nm87S8DbweeO1PHEzGI\nzNijdPsDfy3p8VRf13orVVnmVOArku6y/Yq6tLKB6s8tfr3R/2MT2p0OfK7xF+7/hCr8/4+kx1KV\nd86ejQOL6CZvnkZEFCalmIiIwiTYIyIKk2CPiChMgj0iojAJ9oiIwiTYIyIKk2CPiChMgj0iojD/\nH6OFzIj5tgGOAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10e223748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "draw_histogram(dist,\n",
    "               title='Time series distribution with init={0}'.format(init),\n",
    "               xlabel='States', ylim=(0, 0.35))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Observe that the distribution is close to the (unique) stationary distribution $\\pi$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.048  0.095  0.143  0.048  0.286  0.048  0.095  0.095  0.048  0.095]\n"
     ]
    }
   ],
   "source": [
    "print(pi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, plot the simulated marginal distributions\n",
    "at $T = 10d+1, \\ldots, 11d, 11d+1, \\ldots, 12d$ with initial state `0`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr4AAAGZCAYAAABvxjJmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X20ZHdd5/v353YnQCRPEM2dPEAAAxIkCmrIgMBBsi5t\nYJERRWhABJkx62KQq3PHEHRMK46YWUt5mCAGiMjw1EpADMokyshxEBHIJRAhnUwChHQnPJgASXhQ\n0vT3/rF3J9WVU6eqz6lTtevs92utvbqq9q69v7X7fM751t6/2pWqQpIkSdrs/o95FyBJkiTNgo2v\nJEmSesHGV5IkSb1g4ytJkqResPGVJElSL9j4SpIkqRdsfCVJktQLNr4LIMk3ktzRTvuSfGvg/vYp\nbue/JrkxyW1Jbkhy3tD8NyS5Jsl3k/z8tLYrbSZdyGuShyb5iyRfSXJrksuSPHRa25Y2i47k9f5J\nPpzkliRfT/IPSR47rW3rQDa+C6Cq7ltVh1fV4cAXgKftv19V75zipi4GTqmqI4HHAs9N8lMD8z8J\nvBj4BOA3n0gr6EhejwTeCzwUOBb4GPAXU9y2tCl0JK/fAH4B+L6qOgq4AHhfEnu0DbB13gWoO6rq\n2oG7AfYBDxmY/4cASf5lxqVJGrJaXqvq48DH75qZvBr4jSRHV9XXZlqopHF5/VfgWoC22d0HHA3c\nD7hltpVufr6b2ISSvCzJ10ZMX53guXcAu4HDgHfMpGipp2aU1ycAX7TpldZnI/Oa5Crg2zRnZ95Y\nVTa9G8DGdxOqqt+rqqNHTPeb4LmHA48G3gbcPpOipZ7a6LwmOQG4EPjVDXkBUo9sZF6r6lTgcOA5\nwIc36jX0nY2vVlRVn6R55/lb865F0upG5TXJ9wJ/Dbyuqv50HrVJOtBqf1+r6jtVtRN4WZJTZ15c\nD9j4bkJJXj7wqdTh6WCO4B7CwBhfSdO3UXlNcjRN0/veqnrltOuW+miGf18PAR60vmq1EhvfTaiq\nfnfgU6nD0xErPSeNs5Mc1d4+jeYKDh8YWOaQJPem+bk5NMm9k2Q2r0ranDYir0mOAC4H/r6qXj67\nVyNtbhuU18ck+fEkhya5T5Jzge8DPjq7V9YfNr4a9O+Az9KMO3or8NqqunBg/t8A3wJOB97Q3n78\nrIuUBKye158CfhR44eDRqHa8r6TZWy2v96IZh38LsAfYBjy1qr40j0I3u1StfjnWJH8MPBX4SlU9\ncsQyrwV+kqYRekFVXTntQiWNZ16lxWFepdmb5Ijvm2nefawoyZnA91fVycAvAq+fUm2SDp55lRaH\neZVmbGzjW1UfAla79uPTgbe0y34UOCrJsdMpT9LBMK/S4jCv0uxNY4zv8TQXY95vD+A4MqmbzKu0\nOMyrNGXT+nDb8Cf7Vx84LGmezKu0OMyrNEVbp7COm4ATB+6f0D52gCSGVRpSVbO+HJx5ldaoq3kF\nMyutZKXMTuOI76XA8wGSnA58vaq+PKKAsdP5558/0XLzmLpcW9frs7Z7TnPSm7x2vT5rW6z65mTi\nvML4zPr/ujlr63p9XfsbO/aIb5J3Ak8EjkmyGzif5htFqKqLqur9Sc5Mcj3wTeCFBxFqSVNkXqXF\nYV6l2Rvb+FbV9gmWOWc65UhaD/MqLQ7zKs1e5765bWlpad4ljNTl2qDb9Vnb5tT1fdfl+qxt7bpe\nX1d1fb91ub4u1wbdrq9rtY395rapbSipWW1LWgRJqNl/WGYi5lU6UJfzCmZWGjYqs5074itJkiRt\nBBtfSZIk9YKNryRJknrBxleSJEm9YOMrSZKkXrDxlSRJUi/Y+EqSJKkXbHwlSZLUCza+kiRJ6oWt\n8y5Ak0vW96VBfquPJEnqMxvfhbPW5rWz37QpSZI0Ew51kCRJUi/Y+EqSJKkXbHwlSZLUCza+kiRJ\n6gUbX0mSJPWCja8kSZJ6wcZXkiRJvWDjK0mSpF6w8ZUkSVIvjG18k2xLck2S65Kcu8L8I5O8L8kn\nk3w6yQs2pFJJEzGz0uIwr9JspWr0V+Am2QJcC5wB3AR8HNheVbsGlnk5cHhVnZfkmHb5Y6tq79C6\narVtabwkrOcri93/3ZKEqprqd0lPK7PmVTpQl/PaLmdmpQGjMjvuiO9pwPVVdUNV3QnsBM4aWmYf\ncER7+wjg1uFASpoZMystDvMqzdi4xvd4YPfA/T3tY4MuBE5JcjPwKeCl0ytP0kEys9LiMK/SjG0d\nM3+S8ybbgE9U1ZOSPAT4myQ/VFV3DC+4Y8eOu24vLS2xtLR0EKVKi215eZnl5eWN3szUMmte1WeL\nllcws+q3STM7bozv6cCOqtrW3j8P2FdVFwws85fAK6vqw+39/wmcW1VXDK3L8Ufr5BjfzWWDxgxO\nJbPmVTpQl/PaPm5mpQFrHeN7BXBykpOSHAo8C7h0aJkbaQbmk+RY4GHA59ZfsqQ1MLPS4jCv0oyt\nOtShqvYmOQe4HNgCXFxVu5Kc3c6/CHgF8CdJrgIC/FpVfXWD65a0AjMrLQ7zKs3eqkMdprohT8Os\nm0MdNpeNOHU6LeZVOlCX8wpmVhq21qEOkiRJ0qZg4ytJkqResPGVJElSL9j4SpIkqRdsfCVJktQL\nNr6SJEnqBRtfSZIk9YKNryRJknrBxleSJEm9YOMrSZKkXrDxlSRJUi/Y+EqSJKkXbHwlSZLUCza+\nkiRJ6gUbX0mSJPWCja8kSZJ6wcZXkiRJvWDjK0mSpF6w8ZUkSVIv2PhKkiSpF2x8JUmS1AtjG98k\n25Jck+S6JOeOWGYpyZVJPp1keepVSpqYmZUWh3mVZitVNXpmsgW4FjgDuAn4OLC9qnYNLHMU8GHg\nKVW1J8kxVXXLCuuq1bal8ZIAa92Hwf3fLUmoqkx5nVPJrHmVDtTlvLbLmVlpwKjMjjviexpwfVXd\nUFV3AjuBs4aWeQ7w7qraA7BSICXNjJmVFod5lWZsXON7PLB74P6e9rFBJwP3S/LBJFck+blpFijp\noJhZaXGYV2nGto6ZP8l5k0OARwNPBg4DPpLkH6vquuEFd+zYcdftpaUllpaWJi5UWnTLy8ssLy9v\n9Gamllnzqj5btLyCmVW/TZrZcWN8Twd2VNW29v55wL6qumBgmXOB+1TVjvb+m4DLquqSoXU5/mid\nHOO7uWzQmMGpZNa8Sgfqcl7bx82sNGCtY3yvAE5OclKSQ4FnAZcOLfMXwI8n2ZLkMOAxwNXTKFrS\nQTOz0uIwr9KMrTrUoar2JjkHuBzYAlxcVbuSnN3Ov6iqrklyGXAVsA94Y1UZSmkOzKy0OMyrNHur\nDnWY6oY8DbNuDnXYXDbi1Om0mFfpQF3OK5hZadhahzpIkiRJm4KNryRJknrBxleSJEm9YOMrSZKk\nXrDxlSRJUi/Y+EqSJKkXbHwlSZLUCza+kiRJ6gUbX0mSJPWCja8kSZJ6wcZXkiRJvWDjK0mSpF6w\n8ZUkSVIv2PhKkiSpF7bOuwBJkrT5JVn3OqpqCpWoz2x8JUnSjKyncV1/4yw51EGSJEm9YOMrSZKk\nXrDxlSRJUi/Y+EqSJKkXbHwlSZLUCza+kiRJ6oWxjW+SbUmuSXJdknNXWe7HkuxN8ozplijpYJhZ\naXGYV2m2Vm18k2wBLgS2AacA25M8fMRyFwCX4YX2pLkxs9LiMK/S7I074nsacH1V3VBVdwI7gbNW\nWO4lwCXAP0+5PkkHx8xKi8O8SjM2rvE9Htg9cH9P+9hdkhxPE9TXtw/5fYLS/JhZaXGYV2nGxn1l\n8SQBezXwsqqqNF/EPfI0zI4dO+66vbS0xNLS0gSrlzaH5eVllpeXN3ozU8useVWfLVpewcyq3ybN\nbKpG5y7J6cCOqtrW3j8P2FdVFwws8znuDuIxwLeA/1BVlw6tq1bblsZrfuetdR8G93+3JKGqpjpe\nb1qZNa/Sgbqc13a5zmd2fX/DwL9jOhijMjuu8d0KXAs8GbgZ+Biwvap2jVj+zcD7quo9K8zrfCi7\nzsZ3c9mgP6RTyax5lQ7U5by28zqfWRtfzdKozK461KGq9iY5B7gc2AJcXFW7kpzdzr9oQ6qVtCZm\nVjpQ02ytz0Y1W+ZVmr1Vj/hOdUML8G606zziu7lsxBGkaTGv2iymdZSxy3mFxcisR3w1S6My6ze3\nSZIkqRdsfCVJktQLNr6SJEnqBRtfSZIk9YKNryRJknrBxleSJEm9YOMrSZKkXrDxlSRJUi/Y+EqS\nJKkXbHwlSZLUCza+kiRJ6gUbX0mSJPWCja8kSZJ6Yeu8C5AkLbYk615HVU2hEklanY2vJGkK1tO4\nrr9xlqRJONRBkiRJvWDjK0mSpF6w8ZUkSVIvOMZXkiTdgx9a1GZk4ytJPWNDo8n5oUVtLja+ktRL\nNjSS+meiMb5JtiW5Jsl1Sc5dYf5zk3wqyVVJPpzk1OmXKmkS5lVaHOZVmq2xjW+SLcCFwDbgFGB7\nkocPLfY54AlVdSrwCuAN0y5U0njmVVoc5lWavUmO+J4GXF9VN1TVncBO4KzBBarqI1V1W3v3o8AJ\n0y1T0oTMq7Q4zKs0Y5M0vscDuwfu72kfG+VFwPvXU5SkNTOv0uIwr9KMTfLhtok/AZHkScAvAI9b\naf6OHTvuur20tMTS0tKkq5YW3vLyMsvLyxu9GfMqTcVyOx2YhSmbWl7BzKrfJv0bm3GXpElyOrCj\nqra1988D9lXVBUPLnQq8B9hWVdevsJ7y8jfr01yCaK37MF5+qGOSUFVT/Xi8edUk1ve7BIZ/n0x7\nfdM0rdq6nNd2malntk8/J9p8RmV2kqEOVwAnJzkpyaHAs4BLh1b+AJpQPm9UKCXNhHmVFod5lWZs\n7FCHqtqb5BzgcmALcHFV7Upydjv/IuA3gaOB17cXRr+zqk7buLIlrcS8SovDvK7Per+IxaPH/TR2\nqMPUNuSp03VzqMPmshGnTqfFvG5ufTqF3eWhDtPUx6EO/k3UatYz1EGSJElaeDa+kiRJ6gUbX0mS\nJPWCja8kSZJ6wcZXkiRJvWDjK0mSpF6w8ZUkSVIv2PhKkiSpF2x8JUmS1As2vpIkSeoFG19JkiT1\nwtZ5F6D5aL7jfH38nnNJG8HfT5I2io1vr63nD8P6/zBJ0mj+flqL9b5p8A2DNjsbX0mSNpW1Nq/9\nfcOg/nCMryRJknrBxleSJEm9YOMrSZKkXrDxlSRJUi/Y+EqSJKkXbHwlSZLUCza+kiRJ6oWxjW+S\nbUmuSXJdknNHLPPadv6nkjxq+mVKmpSZlRaHeZVma9XGN8kW4EJgG3AKsD3Jw4eWORP4/qo6GfhF\n4PXrKWh5eXk9T99QXa6tsTzvAkbq8r7rcm0Ha9aZ7fq+63J9Xa6ty79LGsvzLmAqZv83dnntT52J\n5XkXMFK389rt+rpW27gjvqcB11fVDVV1J7ATOGtomacDbwGoqo8CRyU5dq0FdW0HDepybY3leRcw\nUpf3XZdrW4OZZrbr+67L9XW5ti7/Lmksz7uAaZnx39jlNRc6G8vzLmCkbue12/V1rbZxje/xwO6B\n+3vax8Ytc8L6S5O0BmZWWhzmVZqxcY3vpF/4PfwF32v9onBJ62NmpcVhXqVZq6qRE3A6cNnA/fOA\nc4eW+SPg2QP3rwGOXWFd5eTkdOC0Wv7WMjGlzM57vzg5dXHqal7NrJPTytNKudvK6q4ATk5yEnAz\n8Cxg+9AylwLnADuTnA58vaq+PLyiqhp+xypp+qaSWfMqzYR/Y6UZW7Xxraq9Sc4BLge2ABdX1a4k\nZ7fzL6qq9yc5M8n1wDeBF2541ZJWZGalxWFepdlLe4pEkiRJ2tQ6881tk1zEe16SnJjkg0k+k+TT\nSX553jUNS7IlyZVJ3jfvWgYlOSrJJUl2Jbm6PVXXGUl+pf0//ack70hyr3nXtCi6mlnzuj5dzqx5\nXbuu5hXM7Hp0Oa/Qzcx2ovGd5CLec3Yn8CtV9QiaDyP8UsfqA3gpcDXNgO4ueQ3w/qp6OHAqsGvO\n9dwlyfHAS4AfqapH0pxqfPZ8q1oMHc+seV2fTmbWvK5dx/MKZnY9OplX6G5mO9H4MtlFvOemqr5U\nVZ9sb3+D5gfruPlWdbckJwBnAm/inpe9mZskRwKPr6o/hmY8W1XdNueyhm0FDkuyFTgMuGnO9SyK\nzmbWvK7dAmTWvK5NZ/MKZnatFiCv0MHMdqXxneQi3p3Qfvr2UcBH51vJAV4F/Cdg37wLGfIg4J+T\nvDnJJ5K8Mclh8y5qv6q6Cfh94EaaT1R/vao+MN+qFsZCZNa8HrTOZta8rstC5BXM7EHqbF6hu5nt\nSuPbtVMHK0pyX+AS4KXtu9K5S/I04CtVdSUdeifa2go8GvjDqno0zSeSXzbfku6W5GiarwM9iebo\nwn2TPHeuRS2OzmfWvK5JZzNrXtel83kFM7sGnc0rdDezXWl8bwJOHLh/Is070s5IcgjwbuBtVfXe\nedcz4LHA05N8Hngn8BNJ/vuca9pvD7Cnqj7e3r+EJqRdcQbw+aq6tar2Au+h2Z8ar9OZNa9r1uXM\nmte163RewcyuUZfzCh3NbFca37su4p3kUJqLeF8655rukiTAxcDVVfXqedczqKpeXlUnVtWDaAaN\n/21VPX/edUEzbgvYneSh7UNnAJ+ZY0nDvgCcnuQ+7f/xGTQfXtB4nc2seV27jmfWvK5dZ/MKZnat\nOp5X6Ghmx31z20yMuoj3nMsa9DjgecBVSa5sHzuvqi6bY02jdO2U1kuAt7e/bD9Lhy6+XlUfS3IJ\n8Algb/vvG+Zb1WLoeGbN6/p0MrPmde06nlcws+vRybxCdzPrF1hIkiSpF7oy1EGSJEnaUDa+kiRJ\n6gUbX0mSJPWCja8kSZJ6wcZ3AST5RpI72mlfkm8N3N8+xe381yQ3JrktyQ1Jzhux3PPbOl40rW1L\nm0VX8tpue7CWuX+aWuqaDuV1S5LfSXJTktvbb2I7clrb1928qsOCaS+i/aKq+tsNWPfDgJuq6htJ\njgP+GvjPVfXnA8scDXwE+A7w6v3fES7pnuaZ1yT7gIdU1eenvW1pM5pzXn8HOB14YVXtTnIK8Nmq\n+tdp19J3nbiOr7qhqq4duBua7yV/yNBirwReA/zsrOqSdE8T5tWzelIHrJbX9oDSS4FTq2p3u/zc\nv+hhs/KX4iaU5GVJvjZi+uoEz70D2A0cBrxjYN5pNF+H+Ecb+gKkHtmovLb+V5IvJnl3kgdu1GuQ\n+mKD8vpImi94eGab12uTvHhjX0l/2fhuQlX1e1V19IjpfhM893CaBvdtwO3QjD8CXgecU46PkaZm\nI/LaegLwQOAHgJuBv2xzLGmNNiivJwBHAicDJwE/A+xIcsbGvZL+svHViqrqk8C3gd9qH3oxcFVV\nfWxgscy8MEn3sEJeqaq/r6q9VXUbzWnUk2iaYElztEJev93++9tV9a9V9U/ATuDMedS32dn4bkJJ\nXj7wqdTh6fbxa7jLIcCD29s/AfxUexrmi8Bjgd9P8tpp1y/1yZTzOjzG967NDP0raQ02KK9XjVjG\ns6sbwMZ3E6qq362qw0dMR6z0nDTOTnJUe/s0mqO8/7Nd5AU0R4t+CPhh4ApgB/DrG/6CpE1synn9\nQDv/lCQ/3F4i6b7AHwB7gF0ze2HSJrQRea2qzwIfAn49yaFJHg48C/jLWb2uPrHx1aB/B3yWZtzR\nW4HXVtWFAFV1W1V9pZ2+THM5s9ur6o75lSv12si8AsfSnCq9rV3mROBpVfXdeRQqadW8AmynGZN/\nK03D+xtV9cGZV9kDY6/jm+SPgacCX6mqR45Y5rXATwLfAl5QVVdOu1BJ45lXaXGYV2n2Jjni+2Zg\n26iZSc4Evr+qTgZ+EXj9lGqTdPDMq7Q4zKs0Y2Mb36r6EPC1VRZ5OvCWdtmPAkclOXY65Uk6GOZV\nWhzmVZq9aYzxPZ7mYsz77aG5Jp2k7jGv0uIwr9KUTesri4cvkXOPgcNJvCyHNKSq5nF5KfMqrUFX\n8wpmVlrJSpmdxhHfm2g+MbzfCe1jKxUwdjr//PMnWm4eU5dr63p91nbPaU56k9eu12dti1XfnEyc\nVxifWf9fN2dtXa+va39jp9H4Xgo8HyDJ6cDXq7nclaTuMa/S4jCv0pSNHeqQ5J3AE4FjkuwGzqf5\nxhGq6qKqen+SM5NcD3wTeOFGFixpNPMqLQ7zKs3e2Ma3qrZPsMw50ykHlpaWprWqqetybdDt+qxt\nNszrgbpcn7WtXdfrm5R5PVCX6+tybdDt+rpW29gvsJjahpKa1bakRZCEms+HZcYyr9KBupxXMLPS\nsFGZ9SuLJUmS1As2vpIkSeoFG19JkiT1go2vJEmSesHGV5IkSb1g4ytJkqResPGVJElSL9j4SpIk\nqRdsfCVJktQLNr6SJEnqBRtfSZIk9YKNryRJknrBxleSJEm9YOMrSZKkXrDxlSRJUi/Y+EqSJKkX\nbHwlSZLUCza+kiRJ6gUbX0mSJPWCja8kSZJ6YWzjm2RbkmuSXJfk3BXmH5nkfUk+meTTSV6wIZVK\nmoiZlRaHeZVmK1U1emayBbgWOAO4Cfg4sL2qdg0s83Lg8Ko6L8kx7fLHVtXeoXXVatuS+iYJVZUp\nr3MqmTWv0oG6nNd2OTMrDRiV2XFHfE8Drq+qG6rqTmAncNbQMvuAI9rbRwC3DgdS05FkXZN6wcxK\ni8O8SjM2rvE9Htg9cH9P+9igC4FTktwMfAp46fTK0z3VGif1hJmVFod5lWZsXOM7Sce0DfhEVR0H\n/DDwuiSHr7sySWthZqXFYV6lGds6Zv5NwIkD90+keUc66AXAKwGq6rNJPg88DLhieGU7duy46/bS\n0hJLS0sHW6+0sJaXl1leXt7ozUwts+ZVfbZoeQUzq36bNLPjPty2lWYg/ZOBm4GPcc+B938IfLmq\nfivJscD/B5xaVV8dWpcD79epGae71n0Y3P/dskEflplKZs2rdKAu57VdzsxKA0ZldtUjvlW1N8k5\nwOXAFuDiqtqV5Ox2/kXAK4A/SXIVEODXhgMpaTbMrLQ4zKs0e6se8Z3qhnw3um4e8d1cNuII0rSY\nV+lAXc4rmFlp2FovZyZJkiRtCja+kiRJ6gUbX0mSJPWCja8kSZJ6wcZXkiRJvWDjK0mSpF6w8ZUk\nSVIv2PhKkiSpF2x8JUmS1As2vpIkSeoFG19JkiT1go2vJEmSesHGV5IkSb1g4ytJkqResPGVJElS\nL9j4SpIkqRdsfCVJktQLNr6SJEnqBRtfSZIk9YKNryRJknrBxleSJEm9YOMrSZKkXhjb+CbZluSa\nJNclOXfEMktJrkzy6STLU69S0sTMrLQ4zKs0W6mq0TOTLcC1wBnATcDHge1VtWtgmaOADwNPqao9\nSY6pqltWWFetti2NlwRY6z4M7v9uSUJVZcrrnEpmzat0oC7ntV3OzEoDRmV23BHf04Drq+qGqroT\n2AmcNbTMc4B3V9UegJUCKWlmzKy0OMyrNGPjGt/jgd0D9/e0jw06Gbhfkg8muSLJz02zQEkHxcxK\ni8O8SjO2dcz8Sc6bHAI8GngycBjwkST/WFXXDS+4Y8eOu24vLS2xtLQ0caHSolteXmZ5eXmjNzO1\nzJpX9dmi5RXMrPpt0syOG+N7OrCjqra1988D9lXVBQPLnAvcp6p2tPffBFxWVZcMrcvxR+vkGN/N\nZYPGDE4ls+ZVOlCX89o+bmalAWsd43sFcHKSk5IcCjwLuHRomb8AfjzJliSHAY8Brp5G0ZIOmpmV\nFod5lWZs1aEOVbU3yTnA5cAW4OKq2pXk7Hb+RVV1TZLLgKuAfcAbq8pQSnNgZqXFYV6l2Vt1qMNU\nN+RpmHVzqMPmshGnTqfFvEoH6nJewcxKw9Y61EGSJEnaFGx8JUmS1As2vpIkSeoFG19JkiT1go2v\nJEmSesHGV5IkSb1g4ytJkqResPGVJElSL9j4SpIkqRdsfCVJktQLNr6SJEnqBRtfSZIk9YKNryRJ\nknrBxleSJEm9YOMrSZKkXrDxlSRJUi/Y+EqSJKkXbHwlSZLUCza+kiRJ6gUbX0mSJPWCja8kSZJ6\nYWzjm2RbkmuSXJfk3FWW+7Eke5M8Y7olSjoYZlZaHOZVmq1VG98kW4ALgW3AKcD2JA8fsdwFwGVA\nNqBOSRMws9LiMK/S7I074nsacH1V3VBVdwI7gbNWWO4lwCXAP0+5PkkHx8xKi8O8SjM2rvE9Htg9\ncH9P+9hdkhxPE9TXtw/V1KqTdLDMrLQ4zKs0Y+Ma30kC9mrgZVVVNKdgPA0jzY+ZlRaHeZVmbOuY\n+TcBJw7cP5HmHemgHwF2JgE4BvjJJHdW1aXDK9uxY8ddt5eWllhaWjr4iqUFtby8zPLy8kZvZmqZ\nNa/qs0XLK5hZ9dukmU3zJnLEzGQrcC3wZOBm4GPA9qraNWL5NwPvq6r3rDCvVtuWxmt+8a11Hwb3\nf7ckoaqmevRmWpk1r9KBupzXdl7nM9s27+vS9deo7hiV2VWP+FbV3iTnAJcDW4CLq2pXkrPb+Rdt\nSLWS1sTMSoujn3ldT+PqKA+t36pHfKe6oQV4N9p1HvHdXDbiCNK0mFfpQF3OKyxGZtf3Nwz8O6aD\nMSqzfnObJEmSesHGV5IkSb1g4ytJkqResPGVJElSL9j4SpIkqRdsfCVJktQLNr6SJEnqBRtfSZIk\n9cKq39wmSZL6ya8Y1mZk4ytJkkbwK4a1uTjUQZIkSb3gEV9J0qbl6XpJg2x8JUmbnKfrJTUc6iBJ\nkqRe8IivJPWMp/8l9ZWNryT1kqf/JfWPQx0kSZLUCza+kiRJ6gUbX0mSJPWCja8kSZJ6wQ+3SZLW\nxatESFoUNr6SpCnwKhGSum+ioQ5JtiW5Jsl1Sc5dYf5zk3wqyVVJPpzk1OmXKmkS5lVaHOZVmq2x\njW+SLcCFwDbgFGB7kocPLfY54AlVdSrwCuAN0y5U0njmVVoc5lWavUmO+J4GXF9VN1TVncBO4KzB\nBarqI1V1W3v3o8AJ0y1T0oTMq7Q4zKs0Y5M0vscDuwfu72kfG+VFwPvXU5SkNTOv0uIwr9KMTfLh\ntok/sZDkScAvAI9baf6OHTvuur20tMTS0tKkq5YW3vLyMsvLyxu9GfMqTcVyOx2YhSmbWl7BzKrf\nJv0bm3FSX7gNAAAQBElEQVSXkElyOrCjqra1988D9lXVBUPLnQq8B9hWVdevsJ7ycjXr01wyaK37\nMF4uqGOSUFVT/Ti7edUk1ve7BIZ/n0x7fdM0rdq6nNd2malntk8/J9p8RmV2kqEOVwAnJzkpyaHA\ns4BLh1b+AJpQPm9UKCXNhHmVFod5lWZs7FCHqtqb5BzgcmALcHFV7Upydjv/IuA3gaOB17cXMr+z\nqk7buLK1Xl5wfnMyr9oM+vL7ybxKszd2qMPUNuSp03Wb5lAHTznN30acOp0W87q5df0U9jTX1+Wh\nDtPkUAfpQOsZ6iBJkiQtPBtfSZIk9YKNryRJknphkuv4SpIkdcp6PwTpeOF+svGVJGmT6F8zuPYP\nfKufbHwlSdpUbAalURzjK0mSpF6w8ZUkSVIv2PhKkiSpF2x8JUmS1As2vpIkSeoFG19JkiT1go2v\nJEmSesHGV5IkSb1g4ytJkqResPGVJElSL9j4SpIkqRdsfCVJktQLNr6SJEnqBRtfSZIk9YKNryRJ\nknphbOObZFuSa5Jcl+TcEcu8tp3/qSSPWk9By8vL63n6hupybY3leRcwUpf3XZdrW4tZZrbr+67L\n9XW5ti7/Lmksz7uAqZnt39jltT91JpbnXcBI3c5rt+vrWm2rNr5JtgAXAtuAU4DtSR4+tMyZwPdX\n1cnALwKvX09BXdtBg7pcW2N53gWM1OV91+XaDtasM9v1fdfl+rpcW5d/lzSW513AVMz+b+zy2p86\nE8vzLmCkbue12/V1rbZxR3xPA66vqhuq6k5gJ3DW0DJPB94CUFUfBY5KcuzUK5U0CTMrLQ7zKs3Y\nuMb3eGD3wP097WPjljlh/aVJWgMzKy0O8yrNWlWNnICfBt44cP95wH8bWuZ9wOMG7n8AePQK6yon\nJ6cDp9Xyt5aJKWV23vvFyamLU1fzamadnFaeVsrdVlZ3E3DiwP0Tad5trrbMCe1jB6iqjNmWpPWb\nSmbNqzQT/o2VZmzcUIcrgJOTnJTkUOBZwKVDy1wKPB8gyenA16vqy1OvVNIkzKy0OMyrNGOrHvGt\nqr1JzgEuB7YAF1fVriRnt/Mvqqr3JzkzyfXAN4EXbnjVklZkZqXFYV6l2Us7NkiSJEna1DrzzW2T\nXMR7XpKcmOSDST6T5NNJfnneNQ1LsiXJlUneN+9aBiU5KsklSXYlubo9VdcZSX6l/T/9pyTvSHKv\nede0KLqaWfO6Pl3OrHldu67mFczsenQ5r9DNzHai8Z3kIt5zdifwK1X1COB04Jc6Vh/AS4GraT7J\n2CWvAd5fVQ8HTgV2zbmeuyQ5HngJ8CNV9UiaU43Pnm9Vi6HjmTWv69PJzJrXtet4XsHMrkcn8wrd\nzWwnGl8mu4j33FTVl6rqk+3tb9D8YB0336ruluQE4EzgTUBnPtmb5Ejg8VX1x9CMZ6uq2+Zc1rCt\nwGFJtgKHscKnpbWizmbWvK7dAmTWvK5NZ/MKZnatFiCv0MHMdqXxneQi3p2Q5CTgUcBH51vJAV4F\n/Cdg37wLGfIg4J+TvDnJJ5K8Mclh8y5qv6q6Cfh94EbgZppPS39gvlUtjIXIrHk9aJ3NrHldl4XI\nK5jZg9TZvEJ3M9uVxrdrpw5WlOS+wCXAS9t3pXOX5GnAV6rqSjr0TrS1FXg08IdV9WiaTyS/bL4l\n3S3J0TRfB3oSzdGF+yZ57lyLWhydz6x5XZPOZta8rkvn8wpmdg06m1fobma70vhOchHvuUpyCPBu\n4G1V9d551zPgscDTk3weeCfwE0n++5xr2m8PsKeqPt7ev4QmpF1xBvD5qrq1qvYC76HZnxqv05k1\nr2vW5cya17XrdF7BzK5Rl/MKHc1sVxrfSS7iPTdJAlwMXF1Vr553PYOq6uVVdWJVPYhm0PjfVtXz\n510XNOO2gN1JHto+dAbwmTmWNOwLwOlJ7tP+H59B8+EFjdfZzJrXtet4Zs3r2nU2r2Bm16rjeYWO\nZnbcVxbPxKiLeM+5rEGPo/kO9auSXNk+dl5VXTbHmkbp2imtlwBvb3/ZfpYOXXy9qj6W5BLgE8De\n9t83zLeqxdDxzJrX9elkZs3r2nU8r2Bm16OTeYXuZtYvsJAkSVIvdGWogyRJkrShbHwlSZLUCza+\nkiRJ6gUbX0mSJPWCja8kSZJ6wcZXkiRJvWDjK0mSpF6w8ZUkSVIv2PhKkiSpF2x8JUmS1As2vpIk\nSeoFG19JkiT1go2vJEmSesHGV5IkSb1g4ytJkqResPGVJElSL9j4SpIkqRdsfCVJktQLNr6SJEnq\nBRtfSZ2S5NNJnrDB2zgpyb4kE/0OTPInSV7R3n58kmumWMv7k/xce/sFST40xXU/N8nl01rfNCW5\nI8lJq8y/IcmTJ1zXxK+zy/tE0saz8ZU6LMlzklzRNgk3t03S4+Zd17QMNpT7VdUPVtX/mldNI1Q7\nUVUfqqofGPeEJDuSvHXsiqvOrKqxy02wvXs081X19qp6ynrXvRGq6vCqugFW/jlgYJ9PsK6JX+fw\nsu0+e/CEZR8gyZOTXJPkm0n+NskD1rIeSbNj4yt1VJJfBV4F/A7wfcCJwOuAp49YfsvsquulTHVl\nrWmuc/+qN2Cdm91B77MkxwDvBn4dOBq4AvjTKdclacpsfKUOSnIk8FvAi6vqvVX17ar6blX9VVWd\n2y6zI8klSd6a5Dbg55Mcl+TSJLcmuS7Jvx9Y52nt0ePbknwpye+3j987yduS3JLka0k+luT7RtR1\nbpI9SW5vj3T9RPt4krwsyfXtev40ydEDz/vxJP/Qrv/GJD+f5D8AzwF+rT2i/Rftsned4k5yrySv\nTnJTO70qyaHtvKW2ll9N8uX2iPgLBrb51CRXtq/3xiTnH8T+f1SST7Svcydw74F5S0l2r7ZPkmwD\nzgOe1b62K9tll5P8TpIPA98AHtw+9qIDN5//luTrSXbt38fD+2bgZ2D/0eL9R8m/3tZyeoaGTiR5\nbJKPt+v+WJJ/OzBvOclvJ/n79vmXJ7l/O2+in5EkL0xy6cD965L82cD93UlObW/vS/KQJL/ICj8H\nrUcl+VRb784k9xrx/zX8OvclOTvJ/27rvXClZZPs32efarf9zJXWP8IzgE9X1bur6jvADuCHkjz0\nINYhacZsfKVu+rc0zdafj1nu6cC7qupI4B3ATuBG4N8APwP8bpIntcu+BnhVu+yDufvo1M8DRwAn\nAPcDzga+PbyhJA8Dfgn40ao6Avi/gBva2b/c1vKEdttfozk6TZIHAu9vt38M8MPAJ6vqjcDbgQva\n095ntesaPMX968BpwA+102nAbwyUdWxb+3HAi4DXpXnTAE1j+bz29T4V+L+TnMUYbWP9XuAtNEfy\n3gX8NCucdh+1T6rqMuB3gZ3ta3vUwNOeB/x74HDgC9zzlP5jgOuB+wPnA+9JctQK+4ah249v/z2y\nqo6oqn8cqvV+wF8Br6b5f/4D4K8G36AA24EX0JxhOBT4f9vHJ/oZAZb315HkOOAQ4PT2/oOB76mq\nqwbrr6o3sPLPQYBnAk8BHgSc2tY2qacCP9o+72eT3GMoRFXtH0t+arvtdyV5QNssj5qe3T7nEcCn\nBtb1LZr/tx88iBolzZiNr9RN9wduqap9Y5b7h6raf4Tte4HHAudW1Xeq6lPAm4Dnt/O/A5yc5Jiq\n+lZVfWzg8fsDJ1fjyqq6Y4VtfRe4F/CIJIdU1Y1V9bl23tnAb1TVzVV1J83R6p9JM/ziOcDfVNWf\ntketv9rWtt9qp5mfA/x2Vd1SVbe06/25gfl3tvO/W1X/g6bZfRhAVf1dVX2mvf1PNG8KnrjKtvY7\nHdhaVa9p1/tu4OMjll1tn2SF11bAn1TVrqraV1V7V1jnVwa2/WfAtTRN3Eoy4vZKngpc245x3VdV\nO4FruHvoTAFvrqrrq+pfgD+jeZMCE/6MVNXngTuSPIrmTdDlwM3tG4QncvdR6XGvZX89r62qL1XV\n14D3DdQzid+rqturajfwwUmf2/4fHr3KtLNd9HuA24eefjtw34OoUdKM2fhK3XQrcEzGX3Vgz8Dt\n44CvVtU3Bx67ETi+vf0i4KHArvZU9f5m6q00DcrOdjjBBUm2Dm+oqq4H/h+aU7pfTvLOJP+mnX0S\n8Of7j4oBVwN7aY7IngB8bnh9EzqO5qjo4Os5buD+rUNvDr5F23gkeUySDyb5SpKv0zTn959wmzcN\nPfaFlRYcs09G2T1m/krbPm6lBQ/ScTT7b7V1f2ng9re5u4mb6Gek9XfAEs2R379rpyfSNMJ/d5A1\nj6rnYJ/7LZpGdZq+QXMUfNCRwEpvGiV1hI2v1E0fAf4V+KlVlhk+7X0zcL8kg83BA2ib4/ZI3nOq\n6nuBC4BLktynqvZW1W9X1SNojhg/jbuPEh+4wap3VtXjgQe2276gnXUjsG3oyNhhVXUzTaP3kFVe\nw2pupmmqB1/PzWOes987aIYsnFBVRwF/xGS/877I3W8W9nvgqIVX2SejXtu417zStve/5m9yYAP3\nfx7Eem/inq/jgdyz0b6Hg/kZoWlun0TT+C5zdyP8REY3vpNcvWGiKzysVzvU4Y5Vpu3top+hGX6z\n/3nfQ/Nz/plZ1ClpbWx8pQ6qqtuA36QZs3pWksOSHJLkJ5Psb6wy9JzdwD8Ar0zzobBTgV8A3gaQ\n5HlJvrdd/DaaRmJfkicleWQ7LOEOmuED3x2uKclD03xw6140Tfm/DCz3RzTjiR/QLvu9SfafQn87\ncEaSZybZmuT+SfY3DF+mGW88yjuB30hyTJpP0f8mzdHHSdwX+FpVfSfJaTTDJiZpnv4B2Jvkl9t9\n/gzgx1ZacMw++RJwUnKPKzeMG5LwfQPbfibwAzRjpAE+CTy73Y8/yoFjj/8Z2MfoNxn/A3hoku3t\n85/Vrvsvx9U26c9Ia3/je+/2jc/fA9toxgZfOeI5434ORtY2oZWGnQxu+6591g51OHyV6Z3ton8O\n/GCSZyS5N8147E9W1f9eR52SNpiNr9RRVfUHwK/SfJjrKzRHVV/M3R94W+k6p9tpjpDeDLwH+M2q\n+tt23lOATye5g+Yyac+uqn+lGY7wLppm+Gqao3QrNZf3Al5J02B9keaDaue1814DXAr8dZLbaY5Y\nn9a+jt3AmcB/pBnCcSXNB44ALgZOaYdIvGeFbf4OzWWirmqnK9rH7tpNKzxnvxcDv93W85+556Wm\nVnxuO0b5GTQfpLoV+Fmay1at9NzV9sm72n9vTXLFhDUX8I/Aye06XwH8dDvGlfZ1PITmw4M7aN5U\n7K/7W8B/AT6c5KtJHsOB1x++leZI7X8EbqH54NrTquqrI2ob/Pma9GeEqrqOpjn+UHv/duCzwIer\natQH88b9HAzXM27e8HKD84eX3QG8pd32z4xY/z032Iw5/2maff5Vmg/SPXvVJ0mauxz4e0iSJEna\nnDziK0mSpF6w8ZUkSVIv2PhKkiSpF2x8JUmS1As2vpIkSeoFG19JkiT1go2vJEmSesHGV5IkSb1g\n4ytJkqRe+P8ByK6st2Vo4S0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10ea55080>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "init = 0\n",
    "k = 10\n",
    "Ts = [k*d + 1 + i for i in range(2*d)]\n",
    "num_reps = 10**2\n",
    "dists = cross_sectional_dist(mc2, T=Ts, init=init, num_reps=num_reps)\n",
    "\n",
    "fig, axes = plt.subplots(2, d, figsize=(12, 6))\n",
    "for dist, T, ax in zip(dists, Ts, axes.flatten()):\n",
    "    draw_histogram(dist, ax=ax,\n",
    "                   title='T = {T}'.format(T=T))\n",
    "fig.suptitle('Cross sectional distributions with init={init}'.format(init=init),\n",
    "             y=0.05, fontsize=12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare these with the rows of $P^{10d+1}, \\ldots, P^{10d+d}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3: Nearly completely decomposable chain"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the Markov chain given by the following stochastic matrix $P^{\\varepsilon}$,\n",
    "parameterized by $\\varepsilon$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def P_epsilon(eps, p=0.5):\n",
    "    P = np.array([[1-(p+eps), p,         eps],\n",
    "                  [p,         1-(p+eps), eps],\n",
    "                  [eps,       eps,       1-2*eps]])\n",
    "    return P"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If $\\varepsilon = 0$,\n",
    "then the Markovh chain is reducible into two recurrent classes, `[0, 1]` and `[2]`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.5,  0.5,  0. ],\n",
       "       [ 0.5,  0.5,  0. ],\n",
       "       [ 0. ,  0. ,  1. ]])"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "P_epsilon(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0, 1]), array([2])]"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MarkovChain(P_epsilon(0)).recurrent_classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If $\\varepsilon > 0$ but small, the chain is irreducible,\n",
    "but transition within each of the subsets `[0, 1]` and `[2]` is much more likely\n",
    "than that between these sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.499,  0.5  ,  0.001],\n",
       "       [ 0.5  ,  0.499,  0.001],\n",
       "       [ 0.001,  0.001,  0.998]])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "P_epsilon(0.001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([0, 1, 2])]"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MarkovChain(P_epsilon(0.001)).recurrent_classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Analytically, the unique stationary distribution of the chain with $\\varepsilon > 0$\n",
    "is `(1/3, 1/3, 1/3)`, independent of the value of $\\varepsilon$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However,\n",
    "for such matrices with small values of $\\varepsilon > 0$,\n",
    "general purpose eigenvalue solvers are numerically unstable.\n",
    "\n",
    "For example, if we use [`numpy.linalg.eig`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eig.html)\n",
    "to compute the eigenvector that corresponds\n",
    "to the dominant (i.e., largest in magnitude) eigenvalue:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.set_printoptions(precision=15)  # Increase the number of digits printed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epsilon = 1e-12\n",
      "[ 0.33333662324477   0.33333662324477   0.333326753510459]\n",
      "epsilon = 1e-13\n",
      "[ 0.333344849397111  0.333344849397111  0.333310301205779]\n",
      "epsilon = 1e-14\n",
      "[ 0.333000773366967  0.333000773366967  0.333998453266065]\n",
      "epsilon = 1e-15\n",
      "[ 0.329141015582661  0.329141015582661  0.341717968834677]\n",
      "epsilon = 1e-16\n",
      "[ 0.5  0.5  0. ]\n",
      "epsilon = 1e-17\n",
      "[ 0.5  0.5  0. ]\n"
     ]
    }
   ],
   "source": [
    "epsilons = [float('1e-{i}'.format(i=i)) for i in range(12, 18)]\n",
    "for eps in epsilons:\n",
    "    print('epsilon = {eps}'.format(eps=eps))\n",
    "    w, v = np.linalg.eig(P_epsilon(eps).T)\n",
    "    i = w.argmax()\n",
    "    print(v[:, i]/v[:, i].sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output becomes farther from the actual stationary distribution `(1/3, 1/3, 1/3)`\n",
    "as $\\varepsilon$ becomes smaller."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same applies to [`scipy.linalg.eig`](http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.eig.htm):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epsilon = 1e-12\n",
      "[ 0.33333662324477   0.33333662324477   0.333326753510459]\n",
      "epsilon = 1e-13\n",
      "[ 0.333344849397131  0.333344849397131  0.333310301205738]\n",
      "epsilon = 1e-14\n",
      "[ 0.333883550460886  0.333883550460886  0.332232899078228]\n",
      "epsilon = 1e-15\n",
      "[ 0.329141015582661  0.329141015582661  0.341717968834677]\n",
      "epsilon = 1e-16\n",
      "[ 0.344742003948057  0.344742003948057  0.310515992103886]\n",
      "epsilon = 1e-17\n",
      "[ 0.523579944426163  0.523579944426163 -0.047159888852327]\n"
     ]
    }
   ],
   "source": [
    "import scipy.linalg\n",
    "\n",
    "for eps in epsilons:\n",
    "    print('epsilon = {eps}'.format(eps=eps))\n",
    "    w, v = scipy.linalg.eig(P_epsilon(eps), left=True, right=False)\n",
    "    i = w.argmax()\n",
    "    print(v[:, i]/v[:, i].sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`MarkovChain` in `quantecon` employs\n",
    "the algorithm called the \"[GTH algorithm](http://quanteconpy.readthedocs.org/en/stable/markov/gth_solve.html)\",\n",
    "which is a numerically stable variant of Gaussian elimination,\n",
    "specialized for Markov chains."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epsilon = 1e-12\n",
      "[ 0.333333333333333  0.333333333333333  0.333333333333333]\n",
      "epsilon = 1e-13\n",
      "[ 0.333333333333333  0.333333333333333  0.333333333333333]\n",
      "epsilon = 1e-14\n",
      "[ 0.333333333333333  0.333333333333333  0.333333333333333]\n",
      "epsilon = 1e-15\n",
      "[ 0.333333333333333  0.333333333333333  0.333333333333333]\n",
      "epsilon = 1e-16\n",
      "[ 0.333333333333333  0.333333333333333  0.333333333333333]\n",
      "epsilon = 1e-17\n",
      "[ 0.333333333333333  0.333333333333333  0.333333333333333]\n",
      "epsilon = 1e-100\n",
      "[ 0.333333333333333  0.333333333333333  0.333333333333333]\n"
     ]
    }
   ],
   "source": [
    "for eps in epsilons + [1e-100]:\n",
    "    print('epsilon = {eps}'.format(eps=eps))\n",
    "    print(MarkovChain(P_epsilon(eps)).stationary_distributions[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "It succeeds in obtaining the correct stationary distribution for any value of $\\varepsilon$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}