diff --git a/.gitignore b/.gitignore index 663f268d..7b225ae2 100644 --- a/.gitignore +++ b/.gitignore @@ -32,6 +32,7 @@ bin build include lib +share cast-offs develop-eggs development @@ -41,3 +42,10 @@ development .mr.developer.cfg outline_improvements.txt src +html +slides +new_mash +.buildinfo +pip-selfcheck.json +.ipynb_checkpoints +testenvs diff --git a/Makefile b/Makefile index 20506467..84b07bcc 100644 --- a/Makefile +++ b/Makefile @@ -2,8 +2,9 @@ # # You can set these variables from the command line. +BINDIR = ./bin SPHINXOPTS = -SPHINXBUILD = ./bin/sphinx-build +SPHINXBUILD = $(BINDIR)/sphinx-build PAPER = BUILDDIR = build diff --git a/README.rst b/README.rst index d79a8a4e..e7769775 100644 --- a/README.rst +++ b/README.rst @@ -10,35 +10,78 @@ This package provides the source for all lecture materials used for the `Internet Programming in Python`_ section of the `Certificate in Python Programming`_ offered by the `University of Washington Professional & Continuing Education`_ program. This version of the documentation is used for -the Winter 2015 instance of the course, Taught by `Cris Ewing`_ +the Winter 2016 instance of the course, Taught by `Cris Ewing`_ -.. _Internet Programming in Python: http://www.pce.uw.edu/courses/internet-programming-python/downtown-seattle-winter-2015/ +.. _Internet Programming in Python: http://www.pce.uw.edu/courses/internet-programming-python/downtown-seattle-winter-2016/ .. _Certificate in Python Programming: http://www.pce.uw.edu/certificates/python-programming.html .. _University of Washington Professional & Continuing Education: http://www.pce.uw.edu/ .. _Cris Ewing: http://www.linkedin.com/profile/view?id=19741495 +This course is taught using Python 3. + +This documentation builds both an HTML version of the course lectures (for the +students) and a set of slides (for the instructor). It uses the Python-based +documentation tool `Sphinx`_ and the `hieroglyph`_ sphinx extension. Shell +examples use `iPython` and tests are written for `pytest`. The build +environment is managed using `virtualenv` and `pip` + +.. _iPython: http://ipython.org/ +.. _Sphinx: http://sphinx-doc.org/ +.. _hieroglyph: http://docs.hieroglyph.io/en/latest/ +.. _pytest: http://pytest.org/latest/ +.. _virtualenv: https://virtualenv.pypa.io/en/latest/ +.. _pip: https://pip.pypa.io/en/stable + Building The Documentation -------------------------- -This documentation is built using docutils and Sphinx. The package uses -`zc.buildout` to manage setup and dependencies. This package uses the version -2 `bootstrap.py` script. This version of the script will attempt to use -setuptools 0.7 or better. If you have an earlier version of setuptools -installed, please upgrade prior to bootstrapping this buildout. +To build the documentation locally, begin by cloning the project to your +machine: + +.. code-block:: bash + + $ git clone https://github.com/cewing/training.python_web.git + +Change directories into the repository, then create a virtualenv using Python +3: + +.. code-block:: bash + + $ cd training.python_web + $ virtualenv --python /path/to/bin/python3.5 . + Running virtualenv with interpreter /path/to/bin/python3.5 + New python executable in training.python_web/bin/python3.5 + Also creating executable in training.python_web/bin/python + Installing setuptools, pip...done. + +Install the requirements for the documentation using pip: + +.. code-block:: bash + + $ bin/pip install -r requirements.pip + ... + + Successfully installed Babel-2.0 Jinja2-2.8 MarkupSafe-0.23 Pygments-2.0.2 Sphinx-1.3.1 alabaster-0.7.6 appnope-0.1.0 decorator-4.0.2 docutils-0.12 gnureadline-6.3.3 hieroglyph-0.7.1 ipython-4.0.0 ipython-genutils-0.1.0 path.py-8.1 pexpect-3.3 pickleshare-0.5 py-1.4.30 pytest-2.7.2 pytz-2015.4 simplegeneric-0.8.1 six-1.9.0 snowballstemmer-1.2.0 sphinx-rtd-theme-0.1.8 traitlets-4.0.0 + +Once that has successfully completed, you should be able to build both the html +documentation and the slides using the included Makefile. + +.. code-block:: bash + + $ make html + ... + + Build finished. The HTML pages are in build/html. + + (webdocs)$ make slides + ... -After cloning this package from the repository, do the following:: + Build finished. The HTML slides are in build/slides. - $ cd training.python_web # the location of your local copy - $ python bootstrap.py # must be Python 2.6 or 2.7 - $ bin/buildout - $ bin/sphinx # to build the main documentation and course outline - $ bin/build_s5 # to build the class session presentations +.. note:: If you prefer to build your virtualenvs in other ways, you will need + to adjust the `BINDIR` variable in `Makefile` to fit your reality. -At the end of a successful build, you will find a ``build/html`` directory, -containing the completed documentation and presentations. -.. _zc.buildout: https://pypi.python.org/pypi/zc.buildout/ -.. _bootstrap.py: http://downloads.buildout.org/2/bootstrap.py Reading The Documentation ------------------------- diff --git a/bootstrap.py b/bootstrap.py deleted file mode 100644 index 4d7aa17b..00000000 --- a/bootstrap.py +++ /dev/null @@ -1,170 +0,0 @@ -############################################################################## -# -# Copyright (c) 2006 Zope Foundation and Contributors. -# All Rights Reserved. -# -# This software is subject to the provisions of the Zope Public License, -# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution. -# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED -# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS -# FOR A PARTICULAR PURPOSE. -# -############################################################################## -"""Bootstrap a buildout-based project - -Simply run this script in a directory containing a buildout.cfg. -The script accepts buildout command-line options, so you can -use the -c option to specify an alternate configuration file. -""" - -import os -import shutil -import sys -import tempfile - -from optparse import OptionParser - -tmpeggs = tempfile.mkdtemp() - -usage = '''\ -[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options] - -Bootstraps a buildout-based project. - -Simply run this script in a directory containing a buildout.cfg, using the -Python that you want bin/buildout to use. - -Note that by using --find-links to point to local resources, you can keep -this script from going over the network. -''' - -parser = OptionParser(usage=usage) -parser.add_option("-v", "--version", help="use a specific zc.buildout version") - -parser.add_option("-t", "--accept-buildout-test-releases", - dest='accept_buildout_test_releases', - action="store_true", default=False, - help=("Normally, if you do not specify a --version, the " - "bootstrap script and buildout gets the newest " - "*final* versions of zc.buildout and its recipes and " - "extensions for you. If you use this flag, " - "bootstrap and buildout will get the newest releases " - "even if they are alphas or betas.")) -parser.add_option("-c", "--config-file", - help=("Specify the path to the buildout configuration " - "file to be used.")) -parser.add_option("-f", "--find-links", - help=("Specify a URL to search for buildout releases")) - - -options, args = parser.parse_args() - -###################################################################### -# load/install setuptools - -to_reload = False -try: - import pkg_resources - import setuptools -except ImportError: - ez = {} - - try: - from urllib.request import urlopen - except ImportError: - from urllib2 import urlopen - - # XXX use a more permanent ez_setup.py URL when available. - exec(urlopen('https://bitbucket.org/pypa/setuptools/raw/0.7.2/ez_setup.py' - ).read(), ez) - setup_args = dict(to_dir=tmpeggs, download_delay=0) - ez['use_setuptools'](**setup_args) - - if to_reload: - reload(pkg_resources) - import pkg_resources - # This does not (always?) update the default working set. We will - # do it. - for path in sys.path: - if path not in pkg_resources.working_set.entries: - pkg_resources.working_set.add_entry(path) - -###################################################################### -# Install buildout - -ws = pkg_resources.working_set - -cmd = [sys.executable, '-c', - 'from setuptools.command.easy_install import main; main()', - '-mZqNxd', tmpeggs] - -find_links = os.environ.get( - 'bootstrap-testing-find-links', - options.find_links or - ('http://downloads.buildout.org/' - if options.accept_buildout_test_releases else None) - ) -if find_links: - cmd.extend(['-f', find_links]) - -setuptools_path = ws.find( - pkg_resources.Requirement.parse('setuptools')).location - -requirement = 'zc.buildout' -version = options.version -if version is None and not options.accept_buildout_test_releases: - # Figure out the most recent final version of zc.buildout. - import setuptools.package_index - _final_parts = '*final-', '*final' - - def _final_version(parsed_version): - for part in parsed_version: - if (part[:1] == '*') and (part not in _final_parts): - return False - return True - index = setuptools.package_index.PackageIndex( - search_path=[setuptools_path]) - if find_links: - index.add_find_links((find_links,)) - req = pkg_resources.Requirement.parse(requirement) - if index.obtain(req) is not None: - best = [] - bestv = None - for dist in index[req.project_name]: - distv = dist.parsed_version - if _final_version(distv): - if bestv is None or distv > bestv: - best = [dist] - bestv = distv - elif distv == bestv: - best.append(dist) - if best: - best.sort() - version = best[-1].version -if version: - requirement = '=='.join((requirement, version)) -cmd.append(requirement) - -import subprocess -if subprocess.call(cmd, env=dict(os.environ, PYTHONPATH=setuptools_path)) != 0: - raise Exception( - "Failed to execute command:\n%s", - repr(cmd)[1:-1]) - -###################################################################### -# Import and run buildout - -ws.add_entry(tmpeggs) -ws.require(requirement) -import zc.buildout.buildout - -if not [a for a in args if '=' not in a]: - args.append('bootstrap') - -# if -c was provided, we push it back into args for buildout' main function -if options.config_file is not None: - args[0:0] = ['-c', options.config_file] - -zc.buildout.buildout.main(args) -shutil.rmtree(tmpeggs) \ No newline at end of file diff --git a/buildout.cfg b/buildout.cfg deleted file mode 100644 index ed81b6b9..00000000 --- a/buildout.cfg +++ /dev/null @@ -1,57 +0,0 @@ -# -# Buildout to set-up Sphinx -# -[buildout] -parts = - sphinx - -extensions = mr.developer -auto-checkout = * -always-checkout = force - -allow-picked-versions = true -show-picked-versions = true - -versions = versions - -script-in = ${buildout:directory}/commands/build.in - -[sphinx] -recipe = collective.recipe.sphinxbuilder -outputs = - html -source = ${buildout:directory}/source/main -build = ${buildout:directory}/build -eggs = - Sphinx - docutils - Pygments - hieroglyph - ipython - -[versions] -# pin versions for continued sanity -Jinja2 = 2.7.2 -Pygments = 1.6 -Sphinx = 1.2.2 -collective.recipe.sphinxbuilder = 0.8.2 - -#Required by: -#collective.recipe.sphinxbuilder 0.7.1 -docutils = 0.11 - -#Required by: -#collective.recipe.sphinxbuilder 0.7.1 -zc.buildout = 1.5.2 - -#Required by: -#collective.recipe.sphinxbuilder 0.7.1 -zc.recipe.egg = 1.3.2 - -#Required by: -#rjm.recipe.venv 0.8 -virtualenv = 1.10 - - -[sources] -hieroglyph = git https://github.com/nyergler/hieroglyph.git diff --git a/downloads/.gitignore b/downloads/.gitignore deleted file mode 100644 index d6b7ef32..00000000 --- a/downloads/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -* -!.gitignore diff --git a/notebooks/Networking & Sockets.ipynb b/notebooks/Networking & Sockets.ipynb new file mode 100644 index 00000000..09d64583 --- /dev/null +++ b/notebooks/Networking & Sockets.ipynb @@ -0,0 +1,309 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "import socket" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def get_constants(prefix):\n", + " \"\"\"mapping of socket module constants to their names\"\"\"\n", + " return {getattr(socket, n): n\n", + " for n in dir(socket)\n", + " if n.startswith(prefix)\n", + " }" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "families = get_constants('AF_')\n", + "types = get_constants('SOCK_')\n", + "protocols = get_constants('IPPROTO_')" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "{: 'AF_UNSPEC',\n", + " : 'AF_UNIX',\n", + " : 'AF_INET',\n", + " : 'AF_SNA',\n", + " 12: 'AF_DECnet',\n", + " : 'AF_APPLETALK',\n", + " : 'AF_ROUTE',\n", + " : 'AF_LINK',\n", + " : 'AF_IPX',\n", + " : 'AF_INET6',\n", + " : 'AF_SYSTEM'}" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "families" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "{: 'SOCK_STREAM',\n", + " : 'SOCK_DGRAM',\n", + " : 'SOCK_RAW',\n", + " : 'SOCK_RDM',\n", + " : 'SOCK_SEQPACKET'}" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "types" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "{0: 'IPPROTO_IP',\n", + " 1: 'IPPROTO_ICMP',\n", + " 2: 'IPPROTO_IGMP',\n", + " 3: 'IPPROTO_GGP',\n", + " 4: 'IPPROTO_IPV4',\n", + " 6: 'IPPROTO_TCP',\n", + " 8: 'IPPROTO_EGP',\n", + " 12: 'IPPROTO_PUP',\n", + " 17: 'IPPROTO_UDP',\n", + " 22: 'IPPROTO_IDP',\n", + " 29: 'IPPROTO_TP',\n", + " 36: 'IPPROTO_XTP',\n", + " 41: 'IPPROTO_IPV6',\n", + " 43: 'IPPROTO_ROUTING',\n", + " 44: 'IPPROTO_FRAGMENT',\n", + " 46: 'IPPROTO_RSVP',\n", + " 47: 'IPPROTO_GRE',\n", + " 50: 'IPPROTO_ESP',\n", + " 51: 'IPPROTO_AH',\n", + " 58: 'IPPROTO_ICMPV6',\n", + " 59: 'IPPROTO_NONE',\n", + " 60: 'IPPROTO_DSTOPTS',\n", + " 63: 'IPPROTO_HELLO',\n", + " 77: 'IPPROTO_ND',\n", + " 80: 'IPPROTO_EON',\n", + " 103: 'IPPROTO_PIM',\n", + " 108: 'IPPROTO_IPCOMP',\n", + " 132: 'IPPROTO_SCTP',\n", + " 255: 'IPPROTO_RAW',\n", + " 256: 'IPPROTO_MAX'}" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "protocols" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "default_socket = socket.socket()" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'AF_INET'" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "families[default_socket.family]" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'SOCK_STREAM'" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "types[default_socket.type]" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'IPPROTO_IP'" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "protocols[default_socket.proto]" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def get_address_info(host, port):\n", + " for response in socket.getaddrinfo(host, port):\n", + " fam, typ, pro, nam, add = response\n", + " print('family: {}'.format(families[fam]))\n", + " print('type: {}'.format(types[typ]))\n", + " print('protocol: {}'.format(protocols[pro]))\n", + " print('canonical name: {}'.format(nam))\n", + " print('socket address: {}'.format(add))\n", + " print('')" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "family: AF_INET\n", + "type: SOCK_DGRAM\n", + "protocol: IPPROTO_UDP\n", + "canonical name: \n", + "socket address: ('127.0.0.1', 80)\n", + "\n", + "family: AF_INET\n", + "type: SOCK_STREAM\n", + "protocol: IPPROTO_TCP\n", + "canonical name: \n", + "socket address: ('127.0.0.1', 80)\n", + "\n" + ] + } + ], + "source": [ + "get_address_info(socket.gethostname(), 'http')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/requirements.pip b/requirements.pip new file mode 100644 index 00000000..7539549c --- /dev/null +++ b/requirements.pip @@ -0,0 +1,24 @@ +alabaster==0.7.6 +appnope==0.1.0 +Babel==2.0 +decorator==4.0.2 +docutils==0.12 +gnureadline==6.3.3 +hieroglyph==0.7.1 +ipython==4.0.0 +ipython-genutils==0.1.0 +Jinja2==2.8 +MarkupSafe==0.23 +path.py==8.1 +pexpect==3.3 +pickleshare==0.5 +py==1.4.30 +Pygments==2.0.2 +pytest==2.7.2 +pytz==2015.4 +simplegeneric==0.8.1 +six==1.9.0 +snowballstemmer==1.2.0 +Sphinx==1.3.1 +sphinx-rtd-theme==0.1.8 +traitlets==4.0.0 diff --git a/resources/session01/echo_client.py b/resources/session01/echo_client.py new file mode 100644 index 00000000..6b2f0472 --- /dev/null +++ b/resources/session01/echo_client.py @@ -0,0 +1,48 @@ +import socket +import sys + + +def client(msg, log_buffer=sys.stderr): + server_address = ('localhost', 10000) + # TODO: Replace the following line with your code which will instantiate + # a TCP socket with IPv4 Addressing, call the socket you make 'sock' + sock = None + print('connecting to {0} port {1}'.format(*server_address), file=log_buffer) + # TODO: connect your socket to the server here. + + # you can use this variable to accumulate the entire message received back + # from the server + received_message = '' + + # this try/finally block exists purely to allow us to close the socket + # when we are finished with it + try: + print('sending "{0}"'.format(msg), file=log_buffer) + # TODO: send your message to the server here. + + # TODO: the server should be sending you back your message as a series + # of 16-byte chunks. Accumulate the chunks you get to build the + # entire reply from the server. Make sure that you have received + # the entire message and then you can break the loop. + # + # Log each chunk you receive. Use the print statement below to + # do it. This will help in debugging problems + chunk = '' + print('received "{0}"'.format(chunk.decode('utf8')), file=log_buffer) + finally: + # TODO: after you break out of the loop receiving echoed chunks from + # the server you will want to close your client socket. + print('closing socket', file=log_buffer) + + # TODO: when all is said and done, you should return the entire reply + # you received from the server as the return value of this function. + + +if __name__ == '__main__': + if len(sys.argv) != 2: + usage = '\nusage: python echo_client.py "this is my message"\n' + print(usage, file=sys.stderr) + sys.exit(1) + + msg = sys.argv[1] + client(msg) diff --git a/resources/session01/echo_server.py b/resources/session01/echo_server.py new file mode 100644 index 00000000..4103ac6a --- /dev/null +++ b/resources/session01/echo_server.py @@ -0,0 +1,76 @@ +import socket +import sys + + +def server(log_buffer=sys.stderr): + # set an address for our server + address = ('127.0.0.1', 10000) + # TODO: Replace the following line with your code which will instantiate + # a TCP socket with IPv4 Addressing, call the socket you make 'sock' + sock = None + # TODO: You may find that if you repeatedly run the server script it fails, + # claiming that the port is already used. You can set an option on + # your socket that will fix this problem. We DID NOT talk about this + # in class. Find the correct option by reading the very end of the + # socket library documentation: + # http://docs.python.org/3/library/socket.html#example + + # log that we are building a server + print("making a server on {0}:{1}".format(*address), file=log_buffer) + + # TODO: bind your new sock 'sock' to the address above and begin to listen + # for incoming connections + + try: + # the outer loop controls the creation of new connection sockets. The + # server will handle each incoming connection one at a time. + while True: + print('waiting for a connection', file=log_buffer) + + # TODO: make a new socket when a client connects, call it 'conn', + # at the same time you should be able to get the address of + # the client so we can report it below. Replace the + # following line with your code. It is only here to prevent + # syntax errors + addr = ('bar', 'baz') + try: + print('connection - {0}:{1}'.format(*addr), file=log_buffer) + + # the inner loop will receive messages sent by the client in + # buffers. When a complete message has been received, the + # loop will exit + while True: + # TODO: receive 16 bytes of data from the client. Store + # the data you receive as 'data'. Replace the + # following line with your code. It's only here as + # a placeholder to prevent an error in string + # formatting + data = b'' + print('received "{0}"'.format(data.decode('utf8'))) + # TODO: Send the data you received back to the client, log + # the fact using the print statement here. It will help in + # debugging problems. + print('sent "{0}"'.format(data.decode('utf8'))) + # TODO: Check here to see if the message you've received is + # complete. If it is, break out of this inner loop. + + finally: + # TODO: When the inner loop exits, this 'finally' clause will + # be hit. Use that opportunity to close the socket you + # created above when a client connected. + print( + 'echo complete, client connection closed', file=log_buffer + ) + + except KeyboardInterrupt: + # TODO: Use the python KeyboardInterrupt exception as a signal to + # close the server socket and exit from the server function. + # Replace the call to `pass` below, which is only there to + # prevent syntax problems + pass + print('quitting echo server', file=log_buffer) + + +if __name__ == '__main__': + server() + sys.exit(0) diff --git a/resources/session01/socket_tools.py b/resources/session01/socket_tools.py new file mode 100644 index 00000000..d2bc1e18 --- /dev/null +++ b/resources/session01/socket_tools.py @@ -0,0 +1,21 @@ +import socket + + +def get_constants(prefix): + return {getattr(socket, n): n for n in dir(socket) if n.startswith(prefix)} + + +families = get_constants('AF_') +types = get_constants('SOCK_') +protocols = get_constants('IPPROTO_') + + +def get_address_info(host, port): + for response in socket.getaddrinfo(host, port): + fam, typ, pro, nam, add = response + print('family: {}'.format(families[fam])) + print('type: {}'.format(types[typ])) + print('protocol: {}'.format(protocols[pro])) + print('canonical name: {}'.format(nam)) + print('socket address: {}'.format(add)) + print() diff --git a/resources/session01/tasks.txt b/resources/session01/tasks.txt new file mode 100644 index 00000000..8fdab003 --- /dev/null +++ b/resources/session01/tasks.txt @@ -0,0 +1,53 @@ +Session 4 Homework +================== + +Required Tasks: +--------------- + +* Complete the code in ``echo_server.py`` to create a server that sends back + whatever messages it receives from a client + +* Complete the code in ``echo_client.py`` to create a client function that + can send a message and receive a reply. + +* Ensure that the tests in ``tests.py`` pass. + +To run the tests: + +* Open one terminal while in this folder and execute this command: + + $ python echo_server.py + +* Open a second terminal in this same folder and execute this command: + + $ python tests.py + + + + +Optional Tasks: +--------------- + +Simple: + +* Write a python function that lists the services provided by a given range of + ports. + + * accept the lower and upper bounds as arguments + * provide sensible defaults + * Ensure that it only accepts valid port numbers (0-65535) + +Challenging: + +* The echo server as outlined will only process a connection from one client + at a time. If a second client were to attempt a connection, it would have to + wait until the first message was fully echoed before it could be dealt with. + + Python provides a module called `select` that allows waiting for I/O events + in order to control flow. The `select.select` method can be used to allow + our echo server to handle more than one incoming connection in "parallel". + + Read the documentation about the `select` module + (http://docs.python.org/3/library/select.html) and attempt to write a second + version of the echo server that can handle multiple client connections in + "parallel". You do not need to invoke threading of any kind to do this. diff --git a/resources/session01/tests.py b/resources/session01/tests.py new file mode 100644 index 00000000..d4c6c791 --- /dev/null +++ b/resources/session01/tests.py @@ -0,0 +1,46 @@ +from echo_client import client +import socket +import unittest + + +class EchoTestCase(unittest.TestCase): + """tests for the echo server and client""" + + def send_message(self, message): + """Attempt to send a message using the client + + In case of a socket error, fail and report the problem + """ + try: + reply = client(message) + except socket.error as e: + if e.errno == 61: + msg = "Error: {0}, is the server running?" + self.fail(msg.format(e.strerror)) + else: + self.fail("Unexpected Error: {0}".format(str(e))) + return reply + + def test_short_message_echo(self): + """test that a message short than 16 bytes echoes cleanly""" + expected = "short message" + actual = self.send_message(expected) + self.assertEqual( + expected, + actual, + "expected {0}, got {1}".format(expected, actual) + ) + + def test_long_message_echo(self): + """test that a message longer than 16 bytes echoes in 16-byte chunks""" + expected = "Four score and seven years ago our fathers did stuff" + actual = self.send_message(expected) + self.assertEqual( + expected, + actual, + "expected {0}, got {1}".format(expected, actual) + ) + + +if __name__ == '__main__': + unittest.main() diff --git a/resources/session02/homework/http_server.py b/resources/session02/homework/http_server.py new file mode 100644 index 00000000..84ceffe1 --- /dev/null +++ b/resources/session02/homework/http_server.py @@ -0,0 +1,86 @@ +import socket +import sys + + +def response_ok(body=b"this is a pretty minimal response", mimetype=b"text/plain"): + """returns a basic HTTP response""" + resp = [] + resp.append(b"HTTP/1.1 200 OK") + resp.append(b"Content-Type: text/plain") + resp.append(b"") + resp.append(b"this is a pretty minimal response") + return b"\r\n".join(resp) + + +def response_method_not_allowed(): + """returns a 405 Method Not Allowed response""" + resp = [] + resp.append("HTTP/1.1 405 Method Not Allowed") + resp.append("") + return "\r\n".join(resp).encode('utf8') + + +def response_not_found(): + """returns a 404 Not Found response""" + return b"" + + +def parse_request(request): + first_line = request.split("\r\n", 1)[0] + method, uri, protocol = first_line.split() + if method != "GET": + raise NotImplementedError("We only accept GET") + return uri + + +def resolve_uri(uri): + """This method should return appropriate content and a mime type""" + return b"still broken", b"text/plain" + + +def server(log_buffer=sys.stderr): + address = ('127.0.0.1', 10000) + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + print("making a server on {0}:{1}".format(*address), file=log_buffer) + sock.bind(address) + sock.listen(1) + + try: + while True: + print('waiting for a connection', file=log_buffer) + conn, addr = sock.accept() # blocking + try: + print('connection - {0}:{1}'.format(*addr), file=log_buffer) + request = '' + while True: + data = conn.recv(1024) + request += data.decode('utf8') + if len(data) < 1024: + break + + try: + uri = parse_request(request) + except NotImplementedError: + response = response_method_not_allowed() + else: + try: + content, mime_type = resolve_uri(uri) + except NameError: + response = response_not_found() + else: + response = response_ok(content, mime_type) + + print('sending response', file=log_buffer) + conn.sendall(response) + finally: + conn.close() + + except KeyboardInterrupt: + sock.close() + return + + +if __name__ == '__main__': + server() + sys.exit(0) diff --git a/resources/session02/homework/simple_client.py b/resources/session02/homework/simple_client.py new file mode 100644 index 00000000..2c9ed4cd --- /dev/null +++ b/resources/session02/homework/simple_client.py @@ -0,0 +1,44 @@ +import socket +import sys + + +def bytes_client(msg): + server_address = ('localhost', 10000) + sock = socket.socket( + socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP + ) + print( + 'connecting to {0} port {1}'.format(*server_address), + file=sys.stderr + ) + sock.connect(server_address) + response = b'' + done = False + bufsize = 1024 + try: + print('sending "{0}"'.format(msg), file=sys.stderr) + sock.sendall(msg.encode('utf8')) + while not done: + chunk = sock.recv(bufsize) + if len(chunk) < bufsize: + done = True + response += chunk + print('received "{0}"'.format(response), file=sys.stderr) + finally: + print('closing socket', file=sys.stderr) + sock.close() + return response + + +def client(msg): + return bytes_client(msg).decode('utf8') + + +if __name__ == '__main__': + if len(sys.argv) != 2: + usg = '\nusage: python echo_client.py "this is my message"\n' + print(usg, file=sys.stderr) + sys.exit(1) + + msg = sys.argv[1] + client(msg) diff --git a/resources/session02/homework/tests.py b/resources/session02/homework/tests.py new file mode 100644 index 00000000..aed245e4 --- /dev/null +++ b/resources/session02/homework/tests.py @@ -0,0 +1,427 @@ +import mimetypes +import os +import pathlib +import socket +import unittest + + +CRLF = '\r\n' +CRLF_BYTES = CRLF.encode('utf8') +KNOWN_TYPES = set( + map(lambda x: x.encode('utf8'), mimetypes.types_map.values()) +) + + +def extract_response_code(response): + return response.split(CRLF_BYTES, 1)[0].split(b' ', 1)[1].strip() + + +def extract_response_protocol(response): + return response.split(CRLF_BYTES, 1)[0].split(b' ', 1)[0].strip() + + +def extract_headers(response): + return response.split(CRLF_BYTES*2, 1)[0].split(CRLF_BYTES)[1:] + + +def extract_body(response): + return response.split(CRLF_BYTES*2, 1)[1] + + +class ResponseOkTestCase(unittest.TestCase): + """unit tests for the response_ok method in our server + + Becase this is a unit test case, it does not require the server to be + running. + """ + + def call_function_under_test(self, body=b"", mimetype=b"text/plain"): + """call the `response_ok` function from our http_server module""" + from http_server import response_ok + return response_ok(body=body, mimetype=mimetype) + + def test_response_code(self): + ok = self.call_function_under_test() + expected = "200 OK" + actual = extract_response_code(ok) + self.assertEqual(expected.encode('utf8'), actual) + + def test_response_protocol(self): + ok = self.call_function_under_test() + expected = 'HTTP/1.1' + actual = extract_response_protocol(ok) + self.assertEqual(expected.encode('utf8'), actual) + + def test_response_has_content_type_header(self): + ok = self.call_function_under_test() + headers = extract_headers(ok) + expected_name = 'content-type'.encode('utf8') + has_header = False + for header in headers: + name, value = header.split(b':') + actual_name = name.strip().lower() + if actual_name == expected_name: + has_header = True + break + self.assertTrue(has_header) + + def test_response_has_legitimate_content_type(self): + ok = self.call_function_under_test() + headers = extract_headers(ok) + expected_name = 'content-type'.encode('utf8') + for header in headers: + name, value = header.split(b':') + actual_name = name.strip().lower() + if actual_name == expected_name: + self.assertTrue(value.strip() in KNOWN_TYPES) + return + self.fail('no content type header found') + + def test_passed_mimetype_in_response(self): + mimetypes = [ + b'image/jpeg', b'text/html', b'text/x-python', + ] + header_name = b'content-type' + for expected in mimetypes: + ok = self.call_function_under_test(mimetype=expected) + headers = extract_headers(ok) + for header in headers: + name, value = header.split(b':') + if header_name == name.strip().lower(): + actual = value.strip() + self.assertEqual( + expected, + actual, + "expected {0}, got {1}".format(expected, actual) + ) + + def test_passed_body_in_response(self): + bodies = [ + b"a body", + b"a longer body\nwith two lines", + pathlib.Path("webroot/sample.txt").read_bytes(), + ] + for expected in bodies: + ok = self.call_function_under_test(body=expected) + actual = extract_body(ok) + self.assertEqual( + expected, + actual, + "expected {0}, got {1}".format(expected, actual)) + + +class ResponseMethodNotAllowedTestCase(unittest.TestCase): + """unit tests for the response_method_not_allowed function""" + + def call_function_under_test(self): + """call the `response_method_not_allowed` function""" + from http_server import response_method_not_allowed + return response_method_not_allowed() + + def test_response_code(self): + resp = self.call_function_under_test() + expected = "405 Method Not Allowed" + actual = extract_response_code(resp) + self.assertEqual(expected.encode('utf8'), actual) + + def test_response_method(self): + resp = self.call_function_under_test() + expected = 'HTTP/1.1' + actual = extract_response_protocol(resp) + self.assertEqual(expected.encode('utf8'), actual) + + +class ResponseNotFoundTestCase(unittest.TestCase): + """unit tests for the response_not_found function""" + + def call_function_under_test(self): + """call the 'response_not_found' function""" + from http_server import response_not_found + return response_not_found() + + def test_response_code(self): + resp = self.call_function_under_test() + expected = "404 Not Found" + actual = extract_response_code(resp) + self.assertEqual(expected.encode('utf8'), actual) + + def test_response_method(self): + resp = self.call_function_under_test() + expected = 'HTTP/1.1' + actual = extract_response_protocol(resp) + self.assertEqual(expected.encode('utf8'), actual) + + +class ParseRequestTestCase(unittest.TestCase): + """unit tests for the parse_request method""" + + def call_function_under_test(self, request): + """call the `parse_request` function""" + from http_server import parse_request + return parse_request(request) + + def test_get_method(self): + """verify that GET HTTP requests do not raise an error""" + request = "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n" + try: + self.call_function_under_test(request) + except (NotImplementedError, Exception) as e: + self.fail('GET method raises an error {0}'.format(str(e))) + + def test_bad_http_methods(self): + """verify that non-GET HTTP methods raise a NotImplementedError""" + methods = ['POST', 'PUT', 'DELETE', 'HEAD'] + request_template = "{0} / HTTP/1.1\r\nHost: example.com\r\n\r\n" + for method in methods: + request = request_template.format(method) + self.assertRaises( + NotImplementedError, self.call_function_under_test, request + ) + + def test_uri_returned(self): + """verify that the parse_request function returns a URI""" + URIs = [ + '/', '/a_web_page.html', '/sample.txt', '/images/sample_1.png', + ] + request_tmplt = "GET {0} HTTP/1.1" + for expected in URIs: + request = request_tmplt.format(expected) + actual = self.call_function_under_test(request) + self.assertEqual( + expected, + actual, + "expected {0}, got {1}".format(expected, actual) + ) + + +class ResolveURITestCase(unittest.TestCase): + """unit tests for the resolve_uri function""" + + def call_function_under_test(self, uri): + """call the resolve_uri function""" + from http_server import resolve_uri + content, mime_type = resolve_uri(uri) + return content, mime_type.decode('utf8') + + def test_directory_resource(self): + uri = '/' + expected_names = [ + 'a_web_page.html', 'images', 'make_time.py', 'sample.txt', + ] + expected_mimetype = "text/plain" + actual_body, actual_mimetype = self.call_function_under_test(uri) + self.assertEqual( + expected_mimetype, + actual_mimetype, + 'expected {0} got {1}'.format(expected_mimetype, actual_mimetype) + ) + actual_body = actual_body.decode('utf8') + for expected in expected_names: + self.assertTrue( + expected in actual_body, + '"{0}" not in "{1}"'.format(expected, actual_body) + ) + + def test_file_resource(self): + uris_types = { + '/a_web_page.html': 'text/html', + '/make_time.py': 'text/x-python', + '/sample.txt': 'text/plain', + } + for uri, expected_mimetype in uris_types.items(): + path = pathlib.Path("webroot{0}".format(uri)) + expected_body = path.read_bytes() + actual_body, actual_mimetype = self.call_function_under_test(uri) + self.assertEqual( + expected_mimetype, + actual_mimetype, + 'expected {0} got {1}'.format( + expected_mimetype, actual_mimetype + ) + ) + self.assertEqual( + expected_body, + actual_body, + 'expected {0} got {1}'.format( + expected_mimetype, actual_mimetype + ) + ) + + def test_image_resource(self): + names_types = { + 'JPEG_example.jpg': 'image/jpeg', + 'sample_1.png': 'image/png', + } + for filename, expected_mimetype in names_types.items(): + uri = "/images/{0}".format(filename) + path = pathlib.Path("webroot{0}".format(uri)) + expected_body = path.read_bytes() + actual_body, actual_mimetype = self.call_function_under_test(uri) + self.assertEqual( + expected_mimetype, + actual_mimetype, + 'expected {0} got {1}'.format( + expected_mimetype, actual_mimetype + ) + ) + self.assertEqual( + expected_body, + actual_body, + 'expected {0} got {1}'.format( + expected_mimetype, actual_mimetype + ) + ) + + def test_missing_resource(self): + uri = "/missing.html" + self.assertRaises(NameError, self.call_function_under_test, uri) + + +class HTTPServerFunctionalTestCase(unittest.TestCase): + """functional tests of the HTTP Server + + This test case interacts with the http server, and as such requires it to + be running in order for the tests to pass + """ + + def send_message(self, message, use_bytes=False): + """Attempt to send a message using the client and the test buffer + + In case of a socket error, fail and report the problem + """ + response = '' + if not use_bytes: + from simple_client import client + else: + from simple_client import bytes_client as client + + try: + response = client(message) + except socket.error as e: + if e.errno == 61: + msg = "Error: {0}, is the server running?" + self.fail(msg.format(e.strerror)) + else: + self.fail("Unexpected Error: {0}".format(str(e))) + return response + + def test_get_request(self): + message = CRLF.join(['GET / HTTP/1.1', 'Host: example.com', '']) + expected = '200 OK' + actual = self.send_message(message) + self.assertTrue( + expected in actual, '"{0}" not in "{1}"'.format(expected, actual) + ) + + def test_post_request(self): + message = CRLF.join(['POST / HTTP/1.1', 'Host: example.com', '']) + expected = '405 Method Not Allowed' + actual = self.send_message(message) + self.assertTrue( + expected in actual, '"{0}" not in "{1}"'.format(expected, actual) + ) + + def test_webroot_directory_resources(self): + """verify that directory uris are properly served""" + message_tmpl = CRLF.join(['GET {0} HTTP/1.1', 'Host: example.com', '']) + root = "webroot" + for directory, directories, files in os.walk(root): + directory_uri = "/{0}".format(directory[len(root):]) + message = message_tmpl.format(directory_uri) + actual = self.send_message(message) + # verify that directory listings are correct + self.assertTrue( + "200 OK" in actual, + "request for {0} did not result in OK".format(directory_uri)) + for expected in directories + files: + self.assertTrue( + expected in actual, + '"{0}" not in "{1}"'.format(expected, actual) + ) + + def test_webroot_file_uris(self): + """verify that file uris are properly served""" + message_tmpl = CRLF.join(['GET {0} HTTP/1.1', 'Host: example.com', '']) + root = pathlib.Path("webroot") + for file_path in root.iterdir(): + # set up expectations for this file + if file_path.is_dir(): + continue + expected_body = file_path.read_bytes().decode('utf8') + expected_mimetype = mimetypes.types_map[ + os.path.splitext(str(file_path))[1] + ] + file_uri = str(file_path)[len(str(root)):] + message = message_tmpl.format(file_uri) + actual = self.send_message(message) + self.assertTrue( + "200 OK" in actual, + "request for {0} did not result in OK".format( + file_uri + ) + ) + self.assertTrue( + expected_mimetype in actual, + "mimetype {0} not in response for {1}".format( + expected_mimetype, file_uri + ) + ) + self.assertTrue( + expected_body in actual, + "body of {0} not in response for {1}".format( + file_path, file_uri + ) + ) + + def test_webroot_image_uris(self): + """verify that image uris are properly served + + requires using a client that does not attempt to decode the response + body + """ + message_tmpl = CRLF.join(['GET {0} HTTP/1.1', 'Host: example.com', '']) + root = pathlib.Path("webroot") + images_path = root / 'images' + for file_path in images_path.iterdir(): + # set up expectations for this file + if file_path.is_dir(): + continue + expected_body = file_path.read_bytes() + expected_mimetype = mimetypes.types_map[ + os.path.splitext(str(file_path))[1] + ] + file_uri = str(file_path)[len(str(root)):] + message = message_tmpl.format(file_uri) + actual = self.send_message(message, use_bytes=True) + self.assertTrue( + b"200 OK" in actual, + "request for {0} did not result in OK".format( + file_uri + ) + ) + self.assertTrue( + expected_mimetype.encode('utf8') in actual, + "mimetype {0} not in response for {1}".format( + expected_mimetype, file_uri + ) + ) + self.assertTrue( + expected_body in actual, + "body of {0} not in response for {1}".format( + file_path, file_uri + ) + ) + + def test_missing_resource(self): + message = CRLF.join( + ['GET /missing.html HTTP/1.1', 'Host: example.com', ''] + ) + expected = '404 Not Found' + actual = self.send_message(message) + self.assertTrue( + expected in actual, '"{0}" not in "{1}"'.format(expected, actual) + ) + + +if __name__ == '__main__': + unittest.main() diff --git a/resources/session02/homework/webroot/a_web_page.html b/resources/session02/homework/webroot/a_web_page.html new file mode 100644 index 00000000..4635692d --- /dev/null +++ b/resources/session02/homework/webroot/a_web_page.html @@ -0,0 +1,11 @@ + + + + +

North Carolina

+ +

A fine place to spend a week learning web programming!

+ + + + diff --git a/resources/session02/homework/webroot/images/JPEG_example.jpg b/resources/session02/homework/webroot/images/JPEG_example.jpg new file mode 100644 index 00000000..13506f01 Binary files /dev/null and b/resources/session02/homework/webroot/images/JPEG_example.jpg differ diff --git a/resources/session02/homework/webroot/images/Sample_Scene_Balls.jpg b/resources/session02/homework/webroot/images/Sample_Scene_Balls.jpg new file mode 100644 index 00000000..1c0ccade Binary files /dev/null and b/resources/session02/homework/webroot/images/Sample_Scene_Balls.jpg differ diff --git a/resources/session02/homework/webroot/images/sample_1.png b/resources/session02/homework/webroot/images/sample_1.png new file mode 100644 index 00000000..5b2f52df Binary files /dev/null and b/resources/session02/homework/webroot/images/sample_1.png differ diff --git a/resources/session02/homework/webroot/make_time.py b/resources/session02/homework/webroot/make_time.py new file mode 100644 index 00000000..b69acf38 --- /dev/null +++ b/resources/session02/homework/webroot/make_time.py @@ -0,0 +1,22 @@ +#!/usr/bin/env python + +""" +make_time.py + +simple script that returns and HTML page with the current time +""" + +import datetime + +time_str = datetime.datetime.now().isoformat() + +html = """ + + +

The time is:

+

%s

+ + +""" % time_str + +print(html) diff --git a/resources/session02/homework/webroot/sample.txt b/resources/session02/homework/webroot/sample.txt new file mode 100644 index 00000000..1965c7d3 --- /dev/null +++ b/resources/session02/homework/webroot/sample.txt @@ -0,0 +1,3 @@ +This is a very simple text file. +Just to show that we can server it up. +It is three lines long. diff --git a/resources/session02/http_server.py b/resources/session02/http_server.py new file mode 100644 index 00000000..d5aaf480 --- /dev/null +++ b/resources/session02/http_server.py @@ -0,0 +1,39 @@ +import socket +import sys + + +def server(log_buffer=sys.stderr): + address = ('127.0.0.1', 10000) + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + print("making a server on {0}:{1}".format(*address), file=log_buffer) + sock.bind(address) + sock.listen(1) + + try: + while True: + print('waiting for a connection', file=log_buffer) + conn, addr = sock.accept() # blocking + try: + print('connection - {0}:{1}'.format(*addr), file=log_buffer) + while True: + data = conn.recv(16) + print('received "{0}"'.format(data), file=log_buffer) + if data: + print('sending data back to client', file=log_buffer) + conn.sendall(data) + else: + msg = 'no more data from {0}:{1}'.format(*addr) + print(msg, log_buffer) + break + finally: + conn.close() + + except KeyboardInterrupt: + sock.close() + return + + +if __name__ == '__main__': + server() + sys.exit(0) diff --git a/resources/session02/ljshell.py b/resources/session02/ljshell.py deleted file mode 100644 index 40a8b080..00000000 --- a/resources/session02/ljshell.py +++ /dev/null @@ -1,9 +0,0 @@ -from pyramid.paster import get_appsettings, setup_logging -from sqlalchemy import engine_from_config -from sqlalchemy.orm import sessionmaker - -config_uri = 'development.ini' -setup_logging(config_uri) -settings = get_appsettings(config_uri) -engine = engine_from_config(settings, 'sqlalchemy.') -Session = sessionmaker(bind=engine) diff --git a/resources/session02/simple_client.py b/resources/session02/simple_client.py new file mode 100644 index 00000000..74523a2a --- /dev/null +++ b/resources/session02/simple_client.py @@ -0,0 +1,40 @@ +import socket +import sys + + +def client(msg): + server_address = ('localhost', 10000) + sock = socket.socket( + socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP + ) + print( + 'connecting to {0} port {1}'.format(*server_address), + file=sys.stderr + ) + sock.connect(server_address) + response = '' + done = False + bufsize = 1024 + try: + print('sending "{0}"'.format(msg), file=sys.stderr) + sock.sendall(msg.encode('utf8')) + while not done: + chunk = sock.recv(bufsize) + if len(chunk) < bufsize: + done = True + response += chunk.decode('utf8') + print('received "{0}"'.format(response), file=sys.stderr) + finally: + print('closing socket', file=sys.stderr) + sock.close() + return response + + +if __name__ == '__main__': + if len(sys.argv) != 2: + usg = '\nusage: python echo_client.py "this is my message"\n' + print(usg, file=sys.stderr) + sys.exit(1) + + msg = sys.argv[1] + client(msg) diff --git a/resources/session02/tests.py b/resources/session02/tests.py new file mode 100644 index 00000000..a4da1793 --- /dev/null +++ b/resources/session02/tests.py @@ -0,0 +1,165 @@ +import mimetypes +import socket +import unittest + + +CRLF = '\r\n' +CRLF_BYTES = CRLF.encode('utf8') +KNOWN_TYPES = set( + map(lambda x: x.encode('utf8'), mimetypes.types_map.values()) +) + + +def extract_response_code(response): + return response.split(CRLF_BYTES, 1)[0].split(b' ', 1)[1].strip() + + +def extract_response_protocol(response): + return response.split(CRLF_BYTES, 1)[0].split(b' ', 1)[0].strip() + + +def extract_headers(response): + return response.split(CRLF_BYTES*2, 1)[0].split(CRLF_BYTES)[1:] + + +class ResponseOkTestCase(unittest.TestCase): + """unit tests for the response_ok method in our server + + Becase this is a unit test case, it does not require the server to be + running. + """ + + def call_function_under_test(self): + """call the `response_ok` function from our http_server module""" + from http_server import response_ok + return response_ok() + + def test_response_code(self): + ok = self.call_function_under_test() + expected = "200 OK" + actual = extract_response_code(ok) + self.assertEqual(expected.encode('utf8'), actual) + + def test_response_protocol(self): + ok = self.call_function_under_test() + expected = 'HTTP/1.1' + actual = extract_response_protocol(ok) + self.assertEqual(expected.encode('utf8'), actual) + + def test_response_has_content_type_header(self): + ok = self.call_function_under_test() + headers = extract_headers(ok) + expected_name = 'content-type'.encode('utf8') + has_header = False + for header in headers: + name, value = header.split(b':') + actual_name = name.strip().lower() + if actual_name == expected_name: + has_header = True + break + self.assertTrue(has_header) + + def test_response_has_legitimate_content_type(self): + ok = self.call_function_under_test() + headers = extract_headers(ok) + expected_name = 'content-type'.encode('utf8') + for header in headers: + name, value = header.split(b':') + actual_name = name.strip().lower() + if actual_name == expected_name: + self.assertTrue(value.strip() in KNOWN_TYPES) + return + self.fail('no content type header found') + + +class ResponseMethodNotAllowedTestCase(unittest.TestCase): + """unit tests for the response_method_not_allowed function""" + + def call_function_under_test(self): + """call the `response_method_not_allowed` function""" + from http_server import response_method_not_allowed + return response_method_not_allowed() + + def test_response_code(self): + resp = self.call_function_under_test() + expected = "405 Method Not Allowed" + actual = extract_response_code(resp) + self.assertEqual(expected.encode('utf8'), actual) + + def test_response_method(self): + resp = self.call_function_under_test() + expected = 'HTTP/1.1' + actual = extract_response_protocol(resp) + self.assertEqual(expected.encode('utf8'), actual) + + +class ParseRequestTestCase(unittest.TestCase): + """unit tests for the parse_request method""" + + def call_function_under_test(self, request): + """call the `parse_request` function""" + from http_server import parse_request + return parse_request(request) + + def test_get_method(self): + """verify that GET HTTP requests do not raise an error""" + request = "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n" + try: + self.call_function_under_test(request) + except (NotImplementedError, Exception) as e: + self.fail('GET method raises an error {0}'.format(str(e))) + + def test_bad_http_methods(self): + """verify that non-GET HTTP methods raise a NotImplementedError""" + methods = ['POST', 'PUT', 'DELETE', 'HEAD'] + request_template = "{0} / HTTP/1.1\r\nHost: example.com\r\n\r\n" + for method in methods: + request = request_template.format(method) + self.assertRaises( + NotImplementedError, self.call_function_under_test, request + ) + + +class HTTPServerFunctionalTestCase(unittest.TestCase): + """functional tests of the HTTP Server + + This test case interacts with the http server, and as such requires it to + be running in order for the tests to pass + """ + + def send_message(self, message): + """Attempt to send a message using the client and the test buffer + + In case of a socket error, fail and report the problem + """ + from simple_client import client + response = '' + try: + response = client(message) + except socket.error as e: + if e.errno == 61: + msg = "Error: {0}, is the server running?" + self.fail(msg.format(e.strerror)) + else: + self.fail("Unexpected Error: {0}".format(str(e))) + return response + + def test_get_request(self): + message = CRLF.join(['GET / HTTP/1.1', 'Host: example.com', '']) + expected = '200 OK' + actual = self.send_message(message) + self.assertTrue( + expected in actual, '"{0}" not in "{1}"'.format(expected, actual) + ) + + def test_post_request(self): + message = CRLF.join(['POST / HTTP/1.1', 'Host: example.com', '']) + expected = '405 Method Not Allowed' + actual = self.send_message(message) + self.assertTrue( + expected in actual, '"{0}" not in "{1}"'.format(expected, actual) + ) + + +if __name__ == '__main__': + unittest.main() diff --git a/resources/session03/cgi/cgi-bin/cgi_1.py b/resources/session03/cgi/cgi-bin/cgi_1.py new file mode 100755 index 00000000..baa5c3e9 --- /dev/null +++ b/resources/session03/cgi/cgi-bin/cgi_1.py @@ -0,0 +1,5 @@ +#!/usr/bin/env python +import cgi + + +cgi.test() diff --git a/resources/session03/cgi/cgi-bin/cgi_2.py b/resources/session03/cgi/cgi-bin/cgi_2.py new file mode 100755 index 00000000..100ccded --- /dev/null +++ b/resources/session03/cgi/cgi-bin/cgi_2.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python +import cgi +import cgitb +cgitb.enable() +import os +import datetime + + +default = "No Value Present" + + +print("Content-Type: text/html") +print() + +body = """ + +Lab 1 - CGI experiments + + +

Hey there, this page has been generated by {software}, running {script}

+

Today is {month} {date}, {year}.

+

This page was requested by IP Address {client_ip}

+ +""".format( + software=os.environ.get('SERVER_SOFTWARE', default), + script='aaaa', + month='bbbb', + date='cccc', + year='dddd', + client_ip='eeee' +) +print(body) diff --git a/resources/session03/cgi/cgi-bin/cgi_sums.py b/resources/session03/cgi/cgi-bin/cgi_sums.py new file mode 100755 index 00000000..feed2bc8 --- /dev/null +++ b/resources/session03/cgi/cgi-bin/cgi_sums.py @@ -0,0 +1,9 @@ +#!/usr/bin/env python +import cgi +import cgitb + +cgitb.enable() + +print("Content-type: text/plain") +print() +print("Your job is to make this work") diff --git a/resources/session03/cgi/index.html b/resources/session03/cgi/index.html new file mode 100644 index 00000000..d6725d8b --- /dev/null +++ b/resources/session03/cgi/index.html @@ -0,0 +1,17 @@ + + + + Python 200: Session 06 Lab Examples + + +

Python 200

+

Session 06: CGI, WSGI and Living Online

+

CGI Examples

+
    +
  1. CGI Test 1
  2. +
  3. Exercise One
  4. +
  5. CGI Sum Server
  6. +
+

WSGI Examples

+ + diff --git a/resources/session03/http_server.py b/resources/session03/http_server.py new file mode 100644 index 00000000..e42ea086 --- /dev/null +++ b/resources/session03/http_server.py @@ -0,0 +1,112 @@ +import mimetypes +import pathlib +import socket +import sys + + +def response_ok(body=b"this is a pretty minimal response", mimetype=b"text/plain"): + """returns a basic HTTP response as bytes""" + resp = [] + resp.append(b"HTTP/1.1 200 OK") + resp.append(b"Content-Type: " + mimetype) + resp.append(b"") + resp.append(body) + return b"\r\n".join(resp) + + +def response_method_not_allowed(): + """returns a 405 Method Not Allowed response as bytes""" + resp = [] + resp.append("HTTP/1.1 405 Method Not Allowed") + resp.append("") + return "\r\n".join(resp).encode('utf8') + + +def response_not_found(): + """returns a 404 Not Found response as bytes""" + resp = [] + resp.append("HTTP/1.1 404 Not Found") + resp.append("") + return "\r\n".join(resp).encode('utf8') + + +def parse_request(request): + first_line = request.split("\r\n", 1)[0] + method, uri, protocol = first_line.split() + if method != "GET": + raise NotImplementedError("We only accept GET") + return uri + + +def resolve_uri(uri): + """This method should return appropriate content and a mime type + + Both content and mime type should be expressed as bytes + """ + root_path = pathlib.Path('./webroot') + resource_path = root_path / uri.lstrip('/') + if resource_path.is_dir(): + # the resource is a directory, content type is text/html, produce a + # listing of the directory contents + item_template = '* {}' + listing = [] + for item_path in resource_path.iterdir(): + listing.append(item_template.format(str(item_path))) + content = "\n".join(listing).encode('utf8') + mime_type = 'text/plain'.encode('utf8') + elif resource_path.is_file(): + # the resource is a file, figure out its mime type and read + content = resource_path.read_bytes() + mime_type = mimetypes.guess_type(str(resource_path))[0].encode('utf8') + else: + raise NameError() + return content, mime_type + + + +def server(log_buffer=sys.stderr): + address = ('127.0.0.1', 10000) + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + print("making a server on {0}:{1}".format(*address), file=log_buffer) + sock.bind(address) + sock.listen(1) + + try: + while True: + print('waiting for a connection', file=log_buffer) + conn, addr = sock.accept() # blocking + try: + print('connection - {0}:{1}'.format(*addr), file=log_buffer) + request = '' + while True: + data = conn.recv(1024) + request += data.decode('utf8') + if len(data) < 1024: + break + + try: + uri = parse_request(request) + except NotImplementedError: + response = response_method_not_allowed() + else: + try: + content, mime_type = resolve_uri(uri) + except NameError: + response = response_not_found() + else: + response = response_ok(content, mime_type) + + print('sending response', file=log_buffer) + conn.sendall(response) + finally: + conn.close() + + except KeyboardInterrupt: + sock.close() + return + + +if __name__ == '__main__': + server() + sys.exit(0) diff --git a/resources/session03/wsgi/bookapp.py b/resources/session03/wsgi/bookapp.py new file mode 100644 index 00000000..d2284c6f --- /dev/null +++ b/resources/session03/wsgi/bookapp.py @@ -0,0 +1,26 @@ +import re + +from bookdb import BookDB + +DB = BookDB() + + +def book(book_id): + return "

a book with id %s

" % book_id + + +def books(): + return "

a list of books

" + + +def application(environ, start_response): + status = "200 OK" + headers = [('Content-type', 'text/html')] + start_response(status, headers) + return ["

No Progress Yet

".encode('utf8')] + + +if __name__ == '__main__': + from wsgiref.simple_server import make_server + srv = make_server('localhost', 8080, application) + srv.serve_forever() diff --git a/resources/session03/wsgi/bookdb.py b/resources/session03/wsgi/bookdb.py new file mode 100644 index 00000000..f3a72414 --- /dev/null +++ b/resources/session03/wsgi/bookdb.py @@ -0,0 +1,45 @@ + +class BookDB(): + def titles(self): + titles = [ + dict(id=id, title=database[id]['title']) for id in database.keys() + ] + return titles + + def title_info(self, id): + return database.get(id, None) + + +# let's pretend we're getting this information from a database somewhere +database = { + 'id1': { + 'title': 'CherryPy Essentials: Rapid Python Web Application Development', + 'isbn': '978-1904811848', + 'publisher': 'Packt Publishing (March 31, 2007)', + 'author': 'Sylvain Hellegouarch', + }, + 'id2': { + 'title': 'Python for Software Design: How to Think Like a Computer Scientist', + 'isbn': '978-0521725965', + 'publisher': 'Cambridge University Press; 1 edition (March 16, 2009)', + 'author': 'Allen B. Downey', + }, + 'id3': { + 'title': 'Foundations of Python Network Programming', + 'isbn': '978-1430230038', + 'publisher': 'Apress; 2 edition (December 21, 2010)', + 'author': 'John Goerzen', + }, + 'id4': { + 'title': 'Python Cookbook, Second Edition', + 'isbn': '978-0-596-00797-3', + 'publisher': 'O''Reilly Media', + 'author': 'Alex Martelli, Anna Ravenscroft, David Ascher', + }, + 'id5': { + 'title': 'The Pragmatic Programmer: From Journeyman to Master', + 'isbn': '978-0201616224', + 'publisher': 'Addison-Wesley Professional (October 30, 1999)', + 'author': 'Andrew Hunt, David Thomas', + }, +} diff --git a/resources/session03/wsgi/tests.py b/resources/session03/wsgi/tests.py new file mode 100644 index 00000000..b92bc78d --- /dev/null +++ b/resources/session03/wsgi/tests.py @@ -0,0 +1,128 @@ +import unittest + + +class BookAppTestCase(unittest.TestCase): + """shared functionality""" + + def setUp(self): + from bookdb import database + self.db = database + + +class BookDBTestCase(BookAppTestCase): + """tests for the bookdb code""" + + def makeOne(self): + from bookdb import BookDB + return BookDB() + + def test_all_titles_returned(self): + actual_titles = self.makeOne().titles() + self.assertEqual(len(actual_titles), len(self.db)) + + def test_all_titles_correct(self): + actual_titles = self.makeOne().titles() + for actual_title in actual_titles: + self.assertTrue(actual_title['id'] in self.db) + actual = actual_title['title'] + expected = self.db[actual_title['id']]['title'] + self.assertEqual(actual, expected) + + def test_title_info_complete(self): + use_id, expected = self.db.items()[0] + actual = self.makeOne().title_info(use_id) + # demonstrate all actual keys are expected + for key in actual: + self.assertTrue(key in expected) + # demonstrate all expected keys are present in actual + for key in expected: + self.assertTrue(key in actual) + + def test_title_info_correct(self): + for book_id, expected in self.db.items(): + actual = self.makeOne().title_info(book_id) + self.assertEqual(actual, expected) + + +class ResolvePathTestCase(BookAppTestCase): + """tests for the resolve_path function""" + + def call_function_under_test(self, path): + from bookapp import resolve_path + return resolve_path(path) + + def test_root_returns_books_function(self): + """verify that the correct function is returned by the root path""" + from bookapp import books as expected + path = '/' + actual, args = self.call_function_under_test(path) + self.assertTrue(actual is expected) + + def test_root_returns_no_args(self): + """verify that no args are returned for the root path""" + path = '/' + func, actual = self.call_function_under_test(path) + self.assertTrue(not actual) + + def test_book_path_returns_book_function(self): + from bookapp import book as expected + book_id = self.db.keys()[0] + path = '/book/{0}'.format(book_id) + actual, args = self.call_function_under_test(path) + self.assertTrue(actual is expected) + + def test_book_path_returns_bookid_in_args(self): + expected = self.db.keys()[0] + path = '/book/{0}'.format(expected) + func, actual = self.call_function_under_test(path) + self.assertTrue(expected in actual) + + def test_bad_path_raises_name_error(self): + path = '/not/valid/path' + self.assertRaises(NameError, self.call_function_under_test, path) + + +class BooksTestCase(BookAppTestCase): + """tests for the books function""" + + def call_function_under_test(self): + from bookapp import books + return books() + + def test_all_book_titles_in_result(self): + actual = self.call_function_under_test() + for book_id, info in self.db.items(): + expected = info['title'] + self.assertTrue(expected in actual) + + def test_all_book_ids_in_result(self): + actual = self.call_function_under_test() + for expected in self.db: + self.assertTrue(expected in actual) + + +class BookTestCase(BookAppTestCase): + """tests for the book function""" + + def call_function_under_test(self, id): + from bookapp import book + return book(id) + + def test_all_ids_have_results(self): + for book_id in self.db: + actual = self.call_function_under_test(book_id) + self.assertTrue(actual) + + def test_id_returns_correct_results(self): + for book_id, book_info in self.db.items(): + actual = self.call_function_under_test(book_id) + for expected in book_info.values(): + self.assertTrue(expected in actual) + + def test_bad_id_raises_name_error(self): + bad_id = "sponge" + self.assertRaises(NameError, self.call_function_under_test, bad_id) + + +if __name__ == '__main__': + unittest.main() diff --git a/resources/session03/wsgi/wsgi_1.py b/resources/session03/wsgi/wsgi_1.py new file mode 100644 index 00000000..85498d13 --- /dev/null +++ b/resources/session03/wsgi/wsgi_1.py @@ -0,0 +1,42 @@ +#!/usr/bin/env python +import datetime + +default = "No Value Set" + +body = """ + +Lab 3 - WSGI experiments + + +

Hey there, this page has been generated by {software}, running at {path}

+

Today is {month} {date}, {year}.

+

This page was requested by IP Address {client_ip}

+ +""" + + +def application(environ, start_response): + import pprint + pprint.pprint(environ) + + response_body = body.format( + software=environ.get('SERVER_SOFTWARE', default), + path="aaaa", + month="bbbb", + date="cccc", + year="dddd", + client_ip="eeee" + ) + status = '200 OK' + + response_headers = [('Content-Type', 'text/html'), + ('Content-Length', str(len(response_body)))] + start_response(status, response_headers) + + return [response_body.encode('utf8')] + + +if __name__ == '__main__': + from wsgiref.simple_server import make_server + srv = make_server('localhost', 8080, application) + srv.serve_forever() diff --git a/resources/session04/mashup_1.py b/resources/session04/mashup_1.py new file mode 100644 index 00000000..0880e87c --- /dev/null +++ b/resources/session04/mashup_1.py @@ -0,0 +1,51 @@ +from bs4 import BeautifulSoup +import requests + + +INSPECTION_DOMAIN = 'http://info.kingcounty.gov' +INSPECTION_PATH = '/health/ehs/foodsafety/inspections/Results.aspx' +INSPECTION_PARAMS = { + 'Output': 'W', + 'Business_Name': '', + 'Business_Address': '', + 'Longitude': '', + 'Latitude': '', + 'City': '', + 'Zip_Code': '', + 'Inspection_Type': 'All', + 'Inspection_Start': '', + 'Inspection_End': '', + 'Inspection_Closed_Business': 'A', + 'Violation_Points': '', + 'Violation_Red_Points': '', + 'Violation_Descr': '', + 'Fuzzy_Search': 'N', + 'Sort': 'H' +} + + +def get_inspection_page(**kwargs): + url = INSPECTION_DOMAIN + INSPECTION_PATH + params = INSPECTION_PARAMS.copy() + for key, val in kwargs.items(): + if key in INSPECTION_PARAMS: + params[key] = val + resp = requests.get(url, params=params) + resp.raise_for_status() + return resp.text + + +def parse_source(html): + parsed = BeautifulSoup(html) + return parsed + + +if __name__ == '__main__': + use_params = { + 'Inspection_Start': '2/1/2014', + 'Inspection_End': '2/1/2016', + 'Zip_Code': '98101' + } + html = get_inspection_page(**use_params) + parsed = parse_source(html) + print(parsed.prettify()) diff --git a/resources/session04/mashup_2.py b/resources/session04/mashup_2.py new file mode 100644 index 00000000..6d4778c9 --- /dev/null +++ b/resources/session04/mashup_2.py @@ -0,0 +1,66 @@ +from bs4 import BeautifulSoup +import pathlib +import re +import requests + + +INSPECTION_DOMAIN = 'http://info.kingcounty.gov' +INSPECTION_PATH = '/health/ehs/foodsafety/inspections/Results.aspx' +INSPECTION_PARAMS = { + 'Output': 'W', + 'Business_Name': '', + 'Business_Address': '', + 'Longitude': '', + 'Latitude': '', + 'City': '', + 'Zip_Code': '', + 'Inspection_Type': 'All', + 'Inspection_Start': '', + 'Inspection_End': '', + 'Inspection_Closed_Business': 'A', + 'Violation_Points': '', + 'Violation_Red_Points': '', + 'Violation_Descr': '', + 'Fuzzy_Search': 'N', + 'Sort': 'H' +} + + +def get_inspection_page(**kwargs): + url = INSPECTION_DOMAIN + INSPECTION_PATH + params = INSPECTION_PARAMS.copy() + for key, val in kwargs.items(): + if key in INSPECTION_PARAMS: + params[key] = val + resp = requests.get(url, params=params) + resp.raise_for_status() + return resp.text + + +def parse_source(html): + parsed = BeautifulSoup(html) + return parsed + + +def load_inspection_page(name): + file_path = pathlib.Path(name) + return file_path.read_text(encoding='utf8') + + +def restaurant_data_generator(html): + id_finder = re.compile(r'PR[\d]+~') + return html.find_all('div', id=id_finder) + + +if __name__ == '__main__': + use_params = { + 'Inspection_Start': '2/1/2013', + 'Inspection_End': '2/1/2015', + 'Zip_Code': '98101' + } + # html = get_inspection_page(**use_params) + html = load_inspection_page('inspection_page.html') + parsed = parse_source(html) + content_col = parsed.find("td", id="contentcol") + data_list = restaurant_data_generator(content_col) + print(data_list[0].prettify()) diff --git a/resources/session04/mashup_3.py b/resources/session04/mashup_3.py new file mode 100644 index 00000000..79e46632 --- /dev/null +++ b/resources/session04/mashup_3.py @@ -0,0 +1,93 @@ +from bs4 import BeautifulSoup +import pathlib +import re +import requests + + +INSPECTION_DOMAIN = 'http://info.kingcounty.gov' +INSPECTION_PATH = '/health/ehs/foodsafety/inspections/Results.aspx' +INSPECTION_PARAMS = { + 'Output': 'W', + 'Business_Name': '', + 'Business_Address': '', + 'Longitude': '', + 'Latitude': '', + 'City': '', + 'Zip_Code': '', + 'Inspection_Type': 'All', + 'Inspection_Start': '', + 'Inspection_End': '', + 'Inspection_Closed_Business': 'A', + 'Violation_Points': '', + 'Violation_Red_Points': '', + 'Violation_Descr': '', + 'Fuzzy_Search': 'N', + 'Sort': 'H' +} + + +def get_inspection_page(**kwargs): + url = INSPECTION_DOMAIN + INSPECTION_PATH + params = INSPECTION_PARAMS.copy() + for key, val in kwargs.items(): + if key in INSPECTION_PARAMS: + params[key] = val + resp = requests.get(url, params=params) + resp.raise_for_status() + return resp.text + + +def parse_source(html): + parsed = BeautifulSoup(html) + return parsed + + +def load_inspection_page(name): + file_path = pathlib.Path(name) + return file_path.read_text(encoding='utf8') + + +def restaurant_data_generator(html): + id_finder = re.compile(r'PR[\d]+~') + return html.find_all('div', id=id_finder) + + +def has_two_tds(elem): + is_tr = elem.name == 'tr' + td_children = elem.find_all('td', recursive=False) + has_two = len(td_children) == 2 + return is_tr and has_two + + +def clean_data(td): + return td.text.strip(" \n:-") + + +def extract_restaurant_metadata(elem): + restaurant_data_rows = elem.find('tbody').find_all( + has_two_tds, recursive=False + ) + rdata = {} + current_label = '' + for data_row in restaurant_data_rows: + key_cell, val_cell = data_row.find_all('td', recursive=False) + new_label = clean_data(key_cell) + current_label = new_label if new_label else current_label + rdata.setdefault(current_label, []).append(clean_data(val_cell)) + return rdata + + +if __name__ == '__main__': + use_params = { + 'Inspection_Start': '2/1/2013', + 'Inspection_End': '2/1/2015', + 'Zip_Code': '98101' + } + # html = get_inspection_page(**use_params) + html = load_inspection_page('inspection_page.html') + parsed = parse_source(html) + content_col = parsed.find("td", id="contentcol") + data_list = restaurant_data_generator(content_col) + for data_div in data_list: + metadata = extract_restaurant_metadata(data_div) + print(metadata) diff --git a/resources/session04/mashup_4.py b/resources/session04/mashup_4.py new file mode 100644 index 00000000..ab5ebdd5 --- /dev/null +++ b/resources/session04/mashup_4.py @@ -0,0 +1,133 @@ +from bs4 import BeautifulSoup +import pathlib +import re +import requests + + +INSPECTION_DOMAIN = 'http://info.kingcounty.gov' +INSPECTION_PATH = '/health/ehs/foodsafety/inspections/Results.aspx' +INSPECTION_PARAMS = { + 'Output': 'W', + 'Business_Name': '', + 'Business_Address': '', + 'Longitude': '', + 'Latitude': '', + 'City': '', + 'Zip_Code': '', + 'Inspection_Type': 'All', + 'Inspection_Start': '', + 'Inspection_End': '', + 'Inspection_Closed_Business': 'A', + 'Violation_Points': '', + 'Violation_Red_Points': '', + 'Violation_Descr': '', + 'Fuzzy_Search': 'N', + 'Sort': 'H' +} + + +def get_inspection_page(**kwargs): + url = INSPECTION_DOMAIN + INSPECTION_PATH + params = INSPECTION_PARAMS.copy() + for key, val in kwargs.items(): + if key in INSPECTION_PARAMS: + params[key] = val + resp = requests.get(url, params=params) + resp.raise_for_status() + return resp.text + + +def parse_source(html): + parsed = BeautifulSoup(html) + return parsed + + +def load_inspection_page(name): + file_path = pathlib.Path(name) + return file_path.read_text(encoding='utf8') + + +def restaurant_data_generator(html): + id_finder = re.compile(r'PR[\d]+~') + return html.find_all('div', id=id_finder) + + +def has_two_tds(elem): + is_tr = elem.name == 'tr' + td_children = elem.find_all('td', recursive=False) + has_two = len(td_children) == 2 + return is_tr and has_two + + +def clean_data(td): + return td.text.strip(" \n:-") + + +def extract_restaurant_metadata(elem): + restaurant_data_rows = elem.find('tbody').find_all( + has_two_tds, recursive=False + ) + rdata = {} + current_label = '' + for data_row in restaurant_data_rows: + key_cell, val_cell = data_row.find_all('td', recursive=False) + new_label = clean_data(key_cell) + current_label = new_label if new_label else current_label + rdata.setdefault(current_label, []).append(clean_data(val_cell)) + return rdata + + +def is_inspection_data_row(elem): + is_tr = elem.name == 'tr' + if not is_tr: + return False + td_children = elem.find_all('td', recursive=False) + has_four = len(td_children) == 4 + this_text = clean_data(td_children[0]).lower() + contains_word = 'inspection' in this_text + does_not_start = not this_text.startswith('inspection') + return is_tr and has_four and contains_word and does_not_start + + +def get_score_data(elem): + inspection_rows = elem.find_all(is_inspection_data_row) + samples = len(inspection_rows) + total = 0 + high_score = 0 + average = 0 + for row in inspection_rows: + strval = clean_data(row.find_all('td')[2]) + try: + intval = int(strval) + except (ValueError, TypeError): + samples -= 1 + else: + total += intval + high_score = intval if intval > high_score else high_score + + if samples: + average = total/float(samples) + data = { + 'Average Score': average, + 'High Score': high_score, + 'Total Inspections': samples + } + return data + + +if __name__ == '__main__': + use_params = { + 'Inspection_Start': '2/1/2013', + 'Inspection_End': '2/1/2015', + 'Zip_Code': '98101' + } + # html = get_inspection_page(**use_params) + html = load_inspection_page('inspection_page.html') + parsed = parse_source(html) + content_col = parsed.find("td", id="contentcol") + data_list = restaurant_data_generator(content_col) + for data_div in data_list: + metadata = extract_restaurant_metadata(data_div) + inspection_data = get_score_data(data_div) + metadata.update(inspection_data) + print(metadata) diff --git a/resources/session04/mashup_5.py b/resources/session04/mashup_5.py new file mode 100644 index 00000000..20f62b49 --- /dev/null +++ b/resources/session04/mashup_5.py @@ -0,0 +1,164 @@ +from bs4 import BeautifulSoup +import geocoder +import json +import pathlib +import re +import requests + + +INSPECTION_DOMAIN = 'http://info.kingcounty.gov' +INSPECTION_PATH = '/health/ehs/foodsafety/inspections/Results.aspx' +INSPECTION_PARAMS = { + 'Output': 'W', + 'Business_Name': '', + 'Business_Address': '', + 'Longitude': '', + 'Latitude': '', + 'City': '', + 'Zip_Code': '', + 'Inspection_Type': 'All', + 'Inspection_Start': '', + 'Inspection_End': '', + 'Inspection_Closed_Business': 'A', + 'Violation_Points': '', + 'Violation_Red_Points': '', + 'Violation_Descr': '', + 'Fuzzy_Search': 'N', + 'Sort': 'H' +} + + +def get_inspection_page(**kwargs): + url = INSPECTION_DOMAIN + INSPECTION_PATH + params = INSPECTION_PARAMS.copy() + for key, val in kwargs.items(): + if key in INSPECTION_PARAMS: + params[key] = val + resp = requests.get(url, params=params) + resp.raise_for_status() + return resp.text + + +def parse_source(html): + parsed = BeautifulSoup(html) + return parsed + + +def load_inspection_page(name): + file_path = pathlib.Path(name) + return file_path.read_text(encoding='utf8') + + +def restaurant_data_generator(html): + id_finder = re.compile(r'PR[\d]+~') + return html.find_all('div', id=id_finder) + + +def has_two_tds(elem): + is_tr = elem.name == 'tr' + td_children = elem.find_all('td', recursive=False) + has_two = len(td_children) == 2 + return is_tr and has_two + + +def clean_data(td): + return td.text.strip(" \n:-") + + +def extract_restaurant_metadata(elem): + restaurant_data_rows = elem.find('tbody').find_all( + has_two_tds, recursive=False + ) + rdata = {} + current_label = '' + for data_row in restaurant_data_rows: + key_cell, val_cell = data_row.find_all('td', recursive=False) + new_label = clean_data(key_cell) + current_label = new_label if new_label else current_label + rdata.setdefault(current_label, []).append(clean_data(val_cell)) + return rdata + + +def is_inspection_data_row(elem): + is_tr = elem.name == 'tr' + if not is_tr: + return False + td_children = elem.find_all('td', recursive=False) + has_four = len(td_children) == 4 + this_text = clean_data(td_children[0]).lower() + contains_word = 'inspection' in this_text + does_not_start = not this_text.startswith('inspection') + return is_tr and has_four and contains_word and does_not_start + + +def get_score_data(elem): + inspection_rows = elem.find_all(is_inspection_data_row) + samples = len(inspection_rows) + total = 0 + high_score = 0 + average = 0 + for row in inspection_rows: + strval = clean_data(row.find_all('td')[2]) + try: + intval = int(strval) + except (ValueError, TypeError): + samples -= 1 + else: + total += intval + high_score = intval if intval > high_score else high_score + + if samples: + average = total/float(samples) + data = { + u'Average Score': average, + u'High Score': high_score, + u'Total Inspections': samples + } + return data + + +def result_generator(count): + use_params = { + 'Inspection_Start': '2/1/2013', + 'Inspection_End': '2/1/2015', + 'Zip_Code': '98101' + } + # html = get_inspection_page(**use_params) + html = load_inspection_page('inspection_page.html') + parsed = parse_source(html) + content_col = parsed.find("td", id="contentcol") + data_list = restaurant_data_generator(content_col) + for data_div in data_list[:count]: + metadata = extract_restaurant_metadata(data_div) + inspection_data = get_score_data(data_div) + metadata.update(inspection_data) + yield metadata + + +def get_geojson(result): + address = " ".join(result.get('Address', '')) + if not address: + return None + geocoded = geocoder.google(address) + geojson = geocoded.geojson + inspection_data = {} + use_keys = ( + 'Business Name', 'Average Score', 'Total Inspections', 'High Score' + ) + for key, val in result.items(): + if key not in use_keys: + continue + if isinstance(val, list): + val = " ".join(val) + inspection_data[key] = val + geojson['properties'] = inspection_data + return geojson + + +if __name__ == '__main__': + total_result = {'type': 'FeatureCollection', 'features': []} + for result in result_generator(10): + geojson = get_geojson(result) + total_result['features'].append(geojson) + with open('my_map.json', 'w') as fh: + json.dump(total_result, fh) diff --git a/resources/session06/__init__.py b/resources/session06/__init__.py new file mode 100644 index 00000000..32e9cc81 --- /dev/null +++ b/resources/session06/__init__.py @@ -0,0 +1,30 @@ +from pyramid.config import Configurator +from sqlalchemy import engine_from_config + +from .models import ( + DBSession, + Base, + ) + + +def create_session(settings): + from sqlalchemy.orm import sessionmaker + engine = engine_from_config(settings, 'sqlalchemy.') + Session = sessionmaker(bind=engine) + return Session() + + +def main(global_config, **settings): + """ This function returns a Pyramid WSGI application. + """ + engine = engine_from_config(settings, 'sqlalchemy.') + DBSession.configure(bind=engine) + Base.metadata.bind = engine + config = Configurator(settings=settings) + config.include('pyramid_jinja2') + config.add_static_view('static', 'static', cache_max_age=3600) + config.add_route('home', '/') + config.add_route('detail', '/journal/{id:\d+}') + config.add_route('action', '/journal/{action}') + config.scan() + return config.make_wsgi_app() diff --git a/resources/session06/development.ini b/resources/session06/development.ini new file mode 100644 index 00000000..1139ff82 --- /dev/null +++ b/resources/session06/development.ini @@ -0,0 +1,76 @@ +### +# app configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/environment.html +### + +[app:main] +use = egg:learning_journal + +pyramid.reload_templates = true +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en +pyramid.includes = + pyramid_debugtoolbar + pyramid_tm + +sqlalchemy.url = sqlite:///%(here)s/learning_journal.sqlite + +# By default, the toolbar only appears for clients from IP addresses +# '127.0.0.1' and '::1'. +# debugtoolbar.hosts = 127.0.0.1 ::1 + + +[pshell] +create_session = learning_journal.create_session +Entry = learning_journal.models.Entry + +### +# wsgi server configuration +### + +[server:main] +use = egg:waitress#main +host = 0.0.0.0 +port = 6543 + +### +# logging configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/logging.html +### + +[loggers] +keys = root, learning_journal, sqlalchemy + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_learning_journal] +level = DEBUG +handlers = +qualname = learning_journal + +[logger_sqlalchemy] +level = INFO +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s diff --git a/resources/session02/forms.py b/resources/session06/forms.py similarity index 93% rename from resources/session02/forms.py rename to resources/session06/forms.py index 5629d7a9..88d9d348 100644 --- a/resources/session02/forms.py +++ b/resources/session06/forms.py @@ -8,7 +8,7 @@ strip_filter = lambda x: x.strip() if x else None -class BlogCreateForm(Form): +class EntryCreateForm(Form): title = TextField( 'Entry title', [validators.Length(min=1, max=255)], diff --git a/resources/session02/layout.jinja2 b/resources/session06/layout.jinja2 similarity index 88% rename from resources/session02/layout.jinja2 rename to resources/session06/layout.jinja2 index 93e462c0..8dbff846 100644 --- a/resources/session02/layout.jinja2 +++ b/resources/session06/layout.jinja2 @@ -6,7 +6,7 @@ - +
diff --git a/resources/session06/learning_journal/.gitignore b/resources/session06/learning_journal/.gitignore new file mode 100644 index 00000000..c7332211 --- /dev/null +++ b/resources/session06/learning_journal/.gitignore @@ -0,0 +1,3 @@ +*.pyc +.DS_Store +*.egg-info diff --git a/resources/session06/learning_journal/CHANGES.txt b/resources/session06/learning_journal/CHANGES.txt new file mode 100644 index 00000000..35a34f33 --- /dev/null +++ b/resources/session06/learning_journal/CHANGES.txt @@ -0,0 +1,4 @@ +0.0 +--- + +- Initial version diff --git a/resources/session06/learning_journal/MANIFEST.in b/resources/session06/learning_journal/MANIFEST.in new file mode 100644 index 00000000..3a0de395 --- /dev/null +++ b/resources/session06/learning_journal/MANIFEST.in @@ -0,0 +1,2 @@ +include *.txt *.ini *.cfg *.rst +recursive-include learning_journal *.ico *.png *.css *.gif *.jpg *.pt *.txt *.mak *.mako *.js *.html *.xml diff --git a/resources/session06/learning_journal/README.txt b/resources/session06/learning_journal/README.txt new file mode 100644 index 00000000..f49a002c --- /dev/null +++ b/resources/session06/learning_journal/README.txt @@ -0,0 +1,14 @@ +learning_journal README +================== + +Getting Started +--------------- + +- cd + +- $VENV/bin/python setup.py develop + +- $VENV/bin/initialize_learning_journal_db development.ini + +- $VENV/bin/pserve development.ini + diff --git a/resources/session06/learning_journal/development.ini b/resources/session06/learning_journal/development.ini new file mode 100644 index 00000000..1139ff82 --- /dev/null +++ b/resources/session06/learning_journal/development.ini @@ -0,0 +1,76 @@ +### +# app configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/environment.html +### + +[app:main] +use = egg:learning_journal + +pyramid.reload_templates = true +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en +pyramid.includes = + pyramid_debugtoolbar + pyramid_tm + +sqlalchemy.url = sqlite:///%(here)s/learning_journal.sqlite + +# By default, the toolbar only appears for clients from IP addresses +# '127.0.0.1' and '::1'. +# debugtoolbar.hosts = 127.0.0.1 ::1 + + +[pshell] +create_session = learning_journal.create_session +Entry = learning_journal.models.Entry + +### +# wsgi server configuration +### + +[server:main] +use = egg:waitress#main +host = 0.0.0.0 +port = 6543 + +### +# logging configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/logging.html +### + +[loggers] +keys = root, learning_journal, sqlalchemy + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_learning_journal] +level = DEBUG +handlers = +qualname = learning_journal + +[logger_sqlalchemy] +level = INFO +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s diff --git a/resources/session06/learning_journal/learning_journal/__init__.py b/resources/session06/learning_journal/learning_journal/__init__.py new file mode 100644 index 00000000..32e9cc81 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/__init__.py @@ -0,0 +1,30 @@ +from pyramid.config import Configurator +from sqlalchemy import engine_from_config + +from .models import ( + DBSession, + Base, + ) + + +def create_session(settings): + from sqlalchemy.orm import sessionmaker + engine = engine_from_config(settings, 'sqlalchemy.') + Session = sessionmaker(bind=engine) + return Session() + + +def main(global_config, **settings): + """ This function returns a Pyramid WSGI application. + """ + engine = engine_from_config(settings, 'sqlalchemy.') + DBSession.configure(bind=engine) + Base.metadata.bind = engine + config = Configurator(settings=settings) + config.include('pyramid_jinja2') + config.add_static_view('static', 'static', cache_max_age=3600) + config.add_route('home', '/') + config.add_route('detail', '/journal/{id:\d+}') + config.add_route('action', '/journal/{action}') + config.scan() + return config.make_wsgi_app() diff --git a/resources/session06/learning_journal/learning_journal/forms.py b/resources/session06/learning_journal/learning_journal/forms.py new file mode 100644 index 00000000..88d9d348 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/forms.py @@ -0,0 +1,21 @@ +from wtforms import ( + Form, + TextField, + TextAreaField, + validators, +) + +strip_filter = lambda x: x.strip() if x else None + + +class EntryCreateForm(Form): + title = TextField( + 'Entry title', + [validators.Length(min=1, max=255)], + filters=[strip_filter] + ) + body = TextAreaField( + 'Entry body', + [validators.Length(min=1)], + filters=[strip_filter] + ) diff --git a/resources/session06/learning_journal/learning_journal/models.py b/resources/session06/learning_journal/learning_journal/models.py new file mode 100644 index 00000000..7afb0ddb --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/models.py @@ -0,0 +1,64 @@ +import datetime +from sqlalchemy import ( + Column, + DateTime, + Index, + Integer, + Text, + Unicode, + UnicodeText, + ) + +import sqlalchemy as sa +from sqlalchemy.ext.declarative import declarative_base + +from sqlalchemy.orm import ( + scoped_session, + sessionmaker, + ) + +from zope.sqlalchemy import ZopeTransactionExtension + +DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension())) +Base = declarative_base() + + +class MyModel(Base): + __tablename__ = 'models' + id = Column(Integer, primary_key=True) + name = Column(Text) + value = Column(Integer) + +Index('my_index', MyModel.name, unique=True, mysql_length=255) + + +class Entry(Base): + __tablename__ = 'entries' + id = Column(Integer, primary_key=True) + title = Column(Unicode(255), unique=True, nullable=False) + body = Column(UnicodeText, default=u'') + created = Column(DateTime, default=datetime.datetime.utcnow) + edited = Column(DateTime, default=datetime.datetime.utcnow) + + @classmethod + def all(cls, session=None): + """return a query with all entries, ordered by creation date reversed + """ + if session is None: + session = DBSession + return session.query(cls).order_by(sa.desc(cls.created)).all() + + @classmethod + def by_id(cls, id, session=None): + """return a single entry identified by id + + If no entry exists with the provided id, return None + """ + if session is None: + session = DBSession + return session.query(cls).get(id) + + + + + diff --git a/resources/session06/learning_journal/learning_journal/scripts/__init__.py b/resources/session06/learning_journal/learning_journal/scripts/__init__.py new file mode 100644 index 00000000..5bb534f7 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/scripts/__init__.py @@ -0,0 +1 @@ +# package diff --git a/resources/session06/learning_journal/learning_journal/scripts/initializedb.py b/resources/session06/learning_journal/learning_journal/scripts/initializedb.py new file mode 100644 index 00000000..7dfdece1 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/scripts/initializedb.py @@ -0,0 +1,40 @@ +import os +import sys +import transaction + +from sqlalchemy import engine_from_config + +from pyramid.paster import ( + get_appsettings, + setup_logging, + ) + +from pyramid.scripts.common import parse_vars + +from ..models import ( + DBSession, + MyModel, + Base, + ) + + +def usage(argv): + cmd = os.path.basename(argv[0]) + print('usage: %s [var=value]\n' + '(example: "%s development.ini")' % (cmd, cmd)) + sys.exit(1) + + +def main(argv=sys.argv): + if len(argv) < 2: + usage(argv) + config_uri = argv[1] + options = parse_vars(argv[2:]) + setup_logging(config_uri) + settings = get_appsettings(config_uri, options=options) + engine = engine_from_config(settings, 'sqlalchemy.') + DBSession.configure(bind=engine) + Base.metadata.create_all(engine) + with transaction.manager: + model = MyModel(name='one', value=1) + DBSession.add(model) diff --git a/resources/session06/learning_journal/learning_journal/static/pyramid-16x16.png b/resources/session06/learning_journal/learning_journal/static/pyramid-16x16.png new file mode 100644 index 00000000..97920311 Binary files /dev/null and b/resources/session06/learning_journal/learning_journal/static/pyramid-16x16.png differ diff --git a/resources/session06/learning_journal/learning_journal/static/pyramid.png b/resources/session06/learning_journal/learning_journal/static/pyramid.png new file mode 100644 index 00000000..4ab837be Binary files /dev/null and b/resources/session06/learning_journal/learning_journal/static/pyramid.png differ diff --git a/resources/session02/styles.css b/resources/session06/learning_journal/learning_journal/static/styles.css similarity index 100% rename from resources/session02/styles.css rename to resources/session06/learning_journal/learning_journal/static/styles.css diff --git a/resources/session06/learning_journal/learning_journal/static/theme.css b/resources/session06/learning_journal/learning_journal/static/theme.css new file mode 100644 index 00000000..228768e2 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/static/theme.css @@ -0,0 +1,152 @@ +@import url(//fonts.googleapis.com/css?family=Open+Sans:300,400,600,700); +body { + font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; + font-weight: 300; + color: #ffffff; + background: #bc2131; +} +h1, +h2, +h3, +h4, +h5, +h6 { + font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; + font-weight: 300; +} +p { + font-weight: 300; +} +.font-normal { + font-weight: 400; +} +.font-semi-bold { + font-weight: 600; +} +.font-bold { + font-weight: 700; +} +.starter-template { + margin-top: 25px; +} +.starter-template .content { + margin-left: 10px; +} +.starter-template .content h1 { + margin-top: 10px; + font-size: 60px; +} +.starter-template .content h1 .smaller { + font-size: 40px; + color: #f2b7bd; +} +.starter-template .content .lead { + font-size: 25px; + color: #f2b7bd; +} +.starter-template .content .lead .font-normal { + color: #ffffff; +} +.starter-template .links { + float: right; + right: 0; + margin-top: 125px; +} +.starter-template .links ul { + display: block; + padding: 0; + margin: 0; +} +.starter-template .links ul li { + list-style: none; + display: inline; + margin: 0 10px; +} +.starter-template .links ul li:first-child { + margin-left: 0; +} +.starter-template .links ul li:last-child { + margin-right: 0; +} +.starter-template .links ul li.current-version { + color: #f2b7bd; + font-weight: 400; +} +.starter-template .links ul li a { + color: #ffffff; +} +.starter-template .links ul li a:hover { + text-decoration: underline; +} +.starter-template .links ul li .icon-muted { + color: #eb8b95; + margin-right: 5px; +} +.starter-template .links ul li:hover .icon-muted { + color: #ffffff; +} +.starter-template .copyright { + margin-top: 10px; + font-size: 0.9em; + color: #f2b7bd; + text-transform: lowercase; + float: right; + right: 0; +} +@media (max-width: 1199px) { + .starter-template .content h1 { + font-size: 45px; + } + .starter-template .content h1 .smaller { + font-size: 30px; + } + .starter-template .content .lead { + font-size: 20px; + } +} +@media (max-width: 991px) { + .starter-template { + margin-top: 0; + } + .starter-template .logo { + margin: 40px auto; + } + .starter-template .content { + margin-left: 0; + text-align: center; + } + .starter-template .content h1 { + margin-bottom: 20px; + } + .starter-template .links { + float: none; + text-align: center; + margin-top: 60px; + } + .starter-template .copyright { + float: none; + text-align: center; + } +} +@media (max-width: 767px) { + .starter-template .content h1 .smaller { + font-size: 25px; + display: block; + } + .starter-template .content .lead { + font-size: 16px; + } + .starter-template .links { + margin-top: 40px; + } + .starter-template .links ul li { + display: block; + margin: 0; + } + .starter-template .links ul li .icon-muted { + display: none; + } + .starter-template .copyright { + margin-top: 20px; + } +} diff --git a/resources/session06/learning_journal/learning_journal/static/theme.min.css b/resources/session06/learning_journal/learning_journal/static/theme.min.css new file mode 100644 index 00000000..2f924bcc --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/static/theme.min.css @@ -0,0 +1 @@ +@import url(//fonts.googleapis.com/css?family=Open+Sans:300,400,600,700);body{font-family:"Open Sans","Helvetica Neue",Helvetica,Arial,sans-serif;font-weight:300;color:#fff;background:#bc2131}h1,h2,h3,h4,h5,h6{font-family:"Open Sans","Helvetica Neue",Helvetica,Arial,sans-serif;font-weight:300}p{font-weight:300}.font-normal{font-weight:400}.font-semi-bold{font-weight:600}.font-bold{font-weight:700}.starter-template{margin-top:250px}.starter-template .content{margin-left:10px}.starter-template .content h1{margin-top:10px;font-size:60px}.starter-template .content h1 .smaller{font-size:40px;color:#f2b7bd}.starter-template .content .lead{font-size:25px;color:#f2b7bd}.starter-template .content .lead .font-normal{color:#fff}.starter-template .links{float:right;right:0;margin-top:125px}.starter-template .links ul{display:block;padding:0;margin:0}.starter-template .links ul li{list-style:none;display:inline;margin:0 10px}.starter-template .links ul li:first-child{margin-left:0}.starter-template .links ul li:last-child{margin-right:0}.starter-template .links ul li.current-version{color:#f2b7bd;font-weight:400}.starter-template .links ul li a{color:#fff}.starter-template .links ul li a:hover{text-decoration:underline}.starter-template .links ul li .icon-muted{color:#eb8b95;margin-right:5px}.starter-template .links ul li:hover .icon-muted{color:#fff}.starter-template .copyright{margin-top:10px;font-size:.9em;color:#f2b7bd;text-transform:lowercase;float:right;right:0}@media (max-width:1199px){.starter-template .content h1{font-size:45px}.starter-template .content h1 .smaller{font-size:30px}.starter-template .content .lead{font-size:20px}}@media (max-width:991px){.starter-template{margin-top:0}.starter-template .logo{margin:40px auto}.starter-template .content{margin-left:0;text-align:center}.starter-template .content h1{margin-bottom:20px}.starter-template .links{float:none;text-align:center;margin-top:60px}.starter-template .copyright{float:none;text-align:center}}@media (max-width:767px){.starter-template .content h1 .smaller{font-size:25px;display:block}.starter-template .content .lead{font-size:16px}.starter-template .links{margin-top:40px}.starter-template .links ul li{display:block;margin:0}.starter-template .links ul li .icon-muted{display:none}.starter-template .copyright{margin-top:20px}} \ No newline at end of file diff --git a/resources/session06/learning_journal/learning_journal/templates/detail.jinja2 b/resources/session06/learning_journal/learning_journal/templates/detail.jinja2 new file mode 100644 index 00000000..29d736f5 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/templates/detail.jinja2 @@ -0,0 +1,11 @@ +{% extends "layout.jinja2" %} +{% block body %} +
+

{{ entry.title }}

+
+

{{ entry.body }}

+
+

Created {{entry.created}}

+
+

Go Back

+{% endblock %} diff --git a/resources/session06/learning_journal/learning_journal/templates/edit.jinja2 b/resources/session06/learning_journal/learning_journal/templates/edit.jinja2 new file mode 100644 index 00000000..ebe0f6b9 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/templates/edit.jinja2 @@ -0,0 +1,17 @@ +{% extends "templates/layout.jinja2" %} +{% block body %} +

Create a Journal Entry

+
+{% for field in form %} + {% if field.errors %} +
    + {% for error in field.errors %} +
  • {{ error }}
  • + {% endfor %} +
+ {% endif %} +

{{ field.label }}: {{ field }}

+{% endfor %} +

+
+{% endblock %} diff --git a/resources/session06/learning_journal/learning_journal/templates/layout.jinja2 b/resources/session06/learning_journal/learning_journal/templates/layout.jinja2 new file mode 100644 index 00000000..a8b27afd --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/templates/layout.jinja2 @@ -0,0 +1,29 @@ + + + + + Python Learning Journal + + + + +
+ +
+
+

My Python Journal

+
+ {% block body %}{% endblock %} +
+
+
+

Created in the UW PCE Python Certificate Program

+
+ + diff --git a/resources/session06/learning_journal/learning_journal/templates/list.jinja2 b/resources/session06/learning_journal/learning_journal/templates/list.jinja2 new file mode 100644 index 00000000..09c835a8 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/templates/list.jinja2 @@ -0,0 +1,16 @@ +{% extends "layout.jinja2" %} +{% block body %} +{% if entries %} +

Journal Entries

+ +{% else %} +

This journal is empty

+{% endif %} +

New Entry

+{% endblock %} diff --git a/resources/session06/learning_journal/learning_journal/templates/mytemplate.pt b/resources/session06/learning_journal/learning_journal/templates/mytemplate.pt new file mode 100644 index 00000000..9e88dc4b --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/templates/mytemplate.pt @@ -0,0 +1,66 @@ + + + + + + + + + + + Alchemy Scaffold for The Pyramid Web Framework + + + + + + + + + + + + + +
+
+
+
+ +
+
+
+

Pyramid Alchemy scaffold

+

Welcome to ${project}, an application generated by
the Pyramid Web Framework 1.5.2.

+
+
+
+
+ +
+
+ +
+
+
+ + + + + + + + diff --git a/resources/session06/learning_journal/learning_journal/tests.py b/resources/session06/learning_journal/learning_journal/tests.py new file mode 100644 index 00000000..4fc444a6 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/tests.py @@ -0,0 +1,55 @@ +import unittest +import transaction + +from pyramid import testing + +from .models import DBSession + + +class TestMyViewSuccessCondition(unittest.TestCase): + def setUp(self): + self.config = testing.setUp() + from sqlalchemy import create_engine + engine = create_engine('sqlite://') + from .models import ( + Base, + MyModel, + ) + DBSession.configure(bind=engine) + Base.metadata.create_all(engine) + with transaction.manager: + model = MyModel(name='one', value=55) + DBSession.add(model) + + def tearDown(self): + DBSession.remove() + testing.tearDown() + + def test_passing_view(self): + from .views import my_view + request = testing.DummyRequest() + info = my_view(request) + self.assertEqual(info['one'].name, 'one') + self.assertEqual(info['project'], 'learning_journal') + + +class TestMyViewFailureCondition(unittest.TestCase): + def setUp(self): + self.config = testing.setUp() + from sqlalchemy import create_engine + engine = create_engine('sqlite://') + from .models import ( + Base, + MyModel, + ) + DBSession.configure(bind=engine) + + def tearDown(self): + DBSession.remove() + testing.tearDown() + + def test_failing_view(self): + from .views import my_view + request = testing.DummyRequest() + info = my_view(request) + self.assertEqual(info.status_int, 500) diff --git a/resources/session06/learning_journal/learning_journal/views.py b/resources/session06/learning_journal/learning_journal/views.py new file mode 100644 index 00000000..ad76afb5 --- /dev/null +++ b/resources/session06/learning_journal/learning_journal/views.py @@ -0,0 +1,43 @@ +from pyramid.httpexceptions import HTTPFound, HTTPNotFound +from pyramid.view import view_config + +from .models import ( + DBSession, + MyModel, + Entry, + ) + +from .forms import EntryCreateForm + + +@view_config(route_name='home', renderer='templates/list.jinja2') +def index_page(request): + entries = Entry.all() + return {'entries': entries} + + +@view_config(route_name='detail', renderer='templates/detail.jinja2') +def view(request): + this_id = request.matchdict.get('id', -1) + entry = Entry.by_id(this_id) + if not entry: + return HTTPNotFound() + return {'entry': entry} + + +@view_config(route_name='action', match_param='action=create', + renderer='templates/edit.jinja2') +def create(request): + entry = Entry() + form = EntryCreateForm(request.POST) + if request.method == 'POST' and form.validate(): + form.populate_obj(entry) + DBSession.add(entry) + return HTTPFound(location=request.route_url('home')) + return {'form': form, 'action': request.matchdict.get('action')} + + +@view_config(route_name='action', match_param='action=edit', + renderer='string') +def update(request): + return 'edit page' diff --git a/resources/session06/learning_journal/production.ini b/resources/session06/learning_journal/production.ini new file mode 100644 index 00000000..1db7a630 --- /dev/null +++ b/resources/session06/learning_journal/production.ini @@ -0,0 +1,62 @@ +### +# app configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/environment.html +### + +[app:main] +use = egg:learning_journal + +pyramid.reload_templates = false +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en +pyramid.includes = + pyramid_tm + +sqlalchemy.url = sqlite:///%(here)s/learning_journal.sqlite + +[server:main] +use = egg:waitress#main +host = 0.0.0.0 +port = 6543 + +### +# logging configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/logging.html +### + +[loggers] +keys = root, learning_journal, sqlalchemy + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = WARN +handlers = console + +[logger_learning_journal] +level = WARN +handlers = +qualname = learning_journal + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s diff --git a/resources/session06/learning_journal/setup.py b/resources/session06/learning_journal/setup.py new file mode 100644 index 00000000..e4bb0bcd --- /dev/null +++ b/resources/session06/learning_journal/setup.py @@ -0,0 +1,48 @@ +import os + +from setuptools import setup, find_packages + +here = os.path.abspath(os.path.dirname(__file__)) +with open(os.path.join(here, 'README.txt')) as f: + README = f.read() +with open(os.path.join(here, 'CHANGES.txt')) as f: + CHANGES = f.read() + +requires = [ + 'pyramid', + 'pyramid_jinja2', + 'pyramid_debugtoolbar', + 'pyramid_tm', + 'SQLAlchemy', + 'transaction', + 'zope.sqlalchemy', + 'waitress', + 'wtforms', + ] + +setup(name='learning_journal', + version='0.0', + description='learning_journal', + long_description=README + '\n\n' + CHANGES, + classifiers=[ + "Programming Language :: Python", + "Framework :: Pyramid", + "Topic :: Internet :: WWW/HTTP", + "Topic :: Internet :: WWW/HTTP :: WSGI :: Application", + ], + author='', + author_email='', + url='', + keywords='web wsgi bfg pylons pyramid', + packages=find_packages(), + include_package_data=True, + zip_safe=False, + test_suite='learning_journal', + install_requires=requires, + entry_points="""\ + [paste.app_factory] + main = learning_journal:main + [console_scripts] + initialize_learning_journal_db = learning_journal.scripts.initializedb:main + """, + ) diff --git a/resources/session02/models.py b/resources/session06/models.py similarity index 80% rename from resources/session02/models.py rename to resources/session06/models.py index 4d689df5..e87ac2c8 100644 --- a/resources/session02/models.py +++ b/resources/session06/models.py @@ -41,15 +41,19 @@ class Entry(Base): edited = Column(DateTime, default=datetime.datetime.utcnow) @classmethod - def all(cls): + def all(cls, session=None): """return a query with all entries, ordered by creation date reversed """ - return DBSession.query(cls).order_by(sa.desc(cls.created)).all() + if session is None: + session = DBSession + return session.query(cls).order_by(sa.desc(cls.created)).all() @classmethod - def by_id(cls, id): + def by_id(cls, id, session=None): """return a single entry identified by id If no entry exists with the provided id, return None """ - return DBSession.query(cls).get(id) + if session is None: + session = DBSession + return session.query(cls).get(id) diff --git a/resources/session06/styles.css b/resources/session06/styles.css new file mode 100644 index 00000000..951ac84f --- /dev/null +++ b/resources/session06/styles.css @@ -0,0 +1,73 @@ +body{ + color:#111; + padding:0; + margin:0; + background-color: #eee;} +header{ + margin:0; + padding:0 0.75em; + width:100%; + background: #222; + color: #ccc; + border-bottom: 3px solid #fff;} +header:after{ + content:""; + display:table; + clear:both;} +header a, +footer a{ + text-decoration:none} +header a:hover, +footer a:hover { + color:#fff; +} +header a:visited, +footer a:visited { + color:#eee; +} +header aside{ + float:right; + text-align:right; + padding-right:0.75em} +header ul{ + list-style:none; + list-style-type:none; + display:inline-block} +header ul li{ + margin:0 0.25em 0 0} +header ul li a{ + padding:0; + display:inline-block} +main{padding:0 0.75em 1em} +main:after{ + content:""; + display:table; + clear:both} +main article{ + margin-bottom:1em; + padding-left:0.5em} +main article h3{margin-top:0} +main article .entry_body{ + margin:0.5em} +main aside{float:right} +main aside .field{ + margin-bottom:1em} +main aside .field input, +main aside .field label, +main aside .field textarea{ + vertical-align:top} +main aside .field label{ + display:inline-block; + width:15%; + padding-top:2px} +main aside .field input, +main aside .field textarea{ + width:83%} +main aside .control_row input{ + margin-left:16%} +footer{ + padding: 1em 0.75em; + background: #222; + color: #ccc; + border-top: 3px solid #fff; + border-bottom: 3px solid #fff;} diff --git a/resources/session07/detail.jinja2 b/resources/session07/detail.jinja2 new file mode 100644 index 00000000..f80810d3 --- /dev/null +++ b/resources/session07/detail.jinja2 @@ -0,0 +1,15 @@ +{% extends "layout.jinja2" %} +{% block body %} +
+

{{ entry.title }}

+
+

{{ entry.body }}

+
+

Created {{entry.created}}

+
+

+ Go Back :: + + Edit Entry +

+{% endblock %} diff --git a/resources/session07/forms.py b/resources/session07/forms.py new file mode 100644 index 00000000..fad71bd1 --- /dev/null +++ b/resources/session07/forms.py @@ -0,0 +1,26 @@ +from wtforms import ( + Form, + HiddenField, + TextField, + TextAreaField, + validators, +) + +strip_filter = lambda x: x.strip() if x else None + + +class EntryCreateForm(Form): + title = TextField( + 'Entry title', + [validators.Length(min=1, max=255)], + filters=[strip_filter] + ) + body = TextAreaField( + 'Entry body', + [validators.Length(min=1)], + filters=[strip_filter] + ) + + +class EntryEditForm(EntryCreateForm): + id = HiddenField() diff --git a/resources/session07/learning_journal/.gitignore b/resources/session07/learning_journal/.gitignore new file mode 100644 index 00000000..2ffa3242 --- /dev/null +++ b/resources/session07/learning_journal/.gitignore @@ -0,0 +1,4 @@ +*.pyc +.DS_Store +*.egg-info +*.sqlite diff --git a/resources/session07/learning_journal/CHANGES.txt b/resources/session07/learning_journal/CHANGES.txt new file mode 100644 index 00000000..35a34f33 --- /dev/null +++ b/resources/session07/learning_journal/CHANGES.txt @@ -0,0 +1,4 @@ +0.0 +--- + +- Initial version diff --git a/resources/session07/learning_journal/MANIFEST.in b/resources/session07/learning_journal/MANIFEST.in new file mode 100644 index 00000000..3a0de395 --- /dev/null +++ b/resources/session07/learning_journal/MANIFEST.in @@ -0,0 +1,2 @@ +include *.txt *.ini *.cfg *.rst +recursive-include learning_journal *.ico *.png *.css *.gif *.jpg *.pt *.txt *.mak *.mako *.js *.html *.xml diff --git a/resources/session07/learning_journal/Procfile b/resources/session07/learning_journal/Procfile new file mode 100644 index 00000000..e6450506 --- /dev/null +++ b/resources/session07/learning_journal/Procfile @@ -0,0 +1 @@ +web: ./run diff --git a/resources/session07/learning_journal/README.txt b/resources/session07/learning_journal/README.txt new file mode 100644 index 00000000..f49a002c --- /dev/null +++ b/resources/session07/learning_journal/README.txt @@ -0,0 +1,14 @@ +learning_journal README +================== + +Getting Started +--------------- + +- cd + +- $VENV/bin/python setup.py develop + +- $VENV/bin/initialize_learning_journal_db development.ini + +- $VENV/bin/pserve development.ini + diff --git a/resources/session07/learning_journal/build_db b/resources/session07/learning_journal/build_db new file mode 100755 index 00000000..a912dc68 --- /dev/null +++ b/resources/session07/learning_journal/build_db @@ -0,0 +1,3 @@ +#!/bin/bash +python setup.py develop +initialize_learning_journal_db production.ini diff --git a/resources/session07/learning_journal/development.ini b/resources/session07/learning_journal/development.ini new file mode 100644 index 00000000..f52e811f --- /dev/null +++ b/resources/session07/learning_journal/development.ini @@ -0,0 +1,78 @@ +### +# app configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/environment.html +### + +[app:main] +use = egg:learning_journal + +pyramid.reload_templates = true +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en +pyramid.includes = + pyramid_debugtoolbar + pyramid_tm + +sqlalchemy.url = sqlite:///%(here)s/learning_journal.sqlite + +jinja2.filters = + markdown = learning_journal.views.render_markdown + +# By default, the toolbar only appears for clients from IP addresses +# '127.0.0.1' and '::1'. +# debugtoolbar.hosts = 127.0.0.1 ::1 + +[pshell] +create_session = learning_journal.create_session +Entry = learning_journal.models.Entry + +### +# wsgi server configuration +### + +[server:main] +use = egg:waitress#main +host = 0.0.0.0 +port = 6543 + +### +# logging configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/logging.html +### + +[loggers] +keys = root, learning_journal, sqlalchemy + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = INFO +handlers = console + +[logger_learning_journal] +level = DEBUG +handlers = +qualname = learning_journal + +[logger_sqlalchemy] +level = INFO +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s diff --git a/resources/session07/learning_journal/learning_journal/__init__.py b/resources/session07/learning_journal/learning_journal/__init__.py new file mode 100644 index 00000000..6f51ba85 --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/__init__.py @@ -0,0 +1,44 @@ +import os +from pyramid.authentication import AuthTktAuthenticationPolicy +from pyramid.authorization import ACLAuthorizationPolicy +from pyramid.config import Configurator +from sqlalchemy import engine_from_config + +from .models import ( + DBSession, + Base, + ) +from .security import EntryFactory + + +def create_session(settings): + from sqlalchemy.orm import sessionmaker + engine = engine_from_config(settings, 'sqlalchemy.') + Session = sessionmaker(bind=engine) + return Session() + + +def main(global_config, **settings): + """ This function returns a Pyramid WSGI application. + """ + if 'DATABASE_URL' in os.environ: + settings['sqlalchemy.url'] = os.environ['DATABASE_URL'] + engine = engine_from_config(settings, 'sqlalchemy.') + engine = engine_from_config(settings, 'sqlalchemy.') + DBSession.configure(bind=engine) + Base.metadata.bind = engine + secret = os.environ.get('AUTH_SECRET', 'somesecret') + config = Configurator( + settings=settings, + authentication_policy=AuthTktAuthenticationPolicy(secret), + authorization_policy=ACLAuthorizationPolicy(), + default_permission='view' + ) + config.include('pyramid_jinja2') + config.add_static_view('static', 'static', cache_max_age=3600) + config.add_route('home', '/', factory=EntryFactory) + config.add_route('detail', '/journal/{id:\d+}', factory=EntryFactory) + config.add_route('action', '/journal/{action}', factory=EntryFactory) + config.add_route('auth', '/sign/{action}', factory=EntryFactory) + config.scan() + return config.make_wsgi_app() diff --git a/resources/session07/learning_journal/learning_journal/forms.py b/resources/session07/learning_journal/learning_journal/forms.py new file mode 100644 index 00000000..652c286b --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/forms.py @@ -0,0 +1,32 @@ +from wtforms import ( + Form, + HiddenField, + PasswordField, + TextField, + TextAreaField, + validators, +) + +strip_filter = lambda x: x.strip() if x else None + + +class EntryCreateForm(Form): + title = TextField( + 'Entry title', + [validators.Length(min=1, max=255)], + filters=[strip_filter] + ) + body = TextAreaField( + 'Entry body', + [validators.Length(min=1)], + filters=[strip_filter] + ) + + +class EntryEditForm(EntryCreateForm): + id = HiddenField() + + +class LoginForm(Form): + username = TextField('Username', [validators.Length(min=1, max=255)]) + password = PasswordField('Password', [validators.Length(min=1, max=255)]) diff --git a/resources/session07/learning_journal/learning_journal/models.py b/resources/session07/learning_journal/learning_journal/models.py new file mode 100644 index 00000000..6f2290c2 --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/models.py @@ -0,0 +1,79 @@ +import datetime +from passlib.context import CryptContext +from sqlalchemy import ( + Column, + DateTime, + Index, + Integer, + Text, + Unicode, + UnicodeText, + ) + +import sqlalchemy as sa +from sqlalchemy.ext.declarative import declarative_base + +from sqlalchemy.orm import ( + scoped_session, + sessionmaker, + ) + +from zope.sqlalchemy import ZopeTransactionExtension + +DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension())) +Base = declarative_base() + + +password_context = CryptContext(schemes=['pbkdf2_sha512']) + + +class MyModel(Base): + __tablename__ = 'models' + id = Column(Integer, primary_key=True) + name = Column(Text) + value = Column(Integer) + +Index('my_index', MyModel.name, unique=True, mysql_length=255) + + +class Entry(Base): + __tablename__ = 'entries' + id = Column(Integer, primary_key=True) + title = Column(Unicode(255), unique=True, nullable=False) + body = Column(UnicodeText, default=u'') + created = Column(DateTime, default=datetime.datetime.utcnow) + edited = Column(DateTime, default=datetime.datetime.utcnow) + + @classmethod + def all(cls, session=None): + """return a query with all entries, ordered by creation date reversed + """ + if session is None: + session = DBSession + return session.query(cls).order_by(sa.desc(cls.created)).all() + + @classmethod + def by_id(cls, id, session=None): + """return a single entry identified by id + + If no entry exists with the provided id, return None + """ + if session is None: + session = DBSession + return session.query(cls).get(id) + + +class User(Base): + __tablename__ = 'users' + id = Column(Integer, primary_key=True, autoincrement=True) + name = Column(Unicode(255), unique=True, nullable=False) + password = Column(Unicode(255), nullable=False) + + @classmethod + def by_name(cls, name, session=None): + if session is None: + session = DBSession + return DBSession.query(cls).filter(cls.name == name).first() + + def verify_password(self, password): + return password_context.verify(password, self.password) diff --git a/resources/session07/learning_journal/learning_journal/scripts/__init__.py b/resources/session07/learning_journal/learning_journal/scripts/__init__.py new file mode 100644 index 00000000..5bb534f7 --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/scripts/__init__.py @@ -0,0 +1 @@ +# package diff --git a/resources/session07/learning_journal/learning_journal/scripts/initializedb.py b/resources/session07/learning_journal/learning_journal/scripts/initializedb.py new file mode 100644 index 00000000..a5dc7b20 --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/scripts/initializedb.py @@ -0,0 +1,46 @@ +import os +import sys +import transaction + +from sqlalchemy import engine_from_config + +from pyramid.paster import ( + get_appsettings, + setup_logging, + ) + +from pyramid.scripts.common import parse_vars + +from ..models import ( + DBSession, + MyModel, + Base, + User, + password_context + ) + + +def usage(argv): + cmd = os.path.basename(argv[0]) + print('usage: %s [var=value]\n' + '(example: "%s development.ini")' % (cmd, cmd)) + sys.exit(1) + + +def main(argv=sys.argv): + if len(argv) < 2: + usage(argv) + config_uri = argv[1] + options = parse_vars(argv[2:]) + setup_logging(config_uri) + settings = get_appsettings(config_uri, options=options) + if 'DATABASE_URL' in os.environ: + settings['sqlalchemy.url'] = os.environ['DATABASE_URL'] + engine = engine_from_config(settings, 'sqlalchemy.') + DBSession.configure(bind=engine) + Base.metadata.create_all(engine) + with transaction.manager: + password = os.environ.get('ADMIN_PASSWORD', 'admin') + encrypted = password_context.encrypt(password) + admin = User(name=u'admin', password=encrypted) + DBSession.add(admin) diff --git a/resources/session07/learning_journal/learning_journal/security.py b/resources/session07/learning_journal/learning_journal/security.py new file mode 100644 index 00000000..2ee4d4eb --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/security.py @@ -0,0 +1,12 @@ +from pyramid.security import Allow, Everyone, Authenticated + + +class EntryFactory(object): + __acl__ = [ + (Allow, Everyone, 'view'), + (Allow, Authenticated, 'create'), + (Allow, Authenticated, 'edit'), + ] + + def __init__(self, request): + pass diff --git a/resources/session07/learning_journal/learning_journal/static/pyramid-16x16.png b/resources/session07/learning_journal/learning_journal/static/pyramid-16x16.png new file mode 100644 index 00000000..97920311 Binary files /dev/null and b/resources/session07/learning_journal/learning_journal/static/pyramid-16x16.png differ diff --git a/resources/session07/learning_journal/learning_journal/static/pyramid.png b/resources/session07/learning_journal/learning_journal/static/pyramid.png new file mode 100644 index 00000000..4ab837be Binary files /dev/null and b/resources/session07/learning_journal/learning_journal/static/pyramid.png differ diff --git a/resources/session07/learning_journal/learning_journal/static/styles.css b/resources/session07/learning_journal/learning_journal/static/styles.css new file mode 100644 index 00000000..e9299ebd --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/static/styles.css @@ -0,0 +1,138 @@ +body{ + color:#111; + padding:0; + margin:0; + background-color: #eee;} +header{ + margin:0; + padding:0 0.75em; + width:100%; + background: #222; + color: #ccc; + border-bottom: 3px solid #fff;} +header:after{ + content:""; + display:table; + clear:both;} +header a, +footer a{ + text-decoration:none} +header a:hover, +footer a:hover { + color:#fff; +} +header a:visited, +footer a:visited { + color:#eee; +} +header aside{ + float:right; + text-align:right; + padding-right:0.75em} +header ul{ + list-style:none; + list-style-type:none; + display:inline-block} +header ul li{ + margin:0 0.25em 0 0} +header ul li a{ + padding:0; + display:inline-block} +main{padding:0 0.75em 1em} +main:after{ + content:""; + display:table; + clear:both} +main article{ + margin-bottom:1em; + padding-left:0.5em} +main article h3{margin-top:0} +main article .entry_body{ + margin:0.5em} +main aside{float:right} +main aside .field{ + margin-bottom:1em} +main aside .field input, +main aside .field label, +main aside .field textarea{ + vertical-align:top} +main aside .field label{ + display:inline-block; + width:15%; + padding-top:2px} +main aside .field input, +main aside .field textarea{ + width:83%} +main aside .control_row input{ + margin-left:16%} +footer{ + padding: 1em 0.75em; + background: #222; + color: #ccc; + border-top: 3px solid #fff; + border-bottom: 3px solid #fff;} +.codehilite {padding: 0.25em 1em;} +/* Pygments code hilight styles */ +.codehilite .hll { background-color: #ffffcc } +.codehilite { background: #ffffff; } +.codehilite .c { color: #888888 } /* Comment */ +.codehilite .err { color: #FF0000; background-color: #FFAAAA } /* Error */ +.codehilite .k { color: #008800; font-weight: bold } /* Keyword */ +.codehilite .o { color: #333333 } /* Operator */ +.codehilite .cm { color: #888888 } /* Comment.Multiline */ +.codehilite .cp { color: #557799 } /* Comment.Preproc */ +.codehilite .c1 { color: #888888 } /* Comment.Single */ +.codehilite .cs { color: #cc0000; font-weight: bold } /* Comment.Special */ +.codehilite .gd { color: #A00000 } /* Generic.Deleted */ +.codehilite .ge { font-style: italic } /* Generic.Emph */ +.codehilite .gr { color: #FF0000 } /* Generic.Error */ +.codehilite .gh { color: #000080; font-weight: bold } /* Generic.Heading */ +.codehilite .gi { color: #00A000 } /* Generic.Inserted */ +.codehilite .go { color: #888888 } /* Generic.Output */ +.codehilite .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */ +.codehilite .gs { font-weight: bold } /* Generic.Strong */ +.codehilite .gu { color: #800080; font-weight: bold } /* Generic.Subheading */ +.codehilite .gt { color: #0044DD } /* Generic.Traceback */ +.codehilite .kc { color: #008800; font-weight: bold } /* Keyword.Constant */ +.codehilite .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */ +.codehilite .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */ +.codehilite .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */ +.codehilite .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */ +.codehilite .kt { color: #333399; font-weight: bold } /* Keyword.Type */ +.codehilite .m { color: #6600EE; font-weight: bold } /* Literal.Number */ +.codehilite .s { background-color: #fff0f0 } /* Literal.String */ +.codehilite .na { color: #0000CC } /* Name.Attribute */ +.codehilite .nb { color: #007020 } /* Name.Builtin */ +.codehilite .nc { color: #BB0066; font-weight: bold } /* Name.Class */ +.codehilite .no { color: #003366; font-weight: bold } /* Name.Constant */ +.codehilite .nd { color: #555555; font-weight: bold } /* Name.Decorator */ +.codehilite .ni { color: #880000; font-weight: bold } /* Name.Entity */ +.codehilite .ne { color: #FF0000; font-weight: bold } /* Name.Exception */ +.codehilite .nf { color: #0066BB; font-weight: bold } /* Name.Function */ +.codehilite .nl { color: #997700; font-weight: bold } /* Name.Label */ +.codehilite .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */ +.codehilite .nt { color: #007700 } /* Name.Tag */ +.codehilite .nv { color: #996633 } /* Name.Variable */ +.codehilite .ow { color: #000000; font-weight: bold } /* Operator.Word */ +.codehilite .w { color: #bbbbbb } /* Text.Whitespace */ +.codehilite .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */ +.codehilite .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */ +.codehilite .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */ +.codehilite .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */ +.codehilite .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */ +.codehilite .sb { background-color: #fff0f0 } /* Literal.String.Backtick */ +.codehilite .sc { color: #0044DD } /* Literal.String.Char */ +.codehilite .sd { color: #DD4422 } /* Literal.String.Doc */ +.codehilite .s2 { background-color: #fff0f0 } /* Literal.String.Double */ +.codehilite .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */ +.codehilite .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */ +.codehilite .si { background-color: #eeeeee } /* Literal.String.Interpol */ +.codehilite .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */ +.codehilite .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */ +.codehilite .s1 { background-color: #fff0f0 } /* Literal.String.Single */ +.codehilite .ss { color: #AA6600 } /* Literal.String.Symbol */ +.codehilite .bp { color: #007020 } /* Name.Builtin.Pseudo */ +.codehilite .vc { color: #336699 } /* Name.Variable.Class */ +.codehilite .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */ +.codehilite .vi { color: #3333BB } /* Name.Variable.Instance */ +.codehilite .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */ diff --git a/resources/session07/learning_journal/learning_journal/static/theme.css b/resources/session07/learning_journal/learning_journal/static/theme.css new file mode 100644 index 00000000..be50ad42 --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/static/theme.css @@ -0,0 +1,152 @@ +@import url(//fonts.googleapis.com/css?family=Open+Sans:300,400,600,700); +body { + font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; + font-weight: 300; + color: #ffffff; + background: #bc2131; +} +h1, +h2, +h3, +h4, +h5, +h6 { + font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; + font-weight: 300; +} +p { + font-weight: 300; +} +.font-normal { + font-weight: 400; +} +.font-semi-bold { + font-weight: 600; +} +.font-bold { + font-weight: 700; +} +.starter-template { + margin-top: 250px; +} +.starter-template .content { + margin-left: 10px; +} +.starter-template .content h1 { + margin-top: 10px; + font-size: 60px; +} +.starter-template .content h1 .smaller { + font-size: 40px; + color: #f2b7bd; +} +.starter-template .content .lead { + font-size: 25px; + color: #f2b7bd; +} +.starter-template .content .lead .font-normal { + color: #ffffff; +} +.starter-template .links { + float: right; + right: 0; + margin-top: 125px; +} +.starter-template .links ul { + display: block; + padding: 0; + margin: 0; +} +.starter-template .links ul li { + list-style: none; + display: inline; + margin: 0 10px; +} +.starter-template .links ul li:first-child { + margin-left: 0; +} +.starter-template .links ul li:last-child { + margin-right: 0; +} +.starter-template .links ul li.current-version { + color: #f2b7bd; + font-weight: 400; +} +.starter-template .links ul li a { + color: #ffffff; +} +.starter-template .links ul li a:hover { + text-decoration: underline; +} +.starter-template .links ul li .icon-muted { + color: #eb8b95; + margin-right: 5px; +} +.starter-template .links ul li:hover .icon-muted { + color: #ffffff; +} +.starter-template .copyright { + margin-top: 10px; + font-size: 0.9em; + color: #f2b7bd; + text-transform: lowercase; + float: right; + right: 0; +} +@media (max-width: 1199px) { + .starter-template .content h1 { + font-size: 45px; + } + .starter-template .content h1 .smaller { + font-size: 30px; + } + .starter-template .content .lead { + font-size: 20px; + } +} +@media (max-width: 991px) { + .starter-template { + margin-top: 0; + } + .starter-template .logo { + margin: 40px auto; + } + .starter-template .content { + margin-left: 0; + text-align: center; + } + .starter-template .content h1 { + margin-bottom: 20px; + } + .starter-template .links { + float: none; + text-align: center; + margin-top: 60px; + } + .starter-template .copyright { + float: none; + text-align: center; + } +} +@media (max-width: 767px) { + .starter-template .content h1 .smaller { + font-size: 25px; + display: block; + } + .starter-template .content .lead { + font-size: 16px; + } + .starter-template .links { + margin-top: 40px; + } + .starter-template .links ul li { + display: block; + margin: 0; + } + .starter-template .links ul li .icon-muted { + display: none; + } + .starter-template .copyright { + margin-top: 20px; + } +} diff --git a/resources/session07/learning_journal/learning_journal/static/theme.min.css b/resources/session07/learning_journal/learning_journal/static/theme.min.css new file mode 100644 index 00000000..2f924bcc --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/static/theme.min.css @@ -0,0 +1 @@ +@import url(//fonts.googleapis.com/css?family=Open+Sans:300,400,600,700);body{font-family:"Open Sans","Helvetica Neue",Helvetica,Arial,sans-serif;font-weight:300;color:#fff;background:#bc2131}h1,h2,h3,h4,h5,h6{font-family:"Open Sans","Helvetica Neue",Helvetica,Arial,sans-serif;font-weight:300}p{font-weight:300}.font-normal{font-weight:400}.font-semi-bold{font-weight:600}.font-bold{font-weight:700}.starter-template{margin-top:250px}.starter-template .content{margin-left:10px}.starter-template .content h1{margin-top:10px;font-size:60px}.starter-template .content h1 .smaller{font-size:40px;color:#f2b7bd}.starter-template .content .lead{font-size:25px;color:#f2b7bd}.starter-template .content .lead .font-normal{color:#fff}.starter-template .links{float:right;right:0;margin-top:125px}.starter-template .links ul{display:block;padding:0;margin:0}.starter-template .links ul li{list-style:none;display:inline;margin:0 10px}.starter-template .links ul li:first-child{margin-left:0}.starter-template .links ul li:last-child{margin-right:0}.starter-template .links ul li.current-version{color:#f2b7bd;font-weight:400}.starter-template .links ul li a{color:#fff}.starter-template .links ul li a:hover{text-decoration:underline}.starter-template .links ul li .icon-muted{color:#eb8b95;margin-right:5px}.starter-template .links ul li:hover .icon-muted{color:#fff}.starter-template .copyright{margin-top:10px;font-size:.9em;color:#f2b7bd;text-transform:lowercase;float:right;right:0}@media (max-width:1199px){.starter-template .content h1{font-size:45px}.starter-template .content h1 .smaller{font-size:30px}.starter-template .content .lead{font-size:20px}}@media (max-width:991px){.starter-template{margin-top:0}.starter-template .logo{margin:40px auto}.starter-template .content{margin-left:0;text-align:center}.starter-template .content h1{margin-bottom:20px}.starter-template .links{float:none;text-align:center;margin-top:60px}.starter-template .copyright{float:none;text-align:center}}@media (max-width:767px){.starter-template .content h1 .smaller{font-size:25px;display:block}.starter-template .content .lead{font-size:16px}.starter-template .links{margin-top:40px}.starter-template .links ul li{display:block;margin:0}.starter-template .links ul li .icon-muted{display:none}.starter-template .copyright{margin-top:20px}} \ No newline at end of file diff --git a/resources/session07/learning_journal/learning_journal/templates/detail.jinja2 b/resources/session07/learning_journal/learning_journal/templates/detail.jinja2 new file mode 100644 index 00000000..8f966471 --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/templates/detail.jinja2 @@ -0,0 +1,18 @@ +{% extends "layout.jinja2" %} +{% block body %} +
+

{{ entry.title }}

+
+

{{ entry.body|markdown }}

+
+

Created {{entry.created}}

+
+

+ Go Back + {% if logged_in %} + :: + + Edit Entry + {% endif %} +

+{% endblock %} diff --git a/resources/session07/learning_journal/learning_journal/templates/edit.jinja2 b/resources/session07/learning_journal/learning_journal/templates/edit.jinja2 new file mode 100644 index 00000000..ebe0f6b9 --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/templates/edit.jinja2 @@ -0,0 +1,17 @@ +{% extends "templates/layout.jinja2" %} +{% block body %} +

Create a Journal Entry

+
+{% for field in form %} + {% if field.errors %} +
    + {% for error in field.errors %} +
  • {{ error }}
  • + {% endfor %} +
+ {% endif %} +

{{ field.label }}: {{ field }}

+{% endfor %} +

+
+{% endblock %} diff --git a/resources/session07/learning_journal/learning_journal/templates/layout.jinja2 b/resources/session07/learning_journal/learning_journal/templates/layout.jinja2 new file mode 100644 index 00000000..a8b27afd --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/templates/layout.jinja2 @@ -0,0 +1,29 @@ + + + + + Python Learning Journal + + + + +
+ +
+
+

My Python Journal

+
+ {% block body %}{% endblock %} +
+
+
+

Created in the UW PCE Python Certificate Program

+
+ + diff --git a/resources/session07/learning_journal/learning_journal/templates/list.jinja2 b/resources/session07/learning_journal/learning_journal/templates/list.jinja2 new file mode 100644 index 00000000..7f1e4795 --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/templates/list.jinja2 @@ -0,0 +1,31 @@ +{% extends "layout.jinja2" %} +{% block body %} +{% if login_form %} + +{% endif %} +{% if entries %} +

Journal Entries

+ +{% else %} +

This journal is empty

+{% endif %} +{% if not login_form %} +

New Entry

+{% endif %} +{% endblock %} diff --git a/resources/session07/learning_journal/learning_journal/templates/mytemplate.pt b/resources/session07/learning_journal/learning_journal/templates/mytemplate.pt new file mode 100644 index 00000000..9e88dc4b --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/templates/mytemplate.pt @@ -0,0 +1,66 @@ + + + + + + + + + + + Alchemy Scaffold for The Pyramid Web Framework + + + + + + + + + + + + + +
+
+
+
+ +
+
+
+

Pyramid Alchemy scaffold

+

Welcome to ${project}, an application generated by
the Pyramid Web Framework 1.5.2.

+
+
+
+
+ +
+
+ +
+
+
+ + + + + + + + diff --git a/resources/session07/learning_journal/learning_journal/tests.py b/resources/session07/learning_journal/learning_journal/tests.py new file mode 100644 index 00000000..4fc444a6 --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/tests.py @@ -0,0 +1,55 @@ +import unittest +import transaction + +from pyramid import testing + +from .models import DBSession + + +class TestMyViewSuccessCondition(unittest.TestCase): + def setUp(self): + self.config = testing.setUp() + from sqlalchemy import create_engine + engine = create_engine('sqlite://') + from .models import ( + Base, + MyModel, + ) + DBSession.configure(bind=engine) + Base.metadata.create_all(engine) + with transaction.manager: + model = MyModel(name='one', value=55) + DBSession.add(model) + + def tearDown(self): + DBSession.remove() + testing.tearDown() + + def test_passing_view(self): + from .views import my_view + request = testing.DummyRequest() + info = my_view(request) + self.assertEqual(info['one'].name, 'one') + self.assertEqual(info['project'], 'learning_journal') + + +class TestMyViewFailureCondition(unittest.TestCase): + def setUp(self): + self.config = testing.setUp() + from sqlalchemy import create_engine + engine = create_engine('sqlite://') + from .models import ( + Base, + MyModel, + ) + DBSession.configure(bind=engine) + + def tearDown(self): + DBSession.remove() + testing.tearDown() + + def test_failing_view(self): + from .views import my_view + request = testing.DummyRequest() + info = my_view(request) + self.assertEqual(info.status_int, 500) diff --git a/resources/session07/learning_journal/learning_journal/views.py b/resources/session07/learning_journal/learning_journal/views.py new file mode 100644 index 00000000..d6248a0b --- /dev/null +++ b/resources/session07/learning_journal/learning_journal/views.py @@ -0,0 +1,94 @@ +from jinja2 import Markup +import markdown +from pyramid.httpexceptions import HTTPFound, HTTPNotFound +from pyramid.security import forget, remember, authenticated_userid +from pyramid.view import view_config + +from .models import ( + DBSession, + MyModel, + Entry, + User + ) + +from .forms import ( + EntryCreateForm, + EntryEditForm, + LoginForm +) + + +@view_config(route_name='home', renderer='templates/list.jinja2') +def index_page(request): + entries = Entry.all() + form = None + if not authenticated_userid(request): + form = LoginForm() + return {'entries': entries, 'login_form': form} + + +@view_config(route_name='detail', renderer='templates/detail.jinja2') +def view(request): + this_id = request.matchdict.get('id', -1) + entry = Entry.by_id(this_id) + if not entry: + return HTTPNotFound() + logged_in = authenticated_userid(request) + return {'entry': entry, 'logged_in': logged_in} + + +@view_config(route_name='action', match_param='action=create', + renderer='templates/edit.jinja2', + permission='create') +def create(request): + entry = Entry() + form = EntryCreateForm(request.POST) + if request.method == 'POST' and form.validate(): + form.populate_obj(entry) + DBSession.add(entry) + return HTTPFound(location=request.route_url('home')) + return {'form': form, 'action': request.matchdict.get('action')} + + +@view_config(route_name='action', match_param='action=edit', + renderer='templates/edit.jinja2', + permission='edit') +def update(request): + id = int(request.params.get('id', -1)) + entry = Entry.by_id(id) + if not entry: + return HTTPNotFound() + form = EntryEditForm(request.POST, entry) + if request.method == 'POST' and form.validate(): + form.populate_obj(entry) + return HTTPFound(location=request.route_url('detail', id=entry.id)) + return {'form': form, 'action': request.matchdict.get('action')} + + +@view_config(route_name='auth', match_param='action=in', renderer='string', + request_method='POST') +@view_config(route_name='auth', match_param='action=out', renderer='string') +def sign_in(request): + login_form = None + if request.method == 'POST': + login_form = LoginForm(request.POST) + if login_form and login_form.validate(): + user = User.by_name(login_form.username.data) + if user and user.verify_password(login_form.password.data): + headers = remember(request, user.name) + else: + headers = forget(request) + else: + headers = forget(request) + return HTTPFound(location=request.route_url('home'), + headers=headers) + + +def render_markdown(content): + output = Markup( + markdown.markdown( + content, + extensions=['codehilite(pygments_style=colorful)', 'fenced_code'] + ) + ) + return output diff --git a/resources/session07/learning_journal/production.ini b/resources/session07/learning_journal/production.ini new file mode 100644 index 00000000..d203746a --- /dev/null +++ b/resources/session07/learning_journal/production.ini @@ -0,0 +1,66 @@ +### +# app configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/environment.html +### + +[app:main] +use = egg:learning_journal + + +pyramid.reload_templates = false +pyramid.debug_authorization = false +pyramid.debug_notfound = false +pyramid.debug_routematch = false +pyramid.default_locale_name = en +pyramid.includes = + pyramid_tm + +sqlalchemy.url = sqlite:///%(here)s/learning_journal.sqlite + +jinja2.filters = + markdown = learning_journal.views.render_markdown + +[server:main] +use = egg:waitress#main +host = 0.0.0.0 +port = 6543 + +### +# logging configuration +# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/logging.html +### + +[loggers] +keys = root, learning_journal, sqlalchemy + +[handlers] +keys = console + +[formatters] +keys = generic + +[logger_root] +level = WARN +handlers = console + +[logger_learning_journal] +level = WARN +handlers = +qualname = learning_journal + +[logger_sqlalchemy] +level = WARN +handlers = +qualname = sqlalchemy.engine +# "level = INFO" logs SQL queries. +# "level = DEBUG" logs SQL queries and results. +# "level = WARN" logs neither. (Recommended for production systems.) + +[handler_console] +class = StreamHandler +args = (sys.stderr,) +level = NOTSET +formatter = generic + +[formatter_generic] +format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s diff --git a/resources/session07/learning_journal/requirements.txt b/resources/session07/learning_journal/requirements.txt new file mode 100644 index 00000000..871e054a --- /dev/null +++ b/resources/session07/learning_journal/requirements.txt @@ -0,0 +1,34 @@ +appnope==0.1.0 +decorator==4.0.6 +ipython==4.0.1 +ipython-genutils==0.1.0 +Jinja2==2.8 +Mako==1.0.3 +Markdown==2.6.5 +MarkupSafe==0.23 +passlib==1.6.5 +PasteDeploy==1.5.2 +path.py==8.1.2 +pexpect==4.0.1 +pickleshare==0.5 +psycopg2==2.6.1 +ptyprocess==0.5 +Pygments==2.0.2 +pyramid==1.5.7 +pyramid-debugtoolbar==2.4.2 +pyramid-jinja2==2.5 +pyramid-mako==1.0.2 +pyramid-tm==0.12.1 +repoze.lru==0.6 +simplegeneric==0.8.1 +SQLAlchemy==1.0.11 +traitlets==4.0.0 +transaction==1.4.4 +translationstring==1.3 +venusian==1.0 +waitress==0.8.10 +WebOb==1.5.1 +WTForms==2.1 +zope.deprecation==4.1.2 +zope.interface==4.1.3 +zope.sqlalchemy==0.7.6 diff --git a/resources/session07/learning_journal/run b/resources/session07/learning_journal/run new file mode 100755 index 00000000..3689ebe2 --- /dev/null +++ b/resources/session07/learning_journal/run @@ -0,0 +1,3 @@ +#!/bin/bash +python setup.py develop +python runapp.py diff --git a/resources/session07/learning_journal/runapp.py b/resources/session07/learning_journal/runapp.py new file mode 100644 index 00000000..df24540c --- /dev/null +++ b/resources/session07/learning_journal/runapp.py @@ -0,0 +1,10 @@ +import os + +from paste.deploy import loadapp +from waitress import serve + +if __name__ == "__main__": + port = int(os.environ.get("PORT", 5000)) + app = loadapp('config:production.ini', relative_to='.') + + serve(app, host='0.0.0.0', port=port) diff --git a/resources/session07/learning_journal/runtime.txt b/resources/session07/learning_journal/runtime.txt new file mode 100644 index 00000000..294a23e9 --- /dev/null +++ b/resources/session07/learning_journal/runtime.txt @@ -0,0 +1 @@ +python-3.5.0 diff --git a/resources/session07/learning_journal/setup.py b/resources/session07/learning_journal/setup.py new file mode 100644 index 00000000..f681f70b --- /dev/null +++ b/resources/session07/learning_journal/setup.py @@ -0,0 +1,51 @@ +import os + +from setuptools import setup, find_packages + +here = os.path.abspath(os.path.dirname(__file__)) +with open(os.path.join(here, 'README.txt')) as f: + README = f.read() +with open(os.path.join(here, 'CHANGES.txt')) as f: + CHANGES = f.read() + +requires = [ + 'pyramid', + 'pyramid_jinja2', + 'pyramid_debugtoolbar', + 'pyramid_tm', + 'SQLAlchemy', + 'transaction', + 'zope.sqlalchemy', + 'waitress', + 'wtforms', + 'passlib', + 'markdown', + 'pygments', + ] + +setup(name='learning_journal', + version='0.0', + description='learning_journal', + long_description=README + '\n\n' + CHANGES, + classifiers=[ + "Programming Language :: Python", + "Framework :: Pyramid", + "Topic :: Internet :: WWW/HTTP", + "Topic :: Internet :: WWW/HTTP :: WSGI :: Application", + ], + author='', + author_email='', + url='', + keywords='web wsgi bfg pylons pyramid', + packages=find_packages(), + include_package_data=True, + zip_safe=False, + test_suite='learning_journal', + install_requires=requires, + entry_points="""\ + [paste.app_factory] + main = learning_journal:main + [console_scripts] + initialize_learning_journal_db = learning_journal.scripts.initializedb:main + """, + ) diff --git a/resources/session07/models.py b/resources/session07/models.py new file mode 100644 index 00000000..a5625056 --- /dev/null +++ b/resources/session07/models.py @@ -0,0 +1,72 @@ +import datetime +from sqlalchemy import ( + Column, + DateTime, + Index, + Integer, + Text, + Unicode, + UnicodeText, + ) + +import sqlalchemy as sa +from sqlalchemy.ext.declarative import declarative_base + +from sqlalchemy.orm import ( + scoped_session, + sessionmaker, + ) + +from zope.sqlalchemy import ZopeTransactionExtension + +DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension())) +Base = declarative_base() + + +class MyModel(Base): + __tablename__ = 'models' + id = Column(Integer, primary_key=True) + name = Column(Text) + value = Column(Integer) + +Index('my_index', MyModel.name, unique=True, mysql_length=255) + + +class Entry(Base): + __tablename__ = 'entries' + id = Column(Integer, primary_key=True) + title = Column(Unicode(255), unique=True, nullable=False) + body = Column(UnicodeText, default=u'') + created = Column(DateTime, default=datetime.datetime.utcnow) + edited = Column(DateTime, default=datetime.datetime.utcnow) + + @classmethod + def all(cls, session=None): + """return a query with all entries, ordered by creation date reversed + """ + if session is None: + session = DBSession + return session.query(cls).order_by(sa.desc(cls.created)).all() + + @classmethod + def by_id(cls, id, session=None): + """return a single entry identified by id + + If no entry exists with the provided id, return None + """ + if session is None: + session = DBSession + return session.query(cls).get(id) + + +class User(Base): + __tablename__ = 'users' + id = Column(Integer, primary_key=True, autoincrement=True) + name = Column(Unicode(255), unique=True, nullable=False) + password = Column(Unicode(255), nullable=False) + + @classmethod + def by_name(cls, name, session=None): + if session is None: + session = DBSession + return DBSession.query(cls).filter(cls.name == name).first() diff --git a/resources/session07/views.py b/resources/session07/views.py new file mode 100644 index 00000000..35e37963 --- /dev/null +++ b/resources/session07/views.py @@ -0,0 +1,54 @@ +from pyramid.httpexceptions import HTTPFound, HTTPNotFound +from pyramid.view import view_config + +from .models import ( + DBSession, + MyModel, + Entry, + ) + +from .forms import ( + EntryCreateForm, + EntryEditForm, +) + + +@view_config(route_name='home', renderer='templates/list.jinja2') +def index_page(request): + entries = Entry.all() + return {'entries': entries} + + +@view_config(route_name='detail', renderer='templates/detail.jinja2') +def view(request): + this_id = request.matchdict.get('id', -1) + entry = Entry.by_id(this_id) + if not entry: + return HTTPNotFound() + return {'entry': entry} + + +@view_config(route_name='action', match_param='action=create', + renderer='templates/edit.jinja2') +def create(request): + entry = Entry() + form = EntryCreateForm(request.POST) + if request.method == 'POST' and form.validate(): + form.populate_obj(entry) + DBSession.add(entry) + return HTTPFound(location=request.route_url('home')) + return {'form': form, 'action': request.matchdict.get('action')} + + +@view_config(route_name='action', match_param='action=edit', + renderer='templates/edit.jinja2') +def update(request): + id = int(request.params.get('id', -1)) + entry = Entry.by_id(id) + if not entry: + return HTTPNotFound() + form = EntryEditForm(request.POST, entry) + if request.method == 'POST' and form.validate(): + form.populate_obj(entry) + return HTTPFound(location=request.route_url('detail', id=entry.id)) + return {'form': form, 'action': request.matchdict.get('action')} diff --git a/resources/session08/django_blog.css b/resources/session08/django_blog.css new file mode 100644 index 00000000..45a882de --- /dev/null +++ b/resources/session08/django_blog.css @@ -0,0 +1,74 @@ +body { + background-color: #eee; + color: #111; + font-family: "HelveticaNeue-Light", "Helvetica Neue Light", "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif; + margin:0; + padding:0; +} +#container { + margin:0; + padding:0; + margin-top: 0px; +} +#header { + background-color: #333; + border-botton: 1px solid #111; + margin:0; + padding:0; +} +#control-bar { + margin: 0em 0em 1em; + list-style: none; + list-style-type: none; + text-align: right; + color: #eee; + font-size: 80%; + padding-bottom: 0.4em; +} +#control-bar li { + display: inline-block; +} +#control-bar li a { + color: #eee; + padding: 0.5em; + text-decoration: none; +} +#control-bar li a:hover { + color: #cce; +} +#content { + margin: 0em 1em 1em; +} + +ul#entries { + list-style: none; + list-style-type: none; +} +div.entry { + margin-right: 2em; + margin-top: 1em; + border-top: 1px solid #cecece; +} +ul#entries li:first-child div.entry { + border-top: none; + margin-top: 0em; +} +div.entry-body { + margin-left: 2em; +} +.notification { + float: right; + text-align: center; + width: 25%; + padding: 1em; +} +.info { + background-color: #aae; +} +ul.categories { + list-style: none; + list-style-type: none; +} +ul.categories li { + display: inline; +} diff --git a/resources/session08/myblog_test_fixture.json b/resources/session08/myblog_test_fixture.json new file mode 100644 index 00000000..bf5269e9 --- /dev/null +++ b/resources/session08/myblog_test_fixture.json @@ -0,0 +1,38 @@ +[ + { + "pk": 1, + "model": "auth.user", + "fields": { + "username": "admin", + "first_name": "Mr.", + "last_name": "Administrator", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "admin@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + }, + { + "pk": 2, + "model": "auth.user", + "fields": { + "username": "noname", + "first_name": "", + "last_name": "", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "noname@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + } +] diff --git a/resources/session08/mysite_stage_1/manage.py b/resources/session08/mysite_stage_1/manage.py new file mode 100755 index 00000000..8a50ec04 --- /dev/null +++ b/resources/session08/mysite_stage_1/manage.py @@ -0,0 +1,10 @@ +#!/usr/bin/env python +import os +import sys + +if __name__ == "__main__": + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") + + from django.core.management import execute_from_command_line + + execute_from_command_line(sys.argv) diff --git a/resources/session08/mysite_stage_1/myblog/__init__.py b/resources/session08/mysite_stage_1/myblog/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session08/mysite_stage_1/myblog/admin.py b/resources/session08/mysite_stage_1/myblog/admin.py new file mode 100644 index 00000000..310e7294 --- /dev/null +++ b/resources/session08/mysite_stage_1/myblog/admin.py @@ -0,0 +1,8 @@ +from django.contrib import admin + +from myblog.models import Category +from myblog.models import Post + + +admin.site.register(Category) +admin.site.register(Post) diff --git a/resources/session08/mysite_stage_1/myblog/apps.py b/resources/session08/mysite_stage_1/myblog/apps.py new file mode 100644 index 00000000..5e29c8d9 --- /dev/null +++ b/resources/session08/mysite_stage_1/myblog/apps.py @@ -0,0 +1,5 @@ +from django.apps import AppConfig + + +class MyblogConfig(AppConfig): + name = 'myblog' diff --git a/resources/session08/mysite_stage_1/myblog/fixtures/myblog_test_fixture.json b/resources/session08/mysite_stage_1/myblog/fixtures/myblog_test_fixture.json new file mode 100644 index 00000000..bf5269e9 --- /dev/null +++ b/resources/session08/mysite_stage_1/myblog/fixtures/myblog_test_fixture.json @@ -0,0 +1,38 @@ +[ + { + "pk": 1, + "model": "auth.user", + "fields": { + "username": "admin", + "first_name": "Mr.", + "last_name": "Administrator", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "admin@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + }, + { + "pk": 2, + "model": "auth.user", + "fields": { + "username": "noname", + "first_name": "", + "last_name": "", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "noname@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + } +] diff --git a/resources/session08/mysite_stage_1/myblog/migrations/0001_initial.py b/resources/session08/mysite_stage_1/myblog/migrations/0001_initial.py new file mode 100644 index 00000000..18d659f1 --- /dev/null +++ b/resources/session08/mysite_stage_1/myblog/migrations/0001_initial.py @@ -0,0 +1,31 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.9 on 2015-12-31 19:13 +from __future__ import unicode_literals + +from django.conf import settings +from django.db import migrations, models +import django.db.models.deletion + + +class Migration(migrations.Migration): + + initial = True + + dependencies = [ + migrations.swappable_dependency(settings.AUTH_USER_MODEL), + ] + + operations = [ + migrations.CreateModel( + name='Post', + fields=[ + ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), + ('title', models.CharField(max_length=128)), + ('text', models.TextField(blank=True)), + ('created_date', models.DateTimeField(auto_now_add=True)), + ('modified_date', models.DateTimeField(auto_now=True)), + ('published_date', models.DateTimeField(blank=True, null=True)), + ('author', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)), + ], + ), + ] diff --git a/resources/session08/mysite_stage_1/myblog/migrations/0002_category.py b/resources/session08/mysite_stage_1/myblog/migrations/0002_category.py new file mode 100644 index 00000000..218b1052 --- /dev/null +++ b/resources/session08/mysite_stage_1/myblog/migrations/0002_category.py @@ -0,0 +1,24 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.9 on 2015-12-31 21:40 +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('myblog', '0001_initial'), + ] + + operations = [ + migrations.CreateModel( + name='Category', + fields=[ + ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), + ('name', models.CharField(max_length=128)), + ('description', models.TextField(blank=True)), + ('posts', models.ManyToManyField(blank=True, related_name='categories', to='myblog.Post')), + ], + ), + ] diff --git a/resources/session08/mysite_stage_1/myblog/migrations/__init__.py b/resources/session08/mysite_stage_1/myblog/migrations/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session08/mysite_stage_1/myblog/models.py b/resources/session08/mysite_stage_1/myblog/models.py new file mode 100644 index 00000000..cef1beac --- /dev/null +++ b/resources/session08/mysite_stage_1/myblog/models.py @@ -0,0 +1,27 @@ +from django.db import models +from django.contrib.auth.models import User + + +class Post(models.Model): + title = models.CharField(max_length=128) + text = models.TextField(blank=True) + author = models.ForeignKey(User) + created_date = models.DateTimeField(auto_now_add=True) + modified_date = models.DateTimeField(auto_now=True) + published_date = models.DateTimeField(blank=True, null=True) + + def __str__(self): + return self.title + + +class Category(models.Model): + name = models.CharField(max_length=128) + description = models.TextField(blank=True) + posts = models.ManyToManyField( + Post, + blank=True, + related_name='categories' + ) + + def __str__(self): + return self.name diff --git a/resources/session08/mysite_stage_1/myblog/tests.py b/resources/session08/mysite_stage_1/myblog/tests.py new file mode 100644 index 00000000..308dd6f1 --- /dev/null +++ b/resources/session08/mysite_stage_1/myblog/tests.py @@ -0,0 +1,27 @@ +from django.test import TestCase +from django.contrib.auth.models import User + +from myblog.models import Category +from myblog.models import Post + + +class PostTestCase(TestCase): + fixtures = ['myblog_test_fixture.json'] + + def setUp(self): + self.user = User.objects.get(pk=1) + + def test_string_representation(self): + expected = "This is a title" + p1 = Post(title=expected) + actual = str(p1) + self.assertEqual(expected, actual) + + +class CategoryTestCase(TestCase): + + def test_string_representation(self): + expected = "A Category" + c1 = Category(name=expected) + actual = str(c1) + self.assertEqual(expected, actual) diff --git a/resources/session08/mysite_stage_1/myblog/views.py b/resources/session08/mysite_stage_1/myblog/views.py new file mode 100644 index 00000000..91ea44a2 --- /dev/null +++ b/resources/session08/mysite_stage_1/myblog/views.py @@ -0,0 +1,3 @@ +from django.shortcuts import render + +# Create your views here. diff --git a/resources/session08/mysite_stage_1/mysite/__init__.py b/resources/session08/mysite_stage_1/mysite/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session08/mysite_stage_1/mysite/settings.py b/resources/session08/mysite_stage_1/mysite/settings.py new file mode 100644 index 00000000..3753c661 --- /dev/null +++ b/resources/session08/mysite_stage_1/mysite/settings.py @@ -0,0 +1,122 @@ +""" +Django settings for mysite project. + +Generated by 'django-admin startproject' using Django 1.9. + +For more information on this file, see +https://docs.djangoproject.com/en/1.9/topics/settings/ + +For the full list of settings and their values, see +https://docs.djangoproject.com/en/1.9/ref/settings/ +""" + +import os + +# Build paths inside the project like this: os.path.join(BASE_DIR, ...) +BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + + +# Quick-start development settings - unsuitable for production +# See https://docs.djangoproject.com/en/1.9/howto/deployment/checklist/ + +# SECURITY WARNING: keep the secret key used in production secret! +SECRET_KEY = 'i=n^tc%@@gq#8ev6dlymy9+-%@^f!q54sjf0rvikt_k5bl(t1=' + +# SECURITY WARNING: don't run with debug turned on in production! +DEBUG = True + +ALLOWED_HOSTS = [] + + +# Application definition + +INSTALLED_APPS = [ + 'django.contrib.admin', + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.messages', + 'django.contrib.staticfiles', + 'myblog', +] + +MIDDLEWARE_CLASSES = [ + 'django.middleware.security.SecurityMiddleware', + 'django.contrib.sessions.middleware.SessionMiddleware', + 'django.middleware.common.CommonMiddleware', + 'django.middleware.csrf.CsrfViewMiddleware', + 'django.contrib.auth.middleware.AuthenticationMiddleware', + 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', + 'django.contrib.messages.middleware.MessageMiddleware', + 'django.middleware.clickjacking.XFrameOptionsMiddleware', +] + +ROOT_URLCONF = 'mysite.urls' + +TEMPLATES = [ + { + 'BACKEND': 'django.template.backends.django.DjangoTemplates', + 'DIRS': [], + 'APP_DIRS': True, + 'OPTIONS': { + 'context_processors': [ + 'django.template.context_processors.debug', + 'django.template.context_processors.request', + 'django.contrib.auth.context_processors.auth', + 'django.contrib.messages.context_processors.messages', + ], + }, + }, +] + +WSGI_APPLICATION = 'mysite.wsgi.application' + + +# Database +# https://docs.djangoproject.com/en/1.9/ref/settings/#databases + +DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.sqlite3', + 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), + } +} + + +# Password validation +# https://docs.djangoproject.com/en/1.9/ref/settings/#auth-password-validators + +AUTH_PASSWORD_VALIDATORS = [ + { + 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', + }, + { + 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', + }, + { + 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', + }, + { + 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', + }, +] + + +# Internationalization +# https://docs.djangoproject.com/en/1.9/topics/i18n/ + +LANGUAGE_CODE = 'en-us' + +TIME_ZONE = 'UTC' + +USE_I18N = True + +USE_L10N = True + +USE_TZ = True + + +# Static files (CSS, JavaScript, Images) +# https://docs.djangoproject.com/en/1.9/howto/static-files/ + +STATIC_URL = '/static/' diff --git a/resources/session08/mysite_stage_1/mysite/urls.py b/resources/session08/mysite_stage_1/mysite/urls.py new file mode 100644 index 00000000..40cce1f4 --- /dev/null +++ b/resources/session08/mysite_stage_1/mysite/urls.py @@ -0,0 +1,22 @@ +"""mysite URL Configuration + +The `urlpatterns` list routes URLs to views. For more information please see: + https://docs.djangoproject.com/en/1.9/topics/http/urls/ +Examples: +Function views + 1. Add an import: from my_app import views + 2. Add a URL to urlpatterns: url(r'^$', views.home, name='home') +Class-based views + 1. Add an import: from other_app.views import Home + 2. Add a URL to urlpatterns: url(r'^$', Home.as_view(), name='home') +Including another URLconf + 1. Add an import: from blog import urls as blog_urls + 2. Import the include() function: from django.conf.urls import url, include + 3. Add a URL to urlpatterns: url(r'^blog/', include(blog_urls)) +""" +from django.conf.urls import url +from django.contrib import admin + +urlpatterns = [ + url(r'^admin/', admin.site.urls), +] diff --git a/resources/session08/mysite_stage_1/mysite/wsgi.py b/resources/session08/mysite_stage_1/mysite/wsgi.py new file mode 100644 index 00000000..328bae0f --- /dev/null +++ b/resources/session08/mysite_stage_1/mysite/wsgi.py @@ -0,0 +1,16 @@ +""" +WSGI config for mysite project. + +It exposes the WSGI callable as a module-level variable named ``application``. + +For more information on this file, see +https://docs.djangoproject.com/en/1.9/howto/deployment/wsgi/ +""" + +import os + +from django.core.wsgi import get_wsgi_application + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") + +application = get_wsgi_application() diff --git a/resources/session08/mysite_stage_2/manage.py b/resources/session08/mysite_stage_2/manage.py new file mode 100755 index 00000000..8a50ec04 --- /dev/null +++ b/resources/session08/mysite_stage_2/manage.py @@ -0,0 +1,10 @@ +#!/usr/bin/env python +import os +import sys + +if __name__ == "__main__": + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") + + from django.core.management import execute_from_command_line + + execute_from_command_line(sys.argv) diff --git a/resources/session08/mysite_stage_2/myblog/__init__.py b/resources/session08/mysite_stage_2/myblog/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session08/mysite_stage_2/myblog/admin.py b/resources/session08/mysite_stage_2/myblog/admin.py new file mode 100644 index 00000000..310e7294 --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/admin.py @@ -0,0 +1,8 @@ +from django.contrib import admin + +from myblog.models import Category +from myblog.models import Post + + +admin.site.register(Category) +admin.site.register(Post) diff --git a/resources/session08/mysite_stage_2/myblog/apps.py b/resources/session08/mysite_stage_2/myblog/apps.py new file mode 100644 index 00000000..5e29c8d9 --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/apps.py @@ -0,0 +1,5 @@ +from django.apps import AppConfig + + +class MyblogConfig(AppConfig): + name = 'myblog' diff --git a/resources/session08/mysite_stage_2/myblog/fixtures/myblog_test_fixture.json b/resources/session08/mysite_stage_2/myblog/fixtures/myblog_test_fixture.json new file mode 100644 index 00000000..bf5269e9 --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/fixtures/myblog_test_fixture.json @@ -0,0 +1,38 @@ +[ + { + "pk": 1, + "model": "auth.user", + "fields": { + "username": "admin", + "first_name": "Mr.", + "last_name": "Administrator", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "admin@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + }, + { + "pk": 2, + "model": "auth.user", + "fields": { + "username": "noname", + "first_name": "", + "last_name": "", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "noname@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + } +] diff --git a/resources/session08/mysite_stage_2/myblog/migrations/0001_initial.py b/resources/session08/mysite_stage_2/myblog/migrations/0001_initial.py new file mode 100644 index 00000000..18d659f1 --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/migrations/0001_initial.py @@ -0,0 +1,31 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.9 on 2015-12-31 19:13 +from __future__ import unicode_literals + +from django.conf import settings +from django.db import migrations, models +import django.db.models.deletion + + +class Migration(migrations.Migration): + + initial = True + + dependencies = [ + migrations.swappable_dependency(settings.AUTH_USER_MODEL), + ] + + operations = [ + migrations.CreateModel( + name='Post', + fields=[ + ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), + ('title', models.CharField(max_length=128)), + ('text', models.TextField(blank=True)), + ('created_date', models.DateTimeField(auto_now_add=True)), + ('modified_date', models.DateTimeField(auto_now=True)), + ('published_date', models.DateTimeField(blank=True, null=True)), + ('author', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)), + ], + ), + ] diff --git a/resources/session08/mysite_stage_2/myblog/migrations/0002_category.py b/resources/session08/mysite_stage_2/myblog/migrations/0002_category.py new file mode 100644 index 00000000..218b1052 --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/migrations/0002_category.py @@ -0,0 +1,24 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.9 on 2015-12-31 21:40 +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('myblog', '0001_initial'), + ] + + operations = [ + migrations.CreateModel( + name='Category', + fields=[ + ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), + ('name', models.CharField(max_length=128)), + ('description', models.TextField(blank=True)), + ('posts', models.ManyToManyField(blank=True, related_name='categories', to='myblog.Post')), + ], + ), + ] diff --git a/resources/session08/mysite_stage_2/myblog/migrations/__init__.py b/resources/session08/mysite_stage_2/myblog/migrations/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session08/mysite_stage_2/myblog/models.py b/resources/session08/mysite_stage_2/myblog/models.py new file mode 100644 index 00000000..cef1beac --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/models.py @@ -0,0 +1,27 @@ +from django.db import models +from django.contrib.auth.models import User + + +class Post(models.Model): + title = models.CharField(max_length=128) + text = models.TextField(blank=True) + author = models.ForeignKey(User) + created_date = models.DateTimeField(auto_now_add=True) + modified_date = models.DateTimeField(auto_now=True) + published_date = models.DateTimeField(blank=True, null=True) + + def __str__(self): + return self.title + + +class Category(models.Model): + name = models.CharField(max_length=128) + description = models.TextField(blank=True) + posts = models.ManyToManyField( + Post, + blank=True, + related_name='categories' + ) + + def __str__(self): + return self.name diff --git a/resources/session08/mysite_stage_2/myblog/templates/list.html b/resources/session08/mysite_stage_2/myblog/templates/list.html new file mode 100644 index 00000000..e21bc51c --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/templates/list.html @@ -0,0 +1,23 @@ +{% extends "base.html" %} + +{% block content %} +

Recent Posts

+ + {% comment %} here is where the query happens {% endcomment %} + {% for post in posts %} +
+

{{ post }}

+ +
+ {{ post.text }} +
+
    + {% for category in post.categories.all %} +
  • {{ category }}
  • + {% endfor %} +
+
+ {% endfor %} +{% endblock %} diff --git a/resources/session08/mysite_stage_2/myblog/tests.py b/resources/session08/mysite_stage_2/myblog/tests.py new file mode 100644 index 00000000..d9418619 --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/tests.py @@ -0,0 +1,60 @@ +import datetime +from django.contrib.auth.models import User +from django.test import TestCase +from django.utils.timezone import utc + +from myblog.models import Category +from myblog.models import Post + + +class PostTestCase(TestCase): + fixtures = ['myblog_test_fixture.json'] + + def setUp(self): + self.user = User.objects.get(pk=1) + + def test_string_representation(self): + expected = "This is a title" + p1 = Post(title=expected) + actual = str(p1) + self.assertEqual(expected, actual) + + +class CategoryTestCase(TestCase): + + def test_string_representation(self): + expected = "A Category" + c1 = Category(name=expected) + actual = str(c1) + self.assertEqual(expected, actual) + + +class FrontEndTestCase(TestCase): + """test views provided in the front-end""" + fixtures = ['myblog_test_fixture.json', ] + + def setUp(self): + self.now = datetime.datetime.utcnow().replace(tzinfo=utc) + self.timedelta = datetime.timedelta(15) + author = User.objects.get(pk=1) + for count in range(1, 11): + post = Post(title="Post %d Title" % count, + text="foo", + author=author) + if count < 6: + # publish the first five posts + pubdate = self.now - self.timedelta * count + post.published_date = pubdate + post.save() + + def test_list_only_published(self): + resp = self.client.get('/') + # the content of the rendered response is always a bytestring + resp_text = resp.content.decode(resp.charset) + self.assertTrue("Recent Posts" in resp_text) + for count in range(1, 11): + title = "Post %d Title" % count + if count < 6: + self.assertContains(resp, title, count=1) + else: + self.assertNotContains(resp, title) diff --git a/resources/session08/mysite_stage_2/myblog/urls.py b/resources/session08/mysite_stage_2/myblog/urls.py new file mode 100644 index 00000000..8428ffe4 --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/urls.py @@ -0,0 +1,14 @@ +from django.conf.urls import url + +from myblog.views import stub_view +from myblog.views import list_view + + +urlpatterns = [ + url(r'^$', + list_view, + name="blog_index"), + url(r'^posts/(?P\d+)/$', + stub_view, + name='blog_detail'), +] diff --git a/resources/session08/mysite_stage_2/myblog/views.py b/resources/session08/mysite_stage_2/myblog/views.py new file mode 100644 index 00000000..ab45c18b --- /dev/null +++ b/resources/session08/mysite_stage_2/myblog/views.py @@ -0,0 +1,23 @@ +from django.http import HttpResponse, HttpResponseRedirect, Http404 +from django.shortcuts import render +from django.template import RequestContext, loader + +from myblog.models import Post + + +def stub_view(request, *args, **kwargs): + body = "Stub View\n\n" + if args: + body += "Args:\n" + body += "\n".join(["\t%s" % a for a in args]) + if kwargs: + body += "Kwargs:\n" + body += "\n".join(["\t%s: %s" % i for i in kwargs.items()]) + return HttpResponse(body, content_type="text/plain") + + +def list_view(request): + published = Post.objects.exclude(published_date__exact=None) + posts = published.order_by('-published_date') + context = {'posts': posts} + return render(request, 'list.html', context) diff --git a/resources/session08/mysite_stage_2/mysite/__init__.py b/resources/session08/mysite_stage_2/mysite/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session08/mysite_stage_2/mysite/settings.py b/resources/session08/mysite_stage_2/mysite/settings.py new file mode 100644 index 00000000..e81fb826 --- /dev/null +++ b/resources/session08/mysite_stage_2/mysite/settings.py @@ -0,0 +1,122 @@ +""" +Django settings for mysite project. + +Generated by 'django-admin startproject' using Django 1.9. + +For more information on this file, see +https://docs.djangoproject.com/en/1.9/topics/settings/ + +For the full list of settings and their values, see +https://docs.djangoproject.com/en/1.9/ref/settings/ +""" + +import os + +# Build paths inside the project like this: os.path.join(BASE_DIR, ...) +BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + + +# Quick-start development settings - unsuitable for production +# See https://docs.djangoproject.com/en/1.9/howto/deployment/checklist/ + +# SECURITY WARNING: keep the secret key used in production secret! +SECRET_KEY = 'i=n^tc%@@gq#8ev6dlymy9+-%@^f!q54sjf0rvikt_k5bl(t1=' + +# SECURITY WARNING: don't run with debug turned on in production! +DEBUG = True + +ALLOWED_HOSTS = [] + + +# Application definition + +INSTALLED_APPS = [ + 'django.contrib.admin', + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.messages', + 'django.contrib.staticfiles', + 'myblog', +] + +MIDDLEWARE_CLASSES = [ + 'django.middleware.security.SecurityMiddleware', + 'django.contrib.sessions.middleware.SessionMiddleware', + 'django.middleware.common.CommonMiddleware', + 'django.middleware.csrf.CsrfViewMiddleware', + 'django.contrib.auth.middleware.AuthenticationMiddleware', + 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', + 'django.contrib.messages.middleware.MessageMiddleware', + 'django.middleware.clickjacking.XFrameOptionsMiddleware', +] + +ROOT_URLCONF = 'mysite.urls' + +TEMPLATES = [ + { + 'BACKEND': 'django.template.backends.django.DjangoTemplates', + 'DIRS': [os.path.join(BASE_DIR, 'mysite/templates')], + 'APP_DIRS': True, + 'OPTIONS': { + 'context_processors': [ + 'django.template.context_processors.debug', + 'django.template.context_processors.request', + 'django.contrib.auth.context_processors.auth', + 'django.contrib.messages.context_processors.messages', + ], + }, + }, +] + +WSGI_APPLICATION = 'mysite.wsgi.application' + + +# Database +# https://docs.djangoproject.com/en/1.9/ref/settings/#databases + +DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.sqlite3', + 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), + } +} + + +# Password validation +# https://docs.djangoproject.com/en/1.9/ref/settings/#auth-password-validators + +AUTH_PASSWORD_VALIDATORS = [ + { + 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', + }, + { + 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', + }, + { + 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', + }, + { + 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', + }, +] + + +# Internationalization +# https://docs.djangoproject.com/en/1.9/topics/i18n/ + +LANGUAGE_CODE = 'en-us' + +TIME_ZONE = 'UTC' + +USE_I18N = True + +USE_L10N = True + +USE_TZ = True + + +# Static files (CSS, JavaScript, Images) +# https://docs.djangoproject.com/en/1.9/howto/static-files/ + +STATIC_URL = '/static/' diff --git a/resources/session08/mysite_stage_2/mysite/templates/base.html b/resources/session08/mysite_stage_2/mysite/templates/base.html new file mode 100644 index 00000000..2eacafa0 --- /dev/null +++ b/resources/session08/mysite_stage_2/mysite/templates/base.html @@ -0,0 +1,15 @@ + + + + My Django Blog + + +
+
+ {% block content %} + [content will go here] + {% endblock %} +
+
+ + diff --git a/resources/session08/mysite_stage_2/mysite/urls.py b/resources/session08/mysite_stage_2/mysite/urls.py new file mode 100644 index 00000000..bf31d5d5 --- /dev/null +++ b/resources/session08/mysite_stage_2/mysite/urls.py @@ -0,0 +1,24 @@ +"""mysite URL Configuration + +The `urlpatterns` list routes URLs to views. For more information please see: + https://docs.djangoproject.com/en/1.9/topics/http/urls/ +Examples: +Function views + 1. Add an import: from my_app import views + 2. Add a URL to urlpatterns: url(r'^$', views.home, name='home') +Class-based views + 1. Add an import: from other_app.views import Home + 2. Add a URL to urlpatterns: url(r'^$', Home.as_view(), name='home') +Including another URLconf + 1. Add an import: from blog import urls as blog_urls + 2. Import the include() function: from django.conf.urls import url, include + 3. Add a URL to urlpatterns: url(r'^blog/', include(blog_urls)) +""" +from django.conf.urls import url +from django.conf.urls import include +from django.contrib import admin + +urlpatterns = [ + url(r'^', include('myblog.urls')), + url(r'^admin/', admin.site.urls), +] diff --git a/resources/session08/mysite_stage_2/mysite/wsgi.py b/resources/session08/mysite_stage_2/mysite/wsgi.py new file mode 100644 index 00000000..328bae0f --- /dev/null +++ b/resources/session08/mysite_stage_2/mysite/wsgi.py @@ -0,0 +1,16 @@ +""" +WSGI config for mysite project. + +It exposes the WSGI callable as a module-level variable named ``application``. + +For more information on this file, see +https://docs.djangoproject.com/en/1.9/howto/deployment/wsgi/ +""" + +import os + +from django.core.wsgi import get_wsgi_application + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") + +application = get_wsgi_application() diff --git a/resources/session08/mysite_stage_3/manage.py b/resources/session08/mysite_stage_3/manage.py new file mode 100755 index 00000000..8a50ec04 --- /dev/null +++ b/resources/session08/mysite_stage_3/manage.py @@ -0,0 +1,10 @@ +#!/usr/bin/env python +import os +import sys + +if __name__ == "__main__": + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") + + from django.core.management import execute_from_command_line + + execute_from_command_line(sys.argv) diff --git a/resources/session08/mysite_stage_3/myblog/__init__.py b/resources/session08/mysite_stage_3/myblog/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session08/mysite_stage_3/myblog/admin.py b/resources/session08/mysite_stage_3/myblog/admin.py new file mode 100644 index 00000000..310e7294 --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/admin.py @@ -0,0 +1,8 @@ +from django.contrib import admin + +from myblog.models import Category +from myblog.models import Post + + +admin.site.register(Category) +admin.site.register(Post) diff --git a/resources/session08/mysite_stage_3/myblog/apps.py b/resources/session08/mysite_stage_3/myblog/apps.py new file mode 100644 index 00000000..5e29c8d9 --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/apps.py @@ -0,0 +1,5 @@ +from django.apps import AppConfig + + +class MyblogConfig(AppConfig): + name = 'myblog' diff --git a/resources/session08/mysite_stage_3/myblog/fixtures/myblog_test_fixture.json b/resources/session08/mysite_stage_3/myblog/fixtures/myblog_test_fixture.json new file mode 100644 index 00000000..bf5269e9 --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/fixtures/myblog_test_fixture.json @@ -0,0 +1,38 @@ +[ + { + "pk": 1, + "model": "auth.user", + "fields": { + "username": "admin", + "first_name": "Mr.", + "last_name": "Administrator", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "admin@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + }, + { + "pk": 2, + "model": "auth.user", + "fields": { + "username": "noname", + "first_name": "", + "last_name": "", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "noname@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + } +] diff --git a/resources/session08/mysite_stage_3/myblog/migrations/0001_initial.py b/resources/session08/mysite_stage_3/myblog/migrations/0001_initial.py new file mode 100644 index 00000000..18d659f1 --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/migrations/0001_initial.py @@ -0,0 +1,31 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.9 on 2015-12-31 19:13 +from __future__ import unicode_literals + +from django.conf import settings +from django.db import migrations, models +import django.db.models.deletion + + +class Migration(migrations.Migration): + + initial = True + + dependencies = [ + migrations.swappable_dependency(settings.AUTH_USER_MODEL), + ] + + operations = [ + migrations.CreateModel( + name='Post', + fields=[ + ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), + ('title', models.CharField(max_length=128)), + ('text', models.TextField(blank=True)), + ('created_date', models.DateTimeField(auto_now_add=True)), + ('modified_date', models.DateTimeField(auto_now=True)), + ('published_date', models.DateTimeField(blank=True, null=True)), + ('author', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)), + ], + ), + ] diff --git a/resources/session08/mysite_stage_3/myblog/migrations/0002_category.py b/resources/session08/mysite_stage_3/myblog/migrations/0002_category.py new file mode 100644 index 00000000..218b1052 --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/migrations/0002_category.py @@ -0,0 +1,24 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.9 on 2015-12-31 21:40 +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('myblog', '0001_initial'), + ] + + operations = [ + migrations.CreateModel( + name='Category', + fields=[ + ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), + ('name', models.CharField(max_length=128)), + ('description', models.TextField(blank=True)), + ('posts', models.ManyToManyField(blank=True, related_name='categories', to='myblog.Post')), + ], + ), + ] diff --git a/resources/session08/mysite_stage_3/myblog/migrations/__init__.py b/resources/session08/mysite_stage_3/myblog/migrations/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session08/mysite_stage_3/myblog/models.py b/resources/session08/mysite_stage_3/myblog/models.py new file mode 100644 index 00000000..cef1beac --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/models.py @@ -0,0 +1,27 @@ +from django.db import models +from django.contrib.auth.models import User + + +class Post(models.Model): + title = models.CharField(max_length=128) + text = models.TextField(blank=True) + author = models.ForeignKey(User) + created_date = models.DateTimeField(auto_now_add=True) + modified_date = models.DateTimeField(auto_now=True) + published_date = models.DateTimeField(blank=True, null=True) + + def __str__(self): + return self.title + + +class Category(models.Model): + name = models.CharField(max_length=128) + description = models.TextField(blank=True) + posts = models.ManyToManyField( + Post, + blank=True, + related_name='categories' + ) + + def __str__(self): + return self.name diff --git a/resources/session08/mysite_stage_3/myblog/static/django_blog.css b/resources/session08/mysite_stage_3/myblog/static/django_blog.css new file mode 100644 index 00000000..45a882de --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/static/django_blog.css @@ -0,0 +1,74 @@ +body { + background-color: #eee; + color: #111; + font-family: "HelveticaNeue-Light", "Helvetica Neue Light", "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif; + margin:0; + padding:0; +} +#container { + margin:0; + padding:0; + margin-top: 0px; +} +#header { + background-color: #333; + border-botton: 1px solid #111; + margin:0; + padding:0; +} +#control-bar { + margin: 0em 0em 1em; + list-style: none; + list-style-type: none; + text-align: right; + color: #eee; + font-size: 80%; + padding-bottom: 0.4em; +} +#control-bar li { + display: inline-block; +} +#control-bar li a { + color: #eee; + padding: 0.5em; + text-decoration: none; +} +#control-bar li a:hover { + color: #cce; +} +#content { + margin: 0em 1em 1em; +} + +ul#entries { + list-style: none; + list-style-type: none; +} +div.entry { + margin-right: 2em; + margin-top: 1em; + border-top: 1px solid #cecece; +} +ul#entries li:first-child div.entry { + border-top: none; + margin-top: 0em; +} +div.entry-body { + margin-left: 2em; +} +.notification { + float: right; + text-align: center; + width: 25%; + padding: 1em; +} +.info { + background-color: #aae; +} +ul.categories { + list-style: none; + list-style-type: none; +} +ul.categories li { + display: inline; +} diff --git a/resources/session08/mysite_stage_3/myblog/templates/detail.html b/resources/session08/mysite_stage_3/myblog/templates/detail.html new file mode 100644 index 00000000..cd0322ff --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/templates/detail.html @@ -0,0 +1,17 @@ +{% extends "base.html" %} + +{% block content %} +Home +

{{ post }}

+ +
+ {{ post.text }} +
+
    + {% for category in post.categories.all %} +
  • {{ category }}
  • + {% endfor %} +
+{% endblock %} diff --git a/resources/session08/mysite_stage_3/myblog/templates/list.html b/resources/session08/mysite_stage_3/myblog/templates/list.html new file mode 100644 index 00000000..88920817 --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/templates/list.html @@ -0,0 +1,25 @@ +{% extends "base.html" %} + +{% block content %} +

Recent Posts

+ + {% comment %} here is where the query happens {% endcomment %} + {% for post in posts %} +
+

+ {{ post }} +

+ +
+ {{ post.text }} +
+
    + {% for category in post.categories.all %} +
  • {{ category }}
  • + {% endfor %} +
+
+ {% endfor %} +{% endblock %} diff --git a/resources/session08/mysite_stage_3/myblog/tests.py b/resources/session08/mysite_stage_3/myblog/tests.py new file mode 100644 index 00000000..c4f547bd --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/tests.py @@ -0,0 +1,71 @@ +import datetime +from django.contrib.auth.models import User +from django.test import TestCase +from django.utils.timezone import utc + +from myblog.models import Category +from myblog.models import Post + + +class PostTestCase(TestCase): + fixtures = ['myblog_test_fixture.json'] + + def setUp(self): + self.user = User.objects.get(pk=1) + + def test_string_representation(self): + expected = "This is a title" + p1 = Post(title=expected) + actual = str(p1) + self.assertEqual(expected, actual) + + +class CategoryTestCase(TestCase): + + def test_string_representation(self): + expected = "A Category" + c1 = Category(name=expected) + actual = str(c1) + self.assertEqual(expected, actual) + + +class FrontEndTestCase(TestCase): + """test views provided in the front-end""" + fixtures = ['myblog_test_fixture.json', ] + + def setUp(self): + self.now = datetime.datetime.utcnow().replace(tzinfo=utc) + self.timedelta = datetime.timedelta(15) + author = User.objects.get(pk=1) + for count in range(1, 11): + post = Post(title="Post %d Title" % count, + text="foo", + author=author) + if count < 6: + # publish the first five posts + pubdate = self.now - self.timedelta * count + post.published_date = pubdate + post.save() + + def test_list_only_published(self): + resp = self.client.get('/') + # the content of the rendered response is always a bytestring + resp_text = resp.content.decode(resp.charset) + self.assertTrue("Recent Posts" in resp_text) + for count in range(1, 11): + title = "Post %d Title" % count + if count < 6: + self.assertContains(resp, title, count=1) + else: + self.assertNotContains(resp, title) + + def test_details_only_published(self): + for count in range(1, 11): + title = "Post %d Title" % count + post = Post.objects.get(title=title) + resp = self.client.get('/posts/%d/' % post.pk) + if count < 6: + self.assertEqual(resp.status_code, 200) + self.assertContains(resp, title) + else: + self.assertEqual(resp.status_code, 404) diff --git a/resources/session08/mysite_stage_3/myblog/urls.py b/resources/session08/mysite_stage_3/myblog/urls.py new file mode 100644 index 00000000..5caacf17 --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/urls.py @@ -0,0 +1,15 @@ +from django.conf.urls import url + +from myblog.views import stub_view +from myblog.views import list_view +from myblog.views import detail_view + + +urlpatterns = [ + url(r'^$', + list_view, + name="blog_index"), + url(r'^posts/(?P\d+)/$', + detail_view, + name='blog_detail'), +] diff --git a/resources/session08/mysite_stage_3/myblog/views.py b/resources/session08/mysite_stage_3/myblog/views.py new file mode 100644 index 00000000..c1e8c41a --- /dev/null +++ b/resources/session08/mysite_stage_3/myblog/views.py @@ -0,0 +1,33 @@ +from django.http import HttpResponse, HttpResponseRedirect, Http404 +from django.shortcuts import render +from django.template import RequestContext, loader + +from myblog.models import Post + + +def stub_view(request, *args, **kwargs): + body = "Stub View\n\n" + if args: + body += "Args:\n" + body += "\n".join(["\t%s" % a for a in args]) + if kwargs: + body += "Kwargs:\n" + body += "\n".join(["\t%s: %s" % i for i in kwargs.items()]) + return HttpResponse(body, content_type="text/plain") + + +def list_view(request): + published = Post.objects.exclude(published_date__exact=None) + posts = published.order_by('-published_date') + context = {'posts': posts} + return render(request, 'list.html', context) + + +def detail_view(request, post_id): + published = Post.objects.exclude(published_date__exact=None) + try: + post = published.get(pk=post_id) + except Post.DoesNotExist: + raise Http404 + context = {'post': post} + return render(request, 'detail.html', context) diff --git a/resources/session08/mysite_stage_3/mysite/__init__.py b/resources/session08/mysite_stage_3/mysite/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session08/mysite_stage_3/mysite/settings.py b/resources/session08/mysite_stage_3/mysite/settings.py new file mode 100644 index 00000000..3fd7eb4c --- /dev/null +++ b/resources/session08/mysite_stage_3/mysite/settings.py @@ -0,0 +1,125 @@ +""" +Django settings for mysite project. + +Generated by 'django-admin startproject' using Django 1.9. + +For more information on this file, see +https://docs.djangoproject.com/en/1.9/topics/settings/ + +For the full list of settings and their values, see +https://docs.djangoproject.com/en/1.9/ref/settings/ +""" + +import os + +# Build paths inside the project like this: os.path.join(BASE_DIR, ...) +BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + + +# Quick-start development settings - unsuitable for production +# See https://docs.djangoproject.com/en/1.9/howto/deployment/checklist/ + +# SECURITY WARNING: keep the secret key used in production secret! +SECRET_KEY = 'i=n^tc%@@gq#8ev6dlymy9+-%@^f!q54sjf0rvikt_k5bl(t1=' + +# SECURITY WARNING: don't run with debug turned on in production! +DEBUG = True + +ALLOWED_HOSTS = [] + + +# Application definition + +INSTALLED_APPS = [ + 'django.contrib.admin', + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.messages', + 'django.contrib.staticfiles', + 'myblog', +] + +MIDDLEWARE_CLASSES = [ + 'django.middleware.security.SecurityMiddleware', + 'django.contrib.sessions.middleware.SessionMiddleware', + 'django.middleware.common.CommonMiddleware', + 'django.middleware.csrf.CsrfViewMiddleware', + 'django.contrib.auth.middleware.AuthenticationMiddleware', + 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', + 'django.contrib.messages.middleware.MessageMiddleware', + 'django.middleware.clickjacking.XFrameOptionsMiddleware', +] + +ROOT_URLCONF = 'mysite.urls' + +TEMPLATES = [ + { + 'BACKEND': 'django.template.backends.django.DjangoTemplates', + 'DIRS': [os.path.join(BASE_DIR, 'mysite/templates')], + 'APP_DIRS': True, + 'OPTIONS': { + 'context_processors': [ + 'django.template.context_processors.debug', + 'django.template.context_processors.request', + 'django.contrib.auth.context_processors.auth', + 'django.contrib.messages.context_processors.messages', + ], + }, + }, +] + +WSGI_APPLICATION = 'mysite.wsgi.application' + + +# Database +# https://docs.djangoproject.com/en/1.9/ref/settings/#databases + +DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.sqlite3', + 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), + } +} + + +# Password validation +# https://docs.djangoproject.com/en/1.9/ref/settings/#auth-password-validators + +AUTH_PASSWORD_VALIDATORS = [ + { + 'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator', + }, + { + 'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator', + }, + { + 'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator', + }, + { + 'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator', + }, +] + + +# Internationalization +# https://docs.djangoproject.com/en/1.9/topics/i18n/ + +LANGUAGE_CODE = 'en-us' + +TIME_ZONE = 'UTC' + +USE_I18N = True + +USE_L10N = True + +USE_TZ = True + + +# Static files (CSS, JavaScript, Images) +# https://docs.djangoproject.com/en/1.9/howto/static-files/ + +STATIC_URL = '/static/' + +LOGIN_URL = '/login/' +LOGIN_REDIRECT_URL = '/' diff --git a/resources/session08/mysite_stage_3/mysite/templates/base.html b/resources/session08/mysite_stage_3/mysite/templates/base.html new file mode 100644 index 00000000..1529aead --- /dev/null +++ b/resources/session08/mysite_stage_3/mysite/templates/base.html @@ -0,0 +1,27 @@ +{% load staticfiles %} + + + + My Django Blog + + + + +
+
+ {% block content %} + [content will go here] + {% endblock %} +
+
+ + diff --git a/resources/session08/mysite_stage_3/mysite/templates/login.html b/resources/session08/mysite_stage_3/mysite/templates/login.html new file mode 100644 index 00000000..1566d0f7 --- /dev/null +++ b/resources/session08/mysite_stage_3/mysite/templates/login.html @@ -0,0 +1,9 @@ +{% extends "base.html" %} + +{% block content %} +

My Blog Login

+
{% csrf_token %} + {{ form.as_p }} +

+
+{% endblock %} diff --git a/resources/session08/mysite_stage_3/mysite/urls.py b/resources/session08/mysite_stage_3/mysite/urls.py new file mode 100644 index 00000000..91f7819c --- /dev/null +++ b/resources/session08/mysite_stage_3/mysite/urls.py @@ -0,0 +1,33 @@ +"""mysite URL Configuration + +The `urlpatterns` list routes URLs to views. For more information please see: + https://docs.djangoproject.com/en/1.9/topics/http/urls/ +Examples: +Function views + 1. Add an import: from my_app import views + 2. Add a URL to urlpatterns: url(r'^$', views.home, name='home') +Class-based views + 1. Add an import: from other_app.views import Home + 2. Add a URL to urlpatterns: url(r'^$', Home.as_view(), name='home') +Including another URLconf + 1. Add an import: from blog import urls as blog_urls + 2. Import the include() function: from django.conf.urls import url, include + 3. Add a URL to urlpatterns: url(r'^blog/', include(blog_urls)) +""" +from django.conf.urls import url +from django.conf.urls import include +from django.contrib import admin +from django.contrib.auth.views import login, logout + +urlpatterns = [ + url(r'^', include('myblog.urls')), + url(r'^login/$', + login, + {'template_name': 'login.html'}, + name="login"), + url(r'^logout/$', + logout, + {'next_page': '/'}, + name="logout"), + url(r'^admin/', admin.site.urls), +] diff --git a/resources/session08/mysite_stage_3/mysite/wsgi.py b/resources/session08/mysite_stage_3/mysite/wsgi.py new file mode 100644 index 00000000..328bae0f --- /dev/null +++ b/resources/session08/mysite_stage_3/mysite/wsgi.py @@ -0,0 +1,16 @@ +""" +WSGI config for mysite project. + +It exposes the WSGI callable as a module-level variable named ``application``. + +For more information on this file, see +https://docs.djangoproject.com/en/1.9/howto/deployment/wsgi/ +""" + +import os + +from django.core.wsgi import get_wsgi_application + +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") + +application = get_wsgi_application() diff --git a/resources/session09/mysite/manage.py b/resources/session09/mysite/manage.py new file mode 100755 index 00000000..8a50ec04 --- /dev/null +++ b/resources/session09/mysite/manage.py @@ -0,0 +1,10 @@ +#!/usr/bin/env python +import os +import sys + +if __name__ == "__main__": + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") + + from django.core.management import execute_from_command_line + + execute_from_command_line(sys.argv) diff --git a/resources/session09/mysite/myblog/__init__.py b/resources/session09/mysite/myblog/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session09/mysite/myblog/admin.py b/resources/session09/mysite/myblog/admin.py new file mode 100644 index 00000000..d772219e --- /dev/null +++ b/resources/session09/mysite/myblog/admin.py @@ -0,0 +1,44 @@ +import datetime +from django.contrib import admin +from django.core.urlresolvers import reverse +from myblog.models import Post, Category + + +class CategorizationInline(admin.TabularInline): + model = Category.posts.through + + +def make_published(modeladmin, request, queryset): + now = datetime.datetime.now() + queryset.update(published_date=now) +make_published.short_description = "Set publication date for selected posts" + + +class PostAdmin(admin.ModelAdmin): + inlines = [ + CategorizationInline, + ] + list_display = ( + '__unicode__', 'author_for_admin', 'created_date', 'modified_date', 'published_date' + ) + readonly_fields = ( + 'created_date', 'modified_date', + ) + actions = [make_published, ] + + def author_for_admin(self, obj): + author = obj.author + url = reverse('admin:auth_user_change', args=(author.pk,)) + name = author.get_full_name() or author.username + link = '{}'.format(url, name) + return link + author_for_admin.short_description = 'Author' + author_for_admin.allow_tags = True + + +class CategoryAdmin(admin.ModelAdmin): + exclude = ('posts', ) + + +admin.site.register(Post, PostAdmin) +admin.site.register(Category, CategoryAdmin) diff --git a/resources/session09/mysite/myblog/fixtures/myblog_test_fixture.json b/resources/session09/mysite/myblog/fixtures/myblog_test_fixture.json new file mode 100644 index 00000000..bf5269e9 --- /dev/null +++ b/resources/session09/mysite/myblog/fixtures/myblog_test_fixture.json @@ -0,0 +1,38 @@ +[ + { + "pk": 1, + "model": "auth.user", + "fields": { + "username": "admin", + "first_name": "Mr.", + "last_name": "Administrator", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "admin@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + }, + { + "pk": 2, + "model": "auth.user", + "fields": { + "username": "noname", + "first_name": "", + "last_name": "", + "is_active": true, + "is_superuser": true, + "is_staff": true, + "last_login": "2013-05-24T05:35:58.628Z", + "groups": [], + "user_permissions": [], + "password": "pbkdf2_sha256$10000$1rQazFNdOfFt$6aw/uIrv2uASkZ7moXMTajSN+ySYuowBnbP6ILNQntE=", + "email": "noname@example.com", + "date_joined": "2013-05-24T05:35:58.628Z" + } + } +] diff --git a/resources/session09/mysite/myblog/migrations/0001_initial.py b/resources/session09/mysite/myblog/migrations/0001_initial.py new file mode 100644 index 00000000..9772455c --- /dev/null +++ b/resources/session09/mysite/myblog/migrations/0001_initial.py @@ -0,0 +1,30 @@ +# -*- coding: utf-8 -*- +from __future__ import unicode_literals + +from django.db import models, migrations +from django.conf import settings + + +class Migration(migrations.Migration): + + dependencies = [ + migrations.swappable_dependency(settings.AUTH_USER_MODEL), + ] + + operations = [ + migrations.CreateModel( + name='Post', + fields=[ + ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)), + ('title', models.CharField(max_length=128)), + ('text', models.TextField(blank=True)), + ('created_date', models.DateTimeField(auto_now_add=True)), + ('modified_date', models.DateTimeField(auto_now=True)), + ('published_date', models.DateTimeField(null=True, blank=True)), + ('author', models.ForeignKey(to=settings.AUTH_USER_MODEL)), + ], + options={ + }, + bases=(models.Model,), + ), + ] diff --git a/resources/session09/mysite/myblog/migrations/0002_category.py b/resources/session09/mysite/myblog/migrations/0002_category.py new file mode 100644 index 00000000..cd06d71e --- /dev/null +++ b/resources/session09/mysite/myblog/migrations/0002_category.py @@ -0,0 +1,26 @@ +# -*- coding: utf-8 -*- +from __future__ import unicode_literals + +from django.db import models, migrations + + +class Migration(migrations.Migration): + + dependencies = [ + ('myblog', '0001_initial'), + ] + + operations = [ + migrations.CreateModel( + name='Category', + fields=[ + ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)), + ('name', models.CharField(max_length=128)), + ('description', models.TextField(blank=True)), + ('posts', models.ManyToManyField(related_name='categories', null=True, to='myblog.Post', blank=True)), + ], + options={ + }, + bases=(models.Model,), + ), + ] diff --git a/resources/session09/mysite/myblog/migrations/__init__.py b/resources/session09/mysite/myblog/migrations/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session09/mysite/myblog/models.py b/resources/session09/mysite/myblog/models.py new file mode 100644 index 00000000..8b5f59cf --- /dev/null +++ b/resources/session09/mysite/myblog/models.py @@ -0,0 +1,27 @@ +from django.db import models +from django.contrib.auth.models import User + + +class Post(models.Model): + title = models.CharField(max_length=128) + text = models.TextField(blank=True) + author = models.ForeignKey(User) + created_date = models.DateTimeField(auto_now_add=True) + modified_date = models.DateTimeField(auto_now=True) + published_date = models.DateTimeField(blank=True, null=True) + + def __unicode__(self): + return self.title + + +class Category(models.Model): + name = models.CharField(max_length=128) + description = models.TextField(blank=True) + posts = models.ManyToManyField(Post, blank=True, null=True, + related_name='categories') + + class Meta: + verbose_name_plural = 'Categories' + + def __unicode__(self): + return self.name diff --git a/resources/session09/mysite/myblog/static/django_blog.css b/resources/session09/mysite/myblog/static/django_blog.css new file mode 100644 index 00000000..45a882de --- /dev/null +++ b/resources/session09/mysite/myblog/static/django_blog.css @@ -0,0 +1,74 @@ +body { + background-color: #eee; + color: #111; + font-family: "HelveticaNeue-Light", "Helvetica Neue Light", "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif; + margin:0; + padding:0; +} +#container { + margin:0; + padding:0; + margin-top: 0px; +} +#header { + background-color: #333; + border-botton: 1px solid #111; + margin:0; + padding:0; +} +#control-bar { + margin: 0em 0em 1em; + list-style: none; + list-style-type: none; + text-align: right; + color: #eee; + font-size: 80%; + padding-bottom: 0.4em; +} +#control-bar li { + display: inline-block; +} +#control-bar li a { + color: #eee; + padding: 0.5em; + text-decoration: none; +} +#control-bar li a:hover { + color: #cce; +} +#content { + margin: 0em 1em 1em; +} + +ul#entries { + list-style: none; + list-style-type: none; +} +div.entry { + margin-right: 2em; + margin-top: 1em; + border-top: 1px solid #cecece; +} +ul#entries li:first-child div.entry { + border-top: none; + margin-top: 0em; +} +div.entry-body { + margin-left: 2em; +} +.notification { + float: right; + text-align: center; + width: 25%; + padding: 1em; +} +.info { + background-color: #aae; +} +ul.categories { + list-style: none; + list-style-type: none; +} +ul.categories li { + display: inline; +} diff --git a/resources/session09/mysite/myblog/templates/detail.html b/resources/session09/mysite/myblog/templates/detail.html new file mode 100644 index 00000000..cd0322ff --- /dev/null +++ b/resources/session09/mysite/myblog/templates/detail.html @@ -0,0 +1,17 @@ +{% extends "base.html" %} + +{% block content %} +Home +

{{ post }}

+ +
+ {{ post.text }} +
+
    + {% for category in post.categories.all %} +
  • {{ category }}
  • + {% endfor %} +
+{% endblock %} diff --git a/resources/session09/mysite/myblog/templates/list.html b/resources/session09/mysite/myblog/templates/list.html new file mode 100644 index 00000000..88920817 --- /dev/null +++ b/resources/session09/mysite/myblog/templates/list.html @@ -0,0 +1,25 @@ +{% extends "base.html" %} + +{% block content %} +

Recent Posts

+ + {% comment %} here is where the query happens {% endcomment %} + {% for post in posts %} +
+

+ {{ post }} +

+ +
+ {{ post.text }} +
+
    + {% for category in post.categories.all %} +
  • {{ category }}
  • + {% endfor %} +
+
+ {% endfor %} +{% endblock %} diff --git a/resources/session09/mysite/myblog/tests.py b/resources/session09/mysite/myblog/tests.py new file mode 100644 index 00000000..1f967b24 --- /dev/null +++ b/resources/session09/mysite/myblog/tests.py @@ -0,0 +1,67 @@ +import datetime +from django.test import TestCase +from django.contrib.auth.models import User +from django.utils.timezone import utc +from myblog.models import Post, Category + + +class PostTestCase(TestCase): + fixtures = ['myblog_test_fixture.json', ] + + def setUp(self): + self.user = User.objects.get(pk=1) + + def test_unicode(self): + expected = u"This is a title" + p1 = Post(title=expected) + actual = unicode(p1) + self.assertEqual(expected, actual) + + +class CategoryTestCase(TestCase): + + def test_unicode(self): + expected = "A Category" + c1 = Category(name=expected) + actual = unicode(c1) + self.assertEqual(expected, actual) + + +class FrontEndTestCase(TestCase): + """test views provided in the front-end""" + fixtures = ['myblog_test_fixture.json', ] + + def setUp(self): + self.now = datetime.datetime.utcnow().replace(tzinfo=utc) + self.timedelta = datetime.timedelta(15) + author = User.objects.get(pk=1) + for count in range(1, 11): + post = Post(title="Post %d Title" % count, + text="foo", + author=author) + if count < 6: + # publish the first five posts + pubdate = self.now - self.timedelta * count + post.published_date = pubdate + post.save() + + def test_list_only_published(self): + resp = self.client.get('/') + self.assertTrue("Recent Posts" in resp.content) + for count in range(1, 11): + title = "Post %d Title" % count + if count < 6: + self.assertContains(resp, title, count=1) + else: + self.assertNotContains(resp, title) + + def test_details_only_published(self): + for count in range(1, 11): + title = "Post %d Title" % count + post = Post.objects.get(title=title) + resp = self.client.get('/posts/%d/' % post.pk) + if count < 6: + self.assertEqual(resp.status_code, 200) + self.assertContains(resp, title) + else: + self.assertEqual(resp.status_code, 404) diff --git a/resources/session09/mysite/myblog/urls.py b/resources/session09/mysite/myblog/urls.py new file mode 100644 index 00000000..d31e75c9 --- /dev/null +++ b/resources/session09/mysite/myblog/urls.py @@ -0,0 +1,12 @@ +from django.conf.urls import patterns, url + + +urlpatterns = patterns( + 'myblog.views', + url(r'^$', + 'list_view', + name="blog_index"), + url(r'^posts/(?P\d+)/$', + 'detail_view', + name="blog_detail"), +) diff --git a/resources/session09/mysite/myblog/views.py b/resources/session09/mysite/myblog/views.py new file mode 100644 index 00000000..389fa2ed --- /dev/null +++ b/resources/session09/mysite/myblog/views.py @@ -0,0 +1,32 @@ +from django.shortcuts import render +from django.http import HttpResponse, HttpResponseRedirect, Http404 +from django.template import RequestContext, loader +from myblog.models import Post + + +def stub_view(request, *args, **kwargs): + body = "Stub View\n\n" + if args: + body += "Args:\n" + body += "\n".join(["\t%s" % a for a in args]) + if kwargs: + body += "Kwargs:\n" + body += "\n".join(["\t%s: %s" % i for i in kwargs.items()]) + return HttpResponse(body, content_type="text/plain") + + +def list_view(request): + published = Post.objects.exclude(published_date__exact=None) + posts = published.order_by('-published_date') + context = {'posts': posts} + return render(request, 'list.html', context) + + +def detail_view(request, post_id): + published = Post.objects.exclude(published_date__exact=None) + try: + post = published.get(pk=post_id) + except Post.DoesNotExist: + raise Http404 + context = {'post': post} + return render(request, 'detail.html', context) diff --git a/resources/session09/mysite/mysite/__init__.py b/resources/session09/mysite/mysite/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/resources/session09/mysite/mysite/settings.py b/resources/session09/mysite/mysite/settings.py new file mode 100644 index 00000000..8d7e926a --- /dev/null +++ b/resources/session09/mysite/mysite/settings.py @@ -0,0 +1,89 @@ +""" +Django settings for mysite project. + +For more information on this file, see +https://docs.djangoproject.com/en/1.7/topics/settings/ + +For the full list of settings and their values, see +https://docs.djangoproject.com/en/1.7/ref/settings/ +""" + +# Build paths inside the project like this: os.path.join(BASE_DIR, ...) +import os +BASE_DIR = os.path.dirname(os.path.dirname(__file__)) + + +# Quick-start development settings - unsuitable for production +# See https://docs.djangoproject.com/en/1.7/howto/deployment/checklist/ + +# SECURITY WARNING: keep the secret key used in production secret! +SECRET_KEY = 'e@3=0i!#n4l25r*ul*sbx6b$@gh7a6pjee6lr-slw9!ayj#*@f' + +# SECURITY WARNING: don't run with debug turned on in production! +DEBUG = True + +TEMPLATE_DEBUG = True + +ALLOWED_HOSTS = [] + + +# Application definition + +INSTALLED_APPS = ( + 'django.contrib.admin', + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.messages', + 'django.contrib.staticfiles', + 'myblog', +) + +MIDDLEWARE_CLASSES = ( + 'django.contrib.sessions.middleware.SessionMiddleware', + 'django.middleware.common.CommonMiddleware', + 'django.middleware.csrf.CsrfViewMiddleware', + 'django.contrib.auth.middleware.AuthenticationMiddleware', + 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', + 'django.contrib.messages.middleware.MessageMiddleware', + 'django.middleware.clickjacking.XFrameOptionsMiddleware', +) + +ROOT_URLCONF = 'mysite.urls' + +WSGI_APPLICATION = 'mysite.wsgi.application' + + +# Database +# https://docs.djangoproject.com/en/1.7/ref/settings/#databases + +DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.sqlite3', + 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'), + } +} + +# Internationalization +# https://docs.djangoproject.com/en/1.7/topics/i18n/ + +LANGUAGE_CODE = 'en-us' + +TIME_ZONE = 'UTC' + +USE_I18N = True + +USE_L10N = True + +USE_TZ = True + + +# Static files (CSS, JavaScript, Images) +# https://docs.djangoproject.com/en/1.7/howto/static-files/ + +STATIC_URL = '/static/' + + +TEMPLATE_DIRS = (os.path.join(BASE_DIR, 'mysite/templates'), ) +LOGIN_URL = '/login/' +LOGIN_REDIRECT_URL = '/' diff --git a/resources/session09/mysite/mysite/templates/base.html b/resources/session09/mysite/mysite/templates/base.html new file mode 100644 index 00000000..eed438f9 --- /dev/null +++ b/resources/session09/mysite/mysite/templates/base.html @@ -0,0 +1,29 @@ +{% load staticfiles %} + + + + My Django Blog + + + + +
+
+ {% block content %} + [content will go here] + {% endblock %} +
+
+ + diff --git a/resources/session09/mysite/mysite/templates/login.html b/resources/session09/mysite/mysite/templates/login.html new file mode 100644 index 00000000..1566d0f7 --- /dev/null +++ b/resources/session09/mysite/mysite/templates/login.html @@ -0,0 +1,9 @@ +{% extends "base.html" %} + +{% block content %} +

My Blog Login

+
{% csrf_token %} + {{ form.as_p }} +

+
+{% endblock %} diff --git a/resources/session09/mysite/mysite/urls.py b/resources/session09/mysite/mysite/urls.py new file mode 100644 index 00000000..bbcf775f --- /dev/null +++ b/resources/session09/mysite/mysite/urls.py @@ -0,0 +1,15 @@ +from django.conf.urls import patterns, include, url +from django.contrib import admin + +urlpatterns = patterns('', + url(r'^', include('myblog.urls')), + url(r'^login/$', + 'django.contrib.auth.views.login', + {'template_name': 'login.html'}, + name="login"), + url(r'^logout/$', + 'django.contrib.auth.views.logout', + {'next_page': '/'}, + name="logout"), + url(r'^admin/', include(admin.site.urls)), +) diff --git a/resources/session09/mysite/mysite/wsgi.py b/resources/session09/mysite/mysite/wsgi.py new file mode 100644 index 00000000..15c7d49c --- /dev/null +++ b/resources/session09/mysite/mysite/wsgi.py @@ -0,0 +1,14 @@ +""" +WSGI config for mysite project. + +It exposes the WSGI callable as a module-level variable named ``application``. + +For more information on this file, see +https://docs.djangoproject.com/en/1.7/howto/deployment/wsgi/ +""" + +import os +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") + +from django.core.wsgi import get_wsgi_application +application = get_wsgi_application() diff --git a/source/_static/admin_index.png b/source/_static/admin_index.png index ae7a19f9..f42d6e38 100644 Binary files a/source/_static/admin_index.png and b/source/_static/admin_index.png differ diff --git a/source/_static/custom.css b/source/_static/custom.css index 44274406..93e3b1cf 100644 --- a/source/_static/custom.css +++ b/source/_static/custom.css @@ -74,6 +74,15 @@ aside.gdbar { top: 4px; } +.title-slide:first-child hgroup { + top: 35%; +} + +/* line 888, ../scss/default.scss */ +.title-slide hgroup { + top: 0px; +} + em { font-style: italic; } @@ -102,6 +111,14 @@ article .medium { font-weight: bold; font-size: 45px; line-height: 45px; } +article .small { + font-weight: normal; + font-size: 30px; + line-height: 30px; } +article .tiny .highlight pre { + font-size: 16px; + line-height: 20px; +} article .credit { font-size: 75%; text-align: left; } @@ -122,6 +139,11 @@ article .toctree-wrapper li a { border-bottom: none; } article table.docutils tr td { vertical-align: top; } +slide article div.slide-no { + position: absolute; + bottom: 5px; + right: 5px; +} .level-1 h1 { font-size: 65px; @@ -163,4 +185,7 @@ article table.docutils tr td { font-size: 75%; text-align: center; } - +.figure.align-left { + text-align: left; + float: left; +} diff --git a/source/_static/django-admin-login.png b/source/_static/django-admin-login.png index 4ceb5f54..499384a8 100644 Binary files a/source/_static/django-admin-login.png and b/source/_static/django-admin-login.png differ diff --git a/source/_static/django-start.png b/source/_static/django-start.png index 42017110..6c912d30 100644 Binary files a/source/_static/django-start.png and b/source/_static/django-start.png differ diff --git a/source/_static/geojson-io.png b/source/_static/geojson-io.png new file mode 100644 index 00000000..cc437c58 Binary files /dev/null and b/source/_static/geojson-io.png differ diff --git a/source/_static/nginx_hello.png b/source/_static/nginx_hello.png new file mode 100644 index 00000000..f2db39d9 Binary files /dev/null and b/source/_static/nginx_hello.png differ diff --git a/source/_static/no_entry.jpg b/source/_static/no_entry.jpg new file mode 100644 index 00000000..76c18265 Binary files /dev/null and b/source/_static/no_entry.jpg differ diff --git a/source/_templates/end_slide.html b/source/_templates/end_slide.html index 75bbadcc..f87aac00 100644 --- a/source/_templates/end_slide.html +++ b/source/_templates/end_slide.html @@ -1,7 +1,7 @@
-

<Thank You!>

+

Good Night!

 

diff --git a/source/_themes/uwpce_slides2/end_slide.html b/source/_themes/uwpce_slides2/end_slide.html new file mode 100644 index 00000000..09303106 --- /dev/null +++ b/source/_themes/uwpce_slides2/end_slide.html @@ -0,0 +1,8 @@ + +

+

<Thank You!>

+
+

+ +

+
diff --git a/source/_themes/uwpce_slides2/layout.html b/source/_themes/uwpce_slides2/layout.html new file mode 100644 index 00000000..b9fad83e --- /dev/null +++ b/source/_themes/uwpce_slides2/layout.html @@ -0,0 +1,146 @@ + +{%- block doctype -%} + +{%- endblock %} + +{%- set reldelim1 = reldelim1 is not defined and ' »' or reldelim1 %} +{%- set reldelim2 = reldelim2 is not defined and ' |' or reldelim2 %} +{%- set render_sidebar = (not embedded) and (not theme_nosidebar|tobool) and + (sidebars != []) %} +{%- set url_root = pathto('', 1) %} +{# XXX necessary? #} +{%- if url_root == '#' %}{% set url_root = '' %}{% endif %} +{%- if not embedded and docstitle %} + {%- set titlesuffix = " — "|safe + docstitle|e %} +{%- else %} + {%- set titlesuffix = "" %} +{%- endif %} + +{%- macro relbar() %} +{%- endmacro %} + +{%- macro sidebar() %} +{%- endmacro %} + +{%- macro script() %} + + + + + {%- for scriptfile in script_files %} + + {%- endfor %} + {% if theme_custom_js %} + + {% endif %} + +{%- endmacro %} + +{%- macro css() %} + + + + + {% if theme_custom_css %} + + {% endif %} + + + {%- for cssfile in css_files %} + + {%- endfor %} +{%- endmacro %} + + + + {%- block htmltitle %} + {{ title|striptags|e }}{{ titlesuffix }} + {%- endblock %} + + {{ metatags }} + + + + + + + + {{ css() }} + + {%- if not embedded %} + {{ script() }} + {%- if use_opensearch %} + + {%- endif %} + {%- if favicon %} + + {%- endif %} + {%- endif %} +{%- block linktags %} + {%- if hasdoc('about') %} + + {%- endif %} + {%- if hasdoc('genindex') %} + + {%- endif %} + {%- if hasdoc('search') %} + + {%- endif %} + {%- if hasdoc('copyright') %} + + {%- endif %} + + {%- if parents %} + + {%- endif %} + {%- if next %} + + {%- endif %} + {%- if prev %} + + {%- endif %} +{%- endblock %} +{%- block extrahead %} {% endblock %} + + + + + + {% include "title_slide.html" %} + + {% block body %}{% endblock %} + + {% include "end_slide.html" %} + + + + + + + + diff --git a/source/_themes/uwpce_slides2/slide.html b/source/_themes/uwpce_slides2/slide.html new file mode 100644 index 00000000..47c77e34 --- /dev/null +++ b/source/_themes/uwpce_slides2/slide.html @@ -0,0 +1,15 @@ + +
+ {{ title }} +
+
+ {{ content }} + +{% if config.slide_numbers %} +
{{ slide_number }}
+{% endif %} +{% if config.slide_footer %} + +{% endif %} +
+
diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/0fe24dfc41fffed2d6891c797fcd7dee100afa65/_hacks.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/0fe24dfc41fffed2d6891c797fcd7dee100afa65/_hacks.scssc new file mode 100644 index 00000000..d834d4b0 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/0fe24dfc41fffed2d6891c797fcd7dee100afa65/_hacks.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_background-size.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_background-size.scssc new file mode 100644 index 00000000..fcc62701 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_background-size.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_border-radius.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_border-radius.scssc new file mode 100644 index 00000000..e4c708fc Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_border-radius.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_box-shadow.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_box-shadow.scssc new file mode 100644 index 00000000..10a4d0aa Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_box-shadow.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_box-sizing.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_box-sizing.scssc new file mode 100644 index 00000000..db61cdb1 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_box-sizing.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_box.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_box.scssc new file mode 100644 index 00000000..9e240169 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_box.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_columns.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_columns.scssc new file mode 100644 index 00000000..089a3617 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_columns.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_deprecated-support.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_deprecated-support.scssc new file mode 100644 index 00000000..c8e9b8d2 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_deprecated-support.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_images.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_images.scssc new file mode 100644 index 00000000..ad6c7d20 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_images.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_text-shadow.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_text-shadow.scssc new file mode 100644 index 00000000..ecd4d640 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_text-shadow.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_transform.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_transform.scssc new file mode 100644 index 00000000..2994bf98 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_transform.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_transition.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_transition.scssc new file mode 100644 index 00000000..882f0036 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_transition.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_user-interface.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_user-interface.scssc new file mode 100644 index 00000000..9b903473 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/17d03613c125918cd0766f51918feb21dc3c074a/_user-interface.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/65e4c30c131f260ea88c3e4f2e16dfc2ba547e74/_utilities.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/65e4c30c131f260ea88c3e4f2e16dfc2ba547e74/_utilities.scssc new file mode 100644 index 00000000..7fc32451 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/65e4c30c131f260ea88c3e4f2e16dfc2ba547e74/_utilities.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/_base.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/_base.scssc new file mode 100644 index 00000000..51d6e066 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/_base.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/_variables.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/_variables.scssc new file mode 100644 index 00000000..1ada2d1d Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/_variables.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/default.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/default.scssc new file mode 100644 index 00000000..4a68e38a Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/default.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/hieroglyph.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/hieroglyph.scssc new file mode 100644 index 00000000..20d0bc8b Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/hieroglyph.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/io2013.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/io2013.scssc new file mode 100644 index 00000000..f1a05246 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/io2013.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/phone.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/phone.scssc new file mode 100644 index 00000000..b7b3143a Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/a3714e1b6bb8b987fa4c7e14e1704c7e18bb1783/phone.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/af1a5722249b61fe97c5776f7a26e0902a00f406/_reset.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/af1a5722249b61fe97c5776f7a26e0902a00f406/_reset.scssc new file mode 100644 index 00000000..51d20d3c Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/af1a5722249b61fe97c5776f7a26e0902a00f406/_reset.scssc differ diff --git a/source/_themes/uwpce_slides2/static/.sass-cache/af1a5722249b61fe97c5776f7a26e0902a00f406/_support.scssc b/source/_themes/uwpce_slides2/static/.sass-cache/af1a5722249b61fe97c5776f7a26e0902a00f406/_support.scssc new file mode 100644 index 00000000..8672e2e2 Binary files /dev/null and b/source/_themes/uwpce_slides2/static/.sass-cache/af1a5722249b61fe97c5776f7a26e0902a00f406/_support.scssc differ diff --git a/source/_themes/uwpce_slides2/static/README.md b/source/_themes/uwpce_slides2/static/README.md new file mode 100644 index 00000000..1ba53912 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/README.md @@ -0,0 +1,130 @@ + + +

HTML5 Slide Template

+ +## Configuring the slides + +Much of the deck is customized by changing the settings in [`slide_config.js`](slide_config.js). +Some of the customizations include the title, Analytics tracking ID, speaker +information (name, social urls, blog), web fonts to load, themes, and other +general behavior. + +### Customizing the `#io12` hash + +The bottom of the slides include `#io12` by default. If you'd like to change +this, please update the variable `$social-tags: '#io12';` in +[`/theme/scss/default.scss`](theme/scss/default.scss). + +See the next section on "Editing CSS" before you go editing things. + +## Editing CSS + +[Compass](http://compass-style.org/install/) is a CSS preprocessor used to compile +SCSS/SASS into CSS. We chose SCSS for the new slide deck for maintainability, +easier browser compatibility, and because...it's the future! + +That said, if not comfortable working with SCSS or don't want to learn something +new, not a problem. The generated .css files can already be found in +(see [`/theme/css`](theme/css)). You can just edit those and bypass SCSS altogether. +However, our recommendation is to use Compass. It's super easy to install and use. + +### Installing Compass and making changes + +First, install compass: + + sudo gem update --system + sudo gem install compass + +Next, you'll want to watch for changes to the exiting .scss files in [`/theme/scss`](theme/scss) +and any new one you add: + + $ cd io-2012-slides + $ compass watch + +This command automatically recompiles the .scss file when you make a change. +Its corresponding .css file is output to [`/theme/css`](theme/css). Slick. + +By default, [`config.rb`](config.rb) in the main project folder outputs minified +.css. It's a best practice after all! However, if you want unminified files, +run watch with the style output flag: + + compass watch -s expanded + +*Note:* You should not need to edit [`_base.scss`](theme/scss/_base.scss). + +## Running the slides + +The slides can be run locally from `file://` making development easy :) + +### Running from a web server + +If at some point you should need a web server, use [`serve.sh`](serve.sh). It will +launch a simple one and point your default browser to [`http://localhost:8000/template.html`](http://localhost:8000/template.html): + + $ cd io-2012-slides + $ ./serve.sh + +You can also specify a custom port: + + $ ./serve.sh 8080 + +### Presenter mode + +The slides contain a presenter mode feature (beta) to view + control the slides +from a popup window. + +To enable presenter mode, add `presentme=true` to the URL: [http://localhost:8000/template.html?presentme=true](http://localhost:8000/template.html?presentme=true) + +To disable presenter mode, hit [http://localhost:8000/template.html?presentme=false](http://localhost:8000/template.html?presentme=false) + +Presenter mode is sticky, so refreshing the page will persist your settings. + +--- + +That's all she wrote! diff --git a/source/_themes/uwpce_slides2/static/config.rb b/source/_themes/uwpce_slides2/static/config.rb new file mode 100644 index 00000000..e435e43a --- /dev/null +++ b/source/_themes/uwpce_slides2/static/config.rb @@ -0,0 +1,24 @@ +# Require any additional compass plugins here. + +# Set this to the root of your project when deployed: +http_path = "/" +css_dir = "theme/css" +sass_dir = "theme/scss" +images_dir = "images" +javascripts_dir = "js" + +# You can select your preferred output style here (can be overridden via the command line): +output_style = :expanded #:expanded or :nested or :compact or :compressed + +# To enable relative paths to assets via compass helper functions. Uncomment: +# relative_assets = true + +# To disable debugging comments that display the original location of your selectors. Uncomment: +# line_comments = false + + +# If you prefer the indented syntax, you might want to regenerate this +# project again passing --syntax sass, or you can uncomment this: +# preferred_syntax = :sass +# and then run: +# sass-convert -R --from scss --to sass sass scss && rm -rf sass && mv scss sass diff --git a/source/_themes/uwpce_slides2/static/js/hammer.js b/source/_themes/uwpce_slides2/static/js/hammer.js new file mode 100755 index 00000000..44a5802e --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/hammer.js @@ -0,0 +1,586 @@ +/* + * Hammer.JS + * version 0.4 + * author: Eight Media + * https://github.com/EightMedia/hammer.js + */ +function Hammer(element, options, undefined) +{ + var self = this; + + var defaults = { + // prevent the default event or not... might be buggy when false + prevent_default : false, + css_hacks : true, + + drag : true, + drag_vertical : true, + drag_horizontal : true, + // minimum distance before the drag event starts + drag_min_distance : 20, // pixels + + // pinch zoom and rotation + transform : true, + scale_treshold : 0.1, + rotation_treshold : 15, // degrees + + tap : true, + tap_double : true, + tap_max_interval : 300, + tap_double_distance: 20, + + hold : true, + hold_timeout : 500 + }; + options = mergeObject(defaults, options); + + // some css hacks + (function() { + if(!options.css_hacks) { + return false; + } + + var vendors = ['webkit','moz','ms','o','']; + var css_props = { + "userSelect": "none", + "touchCallout": "none", + "userDrag": "none", + "tapHighlightColor": "rgba(0,0,0,0)" + }; + + var prop = ''; + for(var i = 0; i < vendors.length; i++) { + for(var p in css_props) { + prop = p; + if(vendors[i]) { + prop = vendors[i] + prop.substring(0, 1).toUpperCase() + prop.substring(1); + } + element.style[ prop ] = css_props[p]; + } + } + })(); + + // holds the distance that has been moved + var _distance = 0; + + // holds the exact angle that has been moved + var _angle = 0; + + // holds the diraction that has been moved + var _direction = 0; + + // holds position movement for sliding + var _pos = { }; + + // how many fingers are on the screen + var _fingers = 0; + + var _first = false; + + var _gesture = null; + var _prev_gesture = null; + + var _touch_start_time = null; + var _prev_tap_pos = {x: 0, y: 0}; + var _prev_tap_end_time = null; + + var _hold_timer = null; + + var _offset = {}; + + // keep track of the mouse status + var _mousedown = false; + + var _event_start; + var _event_move; + var _event_end; + + + /** + * angle to direction define + * @param float angle + * @return string direction + */ + this.getDirectionFromAngle = function( angle ) + { + var directions = { + down: angle >= 45 && angle < 135, //90 + left: angle >= 135 || angle <= -135, //180 + up: angle < -45 && angle > -135, //270 + right: angle >= -45 && angle <= 45 //0 + }; + + var direction, key; + for(key in directions){ + if(directions[key]){ + direction = key; + break; + } + } + return direction; + }; + + + /** + * count the number of fingers in the event + * when no fingers are detected, one finger is returned (mouse pointer) + * @param event + * @return int fingers + */ + function countFingers( event ) + { + // there is a bug on android (until v4?) that touches is always 1, + // so no multitouch is supported, e.g. no, zoom and rotation... + return event.touches ? event.touches.length : 1; + } + + + /** + * get the x and y positions from the event object + * @param event + * @return array [{ x: int, y: int }] + */ + function getXYfromEvent( event ) + { + event = event || window.event; + + // no touches, use the event pageX and pageY + if(!event.touches) { + var doc = document, + body = doc.body; + + return [{ + x: event.pageX || event.clientX + ( doc && doc.scrollLeft || body && body.scrollLeft || 0 ) - ( doc && doc.clientLeft || body && doc.clientLeft || 0 ), + y: event.pageY || event.clientY + ( doc && doc.scrollTop || body && body.scrollTop || 0 ) - ( doc && doc.clientTop || body && doc.clientTop || 0 ) + }]; + } + // multitouch, return array with positions + else { + var pos = [], src; + for(var t=0, len=event.touches.length; t options.drag_min_distance) || _gesture == 'drag') { + // calculate the angle + _angle = getAngle(_pos.start[0], _pos.move[0]); + _direction = self.getDirectionFromAngle(_angle); + + // check the movement and stop if we go in the wrong direction + var is_vertical = (_direction == 'up' || _direction == 'down'); + if(((is_vertical && !options.drag_vertical) || (!is_vertical && !options.drag_horizontal)) + && (_distance > options.drag_min_distance)) { + return; + } + + _gesture = 'drag'; + + var position = { x: _pos.move[0].x - _offset.left, + y: _pos.move[0].y - _offset.top }; + + var event_obj = { + originalEvent : event, + position : position, + direction : _direction, + distance : _distance, + distanceX : _distance_x, + distanceY : _distance_y, + angle : _angle + }; + + // on the first time trigger the start event + if(_first) { + triggerEvent("dragstart", event_obj); + + _first = false; + } + + // normal slide event + triggerEvent("drag", event_obj); + + cancelEvent(event); + } + }, + + + // transform gesture + // fired on touchmove + transform : function(event) + { + if(options.transform) { + var scale = event.scale || 1; + var rotation = event.rotation || 0; + + if(countFingers(event) != 2) { + return false; + } + + if(_gesture != 'drag' && + (_gesture == 'transform' || Math.abs(1-scale) > options.scale_treshold + || Math.abs(rotation) > options.rotation_treshold)) { + _gesture = 'transform'; + + _pos.center = { x: ((_pos.move[0].x + _pos.move[1].x) / 2) - _offset.left, + y: ((_pos.move[0].y + _pos.move[1].y) / 2) - _offset.top }; + + var event_obj = { + originalEvent : event, + position : _pos.center, + scale : scale, + rotation : rotation + }; + + // on the first time trigger the start event + if(_first) { + triggerEvent("transformstart", event_obj); + _first = false; + } + + triggerEvent("transform", event_obj); + + cancelEvent(event); + + return true; + } + } + + return false; + }, + + + // tap and double tap gesture + // fired on touchend + tap : function(event) + { + // compare the kind of gesture by time + var now = new Date().getTime(); + var touch_time = now - _touch_start_time; + + // dont fire when hold is fired + if(options.hold && !(options.hold && options.hold_timeout > touch_time)) { + return; + } + + // when previous event was tap and the tap was max_interval ms ago + var is_double_tap = (function(){ + if (_prev_tap_pos && options.tap_double && _prev_gesture == 'tap' && (_touch_start_time - _prev_tap_end_time) < options.tap_max_interval) { + var x_distance = Math.abs(_prev_tap_pos[0].x - _pos.start[0].x); + var y_distance = Math.abs(_prev_tap_pos[0].y - _pos.start[0].y); + return (_prev_tap_pos && _pos.start && Math.max(x_distance, y_distance) < options.tap_double_distance); + + } + return false; + })(); + + if(is_double_tap) { + _gesture = 'double_tap'; + _prev_tap_end_time = null; + + triggerEvent("doubletap", { + originalEvent : event, + position : _pos.start + }); + cancelEvent(event); + } + + // single tap is single touch + else { + _gesture = 'tap'; + _prev_tap_end_time = now; + _prev_tap_pos = _pos.start; + + if(options.tap) { + triggerEvent("tap", { + originalEvent : event, + position : _pos.start + }); + cancelEvent(event); + } + } + + } + + }; + + + function handleEvents(event) + { + switch(event.type) + { + case 'mousedown': + case 'touchstart': + _pos.start = getXYfromEvent(event); + _touch_start_time = new Date().getTime(); + _fingers = countFingers(event); + _first = true; + _event_start = event; + + // borrowed from jquery offset https://github.com/jquery/jquery/blob/master/src/offset.js + var box = element.getBoundingClientRect(); + var clientTop = element.clientTop || document.body.clientTop || 0; + var clientLeft = element.clientLeft || document.body.clientLeft || 0; + var scrollTop = window.pageYOffset || element.scrollTop || document.body.scrollTop; + var scrollLeft = window.pageXOffset || element.scrollLeft || document.body.scrollLeft; + + _offset = { + top: box.top + scrollTop - clientTop, + left: box.left + scrollLeft - clientLeft + }; + + _mousedown = true; + + // hold gesture + gestures.hold(event); + + if(options.prevent_default) { + cancelEvent(event); + } + break; + + case 'mousemove': + case 'touchmove': + if(!_mousedown) { + return false; + } + _event_move = event; + _pos.move = getXYfromEvent(event); + + if(!gestures.transform(event)) { + gestures.drag(event); + } + break; + + case 'mouseup': + case 'mouseout': + case 'touchcancel': + case 'touchend': + if(!_mousedown || (_gesture != 'transform' && event.touches && event.touches.length > 0)) { + return false; + } + + _mousedown = false; + _event_end = event; + + // drag gesture + // dragstart is triggered, so dragend is possible + if(_gesture == 'drag') { + triggerEvent("dragend", { + originalEvent : event, + direction : _direction, + distance : _distance, + angle : _angle + }); + } + + // transform + // transformstart is triggered, so transformed is possible + else if(_gesture == 'transform') { + triggerEvent("transformend", { + originalEvent : event, + position : _pos.center, + scale : event.scale, + rotation : event.rotation + }); + } + else { + gestures.tap(_event_start); + } + + _prev_gesture = _gesture; + + // reset vars + reset(); + break; + } + } + + + // bind events for touch devices + // except for windows phone 7.5, it doesnt support touch events..! + if('ontouchstart' in window) { + element.addEventListener("touchstart", handleEvents, false); + element.addEventListener("touchmove", handleEvents, false); + element.addEventListener("touchend", handleEvents, false); + element.addEventListener("touchcancel", handleEvents, false); + } + // for non-touch + else { + + if(element.addEventListener){ // prevent old IE errors + element.addEventListener("mouseout", function(event) { + if(!isInsideHammer(element, event.relatedTarget)) { + handleEvents(event); + } + }, false); + element.addEventListener("mouseup", handleEvents, false); + element.addEventListener("mousedown", handleEvents, false); + element.addEventListener("mousemove", handleEvents, false); + + // events for older IE + }else if(document.attachEvent){ + element.attachEvent("onmouseout", function(event) { + if(!isInsideHammer(element, event.relatedTarget)) { + handleEvents(event); + } + }, false); + element.attachEvent("onmouseup", handleEvents); + element.attachEvent("onmousedown", handleEvents); + element.attachEvent("onmousemove", handleEvents); + } + } + + + /** + * find if element is (inside) given parent element + * @param object element + * @param object parent + * @return bool inside + */ + function isInsideHammer(parent, child) { + // get related target for IE + if(!child && window.event && window.event.toElement){ + child = window.event.toElement; + } + + if(parent === child){ + return true; + } + + // loop over parentNodes of child until we find hammer element + if(child){ + var node = child.parentNode; + while(node !== null){ + if(node === parent){ + return true; + }; + node = node.parentNode; + } + } + return false; + } + + + /** + * merge 2 objects into a new object + * @param object obj1 + * @param object obj2 + * @return object merged object + */ + function mergeObject(obj1, obj2) { + var output = {}; + + if(!obj2) { + return obj1; + } + + for (var prop in obj1) { + if (prop in obj2) { + output[prop] = obj2[prop]; + } else { + output[prop] = obj1[prop]; + } + } + return output; + } + + function isFunction( obj ){ + return Object.prototype.toString.call( obj ) == "[object Function]"; + } +} \ No newline at end of file diff --git a/source/_themes/uwpce_slides2/static/js/modernizr.custom.45394.js b/source/_themes/uwpce_slides2/static/js/modernizr.custom.45394.js new file mode 100644 index 00000000..26f38cdc --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/modernizr.custom.45394.js @@ -0,0 +1,4 @@ +/* Modernizr 2.5.3 (Custom Build) | MIT & BSD + * Build: http://www.modernizr.com/download/#-fontface-backgroundsize-borderimage-borderradius-boxshadow-flexbox-flexbox_legacy-hsla-multiplebgs-opacity-rgba-textshadow-cssanimations-csscolumns-generatedcontent-cssgradients-cssreflections-csstransforms-csstransforms3d-csstransitions-applicationcache-canvas-canvastext-draganddrop-hashchange-history-audio-video-indexeddb-input-inputtypes-localstorage-postmessage-sessionstorage-websockets-websqldatabase-webworkers-geolocation-inlinesvg-smil-svg-svgclippaths-touch-webgl-mq-prefixed-teststyles-testprop-testallprops-hasevent-prefixes-domprefixes-load + */ +;window.Modernizr=function(a,b,c){function C(a){i.cssText=a}function D(a,b){return C(m.join(a+";")+(b||""))}function E(a,b){return typeof a===b}function F(a,b){return!!~(""+a).indexOf(b)}function G(a,b){for(var d in a)if(i[a[d]]!==c)return b=="pfx"?a[d]:!0;return!1}function H(a,b,d){for(var e in a){var f=b[a[e]];if(f!==c)return d===!1?a[e]:E(f,"function")?f.bind(d||b):f}return!1}function I(a,b,c){var d=a.charAt(0).toUpperCase()+a.substr(1),e=(a+" "+o.join(d+" ")+d).split(" ");return E(b,"string")||E(b,"undefined")?G(e,b):(e=(a+" "+p.join(d+" ")+d).split(" "),H(e,b,c))}function K(){e.input=function(c){for(var d=0,e=c.length;d",a,""].join(""),k.id=g,(l?k:m).innerHTML+=h,m.appendChild(k),l||(m.style.background="",f.appendChild(m)),i=c(k,a),l?k.parentNode.removeChild(k):m.parentNode.removeChild(m),!!i},y=function(b){var c=a.matchMedia||a.msMatchMedia;if(c)return c(b).matches;var d;return x("@media "+b+" { #"+g+" { position: absolute; } }",function(b){d=(a.getComputedStyle?getComputedStyle(b,null):b.currentStyle)["position"]=="absolute"}),d},z=function(){function d(d,e){e=e||b.createElement(a[d]||"div"),d="on"+d;var f=d in e;return f||(e.setAttribute||(e=b.createElement("div")),e.setAttribute&&e.removeAttribute&&(e.setAttribute(d,""),f=E(e[d],"function"),E(e[d],"undefined")||(e[d]=c),e.removeAttribute(d))),e=null,f}var a={select:"input",change:"input",submit:"form",reset:"form",error:"img",load:"img",abort:"img"};return d}(),A={}.hasOwnProperty,B;!E(A,"undefined")&&!E(A.call,"undefined")?B=function(a,b){return A.call(a,b)}:B=function(a,b){return b in a&&E(a.constructor.prototype[b],"undefined")},Function.prototype.bind||(Function.prototype.bind=function(b){var c=this;if(typeof c!="function")throw new TypeError;var d=v.call(arguments,1),e=function(){if(this instanceof e){var a=function(){};a.prototype=c.prototype;var f=new a,g=c.apply(f,d.concat(v.call(arguments)));return Object(g)===g?g:f}return c.apply(b,d.concat(v.call(arguments)))};return e});var J=function(c,d){var f=c.join(""),g=d.length;x(f,function(c,d){var f=b.styleSheets[b.styleSheets.length-1],h=f?f.cssRules&&f.cssRules[0]?f.cssRules[0].cssText:f.cssText||"":"",i=c.childNodes,j={};while(g--)j[i[g].id]=i[g];e.touch="ontouchstart"in a||a.DocumentTouch&&b instanceof DocumentTouch||(j.touch&&j.touch.offsetTop)===9,e.csstransforms3d=(j.csstransforms3d&&j.csstransforms3d.offsetLeft)===9&&j.csstransforms3d.offsetHeight===3,e.generatedcontent=(j.generatedcontent&&j.generatedcontent.offsetHeight)>=1,e.fontface=/src/i.test(h)&&h.indexOf(d.split(" ")[0])===0},g,d)}(['@font-face {font-family:"font";src:url("https://")}',["@media (",m.join("touch-enabled),("),g,")","{#touch{top:9px;position:absolute}}"].join(""),["@media (",m.join("transform-3d),("),g,")","{#csstransforms3d{left:9px;position:absolute;height:3px;}}"].join(""),['#generatedcontent:after{content:"',k,'";visibility:hidden}'].join("")],["fontface","touch","csstransforms3d","generatedcontent"]);r.flexbox=function(){return I("flexOrder")},r["flexbox-legacy"]=function(){return I("boxDirection")},r.canvas=function(){var a=b.createElement("canvas");return!!a.getContext&&!!a.getContext("2d")},r.canvastext=function(){return!!e.canvas&&!!E(b.createElement("canvas").getContext("2d").fillText,"function")},r.webgl=function(){try{var d=b.createElement("canvas"),e;e=!(!a.WebGLRenderingContext||!d.getContext("experimental-webgl")&&!d.getContext("webgl")),d=c}catch(f){e=!1}return e},r.touch=function(){return e.touch},r.geolocation=function(){return!!navigator.geolocation},r.postmessage=function(){return!!a.postMessage},r.websqldatabase=function(){return!!a.openDatabase},r.indexedDB=function(){return!!I("indexedDB",a)},r.hashchange=function(){return z("hashchange",a)&&(b.documentMode===c||b.documentMode>7)},r.history=function(){return!!a.history&&!!history.pushState},r.draganddrop=function(){var a=b.createElement("div");return"draggable"in a||"ondragstart"in a&&"ondrop"in a},r.websockets=function(){for(var b=-1,c=o.length;++b0&&g.splice(0,a);setTimeout(function(){b.parentNode.removeChild(b)},15)}}function m(a){var b,c;a.setAttribute("data-orderloaded","loaded");for(a=0;c=h[a];a++)if((b=i[c])&&b.getAttribute("data-orderloaded")==="loaded")delete i[c],require.addScriptToDom(b);else break;a>0&&h.splice(0, +a)}var f=typeof document!=="undefined"&&typeof window!=="undefined"&&document.createElement("script"),n=f&&(f.async||window.opera&&Object.prototype.toString.call(window.opera)==="[object Opera]"||"MozAppearance"in document.documentElement.style),o=f&&f.readyState==="uninitialized",l=/^(complete|loaded)$/,g=[],j={},i={},h=[],f=null;define({version:"1.0.5",load:function(a,b,c,e){var d;b.nameToUrl?(d=b.nameToUrl(a,null),require.s.skipAsync[d]=!0,n||e.isBuild?b([a],c):o?(e=require.s.contexts._,!e.urlFetched[d]&& +!e.loaded[a]&&(e.urlFetched[d]=!0,require.resourcesReady(!1),e.scriptCount+=1,d=require.attach(d,e,a,null,null,m),i[a]=d,h.push(a)),b([a],c)):b.specified(a)?b([a],c):(g.push({name:a,req:b,onLoad:c}),require.attach(d,null,a,k,"script/cache"))):b([a],c)}})})(); diff --git a/source/_themes/uwpce_slides2/static/js/polyfills/classList.min.js b/source/_themes/uwpce_slides2/static/js/polyfills/classList.min.js new file mode 100644 index 00000000..932c7776 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/polyfills/classList.min.js @@ -0,0 +1,2 @@ +/* @source http://purl.eligrey.com/github/classList.js/blob/master/classList.js*/ +"use strict";if(typeof document!=="undefined"&&!("classList" in document.createElement("a"))){(function(a){var f="classList",d="prototype",e=(a.HTMLElement||a.Element)[d],g=Object;strTrim=String[d].trim||function(){return this.replace(/^\s+|\s+$/g,"")},arrIndexOf=Array[d].indexOf||function(k){for(var j=0,h=this.length;j")&&c[0]);return a>4?a:!1}();return a},m.isInternetExplorer=function(){var a=m.isInternetExplorer.cached=typeof m.isInternetExplorer.cached!="undefined"?m.isInternetExplorer.cached:Boolean(m.getInternetExplorerMajorVersion());return a},m.emulated={pushState:!Boolean(a.history&&a.history.pushState&&a.history.replaceState&&!/ Mobile\/([1-7][a-z]|(8([abcde]|f(1[0-8]))))/i.test(e.userAgent)&&!/AppleWebKit\/5([0-2]|3[0-2])/i.test(e.userAgent)),hashChange:Boolean(!("onhashchange"in a||"onhashchange"in d)||m.isInternetExplorer()&&m.getInternetExplorerMajorVersion()<8)},m.enabled=!m.emulated.pushState,m.bugs={setHash:Boolean(!m.emulated.pushState&&e.vendor==="Apple Computer, Inc."&&/AppleWebKit\/5([0-2]|3[0-3])/.test(e.userAgent)),safariPoll:Boolean(!m.emulated.pushState&&e.vendor==="Apple Computer, Inc."&&/AppleWebKit\/5([0-2]|3[0-3])/.test(e.userAgent)),ieDoubleCheck:Boolean(m.isInternetExplorer()&&m.getInternetExplorerMajorVersion()<8),hashEscape:Boolean(m.isInternetExplorer()&&m.getInternetExplorerMajorVersion()<7)},m.isEmptyObject=function(a){for(var b in a)return!1;return!0},m.cloneObject=function(a){var b,c;return a?(b=k.stringify(a),c=k.parse(b)):c={},c},m.getRootUrl=function(){var a=d.location.protocol+"//"+(d.location.hostname||d.location.host);if(d.location.port||!1)a+=":"+d.location.port;return a+="/",a},m.getBaseHref=function(){var a=d.getElementsByTagName("base"),b=null,c="";return a.length===1&&(b=a[0],c=b.href.replace(/[^\/]+$/,"")),c=c.replace(/\/+$/,""),c&&(c+="/"),c},m.getBaseUrl=function(){var a=m.getBaseHref()||m.getBasePageUrl()||m.getRootUrl();return a},m.getPageUrl=function(){var a=m.getState(!1,!1),b=(a||{}).url||d.location.href,c;return c=b.replace(/\/+$/,"").replace(/[^\/]+$/,function(a,b,c){return/\./.test(a)?a:a+"/"}),c},m.getBasePageUrl=function(){var a=d.location.href.replace(/[#\?].*/,"").replace(/[^\/]+$/,function(a,b,c){return/[^\/]$/.test(a)?"":a}).replace(/\/+$/,"")+"/";return a},m.getFullUrl=function(a,b){var c=a,d=a.substring(0,1);return b=typeof b=="undefined"?!0:b,/[a-z]+\:\/\//.test(a)||(d==="/"?c=m.getRootUrl()+a.replace(/^\/+/,""):d==="#"?c=m.getPageUrl().replace(/#.*/,"")+a:d==="?"?c=m.getPageUrl().replace(/[\?#].*/,"")+a:b?c=m.getBaseUrl()+a.replace(/^(\.\/)+/,""):c=m.getBasePageUrl()+a.replace(/^(\.\/)+/,"")),c.replace(/\#$/,"")},m.getShortUrl=function(a){var b=a,c=m.getBaseUrl(),d=m.getRootUrl();return m.emulated.pushState&&(b=b.replace(c,"")),b=b.replace(d,"/"),m.isTraditionalAnchor(b)&&(b="./"+b),b=b.replace(/^(\.\/)+/g,"./").replace(/\#$/,""),b},m.store={},m.idToState=m.idToState||{},m.stateToId=m.stateToId||{},m.urlToId=m.urlToId||{},m.storedStates=m.storedStates||[],m.savedStates=m.savedStates||[],m.normalizeStore=function(){m.store.idToState=m.store.idToState||{},m.store.urlToId=m.store.urlToId||{},m.store.stateToId=m.store.stateToId||{}},m.getState=function(a,b){typeof a=="undefined"&&(a=!0),typeof b=="undefined"&&(b=!0);var c=m.getLastSavedState();return!c&&b&&(c=m.createStateObject()),a&&(c=m.cloneObject(c),c.url=c.cleanUrl||c.url),c},m.getIdByState=function(a){var b=m.extractId(a.url),c;if(!b){c=m.getStateString(a);if(typeof m.stateToId[c]!="undefined")b=m.stateToId[c];else if(typeof m.store.stateToId[c]!="undefined")b=m.store.stateToId[c];else{for(;;){b=(new Date).getTime()+String(Math.random()).replace(/\D/g,"");if(typeof m.idToState[b]=="undefined"&&typeof m.store.idToState[b]=="undefined")break}m.stateToId[c]=b,m.idToState[b]=a}}return b},m.normalizeState=function(a){var b,c;if(!a||typeof a!="object")a={};if(typeof a.normalized!="undefined")return a;if(!a.data||typeof a.data!="object")a.data={};b={},b.normalized=!0,b.title=a.title||"",b.url=m.getFullUrl(m.unescapeString(a.url||d.location.href)),b.hash=m.getShortUrl(b.url),b.data=m.cloneObject(a.data),b.id=m.getIdByState(b),b.cleanUrl=b.url.replace(/\??\&_suid.*/,""),b.url=b.cleanUrl,c=!m.isEmptyObject(b.data);if(b.title||c)b.hash=m.getShortUrl(b.url).replace(/\??\&_suid.*/,""),/\?/.test(b.hash)||(b.hash+="?"),b.hash+="&_suid="+b.id;return b.hashedUrl=m.getFullUrl(b.hash),(m.emulated.pushState||m.bugs.safariPoll)&&m.hasUrlDuplicate(b)&&(b.url=b.hashedUrl),b},m.createStateObject=function(a,b,c){var d={data:a,title:b,url:c};return d=m.normalizeState(d),d},m.getStateById=function(a){a=String(a);var c=m.idToState[a]||m.store.idToState[a]||b;return c},m.getStateString=function(a){var b,c,d;return b=m.normalizeState(a),c={data:b.data,title:a.title,url:a.url},d=k.stringify(c),d},m.getStateId=function(a){var b,c;return b=m.normalizeState(a),c=b.id,c},m.getHashByState=function(a){var b,c;return b=m.normalizeState(a),c=b.hash,c},m.extractId=function(a){var b,c,d;return c=/(.*)\&_suid=([0-9]+)$/.exec(a),d=c?c[1]||a:a,b=c?String(c[2]||""):"",b||!1},m.isTraditionalAnchor=function(a){var b=!/[\/\?\.]/.test(a);return b},m.extractState=function(a,b){var c=null,d,e;return b=b||!1,d=m.extractId(a),d&&(c=m.getStateById(d)),c||(e=m.getFullUrl(a),d=m.getIdByUrl(e)||!1,d&&(c=m.getStateById(d)),!c&&b&&!m.isTraditionalAnchor(a)&&(c=m.createStateObject(null,null,e))),c},m.getIdByUrl=function(a){var c=m.urlToId[a]||m.store.urlToId[a]||b;return c},m.getLastSavedState=function(){return m.savedStates[m.savedStates.length-1]||b},m.getLastStoredState=function(){return m.storedStates[m.storedStates.length-1]||b},m.hasUrlDuplicate=function(a){var b=!1,c;return c=m.extractState(a.url),b=c&&c.id!==a.id,b},m.storeState=function(a){return m.urlToId[a.url]=a.id,m.storedStates.push(m.cloneObject(a)),a},m.isLastSavedState=function(a){var b=!1,c,d,e;return m.savedStates.length&&(c=a.id,d=m.getLastSavedState(),e=d.id,b=c===e),b},m.saveState=function(a){return m.isLastSavedState(a)?!1:(m.savedStates.push(m.cloneObject(a)),!0)},m.getStateByIndex=function(a){var b=null;return typeof a=="undefined"?b=m.savedStates[m.savedStates.length-1]:a<0?b=m.savedStates[m.savedStates.length+a]:b=m.savedStates[a],b},m.getHash=function(){var a=m.unescapeHash(d.location.hash);return a},m.unescapeString=function(b){var c=b,d;for(;;){d=a.unescape(c);if(d===c)break;c=d}return c},m.unescapeHash=function(a){var b=m.normalizeHash(a);return b=m.unescapeString(b),b},m.normalizeHash=function(a){var b=a.replace(/[^#]*#/,"").replace(/#.*/,"");return b},m.setHash=function(a,b){var c,e,f;return b!==!1&&m.busy()?(m.pushQueue({scope:m,callback:m.setHash,args:arguments,queue:b}),!1):(c=m.escapeHash(a),m.busy(!0),e=m.extractState(a,!0),e&&!m.emulated.pushState?m.pushState(e.data,e.title,e.url,!1):d.location.hash!==c&&(m.bugs.setHash?(f=m.getPageUrl(),m.pushState(null,null,f+"#"+c,!1)):d.location.hash=c),m)},m.escapeHash=function(b){var c=m.normalizeHash(b);return c=a.escape(c),m.bugs.hashEscape||(c=c.replace(/\%21/g,"!").replace(/\%26/g,"&").replace(/\%3D/g,"=").replace(/\%3F/g,"?")),c},m.getHashByUrl=function(a){var b=String(a).replace(/([^#]*)#?([^#]*)#?(.*)/,"$2");return b=m.unescapeHash(b),b},m.setTitle=function(a){var b=a.title,c;b||(c=m.getStateByIndex(0),c&&c.url===a.url&&(b=c.title||m.options.initialTitle));try{d.getElementsByTagName("title")[0].innerHTML=b.replace("<","<").replace(">",">").replace(" & "," & ")}catch(e){}return d.title=b,m},m.queues=[],m.busy=function(a){typeof a!="undefined"?m.busy.flag=a:typeof m.busy.flag=="undefined"&&(m.busy.flag=!1);if(!m.busy.flag){h(m.busy.timeout);var b=function(){var a,c,d;if(m.busy.flag)return;for(a=m.queues.length-1;a>=0;--a){c=m.queues[a];if(c.length===0)continue;d=c.shift(),m.fireQueueItem(d),m.busy.timeout=g(b,m.options.busyDelay)}};m.busy.timeout=g(b,m.options.busyDelay)}return m.busy.flag},m.busy.flag=!1,m.fireQueueItem=function(a){return a.callback.apply(a.scope||m,a.args||[])},m.pushQueue=function(a){return m.queues[a.queue||0]=m.queues[a.queue||0]||[],m.queues[a.queue||0].push(a),m},m.queue=function(a,b){return typeof a=="function"&&(a={callback:a}),typeof b!="undefined"&&(a.queue=b),m.busy()?m.pushQueue(a):m.fireQueueItem(a),m},m.clearQueue=function(){return m.busy.flag=!1,m.queues=[],m},m.stateChanged=!1,m.doubleChecker=!1,m.doubleCheckComplete=function(){return m.stateChanged=!0,m.doubleCheckClear(),m},m.doubleCheckClear=function(){return m.doubleChecker&&(h(m.doubleChecker),m.doubleChecker=!1),m},m.doubleCheck=function(a){return m.stateChanged=!1,m.doubleCheckClear(),m.bugs.ieDoubleCheck&&(m.doubleChecker=g(function(){return m.doubleCheckClear(),m.stateChanged||a(),!0},m.options.doubleCheckInterval)),m},m.safariStatePoll=function(){var b=m.extractState(d.location.href),c;if(!m.isLastSavedState(b))c=b;else return;return c||(c=m.createStateObject()),m.Adapter.trigger(a,"popstate"),m},m.back=function(a){return a!==!1&&m.busy()?(m.pushQueue({scope:m,callback:m.back,args:arguments,queue:a}),!1):(m.busy(!0),m.doubleCheck(function(){m.back(!1)}),n.go(-1),!0)},m.forward=function(a){return a!==!1&&m.busy()?(m.pushQueue({scope:m,callback:m.forward,args:arguments,queue:a}),!1):(m.busy(!0),m.doubleCheck(function(){m.forward(!1)}),n.go(1),!0)},m.go=function(a,b){var c;if(a>0)for(c=1;c<=a;++c)m.forward(b);else{if(!(a<0))throw new Error("History.go: History.go requires a positive or negative integer passed.");for(c=-1;c>=a;--c)m.back(b)}return m};if(m.emulated.pushState){var o=function(){};m.pushState=m.pushState||o,m.replaceState=m.replaceState||o}else m.onPopState=function(b,c){var e=!1,f=!1,g,h;return m.doubleCheckComplete(),g=m.getHash(),g?(h=m.extractState(g||d.location.href,!0),h?m.replaceState(h.data,h.title,h.url,!1):(m.Adapter.trigger(a,"anchorchange"),m.busy(!1)),m.expectedStateId=!1,!1):(e=m.Adapter.extractEventData("state",b,c)||!1,e?f=m.getStateById(e):m.expectedStateId?f=m.getStateById(m.expectedStateId):f=m.extractState(d.location.href),f||(f=m.createStateObject(null,null,d.location.href)),m.expectedStateId=!1,m.isLastSavedState(f)?(m.busy(!1),!1):(m.storeState(f),m.saveState(f),m.setTitle(f),m.Adapter.trigger(a,"statechange"),m.busy(!1),!0))},m.Adapter.bind(a,"popstate",m.onPopState),m.pushState=function(b,c,d,e){if(m.getHashByUrl(d)&&m.emulated.pushState)throw new Error("History.js does not support states with fragement-identifiers (hashes/anchors).");if(e!==!1&&m.busy())return m.pushQueue({scope:m,callback:m.pushState,args:arguments,queue:e}),!1;m.busy(!0);var f=m.createStateObject(b,c,d);return m.isLastSavedState(f)?m.busy(!1):(m.storeState(f),m.expectedStateId=f.id,n.pushState(f.id,f.title,f.url),m.Adapter.trigger(a,"popstate")),!0},m.replaceState=function(b,c,d,e){if(m.getHashByUrl(d)&&m.emulated.pushState)throw new Error("History.js does not support states with fragement-identifiers (hashes/anchors).");if(e!==!1&&m.busy())return m.pushQueue({scope:m,callback:m.replaceState,args:arguments,queue:e}),!1;m.busy(!0);var f=m.createStateObject(b,c,d);return m.isLastSavedState(f)?m.busy(!1):(m.storeState(f),m.expectedStateId=f.id,n.replaceState(f.id,f.title,f.url),m.Adapter.trigger(a,"popstate")),!0};if(f){try{m.store=k.parse(f.getItem("History.store"))||{}}catch(p){m.store={}}m.normalizeStore()}else m.store={},m.normalizeStore();m.Adapter.bind(a,"beforeunload",m.clearAllIntervals),m.Adapter.bind(a,"unload",m.clearAllIntervals),m.saveState(m.storeState(m.extractState(d.location.href,!0))),f&&(m.onUnload=function(){var a,b;try{a=k.parse(f.getItem("History.store"))||{}}catch(c){a={}}a.idToState=a.idToState||{},a.urlToId=a.urlToId||{},a.stateToId=a.stateToId||{};for(b in m.idToState){if(!m.idToState.hasOwnProperty(b))continue;a.idToState[b]=m.idToState[b]}for(b in m.urlToId){if(!m.urlToId.hasOwnProperty(b))continue;a.urlToId[b]=m.urlToId[b]}for(b in m.stateToId){if(!m.stateToId.hasOwnProperty(b))continue;a.stateToId[b]=m.stateToId[b]}m.store=a,m.normalizeStore(),f.setItem("History.store",k.stringify(a))},m.intervalList.push(i(m.onUnload,m.options.storeInterval)),m.Adapter.bind(a,"beforeunload",m.onUnload),m.Adapter.bind(a,"unload",m.onUnload));if(!m.emulated.pushState){m.bugs.safariPoll&&m.intervalList.push(i(m.safariStatePoll,m.options.safariPollInterval));if(e.vendor==="Apple Computer, Inc."||(e.appCodeName||"")==="Mozilla")m.Adapter.bind(a,"hashchange",function(){m.Adapter.trigger(a,"popstate")}),m.getHash()&&m.Adapter.onDomLoad(function(){m.Adapter.trigger(a,"hashchange")})}},m.init()})(window) \ No newline at end of file diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-apollo.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-apollo.js new file mode 100644 index 00000000..7098baf4 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-apollo.js @@ -0,0 +1,2 @@ +PR.registerLangHandler(PR.createSimpleLexer([["com",/^#[^\n\r]*/,null,"#"],["pln",/^[\t\n\r \xa0]+/,null,"\t\n\r \xa0"],["str",/^"(?:[^"\\]|\\[\S\s])*(?:"|$)/,null,'"']],[["kwd",/^(?:ADS|AD|AUG|BZF|BZMF|CAE|CAF|CA|CCS|COM|CS|DAS|DCA|DCOM|DCS|DDOUBL|DIM|DOUBLE|DTCB|DTCF|DV|DXCH|EDRUPT|EXTEND|INCR|INDEX|NDX|INHINT|LXCH|MASK|MSK|MP|MSU|NOOP|OVSK|QXCH|RAND|READ|RELINT|RESUME|RETURN|ROR|RXOR|SQUARE|SU|TCR|TCAA|OVSK|TCF|TC|TS|WAND|WOR|WRITE|XCH|XLQ|XXALQ|ZL|ZQ|ADD|ADZ|SUB|SUZ|MPY|MPR|MPZ|DVP|COM|ABS|CLA|CLZ|LDQ|STO|STQ|ALS|LLS|LRS|TRA|TSQ|TMI|TOV|AXT|TIX|DLY|INP|OUT)\s/, +null],["typ",/^(?:-?GENADR|=MINUS|2BCADR|VN|BOF|MM|-?2CADR|-?[1-6]DNADR|ADRES|BBCON|[ES]?BANK=?|BLOCK|BNKSUM|E?CADR|COUNT\*?|2?DEC\*?|-?DNCHAN|-?DNPTR|EQUALS|ERASE|MEMORY|2?OCT|REMADR|SETLOC|SUBRO|ORG|BSS|BES|SYN|EQU|DEFINE|END)\s/,null],["lit",/^'(?:-*(?:\w|\\[!-~])(?:[\w-]*|\\[!-~])[!=?]?)?/],["pln",/^-*(?:[!-z]|\\[!-~])(?:[\w-]*|\\[!-~])[!=?]?/],["pun",/^[^\w\t\n\r "'-);\\\xa0]+/]]),["apollo","agc","aea"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-clj.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-clj.js new file mode 100644 index 00000000..542a2205 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-clj.js @@ -0,0 +1,18 @@ +/* + Copyright (C) 2011 Google Inc. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ +var a=null; +PR.registerLangHandler(PR.createSimpleLexer([["opn",/^[([{]+/,a,"([{"],["clo",/^[)\]}]+/,a,")]}"],["com",/^;[^\n\r]*/,a,";"],["pln",/^[\t\n\r \xa0]+/,a,"\t\n\r \xa0"],["str",/^"(?:[^"\\]|\\[\S\s])*(?:"|$)/,a,'"']],[["kwd",/^(?:def|if|do|let|quote|var|fn|loop|recur|throw|try|monitor-enter|monitor-exit|defmacro|defn|defn-|macroexpand|macroexpand-1|for|doseq|dosync|dotimes|and|or|when|not|assert|doto|proxy|defstruct|first|rest|cons|defprotocol|deftype|defrecord|reify|defmulti|defmethod|meta|with-meta|ns|in-ns|create-ns|import|intern|refer|alias|namespace|resolve|ref|deref|refset|new|set!|memfn|to-array|into-array|aset|gen-class|reduce|map|filter|find|nil?|empty?|hash-map|hash-set|vec|vector|seq|flatten|reverse|assoc|dissoc|list|list?|disj|get|union|difference|intersection|extend|extend-type|extend-protocol|prn)\b/,a], +["typ",/^:[\dA-Za-z-]+/]]),["clj"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-css.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-css.js new file mode 100644 index 00000000..041e1f59 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-css.js @@ -0,0 +1,2 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t\n\f\r ]+/,null," \t\r\n "]],[["str",/^"(?:[^\n\f\r"\\]|\\(?:\r\n?|\n|\f)|\\[\S\s])*"/,null],["str",/^'(?:[^\n\f\r'\\]|\\(?:\r\n?|\n|\f)|\\[\S\s])*'/,null],["lang-css-str",/^url\(([^"')]*)\)/i],["kwd",/^(?:url|rgb|!important|@import|@page|@media|@charset|inherit)(?=[^\w-]|$)/i,null],["lang-css-kw",/^(-?(?:[_a-z]|\\[\da-f]+ ?)(?:[\w-]|\\\\[\da-f]+ ?)*)\s*:/i],["com",/^\/\*[^*]*\*+(?:[^*/][^*]*\*+)*\//],["com", +/^(?:<\!--|--\>)/],["lit",/^(?:\d+|\d*\.\d+)(?:%|[a-z]+)?/i],["lit",/^#[\da-f]{3,6}/i],["pln",/^-?(?:[_a-z]|\\[\da-f]+ ?)(?:[\w-]|\\\\[\da-f]+ ?)*/i],["pun",/^[^\s\w"']+/]]),["css"]);PR.registerLangHandler(PR.createSimpleLexer([],[["kwd",/^-?(?:[_a-z]|\\[\da-f]+ ?)(?:[\w-]|\\\\[\da-f]+ ?)*/i]]),["css-kw"]);PR.registerLangHandler(PR.createSimpleLexer([],[["str",/^[^"')]+/]]),["css-str"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-go.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-go.js new file mode 100644 index 00000000..fc18dc07 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-go.js @@ -0,0 +1 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t\n\r \xa0]+/,null,"\t\n\r \xa0"],["pln",/^(?:"(?:[^"\\]|\\[\S\s])*(?:"|$)|'(?:[^'\\]|\\[\S\s])+(?:'|$)|`[^`]*(?:`|$))/,null,"\"'"]],[["com",/^(?:\/\/[^\n\r]*|\/\*[\S\s]*?\*\/)/],["pln",/^(?:[^"'/`]|\/(?![*/]))+/]]),["go"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-hs.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-hs.js new file mode 100644 index 00000000..9d77b083 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-hs.js @@ -0,0 +1,2 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t-\r ]+/,null,"\t\n \r "],["str",/^"(?:[^\n\f\r"\\]|\\[\S\s])*(?:"|$)/,null,'"'],["str",/^'(?:[^\n\f\r'\\]|\\[^&])'?/,null,"'"],["lit",/^(?:0o[0-7]+|0x[\da-f]+|\d+(?:\.\d+)?(?:e[+-]?\d+)?)/i,null,"0123456789"]],[["com",/^(?:--+[^\n\f\r]*|{-(?:[^-]|-+[^}-])*-})/],["kwd",/^(?:case|class|data|default|deriving|do|else|if|import|in|infix|infixl|infixr|instance|let|module|newtype|of|then|type|where|_)(?=[^\d'A-Za-z]|$)/, +null],["pln",/^(?:[A-Z][\w']*\.)*[A-Za-z][\w']*/],["pun",/^[^\d\t-\r "'A-Za-z]+/]]),["hs"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-lisp.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-lisp.js new file mode 100644 index 00000000..02a30e8d --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-lisp.js @@ -0,0 +1,3 @@ +var a=null; +PR.registerLangHandler(PR.createSimpleLexer([["opn",/^\(+/,a,"("],["clo",/^\)+/,a,")"],["com",/^;[^\n\r]*/,a,";"],["pln",/^[\t\n\r \xa0]+/,a,"\t\n\r \xa0"],["str",/^"(?:[^"\\]|\\[\S\s])*(?:"|$)/,a,'"']],[["kwd",/^(?:block|c[ad]+r|catch|con[ds]|def(?:ine|un)|do|eq|eql|equal|equalp|eval-when|flet|format|go|if|labels|lambda|let|load-time-value|locally|macrolet|multiple-value-call|nil|progn|progv|quote|require|return-from|setq|symbol-macrolet|t|tagbody|the|throw|unwind)\b/,a], +["lit",/^[+-]?(?:[#0]x[\da-f]+|\d+\/\d+|(?:\.\d+|\d+(?:\.\d*)?)(?:[de][+-]?\d+)?)/i],["lit",/^'(?:-*(?:\w|\\[!-~])(?:[\w-]*|\\[!-~])[!=?]?)?/],["pln",/^-*(?:[_a-z]|\\[!-~])(?:[\w-]*|\\[!-~])[!=?]?/i],["pun",/^[^\w\t\n\r "'-);\\\xa0]+/]]),["cl","el","lisp","scm"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-lua.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-lua.js new file mode 100644 index 00000000..e83a3c46 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-lua.js @@ -0,0 +1,2 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t\n\r \xa0]+/,null,"\t\n\r \xa0"],["str",/^(?:"(?:[^"\\]|\\[\S\s])*(?:"|$)|'(?:[^'\\]|\\[\S\s])*(?:'|$))/,null,"\"'"]],[["com",/^--(?:\[(=*)\[[\S\s]*?(?:]\1]|$)|[^\n\r]*)/],["str",/^\[(=*)\[[\S\s]*?(?:]\1]|$)/],["kwd",/^(?:and|break|do|else|elseif|end|false|for|function|if|in|local|nil|not|or|repeat|return|then|true|until|while)\b/,null],["lit",/^[+-]?(?:0x[\da-f]+|(?:\.\d+|\d+(?:\.\d*)?)(?:e[+-]?\d+)?)/i], +["pln",/^[_a-z]\w*/i],["pun",/^[^\w\t\n\r \xa0][^\w\t\n\r "'+=\xa0-]*/]]),["lua"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-ml.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-ml.js new file mode 100644 index 00000000..6df02d72 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-ml.js @@ -0,0 +1,2 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t\n\r \xa0]+/,null,"\t\n\r \xa0"],["com",/^#(?:if[\t\n\r \xa0]+(?:[$_a-z][\w']*|``[^\t\n\r`]*(?:``|$))|else|endif|light)/i,null,"#"],["str",/^(?:"(?:[^"\\]|\\[\S\s])*(?:"|$)|'(?:[^'\\]|\\[\S\s])(?:'|$))/,null,"\"'"]],[["com",/^(?:\/\/[^\n\r]*|\(\*[\S\s]*?\*\))/],["kwd",/^(?:abstract|and|as|assert|begin|class|default|delegate|do|done|downcast|downto|elif|else|end|exception|extern|false|finally|for|fun|function|if|in|inherit|inline|interface|internal|lazy|let|match|member|module|mutable|namespace|new|null|of|open|or|override|private|public|rec|return|static|struct|then|to|true|try|type|upcast|use|val|void|when|while|with|yield|asr|land|lor|lsl|lsr|lxor|mod|sig|atomic|break|checked|component|const|constraint|constructor|continue|eager|event|external|fixed|functor|global|include|method|mixin|object|parallel|process|protected|pure|sealed|trait|virtual|volatile)\b/], +["lit",/^[+-]?(?:0x[\da-f]+|(?:\.\d+|\d+(?:\.\d*)?)(?:e[+-]?\d+)?)/i],["pln",/^(?:[_a-z][\w']*[!#?]?|``[^\t\n\r`]*(?:``|$))/i],["pun",/^[^\w\t\n\r "'\xa0]+/]]),["fs","ml"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-n.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-n.js new file mode 100644 index 00000000..6c2e85b9 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-n.js @@ -0,0 +1,4 @@ +var a=null; +PR.registerLangHandler(PR.createSimpleLexer([["str",/^(?:'(?:[^\n\r'\\]|\\.)*'|"(?:[^\n\r"\\]|\\.)*(?:"|$))/,a,'"'],["com",/^#(?:(?:define|elif|else|endif|error|ifdef|include|ifndef|line|pragma|undef|warning)\b|[^\n\r]*)/,a,"#"],["pln",/^\s+/,a," \r\n\t\xa0"]],[["str",/^@"(?:[^"]|"")*(?:"|$)/,a],["str",/^<#[^#>]*(?:#>|$)/,a],["str",/^<(?:(?:(?:\.\.\/)*|\/?)(?:[\w-]+(?:\/[\w-]+)+)?[\w-]+\.h|[a-z]\w*)>/,a],["com",/^\/\/[^\n\r]*/,a],["com",/^\/\*[\S\s]*?(?:\*\/|$)/, +a],["kwd",/^(?:abstract|and|as|base|catch|class|def|delegate|enum|event|extern|false|finally|fun|implements|interface|internal|is|macro|match|matches|module|mutable|namespace|new|null|out|override|params|partial|private|protected|public|ref|sealed|static|struct|syntax|this|throw|true|try|type|typeof|using|variant|virtual|volatile|when|where|with|assert|assert2|async|break|checked|continue|do|else|ensures|for|foreach|if|late|lock|new|nolate|otherwise|regexp|repeat|requires|return|surroundwith|unchecked|unless|using|while|yield)\b/, +a],["typ",/^(?:array|bool|byte|char|decimal|double|float|int|list|long|object|sbyte|short|string|ulong|uint|ufloat|ulong|ushort|void)\b/,a],["lit",/^@[$_a-z][\w$@]*/i,a],["typ",/^@[A-Z]+[a-z][\w$@]*/,a],["pln",/^'?[$_a-z][\w$@]*/i,a],["lit",/^(?:0x[\da-f]+|(?:\d(?:_\d+)*\d*(?:\.\d*)?|\.\d\+)(?:e[+-]?\d+)?)[a-z]*/i,a,"0123456789"],["pun",/^.[^\s\w"-$'./@`]*/,a]]),["n","nemerle"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-proto.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-proto.js new file mode 100644 index 00000000..f006ad8c --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-proto.js @@ -0,0 +1 @@ +PR.registerLangHandler(PR.sourceDecorator({keywords:"bytes,default,double,enum,extend,extensions,false,group,import,max,message,option,optional,package,repeated,required,returns,rpc,service,syntax,to,true",types:/^(bool|(double|s?fixed|[su]?int)(32|64)|float|string)\b/,cStyleComments:!0}),["proto"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-scala.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-scala.js new file mode 100644 index 00000000..60d034de --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-scala.js @@ -0,0 +1,2 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t\n\r \xa0]+/,null,"\t\n\r \xa0"],["str",/^"(?:""(?:""?(?!")|[^"\\]|\\.)*"{0,3}|(?:[^\n\r"\\]|\\.)*"?)/,null,'"'],["lit",/^`(?:[^\n\r\\`]|\\.)*`?/,null,"`"],["pun",/^[!#%&(--:-@[-^{-~]+/,null,"!#%&()*+,-:;<=>?@[\\]^{|}~"]],[["str",/^'(?:[^\n\r'\\]|\\(?:'|[^\n\r']+))'/],["lit",/^'[$A-Z_a-z][\w$]*(?![\w$'])/],["kwd",/^(?:abstract|case|catch|class|def|do|else|extends|final|finally|for|forSome|if|implicit|import|lazy|match|new|object|override|package|private|protected|requires|return|sealed|super|throw|trait|try|type|val|var|while|with|yield)\b/], +["lit",/^(?:true|false|null|this)\b/],["lit",/^(?:0(?:[0-7]+|x[\da-f]+)l?|(?:0|[1-9]\d*)(?:(?:\.\d+)?(?:e[+-]?\d+)?f?|l?)|\\.\d+(?:e[+-]?\d+)?f?)/i],["typ",/^[$_]*[A-Z][\d$A-Z_]*[a-z][\w$]*/],["pln",/^[$A-Z_a-z][\w$]*/],["com",/^\/(?:\/.*|\*(?:\/|\**[^*/])*(?:\*+\/?)?)/],["pun",/^(?:\.+|\/)/]]),["scala"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-sql.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-sql.js new file mode 100644 index 00000000..da705b0b --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-sql.js @@ -0,0 +1,2 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t\n\r \xa0]+/,null,"\t\n\r \xa0"],["str",/^(?:"(?:[^"\\]|\\.)*"|'(?:[^'\\]|\\.)*')/,null,"\"'"]],[["com",/^(?:--[^\n\r]*|\/\*[\S\s]*?(?:\*\/|$))/],["kwd",/^(?:add|all|alter|and|any|as|asc|authorization|backup|begin|between|break|browse|bulk|by|cascade|case|check|checkpoint|close|clustered|coalesce|collate|column|commit|compute|constraint|contains|containstable|continue|convert|create|cross|current|current_date|current_time|current_timestamp|current_user|cursor|database|dbcc|deallocate|declare|default|delete|deny|desc|disk|distinct|distributed|double|drop|dummy|dump|else|end|errlvl|escape|except|exec|execute|exists|exit|fetch|file|fillfactor|for|foreign|freetext|freetexttable|from|full|function|goto|grant|group|having|holdlock|identity|identitycol|identity_insert|if|in|index|inner|insert|intersect|into|is|join|key|kill|left|like|lineno|load|match|merge|national|nocheck|nonclustered|not|null|nullif|of|off|offsets|on|open|opendatasource|openquery|openrowset|openxml|option|or|order|outer|over|percent|plan|precision|primary|print|proc|procedure|public|raiserror|read|readtext|reconfigure|references|replication|restore|restrict|return|revoke|right|rollback|rowcount|rowguidcol|rule|save|schema|select|session_user|set|setuser|shutdown|some|statistics|system_user|table|textsize|then|to|top|tran|transaction|trigger|truncate|tsequal|union|unique|update|updatetext|use|user|using|values|varying|view|waitfor|when|where|while|with|writetext)(?=[^\w-]|$)/i, +null],["lit",/^[+-]?(?:0x[\da-f]+|(?:\.\d+|\d+(?:\.\d*)?)(?:e[+-]?\d+)?)/i],["pln",/^[_a-z][\w-]*/i],["pun",/^[^\w\t\n\r "'\xa0][^\w\t\n\r "'+\xa0-]*/]]),["sql"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-tex.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-tex.js new file mode 100644 index 00000000..ce96fbbd --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-tex.js @@ -0,0 +1 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t\n\r \xa0]+/,null,"\t\n\r \xa0"],["com",/^%[^\n\r]*/,null,"%"]],[["kwd",/^\\[@-Za-z]+/],["kwd",/^\\./],["typ",/^[$&]/],["lit",/[+-]?(?:\.\d+|\d+(?:\.\d*)?)(cm|em|ex|in|pc|pt|bp|mm)/i],["pun",/^[()=[\]{}]+/]]),["latex","tex"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-vb.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-vb.js new file mode 100644 index 00000000..07506b03 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-vb.js @@ -0,0 +1,2 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t\n\r \xa0\u2028\u2029]+/,null,"\t\n\r \xa0

"],["str",/^(?:["\u201c\u201d](?:[^"\u201c\u201d]|["\u201c\u201d]{2})(?:["\u201c\u201d]c|$)|["\u201c\u201d](?:[^"\u201c\u201d]|["\u201c\u201d]{2})*(?:["\u201c\u201d]|$))/i,null,'"“”'],["com",/^['\u2018\u2019].*/,null,"'‘’"]],[["kwd",/^(?:addhandler|addressof|alias|and|andalso|ansi|as|assembly|auto|boolean|byref|byte|byval|call|case|catch|cbool|cbyte|cchar|cdate|cdbl|cdec|char|cint|class|clng|cobj|const|cshort|csng|cstr|ctype|date|decimal|declare|default|delegate|dim|directcast|do|double|each|else|elseif|end|endif|enum|erase|error|event|exit|finally|for|friend|function|get|gettype|gosub|goto|handles|if|implements|imports|in|inherits|integer|interface|is|let|lib|like|long|loop|me|mod|module|mustinherit|mustoverride|mybase|myclass|namespace|new|next|not|notinheritable|notoverridable|object|on|option|optional|or|orelse|overloads|overridable|overrides|paramarray|preserve|private|property|protected|public|raiseevent|readonly|redim|removehandler|resume|return|select|set|shadows|shared|short|single|static|step|stop|string|structure|sub|synclock|then|throw|to|try|typeof|unicode|until|variant|wend|when|while|with|withevents|writeonly|xor|endif|gosub|let|variant|wend)\b/i, +null],["com",/^rem.*/i],["lit",/^(?:true\b|false\b|nothing\b|\d+(?:e[+-]?\d+[dfr]?|[dfilrs])?|(?:&h[\da-f]+|&o[0-7]+)[ils]?|\d*\.\d+(?:e[+-]?\d+)?[dfr]?|#\s+(?:\d+[/-]\d+[/-]\d+(?:\s+\d+:\d+(?::\d+)?(\s*(?:am|pm))?)?|\d+:\d+(?::\d+)?(\s*(?:am|pm))?)\s+#)/i],["pln",/^(?:(?:[a-z]|_\w)\w*|\[(?:[a-z]|_\w)\w*])/i],["pun",/^[^\w\t\n\r "'[\]\xa0\u2018\u2019\u201c\u201d\u2028\u2029]+/],["pun",/^(?:\[|])/]]),["vb","vbs"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-vhdl.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-vhdl.js new file mode 100644 index 00000000..128b5b6c --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-vhdl.js @@ -0,0 +1,3 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\t\n\r \xa0]+/,null,"\t\n\r \xa0"]],[["str",/^(?:[box]?"(?:[^"]|"")*"|'.')/i],["com",/^--[^\n\r]*/],["kwd",/^(?:abs|access|after|alias|all|and|architecture|array|assert|attribute|begin|block|body|buffer|bus|case|component|configuration|constant|disconnect|downto|else|elsif|end|entity|exit|file|for|function|generate|generic|group|guarded|if|impure|in|inertial|inout|is|label|library|linkage|literal|loop|map|mod|nand|new|next|nor|not|null|of|on|open|or|others|out|package|port|postponed|procedure|process|pure|range|record|register|reject|rem|report|return|rol|ror|select|severity|shared|signal|sla|sll|sra|srl|subtype|then|to|transport|type|unaffected|units|until|use|variable|wait|when|while|with|xnor|xor)(?=[^\w-]|$)/i, +null],["typ",/^(?:bit|bit_vector|character|boolean|integer|real|time|string|severity_level|positive|natural|signed|unsigned|line|text|std_u?logic(?:_vector)?)(?=[^\w-]|$)/i,null],["typ",/^'(?:active|ascending|base|delayed|driving|driving_value|event|high|image|instance_name|last_active|last_event|last_value|left|leftof|length|low|path_name|pos|pred|quiet|range|reverse_range|right|rightof|simple_name|stable|succ|transaction|val|value)(?=[^\w-]|$)/i,null],["lit",/^\d+(?:_\d+)*(?:#[\w.\\]+#(?:[+-]?\d+(?:_\d+)*)?|(?:\.\d+(?:_\d+)*)?(?:e[+-]?\d+(?:_\d+)*)?)/i], +["pln",/^(?:[a-z]\w*|\\[^\\]*\\)/i],["pun",/^[^\w\t\n\r "'\xa0][^\w\t\n\r "'\xa0-]*/]]),["vhdl","vhd"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-wiki.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-wiki.js new file mode 100644 index 00000000..9b0b4487 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-wiki.js @@ -0,0 +1,2 @@ +PR.registerLangHandler(PR.createSimpleLexer([["pln",/^[\d\t a-gi-z\xa0]+/,null,"\t \xa0abcdefgijklmnopqrstuvwxyz0123456789"],["pun",/^[*=[\]^~]+/,null,"=*~^[]"]],[["lang-wiki.meta",/(?:^^|\r\n?|\n)(#[a-z]+)\b/],["lit",/^[A-Z][a-z][\da-z]+[A-Z][a-z][^\W_]+\b/],["lang-",/^{{{([\S\s]+?)}}}/],["lang-",/^`([^\n\r`]+)`/],["str",/^https?:\/\/[^\s#/?]*(?:\/[^\s#?]*)?(?:\?[^\s#]*)?(?:#\S*)?/i],["pln",/^(?:\r\n|[\S\s])[^\n\r#*=A-[^`h{~]*/]]),["wiki"]); +PR.registerLangHandler(PR.createSimpleLexer([["kwd",/^#[a-z]+/i,null,"#"]],[]),["wiki.meta"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-xq.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-xq.js new file mode 100644 index 00000000..e323ae32 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-xq.js @@ -0,0 +1,3 @@ +PR.registerLangHandler(PR.createSimpleLexer([["var pln",/^\$[\w-]+/,null,"$"]],[["pln",/^[\s=][<>][\s=]/],["lit",/^@[\w-]+/],["tag",/^<\/?[a-z](?:[\w-.:]*\w)?|\/?>$/i],["com",/^\(:[\S\s]*?:\)/],["pln",/^[(),/;[\]{}]$/],["str",/^(?:"(?:[^"\\{]|\\[\S\s])*(?:"|$)|'(?:[^'\\{]|\\[\S\s])*(?:'|$))/,null,"\"'"],["kwd",/^(?:xquery|where|version|variable|union|typeswitch|treat|to|then|text|stable|sortby|some|self|schema|satisfies|returns|return|ref|processing-instruction|preceding-sibling|preceding|precedes|parent|only|of|node|namespace|module|let|item|intersect|instance|in|import|if|function|for|follows|following-sibling|following|external|except|every|else|element|descending|descendant-or-self|descendant|define|default|declare|comment|child|cast|case|before|attribute|assert|ascending|as|ancestor-or-self|ancestor|after|eq|order|by|or|and|schema-element|document-node|node|at)\b/], +["typ",/^(?:xs:yearMonthDuration|xs:unsignedLong|xs:time|xs:string|xs:short|xs:QName|xs:Name|xs:long|xs:integer|xs:int|xs:gYearMonth|xs:gYear|xs:gMonthDay|xs:gDay|xs:float|xs:duration|xs:double|xs:decimal|xs:dayTimeDuration|xs:dateTime|xs:date|xs:byte|xs:boolean|xs:anyURI|xf:yearMonthDuration)\b/,null],["fun pln",/^(?:xp:dereference|xinc:node-expand|xinc:link-references|xinc:link-expand|xhtml:restructure|xhtml:clean|xhtml:add-lists|xdmp:zip-manifest|xdmp:zip-get|xdmp:zip-create|xdmp:xquery-version|xdmp:word-convert|xdmp:with-namespaces|xdmp:version|xdmp:value|xdmp:user-roles|xdmp:user-last-login|xdmp:user|xdmp:url-encode|xdmp:url-decode|xdmp:uri-is-file|xdmp:uri-format|xdmp:uri-content-type|xdmp:unquote|xdmp:unpath|xdmp:triggers-database|xdmp:trace|xdmp:to-json|xdmp:tidy|xdmp:subbinary|xdmp:strftime|xdmp:spawn-in|xdmp:spawn|xdmp:sleep|xdmp:shutdown|xdmp:set-session-field|xdmp:set-response-encoding|xdmp:set-response-content-type|xdmp:set-response-code|xdmp:set-request-time-limit|xdmp:set|xdmp:servers|xdmp:server-status|xdmp:server-name|xdmp:server|xdmp:security-database|xdmp:security-assert|xdmp:schema-database|xdmp:save|xdmp:role-roles|xdmp:role|xdmp:rethrow|xdmp:restart|xdmp:request-timestamp|xdmp:request-status|xdmp:request-cancel|xdmp:request|xdmp:redirect-response|xdmp:random|xdmp:quote|xdmp:query-trace|xdmp:query-meters|xdmp:product-edition|xdmp:privilege-roles|xdmp:privilege|xdmp:pretty-print|xdmp:powerpoint-convert|xdmp:platform|xdmp:permission|xdmp:pdf-convert|xdmp:path|xdmp:octal-to-integer|xdmp:node-uri|xdmp:node-replace|xdmp:node-kind|xdmp:node-insert-child|xdmp:node-insert-before|xdmp:node-insert-after|xdmp:node-delete|xdmp:node-database|xdmp:mul64|xdmp:modules-root|xdmp:modules-database|xdmp:merging|xdmp:merge-cancel|xdmp:merge|xdmp:md5|xdmp:logout|xdmp:login|xdmp:log-level|xdmp:log|xdmp:lock-release|xdmp:lock-acquire|xdmp:load|xdmp:invoke-in|xdmp:invoke|xdmp:integer-to-octal|xdmp:integer-to-hex|xdmp:http-put|xdmp:http-post|xdmp:http-options|xdmp:http-head|xdmp:http-get|xdmp:http-delete|xdmp:hosts|xdmp:host-status|xdmp:host-name|xdmp:host|xdmp:hex-to-integer|xdmp:hash64|xdmp:hash32|xdmp:has-privilege|xdmp:groups|xdmp:group-serves|xdmp:group-servers|xdmp:group-name|xdmp:group-hosts|xdmp:group|xdmp:get-session-field-names|xdmp:get-session-field|xdmp:get-response-encoding|xdmp:get-response-code|xdmp:get-request-username|xdmp:get-request-user|xdmp:get-request-url|xdmp:get-request-protocol|xdmp:get-request-path|xdmp:get-request-method|xdmp:get-request-header-names|xdmp:get-request-header|xdmp:get-request-field-names|xdmp:get-request-field-filename|xdmp:get-request-field-content-type|xdmp:get-request-field|xdmp:get-request-client-certificate|xdmp:get-request-client-address|xdmp:get-request-body|xdmp:get-current-user|xdmp:get-current-roles|xdmp:get|xdmp:function-name|xdmp:function-module|xdmp:function|xdmp:from-json|xdmp:forests|xdmp:forest-status|xdmp:forest-restore|xdmp:forest-restart|xdmp:forest-name|xdmp:forest-delete|xdmp:forest-databases|xdmp:forest-counts|xdmp:forest-clear|xdmp:forest-backup|xdmp:forest|xdmp:filesystem-file|xdmp:filesystem-directory|xdmp:exists|xdmp:excel-convert|xdmp:eval-in|xdmp:eval|xdmp:estimate|xdmp:email|xdmp:element-content-type|xdmp:elapsed-time|xdmp:document-set-quality|xdmp:document-set-property|xdmp:document-set-properties|xdmp:document-set-permissions|xdmp:document-set-collections|xdmp:document-remove-properties|xdmp:document-remove-permissions|xdmp:document-remove-collections|xdmp:document-properties|xdmp:document-locks|xdmp:document-load|xdmp:document-insert|xdmp:document-get-quality|xdmp:document-get-properties|xdmp:document-get-permissions|xdmp:document-get-collections|xdmp:document-get|xdmp:document-forest|xdmp:document-delete|xdmp:document-add-properties|xdmp:document-add-permissions|xdmp:document-add-collections|xdmp:directory-properties|xdmp:directory-locks|xdmp:directory-delete|xdmp:directory-create|xdmp:directory|xdmp:diacritic-less|xdmp:describe|xdmp:default-permissions|xdmp:default-collections|xdmp:databases|xdmp:database-restore-validate|xdmp:database-restore-status|xdmp:database-restore-cancel|xdmp:database-restore|xdmp:database-name|xdmp:database-forests|xdmp:database-backup-validate|xdmp:database-backup-status|xdmp:database-backup-purge|xdmp:database-backup-cancel|xdmp:database-backup|xdmp:database|xdmp:collection-properties|xdmp:collection-locks|xdmp:collection-delete|xdmp:collation-canonical-uri|xdmp:castable-as|xdmp:can-grant-roles|xdmp:base64-encode|xdmp:base64-decode|xdmp:architecture|xdmp:apply|xdmp:amp-roles|xdmp:amp|xdmp:add64|xdmp:add-response-header|xdmp:access|trgr:trigger-set-recursive|trgr:trigger-set-permissions|trgr:trigger-set-name|trgr:trigger-set-module|trgr:trigger-set-event|trgr:trigger-set-description|trgr:trigger-remove-permissions|trgr:trigger-module|trgr:trigger-get-permissions|trgr:trigger-enable|trgr:trigger-disable|trgr:trigger-database-online-event|trgr:trigger-data-event|trgr:trigger-add-permissions|trgr:remove-trigger|trgr:property-content|trgr:pre-commit|trgr:post-commit|trgr:get-trigger-by-id|trgr:get-trigger|trgr:document-scope|trgr:document-content|trgr:directory-scope|trgr:create-trigger|trgr:collection-scope|trgr:any-property-content|thsr:set-entry|thsr:remove-term|thsr:remove-synonym|thsr:remove-entry|thsr:query-lookup|thsr:lookup|thsr:load|thsr:insert|thsr:expand|thsr:add-synonym|spell:suggest-detailed|spell:suggest|spell:remove-word|spell:make-dictionary|spell:load|spell:levenshtein-distance|spell:is-correct|spell:insert|spell:double-metaphone|spell:add-word|sec:users-collection|sec:user-set-roles|sec:user-set-password|sec:user-set-name|sec:user-set-description|sec:user-set-default-permissions|sec:user-set-default-collections|sec:user-remove-roles|sec:user-privileges|sec:user-get-roles|sec:user-get-description|sec:user-get-default-permissions|sec:user-get-default-collections|sec:user-doc-permissions|sec:user-doc-collections|sec:user-add-roles|sec:unprotect-collection|sec:uid-for-name|sec:set-realm|sec:security-version|sec:security-namespace|sec:security-installed|sec:security-collection|sec:roles-collection|sec:role-set-roles|sec:role-set-name|sec:role-set-description|sec:role-set-default-permissions|sec:role-set-default-collections|sec:role-remove-roles|sec:role-privileges|sec:role-get-roles|sec:role-get-description|sec:role-get-default-permissions|sec:role-get-default-collections|sec:role-doc-permissions|sec:role-doc-collections|sec:role-add-roles|sec:remove-user|sec:remove-role-from-users|sec:remove-role-from-role|sec:remove-role-from-privileges|sec:remove-role-from-amps|sec:remove-role|sec:remove-privilege|sec:remove-amp|sec:protect-collection|sec:privileges-collection|sec:privilege-set-roles|sec:privilege-set-name|sec:privilege-remove-roles|sec:privilege-get-roles|sec:privilege-add-roles|sec:priv-doc-permissions|sec:priv-doc-collections|sec:get-user-names|sec:get-unique-elem-id|sec:get-role-names|sec:get-role-ids|sec:get-privilege|sec:get-distinct-permissions|sec:get-collection|sec:get-amp|sec:create-user-with-role|sec:create-user|sec:create-role|sec:create-privilege|sec:create-amp|sec:collections-collection|sec:collection-set-permissions|sec:collection-remove-permissions|sec:collection-get-permissions|sec:collection-add-permissions|sec:check-admin|sec:amps-collection|sec:amp-set-roles|sec:amp-remove-roles|sec:amp-get-roles|sec:amp-doc-permissions|sec:amp-doc-collections|sec:amp-add-roles|search:unparse|search:suggest|search:snippet|search:search|search:resolve-nodes|search:resolve|search:remove-constraint|search:parse|search:get-default-options|search:estimate|search:check-options|prof:value|prof:reset|prof:report|prof:invoke|prof:eval|prof:enable|prof:disable|prof:allowed|ppt:clean|pki:template-set-request|pki:template-set-name|pki:template-set-key-type|pki:template-set-key-options|pki:template-set-description|pki:template-in-use|pki:template-get-version|pki:template-get-request|pki:template-get-name|pki:template-get-key-type|pki:template-get-key-options|pki:template-get-id|pki:template-get-description|pki:need-certificate|pki:is-temporary|pki:insert-trusted-certificates|pki:insert-template|pki:insert-signed-certificates|pki:insert-certificate-revocation-list|pki:get-trusted-certificate-ids|pki:get-template-ids|pki:get-template-certificate-authority|pki:get-template-by-name|pki:get-template|pki:get-pending-certificate-requests-xml|pki:get-pending-certificate-requests-pem|pki:get-pending-certificate-request|pki:get-certificates-for-template-xml|pki:get-certificates-for-template|pki:get-certificates|pki:get-certificate-xml|pki:get-certificate-pem|pki:get-certificate|pki:generate-temporary-certificate-if-necessary|pki:generate-temporary-certificate|pki:generate-template-certificate-authority|pki:generate-certificate-request|pki:delete-template|pki:delete-certificate|pki:create-template|pdf:make-toc|pdf:insert-toc-headers|pdf:get-toc|pdf:clean|p:status-transition|p:state-transition|p:remove|p:pipelines|p:insert|p:get-by-id|p:get|p:execute|p:create|p:condition|p:collection|p:action|ooxml:runs-merge|ooxml:package-uris|ooxml:package-parts-insert|ooxml:package-parts|msword:clean|mcgm:polygon|mcgm:point|mcgm:geospatial-query-from-elements|mcgm:geospatial-query|mcgm:circle|math:tanh|math:tan|math:sqrt|math:sinh|math:sin|math:pow|math:modf|math:log10|math:log|math:ldexp|math:frexp|math:fmod|math:floor|math:fabs|math:exp|math:cosh|math:cos|math:ceil|math:atan2|math:atan|math:asin|math:acos|map:put|map:map|map:keys|map:get|map:delete|map:count|map:clear|lnk:to|lnk:remove|lnk:insert|lnk:get|lnk:from|lnk:create|kml:polygon|kml:point|kml:interior-polygon|kml:geospatial-query-from-elements|kml:geospatial-query|kml:circle|kml:box|gml:polygon|gml:point|gml:interior-polygon|gml:geospatial-query-from-elements|gml:geospatial-query|gml:circle|gml:box|georss:point|georss:geospatial-query|georss:circle|geo:polygon|geo:point|geo:interior-polygon|geo:geospatial-query-from-elements|geo:geospatial-query|geo:circle|geo:box|fn:zero-or-one|fn:years-from-duration|fn:year-from-dateTime|fn:year-from-date|fn:upper-case|fn:unordered|fn:true|fn:translate|fn:trace|fn:tokenize|fn:timezone-from-time|fn:timezone-from-dateTime|fn:timezone-from-date|fn:sum|fn:subtract-dateTimes-yielding-yearMonthDuration|fn:subtract-dateTimes-yielding-dayTimeDuration|fn:substring-before|fn:substring-after|fn:substring|fn:subsequence|fn:string-to-codepoints|fn:string-pad|fn:string-length|fn:string-join|fn:string|fn:static-base-uri|fn:starts-with|fn:seconds-from-time|fn:seconds-from-duration|fn:seconds-from-dateTime|fn:round-half-to-even|fn:round|fn:root|fn:reverse|fn:resolve-uri|fn:resolve-QName|fn:replace|fn:remove|fn:QName|fn:prefix-from-QName|fn:position|fn:one-or-more|fn:number|fn:not|fn:normalize-unicode|fn:normalize-space|fn:node-name|fn:node-kind|fn:nilled|fn:namespace-uri-from-QName|fn:namespace-uri-for-prefix|fn:namespace-uri|fn:name|fn:months-from-duration|fn:month-from-dateTime|fn:month-from-date|fn:minutes-from-time|fn:minutes-from-duration|fn:minutes-from-dateTime|fn:min|fn:max|fn:matches|fn:lower-case|fn:local-name-from-QName|fn:local-name|fn:last|fn:lang|fn:iri-to-uri|fn:insert-before|fn:index-of|fn:in-scope-prefixes|fn:implicit-timezone|fn:idref|fn:id|fn:hours-from-time|fn:hours-from-duration|fn:hours-from-dateTime|fn:floor|fn:false|fn:expanded-QName|fn:exists|fn:exactly-one|fn:escape-uri|fn:escape-html-uri|fn:error|fn:ends-with|fn:encode-for-uri|fn:empty|fn:document-uri|fn:doc-available|fn:doc|fn:distinct-values|fn:distinct-nodes|fn:default-collation|fn:deep-equal|fn:days-from-duration|fn:day-from-dateTime|fn:day-from-date|fn:data|fn:current-time|fn:current-dateTime|fn:current-date|fn:count|fn:contains|fn:concat|fn:compare|fn:collection|fn:codepoints-to-string|fn:codepoint-equal|fn:ceiling|fn:boolean|fn:base-uri|fn:avg|fn:adjust-time-to-timezone|fn:adjust-dateTime-to-timezone|fn:adjust-date-to-timezone|fn:abs|feed:unsubscribe|feed:subscription|feed:subscribe|feed:request|feed:item|feed:description|excel:clean|entity:enrich|dom:set-pipelines|dom:set-permissions|dom:set-name|dom:set-evaluation-context|dom:set-domain-scope|dom:set-description|dom:remove-pipeline|dom:remove-permissions|dom:remove|dom:get|dom:evaluation-context|dom:domains|dom:domain-scope|dom:create|dom:configuration-set-restart-user|dom:configuration-set-permissions|dom:configuration-set-evaluation-context|dom:configuration-set-default-domain|dom:configuration-get|dom:configuration-create|dom:collection|dom:add-pipeline|dom:add-permissions|dls:retention-rules|dls:retention-rule-remove|dls:retention-rule-insert|dls:retention-rule|dls:purge|dls:node-expand|dls:link-references|dls:link-expand|dls:documents-query|dls:document-versions-query|dls:document-version-uri|dls:document-version-query|dls:document-version-delete|dls:document-version-as-of|dls:document-version|dls:document-update|dls:document-unmanage|dls:document-set-quality|dls:document-set-property|dls:document-set-properties|dls:document-set-permissions|dls:document-set-collections|dls:document-retention-rules|dls:document-remove-properties|dls:document-remove-permissions|dls:document-remove-collections|dls:document-purge|dls:document-manage|dls:document-is-managed|dls:document-insert-and-manage|dls:document-include-query|dls:document-history|dls:document-get-permissions|dls:document-extract-part|dls:document-delete|dls:document-checkout-status|dls:document-checkout|dls:document-checkin|dls:document-add-properties|dls:document-add-permissions|dls:document-add-collections|dls:break-checkout|dls:author-query|dls:as-of-query|dbk:convert|dbg:wait|dbg:value|dbg:stopped|dbg:stop|dbg:step|dbg:status|dbg:stack|dbg:out|dbg:next|dbg:line|dbg:invoke|dbg:function|dbg:finish|dbg:expr|dbg:eval|dbg:disconnect|dbg:detach|dbg:continue|dbg:connect|dbg:clear|dbg:breakpoints|dbg:break|dbg:attached|dbg:attach|cvt:save-converted-documents|cvt:part-uri|cvt:destination-uri|cvt:basepath|cvt:basename|cts:words|cts:word-query-weight|cts:word-query-text|cts:word-query-options|cts:word-query|cts:word-match|cts:walk|cts:uris|cts:uri-match|cts:train|cts:tokenize|cts:thresholds|cts:stem|cts:similar-query-weight|cts:similar-query-nodes|cts:similar-query|cts:shortest-distance|cts:search|cts:score|cts:reverse-query-weight|cts:reverse-query-nodes|cts:reverse-query|cts:remainder|cts:registered-query-weight|cts:registered-query-options|cts:registered-query-ids|cts:registered-query|cts:register|cts:query|cts:quality|cts:properties-query-query|cts:properties-query|cts:polygon-vertices|cts:polygon|cts:point-longitude|cts:point-latitude|cts:point|cts:or-query-queries|cts:or-query|cts:not-query-weight|cts:not-query-query|cts:not-query|cts:near-query-weight|cts:near-query-queries|cts:near-query-options|cts:near-query-distance|cts:near-query|cts:highlight|cts:geospatial-co-occurrences|cts:frequency|cts:fitness|cts:field-words|cts:field-word-query-weight|cts:field-word-query-text|cts:field-word-query-options|cts:field-word-query-field-name|cts:field-word-query|cts:field-word-match|cts:entity-highlight|cts:element-words|cts:element-word-query-weight|cts:element-word-query-text|cts:element-word-query-options|cts:element-word-query-element-name|cts:element-word-query|cts:element-word-match|cts:element-values|cts:element-value-ranges|cts:element-value-query-weight|cts:element-value-query-text|cts:element-value-query-options|cts:element-value-query-element-name|cts:element-value-query|cts:element-value-match|cts:element-value-geospatial-co-occurrences|cts:element-value-co-occurrences|cts:element-range-query-weight|cts:element-range-query-value|cts:element-range-query-options|cts:element-range-query-operator|cts:element-range-query-element-name|cts:element-range-query|cts:element-query-query|cts:element-query-element-name|cts:element-query|cts:element-pair-geospatial-values|cts:element-pair-geospatial-value-match|cts:element-pair-geospatial-query-weight|cts:element-pair-geospatial-query-region|cts:element-pair-geospatial-query-options|cts:element-pair-geospatial-query-longitude-name|cts:element-pair-geospatial-query-latitude-name|cts:element-pair-geospatial-query-element-name|cts:element-pair-geospatial-query|cts:element-pair-geospatial-boxes|cts:element-geospatial-values|cts:element-geospatial-value-match|cts:element-geospatial-query-weight|cts:element-geospatial-query-region|cts:element-geospatial-query-options|cts:element-geospatial-query-element-name|cts:element-geospatial-query|cts:element-geospatial-boxes|cts:element-child-geospatial-values|cts:element-child-geospatial-value-match|cts:element-child-geospatial-query-weight|cts:element-child-geospatial-query-region|cts:element-child-geospatial-query-options|cts:element-child-geospatial-query-element-name|cts:element-child-geospatial-query-child-name|cts:element-child-geospatial-query|cts:element-child-geospatial-boxes|cts:element-attribute-words|cts:element-attribute-word-query-weight|cts:element-attribute-word-query-text|cts:element-attribute-word-query-options|cts:element-attribute-word-query-element-name|cts:element-attribute-word-query-attribute-name|cts:element-attribute-word-query|cts:element-attribute-word-match|cts:element-attribute-values|cts:element-attribute-value-ranges|cts:element-attribute-value-query-weight|cts:element-attribute-value-query-text|cts:element-attribute-value-query-options|cts:element-attribute-value-query-element-name|cts:element-attribute-value-query-attribute-name|cts:element-attribute-value-query|cts:element-attribute-value-match|cts:element-attribute-value-geospatial-co-occurrences|cts:element-attribute-value-co-occurrences|cts:element-attribute-range-query-weight|cts:element-attribute-range-query-value|cts:element-attribute-range-query-options|cts:element-attribute-range-query-operator|cts:element-attribute-range-query-element-name|cts:element-attribute-range-query-attribute-name|cts:element-attribute-range-query|cts:element-attribute-pair-geospatial-values|cts:element-attribute-pair-geospatial-value-match|cts:element-attribute-pair-geospatial-query-weight|cts:element-attribute-pair-geospatial-query-region|cts:element-attribute-pair-geospatial-query-options|cts:element-attribute-pair-geospatial-query-longitude-name|cts:element-attribute-pair-geospatial-query-latitude-name|cts:element-attribute-pair-geospatial-query-element-name|cts:element-attribute-pair-geospatial-query|cts:element-attribute-pair-geospatial-boxes|cts:document-query-uris|cts:document-query|cts:distance|cts:directory-query-uris|cts:directory-query-depth|cts:directory-query|cts:destination|cts:deregister|cts:contains|cts:confidence|cts:collections|cts:collection-query-uris|cts:collection-query|cts:collection-match|cts:classify|cts:circle-radius|cts:circle-center|cts:circle|cts:box-west|cts:box-south|cts:box-north|cts:box-east|cts:box|cts:bearing|cts:arc-intersection|cts:and-query-queries|cts:and-query-options|cts:and-query|cts:and-not-query-positive-query|cts:and-not-query-negative-query|cts:and-not-query|css:get|css:convert|cpf:success|cpf:failure|cpf:document-set-state|cpf:document-set-processing-status|cpf:document-set-last-updated|cpf:document-set-error|cpf:document-get-state|cpf:document-get-processing-status|cpf:document-get-last-updated|cpf:document-get-error|cpf:check-transition|alert:spawn-matching-actions|alert:rule-user-id-query|alert:rule-set-user-id|alert:rule-set-query|alert:rule-set-options|alert:rule-set-name|alert:rule-set-description|alert:rule-set-action|alert:rule-remove|alert:rule-name-query|alert:rule-insert|alert:rule-id-query|alert:rule-get-user-id|alert:rule-get-query|alert:rule-get-options|alert:rule-get-name|alert:rule-get-id|alert:rule-get-description|alert:rule-get-action|alert:rule-action-query|alert:remove-triggers|alert:make-rule|alert:make-log-action|alert:make-config|alert:make-action|alert:invoke-matching-actions|alert:get-my-rules|alert:get-all-rules|alert:get-actions|alert:find-matching-rules|alert:create-triggers|alert:config-set-uri|alert:config-set-trigger-ids|alert:config-set-options|alert:config-set-name|alert:config-set-description|alert:config-set-cpf-domain-names|alert:config-set-cpf-domain-ids|alert:config-insert|alert:config-get-uri|alert:config-get-trigger-ids|alert:config-get-options|alert:config-get-name|alert:config-get-id|alert:config-get-description|alert:config-get-cpf-domain-names|alert:config-get-cpf-domain-ids|alert:config-get|alert:config-delete|alert:action-set-options|alert:action-set-name|alert:action-set-module-root|alert:action-set-module-db|alert:action-set-module|alert:action-set-description|alert:action-remove|alert:action-insert|alert:action-get-options|alert:action-get-name|alert:action-get-module-root|alert:action-get-module-db|alert:action-get-module|alert:action-get-description|zero-or-one|years-from-duration|year-from-dateTime|year-from-date|upper-case|unordered|true|translate|trace|tokenize|timezone-from-time|timezone-from-dateTime|timezone-from-date|sum|subtract-dateTimes-yielding-yearMonthDuration|subtract-dateTimes-yielding-dayTimeDuration|substring-before|substring-after|substring|subsequence|string-to-codepoints|string-pad|string-length|string-join|string|static-base-uri|starts-with|seconds-from-time|seconds-from-duration|seconds-from-dateTime|round-half-to-even|round|root|reverse|resolve-uri|resolve-QName|replace|remove|QName|prefix-from-QName|position|one-or-more|number|not|normalize-unicode|normalize-space|node-name|node-kind|nilled|namespace-uri-from-QName|namespace-uri-for-prefix|namespace-uri|name|months-from-duration|month-from-dateTime|month-from-date|minutes-from-time|minutes-from-duration|minutes-from-dateTime|min|max|matches|lower-case|local-name-from-QName|local-name|last|lang|iri-to-uri|insert-before|index-of|in-scope-prefixes|implicit-timezone|idref|id|hours-from-time|hours-from-duration|hours-from-dateTime|floor|false|expanded-QName|exists|exactly-one|escape-uri|escape-html-uri|error|ends-with|encode-for-uri|empty|document-uri|doc-available|doc|distinct-values|distinct-nodes|default-collation|deep-equal|days-from-duration|day-from-dateTime|day-from-date|data|current-time|current-dateTime|current-date|count|contains|concat|compare|collection|codepoints-to-string|codepoint-equal|ceiling|boolean|base-uri|avg|adjust-time-to-timezone|adjust-dateTime-to-timezone|adjust-date-to-timezone|abs)\b/], +["pln",/^[\w:-]+/],["pln",/^[\t\n\r \xa0]+/]]),["xq","xquery"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/lang-yaml.js b/source/_themes/uwpce_slides2/static/js/prettify/lang-yaml.js new file mode 100644 index 00000000..c38729b6 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/lang-yaml.js @@ -0,0 +1,2 @@ +var a=null; +PR.registerLangHandler(PR.createSimpleLexer([["pun",/^[:>?|]+/,a,":|>?"],["dec",/^%(?:YAML|TAG)[^\n\r#]+/,a,"%"],["typ",/^&\S+/,a,"&"],["typ",/^!\S*/,a,"!"],["str",/^"(?:[^"\\]|\\.)*(?:"|$)/,a,'"'],["str",/^'(?:[^']|'')*(?:'|$)/,a,"'"],["com",/^#[^\n\r]*/,a,"#"],["pln",/^\s+/,a," \t\r\n"]],[["dec",/^(?:---|\.\.\.)(?:[\n\r]|$)/],["pun",/^-/],["kwd",/^\w+:[\n\r ]/],["pln",/^\w+/]]),["yaml","yml"]); diff --git a/source/_themes/uwpce_slides2/static/js/prettify/prettify.css b/source/_themes/uwpce_slides2/static/js/prettify/prettify.css new file mode 100644 index 00000000..d44b3a22 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/prettify.css @@ -0,0 +1 @@ +.pln{color:#000}@media screen{.str{color:#080}.kwd{color:#008}.com{color:#800}.typ{color:#606}.lit{color:#066}.pun,.opn,.clo{color:#660}.tag{color:#008}.atn{color:#606}.atv{color:#080}.dec,.var{color:#606}.fun{color:red}}@media print,projection{.str{color:#060}.kwd{color:#006;font-weight:bold}.com{color:#600;font-style:italic}.typ{color:#404;font-weight:bold}.lit{color:#044}.pun,.opn,.clo{color:#440}.tag{color:#006;font-weight:bold}.atn{color:#404}.atv{color:#060}}pre.prettyprint{padding:2px;border:1px solid #888}ol.linenums{margin-top:0;margin-bottom:0}li.L0,li.L1,li.L2,li.L3,li.L5,li.L6,li.L7,li.L8{list-style-type:none}li.L1,li.L3,li.L5,li.L7,li.L9{background:#eee} \ No newline at end of file diff --git a/source/_themes/uwpce_slides2/static/js/prettify/prettify.js b/source/_themes/uwpce_slides2/static/js/prettify/prettify.js new file mode 100644 index 00000000..eef5ad7e --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/prettify/prettify.js @@ -0,0 +1,28 @@ +var q=null;window.PR_SHOULD_USE_CONTINUATION=!0; +(function(){function L(a){function m(a){var f=a.charCodeAt(0);if(f!==92)return f;var b=a.charAt(1);return(f=r[b])?f:"0"<=b&&b<="7"?parseInt(a.substring(1),8):b==="u"||b==="x"?parseInt(a.substring(2),16):a.charCodeAt(1)}function e(a){if(a<32)return(a<16?"\\x0":"\\x")+a.toString(16);a=String.fromCharCode(a);if(a==="\\"||a==="-"||a==="["||a==="]")a="\\"+a;return a}function h(a){for(var f=a.substring(1,a.length-1).match(/\\u[\dA-Fa-f]{4}|\\x[\dA-Fa-f]{2}|\\[0-3][0-7]{0,2}|\\[0-7]{1,2}|\\[\S\s]|[^\\]/g),a= +[],b=[],o=f[0]==="^",c=o?1:0,i=f.length;c122||(d<65||j>90||b.push([Math.max(65,j)|32,Math.min(d,90)|32]),d<97||j>122||b.push([Math.max(97,j)&-33,Math.min(d,122)&-33]))}}b.sort(function(a,f){return a[0]-f[0]||f[1]-a[1]});f=[];j=[NaN,NaN];for(c=0;ci[0]&&(i[1]+1>i[0]&&b.push("-"),b.push(e(i[1])));b.push("]");return b.join("")}function y(a){for(var f=a.source.match(/\[(?:[^\\\]]|\\[\S\s])*]|\\u[\dA-Fa-f]{4}|\\x[\dA-Fa-f]{2}|\\\d+|\\[^\dux]|\(\?[!:=]|[()^]|[^()[\\^]+/g),b=f.length,d=[],c=0,i=0;c=2&&a==="["?f[c]=h(j):a!=="\\"&&(f[c]=j.replace(/[A-Za-z]/g,function(a){a=a.charCodeAt(0);return"["+String.fromCharCode(a&-33,a|32)+"]"}));return f.join("")}for(var t=0,s=!1,l=!1,p=0,d=a.length;p=5&&"lang-"===b.substring(0,5))&&!(o&&typeof o[1]==="string"))c=!1,b="src";c||(r[f]=b)}i=d;d+=f.length;if(c){c=o[1];var j=f.indexOf(c),k=j+c.length;o[2]&&(k=f.length-o[2].length,j=k-c.length);b=b.substring(5);B(l+i,f.substring(0,j),e,p);B(l+i+j,c,C(b,c),p);B(l+i+k,f.substring(k),e,p)}else p.push(l+i,b)}a.e=p}var h={},y;(function(){for(var e=a.concat(m), +l=[],p={},d=0,g=e.length;d=0;)h[n.charAt(k)]=r;r=r[1];n=""+r;p.hasOwnProperty(n)||(l.push(r),p[n]=q)}l.push(/[\S\s]/);y=L(l)})();var t=m.length;return e}function u(a){var m=[],e=[];a.tripleQuotedStrings?m.push(["str",/^(?:'''(?:[^'\\]|\\[\S\s]|''?(?=[^']))*(?:'''|$)|"""(?:[^"\\]|\\[\S\s]|""?(?=[^"]))*(?:"""|$)|'(?:[^'\\]|\\[\S\s])*(?:'|$)|"(?:[^"\\]|\\[\S\s])*(?:"|$))/,q,"'\""]):a.multiLineStrings?m.push(["str",/^(?:'(?:[^'\\]|\\[\S\s])*(?:'|$)|"(?:[^"\\]|\\[\S\s])*(?:"|$)|`(?:[^\\`]|\\[\S\s])*(?:`|$))/, +q,"'\"`"]):m.push(["str",/^(?:'(?:[^\n\r'\\]|\\.)*(?:'|$)|"(?:[^\n\r"\\]|\\.)*(?:"|$))/,q,"\"'"]);a.verbatimStrings&&e.push(["str",/^@"(?:[^"]|"")*(?:"|$)/,q]);var h=a.hashComments;h&&(a.cStyleComments?(h>1?m.push(["com",/^#(?:##(?:[^#]|#(?!##))*(?:###|$)|.*)/,q,"#"]):m.push(["com",/^#(?:(?:define|elif|else|endif|error|ifdef|include|ifndef|line|pragma|undef|warning)\b|[^\n\r]*)/,q,"#"]),e.push(["str",/^<(?:(?:(?:\.\.\/)*|\/?)(?:[\w-]+(?:\/[\w-]+)+)?[\w-]+\.h|[a-z]\w*)>/,q])):m.push(["com",/^#[^\n\r]*/, +q,"#"]));a.cStyleComments&&(e.push(["com",/^\/\/[^\n\r]*/,q]),e.push(["com",/^\/\*[\S\s]*?(?:\*\/|$)/,q]));a.regexLiterals&&e.push(["lang-regex",/^(?:^^\.?|[!+-]|!=|!==|#|%|%=|&|&&|&&=|&=|\(|\*|\*=|\+=|,|-=|->|\/|\/=|:|::|;|<|<<|<<=|<=|=|==|===|>|>=|>>|>>=|>>>|>>>=|[?@[^]|\^=|\^\^|\^\^=|{|\||\|=|\|\||\|\|=|~|break|case|continue|delete|do|else|finally|instanceof|return|throw|try|typeof)\s*(\/(?=[^*/])(?:[^/[\\]|\\[\S\s]|\[(?:[^\\\]]|\\[\S\s])*(?:]|$))+\/)/]);(h=a.types)&&e.push(["typ",h]);a=(""+a.keywords).replace(/^ | $/g, +"");a.length&&e.push(["kwd",RegExp("^(?:"+a.replace(/[\s,]+/g,"|")+")\\b"),q]);m.push(["pln",/^\s+/,q," \r\n\t\xa0"]);e.push(["lit",/^@[$_a-z][\w$@]*/i,q],["typ",/^(?:[@_]?[A-Z]+[a-z][\w$@]*|\w+_t\b)/,q],["pln",/^[$_a-z][\w$@]*/i,q],["lit",/^(?:0x[\da-f]+|(?:\d(?:_\d+)*\d*(?:\.\d*)?|\.\d\+)(?:e[+-]?\d+)?)[a-z]*/i,q,"0123456789"],["pln",/^\\[\S\s]?/,q],["pun",/^.[^\s\w"-$'./@\\`]*/,q]);return x(m,e)}function D(a,m){function e(a){switch(a.nodeType){case 1:if(k.test(a.className))break;if("BR"===a.nodeName)h(a), +a.parentNode&&a.parentNode.removeChild(a);else for(a=a.firstChild;a;a=a.nextSibling)e(a);break;case 3:case 4:if(p){var b=a.nodeValue,d=b.match(t);if(d){var c=b.substring(0,d.index);a.nodeValue=c;(b=b.substring(d.index+d[0].length))&&a.parentNode.insertBefore(s.createTextNode(b),a.nextSibling);h(a);c||a.parentNode.removeChild(a)}}}}function h(a){function b(a,d){var e=d?a.cloneNode(!1):a,f=a.parentNode;if(f){var f=b(f,1),g=a.nextSibling;f.appendChild(e);for(var h=g;h;h=g)g=h.nextSibling,f.appendChild(h)}return e} +for(;!a.nextSibling;)if(a=a.parentNode,!a)return;for(var a=b(a.nextSibling,0),e;(e=a.parentNode)&&e.nodeType===1;)a=e;d.push(a)}var k=/(?:^|\s)nocode(?:\s|$)/,t=/\r\n?|\n/,s=a.ownerDocument,l;a.currentStyle?l=a.currentStyle.whiteSpace:window.getComputedStyle&&(l=s.defaultView.getComputedStyle(a,q).getPropertyValue("white-space"));var p=l&&"pre"===l.substring(0,3);for(l=s.createElement("LI");a.firstChild;)l.appendChild(a.firstChild);for(var d=[l],g=0;g=0;){var h=m[e];A.hasOwnProperty(h)?window.console&&console.warn("cannot override language handler %s",h):A[h]=a}}function C(a,m){if(!a||!A.hasOwnProperty(a))a=/^\s*=o&&(h+=2);e>=c&&(a+=2)}}catch(w){"console"in window&&console.log(w&&w.stack?w.stack:w)}}var v=["break,continue,do,else,for,if,return,while"],w=[[v,"auto,case,char,const,default,double,enum,extern,float,goto,int,long,register,short,signed,sizeof,static,struct,switch,typedef,union,unsigned,void,volatile"], +"catch,class,delete,false,import,new,operator,private,protected,public,this,throw,true,try,typeof"],F=[w,"alignof,align_union,asm,axiom,bool,concept,concept_map,const_cast,constexpr,decltype,dynamic_cast,explicit,export,friend,inline,late_check,mutable,namespace,nullptr,reinterpret_cast,static_assert,static_cast,template,typeid,typename,using,virtual,where"],G=[w,"abstract,boolean,byte,extends,final,finally,implements,import,instanceof,null,native,package,strictfp,super,synchronized,throws,transient"], +H=[G,"as,base,by,checked,decimal,delegate,descending,dynamic,event,fixed,foreach,from,group,implicit,in,interface,internal,into,is,lock,object,out,override,orderby,params,partial,readonly,ref,sbyte,sealed,stackalloc,string,select,uint,ulong,unchecked,unsafe,ushort,var"],w=[w,"debugger,eval,export,function,get,null,set,undefined,var,with,Infinity,NaN"],I=[v,"and,as,assert,class,def,del,elif,except,exec,finally,from,global,import,in,is,lambda,nonlocal,not,or,pass,print,raise,try,with,yield,False,True,None"], +J=[v,"alias,and,begin,case,class,def,defined,elsif,end,ensure,false,in,module,next,nil,not,or,redo,rescue,retry,self,super,then,true,undef,unless,until,when,yield,BEGIN,END"],v=[v,"case,done,elif,esac,eval,fi,function,in,local,set,then,until"],K=/^(DIR|FILE|vector|(de|priority_)?queue|list|stack|(const_)?iterator|(multi)?(set|map)|bitset|u?(int|float)\d*)/,N=/\S/,O=u({keywords:[F,H,w,"caller,delete,die,do,dump,elsif,eval,exit,foreach,for,goto,if,import,last,local,my,next,no,our,print,package,redo,require,sub,undef,unless,until,use,wantarray,while,BEGIN,END"+ +I,J,v],hashComments:!0,cStyleComments:!0,multiLineStrings:!0,regexLiterals:!0}),A={};k(O,["default-code"]);k(x([],[["pln",/^[^]*(?:>|$)/],["com",/^<\!--[\S\s]*?(?:--\>|$)/],["lang-",/^<\?([\S\s]+?)(?:\?>|$)/],["lang-",/^<%([\S\s]+?)(?:%>|$)/],["pun",/^(?:<[%?]|[%?]>)/],["lang-",/^]*>([\S\s]+?)<\/xmp\b[^>]*>/i],["lang-js",/^]*>([\S\s]*?)(<\/script\b[^>]*>)/i],["lang-css",/^]*>([\S\s]*?)(<\/style\b[^>]*>)/i],["lang-in.tag",/^(<\/?[a-z][^<>]*>)/i]]), +["default-markup","htm","html","mxml","xhtml","xml","xsl"]);k(x([["pln",/^\s+/,q," \t\r\n"],["atv",/^(?:"[^"]*"?|'[^']*'?)/,q,"\"'"]],[["tag",/^^<\/?[a-z](?:[\w-.:]*\w)?|\/?>$/i],["atn",/^(?!style[\s=]|on)[a-z](?:[\w:-]*\w)?/i],["lang-uq.val",/^=\s*([^\s"'>]*(?:[^\s"'/>]|\/(?=\s)))/],["pun",/^[/<->]+/],["lang-js",/^on\w+\s*=\s*"([^"]+)"/i],["lang-js",/^on\w+\s*=\s*'([^']+)'/i],["lang-js",/^on\w+\s*=\s*([^\s"'>]+)/i],["lang-css",/^style\s*=\s*"([^"]+)"/i],["lang-css",/^style\s*=\s*'([^']+)'/i],["lang-css", +/^style\s*=\s*([^\s"'>]+)/i]]),["in.tag"]);k(x([],[["atv",/^[\S\s]+/]]),["uq.val"]);k(u({keywords:F,hashComments:!0,cStyleComments:!0,types:K}),["c","cc","cpp","cxx","cyc","m"]);k(u({keywords:"null,true,false"}),["json"]);k(u({keywords:H,hashComments:!0,cStyleComments:!0,verbatimStrings:!0,types:K}),["cs"]);k(u({keywords:G,cStyleComments:!0}),["java"]);k(u({keywords:v,hashComments:!0,multiLineStrings:!0}),["bsh","csh","sh"]);k(u({keywords:I,hashComments:!0,multiLineStrings:!0,tripleQuotedStrings:!0}), +["cv","py"]);k(u({keywords:"caller,delete,die,do,dump,elsif,eval,exit,foreach,for,goto,if,import,last,local,my,next,no,our,print,package,redo,require,sub,undef,unless,until,use,wantarray,while,BEGIN,END",hashComments:!0,multiLineStrings:!0,regexLiterals:!0}),["perl","pl","pm"]);k(u({keywords:J,hashComments:!0,multiLineStrings:!0,regexLiterals:!0}),["rb"]);k(u({keywords:w,cStyleComments:!0,regexLiterals:!0}),["js"]);k(u({keywords:"all,and,by,catch,class,else,extends,false,finally,for,if,in,is,isnt,loop,new,no,not,null,of,off,on,or,return,super,then,true,try,unless,until,when,while,yes", +hashComments:3,cStyleComments:!0,multilineStrings:!0,tripleQuotedStrings:!0,regexLiterals:!0}),["coffee"]);k(x([],[["str",/^[\S\s]+/]]),["regex"]);window.prettyPrintOne=function(a,m,e){var h=document.createElement("PRE");h.innerHTML=a;e&&D(h,e);E({g:m,i:e,h:h});return h.innerHTML};window.prettyPrint=function(a){function m(){for(var e=window.PR_SHOULD_USE_CONTINUATION?l.now()+250:Infinity;p=0){var k=k.match(g),f,b;if(b= +!k){b=n;for(var o=void 0,c=b.firstChild;c;c=c.nextSibling)var i=c.nodeType,o=i===1?o?b:c:i===3?N.test(c.nodeValue)?b:o:o;b=(f=o===b?void 0:o)&&"CODE"===f.tagName}b&&(k=f.className.match(g));k&&(k=k[1]);b=!1;for(o=n.parentNode;o;o=o.parentNode)if((o.tagName==="pre"||o.tagName==="code"||o.tagName==="xmp")&&o.className&&o.className.indexOf("prettyprint")>=0){b=!0;break}b||((b=(b=n.className.match(/\blinenums\b(?::(\d+))?/))?b[1]&&b[1].length?+b[1]:!0:!1)&&D(n,b),d={g:k,h:n,i:b},E(d))}}p0&&(g.splice(m-1,2),m-=2);m=q.pkgs[g=b[0]];b=b.join("/");m&&b===g+"/"+m.main&&(b=g)}else b.indexOf("./")=== +0&&(b=b.substring(2));return b}function l(b,f){var g=b?b.indexOf("!"):-1,m=null,a=f?f.name:null,h=b,e,d;g!==-1&&(m=b.substring(0,g),b=b.substring(g+1,b.length));m&&(m=c(m,a));b&&(m?e=(g=n[m])&&g.normalize?g.normalize(b,function(b){return c(b,a)}):c(b,a):(e=c(b,a),d=G[e],d||(d=i.nameToUrl(b,null,f),G[e]=d)));return{prefix:m,name:e,parentMap:f,url:d,originalName:h,fullName:m?m+"!"+(e||""):e}}function j(){var b=!0,f=q.priorityWait,g,a;if(f){for(a=0;g=f[a];a++)if(!s[g]){b=!1;break}b&&delete q.priorityWait}return b} +function k(b,f,g){return function(){var a=ha.call(arguments,0),c;if(g&&K(c=a[a.length-1]))c.__requireJsBuild=!0;a.push(f);return b.apply(null,a)}}function t(b,f,g){f=k(g||i.require,b,f);$(f,{nameToUrl:k(i.nameToUrl,b),toUrl:k(i.toUrl,b),defined:k(i.requireDefined,b),specified:k(i.requireSpecified,b),isBrowser:d.isBrowser});return f}function p(b){var f,g,a,c=b.callback,h=b.map,e=h.fullName,ca=b.deps;a=b.listeners;var j=q.requireExecCb||d.execCb;if(c&&K(c)){if(q.catchError.define)try{g=j(e,b.callback, +ca,n[e])}catch(k){f=k}else g=j(e,b.callback,ca,n[e]);if(e)(c=b.cjsModule)&&c.exports!==r&&c.exports!==n[e]?g=n[e]=b.cjsModule.exports:g===r&&b.usingExports?g=n[e]:(n[e]=g,H[e]&&(T[e]=!0))}else e&&(g=n[e]=c,H[e]&&(T[e]=!0));if(x[b.id])delete x[b.id],b.isDone=!0,i.waitCount-=1,i.waitCount===0&&(J=[]);delete M[e];if(d.onResourceLoad&&!b.placeholder)d.onResourceLoad(i,h,b.depArray);if(f)return g=(e?l(e).url:"")||f.fileName||f.sourceURL,a=f.moduleTree,f=P("defineerror",'Error evaluating module "'+e+'" at location "'+ +g+'":\n'+f+"\nfileName:"+g+"\nlineNumber: "+(f.lineNumber||f.line),f),f.moduleName=e,f.moduleTree=a,d.onError(f);for(f=0;c=a[f];f++)c(g);return r}function u(b,f){return function(g){b.depDone[f]||(b.depDone[f]=!0,b.deps[f]=g,b.depCount-=1,b.depCount||p(b))}}function o(b,f){var g=f.map,a=g.fullName,c=g.name,h=N[b]||(N[b]=n[b]),e;if(!f.loading)f.loading=!0,e=function(b){f.callback=function(){return b};p(f);s[f.id]=!0;A()},e.fromText=function(b,f){var g=Q;s[b]=!1;i.scriptCount+=1;i.fake[b]=!0;g&&(Q=!1); +d.exec(f);g&&(Q=!0);i.completeLoad(b)},a in n?e(n[a]):h.load(c,t(g.parentMap,!0,function(b,a){var c=[],e,m;for(e=0;m=b[e];e++)m=l(m,g.parentMap),b[e]=m.fullName,m.prefix||c.push(b[e]);f.moduleDeps=(f.moduleDeps||[]).concat(c);return i.require(b,a)}),e,q)}function y(b){x[b.id]||(x[b.id]=b,J.push(b),i.waitCount+=1)}function D(b){this.listeners.push(b)}function v(b,f){var g=b.fullName,a=b.prefix,c=a?N[a]||(N[a]=n[a]):null,h,e;g&&(h=M[g]);if(!h&&(e=!0,h={id:(a&&!c?O++ +"__p@:":"")+(g||"__r@"+O++),map:b, +depCount:0,depDone:[],depCallbacks:[],deps:[],listeners:[],add:D},B[h.id]=!0,g&&(!a||N[a])))M[g]=h;a&&!c?(g=l(a),a in n&&!n[a]&&(delete n[a],delete R[g.url]),a=v(g,!0),a.add(function(){var f=l(b.originalName,b.parentMap),f=v(f,!0);h.placeholder=!0;f.add(function(b){h.callback=function(){return b};p(h)})})):e&&f&&(s[h.id]=!1,i.paused.push(h),y(h));return h}function C(b,f,a,c){var b=l(b,c),d=b.name,h=b.fullName,e=v(b),j=e.id,k=e.deps,o;if(h){if(h in n||s[j]===!0||h==="jquery"&&q.jQuery&&q.jQuery!== +a().fn.jquery)return;B[j]=!0;s[j]=!0;h==="jquery"&&a&&W(a())}e.depArray=f;e.callback=a;for(a=0;a0)return r;if(q.priorityWait)if(j())A();else return r;for(h in s)if(!(h in L)&&(c=!0,!s[h]))if(b)a+=h+" ";else if(l=!0,h.indexOf("!")===-1){k=[];break}else(e=M[h]&&M[h].moduleDeps)&&k.push.apply(k,e);if(!c&&!i.waitCount)return r;if(b&&a)return b=P("timeout","Load timeout for modules: "+a),b.requireType="timeout",b.requireModules=a,b.contextName=i.contextName,d.onError(b); +if(l&&k.length)for(a=0;h=x[k[a]];a++)if(h=F(h,{})){z(h,{});break}if(!b&&(l||i.scriptCount)){if((I||da)&&!X)X=setTimeout(function(){X=0;E()},50);return r}if(i.waitCount){for(a=0;h=J[a];a++)z(h,{});i.paused.length&&A();Y<5&&(Y+=1,E())}Y=0;d.checkReadyState();return r}var i,A,q={waitSeconds:7,baseUrl:"./",paths:{},pkgs:{},catchError:{}},S=[],B={require:!0,exports:!0,module:!0},G={},n={},s={},x={},J=[],R={},O=0,M={},N={},H={},T={},Z=0;W=function(b){if(!i.jQuery&&(b=b||(typeof jQuery!=="undefined"?jQuery: +null))&&!(q.jQuery&&b.fn.jquery!==q.jQuery)&&("holdReady"in b||"readyWait"in b))if(i.jQuery=b,w(["jquery",[],function(){return jQuery}]),i.scriptCount)V(b,!0),i.jQueryIncremented=!0};A=function(){var b,a,c,l,k,h;i.takeGlobalQueue();Z+=1;if(i.scriptCount<=0)i.scriptCount=0;for(;S.length;)if(b=S.shift(),b[0]===null)return d.onError(P("mismatch","Mismatched anonymous define() module: "+b[b.length-1]));else w(b);if(!q.priorityWait||j())for(;i.paused.length;){k=i.paused;i.pausedCount+=k.length;i.paused= +[];for(l=0;b=k[l];l++)a=b.map,c=a.url,h=a.fullName,a.prefix?o(a.prefix,b):!R[c]&&!s[h]&&((q.requireLoad||d.load)(i,h,c),c.indexOf("empty:")!==0&&(R[c]=!0));i.startTime=(new Date).getTime();i.pausedCount-=k.length}Z===1&&E();Z-=1;return r};i={contextName:a,config:q,defQueue:S,waiting:x,waitCount:0,specified:B,loaded:s,urlMap:G,urlFetched:R,scriptCount:0,defined:n,paused:[],pausedCount:0,plugins:N,needFullExec:H,fake:{},fullExec:T,managerCallbacks:M,makeModuleMap:l,normalize:c,configure:function(b){var a, +c,d;b.baseUrl&&b.baseUrl.charAt(b.baseUrl.length-1)!=="/"&&(b.baseUrl+="/");a=q.paths;d=q.pkgs;$(q,b,!0);if(b.paths){for(c in b.paths)c in L||(a[c]=b.paths[c]);q.paths=a}if((a=b.packagePaths)||b.packages){if(a)for(c in a)c in L||aa(d,a[c],c);b.packages&&aa(d,b.packages);q.pkgs=d}if(b.priority)c=i.requireWait,i.requireWait=!1,A(),i.require(b.priority),A(),i.requireWait=c,q.priorityWait=b.priority;if(b.deps||b.callback)i.require(b.deps||[],b.callback)},requireDefined:function(b,a){return l(b,a).fullName in +n},requireSpecified:function(b,a){return l(b,a).fullName in B},require:function(b,c,g){if(typeof b==="string"){if(K(c))return d.onError(P("requireargs","Invalid require call"));if(d.get)return d.get(i,b,c);c=l(b,c);b=c.fullName;return!(b in n)?d.onError(P("notloaded","Module name '"+c.fullName+"' has not been loaded yet for context: "+a)):n[b]}(b&&b.length||c)&&C(null,b,c,g);if(!i.requireWait)for(;!i.scriptCount&&i.paused.length;)A();return i.require},takeGlobalQueue:function(){U.length&&(ja.apply(i.defQueue, +[i.defQueue.length-1,0].concat(U)),U=[])},completeLoad:function(b){var a;for(i.takeGlobalQueue();S.length;)if(a=S.shift(),a[0]===null){a[0]=b;break}else if(a[0]===b)break;else w(a),a=null;a?w(a):w([b,[],b==="jquery"&&typeof jQuery!=="undefined"?function(){return jQuery}:null]);d.isAsync&&(i.scriptCount-=1);A();d.isAsync||(i.scriptCount-=1)},toUrl:function(b,a){var c=b.lastIndexOf("."),d=null;c!==-1&&(d=b.substring(c,b.length),b=b.substring(0,c));return i.nameToUrl(b,d,a)},nameToUrl:function(b,a,g){var l, +k,h,e,j=i.config,b=c(b,g&&g.fullName);if(d.jsExtRegExp.test(b))a=b+(a?a:"");else{l=j.paths;k=j.pkgs;g=b.split("/");for(e=g.length;e>0;e--)if(h=g.slice(0,e).join("/"),l[h]){g.splice(0,e,l[h]);break}else if(h=k[h]){b=b===h.name?h.location+"/"+h.main:h.location;g.splice(0,e,b);break}a=g.join("/")+(a||".js");a=(a.charAt(0)==="/"||a.match(/^[\w\+\.\-]+:/)?"":j.baseUrl)+a}return j.urlArgs?a+((a.indexOf("?")===-1?"?":"&")+j.urlArgs):a}};i.jQueryCheck=W;i.resume=A;return i}function ka(){var a,c,d;if(C&&C.readyState=== +"interactive")return C;a=document.getElementsByTagName("script");for(c=a.length-1;c>-1&&(d=a[c]);c--)if(d.readyState==="interactive")return C=d;return null}var la=/(\/\*([\s\S]*?)\*\/|([^:]|^)\/\/(.*)$)/mg,ma=/require\(\s*["']([^'"\s]+)["']\s*\)/g,fa=/^\.\//,ba=/\.js$/,O=Object.prototype.toString,u=Array.prototype,ha=u.slice,ja=u.splice,I=!!(typeof window!=="undefined"&&navigator&&document),da=!I&&typeof importScripts!=="undefined",na=I&&navigator.platform==="PLAYSTATION 3"?/^complete$/:/^(complete|loaded)$/, +ea=typeof opera!=="undefined"&&opera.toString()==="[object Opera]",L={},D={},U=[],C=null,Y=0,Q=!1,ia={require:!0,module:!0,exports:!0},d,u={},J,y,v,E,o,w,F,B,z,W,X;if(typeof define==="undefined"){if(typeof requirejs!=="undefined")if(K(requirejs))return;else u=requirejs,requirejs=r;typeof require!=="undefined"&&!K(require)&&(u=require,require=r);d=requirejs=function(a,c,d){var j="_",k;!G(a)&&typeof a!=="string"&&(k=a,G(c)?(a=c,c=d):a=[]);if(k&&k.context)j=k.context;d=D[j]||(D[j]=ga(j));k&&d.configure(k); +return d.require(a,c)};d.config=function(a){return d(a)};require||(require=d);d.toUrl=function(a){return D._.toUrl(a)};d.version="1.0.8";d.jsExtRegExp=/^\/|:|\?|\.js$/;y=d.s={contexts:D,skipAsync:{}};if(d.isAsync=d.isBrowser=I)if(v=y.head=document.getElementsByTagName("head")[0],E=document.getElementsByTagName("base")[0])v=y.head=E.parentNode;d.onError=function(a){throw a;};d.load=function(a,c,l){d.resourcesReady(!1);a.scriptCount+=1;d.attach(l,a,c);if(a.jQuery&&!a.jQueryIncremented)V(a.jQuery,!0), +a.jQueryIncremented=!0};define=function(a,c,d){var j,k;typeof a!=="string"&&(d=c,c=a,a=null);G(c)||(d=c,c=[]);!c.length&&K(d)&&d.length&&(d.toString().replace(la,"").replace(ma,function(a,d){c.push(d)}),c=(d.length===1?["require"]:["require","exports","module"]).concat(c));if(Q&&(j=J||ka()))a||(a=j.getAttribute("data-requiremodule")),k=D[j.getAttribute("data-requirecontext")];(k?k.defQueue:U).push([a,c,d]);return r};define.amd={multiversion:!0,plugins:!0,jQuery:!0};d.exec=function(a){return eval(a)}; +d.execCb=function(a,c,d,j){return c.apply(j,d)};d.addScriptToDom=function(a){J=a;E?v.insertBefore(a,E):v.appendChild(a);J=null};d.onScriptLoad=function(a){var c=a.currentTarget||a.srcElement,l;if(a.type==="load"||c&&na.test(c.readyState))C=null,a=c.getAttribute("data-requirecontext"),l=c.getAttribute("data-requiremodule"),D[a].completeLoad(l),c.detachEvent&&!ea?c.detachEvent("onreadystatechange",d.onScriptLoad):c.removeEventListener("load",d.onScriptLoad,!1)};d.attach=function(a,c,l,j,k,o){var p; +if(I)return j=j||d.onScriptLoad,p=c&&c.config&&c.config.xhtml?document.createElementNS("http://www.w3.org/1999/xhtml","html:script"):document.createElement("script"),p.type=k||c&&c.config.scriptType||"text/javascript",p.charset="utf-8",p.async=!y.skipAsync[a],c&&p.setAttribute("data-requirecontext",c.contextName),p.setAttribute("data-requiremodule",l),p.attachEvent&&!(p.attachEvent.toString&&p.attachEvent.toString().indexOf("[native code]")<0)&&!ea?(Q=!0,o?p.onreadystatechange=function(){if(p.readyState=== +"loaded")p.onreadystatechange=null,p.attachEvent("onreadystatechange",j),o(p)}:p.attachEvent("onreadystatechange",j)):p.addEventListener("load",j,!1),p.src=a,o||d.addScriptToDom(p),p;else da&&(importScripts(a),c.completeLoad(l));return null};if(I){o=document.getElementsByTagName("script");for(B=o.length-1;B>-1&&(w=o[B]);B--){if(!v)v=w.parentNode;if(F=w.getAttribute("data-main")){if(!u.baseUrl)o=F.split("/"),w=o.pop(),o=o.length?o.join("/")+"/":"./",u.baseUrl=o,F=w.replace(ba,"");u.deps=u.deps?u.deps.concat(F): +[F];break}}}d.checkReadyState=function(){var a=y.contexts,c;for(c in a)if(!(c in L)&&a[c].waitCount)return;d.resourcesReady(!0)};d.resourcesReady=function(a){var c,l;d.resourcesDone=a;if(d.resourcesDone)for(l in a=y.contexts,a)if(!(l in L)&&(c=a[l],c.jQueryIncremented))V(c.jQuery,!1),c.jQueryIncremented=!1};d.pageLoaded=function(){if(document.readyState!=="complete")document.readyState="complete"};if(I&&document.addEventListener&&!document.readyState)document.readyState="loading",window.addEventListener("load", +d.pageLoaded,!1);d(u);if(d.isAsync&&typeof setTimeout!=="undefined")z=y.contexts[u.context||"_"],z.requireWait=!0,setTimeout(function(){z.requireWait=!1;z.scriptCount||z.resume();d.checkReadyState()},0)}})(); diff --git a/source/_themes/uwpce_slides2/static/js/slide-controller.js b/source/_themes/uwpce_slides2/static/js/slide-controller.js new file mode 100644 index 00000000..571317b9 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/slide-controller.js @@ -0,0 +1,109 @@ +(function(window) { + +var ORIGIN_ = location.protocol + '//' + location.host; + +function SlideController() { + this.popup = null; + this.isPopup = window.opener; + + if (this.setupDone()) { + window.addEventListener('message', this.onMessage_.bind(this), false); + + // Close popups if we reload the main window. + window.addEventListener('beforeunload', function(e) { + if (this.popup) { + this.popup.close(); + } + }.bind(this), false); + } +} + +SlideController.PRESENTER_MODE_PARAM = 'presentme'; + +SlideController.prototype.setupDone = function() { + var params = location.search.substring(1).split('&').map(function(el) { + return el.split('='); + }); + + var presentMe = null; + for (var i = 0, param; param = params[i]; ++i) { + if (param[0].toLowerCase() == SlideController.PRESENTER_MODE_PARAM) { + presentMe = param[1] == 'true'; + break; + } + } + + if (presentMe !== null) { + localStorage.ENABLE_PRESENTOR_MODE = presentMe; + // TODO: use window.history.pushState to update URL instead of the redirect. + if (window.history.replaceState) { + window.history.replaceState({}, '', location.pathname); + } else { + location.replace(location.pathname); + return false; + } + } + + var enablePresenterMode = localStorage.getItem('ENABLE_PRESENTOR_MODE'); + if (enablePresenterMode && JSON.parse(enablePresenterMode)) { + // Only open popup from main deck. Don't want recursive popup opening! + if (!this.isPopup) { + var opts = 'menubar=no,location=yes,resizable=yes,scrollbars=no,status=no'; + this.popup = window.open(location.href, 'mywindow', opts); + + // Loading in the popup? Trigger the hotkey for turning presenter mode on. + this.popup.addEventListener('load', function(e) { + var evt = this.popup.document.createEvent('Event'); + evt.initEvent('keydown', true, true); + evt.keyCode = 'P'.charCodeAt(0); + this.popup.document.dispatchEvent(evt); + // this.popup.document.body.classList.add('with-notes'); + // document.body.classList.add('popup'); + }.bind(this), false); + } + } + + return true; +} + +SlideController.prototype.onMessage_ = function(e) { + var data = e.data; + + // Restrict messages to being from this origin. Allow local developmet + // from file:// though. + // TODO: It would be dope if FF implemented location.origin! + if (e.origin != ORIGIN_ && ORIGIN_.indexOf('file://') != 0) { + alert('Someone tried to postMessage from an unknown origin'); + return; + } + + // if (e.source.location.hostname != 'localhost') { + // alert('Someone tried to postMessage from an unknown origin'); + // return; + // } + + if ('keyCode' in data) { + var evt = document.createEvent('Event'); + evt.initEvent('keydown', true, true); + evt.keyCode = data.keyCode; + document.dispatchEvent(evt); + } +}; + +SlideController.prototype.sendMsg = function(msg) { + // // Send message to popup window. + // if (this.popup) { + // this.popup.postMessage(msg, ORIGIN_); + // } + + // Send message to main window. + if (this.isPopup) { + // TODO: It would be dope if FF implemented location.origin. + window.opener.postMessage(msg, '*'); + } +}; + +window.SlideController = SlideController; + +})(window); + diff --git a/source/_themes/uwpce_slides2/static/js/slide-deck-instantiate.js b/source/_themes/uwpce_slides2/static/js/slide-deck-instantiate.js new file mode 100644 index 00000000..08b2ebdc --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/slide-deck-instantiate.js @@ -0,0 +1,13 @@ + +// Polyfill missing APIs (if we need to), then create the slide deck. +// iOS < 5 needs classList, dataset, and window.matchMedia. Modernizr contains +// the last one. +(function() { + Modernizr.load({ + test: !!document.body.classList && !!document.body.dataset, + nope: ['js/polyfills/classList.min.js', 'js/polyfills/dataset.min.js'], + complete: function() { + window.slidedeck = new SlideDeck(); + } + }); +})(); diff --git a/source/_themes/uwpce_slides2/static/js/slide-deck.js b/source/_themes/uwpce_slides2/static/js/slide-deck.js new file mode 100644 index 00000000..60b9681c --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/slide-deck.js @@ -0,0 +1,900 @@ +/** + * @authors Luke Mahe + * @authors Eric Bidelman + * @fileoverview TODO + */ +document.cancelFullScreen = document.webkitCancelFullScreen || + document.mozCancelFullScreen; + +/** + * @constructor + */ +function SlideDeck(el) { + this.curSlide_ = 0; + this.prevSlide_ = 0; + this.config_ = null; + this.container = el || document.querySelector('slides'); + this.slides = []; + this.controller = null; + + this.getCurrentSlideFromHash_(); + + // Call this explicitly. Modernizr.load won't be done until after DOM load. + this.onDomLoaded_.bind(this)(); +} + +/** + * @const + * @private + */ +SlideDeck.prototype.SLIDE_CLASSES_ = [ + 'far-past', 'past', 'current', 'next', 'far-next']; + +/** + * @const + * @private + */ +SlideDeck.prototype.CSS_DIR_ = '_static/theme/css/'; + + +/** + * @private + */ +SlideDeck.prototype.findSlideById = function(title_id) { + // Return the 1-base index of the Slide with id ``title_id`` + // + // The index must be 1-based, as it's passed to code which assumes + // it was specified as the location fragment. + + console.log('findSlideById ', title_id); + + slideEls = document.querySelectorAll('slides > slide'); + console.log(slideEls); + + for (var i = 0; i < slideEls.length; i++) { + if (slideEls.item(i).id == title_id) { + return i + 1; + } + } + + // no match on a slide, perhaps it's an explicit reference? + var + target_link = document.querySelector("span[id='" + title_id + "']"), + // XXX this is pretty strict, may need to be more flexible in the future + slide = (target_link && target_link.parentNode); + + if (slide && slide.tagName == 'SLIDE') { + return this.findSlideById(slide.id); + } + + return false; + +}; + +/** + * @private + */ +SlideDeck.prototype.getCurrentSlideFromHash_ = function() { + var slideNo = parseInt(document.location.hash.substr(1)); + + if (isNaN(slideNo)) { + // must be a section title reference + slideNo = this.findSlideById(location.hash.substr(1)); + } + + if (slideNo) { + this.curSlide_ = slideNo - 1; + } else { + this.curSlide_ = 0; + } +}; + +/** + * @param {number} slideNo + */ +SlideDeck.prototype.loadSlide = function(slideNo) { + if (slideNo) { + this.curSlide_ = slideNo - 1; + this.updateSlides_(); + } +}; + +/** + * @private + */ +SlideDeck.prototype.onDomLoaded_ = function(e) { + document.body.classList.add('loaded'); // Add loaded class for templates to use. + + this.slides = this.container.querySelectorAll('slide:not([hidden]):not(.hidden):not(.backdrop)'); + + // If we're on a smartphone, apply special sauce. + if (Modernizr.mq('only screen and (max-device-width: 480px)')) { + // var style = document.createElement('link'); + // style.rel = 'stylesheet'; + // style.type = 'text/css'; + // style.href = this.CSS_DIR_ + 'phone.css'; + // document.querySelector('head').appendChild(style); + + // No need for widescreen layout on a phone. + this.container.classList.remove('layout-widescreen'); + } + + this.loadConfig_(SLIDE_CONFIG); + this.addEventListeners_(); + this.updateSlides_(); + + // Add slide numbers and total slide count metadata to each slide. + var that = this; + for (var i = 0, slide; slide = this.slides[i]; ++i) { + slide.dataset.slideNum = i + 1; + slide.dataset.totalSlides = this.slides.length; + + slide.addEventListener('click', function(e) { + if (document.body.classList.contains('overview')) { + that.loadSlide(this.dataset.slideNum); + e.preventDefault(); + window.setTimeout(function() { + that.toggleOverview(); + }, 500); + } + }, false); + } + + // Note: this needs to come after addEventListeners_(), which adds a + // 'keydown' listener that this controller relies on. + + // Modernizr.touch isn't a sufficient check for devices that support both + // touch and mouse. Create the controller in all cases. + // // Also, no need to set this up if we're on mobile. + // if (!Modernizr.touch) { + this.controller = new SlideController(this); + if (this.controller.isPopup) { + document.body.classList.add('popup'); + } + //} +}; + +/** + * @private + */ +SlideDeck.prototype.addEventListeners_ = function() { + document.addEventListener('keydown', this.onBodyKeyDown_.bind(this), false); + window.addEventListener('popstate', this.onPopState_.bind(this), false); + + // var transEndEventNames = { + // 'WebkitTransition': 'webkitTransitionEnd', + // 'MozTransition': 'transitionend', + // 'OTransition': 'oTransitionEnd', + // 'msTransition': 'MSTransitionEnd', + // 'transition': 'transitionend' + // }; + // + // // Find the correct transitionEnd vendor prefix. + // window.transEndEventName = transEndEventNames[ + // Modernizr.prefixed('transition')]; + // + // // When slides are done transitioning, kickoff loading iframes. + // // Note: we're only looking at a single transition (on the slide). This + // // doesn't include autobuilds the slides may have. Also, if the slide + // // transitions on multiple properties (e.g. not just 'all'), this doesn't + // // handle that case. + // this.container.addEventListener(transEndEventName, function(e) { + // this.enableSlideFrames_(this.curSlide_); + // }.bind(this), false); + + // document.addEventListener('slideenter', function(e) { + // var slide = e.target; + // window.setTimeout(function() { + // this.enableSlideFrames_(e.slideNumber); + // this.enableSlideFrames_(e.slideNumber + 1); + // }.bind(this), 300); + // }.bind(this), false); +}; + +/** + * @private + * @param {Event} e The pop event. + */ +SlideDeck.prototype.onPopState_ = function(e) { + if (e.state != null) { + this.curSlide_ = e.state; + this.updateSlides_(true); + } +}; + +/** + * @param {Event} e + */ +SlideDeck.prototype.onBodyKeyDown_ = function(e) { + if (/^(input|textarea)$/i.test(e.target.nodeName) || + e.target.isContentEditable) { + return; + } + + // Forward keydowns to the main slides if we're the popup. + if (this.controller && this.controller.isPopup) { + this.controller.sendMsg({keyCode: e.keyCode}); + } + + switch (e.keyCode) { + case 13: // Enter + if (document.body.classList.contains('overview')) { + this.toggleOverview(); + } + break; + + case 39: // right arrow + case 32: // space + case 34: // PgDn + this.nextSlide(); + e.preventDefault(); + break; + + case 37: // left arrow + case 8: // Backspace + case 33: // PgUp + this.prevSlide(); + e.preventDefault(); + break; + + case 40: // down arrow + this.nextSlide(); + e.preventDefault(); + break; + + case 38: // up arrow + this.prevSlide(); + e.preventDefault(); + break; + + case 72: // H: Toggle code highlighting + document.body.classList.toggle('highlight-code'); + break; + + case 79: // O: Toggle overview + this.toggleOverview(); + break; + + case 80: // P + if (this.controller && this.controller.isPopup) { + document.body.classList.toggle('with-notes'); + } else if (this.controller && !this.controller.popup) { + document.body.classList.toggle('with-notes'); + } + break; + + case 82: // R + // TODO: implement refresh on main slides when popup is refreshed. + break; + + case 27: // ESC: Hide notes and highlighting + document.body.classList.remove('with-notes'); + document.body.classList.remove('highlight-code'); + + if (document.body.classList.contains('overview')) { + this.toggleOverview(); + } + break; + + case 70: // F: Toggle fullscreen + // Only respect 'f' on body. Don't want to capture keys from an . + // Also, ignore browser's fullscreen shortcut (cmd+shift+f) so we don't + // get trapped in fullscreen! + if (e.target == document.body && !(e.shiftKey && e.metaKey)) { + if (document.mozFullScreen !== undefined && !document.mozFullScreen) { + document.body.mozRequestFullScreen(Element.ALLOW_KEYBOARD_INPUT); + } else if (document.webkitIsFullScreen !== undefined && !document.webkitIsFullScreen) { + document.body.webkitRequestFullScreen(Element.ALLOW_KEYBOARD_INPUT); + } else { + document.cancelFullScreen(); + } + } + break; + + case 87: // W: Toggle widescreen + // Only respect 'w' on body. Don't want to capture keys from an . + if (e.target == document.body && !(e.shiftKey && e.metaKey)) { + this.container.classList.toggle('layout-widescreen'); + } + break; + } +}; + +/** + * + */ +SlideDeck.prototype.focusOverview_ = function() { + var overview = document.body.classList.contains('overview'); + + for (var i = 0, slide; slide = this.slides[i]; i++) { + slide.style[Modernizr.prefixed('transform')] = overview ? + 'translateZ(-2500px) translate(' + (( i - this.curSlide_ ) * 105) + + '%, 0%)' : ''; + } +}; + +/** + */ +SlideDeck.prototype.toggleOverview = function() { + document.body.classList.toggle('overview'); + + this.focusOverview_(); +}; + +/** + * @private + */ +SlideDeck.prototype.loadConfig_ = function(config) { + if (!config) { + return; + } + + this.config_ = config; + + var settings = this.config_.settings; + + this.loadTheme_(settings.theme || []); + + if (settings.favIcon) { + this.addFavIcon_(settings.favIcon); + } + + // Prettyprint. Default to on. + if (!!!('usePrettify' in settings) || settings.usePrettify) { + prettyPrint(); + } + + if (settings.analytics) { + this.loadAnalytics_(); + } + + if (settings.fonts) { + this.addFonts_(settings.fonts); + } + + // Builds. Default to on. + if (!!!('useBuilds' in settings) || settings.useBuilds) { + this.makeBuildLists_(); + } + + if (settings.title) { + document.title = settings.title.replace(//, ' '); + if (settings.eventInfo && settings.eventInfo.title) { + document.title += ' - ' + settings.eventInfo.title; + } + document.querySelector('[data-config-title]').innerHTML = settings.title; + } + + if (settings.subtitle) { + document.querySelector('[data-config-subtitle]').innerHTML = settings.subtitle; + } + + if (this.config_.presenters) { + var presenters = this.config_.presenters; + var dataConfigContact = document.querySelector('[data-config-contact]'); + + var html = []; + if (presenters.length == 1) { + var p = presenters[0]; + + var presenterTitle = [p.name]; + if (p.company) { + presenterTitle.push(p.company); + } + html = presenterTitle.join(' - ') + '
'; + + var gplus = p.gplus ? 'g+' + p.gplus.replace(/https?:\/\//, '') + '' : ''; + + var twitter = p.twitter ? 'twitter' + + '' + + p.twitter + '' : ''; + + var www = p.www ? 'www' + p.www.replace(/https?:\/\//, '') + '' : ''; + + var github = p.github ? 'github' + p.github.replace(/https?:\/\//, '') + '' : ''; + + var html2 = [gplus, twitter, www, github].join('
'); + + if (dataConfigContact) { + dataConfigContact.innerHTML = html2; + } + } else { + for (var i = 0, p; p = presenters[i]; ++i) { + html.push(p.name + ' - ' + p.company); + } + html = html.join('
'); + if (dataConfigContact) { + dataConfigContact.innerHTML = html; + } + } + + var dataConfigPresenter = document.querySelector('[data-config-presenter]'); + if (dataConfigPresenter) { + dataConfigPresenter.innerHTML = html; + if (settings.eventInfo) { + var date = settings.eventInfo.date; + var dateInfo = date ? ' - ' : ''; + dataConfigPresenter.innerHTML += settings.eventInfo.title + dateInfo; + } + } + } + + /* Left/Right tap areas. Default to including. */ + if (!!!('enableSlideAreas' in settings) || settings.enableSlideAreas) { + var el = document.createElement('div'); + el.classList.add('slide-area'); + el.id = 'prev-slide-area'; + el.addEventListener('click', this.prevSlide.bind(this), false); + this.container.appendChild(el); + + var el = document.createElement('div'); + el.classList.add('slide-area'); + el.id = 'next-slide-area'; + el.addEventListener('click', this.nextSlide.bind(this), false); + this.container.appendChild(el); + } + + if (Modernizr.touch && (!!!('enableTouch' in settings) || + settings.enableTouch)) { + var self = this; + + // Note: this prevents mobile zoom in/out but prevents iOS from doing + // it's crazy scroll over effect and disaligning the slides. + window.addEventListener('touchstart', function(e) { + e.preventDefault(); + }, false); + + var hammer = new Hammer(this.container); + hammer.ondragend = function(e) { + if (e.direction == 'right' || e.direction == 'down') { + self.prevSlide(); + } else if (e.direction == 'left' || e.direction == 'up') { + self.nextSlide(); + } + }; + } +}; + +/** + * @private + * @param {Array.} fonts + */ +SlideDeck.prototype.addFonts_ = function(fonts) { + var el = document.createElement('link'); + el.rel = 'stylesheet'; + el.href = ('https:' == document.location.protocol ? 'https' : 'http') + + '://fonts.googleapis.com/css?family=' + fonts.join('|') + '&v2'; + document.querySelector('head').appendChild(el); +}; + +/** + * @private + */ +SlideDeck.prototype.buildNextBuildItem_ = function() { + var slide = this.slides[this.curSlide_]; + var toBuild = slide.querySelector('.to-build'); + var built = slide.querySelector('.build-current'); + + if (built) { + built.classList.remove('build-current'); + if (built.classList.contains('fade')) { + built.classList.add('build-fade'); + } + } + + if (!toBuild) { + var items = slide.querySelectorAll('.build-fade'); + for (var j = 0, item; item = items[j]; j++) { + item.classList.remove('build-fade'); + } + return false; + } + + toBuild.classList.remove('to-build'); + toBuild.classList.add('build-current'); + + return true; +}; + +SlideDeck.prototype.buildNextItem_ = function() { + + var slide = this.slides[this.curSlide_]; + var built = slide.querySelectorAll('.build-current'); + + var buildItems = slide.querySelectorAll('[class*="build-item-"]'); + var show_items; + + // Remove the classes from the previously built item + if (built) { + for (var j = 0, built_item; built_item = built[j]; ++j) { + built_item.classList.remove('build-current'); + if (built_item.classList.contains('fade')) { + built_item.classList.add('build-fade'); + } + + if (built_item.getAttribute('data-build-show-only')) { + + if (built_item.getAttribute('data-build-class')) { + built_item.classList.remove( + built_item.getAttribute('data-build-class') + ); + } else { + built_item.classList.add('build-hide'); + } + } + }; + } + + if (slide._buildItems && slide._buildItems.length) { + while ((show_items = slide._buildItems.shift()) === undefined) {}; + if (show_items) { + + // show the next items + show_items.forEach(function(item, index, items) { + item.classList.remove('to-build'); + item.classList.add('build-current'); + + if (item.getAttribute('data-build-class')) { + item.classList.add(item.getAttribute('data-build-class')); + } + }); + + return true; + } + } + + return this.buildNextBuildItem_(); + +}; + +/** + * @param {boolean=} opt_dontPush + */ +SlideDeck.prototype.prevSlide = function(opt_dontPush) { + if (this.curSlide_ > 0) { + var bodyClassList = document.body.classList; + bodyClassList.remove('highlight-code'); + + // Toggle off speaker notes if they're showing when we move backwards on the + // main slides. If we're the speaker notes popup, leave them up. + if (this.controller && !this.controller.isPopup) { + bodyClassList.remove('with-notes'); + } else if (!this.controller) { + bodyClassList.remove('with-notes'); + } + + this.prevSlide_ = this.curSlide_--; + + this.updateSlides_(opt_dontPush); + } +}; + +/** + * @param {boolean=} opt_dontPush + */ +SlideDeck.prototype.nextSlide = function(opt_dontPush) { + if (!document.body.classList.contains('overview') && this.buildNextItem_()) { + return; + } + + if (this.curSlide_ < this.slides.length - 1) { + var bodyClassList = document.body.classList; + bodyClassList.remove('highlight-code'); + + // Toggle off speaker notes if they're showing when we advanced on the main + // slides. If we're the speaker notes popup, leave them up. + if (this.controller && !this.controller.isPopup) { + bodyClassList.remove('with-notes'); + } else if (!this.controller) { + bodyClassList.remove('with-notes'); + } + + this.prevSlide_ = this.curSlide_++; + + this.updateSlides_(opt_dontPush); + } +}; + +/* Slide events */ + +/** + * Triggered when a slide enter/leave event should be dispatched. + * + * @param {string} type The type of event to trigger + * (e.g. 'slideenter', 'slideleave'). + * @param {number} slideNo The index of the slide that is being left. + */ +SlideDeck.prototype.triggerSlideEvent = function(type, slideNo) { + var el = this.getSlideEl_(slideNo); + if (!el) { + return; + } + + // Call onslideenter/onslideleave if the attribute is defined on this slide. + var func = el.getAttribute(type); + if (func) { + new Function(func).call(el); // TODO: Don't use new Function() :( + } + + // Dispatch event to listeners setup using addEventListener. + var evt = document.createEvent('Event'); + evt.initEvent(type, true, true); + evt.slideNumber = slideNo + 1; // Make it readable + evt.slide = el; + + el.dispatchEvent(evt); +}; + +/** + * @private + */ +SlideDeck.prototype.updateSlides_ = function(opt_dontPush) { + var dontPush = opt_dontPush || false; + + var curSlide = this.curSlide_; + for (var i = 0; i < this.slides.length; ++i) { + switch (i) { + case curSlide - 2: + this.updateSlideClass_(i, 'far-past'); + break; + case curSlide - 1: + this.updateSlideClass_(i, 'past'); + break; + case curSlide: + this.updateSlideClass_(i, 'current'); + break; + case curSlide + 1: + this.updateSlideClass_(i, 'next'); + break; + case curSlide + 2: + this.updateSlideClass_(i, 'far-next'); + break; + default: + this.updateSlideClass_(i); + break; + } + }; + + this.triggerSlideEvent('slideleave', this.prevSlide_); + this.triggerSlideEvent('slideenter', curSlide); + +// window.setTimeout(this.disableSlideFrames_.bind(this, curSlide - 2), 301); +// +// this.enableSlideFrames_(curSlide - 1); // Previous slide. +// this.enableSlideFrames_(curSlide + 1); // Current slide. +// this.enableSlideFrames_(curSlide + 2); // Next slide. + + // Enable current slide's iframes (needed for page loat at current slide). + this.enableSlideFrames_(curSlide + 1); + + // No way to tell when all slide transitions + auto builds are done. + // Give ourselves a good buffer to preload the next slide's iframes. + window.setTimeout(this.enableSlideFrames_.bind(this, curSlide + 2), 1000); + + this.updateHash_(dontPush); + + if (document.body.classList.contains('overview')) { + this.focusOverview_(); + return; + } + +}; + +/** + * @private + * @param {number} slideNo + */ +SlideDeck.prototype.enableSlideFrames_ = function(slideNo) { + var el = this.slides[slideNo - 1]; + if (!el) { + return; + } + + var frames = el.querySelectorAll('iframe'); + for (var i = 0, frame; frame = frames[i]; i++) { + this.enableFrame_(frame); + } +}; + +/** + * @private + * @param {number} slideNo + */ +SlideDeck.prototype.enableFrame_ = function(frame) { + var src = frame.dataset.src; + if (src && frame.src != src) { + frame.src = src; + } +}; + +/** + * @private + * @param {number} slideNo + */ +SlideDeck.prototype.disableSlideFrames_ = function(slideNo) { + var el = this.slides[slideNo - 1]; + if (!el) { + return; + } + + var frames = el.querySelectorAll('iframe'); + for (var i = 0, frame; frame = frames[i]; i++) { + this.disableFrame_(frame); + } +}; + +/** + * @private + * @param {Node} frame + */ +SlideDeck.prototype.disableFrame_ = function(frame) { + frame.src = 'about:blank'; +}; + +/** + * @private + * @param {number} slideNo + */ +SlideDeck.prototype.getSlideEl_ = function(no) { + if ((no < 0) || (no >= this.slides.length)) { + return null; + } else { + return this.slides[no]; + } +}; + +/** + * @private + * @param {number} slideNo + * @param {string} className + */ +SlideDeck.prototype.updateSlideClass_ = function(slideNo, className) { + var el = this.getSlideEl_(slideNo); + + if (!el) { + return; + } + + if (className) { + el.classList.add(className); + } + + for (var i = 0, slideClass; slideClass = this.SLIDE_CLASSES_[i]; ++i) { + if (className != slideClass) { + el.classList.remove(slideClass); + } + } +}; + +/** + * @private + */ +SlideDeck.prototype.BUILD_ITEM_RE = /build-item-(\d+)(-class-(\w+))?(-only)?/; + +SlideDeck.prototype.makeBuildLists_ = function () { + for (var i = this.curSlide_, slide; slide = this.slides[i]; ++i) { + var items = slide.querySelectorAll('.build > *'); + + for (var j = 0, item; item = items[j]; ++j) { + if (item.classList) { + item.classList.add('to-build'); + if (item.parentNode.classList.contains('fade')) { + item.classList.add('fade'); + } + } + } + + var items = slide.querySelectorAll('[class*="build-item-"]'); + if (items.length) { + slide._buildItems = []; + }; + for (var j = 0, item; item = items[j]; ++j) { + if (item.classList) { + item.classList.add('to-build'); + if (!item.parentNode.classList.contains('build')) { + item.parentNode.classList.add('build'); + } + if (item.parentNode.classList.contains('fade')) { + item.classList.add('fade'); + } + } + + var build_info = this.BUILD_ITEM_RE.exec(item.classList), + build_index = build_info[1], + build_class = build_info[3], + build_only = build_info[4]; + + if (slide._buildItems[build_index] === undefined) { + slide._buildItems[build_index] = []; + } + slide._buildItems[build_index].push(item); + + if (build_class) { + item.setAttribute('data-build-class', build_class); + } + + if (build_only) { + // add the data-attribute + item.setAttribute('data-build-show-only', build_index); + } + + } + + } +}; + +/** + * @private + * @param {boolean} dontPush + */ +SlideDeck.prototype.updateHash_ = function(dontPush) { + if (!dontPush) { + var slideNo = this.curSlide_ + 1; + var hash = '#' + slideNo; + if (window.history.pushState) { + window.history.pushState(this.curSlide_, 'Slide ' + slideNo, hash); + } else { + window.location.replace(hash); + } + + // Record GA hit on this slide. + window['_gaq'] && window['_gaq'].push(['_trackPageview', + document.location.href]); + } +}; + + +/** + * @private + * @param {string} favIcon + */ +SlideDeck.prototype.addFavIcon_ = function(favIcon) { + var el = document.createElement('link'); + el.rel = 'icon'; + el.type = 'image/png'; + el.href = favIcon; + document.querySelector('head').appendChild(el); +}; + +/** + * @private + * @param {string} theme + */ +SlideDeck.prototype.loadTheme_ = function(theme) { + var styles = []; + if (theme.constructor.name === 'String') { + styles.push(theme); + } else { + styles = theme; + } + + for (var i = 0, style; themeUrl = styles[i]; i++) { + var style = document.createElement('link'); + style.rel = 'stylesheet'; + style.type = 'text/css'; + if (themeUrl.indexOf('http') == -1) { + style.href = this.CSS_DIR_ + themeUrl + '.css'; + } else { + style.href = themeUrl; + } + document.querySelector('head').appendChild(style); + } +}; + +/** + * @private + */ +SlideDeck.prototype.loadAnalytics_ = function() { + var _gaq = window['_gaq'] || []; + _gaq.push(['_setAccount', this.config_.settings.analytics]); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); +}; diff --git a/source/_themes/uwpce_slides2/static/js/slide-testing.js b/source/_themes/uwpce_slides2/static/js/slide-testing.js new file mode 100644 index 00000000..def9cb1b --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/slide-testing.js @@ -0,0 +1,6 @@ +require(['order!modernizr.custom.45394', + 'order!prettify/prettify', 'order!hammer', 'order!slide-controller', + 'order!slide-deck', + 'order!slide-deck-instantiate'], function(someModule) { + +}); diff --git a/source/_themes/uwpce_slides2/static/js/slides.js b/source/_themes/uwpce_slides2/static/js/slides.js new file mode 100644 index 00000000..ba5a3699 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/js/slides.js @@ -0,0 +1,6 @@ +require(['order!../slide_config', 'order!modernizr.custom.45394', + 'order!prettify/prettify', 'order!hammer', 'order!slide-controller', + 'order!slide-deck', + 'order!slide-deck-instantiate'], function(someModule) { + +}); diff --git a/source/_themes/uwpce_slides2/static/scripts/md/README.md b/source/_themes/uwpce_slides2/static/scripts/md/README.md new file mode 100644 index 00000000..3188b3fa --- /dev/null +++ b/source/_themes/uwpce_slides2/static/scripts/md/README.md @@ -0,0 +1,5 @@ +### Want to use markdown to write your slides? + +`python render.py` can do that for you. + +Dependencies: jinja2, markdown. diff --git a/source/_themes/uwpce_slides2/static/scripts/md/base.html b/source/_themes/uwpce_slides2/static/scripts/md/base.html new file mode 100644 index 00000000..acc79811 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/scripts/md/base.html @@ -0,0 +1,104 @@ + + + + + Google IO 2012 + + + + + + + + + + + + + + + + + + + + + +
+

+

+

+
+
+ +{% for slide in slides %} + + {% if 'segue' in slide.class %} + +
+

{{- slide.title -}}

+

{{- slide.subtitle -}}

+
+ {% else %} +
+

{{- slide.title -}}

+

{{- slide.subtitle -}}

+
+
+ {{- slide.content -}} +
+ {% endif %} +
+{% endfor %} + + + +
+

<Thank You!>

+

Important contact information goes here.

+
+

+ +

+
+ + + + + +
+ + + + + + diff --git a/source/_themes/uwpce_slides2/static/scripts/md/render.py b/source/_themes/uwpce_slides2/static/scripts/md/render.py new file mode 100755 index 00000000..a035b90a --- /dev/null +++ b/source/_themes/uwpce_slides2/static/scripts/md/render.py @@ -0,0 +1,57 @@ +#!/usr/bin/env python + +import codecs +import re +import jinja2 +import markdown + +def process_slides(): + with codecs.open('../../presentation-output.html', 'w', encoding='utf8') as outfile: + md = codecs.open('slides.md', encoding='utf8').read() + md_slides = md.split('\n---\n') + print 'Compiled %s slides.' % len(md_slides) + + slides = [] + # Process each slide separately. + for md_slide in md_slides: + slide = {} + sections = md_slide.split('\n\n') + # Extract metadata at the beginning of the slide (look for key: value) + # pairs. + metadata_section = sections[0] + metadata = parse_metadata(metadata_section) + slide.update(metadata) + remainder_index = metadata and 1 or 0 + # Get the content from the rest of the slide. + content_section = '\n\n'.join(sections[remainder_index:]) + html = markdown.markdown(content_section) + slide['content'] = postprocess_html(html, metadata) + + slides.append(slide) + + template = jinja2.Template(open('base.html').read()) + + outfile.write(template.render(locals())) + +def parse_metadata(section): + """Given the first part of a slide, returns metadata associated with it.""" + metadata = {} + metadata_lines = section.split('\n') + for line in metadata_lines: + colon_index = line.find(':') + if colon_index != -1: + key = line[:colon_index].strip() + val = line[colon_index + 1:].strip() + metadata[key] = val + + return metadata + +def postprocess_html(html, metadata): + """Returns processed HTML to fit into the slide template format.""" + if metadata.get('build_lists') and metadata['build_lists'] == 'true': + html = html.replace('
    ', '
      ') + html = html.replace('
        ', '
          ') + return html + +if __name__ == '__main__': + process_slides() diff --git a/source/_themes/uwpce_slides2/static/scripts/md/slides.md b/source/_themes/uwpce_slides2/static/scripts/md/slides.md new file mode 100644 index 00000000..f8155aca --- /dev/null +++ b/source/_themes/uwpce_slides2/static/scripts/md/slides.md @@ -0,0 +1,78 @@ +title: Slide Title +subtitle: Subtitle +class: image + +![Mobile vs desktop users](image.png) + +--- + +title: Segue Slide +subtitle: Subtitle +class: segue dark nobackground + +--- + +title: Agenda +class: big +build_lists: true + +Things we'll cover (list should build): + +- Bullet1 +- Bullet2 +- Bullet3 + +--- + +title: Today +class: nobackground fill + +![Many kinds of devices.](image.png) + +
          source: place source info here
          + +--- + +title: Big Title Slide +class: title-slide + +--- + +title: Code Example + +Media Queries are sweet: + +
          +@media screen and (max-width: 640px) {
          +  #sidebar { display: none; }
          +}
          +
          + +--- + +title: Once more, with JavaScript + +
          +function isSmall() {
          +  return window.matchMedia("(min-device-width: ???)").matches;
          +}
          +
          +function hasTouch() {
          +  return Modernizr.touch;
          +}
          +
          +function detectFormFactor() {
          +  var device = DESKTOP;
          +  if (hasTouch()) {
          +    device = isSmall() ? PHONE : TABLET;
          +  }
          +  return device;
          +}
          +
          + +--- + +title: Centered content +content_class: flexbox vcenter + +This content should be centered! diff --git a/source/_themes/uwpce_slides2/static/slide_config.js b/source/_themes/uwpce_slides2/static/slide_config.js new file mode 100644 index 00000000..0d9b7c6f --- /dev/null +++ b/source/_themes/uwpce_slides2/static/slide_config.js @@ -0,0 +1,40 @@ +var SLIDE_CONFIG = { + // Slide settings + settings: { + title: 'Title Goes Here
          Up To Two Lines', + subtitle: 'Subtitle Goes Here', + //eventInfo: { + // title: 'Google I/O', + // date: '6/x/2013' + //}, + useBuilds: true, // Default: true. False will turn off slide animation builds. + usePrettify: true, // Default: true + enableSlideAreas: true, // Default: true. False turns off the click areas on either slide of the slides. + enableTouch: true, // Default: true. If touch support should enabled. Note: the device must support touch. + //analytics: 'UA-XXXXXXXX-1', // TODO: Using this breaks GA for some reason (probably requirejs). Update your tracking code in template.html instead. + favIcon: 'images/google_developers_logo_tiny.png', + fonts: [ + 'Open Sans:regular,semibold,italic,italicsemibold', + 'Source Code Pro' + ], + //theme: ['mytheme'], // Add your own custom themes or styles in /theme/css. Leave off the .css extension. + }, + + // Author information + presenters: [{ + name: 'Firstname Lastname', + company: 'Job Title
          Google', + gplus: 'http://plus.google.com/1234567890', + twitter: '@yourhandle', + www: 'http://www.you.com', + github: 'http://github.com/you' + }/*, { + name: 'Second Name', + company: 'Job Title, Google', + gplus: 'http://plus.google.com/1234567890', + twitter: '@yourhandle', + www: 'http://www.you.com', + github: 'http://github.com/you' + }*/] +}; + diff --git a/source/_themes/uwpce_slides2/static/slide_config.js_t b/source/_themes/uwpce_slides2/static/slide_config.js_t new file mode 100644 index 00000000..62339175 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/slide_config.js_t @@ -0,0 +1,27 @@ +var SLIDE_CONFIG = { + // Slide settings + settings: { + title: '{{ docstitle|e }}', + subtitle: '{{ theme_subtitle|e }}', + //eventInfo: { + // title: 'Google I/O', + // date: '6/x/2013' + //}, + useBuilds: {{ theme_use_builds }}, // Default: true. False will turn off slide animation builds. + usePrettify: {{ theme_use_prettify }}, // Default: true + enableSlideAreas: {{ theme_enable_slide_areas }}, // Default: true. False turns off the click areas on either slide of the slides. + enableTouch: {{ theme_enable_touch }}, // Default: true. If touch support should enabled. Note: the device must support touch. + //analytics: 'UA-XXXXXXXX-1', // TODO: Using this breaks GA for some reason (probably requirejs). Update your tracking code in template.html instead. + favIcon: {{ theme_favicon }}, + fonts: [ + 'Open Sans:regular,semibold,italic,italicsemibold', + 'Source Code Pro' + ], + //theme: ['mytheme'], // Add your own custom themes or styles in /theme/css. Leave off the .css extension. + }, + + // Author information + presenters: {% if theme_presenters %}{{ theme_presenters|json }} + {% else %}[] + {% endif %} +}; diff --git a/source/_themes/uwpce_slides2/static/template.html b/source/_themes/uwpce_slides2/static/template.html new file mode 100644 index 00000000..b4e7d33b --- /dev/null +++ b/source/_themes/uwpce_slides2/static/template.html @@ -0,0 +1,416 @@ + + + + + + + + + + + + + + + + + + + + + + +
          + +
          +
          + + + + +
          +

          +

          +

          +
          +
          + + +
          +

          Slide with Bullets

          +
          +
          +
            +
          • Titles are formatted as Open Sans with bold applied and font size is set at 45
          • +
          • Title capitalization is title case +
              +
            • Subtitle capitalization is title case
            • +
            +
          • +
          • Subtitle capitalization is title case
          • +
          • Titles and subtitles should never have a period at the end
          • +
          +
          +
          + + +
          +

          Slide with Bullets that Build

          +

          Subtitle Placeholder

          +
          +
          +

          A list where items build:

          +
            +
          • Pressing 'h' highlights code snippets
          • +
          • Pressing 'p' toggles speaker notes (if they're on the current slide)
          • +
          • Pressing 'f' toggles fullscreen viewing
          • +
          • Pressing 'w' toggles widescreen
          • +
          • Pressing 'o' toggles overview mode
          • +
          • Pressing 'ESC' toggles off these goodies
          • +
          +

          Another list, but items fade as they build:

          +
            +
          • Hover over me!
          • +
          • Hover over me!
          • +
          • Hover over me!
          • +
          +
          +
          + + +
          +

          Slide with (Smaller Font)

          +
          +
          +
            +
          • All links open in new tabs.
          • +
          • To change that this, add target="_self" to the link.
          • +
          +
          +
          + + + + +
          +

          Code Slide (with Subtitle Placeholder)

          +

          Subtitle Placeholder

          +
          +
          +

          Press 'h' to highlight important sections of code (wrapped in <b>).

          +
          +<script type='text/javascript'>
          +  // Say hello world until the user starts questioning
          +  // the meaningfulness of their existence.
          +  function helloWorld(world) {
          +    for (var i = 42; --i >= 0;) {
          +      alert('Hello ' + String(world));
          +    }
          +  }
          +</script>
          +
          +
          +
          + + +
          +

          Code Slide (Smaller Font)

          +
          +
          +
          +// Say hello world until the user starts questioning
          +// the meaningfulness of their existence.
          +function helloWorld(world) {
          +  for (var i = 42; --i >= 0;) {
          +    alert('Hello ' + String(world));
          +  }
          +}
          +
          +
          +<style>
          +  p { color: pink }
          +  b { color: blue }
          +</style>
          +
          +
          +<!DOCTYPE html>
          +<html>
          +<head>
          +  <title>My Awesome Page</title>
          +</head>
          +<body>
          +  <p>Hello world</p>
          +<body>
          +</html>
          +
          +
          +
          + + + +
          +

          Slide with Speaker Notes

          +
          +
          +

          Press 'p' to toggle speaker notes.

          +
          +
          + + + +
          +

          Presenter Mode

          +
          +
          +

          Add ?presentme=true to the URL to enabled presenter mode. + This setting is sticky, meaning refreshing the page will persist presenter + mode.

          +

          Hit ?presentme=false to disable presenter mode.

          +
          +
          + + +
          +

          Slide with Image

          +
          +
          + Description +
          source: place source info here
          +
          +
          + + +
          +

          Slide with Image (Centered horz/vert)

          +
          +
          + Description +
          source: place source info here
          +
          +
          + + +
          +

          Table Option A

          +

          Subtitle Placeholder

          +
          +
          + + + + + + + + + + + + + + + + + + + +
          Column 1Column 2Column 3Column 4
          Row 1placeholderplaceholderplaceholderplaceholder
          Row 2placeholderplaceholderplaceholderplaceholder
          Row 3placeholderplaceholderplaceholderplaceholder
          Row 4placeholderplaceholderplaceholderplaceholder
          Row 5placeholderplaceholderplaceholderplaceholder
          +
          +
          + + +
          +

          Table Option A (Smaller Text)

          +

          Subtitle Placeholder

          +
          +
          + + + + + + + + + + + + + + + + + + + +
          Column 1Column 2Column 3Column 4
          Row 1placeholderplaceholderplaceholderplaceholder
          Row 2placeholderplaceholderplaceholderplaceholder
          Row 3placeholderplaceholderplaceholderplaceholder
          Row 4placeholderplaceholderplaceholderplaceholder
          Row 5placeholderplaceholderplaceholderplaceholder
          +
          +
          + + +
          +

          Table Option B

          +

          Subtitle Placeholder

          +
          +
          + + + + + + + + + + + + + + + + +
          Header 1placeholderplaceholderplaceholder
          Header 2placeholderplaceholderplaceholder
          Header 3placeholderplaceholderplaceholder
          Header 4placeholderplaceholderplaceholder
          Header 5placeholderplaceholderplaceholder
          +
          +
          + + +
          +

          Slide Styles

          +
          +
          +
          +
            +
          • class="red"
          • +
          • class="red2"
          • +
          • class="red3"
          • +
          • class="blue"
          • +
          • class="blue2"
          • +
          • class="blue3"
          • +
          • class="green"
          • +
          • class="green2"
          • +
          +
            +
          • class="green3"
          • +
          • class="yellow"
          • +
          • class="yellow2"
          • +
          • class="yellow3"
          • +
          • class="gray"
          • +
          • class="gray2"
          • +
          • class="gray3"
          • +
          • class="gray4"
          • +
          +
          +
          + I am centered text with a and button. +
          +
          +
          + + + +
          +

          Segue Slide

          +

          Subtitle Placeholder

          +
          +
          + + +
          +

          Full Image (with Optional Header)

          +
          +
          www.flickr.com/photos/25797459@N06/5438799763/
          +
          + + + +
          + + This is an example of quote text. + +
          + Name
          + Company +
          +
          +
          + + +
          +

          Slide with Iframe

          +
          +
          + +
          +
          + + +
          + +
          +
          + + + +
          +

          <Thank You!>

          +

          Important contact information goes here.

          +
          +

          + +

          +
          + + +
          + +
          +
          + + + +
          + + + + + + diff --git a/source/_themes/uwpce_slides2/static/theme/css/default.css b/source/_themes/uwpce_slides2/static/theme/css/default.css new file mode 100644 index 00000000..b78086c0 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/css/default.css @@ -0,0 +1,1450 @@ +@charset "UTF-8"; +/* line 5, ../../../../../../../../../../../Library/Ruby/Gems/2.0.0/gems/compass-core-1.0.3/stylesheets/compass/reset/_utilities.scss */ +html, body, div, span, applet, object, iframe, +h1, h2, h3, h4, h5, h6, p, blockquote, pre, +a, abbr, acronym, address, big, cite, code, +del, dfn, em, img, ins, kbd, q, s, samp, +small, strike, strong, sub, sup, tt, var, +b, u, i, center, +dl, dt, dd, ol, ul, li, +fieldset, form, label, legend, +table, caption, tbody, tfoot, thead, tr, th, td, +article, aside, canvas, details, embed, +figure, figcaption, footer, header, hgroup, +menu, nav, output, ruby, section, summary, +time, mark, audio, video { + margin: 0; + padding: 0; + border: 0; + font: inherit; + font-size: 100%; + vertical-align: baseline; +} + +/* line 22, ../../../../../../../../../../../Library/Ruby/Gems/2.0.0/gems/compass-core-1.0.3/stylesheets/compass/reset/_utilities.scss */ +html { + line-height: 1; +} + +/* line 24, ../../../../../../../../../../../Library/Ruby/Gems/2.0.0/gems/compass-core-1.0.3/stylesheets/compass/reset/_utilities.scss */ +ol, ul { + list-style: none; +} + +/* line 26, ../../../../../../../../../../../Library/Ruby/Gems/2.0.0/gems/compass-core-1.0.3/stylesheets/compass/reset/_utilities.scss */ +table { + border-collapse: collapse; + border-spacing: 0; +} + +/* line 28, ../../../../../../../../../../../Library/Ruby/Gems/2.0.0/gems/compass-core-1.0.3/stylesheets/compass/reset/_utilities.scss */ +caption, th, td { + text-align: left; + font-weight: normal; + vertical-align: middle; +} + +/* line 30, ../../../../../../../../../../../Library/Ruby/Gems/2.0.0/gems/compass-core-1.0.3/stylesheets/compass/reset/_utilities.scss */ +q, blockquote { + quotes: none; +} +/* line 103, ../../../../../../../../../../../Library/Ruby/Gems/2.0.0/gems/compass-core-1.0.3/stylesheets/compass/reset/_utilities.scss */ +q:before, q:after, blockquote:before, blockquote:after { + content: ""; + content: none; +} + +/* line 32, ../../../../../../../../../../../Library/Ruby/Gems/2.0.0/gems/compass-core-1.0.3/stylesheets/compass/reset/_utilities.scss */ +a img { + border: none; +} + +/* line 116, ../../../../../../../../../../../Library/Ruby/Gems/2.0.0/gems/compass-core-1.0.3/stylesheets/compass/reset/_utilities.scss */ +article, aside, details, figcaption, figure, footer, header, hgroup, main, menu, nav, section, summary { + display: block; +} + +/** + * Base SlideDeck Styles + */ +/* line 52, ../scss/_base.scss */ +html { + height: 100%; + overflow: hidden; +} + +/* line 57, ../scss/_base.scss */ +body { + margin: 0; + padding: 0; + opacity: 0; + height: 100%; + min-height: 740px; + width: 100%; + overflow: hidden; + color: #fff; + -webkit-font-smoothing: antialiased; + -moz-font-smoothing: antialiased; + -ms-font-smoothing: antialiased; + -o-font-smoothing: antialiased; + -moz-transition: opacity 800ms ease-in 100ms; + -o-transition: opacity 800ms ease-in 100ms; + -webkit-transition: opacity 800ms ease-in; + -webkit-transition-delay: 100ms; + transition: opacity 800ms ease-in 100ms; +} +/* line 73, ../scss/_base.scss */ +body.loaded { + opacity: 1 !important; +} + +/* line 78, ../scss/_base.scss */ +input, button { + vertical-align: middle; +} + +/* line 82, ../scss/_base.scss */ +slides > slide[hidden] { + display: none !important; +} + +/* line 86, ../scss/_base.scss */ +slides { + width: 100%; + height: 100%; + position: absolute; + left: 0; + top: 0; + -moz-transform: translate3d(0, 0, 0); + -ms-transform: translate3d(0, 0, 0); + -webkit-transform: translate3d(0, 0, 0); + transform: translate3d(0, 0, 0); + -moz-perspective: 1000; + -webkit-perspective: 1000; + perspective: 1000; + -moz-transform-style: preserve-3d; + -webkit-transform-style: preserve-3d; + transform-style: preserve-3d; + -moz-transition: opacity 800ms ease-in 100ms; + -o-transition: opacity 800ms ease-in 100ms; + -webkit-transition: opacity 800ms ease-in; + -webkit-transition-delay: 100ms; + transition: opacity 800ms ease-in 100ms; +} + +/* line 98, ../scss/_base.scss */ +slides > slide { + display: block; + position: absolute; + overflow: hidden; + left: 50%; + top: 50%; + -moz-box-sizing: border-box; + -webkit-box-sizing: border-box; + box-sizing: border-box; +} + +/* Slide styles */ +/*article.fill iframe { + position: absolute; + left: 0; + top: 0; + width: 100%; + height: 100%; + + border: 0; + margin: 0; + + @include border-radius(10px); + + z-index: -1; +} + +slide.fill { + background-repeat: no-repeat; + @include background-size(cover); +} + +slide.fill img { + position: absolute; + left: 0; + top: 0; + min-width: 100%; + min-height: 100%; + + z-index: -1; +} +*/ +/** + * Theme Styles + */ +/* line 22, ../scss/default.scss */ +::selection { + color: white; + background-color: #ffd14d; + text-shadow: none; +} + +/* line 28, ../scss/default.scss */ +::-webkit-scrollbar { + height: 16px; + overflow: visible; + width: 16px; +} + +/* line 33, ../scss/default.scss */ +::-webkit-scrollbar-thumb { + background-color: rgba(0, 0, 0, 0.1); + background-clip: padding-box; + border: solid transparent; + min-height: 28px; + padding: 100px 0 0; + -moz-box-shadow: inset 1px 1px 0 rgba(0, 0, 0, 0.1), inset 0 -1px 0 rgba(0, 0, 0, 0.07); + -webkit-box-shadow: inset 1px 1px 0 rgba(0, 0, 0, 0.1), inset 0 -1px 0 rgba(0, 0, 0, 0.07); + box-shadow: inset 1px 1px 0 rgba(0, 0, 0, 0.1), inset 0 -1px 0 rgba(0, 0, 0, 0.07); + border-width: 1px 1px 1px 6px; +} + +/* line 42, ../scss/default.scss */ +::-webkit-scrollbar-thumb:hover { + background-color: rgba(0, 0, 0, 0.5); +} + +/* line 45, ../scss/default.scss */ +::-webkit-scrollbar-button { + height: 0; + width: 0; +} + +/* line 49, ../scss/default.scss */ +::-webkit-scrollbar-track { + background-clip: padding-box; + border: solid transparent; + border-width: 0 0 0 4px; +} + +/* line 54, ../scss/default.scss */ +::-webkit-scrollbar-corner { + background: transparent; +} + +/* line 58, ../scss/default.scss */ +body { + background: black; +} + +/* line 62, ../scss/default.scss */ +slides > slide { + display: none; + font-family: 'Open Sans', Arial, sans-serif; + font-size: 26px; + color: #797979; + width: 900px; + height: 700px; + margin-left: -450px; + margin-top: -350px; + padding: 40px 60px; + -moz-border-radius: 5px; + -webkit-border-radius: 5px; + border-radius: 5px; + -moz-transition: all 0.6s ease-in-out; + -o-transition: all 0.6s ease-in-out; + -webkit-transition: all 0.6s ease-in-out; + transition: all 0.6s ease-in-out; +} +/* line 83, ../scss/default.scss */ +slides > slide.far-past { + display: none; +} +/* line 90, ../scss/default.scss */ +slides > slide.past { + display: block; + opacity: 0; +} +/* line 97, ../scss/default.scss */ +slides > slide.current { + display: block; + opacity: 1; +} +/* line 103, ../scss/default.scss */ +slides > slide.current .auto-fadein { + opacity: 1; +} +/* line 107, ../scss/default.scss */ +slides > slide.current .gdbar { + -moz-background-size: 100% 100%; + -o-background-size: 100% 100%; + -webkit-background-size: 100% 100%; + background-size: 100% 100%; +} +/* line 112, ../scss/default.scss */ +slides > slide.next { + display: block; + opacity: 0; + pointer-events: none; +} +/* line 120, ../scss/default.scss */ +slides > slide.far-next { + display: none; +} +/* line 127, ../scss/default.scss */ +slides > slide.dark { + background: #515151 !important; +} +/* line 135, ../scss/default.scss */ +slides > slide:not(.nobackground):before { + font-size: 12pt; + content: ""; + position: absolute; + bottom: 20px; + left: 60px; + -moz-background-size: 30px 30px; + -o-background-size: 30px 30px; + -webkit-background-size: 30px 30px; + background-size: 30px 30px; + padding-left: 40px; + height: 30px; + line-height: 1.9; +} +/* line 147, ../scss/default.scss */ +slides > slide:not(.nobackground):after { + font-size: 12pt; + content: ""; + position: absolute; + bottom: 20px; + right: 60px; + line-height: 1.9; +} +/* line 158, ../scss/default.scss */ +slides > slide.title-slide:after { + content: ''; + position: absolute; + bottom: 40px; + right: 40px; + width: 100%; + height: 60px; +} +/* line 170, ../scss/default.scss */ +slides > slide.backdrop { + z-index: -10; + display: block !important; + background: url('data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4gPHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGRlZnM+PGxpbmVhckdyYWRpZW50IGlkPSJncmFkIiBncmFkaWVudFVuaXRzPSJvYmplY3RCb3VuZGluZ0JveCIgeDE9IjAuNSIgeTE9IjAuMCIgeDI9IjAuNSIgeTI9IjEuMCI+PHN0b3Agb2Zmc2V0PSIwJSIgc3RvcC1jb2xvcj0iI2ZmZmZmZiIvPjxzdG9wIG9mZnNldD0iODUlIiBzdG9wLWNvbG9yPSIjZmZmZmZmIi8+PHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjZTZlNmU2Ii8+PC9saW5lYXJHcmFkaWVudD48L2RlZnM+PHJlY3QgeD0iMCIgeT0iMCIgd2lkdGg9IjEwMCUiIGhlaWdodD0iMTAwJSIgZmlsbD0idXJsKCNncmFkKSIgLz48L3N2Zz4g'); + background: -webkit-gradient(linear, 50% 0%, 50% 100%, color-stop(0%, #ffffff), color-stop(85%, #ffffff), color-stop(100%, #e6e6e6)); + background: -moz-linear-gradient(#ffffff, #ffffff 85%, #e6e6e6); + background: -webkit-linear-gradient(#ffffff, #ffffff 85%, #e6e6e6); + background: linear-gradient(#ffffff, #ffffff 85%, #e6e6e6); + background-color: white; +} +/* line 175, ../scss/default.scss */ +slides > slide.backdrop:after, slides > slide.backdrop:before { + display: none; +} +/* line 180, ../scss/default.scss */ +slides > slide > hgroup + article { + margin-top: 45px; +} +/* line 184, ../scss/default.scss */ +slides > slide > hgroup + article.flexbox.vcenter, slides > slide > hgroup + article.flexbox.vleft, slides > slide > hgroup + article.flexbox.vright { + height: 80%; +} +/* line 189, ../scss/default.scss */ +slides > slide > hgroup + article p { + margin-bottom: 1em; +} +/* line 194, ../scss/default.scss */ +slides > slide > article:only-child { + height: 100%; +} +/* line 197, ../scss/default.scss */ +slides > slide > article:only-child > iframe { + height: 98%; +} + +/* line 203, ../scss/default.scss */ +slides.layout-faux-widescreen > slide { + padding: 40px 160px; +} + +/* line 212, ../scss/default.scss */ +slides.layout-widescreen > slide, +slides.layout-faux-widescreen > slide { + margin-left: -550px; + width: 1100px; +} +/* line 217, ../scss/default.scss */ +slides.layout-widescreen > slide.far-past, +slides.layout-faux-widescreen > slide.far-past { + display: block; + display: none; + -moz-transform: translate(-2260px); + -ms-transform: translate(-2260px); + -webkit-transform: translate(-2260px); + transform: translate(-2260px); + -moz-transform: translate3d(-2260px, 0, 0); + -ms-transform: translate3d(-2260px, 0, 0); + -webkit-transform: translate3d(-2260px, 0, 0); + transform: translate3d(-2260px, 0, 0); +} +/* line 224, ../scss/default.scss */ +slides.layout-widescreen > slide.past, +slides.layout-faux-widescreen > slide.past { + display: block; + opacity: 0; +} +/* line 231, ../scss/default.scss */ +slides.layout-widescreen > slide.current, +slides.layout-faux-widescreen > slide.current { + display: block; + opacity: 1; +} +/* line 238, ../scss/default.scss */ +slides.layout-widescreen > slide.next, +slides.layout-faux-widescreen > slide.next { + display: block; + opacity: 0; + pointer-events: none; +} +/* line 246, ../scss/default.scss */ +slides.layout-widescreen > slide.far-next, +slides.layout-faux-widescreen > slide.far-next { + display: block; + display: none; + -moz-transform: translate(2260px); + -ms-transform: translate(2260px); + -webkit-transform: translate(2260px); + transform: translate(2260px); + -moz-transform: translate3d(2260px, 0, 0); + -ms-transform: translate3d(2260px, 0, 0); + -webkit-transform: translate3d(2260px, 0, 0); + transform: translate3d(2260px, 0, 0); +} +/* line 253, ../scss/default.scss */ +slides.layout-widescreen #prev-slide-area, +slides.layout-faux-widescreen #prev-slide-area { + margin-left: -650px; +} +/* line 257, ../scss/default.scss */ +slides.layout-widescreen #next-slide-area, +slides.layout-faux-widescreen #next-slide-area { + margin-left: 550px; +} + +/* line 262, ../scss/default.scss */ +b { + font-weight: 600; +} + +/* line 266, ../scss/default.scss */ +a { + color: #2a7cdf; + text-decoration: none; + border-bottom: 1px solid rgba(42, 124, 223, 0.5); +} +/* line 271, ../scss/default.scss */ +a:hover { + color: black !important; +} + +/* line 276, ../scss/default.scss */ +h1, h2, h3 { + font-weight: 600; +} + +/* line 280, ../scss/default.scss */ +h2 { + font-size: 45px; + line-height: 45px; + letter-spacing: -2px; + color: #515151; +} + +/* line 287, ../scss/default.scss */ +h3 { + font-size: 30px; + letter-spacing: -1px; + line-height: 2; + font-weight: inherit; + color: #797979; +} + +/* line 295, ../scss/default.scss */ +ul { + margin-left: 1.2em; + margin-bottom: 1em; + position: relative; +} +/* line 300, ../scss/default.scss */ +ul li { + margin-bottom: 0.5em; +} +/* line 303, ../scss/default.scss */ +ul li ul { + margin-left: 2em; + margin-bottom: 0; +} +/* line 307, ../scss/default.scss */ +ul li ul li:before { + content: '-'; + font-weight: 600; +} +/* line 314, ../scss/default.scss */ +ul > li:before { + content: '\00B7'; + margin-left: -1em; + position: absolute; + font-weight: 600; +} +/* line 321, ../scss/default.scss */ +ul ul { + margin-top: .5em; +} + +/* line 328, ../scss/default.scss */ +.highlight-code slide.current pre > * { + opacity: 0.25; + -moz-transition: opacity 0.5s ease-in; + -o-transition: opacity 0.5s ease-in; + -webkit-transition: opacity 0.5s ease-in; + transition: opacity 0.5s ease-in; +} +/* line 332, ../scss/default.scss */ +.highlight-code slide.current b { + opacity: 1; +} + +/* line 337, ../scss/default.scss */ +pre { + font-family: 'Source Code Pro', 'Courier New', monospace; + font-size: 20px; + line-height: 28px; + padding: 10px 0 10px 60px; + letter-spacing: -1px; + margin-bottom: 20px; + width: 106%; + background-color: #e6e6e6; + left: -60px; + position: relative; + -moz-box-sizing: border-box; + -webkit-box-sizing: border-box; + box-sizing: border-box; + /*overflow: hidden;*/ +} +/* line 351, ../scss/default.scss */ +pre[data-lang]:after { + content: attr(data-lang); + background-color: #a9a9a9; + right: 0; + top: 0; + position: absolute; + font-size: 16pt; + color: white; + padding: 2px 25px; + text-transform: uppercase; +} + +/* line 364, ../scss/default.scss */ +pre[data-lang="go"] { + color: #333; +} + +/* line 368, ../scss/default.scss */ +code { + font-size: 95%; + font-family: 'Source Code Pro', 'Courier New', monospace; + color: black; +} + +/* line 374, ../scss/default.scss */ +iframe { + width: 100%; + height: 530px; + background: white; + border: 1px solid #e6e6e6; + -moz-box-sizing: border-box; + -webkit-box-sizing: border-box; + box-sizing: border-box; +} + +/* line 382, ../scss/default.scss */ +dt { + font-weight: bold; +} + +/* line 386, ../scss/default.scss */ +button { + display: inline-block; + background: url('data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4gPHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGRlZnM+PGxpbmVhckdyYWRpZW50IGlkPSJncmFkIiBncmFkaWVudFVuaXRzPSJvYmplY3RCb3VuZGluZ0JveCIgeDE9IjAuNSIgeTE9IjAuMCIgeDI9IjAuNSIgeTI9IjEuMCI+PHN0b3Agb2Zmc2V0PSI0MCUiIHN0b3AtY29sb3I9IiNmOWY5ZjkiLz48c3RvcCBvZmZzZXQ9IjcwJSIgc3RvcC1jb2xvcj0iI2UzZTNlMyIvPjwvbGluZWFyR3JhZGllbnQ+PC9kZWZzPjxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSIxMDAlIiBoZWlnaHQ9IjEwMCUiIGZpbGw9InVybCgjZ3JhZCkiIC8+PC9zdmc+IA=='); + background: -webkit-gradient(linear, 50% 0%, 50% 100%, color-stop(40%, #f9f9f9), color-stop(70%, #e3e3e3)); + background: -moz-linear-gradient(#f9f9f9 40%, #e3e3e3 70%); + background: -webkit-linear-gradient(#f9f9f9 40%, #e3e3e3 70%); + background: linear-gradient(#f9f9f9 40%, #e3e3e3 70%); + border: 1px solid #a9a9a9; + -moz-border-radius: 3px; + -webkit-border-radius: 3px; + border-radius: 3px; + padding: 5px 8px; + outline: none; + white-space: nowrap; + -moz-user-select: -moz-none; + -ms-user-select: none; + -webkit-user-select: none; + user-select: none; + cursor: pointer; + text-shadow: 1px 1px #fff; + font-size: 10pt; +} + +/* line 400, ../scss/default.scss */ +button:not(:disabled):hover { + border-color: #515151; +} + +/* line 404, ../scss/default.scss */ +button:not(:disabled):active { + background: url('data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4gPHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGRlZnM+PGxpbmVhckdyYWRpZW50IGlkPSJncmFkIiBncmFkaWVudFVuaXRzPSJvYmplY3RCb3VuZGluZ0JveCIgeDE9IjAuNSIgeTE9IjAuMCIgeDI9IjAuNSIgeTI9IjEuMCI+PHN0b3Agb2Zmc2V0PSI0MCUiIHN0b3AtY29sb3I9IiNlM2UzZTMiLz48c3RvcCBvZmZzZXQ9IjcwJSIgc3RvcC1jb2xvcj0iI2Y5ZjlmOSIvPjwvbGluZWFyR3JhZGllbnQ+PC9kZWZzPjxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSIxMDAlIiBoZWlnaHQ9IjEwMCUiIGZpbGw9InVybCgjZ3JhZCkiIC8+PC9zdmc+IA=='); + background: -webkit-gradient(linear, 50% 0%, 50% 100%, color-stop(40%, #e3e3e3), color-stop(70%, #f9f9f9)); + background: -moz-linear-gradient(#e3e3e3 40%, #f9f9f9 70%); + background: -webkit-linear-gradient(#e3e3e3 40%, #f9f9f9 70%); + background: linear-gradient(#e3e3e3 40%, #f9f9f9 70%); +} + +/* line 408, ../scss/default.scss */ +:disabled { + color: #a9a9a9; +} + +/* line 412, ../scss/default.scss */ +.blue { + color: #4387fd; +} + +/* line 415, ../scss/default.scss */ +.blue2 { + color: #3c8ef3; +} + +/* line 418, ../scss/default.scss */ +.blue3 { + color: #2a7cdf; +} + +/* line 421, ../scss/default.scss */ +.yellow { + color: #ffd14d; +} + +/* line 424, ../scss/default.scss */ +.yellow2 { + color: #f9cc46; +} + +/* line 427, ../scss/default.scss */ +.yellow3 { + color: #f6c000; +} + +/* line 430, ../scss/default.scss */ +.green { + color: #0da861; +} + +/* line 433, ../scss/default.scss */ +.green2 { + color: #00a86d; +} + +/* line 436, ../scss/default.scss */ +.green3 { + color: #009f5d; +} + +/* line 439, ../scss/default.scss */ +.red { + color: #f44a3f; +} + +/* line 442, ../scss/default.scss */ +.red2 { + color: #e0543e; +} + +/* line 445, ../scss/default.scss */ +.red3 { + color: #d94d3a; +} + +/* line 448, ../scss/default.scss */ +.gray { + color: #e6e6e6; +} + +/* line 451, ../scss/default.scss */ +.gray2 { + color: #a9a9a9; +} + +/* line 454, ../scss/default.scss */ +.gray3 { + color: #797979; +} + +/* line 457, ../scss/default.scss */ +.gray4 { + color: #515151; +} + +/* line 461, ../scss/default.scss */ +.white { + color: white !important; +} + +/* line 464, ../scss/default.scss */ +.black { + color: black !important; +} + +/* line 468, ../scss/default.scss */ +.columns-2 { + -moz-column-count: 2; + -webkit-column-count: 2; + column-count: 2; +} + +/* line 472, ../scss/default.scss */ +table { + width: 100%; + border-collapse: -moz-initial; + border-collapse: initial; + border-spacing: 2px; + border-bottom: 1px solid #797979; +} +/* line 479, ../scss/default.scss */ +table tr > td:first-child, table th { + font-weight: 600; + color: #515151; +} +/* line 484, ../scss/default.scss */ +table tr:nth-child(odd) { + background-color: #e6e6e6; +} +/* line 488, ../scss/default.scss */ +table th { + color: white; + font-size: 18px; + background: url('data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4gPHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGRlZnM+PGxpbmVhckdyYWRpZW50IGlkPSJncmFkIiBncmFkaWVudFVuaXRzPSJvYmplY3RCb3VuZGluZ0JveCIgeDE9IjAuNSIgeTE9IjAuMCIgeDI9IjAuNSIgeTI9IjEuMCI+PHN0b3Agb2Zmc2V0PSI0MCUiIHN0b3AtY29sb3I9IiM0Mzg3ZmQiLz48c3RvcCBvZmZzZXQ9IjgwJSIgc3RvcC1jb2xvcj0iIzJhN2NkZiIvPjwvbGluZWFyR3JhZGllbnQ+PC9kZWZzPjxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSIxMDAlIiBoZWlnaHQ9IjEwMCUiIGZpbGw9InVybCgjZ3JhZCkiIC8+PC9zdmc+IA==') no-repeat; + background: -webkit-gradient(linear, 50% 0%, 50% 100%, color-stop(40%, #4387fd), color-stop(80%, #2a7cdf)) no-repeat; + background: -moz-linear-gradient(top, #4387fd 40%, #2a7cdf 80%) no-repeat; + background: -webkit-linear-gradient(top, #4387fd 40%, #2a7cdf 80%) no-repeat; + background: linear-gradient(to bottom, #4387fd 40%, #2a7cdf 80%) no-repeat; +} +/* line 494, ../scss/default.scss */ +table td, table th { + font-size: 18px; + padding: 1em 0.5em; +} +/* line 499, ../scss/default.scss */ +table td.highlight { + color: #515151; + background: url('data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4gPHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGRlZnM+PGxpbmVhckdyYWRpZW50IGlkPSJncmFkIiBncmFkaWVudFVuaXRzPSJvYmplY3RCb3VuZGluZ0JveCIgeDE9IjAuNSIgeTE9IjAuMCIgeDI9IjAuNSIgeTI9IjEuMCI+PHN0b3Agb2Zmc2V0PSI0MCUiIHN0b3AtY29sb3I9IiNmZmQxNGQiLz48c3RvcCBvZmZzZXQ9IjgwJSIgc3RvcC1jb2xvcj0iI2Y2YzAwMCIvPjwvbGluZWFyR3JhZGllbnQ+PC9kZWZzPjxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSIxMDAlIiBoZWlnaHQ9IjEwMCUiIGZpbGw9InVybCgjZ3JhZCkiIC8+PC9zdmc+IA==') no-repeat; + background: -webkit-gradient(linear, 50% 0%, 50% 100%, color-stop(40%, #ffd14d), color-stop(80%, #f6c000)) no-repeat; + background: -moz-linear-gradient(top, #ffd14d 40%, #f6c000 80%) no-repeat; + background: -webkit-linear-gradient(top, #ffd14d 40%, #f6c000 80%) no-repeat; + background: linear-gradient(to bottom, #ffd14d 40%, #f6c000 80%) no-repeat; +} +/* line 504, ../scss/default.scss */ +table.rows { + border-bottom: none; + border-right: 1px solid #797979; +} + +/* line 510, ../scss/default.scss */ +q { + font-size: 45px; + line-height: 72px; +} +/* line 514, ../scss/default.scss */ +q:before { + content: '“'; + position: absolute; + margin-left: -0.5em; +} +/* line 519, ../scss/default.scss */ +q:after { + content: '”'; + position: absolute; + margin-left: 0.1em; +} + +/* line 526, ../scss/default.scss */ +slide.fill { + background-repeat: no-repeat; + -moz-border-radius: 5px; + -webkit-border-radius: 5px; + border-radius: 5px; + -moz-background-size: cover; + -o-background-size: cover; + -webkit-background-size: cover; + background-size: cover; +} + +/* Size variants */ +/* line 535, ../scss/default.scss */ +article.smaller p, article.smaller ul { + font-size: 20px; + line-height: 24px; + letter-spacing: 0; +} +/* line 541, ../scss/default.scss */ +article.smaller table td, article.smaller table th { + font-size: 14px; +} +/* line 545, ../scss/default.scss */ +article.smaller pre { + font-size: 15px; + line-height: 20px; + letter-spacing: 0; +} +/* line 550, ../scss/default.scss */ +article.smaller q { + font-size: 40px; + line-height: 48px; +} +/* line 554, ../scss/default.scss */ +article.smaller q:before, article.smaller q:after { + font-size: 60px; +} + +/* Builds */ +/* line 563, ../scss/default.scss */ +.build > * { + -moz-transition: opacity 0.5s ease-in-out 0.2s; + -o-transition: opacity 0.5s ease-in-out 0.2s; + -webkit-transition: opacity 0.5s ease-in-out; + -webkit-transition-delay: 0.2s; + transition: opacity 0.5s ease-in-out 0.2s; +} +/* line 567, ../scss/default.scss */ +.build .to-build { + opacity: 0; +} +/* line 571, ../scss/default.scss */ +.build .build-fade { + opacity: 0.3; +} +/* line 574, ../scss/default.scss */ +.build .build-fade:hover { + opacity: 1.0; +} + +/* line 581, ../scss/default.scss */ +.popup .next .build .to-build { + opacity: 1; +} +/* line 585, ../scss/default.scss */ +.popup .next .build .build-fade { + opacity: 1; +} + +/* Pretty print */ +/* line 592, ../scss/default.scss */ +.prettyprint .str, +.prettyprint .atv { + /* a markup attribute value */ + color: #009f5d; +} + +/* line 596, ../scss/default.scss */ +.prettyprint .kwd, +.prettyprint .tag { + /* a markup tag name */ + color: #0066cc; +} + +/* line 600, ../scss/default.scss */ +.prettyprint .com { + /* a comment */ + color: #797979; + font-style: italic; +} + +/* line 604, ../scss/default.scss */ +.prettyprint .lit { + /* a literal value */ + color: #7f0000; +} + +/* line 607, ../scss/default.scss */ +.prettyprint .pun, +.prettyprint .opn, +.prettyprint .clo { + color: #515151; +} + +/* line 612, ../scss/default.scss */ +.prettyprint .typ, +.prettyprint .atn, +.prettyprint .dec, +.prettyprint .var { + /* a declaration; a variable name */ + color: #d94d3a; +} + +/* line 618, ../scss/default.scss */ +.prettyprint .pln { + color: #515151; +} + +/* line 622, ../scss/default.scss */ +.note { + position: absolute; + z-index: 100; + width: 100%; + height: 100%; + top: 0; + left: 0; + padding: 1em; + background: rgba(0, 0, 0, 0.3); + opacity: 0; + pointer-events: none; + display: -webkit-box !important; + display: -moz-box !important; + display: -ms-box !important; + display: -o-box !important; + display: box !important; + -webkit-box-orient: vertical; + -moz-box-orient: vertical; + -ms-box-orient: vertical; + box-orient: vertical; + -webkit-box-align: center; + -moz-box-align: center; + -ms-box-align: center; + box-align: center; + -webkit-box-pack: center; + -moz-box-pack: center; + -ms-box-pack: center; + box-pack: center; + -moz-border-radius: 5px; + -webkit-border-radius: 5px; + border-radius: 5px; + -moz-box-sizing: border-box; + -webkit-box-sizing: border-box; + box-sizing: border-box; + -moz-transform: translateY(350px); + -ms-transform: translateY(350px); + -webkit-transform: translateY(350px); + transform: translateY(350px); + -moz-transition: all 0.4s ease-in-out; + -o-transition: all 0.4s ease-in-out; + -webkit-transition: all 0.4s ease-in-out; + transition: all 0.4s ease-in-out; +} +/* line 640, ../scss/default.scss */ +.note > section { + background: #fff; + -moz-border-radius: 5px; + -webkit-border-radius: 5px; + border-radius: 5px; + -moz-box-shadow: 0 0 10px #797979; + -webkit-box-shadow: 0 0 10px #797979; + box-shadow: 0 0 10px #797979; + width: 60%; + padding: 2em; +} + +/* line 657, ../scss/default.scss */ +.with-notes.popup slides.layout-widescreen slide.next, +.with-notes.popup slides.layout-faux-widescreen slide.next { + -moz-transform: translate3d(690px, 80px, 0) scale(0.35); + -ms-transform: translate3d(690px, 80px, 0) scale(0.35); + -webkit-transform: translate3d(690px, 80px, 0) scale(0.35); + transform: translate3d(690px, 80px, 0) scale(0.35); +} +/* line 660, ../scss/default.scss */ +.with-notes.popup slides.layout-widescreen slide .note, +.with-notes.popup slides.layout-faux-widescreen slide .note { + -moz-transform: translate3d(300px, 800px, 0) scale(1.5); + -ms-transform: translate3d(300px, 800px, 0) scale(1.5); + -webkit-transform: translate3d(300px, 800px, 0) scale(1.5); + transform: translate3d(300px, 800px, 0) scale(1.5); +} +/* line 666, ../scss/default.scss */ +.with-notes.popup slide { + overflow: visible; + background: white; + -moz-transition: none; + -o-transition: none; + -webkit-transition: none; + transition: none; + pointer-events: none; + -moz-transform-origin: 0 0; + -ms-transform-origin: 0 0; + -webkit-transform-origin: 0 0; + transform-origin: 0 0; +} +/* line 673, ../scss/default.scss */ +.with-notes.popup slide:not(.backdrop) { + -moz-transform: scale(0.6) translate3d(0.5em, 0.5em, 0); + -ms-transform: scale(0.6) translate3d(0.5em, 0.5em, 0); + -webkit-transform: scale(0.6) translate3d(0.5em, 0.5em, 0); + transform: scale(0.6) translate3d(0.5em, 0.5em, 0); + -moz-box-shadow: 0 0 10px #797979; + -webkit-box-shadow: 0 0 10px #797979; + box-shadow: 0 0 10px #797979; +} +/* line 678, ../scss/default.scss */ +.with-notes.popup slide.backdrop { + background-image: url('data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4gPHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGRlZnM+PHJhZGlhbEdyYWRpZW50IGlkPSJncmFkIiBncmFkaWVudFVuaXRzPSJ1c2VyU3BhY2VPblVzZSIgY3g9IjUwJSIgY3k9IjUwJSIgcj0iNjAwJSI+PHN0b3Agb2Zmc2V0PSIwJSIgc3RvcC1jb2xvcj0iI2IxZGZmZiIvPjxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iIzQzODdmZCIvPjwvcmFkaWFsR3JhZGllbnQ+PC9kZWZzPjxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSIxMDAlIiBoZWlnaHQ9IjEwMCUiIGZpbGw9InVybCgjZ3JhZCkiIC8+PC9zdmc+IA=='); + background-size: 100%; + background-image: -moz-radial-gradient(50% 50%, #b1dfff 0%, #4387fd 600px); + background-image: -webkit-radial-gradient(50% 50%, #b1dfff 0%, #4387fd 600px); + background-image: radial-gradient(50% 50%, #b1dfff 0%, #4387fd 600px); +} +/* line 684, ../scss/default.scss */ +.with-notes.popup slide.next { + -moz-transform: translate3d(570px, 80px, 0) scale(0.35); + -ms-transform: translate3d(570px, 80px, 0) scale(0.35); + -webkit-transform: translate3d(570px, 80px, 0) scale(0.35); + transform: translate3d(570px, 80px, 0) scale(0.35); + opacity: 1 !important; +} +/* line 688, ../scss/default.scss */ +.with-notes.popup slide.next .note { + display: none !important; +} +/* line 694, ../scss/default.scss */ +.with-notes.popup .note { + width: 109%; + height: 260px; + background: #e6e6e6; + padding: 0; + -moz-box-shadow: 0 0 10px #797979; + -webkit-box-shadow: 0 0 10px #797979; + box-shadow: 0 0 10px #797979; + -moz-transform: translate3d(250px, 800px, 0) scale(1.5); + -ms-transform: translate3d(250px, 800px, 0) scale(1.5); + -webkit-transform: translate3d(250px, 800px, 0) scale(1.5); + transform: translate3d(250px, 800px, 0) scale(1.5); + -moz-transition: opacity 400ms ease-in-out; + -o-transition: opacity 400ms ease-in-out; + -webkit-transition: opacity 400ms ease-in-out; + transition: opacity 400ms ease-in-out; +} +/* line 705, ../scss/default.scss */ +.with-notes.popup .note > section { + background: #fff; + -moz-border-radius: 5px; + -webkit-border-radius: 5px; + border-radius: 5px; + height: 100%; + width: 100%; + -moz-box-sizing: border-box; + -webkit-box-sizing: border-box; + box-sizing: border-box; + -moz-box-shadow: none; + -webkit-box-shadow: none; + box-shadow: none; + overflow: auto; + padding: 1em; +} +/* line 718, ../scss/default.scss */ +.with-notes .note { + opacity: 1; + -moz-transform: translateY(0); + -ms-transform: translateY(0); + -webkit-transform: translateY(0); + transform: translateY(0); + pointer-events: auto; +} + +/* line 725, ../scss/default.scss */ +.source { + font-size: 14px; + color: #a9a9a9; + position: absolute; + bottom: 70px; + left: 60px; +} + +/* line 733, ../scss/default.scss */ +.centered { + text-align: center; +} + +/* line 737, ../scss/default.scss */ +.reflect { + -webkit-box-reflect: below 3px -webkit-linear-gradient(rgba(255, 255, 255, 0) 85%, white 150%); + -moz-box-reflect: below 3px -moz-linear-gradient(rgba(255, 255, 255, 0) 85%, white 150%); + -o-box-reflect: below 3px -o-linear-gradient(rgba(255, 255, 255, 0) 85%, white 150%); + -ms-box-reflect: below 3px -ms-linear-gradient(rgba(255, 255, 255, 0) 85%, white 150%); + box-reflect: below 3px linear-gradient(rgba(255, 255, 255, 0) 85%, #ffffff 150%); +} + +/* line 745, ../scss/default.scss */ +.flexbox { + display: -webkit-box !important; + display: -moz-box !important; + display: -ms-box !important; + display: -o-box !important; + display: box !important; +} + +/* line 749, ../scss/default.scss */ +.flexbox.vcenter { + -webkit-box-orient: vertical; + -moz-box-orient: vertical; + -ms-box-orient: vertical; + box-orient: vertical; + -webkit-box-align: center; + -moz-box-align: center; + -ms-box-align: center; + box-align: center; + -webkit-box-pack: center; + -moz-box-pack: center; + -ms-box-pack: center; + box-pack: center; + height: 100%; + width: 100%; +} + +/* line 755, ../scss/default.scss */ +.flexbox.vleft { + -webkit-box-orient: vertical; + -moz-box-orient: vertical; + -ms-box-orient: vertical; + box-orient: vertical; + -webkit-box-align: left; + -moz-box-align: left; + -ms-box-align: left; + box-align: left; + -webkit-box-pack: center; + -moz-box-pack: center; + -ms-box-pack: center; + box-pack: center; + height: 100%; + width: 100%; +} + +/* line 761, ../scss/default.scss */ +.flexbox.vright { + -webkit-box-orient: vertical; + -moz-box-orient: vertical; + -ms-box-orient: vertical; + box-orient: vertical; + -webkit-box-align: end; + -moz-box-align: end; + -ms-box-align: end; + box-align: end; + -webkit-box-pack: center; + -moz-box-pack: center; + -ms-box-pack: center; + box-pack: center; + height: 100%; + width: 100%; +} + +/* line 767, ../scss/default.scss */ +.auto-fadein { + -moz-transition: opacity 0.6s ease-in 1s; + -o-transition: opacity 0.6s ease-in 1s; + -webkit-transition: opacity 0.6s ease-in; + -webkit-transition-delay: 1s; + transition: opacity 0.6s ease-in 1s; + opacity: 0; +} + +/* Clickable/tappable areas */ +/* line 773, ../scss/default.scss */ +.slide-area { + z-index: 1000; + position: absolute; + left: 0; + top: 0; + width: 100px; + height: 700px; + left: 50%; + top: 50%; + cursor: pointer; + margin-top: -350px; +} + +/* line 790, ../scss/default.scss */ +#prev-slide-area { + margin-left: -550px; +} + +/* line 795, ../scss/default.scss */ +#next-slide-area { + margin-left: 450px; +} + +/* ===== SLIDE CONTENT ===== */ +/* line 803, ../scss/default.scss */ +.logoslide img { + width: 383px; + height: 92px; +} + +/* line 809, ../scss/default.scss */ +.segue { + padding: 60px 120px; +} +/* line 812, ../scss/default.scss */ +.segue h2 { + color: #e6e6e6; + font-size: 60px; +} +/* line 816, ../scss/default.scss */ +.segue h3 { + color: #e6e6e6; + line-height: 2.8; +} +/* line 820, ../scss/default.scss */ +.segue hgroup { + position: absolute; + bottom: 225px; +} + +/* line 826, ../scss/default.scss */ +.thank-you-slide { + background: #4387fd !important; + color: white; +} +/* line 830, ../scss/default.scss */ +.thank-you-slide h2 { + font-size: 60px; + color: inherit; +} +/* line 835, ../scss/default.scss */ +.thank-you-slide article > p { + margin-top: 2em; + font-size: 20pt; +} +/* line 840, ../scss/default.scss */ +.thank-you-slide > p { + position: absolute; + bottom: 80px; + font-size: 24pt; + line-height: 1.3; +} + +/* line 848, ../scss/default.scss */ +aside.gdbar { + height: 97px; + width: 215px; + position: absolute; + left: -1px; + top: 125px; + -moz-border-radius: 0 10px 10px 0; + -webkit-border-radius: 0; + border-radius: 0 10px 10px 0; + background: url('data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4gPHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGRlZnM+PGxpbmVhckdyYWRpZW50IGlkPSJncmFkIiBncmFkaWVudFVuaXRzPSJvYmplY3RCb3VuZGluZ0JveCIgeDE9IjAuMCIgeTE9IjAuNSIgeDI9IjEuMCIgeTI9IjAuNSI+PHN0b3Agb2Zmc2V0PSIwJSIgc3RvcC1jb2xvcj0iI2U2ZTZlNiIvPjxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI2U2ZTZlNiIvPjwvbGluZWFyR3JhZGllbnQ+PC9kZWZzPjxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSIxMDAlIiBoZWlnaHQ9IjEwMCUiIGZpbGw9InVybCgjZ3JhZCkiIC8+PC9zdmc+IA==') no-repeat; + background: -webkit-gradient(linear, 0% 50%, 100% 50%, color-stop(0%, #e6e6e6), color-stop(100%, #e6e6e6)) no-repeat; + background: -moz-linear-gradient(left, #e6e6e6, #e6e6e6) no-repeat; + background: -webkit-linear-gradient(left, #e6e6e6, #e6e6e6) no-repeat; + background: linear-gradient(to right, #e6e6e6, #e6e6e6) no-repeat; + -moz-background-size: 0% 100%; + -o-background-size: 0% 100%; + -webkit-background-size: 0% 100%; + background-size: 0% 100%; + -moz-transition: all 0.5s ease-out 0.5s; + -o-transition: all 0.5s ease-out 0.5s; + -webkit-transition: all 0.5s ease-out; + -webkit-transition-delay: 0.5s; + transition: all 0.5s ease-out 0.5s; + /* Better to transition only on background-size, but not sure how to do that with the mixin. */ +} +/* line 859, ../scss/default.scss */ +aside.gdbar.right { + right: 0; + left: -moz-initial; + left: initial; + top: 254px; + /* 96 is height of gray icon bar */ + -moz-transform: rotateZ(180deg); + -ms-transform: rotateZ(180deg); + -webkit-transform: rotateZ(180deg); + transform: rotateZ(180deg); +} +/* line 866, ../scss/default.scss */ +aside.gdbar.right img { + -moz-transform: rotateZ(180deg); + -ms-transform: rotateZ(180deg); + -webkit-transform: rotateZ(180deg); + transform: rotateZ(180deg); +} +/* line 871, ../scss/default.scss */ +aside.gdbar.bottom { + top: -moz-initial; + top: initial; + bottom: 60px; +} +/* line 877, ../scss/default.scss */ +aside.gdbar img { + width: 85px; + height: 85px; + position: absolute; + right: 0; + margin: 8px 15px; +} + +/* line 888, ../scss/default.scss */ +.title-slide hgroup { + bottom: 100px; +} +/* line 891, ../scss/default.scss */ +.title-slide hgroup h1 { + font-size: 65px; + line-height: 1.4; + letter-spacing: -3px; + color: #515151; +} +/* line 898, ../scss/default.scss */ +.title-slide hgroup h2 { + font-size: 34px; + color: #a9a9a9; + font-weight: inherit; +} +/* line 904, ../scss/default.scss */ +.title-slide hgroup p { + font-size: 20px; + color: #797979; + line-height: 1.3; + margin-top: 2em; +} + +/* line 913, ../scss/default.scss */ +.quote { + color: #e6e6e6; +} +/* line 916, ../scss/default.scss */ +.quote .author { + font-size: 24px; + position: absolute; + bottom: 80px; + line-height: 1.4; +} + +/* line 925, ../scss/default.scss */ +[data-config-contact] a { + color: white; + border-bottom: none; +} +/* line 929, ../scss/default.scss */ +[data-config-contact] span { + width: 115px; + display: inline-block; +} + +/* line 938, ../scss/default.scss */ +.overview.popup .note { + display: none !important; +} +/* line 944, ../scss/default.scss */ +.overview slides slide { + display: block; + cursor: pointer; + opacity: 0.5; + pointer-events: auto !important; + background: url('data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4gPHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGRlZnM+PGxpbmVhckdyYWRpZW50IGlkPSJncmFkIiBncmFkaWVudFVuaXRzPSJvYmplY3RCb3VuZGluZ0JveCIgeDE9IjAuNSIgeTE9IjAuMCIgeDI9IjAuNSIgeTI9IjEuMCI+PHN0b3Agb2Zmc2V0PSIwJSIgc3RvcC1jb2xvcj0iI2ZmZmZmZiIvPjxzdG9wIG9mZnNldD0iODUlIiBzdG9wLWNvbG9yPSIjZmZmZmZmIi8+PHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjZTZlNmU2Ii8+PC9saW5lYXJHcmFkaWVudD48L2RlZnM+PHJlY3QgeD0iMCIgeT0iMCIgd2lkdGg9IjEwMCUiIGhlaWdodD0iMTAwJSIgZmlsbD0idXJsKCNncmFkKSIgLz48L3N2Zz4g'); + background: -webkit-gradient(linear, 50% 0%, 50% 100%, color-stop(0%, #ffffff), color-stop(85%, #ffffff), color-stop(100%, #e6e6e6)); + background: -moz-linear-gradient(#ffffff, #ffffff 85%, #e6e6e6); + background: -webkit-linear-gradient(#ffffff, #ffffff 85%, #e6e6e6); + background: linear-gradient(#ffffff, #ffffff 85%, #e6e6e6); + background-color: white; +} +/* line 945, ../scss/default.scss */ +.overview slides slide.backdrop { + display: none !important; +} +/* line 956, ../scss/default.scss */ +.overview slides slide.far-past, .overview slides slide.past, .overview slides slide.next, .overview slides slide.far-next, .overview slides slide.far-past { + opacity: 0.5; + display: block; +} +/* line 965, ../scss/default.scss */ +.overview slides slide.current { + opacity: 1; +} +/* line 971, ../scss/default.scss */ +.overview .slide-area { + display: none; +} + +@media print { + /* line 978, ../scss/default.scss */ + slides slide { + display: block !important; + position: relative; + background: url('data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4gPHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PGRlZnM+PGxpbmVhckdyYWRpZW50IGlkPSJncmFkIiBncmFkaWVudFVuaXRzPSJvYmplY3RCb3VuZGluZ0JveCIgeDE9IjAuNSIgeTE9IjAuMCIgeDI9IjAuNSIgeTI9IjEuMCI+PHN0b3Agb2Zmc2V0PSIwJSIgc3RvcC1jb2xvcj0iI2ZmZmZmZiIvPjxzdG9wIG9mZnNldD0iODUlIiBzdG9wLWNvbG9yPSIjZmZmZmZmIi8+PHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjZTZlNmU2Ii8+PC9saW5lYXJHcmFkaWVudD48L2RlZnM+PHJlY3QgeD0iMCIgeT0iMCIgd2lkdGg9IjEwMCUiIGhlaWdodD0iMTAwJSIgZmlsbD0idXJsKCNncmFkKSIgLz48L3N2Zz4g'); + background: -webkit-gradient(linear, 50% 0%, 50% 100%, color-stop(0%, #ffffff), color-stop(85%, #ffffff), color-stop(100%, #e6e6e6)); + background: -moz-linear-gradient(#ffffff, #ffffff 85%, #e6e6e6); + background: -webkit-linear-gradient(#ffffff, #ffffff 85%, #e6e6e6); + background: linear-gradient(#ffffff, #ffffff 85%, #e6e6e6); + background-color: white; + -moz-transform: none !important; + -ms-transform: none !important; + -webkit-transform: none !important; + transform: none !important; + width: 100%; + height: 100%; + page-break-after: always; + top: auto !important; + left: auto !important; + margin-top: 0 !important; + margin-left: 0 !important; + opacity: 1 !important; + color: #555; + } + /* line 993, ../scss/default.scss */ + slides slide.far-past, slides slide.past, slides slide.next, slides slide.far-next, slides slide.far-past, slides slide.current { + opacity: 1 !important; + display: block !important; + } + /* line 1004, ../scss/default.scss */ + slides slide .build > * { + -moz-transition: none; + -o-transition: none; + -webkit-transition: none; + transition: none; + } + /* line 1008, ../scss/default.scss */ + slides slide .build .to-build, + slides slide .build .build-fade { + opacity: 1; + } + /* line 1014, ../scss/default.scss */ + slides slide .auto-fadein { + opacity: 1 !important; + } + /* line 1018, ../scss/default.scss */ + slides slide.backdrop { + display: none !important; + } + /* line 1022, ../scss/default.scss */ + slides slide table.rows { + border-right: 0; + } + /* line 1027, ../scss/default.scss */ + slides slide[hidden] { + display: none !important; + } + + /* line 1032, ../scss/default.scss */ + .slide-area { + display: none; + } + + /* line 1036, ../scss/default.scss */ + .reflect { + -webkit-box-reflect: none; + -moz-box-reflect: none; + -o-box-reflect: none; + -ms-box-reflect: none; + box-reflect: none; + } + + /* line 1044, ../scss/default.scss */ + pre, code { + font-family: monospace !important; + } +} diff --git a/source/_themes/uwpce_slides2/static/theme/css/hieroglyph.css b/source/_themes/uwpce_slides2/static/theme/css/hieroglyph.css new file mode 100644 index 00000000..a919b034 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/css/hieroglyph.css @@ -0,0 +1,84 @@ +/* line 5, ../scss/hieroglyph.scss */ +ol { + margin-left: 1.2em; + margin-bottom: 1em; + position: relative; + list-style: decimal; +} +/* line 11, ../scss/hieroglyph.scss */ +ol li { + margin-bottom: 0.5em; +} +/* line 14, ../scss/hieroglyph.scss */ +ol li ol { + margin-left: 2em; + margin-bottom: 0; + list-style: decimal; +} +/* line 19, ../scss/hieroglyph.scss */ +ol li ol li:before { + font-weight: 600; +} +/* line 25, ../scss/hieroglyph.scss */ +ol ol { + margin-top: .5em; + list-style: decimal; +} + +/* line 32, ../scss/hieroglyph.scss */ +slide.title-image { + padding-right: 0px; +} +/* line 36, ../scss/hieroglyph.scss */ +slide.title-image hgroup { + position: static !important; + margin-top: 35%; + padding-left: 30px; + background: rgba(255, 255, 255, 0.7); + border-top-left-radius: 5px; + -webkit-border-top-left-radius: 5px; + -moz-border-top-left-radius: 5px; + -o-border-top-left-radius: 5px; +} +/* line 50, ../scss/hieroglyph.scss */ +slide.title-image hgroup + article { + background: rgba(255, 255, 255, 0.7); + margin-top: 0px; + padding-left: 30px; + border-bottom-left-radius: 5px; + -webkit-border-bottom-left-radius: 5px; + -moz-border-bottom-left-radius: 5px; + -o-border-bottom-left-radius: 5px; +} +/* line 62, ../scss/hieroglyph.scss */ +slide.title-image h1 { + color: #222; + font-size: 3.2em; + line-height: 1.5em; + font-weight: 500; +} +/* line 72, ../scss/hieroglyph.scss */ +slide.title-image div.figure img { + position: absolute; + left: 0; + top: 0; + min-width: 100%; + min-height: 100%; + border-radius: 5px; + -o-border-radius: 5px; + -moz-border-radius: 5px; + -webkit-border-radius: 5px; + z-index: -1; +} +/* line 87, ../scss/hieroglyph.scss */ +slide.title-image div.figure .caption { + color: black; + background: rgba(255, 255, 255, 0.25); + padding: 0 5px; + border-bottom-left-radius: 5px; + border-top-right-radius: 5px; + position: absolute; + left: 0; + bottom: 0; + margin-bottom: 0; +} diff --git a/source/_themes/uwpce_slides2/static/theme/css/io2013.css b/source/_themes/uwpce_slides2/static/theme/css/io2013.css new file mode 100644 index 00000000..b42982b2 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/css/io2013.css @@ -0,0 +1,55 @@ +/* line 5, ../scss/io2013.scss */ +* { + line-height: 1.3; +} + +/* line 9, ../scss/io2013.scss */ +h2 { + font-weight: bold; +} + +/* line 12, ../scss/io2013.scss */ +h2, h3 { + color: #515151; +} + +/* line 16, ../scss/io2013.scss */ +q, blockquote { + font-weight: bold; +} + +/* line 20, ../scss/io2013.scss */ +slides > slide { + color: #515151; +} +/* line 24, ../scss/io2013.scss */ +slides > slide.title-slide:after { + content: ''; + background: url(../../images/io2013/google-io-lockup-1.png) no-repeat 100% 50%; + -moz-background-size: contain; + -o-background-size: contain; + -webkit-background-size: contain; + background-size: contain; + position: absolute; + bottom: 80px; + right: 40px; + width: 100%; + height: 90px; +} +/* line 36, ../scss/io2013.scss */ +slides > slide.title-slide hgroup h1 { + font-weight: bold; + line-height: 1.1; +} +/* line 40, ../scss/io2013.scss */ +slides > slide.title-slide hgroup h2, slides > slide.title-slide hgroup p { + color: #515151; +} +/* line 43, ../scss/io2013.scss */ +slides > slide.title-slide hgroup h2 { + margin-top: 0.25em; +} +/* line 46, ../scss/io2013.scss */ +slides > slide.title-slide hgroup p { + margin-top: 3em; +} diff --git a/source/_themes/uwpce_slides2/static/theme/css/phone.css b/source/_themes/uwpce_slides2/static/theme/css/phone.css new file mode 100644 index 00000000..017c7bbf --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/css/phone.css @@ -0,0 +1,26 @@ +/*Smartphones (portrait and landscape) ----------- */ +/*@media only screen +and (min-width : 320px) +and (max-width : 480px) { + +}*/ +/* Smartphones (portrait) ----------- */ +/* Styles */ +/* line 17, ../scss/phone.scss */ +slides > slide { + /* width: $slide-width !important; + height: $slide-height !important; + margin-left: -$slide-width / 2 !important; + margin-top: -$slide-height / 2 !important; + */ + -webkit-transition: none !important; + -moz-transition: none !important; + -o-transition: none !important; + -webkit-transition: none !important; + transition: none !important; +} + +/* iPhone 4 ----------- */ +@media only screen and (-webkit-min-device-pixel-ratio: 1.5), only screen and (min-device-pixel-ratio: 1.5) { + /* Styles */ +} diff --git a/source/_themes/uwpce_slides2/static/theme/scss/_base.scss b/source/_themes/uwpce_slides2/static/theme/scss/_base.scss new file mode 100644 index 00000000..50504db9 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/scss/_base.scss @@ -0,0 +1,139 @@ +@charset "UTF-8"; + +@import "compass/reset"; +@import "compass/css3/border-radius"; +@import "compass/css3/box"; +@import "compass/css3/box-shadow"; +@import "compass/css3/box-sizing"; +@import "compass/css3/images"; +@import "compass/css3/text-shadow"; +@import "compass/css3/background-size"; +@import "compass/css3/transform"; +@import "compass/css3/transition"; + +@import "variables"; + +@mixin font-smoothing($val: antialiased) { + -webkit-font-smoothing: $val; + -moz-font-smoothing: $val; + -ms-font-smoothing: $val; + -o-font-smoothing: $val; +} + +@mixin flexbox { + display: -webkit-box !important; + display: -moz-box !important; + display: -ms-box !important; + display: -o-box !important; + display: box !important; +} + +@mixin flex-center-center { + @include box-orient(vertical); + @include box-align(center); + @include box-pack(center); +} + +@mixin flex-left-center { + @include box-orient(vertical); + @include box-align(left); + @include box-pack(center); +} + +@mixin flex-right-center { + @include box-orient(vertical); + @include box-align(end); + @include box-pack(center); +} + +/** + * Base SlideDeck Styles + */ +html { + height: 100%; + overflow: hidden; +} + +body { + margin: 0; + padding: 0; + + opacity: 0; + + height: 100%; + min-height: 740px; + width: 100%; + + overflow: hidden; + + color: #fff; + @include font-smoothing(antialiased); + @include transition(opacity 800ms ease-in 100ms); // Add small delay to prevent jank. + + &.loaded { + opacity: 1 !important; + } +} + +input, button { + vertical-align: middle; +} + +slides > slide[hidden] { + display: none !important; +} + +slides { + width: 100%; + height: 100%; + position: absolute; + left: 0; + top: 0; + @include transform(translate3d(0, 0, 0)); + @include perspective(1000); + @include transform-style(preserve-3d); + @include transition(opacity 800ms ease-in 100ms); // Add small delay to prevent jank. +} + +slides > slide { + display: block; + position: absolute; + overflow: hidden; + left: 50%; + top: 50%; + @include box-sizing(border-box); +} + +/* Slide styles */ + + +/*article.fill iframe { + position: absolute; + left: 0; + top: 0; + width: 100%; + height: 100%; + + border: 0; + margin: 0; + + @include border-radius(10px); + + z-index: -1; +} + +slide.fill { + background-repeat: no-repeat; + @include background-size(cover); +} + +slide.fill img { + position: absolute; + left: 0; + top: 0; + min-width: 100%; + min-height: 100%; + + z-index: -1; +} +*/ diff --git a/source/_themes/uwpce_slides2/static/theme/scss/_variables.scss b/source/_themes/uwpce_slides2/static/theme/scss/_variables.scss new file mode 100644 index 00000000..d07f9072 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/scss/_variables.scss @@ -0,0 +1,34 @@ +$social-tags: ''; +$brand-small-icon-size: 30px; + +$gray-1: #e6e6e6; +$gray-2: #a9a9a9; +$gray-3: #797979; +$gray-4: #515151; + +$brand-blue: rgb(67, 135, 253); +$brand-blue-secondary: #3c8ef3; +$brand-blue-secondary2: #2a7cdf; + +$brand-red: rgb(244, 74, 63); +$brand-red-secondary: #e0543e; +$brand-red-secondary2: #d94d3a; + +$brand-yellow: rgb(255, 209, 77); +$brand-yellow-secondary: #f9cc46; +$brand-yellow-secondary2: #f6c000; + +$brand-green: rgb(13, 168, 97); +$brand-green-secondary: #00a86d; +$brand-green-secondary2: #009f5d; + +$slide-width: 900px; +$slide-height: 700px; +$slide-width-widescreen: 1100px; +$slide-top-bottom-padding: 40px; +$slide-left-right-padding: 60px; +$slide-border-radius: 5px; + +$slide-tap-area-width: 100px; + +$article-content-top-padding: 45px; diff --git a/source/_themes/uwpce_slides2/static/theme/scss/default.scss b/source/_themes/uwpce_slides2/static/theme/scss/default.scss new file mode 100644 index 00000000..b8c83b42 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/scss/default.scss @@ -0,0 +1,1047 @@ +@import "base"; +@import "compass/css3/columns"; +@import "compass/css3/user-interface"; + +@mixin highlight-color($color: $brand-yellow) { + -webkit-tap-highlight-color: $color; + -moz-tap-highlight-color: $color; + -ms-tap-highlight-color: $color; + -o-tap-highlight-color: $color; + tap-highlight-color: $color; +} + +@mixin backdrop { + @include background(linear-gradient(white, white 85%, $gray-1)); + background-color: white; +} + + +/** + * Theme Styles + */ +::selection { + color: white; + background-color: $brand-yellow; + @include text-shadow(none); +} + +::-webkit-scrollbar { + height: 16px; + overflow: visible; + width: 16px; +} +::-webkit-scrollbar-thumb { + background-color: rgba(0, 0, 0, .1); + background-clip: padding-box; + border: solid transparent; + min-height: 28px; + padding: 100px 0 0; + @include box-shadow(inset 1px 1px 0 rgba(0,0,0,.1),inset 0 -1px 0 rgba(0,0,0,.07)); + border-width: 1px 1px 1px 6px; +} +::-webkit-scrollbar-thumb:hover { + background-color: rgba(0, 0, 0, 0.5); +} +::-webkit-scrollbar-button { + height: 0; + width: 0; +} +::-webkit-scrollbar-track { + background-clip: padding-box; + border: solid transparent; + border-width: 0 0 0 4px; +} +::-webkit-scrollbar-corner { + background: transparent; +} + +body { + background: black; +} + +slides > slide { + display: none; + font-family: 'Open Sans', Arial, sans-serif; + font-size: 26px; + color: $gray-3; + //@include background(linear-gradient(white, white 85%, $gray-1)); + //background-color: white; + width: $slide-width; + height: $slide-height; + margin-left: -$slide-width / 2; + margin-top: -$slide-height / 2; + padding: $slide-top-bottom-padding $slide-left-right-padding; + + @include border-radius($slide-border-radius); + //@include box-shadow(5px 5px 20px $gray-4); + @include transition(all 0.6s ease-in-out); + + //$translateX: 1020px; + //$rotateY: 30deg; + //$rotateX: 45deg; + + &.far-past { + //display: block; + display: none; + //@include transform(translate(-$translateX * 2)); + //@include transform(translate3d(-$translateX * 2, 0, 0)); + } + + &.past { + display: block; + //@include transform(translate(-$translateX) rotateY($rotateY) rotateX($rotateX)); + //@include transform(translate3d(-$translateX, 0, 0) rotateY($rotateY) rotateX($rotateX)); + opacity: 0; + } + + &.current { + display: block; + //@include transform(translate(0)); + //@include transform(translate3d(0, 0, 0)); + opacity: 1; + + .auto-fadein { + opacity: 1; + } + + .gdbar { + @include background-size(100% 100%); + } + } + + &.next { + display: block; + //@include transform(translate($translateX) rotateY(-$rotateY) rotateX($rotateX)); + //@include transform(translate3d($translateX, 0, 0) rotateY(-$rotateY) rotateX($rotateX)); + opacity: 0; + pointer-events: none; + } + + &.far-next { + //display: block; + display: none; + //@include transform(translate($translateX * 2)); + //@include transform(translate3d($translateX * 2, 0, 0)); + } + + &.dark { + background: $gray-4 !important; + } + + &:not(.nobackground) { + //background: white url(../../images/google_developers_icon_128.png) ($brand-small-icon-size * 2) 98% no-repeat; + //@include background-size($brand-small-icon-size $brand-small-icon-size); + + &:before { + font-size: 12pt; + content: $social-tags; + position: absolute; + bottom: $slide-top-bottom-padding / 2; + left: $slide-left-right-padding; + // background: url(../../images/google_developers_icon_128.png) no-repeat 0 50%; + @include background-size($brand-small-icon-size $brand-small-icon-size); + padding-left: $brand-small-icon-size + 10; + height: $brand-small-icon-size; + line-height: 1.9; + } + &:after { + font-size: 12pt; + content: attr(data-slide-num) '/' attr(data-total-slides); + position: absolute; + bottom: $slide-top-bottom-padding / 2; + right: $slide-left-right-padding; + line-height: 1.9; + } + } + + &.title-slide { + &:after { + content: ''; + //background: url(../../images/io2012_logo.png) no-repeat 100% 50%; + //@include background-size(contain); + position: absolute; + bottom: $slide-top-bottom-padding; + right: $slide-top-bottom-padding; + width: 100%; + height: 60px; + } + } + + &.backdrop { + z-index: -10; + display: block !important; + @include backdrop; + + &:after, &:before { + display: none; // Prevent double set of slide nums and footer icons. + } + } + + > hgroup + article { + margin-top: $article-content-top-padding; + + &.flexbox { + &.vcenter, &.vleft, &.vright { + height: 80%; + } + } + + p { + margin-bottom: 1em; + } + } + + > article:only-child { + height: 100%; + + > iframe { + height: 98%; + } + } +} + +slides.layout-faux-widescreen > slide { + padding: $slide-top-bottom-padding 160px; +} + +slides.layout-widescreen, +slides.layout-faux-widescreen { + + $translateX: 1130px; + + > slide { + margin-left: -$slide-width-widescreen / 2; + width: $slide-width-widescreen; + } + + > slide.far-past { + display: block; + display: none; + @include transform(translate(-$translateX * 2)); + @include transform(translate3d(-$translateX * 2, 0, 0)); + } + + > slide.past { + display: block; + //@include transform(translate(-$translateX)); + //@include transform(translate3d(-$translateX, 0, 0)); + opacity: 0; + } + + > slide.current { + display: block; + //@include transform(translate(0)); + //@include transform(translate3d(0, 0, 0)); + opacity: 1; + } + + > slide.next { + display: block; + //@include transform(translate($translateX)); + //@include transform(translate3d($translateX, 0, 0)); + opacity: 0; + pointer-events: none; + } + + > slide.far-next { + display: block; + display: none; + @include transform(translate($translateX * 2)); + @include transform(translate3d($translateX * 2, 0, 0)); + } + + #prev-slide-area { + margin-left: -$slide-width-widescreen / 2 - $slide-tap-area-width; + } + + #next-slide-area { + margin-left: $slide-width-widescreen / 2; + } +} + +b { + font-weight: 600; +} + +a { + color: $brand-blue-secondary2; + text-decoration: none; + border-bottom: 1px solid rgba(42, 124, 223, 0.5); + + &:hover { + color: black !important; + } +} + +h1, h2, h3 { + font-weight: 600; +} + +h2 { + font-size: 45px; + line-height: 45px; + letter-spacing: -2px; + color: $gray-4; +} + +h3 { + font-size: 30px; + letter-spacing: -1px; + line-height: 2; + font-weight: inherit; + color: $gray-3; +} + +ul { + margin-left: 1.2em; + margin-bottom: 1em; + position: relative; + + li { + margin-bottom: 0.5em; + + ul { + margin-left: 2em; + margin-bottom: 0; + + li:before { + content: '-'; + font-weight: 600; + } + } + } + + > li:before { + content: '\00B7'; + margin-left: -1em; + position: absolute; + font-weight: 600; + } + + ul { + margin-top: .5em; + } +} + +// Code highlighting only effects the current slide. +.highlight-code slide.current { + pre > * { + opacity: 0.25; + @include transition(opacity 0.5s ease-in); + } + b { + opacity: 1; + } +} + +pre { + font-family: 'Source Code Pro', 'Courier New', monospace; + font-size: 20px; + line-height: 28px; + padding: 10px 0 10px $slide-left-right-padding; + letter-spacing: -1px; + margin-bottom: 20px; + width: 106%; + background-color: $gray-1; + left: -$slide-left-right-padding; + position: relative; + @include box-sizing(border-box); + /*overflow: hidden;*/ + + &[data-lang]:after { + content: attr(data-lang); + background-color: $gray-2; + right: 0; + top: 0; + position: absolute; + font-size: 16pt; + color: white; + padding: 2px 25px; + text-transform: uppercase; + } +} + +pre[data-lang="go"] { + color: #333; +} + +code { + font-size: 95%; + font-family: 'Source Code Pro', 'Courier New', monospace; + color: black; +} + +iframe { + width: 100%; + height: $slide-height - ($slide-top-bottom-padding * 2) - ($article-content-top-padding * 2); + background: white; + border: 1px solid $gray-1; + @include box-sizing(border-box); +} + +dt { + font-weight: bold; +} + +button { + display: inline-block; + @include background(linear-gradient(#F9F9F9 40%, #E3E3E3 70%)); + border: 1px solid $gray-2; + @include border-radius(3px); + padding: 5px 8px; + outline: none; + white-space: nowrap; + @include user-select(none); + cursor: pointer; + @include text-shadow(1px 1px #fff); + font-size: 10pt; +} + +button:not(:disabled):hover { + border-color: $gray-4; +} + +button:not(:disabled):active { + @include background(linear-gradient(#E3E3E3 40%, #F9F9F9 70%)); +} + +:disabled { + color: $gray-2; +} + +.blue { + color: $brand-blue; +} +.blue2 { + color: $brand-blue-secondary; +} +.blue3 { + color: $brand-blue-secondary2; +} +.yellow { + color: $brand-yellow; +} +.yellow2 { + color: $brand-yellow-secondary; +} +.yellow3 { + color: $brand-yellow-secondary2; +} +.green { + color: $brand-green; +} +.green2 { + color: $brand-green-secondary; +} +.green3 { + color: $brand-green-secondary2; +} +.red { + color: $brand-red; +} +.red2 { + color: $brand-red-secondary; +} +.red3 { + color: $brand-red-secondary2; +} +.gray { + color: $gray-1; +} +.gray2 { + color: $gray-2; +} +.gray3 { + color: $gray-3; +} +.gray4 { + color: $gray-4; +} + +.white { + color: white !important; +} +.black { + color: black !important; +} + +.columns-2 { + @include column-count(2); +} + +table { + width: 100%; + border-collapse: -moz-initial; + border-collapse: initial; + border-spacing: 2px; + border-bottom: 1px solid $gray-3; + + tr > td:first-child, th { + font-weight: 600; + color: $gray-4; + } + + tr:nth-child(odd) { + background-color: $gray-1; + } + + th { + color: white; + font-size: 18px; + @include background(linear-gradient(top, $brand-blue 40%, $brand-blue-secondary2 80%) no-repeat); + } + + td, th { + font-size: 18px; + padding: 1em 0.5em; + } + + td.highlight { + color: $gray-4; + @include background(linear-gradient(top, $brand-yellow 40%, $brand-yellow-secondary2 80%) no-repeat); + } + + &.rows { + border-bottom: none; + border-right: 1px solid $gray-3; + } +} + +q { + font-size: 45px; + line-height: 72px; + + &:before { + content: '“'; + position: absolute; + margin-left: -0.5em; + } + &:after { + content: '”'; + position: absolute; + margin-left: 0.1em; + } +} + +slide.fill { + background-repeat: no-repeat; + @include border-radius($slide-border-radius); + @include background-size(cover); +} + +/* Size variants */ + +article.smaller { + p, ul { + font-size: 20px; + line-height: 24px; + letter-spacing: 0; + } + table { + td, th { + font-size: 14px; + } + } + pre { + font-size: 15px; + line-height: 20px; + letter-spacing: 0; + } + q { + font-size: 40px; + line-height: 48px; + + &:before, &:after { + font-size: 60px; + } + } +} + +/* Builds */ + +.build { + > * { + @include transition(opacity 0.5s ease-in-out 0.2s); + } + + .to-build { + opacity: 0; + } + + .build-fade { + opacity: 0.3; + + &:hover { + opacity: 1.0; + } + } +} + +.popup .next .build { + .to-build { + opacity: 1; + } + + .build-fade { + opacity: 1; + } +} + +/* Pretty print */ + +.prettyprint .str, /* string content */ +.prettyprint .atv { /* a markup attribute value */ + color: $brand-green-secondary2; //rgb(0, 138, 53); +} +.prettyprint .kwd, /* a keyword */ +.prettyprint .tag { /* a markup tag name */ + color: rgb(0, 102, 204); +} +.prettyprint .com { /* a comment */ + color: $gray-3; //rgb(127, 127, 127); + font-style: italic; +} +.prettyprint .lit { /* a literal value */ + color: rgb(127, 0, 0); +} +.prettyprint .pun, /* punctuation, lisp open bracket, lisp close bracket */ +.prettyprint .opn, +.prettyprint .clo { + color: $gray-4; //rgb(127, 127, 127); +} +.prettyprint .typ, /* a type name */ +.prettyprint .atn, /* a markup attribute name */ +.prettyprint .dec, +.prettyprint .var { /* a declaration; a variable name */ + color: $brand-red-secondary2; //rgb(127, 0, 127); +} +.prettyprint .pln { + color: $gray-4; +} + +.note { + position: absolute; + z-index: 100; + width: 100%; + height: 100%; + top: 0; + left: 0; + padding: 1em; + background: rgba(0, 0, 0, 0.3); + opacity: 0; + pointer-events: none; + @include flexbox; + @include flex-center-center; + @include border-radius($slide-border-radius); + + @include box-sizing(border-box); + @include transform(translateY($slide-height / 2));@include transition(all 0.4s ease-in-out); + + > section { + background: #fff; + @include border-radius($slide-border-radius); + @include box-shadow(0 0 10px $gray-3); + width: 60%; + padding: 2em; + } +} + +// Speaker notes only show the current slide. +.with-notes { + + &.popup { + + slides.layout-widescreen, + slides.layout-faux-widescreen { + slide { + &.next { + @include transform(translate3d($slide-width-widescreen / 2 + 140, 80px, 0) scale(0.35)); + } + .note { + @include transform(translate3d(300px, $slide-height + 100, 0) scale(1.5)); + } + } + } + + slide { + overflow: visible; + background: white; + @include transition(none); // No slide transition goodies when in presenter mode. + pointer-events: none; + @include transform-origin(0, 0); // For speaker note transition. + + &:not(.backdrop) { + @include transform(scale(0.6) translate3d(0.5em, 0.5em, 0)); + @include box-shadow(0 0 10px $gray-3); + } + + &.backdrop { + //@include background(linear-gradient($gray-1, white 30%, white 60%, $gray-1)); + @include background-image(radial-gradient(50% 50%, #b1dfff 0%, + $brand-blue 600px)); + } + + &.next { + @include transform(translate3d($slide-width / 2 + 120, 80px, 0) scale(0.35)); + opacity: 1 !important; + + .note { + display: none !important; // Prevents seeing notes if we go to previous slide. + } + } + } + + .note { + width: 109%; + height: $slide-height / 2 - 90; + background: $gray-1; + padding: 0; + + @include box-shadow(0 0 10px $gray-3); + + @include transform(translate3d(250px, $slide-height + 100, 0) scale(1.5)); + @include transition(opacity 400ms ease-in-out); + + > section { + background: #fff; + @include border-radius($slide-border-radius); + height: 100%; + width: 100%; + @include box-sizing(border-box); + @include box-shadow(none); + overflow: auto; + padding: 1em; + } + } + } + + .note { + opacity: 1; + @include transform(translateY(0)); + pointer-events: auto; // Allow people to do things like open links embedded in the speaker notes. + } +} + +.source { + font-size: 14px; + color: $gray-2; + position: absolute; + bottom: $slide-top-bottom-padding + 30px; + left: $slide-left-right-padding; +} + +.centered { + text-align: center; +} + +.reflect { + -webkit-box-reflect: below 3px -webkit-linear-gradient(rgba(255,255,255,0) 85%, white 150%); + -moz-box-reflect: below 3px -moz-linear-gradient(rgba(255,255,255,0) 85%, white 150%); + -o-box-reflect: below 3px -o-linear-gradient(rgba(255,255,255,0) 85%, white 150%); + -ms-box-reflect: below 3px -ms-linear-gradient(rgba(255,255,255,0) 85%, white 150%); + box-reflect: below 3px linear-gradient(rgba(255,255,255,0) 85%, white 150%); +} + +.flexbox { + @include flexbox; +} + +.flexbox.vcenter { + @include flex-center-center; + height: 100%; + width: 100%; +} + +.flexbox.vleft { + @include flex-left-center; + height: 100%; + width: 100%; +} + +.flexbox.vright { + @include flex-right-center; + height: 100%; + width: 100%; +} + +.auto-fadein { + @include transition(opacity 0.6s ease-in 1s); + opacity: 0; +} + +/* Clickable/tappable areas */ +.slide-area { + z-index: 1000; + + position: absolute; + left: 0; + top: 0; + width: $slide-tap-area-width; + height: $slide-height; + + left: 50%; + top: 50%; + + cursor: pointer; + margin-top: -$slide-height / 2; + + //@include highlight-color(rgba(51, 51, 51, 0.5)); +} +#prev-slide-area { + margin-left: -$slide-width-widescreen / 2; + //@include border-radius(10px 0 0 10px); + //@include box-shadow(-5px 0 10px #222 inset); +} +#next-slide-area { + margin-left: $slide-width / 2; + //@include border-radius(0 10px 10px 0); + //@include box-shadow(5px 0 10px #222 inset); +} + +/* ===== SLIDE CONTENT ===== */ +.logoslide { + img { + width: 383px; + height: 92px; + } +} + +.segue { + padding: $slide-left-right-padding $slide-left-right-padding * 2; + + h2 { + color: $gray-1; + font-size: 60px; + } + h3 { + color: $gray-1; + line-height: 2.8; + } + hgroup { + position: absolute; + bottom: 225px; + } +} + +.thank-you-slide { + background: $brand-blue !important; + color: white; + + h2 { + font-size: 60px; + color: inherit; + } + + article > p { + margin-top: 2em; + font-size: 20pt; + } + + > p { + position: absolute; + bottom: $slide-top-bottom-padding * 2; + font-size: 24pt; + line-height: 1.3; + } +} + +aside.gdbar { + height: 97px; + width: 215px; + position: absolute; + left: -1px; + top: 125px; + @include border-radius(0 10px 10px 0); + @include background(linear-gradient(left, $gray-1, $gray-1) no-repeat); + @include background-size(0% 100%); + @include transition(all 0.5s ease-out 0.5s); /* Better to transition only on background-size, but not sure how to do that with the mixin. */ + + &.right { + right: 0; + left: -moz-initial; + left: initial; + top: ($slide-height / 2) - 96; /* 96 is height of gray icon bar */ + @include transform(rotateZ(180deg)); + + img { + @include transform(rotateZ(180deg)); + } + } + + &.bottom { + top: -moz-initial; + top: initial; + bottom: $slide-left-right-padding; + } + + img { + width: 85px; + height: 85px; + position: absolute; + right: 0; + margin: 8px 15px; + } +} + +.title-slide { + + hgroup { + bottom: 100px; + + h1 { + font-size: 65px; + line-height: 1.4; + letter-spacing: -3px; + color: $gray-4; + } + + h2 { + font-size: 34px; + color: $gray-2; + font-weight: inherit; + } + + p { + font-size: 20px; + color: $gray-3; + line-height: 1.3; + margin-top: 2em; + } + } +} + +.quote { + color: $gray-1; + + .author { + font-size: 24px; + position: absolute; + bottom: 80px; + line-height: 1.4; + } +} + +[data-config-contact] { + a { + color: rgb(255, 255, 255); + border-bottom: none; + } + span { + width: 115px; + display: inline-block; + } +} + +.overview { + + &.popup { + .note { + display: none !important; + } + } + + slides { + slide { + &.backdrop { + display: none !important; + } + + display: block; + cursor: pointer; + opacity: 0.5; + pointer-events: auto !important; + + @include backdrop(); + + &.far-past, + &.past, + &.next, + &.far-next, + &.far-past { + opacity: 0.5; + display: block; + } + + &.current { + opacity: 1; + } + } + } + + .slide-area { + display: none; + } +} + +@media print { + slides { + slide { + display: block !important; + position: relative; + @include backdrop(); + @include transform(none !important); + width: 100%; + height: 100%; + page-break-after:always; + top: auto !important; + left: auto !important; + margin-top: 0 !important; + margin-left: 0 !important; + opacity: 1 !important; + color: #555; + + &.far-past, + &.past, + &.next, + &.far-next, + &.far-past, + &.current { + opacity: 1 !important; + display: block !important; + } + + .build { + > * { + @include transition(none); + } + + .to-build, + .build-fade { + opacity: 1; + } + } + + .auto-fadein { + opacity: 1 !important; + } + + &.backdrop { + display: none !important; + } + + table.rows { + border-right: 0; + } + } + + slide[hidden] { + display: none !important; + } + } + + .slide-area { + display: none; + } + + .reflect { + -webkit-box-reflect: none; + -moz-box-reflect: none; + -o-box-reflect: none; + -ms-box-reflect: none; + box-reflect: none; + } + + pre, code { + font-family: monospace !important; + } +} diff --git a/source/_themes/uwpce_slides2/static/theme/scss/hieroglyph.scss b/source/_themes/uwpce_slides2/static/theme/scss/hieroglyph.scss new file mode 100644 index 00000000..e4060852 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/scss/hieroglyph.scss @@ -0,0 +1,100 @@ +@import "compass/css3/background-size"; + +@import "variables"; + +ol { + margin-left: 1.2em; + margin-bottom: 1em; + position: relative; + list-style: decimal; + + li { + margin-bottom: 0.5em; + + ol { + margin-left: 2em; + margin-bottom: 0; + list-style: decimal; + + li:before { + font-weight: 600; + } + } + } + + ol { + margin-top: .5em; + list-style: decimal; + + } +} + +slide.title-image { + + padding-right: 0px; + + hgroup { + position: static !important; + + margin-top: 35%; + padding-left: 30px; + + background: rgba(255, 255, 255, 0.7); + + border-top-left-radius: $slide-border-radius; + -webkit-border-top-left-radius: $slide-border-radius; + -moz-border-top-left-radius: $slide-border-radius; + -o-border-top-left-radius: $slide-border-radius; + } + + hgroup + article { + background: rgba(255, 255, 255, 0.7); + + margin-top: 0px; + padding-left: 30px; + + border-bottom-left-radius: $slide-border-radius; + -webkit-border-bottom-left-radius: $slide-border-radius; + -moz-border-bottom-left-radius: $slide-border-radius; + -o-border-bottom-left-radius: $slide-border-radius; + } + + h1 { + color: #222; + font-size: 3.2em; + + line-height: 1.5em; + font-weight: 500; + } + + div.figure { + + img { + position: absolute; + left: 0; + top: 0; + min-width: 100%; + min-height: 100%; + + border-radius: $slide-border-radius; + -o-border-radius: $slide-border-radius; + -moz-border-radius: $slide-border-radius; + -webkit-border-radius: $slide-border-radius; + + z-index: -1; + } + + .caption { + color: black; + background: rgba(255, 255, 255, 0.25); + padding: 0 5px; + border-bottom-left-radius: $slide-border-radius; + border-top-right-radius: $slide-border-radius; + + position: absolute; + left: 0; + bottom: 0; + margin-bottom: 0; + } + } +} diff --git a/source/_themes/uwpce_slides2/static/theme/scss/io2013.scss b/source/_themes/uwpce_slides2/static/theme/scss/io2013.scss new file mode 100644 index 00000000..c728cfbf --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/scss/io2013.scss @@ -0,0 +1,51 @@ +@import "compass/css3/background-size"; + +@import "variables"; + +* { + line-height: 1.3; +} + +h2 { + font-weight: bold; +} +h2, h3 { + color: $gray-4; +} + +q, blockquote { + font-weight: bold; +} + +slides > slide { + color: $gray-4; + + &.title-slide { + &:after { + content: ''; + background: url(../../images/io2013/google-io-lockup-1.png) no-repeat 100% 50%; + @include background-size(contain); + position: absolute; + bottom: $slide-top-bottom-padding + 40; + right: $slide-top-bottom-padding; + width: 100%; + height: 90px; + } + + hgroup { + h1 { + font-weight: bold; + line-height: 1.1; + } + h2, p { + color: $gray-4; + } + h2 { + margin-top: 0.25em; + } + p { + margin-top: 3em; + } + } + } +} \ No newline at end of file diff --git a/source/_themes/uwpce_slides2/static/theme/scss/phone.scss b/source/_themes/uwpce_slides2/static/theme/scss/phone.scss new file mode 100644 index 00000000..c6a40432 --- /dev/null +++ b/source/_themes/uwpce_slides2/static/theme/scss/phone.scss @@ -0,0 +1,35 @@ +@import "compass/css3/transition"; + + +/*Smartphones (portrait and landscape) ----------- */ +/*@media only screen +and (min-width : 320px) +and (max-width : 480px) { + +}*/ + +/* Smartphones (portrait) ----------- */ +//@media only screen and (max-device-width: 480px) { +/* Styles */ +//$slide-width: 350px; +//$slide-height: 500px; + +slides > slide { +/* width: $slide-width !important; + height: $slide-height !important; + margin-left: -$slide-width / 2 !important; + margin-top: -$slide-height / 2 !important; +*/ + // Don't do full slide transitions on mobile. + -webkit-transition: none !important; // Bug in compass? Not sure why the below is not working + @include transition(none !important); +} + +//} + +/* iPhone 4 ----------- */ +@media +only screen and (-webkit-min-device-pixel-ratio : 1.5), +only screen and (min-device-pixel-ratio : 1.5) { +/* Styles */ +} \ No newline at end of file diff --git a/source/_themes/uwpce_slides2/theme.conf b/source/_themes/uwpce_slides2/theme.conf new file mode 100644 index 00000000..8919ebcc --- /dev/null +++ b/source/_themes/uwpce_slides2/theme.conf @@ -0,0 +1,15 @@ +[theme] +inherit = slides2 +stylesheet = slides.css + +[options] +custom_css = +custom_js = + +subtitle = +use_builds = true +use_prettify = true +enable_slide_areas = true +enable_touch = true +favicon = '' +presenters = diff --git a/source/_themes/uwpce_slides2/title_slide.html b/source/_themes/uwpce_slides2/title_slide.html new file mode 100644 index 00000000..e69de29b diff --git a/source/conf.py b/source/conf.py index 9959eaf8..9609e03c 100644 --- a/source/conf.py +++ b/source/conf.py @@ -50,16 +50,16 @@ # General information about the project. project = u'Internet Programming with Python' -copyright = u'2012-2015, Cris Ewing' +copyright = u'2012-2016, Cris Ewing' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. -version = '2.0' +version = '3.0' # The full version, including alpha/beta/rc tags. -release = '2.0' +release = '3.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. @@ -256,11 +256,16 @@ 'hieroglyph', ] +# TODO: open bug report with hieroglyph plus pr that fixes the documentation +# for this. Current docs suggest that html_theme_path is the setting needed, +# but it is not. +slide_theme_path = ['_themes'] slide_title = "Internet Programming with Python" -slide_theme = 'slides2' +slide_theme = 'uwpce_slides2' slide_levels = 3 slide_link_html_to_slides = True slide_relative_path = './slides' +slide_numbers = True # Place custom static assets in the _static directory and uncomment # the following lines to include them @@ -271,11 +276,11 @@ # 'custom_js': 'custom.js', 'presenters': [ { - 'name': 'Cris Ewing', - 'twitter': '@crisewing', - 'www': 'http://crisewing.com', - 'github': 'http://github.com/cewing', - 'company': 'Cris Ewing, Developer LLC' + 'name': 'Christy Heaton', + 'twitter': '@christytoes', + 'www': 'http://christyheaton.github.io', + 'github': 'http://github.com/christyheaton', + 'company': 'University of Washington' }, ] } diff --git a/source/index.rst b/source/index.rst index 1ff19ba8..f85bcb62 100644 --- a/source/index.rst +++ b/source/index.rst @@ -14,7 +14,10 @@ Internet Programming with Python .. slide:: Internet Programming with Python :level: 1 - This document contains no slides. + .. rst-class:: large center + .. container:: + + No Content .. sidebar:: In This Class @@ -22,32 +25,32 @@ Internet Programming with Python :maxdepth: 2 outline - readings presentations/index + readings -Winter Term, 2015 (10 sessions) +Winter Term, 2017 (10 sessions) -Tuesdays, 6-9 pm, January 6 - March 10 +Tuesdays, 6-9 pm, January 10 - March 14 Overview ======== -This course emphasizes network-based programming and Web -applications, how they work and how to program them in Python. Explore the -underlying principles and their expression in the Python libraries. Learn -contrasting approaches in creating applications: programming with the -low-level libraries versus using highly integrated frameworks +This course emphasizes network-based programming and Web applications, how they +work and how to program them in Python. Explore the underlying principles and +their expression in the Python libraries. Learn contrasting approaches in +creating applications: programming with the low-level libraries versus using +highly integrated frameworks Prerequisites ============= -To attend this course you should have a working knowledge of the basic -syntax and structures of the Python programming language. You will also need -to be comfortable working at the command line to navigate a file system, -create and delete files, and execute commands. Finally, you should have some -basic knowledge of HTML. +To attend this course you should have a working knowledge of the basic syntax +and structures of the Python programming language. You will also need to be +comfortable working at the command line to navigate a file system, create and +delete files, and execute commands. Finally, you should have some basic +knowledge of HTML. Requirements @@ -57,14 +60,15 @@ This workshop does not provide a computer laboratory. You will have to have a portable computer in order to participate. Network access is provided, but you will need to know how to operate the network settings for your computer. -Your computer must have Python version 2.6 or 2.7 installed. No additional +Your computer must have Python version 3.4 or later installed. No additional libraries will be required, but we will be installing some as the workshop progresses. -You will want to familiarize yourself with `virtualenv -`_. We will use it extensively in class -in order to keep our development environments clean and separate. +To keep clean and isolated development environments, we will make use in class +of the `venv`_ module, a standard library module used to create and maintain +lightweight sandbox environments. +.. _venv: https://docs.python.org/3/library/venv.html What to Expect ============== @@ -96,20 +100,24 @@ frameworks covered as well as the choices and compromises that shape them. References ========== -`Python 2 Documentation `_: Complete +`Python 3 Documentation `_: Complete documentation of the language. -`Python 2.7 Quick Reference `_: -Dense and complete. Good for jogging your memory, but don't start here. +`Python 3 Language Reference `_: Terse +and complete reference to the language structures of Python 3. + +Python Standard Library - +`Internet Protocols and Support `_: +All the supported internet protocols as implemented in Python. -`Python Standard Library - Internet Protocols and Support -`_: All the supported internet -protocols as implemented in Python. +Python Module of the Week (`py2`_, `py3`_): A fantastic reference for many +modules in Python 2 and 3. Examples and usage are provided throughout. Don't be +shy about trying the Python 2 docs in Python 3, often they will work still. -`Python Module of the Week `_: A fantastic reference for -any module in python. Examples and usage are provided throughout. +.. _py2: https://pymotw.com/2/contents.html +.. _py3: https://pymotw.com/3/ -`Lecture Presentations `_: Slides from the workshop +`Lecture Presentations `_: Slides from the course presentations. diff --git a/source/outline.rst b/source/outline.rst index 9e5ec8b5..13e5f514 100644 --- a/source/outline.rst +++ b/source/outline.rst @@ -15,41 +15,7 @@ prompt. Each session has associated assignments which you will complete between sessions. -Session 1 - MVC Applications and Data Persistence -------------------------------------------------- - -In this session we will begin by introducing the idea of an MVC (*Model View -Controller*) application. We'll discuss this popular application design -pattern and talk about the ways in which it does and does not apply to the -world of web applications. - -We'll get started with our first application, a learning journal written in the -lignt but powerful *Pyramid* web framework. We'll set up a development -environment and install the framework and dependencies. We'll create our first -*models* and experiment with persisting data to a database. - -References -********** - - -Preparation for Session 2 -************************* - -In preparation for session 2, please read the following materials: - -* `Jinja2 Template Tutorial - `_ -* `HTML5 Site Layout Tutorial - `_ - -Session 2 - Pyramid Views, Renderers and Forms ----------------------------------------------- - -Sesstion 3 - Pyramid Authentication and Deployment --------------------------------------------------- - - -Session 4 - TCP/IP and Sockets +Session 1 - TCP/IP and Sockets ------------------------------ We will begin with a disucssion of the fundamental concepts and structures @@ -71,7 +37,7 @@ References * `Python Module of the Week - socket `_ -Session 5 - Web Protocols +Session 2 - Web Protocols ------------------------- Protocols are the languages of the Internet. They govern how machines speak to @@ -100,8 +66,22 @@ clear and concise and have some great code examples. .. _Internet Protocols and Support: http://pymotw.com/2/internet_protocols.html +Session 3 - CGI and WSGI +------------------------ + +In this class we will explore ways of moving data from HTTP requests into the +dynamic scripts that process data. We will begin by looking at the original +specification for passing data, CGI (Common Gateway Interface). We'll look at +the benefits and drawbacks of the specification, and use it to create some +simple interactions. + +Then we will investigate a more modern take on the same problem, WSGI (Web +Services Gateway Interface). We'll see the ways in which WSGI is similar to +CGI, and look at the ways in which it differs. We'll create a simple interaction +using WSGI and see what benefits and drawbacks it confers. + -Session 6 - APIs and Mashups +Session 4 - APIs and Mashups ---------------------------- The internet is a treasure trove of information. But meaning can be hard to @@ -121,33 +101,91 @@ build a script that can produce derived meaning out of data we find online. References ********** +* `BeautifulSoup `_ +* `requests `_ +* `json `_ +* `geocoder `_ * `httplib `_ * `htmlparser `_ * `xmlrpclib `_ * `DocXMLRPCServer `_ -* `json `_ -Session 7 - CGI and WSGI ------------------------- +Session 5 - MVC Applications and Data Persistence +------------------------------------------------- -In this class we will explore ways of moving data from HTTP requests into the -dynamic scripts that process data. We will begin by looking at the original -specification for passing data, CGI (Common Gateway Interface). We'll look at -the benefits and drawbacks of the specification, and use it to create some -simple interactions. +In this session we will begin by introducing the idea of an MVC (*Model View +Controller*) application. We'll discuss this popular application design +pattern and talk about the ways in which it does and does not apply to the +world of web applications. + +We'll get started with our first application, a learning journal written in the +lignt but powerful *Pyramid* web framework. We'll set up a development +environment and install the framework and dependencies. We'll create our first +*models* and experiment with persisting data to a database. + +References +********** + + +Preparation for Session 6 +************************* + +In preparation for session 6, please read the following materials: + +* `Jinja2 Template Tutorial + `_ +* `HTML5 Site Layout Tutorial + `_ + +Session 6 - Pyramid Views, Renderers and Forms +---------------------------------------------- + +In this session we extend our understanding of the MVC design pattern by +learning how Pyramid implements the *view* and *controller* aspects. + +Pyramid *views* represent the *controller* part of the MVC pattern, and we'll +create a number of them. We'll also learn how Pyramid uses *routes* to properly +connect the *path* requested by a client to the *views* run by a server. + +We'll meet with Pyramid's *renderers*, the *view* in MVC. We'll start by using +a built-in renderer that simply turns view data into strings sent back to the +client as plain text responses. We'll then install a template-based renderer +and use the *jinja2* template language to create visible HTML pages the brower +can load to show our learning journal entries. + +Prepraration for Session 7 +************************** + +In preparation for session 7, please read up on getting started with `Heroku +and Python`_. We'll be deploying our learning journal to Heroku by the end of +that session. + +.. _Heroku and Python: https://devcenter.heroku.com/articles/getting-started-with-python#introduction + +Sesstion 7 - Pyramid Authentication and Deployment +-------------------------------------------------- + +In this session we will learn the basic elements of access control: +authentication and authorization. We'll learn how Pyramid implements these two +aspects of security, and will implement a basic security policy for our +learning journal. + +Once complete, we will deploy our application to Heroku. We'll make a few +changes to how our app is configured to fit with the Heroku model and will be +able to see our application in action by the end of the session. + +Time permitting, we will enhance our application with a few special features +such as Markdown formatting, and code highlighting. A list of potential future +enhancements will give you plenty to think about for the rest of the week. -Then we will investigate a more modern take on the same problem, WSGI (Web -Services Gateway Interface). We'll see the ways in which WSGI is similar to -CGI, and look at the ways in which it differs. We'll create a simple interaction -using WSGI and see what benefits and drawbacks it confers. Preparation for Session 8 ************************* Please walk through this tutorial before session 8 begins. -* `An Introduction to Django `_ +* `An Introduction to Django `_ Session 8 - Basic Django @@ -165,7 +203,7 @@ blog app in Django. We'll learn how to use the tools Django provides to explore and interact with your models while designing them. We'll also get a brief introduction to the Django admin, Django's *killer feature*. -.. _get started building: presentations/django_intro-plain.html +.. _get started building: presentations/django_intro.html Along the way, we'll build a nicely functional blog application. We'll learn @@ -184,16 +222,23 @@ Session 9 - Extending Django ---------------------------- During this session, we will continue our exploration of Django, and of pair -programming. Students will once again pair up and work together to implement -one or more feature extending the basic Django app we created previously. +programming. Students will pair up and work together to implement one or more +feature extending the basic Django app we created previously. Finally, we'll discuss some of the strengths and weaknesses of Django. What makes it a good choice for some projects but not for others. -`Lecture Slides `_ +Preparation for Session 10 +************************** + +In preparation for session 10, you'll need to sign up for an account with +Amazon Web Services. Session 10 - Deploying Django ----------------------------- +During this session, we will deploy our Django application to Amazon Web +Services. To do so, we'll use a popular Python-based configuration management +tool, Ansible. diff --git a/source/presentations/django_intro.rst b/source/presentations/django_intro.rst new file mode 100644 index 00000000..c0ee5100 --- /dev/null +++ b/source/presentations/django_intro.rst @@ -0,0 +1,1051 @@ +.. slideconf:: + :autoslides: False + +************************* +An Introduction To Django +************************* + +.. slide:: Internet Programming with Python + :level: 1 + + This document contains no slides. + +In this tutorial, you'll walk through creating a very simple microblog +application using Django. + +Practice Safe Development +========================= + +We'll install Django and any other packages we use with it in a virtualenv. + +This will ensure that it is isolated from everything else we do in class +(and vice versa) + +Remember the basic format for creating a virtualenv:: + + $ python -m venv [options] + + $ pyvenv [options] + + +Set Up a VirtualEnv +------------------- + +Start by creating your virtualenv:: + + $ python -m venv djangoenv + + $ pyvenv djangoenv + ... + +Then, activate it:: + + $ source djangoenv/bin/activate + + C:\> djangoenv\Scripts\activate.bat + + +Install Django +-------------- + +Finally, install Django 1.7.4 using ``pip``:: + + (djangoenv)$ pip install Django==1.9 + Collecting Django==1.9 + Downloading Django-1.9-py2.py3-none-any.whl (6.6MB) + 100% |████████████████████████████████| 6.6MB 47kB/s + Installing collected packages: Django + Successfully installed Django-1.9 + (djangoenv)$ + + +Our Project +=========== + +Everything in Django stems from the *project*. To get started learning, we'll +create one. We'll use a script installed by Django, ``django-admin.py``: + +.. code-block:: bash + + (djangoenv)$ django-admin startproject mysite + +If you're on windows, that command is slightly different: + +.. code-block:: bash + + django-admin.exe startproject mysite + +.. note:: If you run into trouble at this stage, please consult the + `installation documentation`_. For windows users, see also + `this guide to installation on Windows`_ + +.. _installation documentation: https://docs.djangoproject.com/en/1.9/intro/install/ +.. _this guide to installation on Windows: https://docs.djangoproject.com/en/1.9/howto/windows/ + + +This will create a folder called 'mysite'. The folder contains the following +structure:: + + mysite + ├── manage.py + └── mysite + ├── __init__.py + ├── settings.py + ├── urls.py + └── wsgi.py + +If what you see doesn't match that, you're using an older version of Django. +Make sure you've installed 1.7.4 + + +What Got Created +---------------- + +* **outer *mysite* folder**: this is just a container and can be renamed or + moved at will +* **inner *mysite* folder**: this is your project directory. It should not be + renamed. +* **__init__.py**: magic file that makes *mysite* a python package. +* **settings.py**: file which holds configuration for your project, more soon. +* **urls.py**: file which holds top-level URL configuration for your project, + more soon. +* **wsgi.py**: binds a wsgi application created from your project to the + symbol ``application`` +* **manage.py**: a management control script. + +*django-admin* provides a hook for administrative tasks and abilities: + +* creating a new project or app +* running the development server +* executing tests +* entering a python interpreter +* entering a database shell session with your database +* much much more (run ``django-admin`` without an argument) + +*manage.py* wraps this functionality, adding the full environment of your +project. + + +How *manage.py* Works +--------------------- + +Look in the ``manage.py`` script Django created for you. You'll see this: + +.. code-block:: python + + #!/usr/bin/env python + import os + import sys + + if __name__ == "__main__": + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") + + from django.core.management import execute_from_command_line + + execute_from_command_line(sys.argv) + +The environmental var ``DJANGO_SETTINGS_MODULE`` is how the ``manage.py`` +script is made aware of your project's environment. This is why you shouldn't +rename the project package. + + +Development Server +------------------ + +At this point, you should be ready to use the development server:: + + (djangoenv)$ cd mysite + (djangoenv)$ ./manage.py runserver + ... + +You'll see a scary warning about unapplied migrations. Ignore it for a moment. +Instead, load ``http://localhost:8000`` in your browser. You should see this: + +.. figure:: /_static/django-start.png + :align: center + :width: 98% + +.. rst-class:: build center + +**Do you?** + + +Connecting A Database +--------------------- + +Django supplies its own ORM (Object-Relational Mapper). This ORM sits on top of +the DB-API implementation you choose. You must provide connection information +through Django configuration. + +All Django configuration takes place in ``settings.py`` in your project +folder. + +Edit your ``settings.py`` to match: + +.. code-block:: python + + + DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.sqlite3', + 'NAME': 'mysite.db', + } + } + +There are other database settings, but they are not used with sqlite3, we'll +ignore them for now. + +Django's ORM provides a layer of *abstraction* between you and SQL. You write +Python classes called *models* describing the objects that make up your system. +The ORM handles converting data from these objects into SQL statements (and +back). We'll learn much more about this in a bit. + +The final step in preparing to work is to set up the database. You do this by +running *migrations*. These migrations create the tables needed to support the +models that are required by Django out of the box. + +Run the following command: + +.. code-block:: bash + + (djangoenv)$ ./manage.py migrate + Operations to perform: + Apply all migrations: admin, contenttypes, auth, sessions + Running migrations: + Applying contenttypes.0001_initial... OK + Applying auth.0001_initial... OK + Applying admin.0001_initial... OK + Applying sessions.0001_initial... OK + +Great! Now we can set up an initial user who'll be able to do anything, a +*superuser*. Again, we'll use ``manage.py``: + +.. code-block:: bash + + (djangoenv)$ ./manage.py createsuperuser + Username (leave blank to use 'cewing'): + Email address: cris@crisewing.com + Password: + Password (again): + Superuser created successfully. + +Notice that as you type your password, it will not appear on the screen. Don't +worry, it's actually being recorded. You just can't see it (and neither can +that snoopy git looking over your shoulder). + +Projects and Apps +================= + +We've created a Django *project*. In Django a project represents a whole +website: + +* global configuration settings +* inclusion points for additional functionality +* master list of URL endpoints + +A Django *app* encapsulates a unit of functionality: + +* A blog section +* A discussion forum +* A content tagging system + +.. important:: One *project* can (and likely will) consist of many *apps* + +Django already includes some *apps* for you. + +.. container:: incremental + + They're in ``settings.py`` in the ``INSTALLED_APPS`` setting: + + .. code-block:: python + + + INSTALLED_APPS = ( + 'django.contrib.admin', + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.messages', + 'django.contrib.staticfiles', + ) + + +Our Class App +------------- + +We are going to build an *app* to add to our *project*. To start with our app +will be a simple blog. As stated above, an *app* represents a unit within a +system, the *project*. We have a project, we need to create an *app* + +This is accomplished using ``manage.py``. In your terminal, make sure you are +in the *outer* mysite directory, where the file ``manage.py`` is located. +Then: + +.. code-block:: bash + + (djangoenv)$ ./manage.py startapp myblog + +This should leave you with the following structure: + +.. class:: small + +:: + + mysite + ├── db.sqlite3 + ├── manage.py + ├── myblog + │   ├── __init__.py + │   ├── admin.py + │   ├── migrations + │   │   └── __init__.py + │   ├── models.py + │   ├── tests.py + │   └── views.py + └── mysite + ├── __init__.py + ├── settings.py + ├── urls.py + └── wsgi.py + +Like our Pyramid site, Django divides up functionality by module. You'll create +ORM model classes in the ``models.py`` file, view code in the ``views.py`` +file, and so on. + +We'll start by defining the main Python class for our blog system, a ``Post``. + + +Django Models +------------- + +Any Python class in Django that is meant to be persisted *must* inherit from +the Django ``Model`` class. This base class hooks in to the ORM functionality +converting Python code to SQL. You can override methods from the base ``Model`` +class to alter how this works or write new methods to add functionality. + +Learn more about `models `_ + + +Our Post Model +-------------- + +Open the ``models.py`` file created in our ``myblog`` package. Add the +following: + +.. code-block:: python + + from django.db import models #<-- This is already in the file + from django.contrib.auth.models import User + + class Post(models.Model): + title = models.CharField(max_length=128) + text = models.TextField(blank=True) + author = models.ForeignKey(User) + created_date = models.DateTimeField(auto_now_add=True) + modified_date = models.DateTimeField(auto_now=True) + published_date = models.DateTimeField(blank=True, null=True) + +This code defines a subclass of the Django ``Model`` class and added a bunch of +attributes. + +* These attributes are all instances of ``Field`` classes defined in Django +* Field attributes on a model map to columns in a database table +* The arguments you provide to each Field customize how it works + + * This means *both* how it operates in Django *and* how it is defined in SQL + +* There are arguments shared by all Field types +* There are also arguments specific to individual types + +You can read much more about +`Model Fields and options `_ + +There are some features of our fields worth mentioning in specific. Notice we +have no field that is designated as the *primary key* + +* You *can* make a field the primary key by adding ``primary_key=True`` in the + arguments +* If you do not, Django will **automatically** create one. This field is always + called ``id`` +* No matter what the primary key field is called, its *value* is always + available on a model instance as the ``pk`` attribute: ``instance.pk`` + + +Field Details +------------- + +.. code-block:: python + + title = models.CharField(max_length=128) + +* The required ``max_length`` argument is specific to ``CharField`` fields. +* It affects *both* the Python and SQL behavior of a field. +* In python, it is used to *validate* supplied values during *model validation* +* In SQL it is used in the column definition: ``VARCHAR(128)`` + +.. code-block:: python + + author = models.ForeignKey(User) + +* Django also models SQL *relationships* as specific field types. +* The required positional argument is the class of the related Model. +* By default, the reverse relation is implemented as the attribute + ``_set``. +* You can override this by providing the ``related_name`` argument. + +.. code-block:: python + + created_date = models.DateTimeField(auto_now_add=True) + modified_date = models.DateTimeField(auto_now=True) + +* ``auto_now_add`` is available on all date and time fields. It sets the value + of the field to *now* when an instance is first saved. +* ``auto_now`` is similar, but sets the value anew each time an instance is + saved. +* Setting either of these will cause the ``editable`` attribute of a field to + be set to ``False``. +* This does not mean you can't update these values, only that they will not + show in forms by default. + +.. code-block:: python + + text = models.TextField(blank=True) + # ... + published_date = models.DateTimeField(blank=True, null=True) + +* The argument ``blank`` is shared across all field types. The default is + ``False`` +* This argument affects only the Python behavior of a field, determining if the + field is *required* +* The related ``null`` argument affects the SQL definition of a field: is the + column NULL or NOT NULL +* Django recommends that you **not** use the ``null`` option for text fields. + It will automatically insert an empty string into the database if the field + is left blank. + + +Installing Apps +--------------- + +In order to use our new model, we need Django to know about our *app*. This is +accomplished by configuration in the ``settings.py`` file. Open that file now, +in your editor, and find the INSTALLED_APPS setting. + +You extend Django functionality by *installing apps*. This is pretty simple: + +.. code-block:: python + + + INSTALLED_APPS = ( + 'django.contrib.admin', + 'django.contrib.auth', + 'django.contrib.contenttypes', + 'django.contrib.sessions', + 'django.contrib.messages', + 'django.contrib.staticfiles', + 'myblog', # <- YOU ADD THIS PART + ) + +Once Django is made aware of the existence of this new app, it can make a new +*migration* that will set up the tables for this new class automatically. + +.. code-block:: bash + + (djangoenv)$ ./manage.py makemigrations myblog + Migrations for 'myblog': + 0001_initial.py: + - Create model Post + +And now you can run that migration to make the changes to your database: + +.. code-block:: bash + + (djangoenv)$ ./manage.py migrate + Operations to perform: + Apply all migrations: sessions, myblog, contenttypes, auth, admin + Running migrations: + Rendering model states... DONE + Applying myblog.0001_initial... OK + + +The Django Shell +================ + +Django provides a management command ``shell``: + +* Shares the same ``sys.path`` as your project, so all installed python + packages are present. +* Imports the ``settings.py`` file from your project, and so shares all + installed apps and other settings. +* Handles connections to your database, so you can interact with live data + directly. + +The Django ``shell`` will use more advanced Python interpreters such as +``iPython`` if they are available. Let's go ahead and install iPython in our +``djangoenv`` to get this advantage: + +.. code-block:: bash + + (djangoenv)$ pip install ipython + ... + +Let's explore the Model Instance API directly using this shell: + +:: + + (djangoenv)$ ./manage.py shell + +Instances of our model can be created by simple instantiation: + +.. code-block:: ipython + + In [1]: from myblog.models import Post + In [2]: p1 = Post(title='My First Post', + ...: text='This is the first post I\'ve written') + In [3]: p1 + Out[3]: + +We can also validate that our new object is okay before we try to save it: + +.. code-block:: ipython + + In [4]: p1.full_clean() + ... + + ValidationError: {'author': ['This field cannot be null.']} + + +Django Model Managers +--------------------- + +We have to hook our ``Post`` to an author, which must be a ``User``. To do +this, we need to have an instance of the ``User`` class. We can use the +``User`` *model manager* to run table-level operations like ``SELECT``. + + +All Django models have a *manager*. By default it is accessed through the +``objects`` class attribute. + +Let's use the *manager* to get an instance of the ``User`` class: + +.. code-block:: ipython + + In [5]: from django.contrib.auth.models import User + In [6]: all_users = User.objects.all() + In [7]: all_users + Out[7]: [] + In [8]: p1.author = all_users[0] + +And now our instance should validate properly: + +.. code-block:: ipython + + In [9]: p1.full_clean() + In [10]: + + +Saving New Objects +------------------ + +Our model has three date fields, two of which are supposed to be +auto-populated: + +.. code-block:: ipython + + In [11]: print(p1.created_date) + None + In [12]: print(p1.modified_date) + None + +Although we've instantiated a Post object, it doesn't have these values yet. +That's because a model is not *created* until it's saved into the database. +When we save our post, these fields will get values assigned: + +.. code-block:: ipython + + In [13]: p1.save() + In [14]: print(p1.created_date) + 2015-12-31 19:24:29.019293+00:00 + In [15]: print(p1.modified_date) + 2015-12-31 19:24:29.019532+00:00 + + +Updating An Instance +-------------------- + +Models operate much like 'normal' python objects. To change the value of a +field, simply set the instance attribute to a new value. Call ``save()`` to +persist the change: + +.. code-block:: ipython + + In [16]: p1.title = p1.title + " (updated)" + In [17]: p1.save() + In [18]: p1.title + Out[18]: 'My First Post (updated)' + + +Create a Few Posts +------------------ + +Let's create a few more posts so we can explore the Django model manager query +API: + +.. code-block:: ipython + + In [20]: p2 = Post(title="Another post", + ....: text="The second one created", + ....: author=all_users[0]).save() + In [21]: p3 = Post(title="The third one", + ....: text="With the word 'heffalump'", + ....: author=all_users[0]).save() + In [22]: p4 = Post(title="Posters are a great decoration", + ....: text="When you are a poor college student", + ....: author=all_users[0]).save() + ....: Post.objects.count() + Out[22]: 4 + + +The Django Query API +-------------------- + +The *manager* on each model class supports a full-featured query API. API +methods take keyword arguments, where the keywords are special constructions +combining field names with field *lookups*. The double-underscore character +separates the name of a field from the *lookup* value. + +.. rst-class:: build small + +* title__exact="The exact title" +* text__contains="decoration" +* id__in=range(1,4) +* published_date__lte=datetime.datetime.now() + +Each keyword argument adds to the query that will be used to find matching +objects. + + +QuerySets +--------- + +A ``QuerySet`` is a special type of object that maintains a relationship to the +database. Query API methods can be divided into two basic groups: methods that +return ``QuerySets`` and those that do not. + +The former may be chained without hitting the database: + +.. code-block:: ipython + + In [24]: a = Post.objects.all() #<-- no query yet + In [25]: b = a.filter(title__icontains="post") #<- not yet + In [26]: c = b.exclude(text__contains="created") #<-- nope + In [27]: [(p.title, p.text) for p in c] #<-- This will issue the query + Out[27]: + [('My First Post (updated)', "This is the first post I've written"), + ('Posters are a great decoration', 'When you are a poor college student')] + +Conversely, the latter will issue an SQL query when executed. + +.. code-block:: ipython + + In [28]: a.count() #<-- immediately executes an SQL query + Out[28]: 4 + + +QuerySets and SQL +----------------- + +If you are curious, you can see the SQL that a given QuerySet will use: + +.. code-block:: ipython + + In [29]: print(c.query) + SELECT "myblog_post"."id", "myblog_post"."title", "myblog_post"."text", + "myblog_post"."author_id", "myblog_post"."created_date", + "myblog_post"."modified_date", "myblog_post"."published_date" + FROM "myblog_post" + WHERE ( + "myblog_post"."title" LIKE %post% ESCAPE '\' + AND NOT ("myblog_post"."text" LIKE %created% ESCAPE '\') + ) + +The SQL will vary depending on which DBAPI backend you use (yay ORM!!!) + +.. note:: Incidentally, using this as a way to learn SQL is not a bad idea. + + +Exploring the QuerySet API +-------------------------- + +See https://docs.djangoproject.com/en/1.9/ref/models/querysets + + +.. code-block:: ipython + + In [3]: [p.pk for p in Post.objects.all().order_by('created_date')] + Out[3]: [1, 2, 3, 4] + In [4]: [p.pk for p in Post.objects.all().order_by('-created_date')] + Out[4]: [4, 3, 2, 1] + In [5]: [p.pk for p in Post.objects.filter(title__contains='post')] + Out[5]: [1, 2, 4] + In [6]: [p.pk for p in Post.objects.exclude(title__contains='post')] + Out[6]: [3] + In [7]: qs = Post.objects.exclude(title__contains='post') + In [8]: qs = qs.exclude(id__exact=3) + In [9]: [p.pk for p in qs] + Out[9]: [] + In [10]: qs = Post.objects.exclude(title__contains='post', id__exact=3) + In [11]: [p.pk for p in qs] + Out[11]: [1, 2, 3, 4] + +Do all of those make sense to you? Especially consider the difference between +those last two results? Can you explain that? + + +Updating via QuerySets +---------------------- + +You can update all the objects in a QuerySet at the same time. Changes are persisted +without calling the ``save`` instance method: + +.. code-block:: ipython + + In [12]: qs = Post.objects.all() + In [13]: [p.published_date for p in qs] + Out[13]: [None, None, None, None] + In [14]: from datetime import datetime + In [15]: from django.utils.timezone import UTC + In [16]: utc = UTC() + In [17]: now = datetime.now(utc) + In [18]: qs.update(published_date=now) + Out[18]: 4 + In [19]: [p.published_date for p in qs] + Out[19]: + [datetime.datetime(2015, 12, 31, 19, 50, 4, 99980, tzinfo=), + datetime.datetime(2015, 12, 31, 19, 50, 4, 99980, tzinfo=), + datetime.datetime(2015, 12, 31, 19, 50, 4, 99980, tzinfo=), + datetime.datetime(2015, 12, 31, 19, 50, 4, 99980, tzinfo=)] + + +Testing Our Model +================= + +As with any project, we want to test our work. Django provides a testing +framework to allow this. Django supports both *unit tests* and *doctests*. I +strongly suggest using *unit tests*. You add tests for your *app* to the file +``tests.py``, which should be at the same package level as ``models.py``. + +Locate and open this file in your editor. + + +Django TestCase Classes +----------------------- + +**SimpleTestCase** is for basic unit testing with no ORM requirements + +**TransactionTestCase** is useful if you need to test transactional +actions (commit and rollback) in the ORM + +**TestCase** is used when you require ORM access and a test client + +**LiveServerTestCase** launches the django server during test runs for +front-end acceptance tests. + +Sometimes testing requires base data to be present. We need a User for ours. +Django provides *fixtures* to handle this need. Create a directory called +``fixtures`` inside your ``myblog`` app directory. This new folder should be +adjacent to the ``tests.py`` file. + +.. rst-class:: build + +Copy the file ``myblog_test_fixture.json`` from the ``resources/session08`` +into this directory, it contains users for our tests. + +Now that we have a fixture, we need to instruct our tests to use it. + +Edit ``tests.py`` to look like this: + +.. code-block:: python + + + from django.test import TestCase + from django.contrib.auth.models import User + + class PostTestCase(TestCase): + fixtures = ['myblog_test_fixture.json', ] + + def setUp(self): + self.user = User.objects.get(pk=1) + + +Our First Enhancement +--------------------- + +Look at the way our Post represents itself in the Django shell: + +.. code-block:: python + + In [2]: [p for p in Post.objects.all()] + Out[2]: + [, + , + , + ] + +Wouldn't it be nice if the posts showed their titles instead? In Django, the +``__str__`` method is used to determine how a Model instance represents +itself. Then, calling ``str(instance)`` gives the desired result. + +Let's write a test that demonstrates our desired outcome: + +.. code-block:: python + + # add this import at the top + from myblog.models import Post + + # and this test method to the PostTestCase + def test_string_representation(self): + expected = "This is a title" + p1 = Post(title=expected) + actual = str(p1) + self.assertEqual(expected, actual) + + +To run tests, use the ``test`` management command. Without arguments, it will +run all TestCases it finds in all installed *apps*. You can pass the name of a +single app to focus on those tests. + +Quit your Django shell and in your terminal run the test we wrote: + +.. code-block:: bash + + (djangoenv)$ ./manage.py test myblog + +We have yet to implement this enhancement, so our test should fail: + +:: + + Creating test database for alias 'default'... + F + ====================================================================== + FAIL: test_string_representation (myblog.tests.PostTestCase) + ---------------------------------------------------------------------- + Traceback (most recent call last): + File "/Users/cewing/projects/training/uw_pce/training.python_web/scripts/session07/mysite/myblog/tests.py", line 15, in test_string_representation + self.assertEqual(expected, actual) + AssertionError: 'This is a title' != u'Post object' + + ---------------------------------------------------------------------- + Ran 1 test in 0.007s + + FAILED (failures=1) + Destroying test database for alias 'default'... + +Let's add an appropriate ``__str__`` method to our Post class. + +* It will take ``self`` as its only argument +* And it should return its own title as the result +* Go ahead and take a stab at this in ``models.py`` + +.. code-block:: python + + class Post(models.Model): + #... + + def __str__(self): + return self.title + +Re-run the tests to see if that worked:: + + (djangoenv)$ ./manage.py test myblog + Creating test database for alias 'default'... + . + ---------------------------------------------------------------------- + Ran 1 test in 0.007s + + OK + Destroying test database for alias 'default'... + +.. rst-class:: centered + +**YIPEEEE!** + + +What to Test +------------ + +In any framework, the question arises of what to test. Much of your app's +functionality is provided by framework tools. Does that need testing? I +*usually* don't write tests covering features provided directly by the +framework. I *do* write tests for functionality I add, and for places where I +make changes to how the default functionality works. This is largely a matter +of style and taste (and of budget). + +We've only begun to test our blog app. We'll be adding many more tests later. +In between, you might want to take a look at the `Django testing documentation`_: + +.. _Django testing documentation: https://docs.djangoproject.com/en/1.9/topics/testing/ + + +The Django Admin +================ + +There are some who believe that Django has been Python's *killer app*. And +without doubt the Django Admin is a *killer feature* for Django. To demonstrate +this, we are going to set up the admin for our blog + +The Django Admin is, itself, an *app*, installed by default (as of 1.6). Open +the ``settings.py`` file from our ``mysite`` project package and verify that +you see it in the list: + +.. code-block:: python + + INSTALLED_APPS = ( + 'django.contrib.admin', # <- already present + # ... + ) + +What we need now is to allow the admin to be seen through a web browser. To do +that, we'll have to add some URLs to our project. + + +Django URL Resolution +--------------------- + +Like Pyramid, Django has a system for dispatching requests to code: the *urlconf*. + +* A urlconf is an iterable of calls to the ``django.conf.urls.url`` function +* This function takes: + + * a regexp *rule*, representing the URL + * a ``callable`` to be invoked (or a name identifying one) + * an optional *name* kwarg, used to *reverse* the URL + * other optional arguments we will skip for now + +* The function returns a *resolver* that matches the request path to the + callable + +I said above that a urlconf is an iterable. That iterable is generally built by +calling the ``django.conf.urls.patterns`` function. It's best to build it that +way, but in reality, any iterable will do. + +However, the name you give this iterable is **not flexible**. Django will load +the urlconf named ``urlpatterns`` that it finds in the file named in +``settings.ROOT_URLCONF``. + +Many Django add-on *apps*, like the Django Admin, come with their own urlconf. +It is standard to include these urlconfs by rooting them at some path in your +site. + +You can do this by using the ``django.conf.urls.include`` function as the +callable in a ``url`` call: + +.. code-block:: python + + url(r'^forum/', include('random.forum.app.urls')) + + +Including the Admin +------------------- + +We can use this to add *all* the URLs provided by the Django admin in one +stroke. + + verify the following lines in ``urls.py``: + + .. code-block:: python + + + from django.contrib import admin # <- should be present already + + urlpatterns = [ + ... + url(r'^admin/', include(admin.site.urls)), #<- this should be too + ] + +We can now view the admin. We'll use the Django development server. + +.. rst-class:: build + +In your terminal, use the ``runserver`` management command to start the +development server: + +.. rst-class:: build + +:: + + (djangoenv)$ ./manage.py runserver + Validating models... + + 0 errors found + Django version 1.4.3, using settings 'mysite.settings' + Development server is running at http://127.0.0.1:8000/ + Quit the server with CONTROL-C. + + +Viewing the Admin +----------------- + +Load ``http://localhost:8000/admin/``. You should see this: + +.. figure:: /_static/django-admin-login.png + :align: center + :width: 50% + +.. rst-class:: build + +Login with the name and password you created before. + + +The Admin Index +--------------- + +The index will provide a list of all the installed *apps* and each model +registered. You should see this: + +.. image:: /_static/admin_index.png + :align: center + :width: 90% + +.. rst-class:: build + +Click on ``Users``. Find yourself? Edit yourself, but **don't** uncheck +``superuser``. + + +Add Posts to the Admin +---------------------- + +Okay, let's add our app model to the admin. Find the ``admin.py`` file in the +``myblog`` package. Open it, add the following and save the file: + +.. code-block:: python + + from django.contrib import admin # <- this is already there. + from myblog.models import Post + + admin.site.register(Post) + +Reload the admin index page in your browser. You should now see a listing for +the Myblog app, and an entry for Posts. + +Visit the admin page for Posts. You should see the posts we created earlier in +the Django shell. Look at the listing of Posts. Because of our ``__str__`` +method we see a nice title. + +Are there other fields you'd like to see listed? Click on a Post, note what is +and is not shown. + + +Next Steps +---------- + +We've learned a great deal about Django's ORM and Models. We've also spent some +time getting to know the Query API provided by model managers and QuerySets. + +We've also hooked up the Django Admin and noted some shortcomings. + +In class we'll learn how to put a front end on this, add new models, and +customize the admin experience. + + diff --git a/source/presentations/django_intro.rst.norender b/source/presentations/django_intro.rst.norender deleted file mode 100644 index ea57c5c8..00000000 --- a/source/presentations/django_intro.rst.norender +++ /dev/null @@ -1,1449 +0,0 @@ -************************* -An Introduction To Django -************************* - -In this tutorial, you'll walk through creating a very simple microblog -application using Django. - -Practice Safe Development -------------------------- - -We'll install Django and any other packages we use with it in a virtualenv. - -.. class:: incremental - -This will ensure that it is isolated from everything else we do in class (and -vice versa) - -.. container:: incremental - - Remember the basic format for creating a virtualenv: - - .. class:: small - - :: - - $ python virtualenv.py [options] - - $ virtualenv [options] - - -Set Up a VirtualEnv -------------------- - -Start by creating your virtualenv:: - - $ python virtualenv.py djangoenv - - $ virtualenv djangoenv - ... - -.. container:: incremental - - Then, activate it:: - - $ source djangoenv/bin/activate - - C:\> djangoenv\Scripts\activate - - -Install Django --------------- - -Finally, install Django 1.6.2 using ``pip``: - -.. class:: small - -:: - - (djangoenv)$ pip install Django==1.6.2 - Downloading/unpacking Django==1.5.2 - Downloading Django-1.6.2.tar.gz (8.0MB): 8.0MB downloaded - Running setup.py egg_info for package Django - changing mode of /path/to/djangoenv/bin/django-admin.py to 755 - Successfully installed Django - Cleaning up... - (djangoenv)$ - - -Starting a Project ------------------- - -Everything in Django stems from the *project* - -.. class:: incremental - -To get started learning, we'll create one - -.. class:: incremental - -We'll use a script installed by Django, ``django-admin.py``: - -.. code-block:: - :class: incremental - - (djangoenv)$ django-admin.py startproject mysite - -.. class:: incremental - -This will create a folder called 'mysite'. Let's take a look at it: - - -Project Layout --------------- - -The folder created by ``django-admin.py`` contains the following structure: - -.. code-block:: - - mysite - ├── manage.py - └── mysite - ├── __init__.py - ├── settings.py - ├── urls.py - └── wsgi.py - -.. class:: incremental - -If what you see doesn't match that, you're using an older version of Django. -Make sure you've installed 1.6.2. - - -What Got Created ----------------- - -.. class:: incremental - -* **outer *mysite* folder**: this is just a container and can be renamed or - moved at will -* **inner *mysite* folder**: this is your project directory. It should not be - renamed. -* **__init__.py**: magic file that makes *mysite* a python package. -* **settings.py**: file which holds configuration for your project, more soon. -* **urls.py**: file which holds top-level URL configuration for your project, - more soon. -* **wsgi.py**: binds a wsgi application created from your project to the - symbol ``application`` -* **manage.py**: a management control script. - - -*django-admin.py* and *manage.py* ---------------------------------- - -*django-admin.py* provides a hook for administrative tasks and abilities: - -.. class:: incremental - -* creating a new project or app -* running the development server -* executing tests -* entering a python interpreter -* entering a database shell session with your database -* much much more (run ``django-admin.py`` without an argument) - -.. class:: incremental - -*manage.py* wraps this functionality, adding the full environment of your -project. - - -How *manage.py* Works ---------------------- - -Look in the ``manage.py`` script Django created for you. You'll see this: - -.. code-block:: python - :class: small - - #!/usr/bin/env python - import os - import sys - - if __name__ == "__main__": - os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings") - ... - -.. class:: incremental - -The environmental var ``DJANGO_SETTINGS_MODULE`` is how the ``manage.py`` -script is made aware of your project's environment. This is why you shouldn't -rename the project package. - - -Development Server ------------------- - -At this point, you should be ready to use the development server:: - - (djangoenv)$ cd mysite - (djangoenv)$ python manage.py runserver - ... - -.. class:: incremental - -Load ``http://localhost:8000`` in your browser. - - -A Blank Slate -------------- - -You should see this: - -.. image:: img/django-start.png - :align: center - :width: 98% - -.. class:: incremental center - -**Do you?** - - -Connecting A Database ---------------------- - -Django supplies its own ORM (Object-Relational Mapper) - -.. class:: incremental - -This ORM sits on top of the DB-API implementation you choose. - -.. class:: incremental - -You must provide connection information through Django configuration. - -.. class:: incremental - -All Django configuration takes place in ``settings.py`` in your project -folder. - - -Your Database Settings ----------------------- - -Edit your ``settings.py`` to match: - -.. code-block:: python - :class: small - - DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.sqlite3', - 'NAME': 'mysite.db', - } - } - -.. class:: incremental - -There are other database settings, but they are not used with sqlite3, we'll -ignore them for now. - - -Django and Your Database ------------------------- - -Django's ORM provides a layer of *abstraction* between you and SQL - -.. class:: incremental - -You write Python classes called *models* describing the objects that make up -your system. - -.. class:: incremental - -The ORM handles converting data from these objects into SQL statements (and -back) - -.. class:: incremental - -We'll learn much more about this in a bit - - -Django Organization -------------------- - -We've created a Django *project*. In Django a project represents a whole -website: - -.. class:: incremental - -* global configuration settings -* inclusion points for additional functionality -* master list of URL endpoints - -.. class:: incremental - -A Django *app* encapsulates a unit of functionality: - -.. class:: incremental - -* A blog section -* A discussion forum -* A content tagging system - - -Apps Make Up a Project ----------------------- - -.. class:: big-centered - -One *project* can (and likely will) consist of many *apps* - - -Core Django *Apps* ------------------- - -Django already includes some *apps* for you. - -.. container:: incremental - - They're in ``settings.py`` in the ``INSTALLED_APPS`` setting: - - .. code-block:: python - :class: small - - INSTALLED_APPS = ( - 'django.contrib.admin', - 'django.contrib.auth', - 'django.contrib.contenttypes', - 'django.contrib.sessions', - 'django.contrib.messages', - 'django.contrib.staticfiles', - ) - - -Creating the Database ---------------------- - -These *apps* define models of their own, tables must be created. - -.. container:: incremental - - You make them by running the ``syncdb`` management command: - - .. class:: small - - :: - - (djangoenv)$ python manage.py syncdb - Creating tables ... - Creating table auth_permission - Creating table auth_group_permissions - Creating table auth_group - ... - You just installed Django's auth system, ... - Would you like to create one now? (yes/no): - -.. class:: incremental - -Add your first user at this prompt. I strongly suggest you use the username -'admin' and give it the password 'admin'. If you don't, make sure you remember -the values you use. - - -Our Class App -------------- - -We are going to build an *app* to add to our *project*. To start with our app -will be a lot like the Flask app we finished last time. - -.. class:: incremental - -As stated above, an *app* represents a unit within a system, the *project*. We -have a project, we need to create an *app* - - -Create an App -------------- - -This is accomplished using ``manage.py``. - -.. class:: incremental - -In your terminal, make sure you are in the *outer* mysite directory, where the -file ``manage.py`` is located. Then: - -.. class:: incremental - -:: - - (djangoenv)$ python manage.py startapp myblog - - -What is Created ---------------- - -This should leave you with the following structure: - -.. class:: small - -:: - - mysite - ├── manage.py - ├── myblog - │   ├── __init__.py - │   ├── admin.py - │   ├── models.py - │   ├── tests.py - │   └── views.py - └── mysite - ├── __init__.py - ... - -.. class:: incremental - -We'll start by defining the main Python class for our blog system, a ``Post``. - - -Django Models -------------- - -Any Python class in Django that is meant to be persisted *must* inherit from -the Django ``Model`` class. - -.. class:: incremental - -This base class hooks in to the ORM functionality converting Python code to -SQL. - -.. class:: incremental - -You can override methods from the base ``Model`` class to alter how this works -or write new methods to add functionality. - -.. class:: incremental - -Learn more about `models -`_ - - -Our Post Model --------------- - -Open the ``models.py`` file created in our ``myblog`` package. Add the -following: - -.. code-block:: python - :class: small - - from django.db import models #<-- This is already in the file - from django.contrib.auth.models import User - - class Post(models.Model): - title = models.CharField(max_length=128) - text = models.TextField(blank=True) - author = models.ForeignKey(User) - created_date = models.DateTimeField(auto_now_add=True) - modified_date = models.DateTimeField(auto_now=True) - published_date = models.DateTimeField(blank=True, null=True) - - -Model Fields ------------- - -We've created a subclass of the Django ``Model`` class and added a bunch of -attributes. - -.. class:: incremental - -* These attributes are all instances of ``Field`` classes defined in Django -* Field attributes on a model map to columns in a database table -* The arguments you provide to each Field customize how it works - - * This means *both* how it operates in Django *and* how it is defined in SQL - -* There are arguments shared by all Field types -* There are also arguments specific to individual types - -.. class:: incremental - -You can read much more about `Model Fields and options -`_ - - -Field Details -------------- - -There are some features of our fields worth mentioning in specific: - -.. class:: incremental - -Notice we have no field that is designated as the *primary key* - -.. class:: incremental - -* You *can* make a field the primary key by adding ``primary_key=True`` in the - arguments -* If you do not, Django will **automatically** create one. This field is always - called ``id`` -* No matter what the primary key field is called, its value is always - available on a model instance as the ``pk`` attribute. - - -Field Details -------------- - -.. code-block:: python - :class: small - - title = models.CharField(max_length=128) - -.. class:: incremental - -The required ``max_length`` argument is specific to ``CharField`` fields. - -.. class:: incremental - -It affects *both* the Python and SQL behavior of a field. - -.. class:: incremental - -In python, it is used to *validate* supplied values during *model validation* - -.. class:: incremental - -In SQL it is used in the column definition: ``VARCHAR(128)`` - - -Field Details -------------- - -.. code-block:: python - :class: small - - author = models.ForeignKey(User) - -.. class:: incremental - -Django also models SQL *relationships* as specific field types. - -.. class:: incremental - -The required positional argument is the class of the related Model. - -.. class:: incremental - -By default, the reverse relation is implemented as the attribute -``_set``. - -.. class:: incremental - -You can override this by providing the ``related_name`` argument. - - -Field Details -------------- - -.. code-block:: python - :class: small - - created_date = models.DateTimeField(auto_now_add=True) - modified_date = models.DateTimeField(auto_now=True) - -.. class:: incremental - -``auto_now_add`` is available on all date and time fields. It sets the value -of the field to *now* when an instance is first saved. - -.. class:: incremental - -``auto_now`` is similar, but sets the value anew each time an instance is -saved. - -.. class:: incremental - -Setting either of these will cause the ``editable`` attribute of a field to be -set to ``False``. - - -Field Details -------------- - -.. code-block:: python - :class: small - - text = models.TextField(blank=True) - # ... - published_date = models.DateTimeField(blank=True, null=True) - -.. class:: incremental - -The argument ``blank`` is shared across all field types. The default is -``False`` - -.. class:: incremental - -This argument affects only the Python behavior of a field, determining if the -field is *required* - -.. class:: incremental - -The related ``null`` argument affects the SQL definition of a field: is the -column NULL or NOT NULL - - -Hooking it Up -------------- - -In order to use our new model, we need Django to know about our *app* - -.. class:: incremental - -This is accomplished by configuration in the ``settings.py`` file. - -.. class:: incremental - -Open that file now, in your editor, and find the INSTALLED_APPS setting. - - -Installing Apps ---------------- - -You extend Django functionality by *installing apps*. This is pretty simple: - -.. code-block:: python - :class: small - - INSTALLED_APPS = ( - 'django.contrib.admin', - 'django.contrib.auth', - 'django.contrib.contenttypes', - 'django.contrib.sessions', - 'django.contrib.messages', - 'django.contrib.staticfiles', - 'myblog', # <- YOU ADD THIS PART - ) - - -Setting Up the Database ------------------------ - -You know what the next step will be: - -.. code-block:: - :class: incremental - - (djangoenv)$ python manage.py syncdb - Creating tables ... - Creating table myblog_post - Installing custom SQL ... - Installing indexes ... - Installed 0 object(s) from 0 fixture(s) - -.. class:: incremental - -Django has now created a table for our model. - -.. class:: incremental - -Notice that the table name is a combination of the name of our app *and* the -name of our model. - - -The Django Shell ----------------- - -Django provides a management command ``shell``: - -.. class:: incremental - -* Shares the same ``sys.path`` as your project, so all installed python - packages are present. -* Imports the ``settings.py`` file from your project, and so shares all - installed apps and other settings. -* Handles connections to your database, so you can interact with live data - directly. - -.. class:: incremental - -Let's explore the Model Instance API directly using this shell: - -.. class:: incremental - -:: - - (djangoenv)$ python manage.py shell - - -Creating Instances ------------------- - -Instances of our model can be created by simple instantiation: - -.. code-block:: python - :class: small - - >>> from myblog.models import Post - >>> p1 = Post(title="My first post", - ... text="This is the first post I've written") - >>> p1 - - -.. container:: incremental - - We can also validate that our new object is okay before we try to save it: - - .. code-block:: python - :class: small - - >>> p1.full_clean() - Traceback (most recent call last): - ... - ValidationError: {'author': [u'This field cannot be null.']} - - -Django Model Managers ---------------------- - -We have to hook our ``Post`` to an author, which must be a ``User``. - -.. class:: incremental - -To do this, we need to have an instance of the ``User`` class. - -.. class:: incremental - -We can use the ``User`` *model manager* to run table-level operations like -``SELECT``: - -.. class:: incremental - -All Django models have a *manager*. By default it is accessed through the -``objects`` class attribute. - - -Making a ForeignKey Relation ----------------------------- - -Let's use the *manager* to get an instance of the ``User`` class: - -.. code-block:: python - :class: small - - >>> from django.contrib.auth.models import User - >>> all_users = User.objects.all() - >>> all_users - [] - >>> u1 = all_users[0] - >>> p1.author = u1 - -.. container:: incremental - - And now our instance should validate properly: - - .. code-block:: python - :class: small - - >>> p1.full_clean() - >>> - - -Saving New Objects ------------------- - -Our model has three date fields, two of which are supposed to be -auto-populated: - -.. class:: python - :class: small - - >>> print(p1.created_date) - None - >>> print(p1.modified_date) - None - -.. container:: incremental - - When we save our post, these fields will get values assigned: - - .. code-block:: python - :class: small - - >>> p1.save() - >>> p1.created_date - datetime.datetime(2013, 7, 26, 20, 2, 38, 104217, tzinfo=) - >>> p1.modified_date - datetime.datetime(2013, 7, 26, 20, 2, 38, 104826, tzinfo=) - - -Updating An Instance --------------------- - -Models operate much like 'normal' python objects. - -.. container:: incremental - - To change the value of a field, simply set the instance attribute to a new - value. Call ``save()`` to persist the change: - - .. code-block:: python - :class: small - - >>> p1.title = p1.title + " (updated)" - >>> p1.save() - >>> p1.title - 'My first post (updated)' - - -Create a Few Posts ------------------- - -Let's create a few more posts so we can explore the Django model manager query -API: - -.. code-block:: python - :class: small - - >>> p2 = Post(title="Another post", - ... text="The second one created", - ... author=u1).save() - >>> p3 = Post(title="The third one", - ... text="With the word 'heffalump'", - ... author=u1).save() - >>> p4 = Post(title="Posters are great decoration", - ... text="When you are a poor college student", - ... author=u1).save() - >>> Post.objects.count() - 4 - - -The Django Query API --------------------- - -The *manager* on each model class supports a full-featured query API. - -.. class:: incremental - -API methods take keyword arguments, where the keywords are special -constructions combining field names with field *lookups*: - -.. class:: incremental small - -* title__exact="The exact title" -* text__contains="decoration" -* id__in=range(1,4) -* published_date__lte=datetime.datetime.now() - -.. class:: incremental - -Each keyword argument generates an SQL clause. - - -QuerySets ---------- - -API methods can be divided into two basic groups: methods that return -``QuerySets`` and those that do not. - -.. class:: incremental - -The former may be chained without hitting the database: - -.. code-block:: python - :class: small incremental - - >>> a = Post.objects.all() #<-- no query yet - >>> b = a.filter(title__icontains="post") #<-- not yet - >>> c = b.exclude(text__contains="created") #<-- nope - >>> [(p.title, p.text) for p in c] #<-- This will issue the query - -.. container:: incremental - - Conversely, the latter will issue an SQL query when executed. - - .. code-block:: python - :class: small - - >>> a.count() # immediately executes an SQL query - - -QuerySets and SQL ------------------ - -If you are curious, you can see the SQL that a given QuerySet will use: - -.. code-block:: python - :class: small incremental - - >>> print(c.query) - SELECT "myblog_post"."id", "myblog_post"."title", - "myblog_post"."text", "myblog_post"."author_id", - "myblog_post"."created_date", "myblog_post"."modified_date", - "myblog_post"."published_date" - FROM "myblog_post" - WHERE ("myblog_post"."title" LIKE %post% ESCAPE '\' - AND NOT ("myblog_post"."text" LIKE %created% ESCAPE '\' ) - ) - -.. class:: incremental - -The SQL will vary depending on which DBAPI backend you use (yay ORM!!!) - - -Exploring the QuerySet API --------------------------- - -See https://docs.djangoproject.com/en/1.6/ref/models/querysets - - -.. code-block:: python - :class: small - - >>> [p.pk for p in Post.objects.all().order_by('created_date')] - [1, 2, 3, 4] - >>> [p.pk for p in Post.objects.all().order_by('-created_date')] - [4, 3, 2, 1] - >>> [p.pk for p in Post.objects.filter(title__contains='post')] - [1, 2, 4] - >>> [p.pk for p in Post.objects.exclude(title__contains='post')] - [3] - >>> qs = Post.objects.exclude(title__contains='post') - >>> qs = qs.exclude(id__exact=3) - >>> [p.pk for p in qs] - [] - >>> qs = Post.objects.exclude(title__contains='post', id__exact=3) - >>> [p.pk for p in qs] - [1, 2, 3, 4] - - -Updating via QuerySets ----------------------- - -You can update all selected objects at the same time. - -.. class:: incremental - -Changes are persisted without needing to call ``save``. - -.. code-block:: python - :class: small incremental - - >>> qs = Post.objects.all() - >>> [p.published_date for p in qs] - [None, None, None, None] - >>> from datetime import datetime - >>> from django.utils.timezone import UTC - >>> utc = UTC() - >>> now = datetime.now(utc) - >>> qs.update(published_date=now) - 4 - >>> [p.published_date for p in qs] - [datetime.datetime(2013, 7, 27, 1, 20, 30, 505307, tzinfo=), - ...] - - -Testing Our Model ------------------ - -As with any project, we want to test our work. Django provides a testing -framework to allow this. - -.. class:: incremental - -Django supports both *unit tests* and *doctests*. I strongly suggest using -*unit tests*. - -.. class:: incremental - -You add tests for your *app* to the file ``tests.py``, which should be at the -same package level as ``models.py``. - -.. class:: incremental - -Locate and open this file in your editor. - - -Django TestCase Classes ------------------------ - -**SimpleTestCase** is for basic unit testing with no ORM requirements - -.. class:: incremental - -**TransactionTestCase** is useful if you need to test transactional -actions (commit and rollback) in the ORM - -.. class:: incremental - -**TestCase** is used when you require ORM access and a test client - -.. class:: incremental - -**LiveServerTestCase** launches the django server during test runs for -front-end acceptance tests. - - -Testing Data ------------- - -Sometimes testing requires base data to be present. We need a User for ours. - -.. class:: incremental - -Django provides *fixtures* to handle this need. - -.. class:: incremental - -Create a directory called ``fixtures`` inside your ``myblog`` app directory. - -.. class:: incremental - -Copy the file ``myblog_test_fixture.json`` from the class resources into this -directory, it contains users for our tests. - - -Setting Up Tests ----------------- - -Now that we have a fixture, we need to instruct our tests to use it. - -.. container:: incremental - - Edit ``tests.py`` to look like this: - - .. code-block:: python - :class: small - - from django.test import TestCase - from django.contrib.auth.models import User - - class PostTestCase(TestCase): - fixtures = ['myblog_test_fixture.json', ] - - def setUp(self): - self.user = User.objects.get(pk=1) - - -Our First Enhancement ---------------------- - -Look at the way our Post represents itself in the Django shell: - -.. code-block:: python - :class: small - - >>> [p for p in Post.objects.all()] - [, , - , ] - -.. class:: incremental - -Wouldn't it be nice if the posts showed their titles instead? - -.. class:: incremental - -In Django, the ``__unicode__`` method is used to determine how a Model -instance represents itself. - -.. class:: incremental - -Then, calling ``unicode(instance)`` gives the desired result. - - -Write The Test --------------- - -Let's write a test that demonstrates our desired outcome: - -.. code-block:: python - :class: small - - # add this import at the top - from myblog.models import Post - - # and this test method to the PostTestCase - def test_unicode(self): - expected = "This is a title" - p1 = Post(title=expected) - actual = unicode(p1) - self.assertEqual(expected, actual) - - -Run The Test ------------- - -To run tests, use the ``test`` management command - -.. class:: incremental - -Without arguments, it will run all TestCases it finds in all installed *apps* - -.. class:: incremental - -You can pass the name of a single app to focus on those tests - -.. class:: incremental - -Quit your Django shell and in your terminal run the test we wrote: - -.. code-block:: bash - :class: small incremental - - (djangoenv)$ python manage.py test myblog - - -The Result ----------- - -We have yet to implement this enhancement, so our test should fail: - -.. class:: small - -:: - - Creating test database for alias 'default'... - F - ====================================================================== - FAIL: test_unicode (myblog.tests.PostTestCase) - ---------------------------------------------------------------------- - Traceback (most recent call last): - File "/Users/cewing/projects/training/uw_pce/training.python_web/scripts/session07/mysite/myblog/tests.py", line 15, in test_unicode - self.assertEqual(expected, actual) - AssertionError: 'This is a title' != u'Post object' - - ---------------------------------------------------------------------- - Ran 1 test in 0.007s - - FAILED (failures=1) - Destroying test database for alias 'default'... - - -Make it Pass ------------- - -Let's add an appropriate ``__unicode__`` method to our Post class - -.. class:: incremental - -It will take ``self`` as its only argument - -.. class:: incremental - -And it should return its own title as the result - -.. class:: incremental - -Go ahead and take a stab at this in ``models.py`` - -.. code-block:: python - :class: small incremental - - class Post(models.Model): - #... - - def __unicode__(self): - return self.title - - -Did It Work? ------------- - -Re-run the tests to see: - -.. code-block:: bash - :class: small - - (djangoenv)$ python manage.py test myblog - Creating test database for alias 'default'... - . - ---------------------------------------------------------------------- - Ran 1 test in 0.007s - - OK - Destroying test database for alias 'default'... - -.. class:: incremental center - -**YIPEEEE!** - - -What to Test ------------- - -In any framework, the question arises of what to test. Much of your app's -functionality is provided by framework tools. Does that need testing? - -.. class:: incremental - -I *usually* don't write tests covering features provided directly by the -framework. - -.. class:: incremental - -I *do* write tests for functionality I add, and for places where I make -changes to how the default functionality works. - -.. class:: incremental - -This is largely a matter of style and taste (and of budget). - - -More Later ----------- - -We've only begun to test our blog app. - -.. class:: incremental - -We'll be adding many more tests later - -.. class:: incremental - -In between, you might want to take a look at the Django testing documentation: - -.. class:: incremental center - -https://docs.djangoproject.com/en/1.6/topics/testing/ - - -The Django Admin ----------------- - -There are some who believe that Django has been Python's *killer app* - -.. class:: incremental - -And without doubt the Django Admin is a *killer feature* for Django. - -.. class:: incremental - -To demonstrate this, we are going to set up the admin for our blog - - -Using the Admin ---------------- - -The Django Admin is, itself, an *app*, installed by default (as of 1.6). - -.. class:: incremental - -Open the ``settings.py`` file from our ``mysite`` project package and -verify that you see it in the list: - -.. code-block:: python - :class: incremental small - - INSTALLED_APPS = ( - 'django.contrib.admin', # <- already present - # ... - 'django.contrib.staticfiles', # <- already present - 'myblog', # <- already present - ) - - -Accessing the Admin -------------------- - -What we need now is to allow the admin to be seen through a web browser. - -.. class:: incremental - -To do that, we'll have to add some URLs to our project. - - -Django URL Resolution ---------------------- - -Django too has a system for dispatching requests to code: the *urlconf*. - -.. class:: incremental - -* A urlconf is an iterable of calls to the ``django.conf.urls.url`` function -* This function takes: - - * a regexp *rule*, representing the URL - - * a ``callable`` to be invoked (or a name identifying one) - - * an optional *name* kwarg, used to *reverse* the URL - - * other optional arguments we will skip for now - -* The function returns a *resolver* that matches the request path to the - callable - - -*urlpatterns* -------------- - -I said above that a urlconf is an iterable. - -.. class:: incremental - -That iterable is generally built by calling the ``django.conf.urls.patterns`` -function. - -.. class:: incremental - -It's best to build it that way, but in reality, any iterable will do. - -.. class:: incremental - -However, the name you give this iterable is **not flexible**. - -.. class:: incremental - -Django will load the urlconf named ``urlpatterns`` that it finds in the file -named in ``settings.ROOT_URLCONF``. - - -Including URLs --------------- - -Many Django add-on *apps*, like the Django Admin, come with their own urlconf - -.. class:: incremental - -It is standard to include these urlconfs by rooting them at some path in your -site. - -.. container:: incremental - - You can do this by using the ``django.conf.urls.include`` function as the - callable in a ``url`` call: - - .. code-block:: python - :class: small - - url(r'^forum/', include('random.forum.app.urls')) - - -Including the Admin -------------------- - -We can use this to add *all* the URLs provided by the Django admin in one -stroke. - -.. container:: incremental - - verify the following lines in ``urls.py``: - - .. code-block:: python - :class: small - - from django.contrib import admin #<- make sure these two are - admin.autodiscover() #<- present and uncommented - - urlpatterns = patterns('', - ... - url(r'^admin/', include(admin.site.urls)), #<- and this - ) - - -Using the Development Server ----------------------------- - -We can now view the admin. We'll use the Django development server. - -.. class:: incremental - -In your terminal, use the ``runserver`` management command to start the -development server: - -.. class:: incremental - -:: - - (djangoenv)$ python manage.py runserver - Validating models... - - 0 errors found - Django version 1.4.3, using settings 'mysite.settings' - Development server is running at http://127.0.0.1:8000/ - Quit the server with CONTROL-C. - - -Viewing the Admin ------------------ - -Load ``http://localhost:8000/admin/``. You should see this: - -.. image:: img/django-admin-login.png - :align: center - :width: 50% - -.. class:: incremental - -Login with the name and password you created before. - - -The Admin Index ---------------- - -The index will provide a list of all the installed *apps* and each model -registered. You should see this: - -.. image:: img/admin_index.png - :align: center - :width: 90% - -.. class:: incremental - -Click on ``Users``. Find yourself? Edit yourself, but **don't** uncheck -``superuser``. - - -Add Posts to the Admin ----------------------- - -Okay, let's add our app model to the admin. - -.. class:: incremental - -Find the ``admin.py`` file in the ``myblog`` package. Open it, add the -following and save the file: - -.. code-block:: python - :class: incremental - - from django.contrib import admin # <- this is already there. - from myblog.models import Post - - admin.site.register(Post) - -.. class:: incremental - -Reload the admin index page. - - -Play A Bit ----------- - -Visit the admin page for Posts. You should see the posts we created earlier in -the Django shell. - -.. class:: incremental - -Look at the listing of Posts. Because of our ``__unicode__`` method we see a -nice title. - -.. class:: incremental - -Are there other fields you'd like to see listed? - -.. class:: incremental - -Click on a Post, note what is and is not shown. - - -Next Steps ----------- - -We've learned a great deal about Django's ORM and Models. - -.. class:: incremental - -We've also spent some time getting to know the Query API provided by model -managers and QuerySets. - -.. class:: incremental - -We've also hooked up the Django Admin and noted some shortcomings. - -.. class:: incremental - -In class we'll learn how to put a front end on this, add new models, and -customize the admin experience. - - diff --git a/source/presentations/index.rst b/source/presentations/index.rst index 5fae542a..79588fbb 100644 --- a/source/presentations/index.rst +++ b/source/presentations/index.rst @@ -17,3 +17,11 @@ course. session01 session02 + session03 + session04 + session05 + session06 + session07 + session08 + session09 + session10 diff --git a/source/presentations/session01.rst b/source/presentations/session01.rst index d0d5e7be..f28b6346 100644 --- a/source/presentations/session01.rst +++ b/source/presentations/session01.rst @@ -1,22 +1,19 @@ -.. slideconf:: - :autoslides: True - ********** Session 01 ********** -.. image:: /_static/python.png +.. figure:: /_static/python.png :align: center - :width: 43% + :width: 50% + **Networking and Sockets** -Introductions -============= +Computer Communications +======================= .. rst-class:: large centered -Wherin we learn about the Model View Controller approach to app design and -explore data persistence in Python. +Wherein we learn about how computers speak to each-other over a network. But First --------- @@ -94,1711 +91,1207 @@ Please check frequently. I will update with great regularity Introductions -Working with Virtualenv -======================= - -.. rst-class:: large +TCP/IP +------ -| For every -| add-on package installed -| in a system Python, -| the gods kill a kitten -| -| - me +.. figure:: /_static/network_topology.png + :align: left -Why Virtualenv? ---------------- + http://en.wikipedia.org/wiki/Internet_Protocol_Suite .. rst-class:: build -* You will need to install packages that aren't in the Python standard - Library -* You often need to install *different* versions of the *same* library for - different projects -* Conflicts arising from having the wrong version of a dependency installed can - cause long-term nightmares -* Use `virtualenv`_ ... -* **Always** - -.. _virtualenv: http://www.virtualenv.org/ - -Installing Virtualenv ---------------------- - -The best way is to install directly in your system Python (one exception to the -rule). +* processes can communicate +* inside one machine +* between two machines +* among many machines -.. rst-class:: build -.. container:: - To do so you will have to have `pip`_ installed. +.. nextslide:: - Try the following command: +.. figure:: /_static/data_in_tcpip_stack.png + :align: left + :width: 100% - .. code-block:: bash + http://en.wikipedia.org/wiki/Internet_Protocol_Suite - $ which pip - /usr/local/bin/pip +.. rst-class:: build - If the ``which`` command returns no value for you, then ``pip`` is not - installed in your system. To fix this, follow `the instructions here`_. +* Process divided into 'layers' +* 'Layers' are mostly arbitrary +* Different descriptions have different layers +* Most common is the 'TCP/IP Stack' -.. _pip: https://pip.pypa.io/en/latest/index.html -.. _the instructions here: https://pip.pypa.io/en/latest/installing.html -.. nextslide:: +The TCP/IP Stack - Link +----------------------- -Once you have ``pip`` installed in your system, you can use it to install -`virtualenv`_. +The bottom layer is the 'Link Layer' .. rst-class:: build -.. container:: - Because you are installing it into your system python, you will most likely - need ``superuser`` privileges to do so: +* Deals with the physical connections between machines, 'the wire' - .. code-block:: bash +* Packages data for physical transport - $ sudo pip install virtualenv - Downloading/unpacking virtualenv - Downloading virtualenv-1.11.2-py2.py3-none-any.whl (2.8MB): 2.8MB downloaded - Installing collected packages: virtualenv - Successfully installed virtualenv - Cleaning up... +* Executes transmission over a physical medium -.. nextslide:: + .. rst-class:: build -Great. Once that's done, you should find that you have a ``virtualenv`` -command available to you from your shell: + * what that medium is is arbitrary -.. code-block:: bash +* Implemented in the Network Interface Card(s) (NIC) in your computer - $ virtualenv --help - Usage: virtualenv [OPTIONS] DEST_DIR - Options: - --version show program's version number and exit - -h, --help ... +The TCP/IP Stack - Internet +--------------------------- -Using Virtuelenv ----------------- - -Creating a new virtualenv is very very simple: +Moving up, we have the 'Internet Layer' .. rst-class:: build -.. container:: - - .. code-block:: bash - - $ virtualenv [options] +* Deals with addressing and routing - ```` is just the name of the environment you want to create. + .. rst-class:: build - It's arbitrary, so name them to be easily remembered. + * Where are we going and how do we get there? -.. nextslide:: - -Let's make one for demonstration purposes: +* Agnostic as to physical medium (IP over Avian Carrier - IPoAC) -.. code-block:: bash +* Makes no promises of reliability - $ virtualenv demoenv - New python executable in demoenv/bin/python - Installing setuptools, pip...done. +* Two addressing systems + .. rst-class:: build -.. nextslide:: What Happened? + * IPv4 (current, limited '192.168.1.100') -When you ran that command, a couple of things took place: + * IPv6 (future, 3.4 x 10^38 addresses, '2001:0db8:85a3:0042:0000:8a2e:0370:7334') -.. rst-class:: build -* A new directory with your requested name was created -* A new Python executable was created in /bin (/Scripts on Windows) -* The new Python was cloned from your system Python (where virtualenv was - installed) -* The new Python was isolated from any libraries installed in the old Python -* Setuptools was installed so you have ``easy_install`` for this new python -* Pip was installed so you have ``pip`` for this new python - -Activation ----------- - -Every virtualenv you create contains an executable Python command. - -.. rst-class:: build -.. container:: - - If you do a quick check to see which Python executable is found by your - terminal, you'll see that it is not the one: - - .. code-block:: bash - - $ which python - /usr/bin/python +.. nextslide:: - You can execute the new Python by explicitly pointing to it: +.. rst-class:: large center - .. code-block:: bash +That's 4.3 x 10^28 addresses *per person alive today* - $ ./demoenv/bin/python -V - Python 2.7.5 -.. nextslide:: +The TCP/IP Stack - Transport +---------------------------- -But that's tedious and hard to remember. +Next up is the 'Transport Layer' .. rst-class:: build -.. container:: - Instead, ``activate`` your virtualenv using the ``source`` shell command: +* Deals with transmission and reception of data - .. code-block:: bash - - $ source demoenv/bin/activate - (demoenv)$ which python - /Users/cewing/demoenv/bin/python + * error correction, flow control, congestion management - Notice that when a virtualenv is *active* you can see it in your command - prompt. +* Common protocols include TCP & UDP - So long as the virtualenv is *active* the ``python`` executable that will - be used will be the new one in your ``demoenv``. + * TCP: Tranmission Control Protocol -Installing Packages -------------------- + * UDP: User Datagram Protocol -Since ``pip`` is also installed, the ``pip`` that is used to install new -software will also be the one in ``demoenv``. +* Not all Transport Protocols are 'reliable' -.. code-block:: bash + .. rst-class:: build - (demoenv)$ which pip - /Users/cewing/demoenv/bin/pip + * TCP ensures that dropped packets are resent -.. rst-class:: build -.. container:: + * UDP makes no such assurance - This means that using these tools to install packages will install them - *into your virtual environment only* + * Reliability is slow and expensive - The are not installed into the system Python. - - Let's see this in action. .. nextslide:: -We'll install a package called ``docutils`` +The 'Transport Layer' also establishes the concept of a **port** .. rst-class:: build .. container:: - It provides tools for creating documentation using ReStructuredText - - Install it using pip (while your virtualenv is active): - - .. code-block:: bash - - (demoenv)$ pip install docutils - Downloading/unpacking docutils - Downloading docutils-0.11.tar.gz (1.6MB): 1.6MB downloaded - Running setup.py (path:/Users/cewing/demoenv/build/docutils/setup.py) egg_info for package docutils - ... - changing mode of /Users/cewing/demoenv/bin/rst2xml.py to 755 - changing mode of /Users/cewing/demoenv/bin/rstpep2html.py to 755 - Successfully installed docutils - Cleaning up... - -.. nextslide:: - -And now, when we fire up our Python interpreter, the docutils package is -available to us: - -.. code-block:: pycon - - (demoenv)$ python - Python 2.7.5 (default, Aug 25 2013, 00:04:04) - [GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin - Type "help", "copyright", "credits" or "license" for more information. - >>> import docutils - >>> docutils.__path__ - ['/Users/cewing/demoenv/lib/python2.7/site-packages/docutils'] - >>> ^d - (demoenv)$ - -.. nextslide:: Side Effects - -Like some other Python libraries, the ``docutils`` package provides a number of -executable scripts when it is installed. + .. rst-class:: build -.. rst-class:: build -.. container:: + * IP Addresses designate a specific *machine* on the network - You can see these in the ``bin`` directory inside your virtualenv: + * A **port** provides addressing for individual *applications* in a single + host - .. code-block:: bash + * 192.168.1.100:80 (the *:80* part is the **port**) - (demoenv)$ ls ./demoenv/bin - ... - python - rst2html.py - rst2latex.py - ... + * [2001:db8:85a3:8d3:1319:8a2e:370:7348]:443 (*:443* is the **port**) - These scripts are set up to execute using the Python with which they were - built. + This means that you don't have to worry about information intended for your + web browser being accidentally read by your email client. - Running these scripts will use the Python executable in your virtualenv, - *even if that virtualenv is not active*! -Deactivation ------------- +.. nextslide:: -So you've got a virtual environment created and activated so you can work with -it. +There are certain **ports** which are commonly understood to belong to given +applications or protocols: .. rst-class:: build .. container:: - Eventually you'll need to stop working with this ``virtualenv`` and switch - to another - - It's a good idea to keep a separate ``virtualenv`` for every project you - work on. + .. rst-class:: build - When a ``virtualenv`` is active, all you have to do is use the - ``deactivate`` command: + * 80/443 - HTTP/HTTPS + * 20 - FTP + * 22 - SSH + * 23 - Telnet + * 25 - SMTP + * ... - .. code-block:: bash + These ports are often referred to as **well-known ports** - (demoenv)$ deactivate - $ which python - /usr/bin/python + .. rst-class:: small - Note that your shell prompt returns to normal, and now the executable - Python found when you check ``python`` is the system one again. + (see http://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers) -Cleaning Up ------------ +.. nextslide:: -The final advantage that ``virtualenv`` offers you as a developer is -the ability to easily remove a batch of installed Python software from your -system. +Ports are grouped into a few different classes .. rst-class:: build -.. container:: - Consider a situation where you installed a library that breaks your Python - (it happens) +* Ports numbered 0 - 1023 are *reserved* - If you are working in your system Python, you now have to figure out what - that package installed +* Ports numbered 1024 - 65535 are *open* - You have to figure out where it is +* Ports numbered 1024 - 49151 may be *registered* - And you have to go clean it out manually. +* Ports numbered 49152 - 65535 are called *ephemeral* - With ``virtualenv`` you simply remove the directory ``virtualenv`` created - when you started out. -.. nextslide:: +The TCP/IP Stack - Application +------------------------------ -Let's do that with our ``demoenv``: +The topmost layer is the 'Application Layer' .. rst-class:: build .. container:: - .. code-block:: bash - - $ rm -rf demoenv - - And that's it. - - The entire environment and all the packages you installed into it are now - gone. + .. rst-class:: build - There are no traces left to pollute your world. + * Deals directly with data produced or consumed by an application -.. nextslide:: Break Time + * Reads or writes data using a set of understood, well-defined **protocols** -Let's take a moment to rest up and absorb what we've learned. + * HTTP, SMTP, FTP etc. -When we return, we'll begin talking about a particular approach to thinking -about application design: + * Does not know (or need to know) about lower layer functionality -.. rst-class:: centered + * The exception to this rule is **endpoint** data (or IP:Port) -**Model View Controller** + .. rst-class:: centered -MVC Applications -================ + **this is where we live and work** -.. figure:: http://upload.wikimedia.org/wikipedia/commons/4/40/MVC_passive_view.png - :align: center - :width: 50% - By Alan Evangelista (Own work) [CC0], via Wikimedia Commons +Sockets +------- -Separation of Concerns ----------------------- +Think back for a second to what we just finished discussing, the TCP/IP stack. .. rst-class:: build .. container:: - In the first part of this course, you were introduced to the concept of - *Object Oriented Programming* - - OOP was `first formalized`_ in the 1970s in *Smalltalk*, invented by Alan - Kay at *Xerox PARC* - - *Smalltalk* was also the first language which utilized the - `Model View Controller`_ design pattern. - - This pattern (like all `design patterns`_) seeks to provide a way of - thinking that helps to make software design easier. - - In this case, the goal is to help clarify the high-level *separation of - concerns* in a system. - -.. _first formalized: http://en.wikipedia.org/wiki/Object-oriented_programming#History -.. _Model View Controller: http://en.wikipedia.org/wiki/Model–view–controller -.. _design patterns: http://en.wikipedia.org/wiki/Software_design_pattern - -Three Components ----------------- - -The pattern divides the elements of a system into three parts: + .. rst-class:: build -.. rst-class:: build + * The *Internet* layer gives us an **IP Address** -Model: - This component represents the *data* that comprises the system, and the - *logic* used to manipulate that data. + * The *Transport* layer establishes the idea of a **port**. -View: - This component can be any *representation* of the data to the outside world: - a chart, diagram, table, user interface, etc. + * The *Application* layer doesn't care about what happens below... - It also includes representations of the *actions* available in the system. + * *Except for* **endpoint data** (IP:Port) -Controller: - This component coordinates the Model and the View in a system. + A **Socket** is the software representation of that endpoint. - It accepts input from a user and channels that input into the Model. + Opening a **socket** creates a kind of transceiver that can send and/or + receive *bytes* at a given IP address and Port. - It accepts information about the current state of the Model and transmits - that information to the View. -On the Web ----------- +Sockets in Python +----------------- -This pattern has proven useful for thinking about the applications we build for -the web. +Python provides a standard library module which provides socket functionality. +It is called **socket**. .. rst-class:: build .. container:: - A web browser provides a convenient container for *views* of data. + The library is really just a very thin wrapper around the system + implementation of *BSD Sockets* - These *views* are created by *controller* software hosted on a server. + Let's spend a few minutes getting to know this module. - This *controller* software accepts input from users via *HTTP requests*, - channeling it into a *data model* usually stored in some database. + We're going to do this next part together, so open up a terminal and start + an iPython interpreter - The *controller* returns information about the state of the *data model* to - the user via *HTTP responses* .. nextslide:: -This approach is so common, that it has been formalized into any number of *web -frameworks* +The Python sockets library allows us to find out what port a *service* uses: .. rst-class:: build .. container:: - *Web frameworks* abstract away the specifics of the *HTTP request/response - cycle*, leaving simple MVC components for the developer to use. - - *Web frameworks* exist in nearly all modern languages. - - Python has scores of them. - - Over the weeks to come, we'll learn about two of them, `Pyramid`_ and - `Django`_. + .. code-block:: ipython -.. _Pyramid: http://www.pylonsproject.org/projects/pyramid/about -.. _Django: https://www.djangoproject.com/ - -A Word About Terminology ------------------------- - -Although the MVC pattern is a useful abstraction, there are a few differences -in how things are named in Python web frameworks - -.. rst-class:: build centered -.. container:: + In [1]: import socket - model <--> model + In [2]: socket.getservbyname('ssh') + Out[2]: 22 - controller <--> view + You can also do a *reverse lookup*, finding what service uses a given *port*: - view <--> template (or even HTTP response) + .. code-block:: ipython - .. rst-class:: left + In [3]: socket.getservbyport(80) + Out[3]: 'http' - For more on this difference, you can `read this`_ from the Pyramid design - documentation. -.. _read this: http://docs.pylonsproject.org/projects/pyramid/en/latest/designdefense.html#pyramid-gets-its-terminology-wrong-mvc - -Our First Application -===================== - -.. rst-class:: left +.. nextslide:: -But enough abstract blabbering. +The sockets library also provides tools for finding out information about +*hosts*. For example, you can find out about the hostname and IP address of +the machine you are currently using: -.. rst-class:: build left -.. container:: +.. code-block:: ipython - There's no better way to make concepts like these concrete than to build - something using them. + In [4]: socket.gethostname() + Out[4]: 'Banks' - Let's make an application! + In [5]: socket.gethostbyname(socket.gethostname()) + Out[5]: '127.0.0.1' - We're going to build a Learning Journal. +.. nextslide:: - When we're done, you'll have a live, online application you can use to keep - note of the things you are learning about Python development. +You can also find out about machines that are located elsewhere, assuming you +know their hostname. For example: - We'll use one of our Python web framework to do this: `Pyramid`_ +.. code-block:: ipython -Pyramid -------- + In [6]: socket.gethostbyname('google.com') + Out[6]: '173.194.33.100' -First published in 2010, `Pyramid`_ is a powerful, flexible web framework. + In [7]: socket.gethostbyname('uw.edu') + Out[7]: '128.95.155.134' -.. rst-class:: build -.. container:: + In [8]: socket.gethostbyname('crisewing.com') + Out[8]: '108.168.213.86' - You can create compelling one-page applications, much like in - microframeworks like Flask - You can also create powerful, scalable applications using the full - power of Python +.. nextslide:: - Created by the combined powers of the teams behind Pylons and Zope +The ``gethostbyname_ex`` method of the ``socket`` library provides more +information about the machines we are exploring: - It represents the first true second-generation web framework in - existence. +.. code-block:: ipython -Starting the Project --------------------- + In [9]: socket.gethostbyname_ex('crisewing.com') + Out[9]: ('crisewing.com', [], ['108.168.213.86']) -The first step is to prepare for the project. + In [10]: socket.gethostbyname_ex('google.com') + Out[10]: + ('google.com', + [], + ['173.194.33.100', '173.194.33.103', + ... + '173.194.33.97', '173.194.33.104']) -.. rst-class:: build -.. container:: +.. nextslide:: - Begin by creating a location where you'll do your work. +To create a socket, you use the **socket** method of the ``socket`` library. +It takes up to three optional positional arguments (here we use none to get +the default behavior): - I generally put all my work in a folder called ``projects`` in my home - directory: +.. code-block:: ipython - .. code-block:: bash + In [11]: foo = socket.socket() - $ cd - $ mkdir projects - $ cd projects - $ mkdir learning-journal - $ cd learning-journal - $ pwd - /Users/cewing/project/learning-journal + In [12]: foo + Out[12]: -.. nextslide:: Creating an Environment +.. nextslide:: -We continue our preparations by creating a virtualenv we will use for it. +A socket has some properties that are immediately important to us. These +include the *family*, *type* and *protocol* of the socket: .. rst-class:: build .. container:: - Again, this will help us to keep our work here isolated from anything else - we do. + .. code-block:: ipython - Remember how to make a new virtualenv? + In [13]: foo.family + Out[13]: - .. code-block:: bash + In [14]: foo.type + Out[14]: - $ virtualenv ljenv - New python executable in ljenv/bin/python - Installing setuptools, pip...done. + In [15]: foo.proto + Out[15]: 0 - And then, how to activate it? + You might notice that the values for these properties are integers. In + fact, these integers are **constants** defined in the socket library. - .. code-block:: bash - $ source ljenv/bin/activate - (ljenv)$ +.. nextslide:: A quick utility method -.. nextslide:: Installing Pyramid - -Next, we install the Pyramid web framework into our new virtualenv. +Let's define a method in place to help us see these constants. It will take a +single argument, the shared prefix for a defined set of constants: .. rst-class:: build .. container:: - We can do this with the ``pip`` in our active ``ljenv``: + (you can also find this in ``resources/session01/socket_tools.py``) - .. code-block:: bash + .. code-block:: ipython - (ljenv)$ pip install pyramid - Collecting pyramid - Downloading pyramid-1.5.2-py2.py3-none-any.whl (545kB) - 100% |################################| 548kB 172kB/s - ... - Successfully installed PasteDeploy-1.5.2 WebOb-1.4 - pyramid-1.5.2 repoze.lru-0.6 translationstring-1.3 - venusian-1.0 zope.deprecation-4.1.1 zope.interface-4.1.2 + In [37]: def get_constants(prefix): + ....: """mapping of socket module constants to their names""" + ....: return {getattr(socket, n): n + ....: for n in dir(socket) + ....: if n.startswith(prefix) + ....: } + ....: - Once that is complete, we are ready to create a *scaffold* for our project. -Working with Pyramid --------------------- +Socket Families +--------------- -Many web frameworks require at least a bit of *boilerplate* code to get -started. +Think back a moment to our discussion of the *Internet* layer of the TCP/IP +stack. There were a couple of different types of IP addresses: .. rst-class:: build .. container:: - Pyramid does not. + .. rst-class:: build - However, our application will require a database and handling that does - require some. + * IPv4 ('192.168.1.100') - Pyramid provides a system for creating boilerplate called ``pcreate``. + * IPv6 ('2001:0db8:85a3:0042:0000:8a2e:0370:7334') - You use it to generate the skeleton for a project based on some pattern: - .. code-block:: bash + The **family** of a socket corresponds to the *addressing system* it uses + for connecting. + +.. nextslide:: - (ljenv)$ pcreate -s alchemy learning_journal - Creating directory /Users/cewing/projects/learning-journal/learning_journal - ... - Welcome to Pyramid. Sorry for the convenience. - =============================================================================== - - Let's take a quick look at what that did - -.. nextslide:: What You Get - -.. code-block:: bash - - (ljenv)$ tree learning_journal/ - learning_journal/ - ... - ├── development.ini - ├── learning_journal - │   ├── __init__.py - │   ├── models.py - │   ├── scripts - │   │   ├── __init__.py - │   │   └── initializedb.py - │   ├── static - ... - │   ├── templates - │   │   └── mytemplate.pt - │   ├── tests.py - │   └── views.py - ├── production.ini - └── setup.py - -.. nextslide:: Saving Your Work - -You've now created something worth saving. +Families defined in the ``socket`` library are prefixed by ``AF_``: .. rst-class:: build .. container:: - Start by initializing a new git repository in the `learning_journal` folder - you just created: - - .. code-block:: bash - - (ljenv)$ cd learning_journal - (ljenv)$ git init - Initialized empty Git repository in - /Users/cewing/projects/learning-journal/learning_journal/.git/ - -.. nextslide:: Saving Your Work + .. code-block:: ipython -Check ``git status`` to see where things stand: + In [39]: families = get_constants('AF_') -.. code-block:: bash + In [40]: families + Out[40]: + {: 'AF_UNSPEC', + : 'AF_UNIX', + : 'AF_INET', + ... + : 'AF_INET6', + : 'AF_SYSTEM'} - (ljenv)$ git status - On branch master + *Your results may vary* - Initial commit + Of all of these, the ones we care most about are ``2`` (IPv4) and ``30`` + (IPv6). - Untracked files: - (use "git add ..." to include in what will be committed) - CHANGES.txt - MANIFEST.in - README.txt - development.ini - learning_journal/ - production.ini - setup.py +.. nextslide:: Unix Domain Sockets -.. nextslide:: Add the Project Code -Add your work to this new repository: - -.. code-block:: bash - - (ljenv)$ git add . - (ljenv)$ git status - ... - Changes to be committed: - (use "git rm --cached ..." to unstage) - - new file: CHANGES.txt - new file: MANIFEST.in - ... - new file: production.ini - new file: setup.py - -.. nextslide:: Ignore Irrelevant Files - -Python creates ``.pyc`` files when it executes your code. +When you are on a machine with an operating system that is Unix-like, you will +find another generally useful socket family: ``AF_UNIX``, or Unix Domain +Sockets. Sockets in this family: .. rst-class:: build -.. container:: - - There are many other files you don't want or need in your repository - - You can ignore this in ``git`` with the ``.gitignore`` file. - Create one now, in this same directory, and add the following basic lines:: +* connect processes **on the same machine** - *.pyc - .DS_Store +* are generally a bit slower than IPC connnections - Finally, add this new file to your repository, too. +* have the benefit of allowing the same API for programs that might run on one + machine __or__ across the network -.. nextslide:: Make It Permanent +* use an 'address' that looks like a pathname ('/tmp/foo.sock') -To preserve all these changes, you'll need to commit what you've done: -.. code-block:: bash +.. nextslide:: Test your skills - (ljenv)$ git commit -m "initial commit of the Pyramid learning journal" +What is the *default* family for the socket we created just a moment ago? .. rst-class:: build .. container:: - This will make a first commit here in this local repository. + (remember we bound the socket to the symbol ``foo``) - For homework, you'll put this into GitHub, but this is enough for now. + How did you figure this out? - Let's move on to learning about what we've built so far. -.. nextslide:: Project Structure +Socket Types +------------ -When you ran the ``pcreate`` command, a new folder was created: -``learning_journal``. +The socket *type* determines the semantics of socket communications. .. rst-class:: build .. container:: - This folder contains your *project*. + Look up socket type constants with the ``SOCK_`` prefix: - At the top level, you have *configuration* (.ini files) + .. code-block:: ipython - You also have a file called ``setup.py`` + In [42]: types = get_constants('SOCK_') - This file turns this collection of Python code and configuration into an - *installable Python distribution* + In [43]: types + Out[43]: + {: 'SOCK_STREAM', + : 'SOCK_DGRAM', + : 'SOCK_RAW', + : 'SOCK_RDM', + : 'SOCK_SEQPACKET'} - Let's take a moment to look over the code in that file + The most common are ``1`` (Stream communication (TCP)) and ``2`` (Datagram + communication (UDP)). -.. nextslide:: ``setup.py`` -.. code-block:: python +.. nextslide:: Test your skills - from setuptools import setup, find_packages - ... - requires = [ - 'pyramid', - ... # packages on which this software depends (dependencies) - ] - setup(name='learning_journal', - version='0.0', - ... # package metadata (used by PyPI) - install_requires=requires, - # Entry points are ways that we can run our code once installed - entry_points="""\ - [paste.app_factory] - main = learning_journal:main - [console_scripts] - initialize_learning_journal_db = learning_journal.scripts.initializedb:main - """, - ) - -Pyramid is Python ------------------ - -In the ``__init__.py`` file of your app *package*, you'll find a ``main`` -function: - -.. code-block:: python - - def main(global_config, **settings): - """ This function returns a Pyramid WSGI application. - """ - engine = engine_from_config(settings, 'sqlalchemy.') - DBSession.configure(bind=engine) - Base.metadata.bind = engine - config = Configurator(settings=settings) - config.include('pyramid_chameleon') - config.add_static_view('static', 'static', cache_max_age=3600) - config.add_route('home', '/') - config.scan() - return config.make_wsgi_app() +What is the *default* type for our generic socket, ``foo``? -Let's take a closer look at this, line by line. -.. nextslide:: System Configuration - -.. code-block:: python - - def main(global_config, **settings): +Socket Protocols +---------------- -Configuration is passed in to an application after being read from the -``.ini`` file we saw above. +A socket also has a designated *protocol*. The constants for these are +prefixed by ``IPPROTO_``: .. rst-class:: build .. container:: - These files contain sections (``[app:main]``) containing ``name = value`` - pairs of *configuration data* + .. code-block:: ipython - This data is parsed with the Python - `ConfigParser `_ module. + In [45]: protocols = get_constants('IPPROTO_') - The result is a dict of values: + In [46]: protocols + Out[46]: + {0: 'IPPROTO_IP', + ... + 6: 'IPPROTO_TCP', + ... + 17: 'IPPROTO_UDP', + ...} - .. code-block:: python + The choice of which protocol to use for a socket is determined by the + *internet layer* protocol you intend to use. ``TCP``? ``UDP``? ``ICMP``? + ``IGMP``? - {'app:main': {'pyramid.reload_templates': True, ...}, ...} - The default section of the file is passed in as ``global_config``, the - section for *this app* as ``settings``. +.. nextslide:: Test your skills -.. nextslide:: Database Configuration +What is the *default* protocol used by our generic socket, ``foo``? -.. code-block:: python - from sqlalchemy import engine_from_config - from .models import DBSession, Base - ... - engine = engine_from_config(settings, 'sqlalchemy.') - DBSession.configure(bind=engine) - Base.metadata.bind = engine +Customizing Sockets +------------------- -We will use a package called ``SQLAlchemy`` to interact with our database. +These three properties of a socket correspond to the three positional +arguments you may pass to the socket constructor. .. rst-class:: build .. container:: - Our connection is set up using settings read from the ``.ini`` file. - - Can you find the settings for the database? + Using them allows you to create sockets with specific communications + profiles: - The ``DBSession`` ensures that each *database transaction* is tied to HTTP - requests. + .. code-block:: ipython - The ``Base`` provides a parent class that will hook our *models* to the - database. + In [3]: socket.socket(socket.AF_INET, + ...: socket.SOCK_DGRAM, + ...: socket.IPPROTO_UDP) + Out[3]: -.. nextslide:: App Configuration - -.. code-block:: python - config = Configurator(settings=settings) - config.include('pyramid_chameleon') - config.add_static_view('static', 'static', cache_max_age=3600) - config.add_route('home', '/') - config.scan() +Break Time +---------- -Pyramid controlls application-level configuration using a ``Configurator`` class. +So far we have: .. rst-class:: build .. container:: - It uses app-specific settings passed in from the ``.ini`` file + .. rst-class:: build - We can also ``include`` configuration from other add-on packages + * learned about the "layers" of the TCP/IP Stack + * discussed *families*, *types* and *protocols* in sockets + * learned how to create sockets with a specific communications profile. - Additionally, we can configure *routes* and *views* needed to connect our - application to the outside world here (more on this next week). + When we return we'll learn how to find the communcations profiles of remote + sockets, how to connect to them, and how to send and receive messages. - Finally, the ``Configurator`` instance performs a ``scan`` to ensure there - are no problems with what we've created. + Take a few minutes now to clear your head (do not quit your python + interpreter). -.. nextslide:: A Last Word on Configuration -We will return to the configuration of our application repeatedly over the next -sessions. +Address Information +------------------- + +When you are creating a socket to communicate with a remote service, the +remote socket will have a specific communications profile. .. rst-class:: build .. container:: - Pyramid configuration is powerful and flexible. - - We'll use a few of its features + The local socket you create must match that communications profile. - But there's a lot more you could (and should) learn. + How can you determine the *correct* values to use? - Read about it in the `configuration chapter`_ of the Pyramid documentation. + .. rst-class:: centered -.. _configuration chapter: http://docs.pylonsproject.org/projects/pyramid/en/latest/api/config.html + **You ask.** -.. nextslide:: Break Time - -Let's take a moment to rest up and absorb what we've learned. - -When we return, we'll see how we can create *models* that will embody the data -for our Learning Journal application. - -.. rst-class:: centered +.. nextslide:: -**Pyramid Models** +The function ``socket.getaddrinfo`` provides information about available +connections on a given host. +.. code-block:: python -Models in Pyramid -================= + socket.getaddrinfo('127.0.0.1', 80) -.. rst-class:: left +.. rst-class:: build .. container:: - The central component of MVC, the model, captures the behavior of the - application in terms of its problem domain, independent of the user - interface. The model directly manages the data, logic and rules of the - application + This provides all you need to make a proper connection to a socket on a + remote host. The value returned is a tuple of: - -- from the Wikipedia article on `Model-view-controller`_ + .. rst-class:: build -.. _Model-view-controller: http://en.wikipedia.org/wiki/Model–view–controller + * socket family + * socket type + * socket protocol + * canonical name (usually empty, unless requested by flag) + * socket address (tuple of IP and Port) -Models and ORMs ---------------- -In an MVC application, we define the *problem domain* by creating one or more -*Models*. +.. nextslide:: A quick utility method -.. rst-class:: build -.. container:: - - These capture relevant details about the information we want to preserve - and how we want to interact with it. +Again, let's create a utility method in-place so we can see this in action: - In Python-based MVC applications, these *Models* are implemented as Python - classes. +.. code-block:: ipython - The individual bits of data we want to know about are *attributes* of our - classes. + In [10]: def get_address_info(host, port): + ....: for response in socket.getaddrinfo(host, port): + ....: fam, typ, pro, nam, add = response + ....: print('family: {}'.format(families[fam])) + ....: print('type: {}'.format(types[typ])) + ....: print('protocol: {}'.format(protocols[pro])) + ....: print('canonical name: {}'.format(nam)) + ....: print('socket address: {}'.format(add)) + ....: print('') + ....: - The actions we want to take using that data are *methods* of our classes. +(you can also find this in ``resources/session01/socket_tools.py``) - Together, we can refer to this as the *API* of our system. -.. nextslide:: Persistence +.. nextslide:: On Your Own Machine -It's all well and good to have a set of Python classes that represent your -system. +Now, ask your own machine what possible connections are available for 'http': .. rst-class:: build .. container:: - But what happens when you want to *save* information. - - What happens to a instance of a Python class when you quit the interprer? + .. code-block:: ipython - When your script stops running? + In [11]: get_address_info(socket.gethostname(), 'http') + family: AF_INET + type: SOCK_DGRAM + protocol: IPPROTO_UDP + canonical name: + socket address: ('127.0.0.1', 80) - The code in a website runs when an HTTP request comes in from a client. + family: AF_INET + type: SOCK_STREAM + protocol: IPPROTO_TCP + canonical name: + socket address: ('127.0.0.1', 80) - It stops running when an HTTP response goes back out to the client. + What answers do you get? - So what happens to the data in your system in-between these moments? - The data must be *persisted* +.. nextslide:: On the Internet -.. nextslide:: Alternatives +.. code-block:: ipython -In the last class from part one of this series, you explored a number of -alternatives for persistence - -.. rst-class:: build + In [12]: get_address_info('crisewing.com', 'http') + family: AF_INET + type: SOCK_DGRAM + protocol: IPPROTO_UDP + canonical name: + socket address: ('108.168.213.86', 80) -* Python Literals -* Pickle/Shelf -* Interchange Files (CSV, XML, INI) -* Object Stores (ZODB, Durus) -* NoSQL Databases (MongoDB, CouchDB) -* SQL Databases (sqlite, MySQL, PostgreSQL, Oracle, SQLServer) + family: AF_INET + type: SOCK_STREAM + protocol: IPPROTO_TCP + canonical name: + socket address: ('108.168.213.86', 80) .. rst-class:: build .. container:: - Any of these might be useful for certain types of applications. + Try a few other servers you know about. - On the web, you tend to see two used the most: - .. rst-class:: build - - * NoSQL - * SQL - -.. nextslide:: Choosing One - -How do you choose one over the other? +Client Side +=========== .. rst-class:: build .. container:: - In general, the telling factor is going to be how you intend to use your - data. + .. rst-class:: large - In systems where the dominant feature is viewing/interacting with - individual objects, a NoSQL storage solution might be the best way to go. + Let's put this to use - In systems with objects that are related to eachother, SQL-based Relational - Databases are a better choice. + We'll communicate with a remote server as a *client* - Our system is more like this latter type (trust me on that one for now). - We'll be using SQL (sqlite to start with). +Construct a Socket +------------------ +We've already made a socket ``foo`` using the generic constructor without any +arguments. We can make a better one now by using real address information from +a real server online [**do not type this yet**]: -.. nextslide:: Objects and Tables +.. code-block:: ipython -So we have a system where our data is captured in Python *objects* + In [13]: streams = [info + ....: for info in socket.getaddrinfo('crisewing.com', 'http') + ....: if info[1] == socket.SOCK_STREAM] + ....: + In [14]: streams + Out[14]: + [(, + , + 6, + '', + ('108.168.213.86', 80))] + In [15]: info = streams[0] + In [16]: cewing_socket = socket.socket(*info[:3]) -.. rst-class:: build -.. container:: - And a storage system where our data must be rendered as database *tables* +Connecting a Socket +------------------- - Python provides a specification for interacting directly with databases: - `dbapi2`_ +Once the socket is constructed with the appropriate *family*, *type* and +*protocol*, we can connect it to the address of our remote server: - And there are multiple Python packages that implement this specification - for various databases: +.. code-block:: ipython - .. rst-class:: build + In [18]: cewing_socket.connect(info[-1]) - * sqlite3 - * python-mysql - * psycopg2 - * ... +.. rst-class:: build - With these, you can write SQL to save your Python objects into your - database. +* a successful connection returns ``None`` -.. _dbapi2: https://www.python.org/dev/peps/pep-0249/ +* a failed connection raises an error -.. nextslide:: ORMs +* you can use the *type* of error returned to tell why the connection failed. -But that's a pain. -.. rst-class:: build -.. container:: +Sending a Message +----------------- - SQL, while not impossible, is yet another language to learn. +Send a message to the server on the other end of our connection (we'll +learn in session 2 about the message we are sending): - And there is a viable alternative in using an *Object Relational Manager* - (ORM) +.. code-block:: ipython - An ORM provides a layer of *abstraction* between you and SQL + In [19]: msg = "GET / HTTP/1.1\r\n" + In [20]: msg += "Host: crisewing.com\r\n\r\n" + In [21]: msg = msg.encode('utf8') + In [22]: msg + Out[22]: b'GET / HTTP/1.1\r\nHost: crisewing.com\r\n\r\n' + In [23]: cewing_socket.sendall(msg) - You instantiate Python objects and set attributes on them +.. rst-class:: build small - The ORM handles converting data from these objects into SQL statements (and - back) +* the transmission continues until all data is sent or an error occurs +* success returns ``None`` +* failure to send raises an error +* the type of error can tell you why the transmission failed +* but you **cannot** know how much, if any, of your data was sent -SQLAlchemy ----------- -In our project we will be using the `SQLAlchemy`_ ORM. +Messages Are Bytes +------------------ -.. rst-class:: build -.. container:: +One detail from the previous code should stand out: - You can find SQLAlchemy among the packages in ``requires`` in ``setup.py`` - in our new ``learning_journal`` package. +.. code-block:: ipython - However, we don't yet have that code installed. + In [21]: msg = msg.encode('utf8') + In [22]: msg + Out[22]: b'GET / HTTP/1.1\r\nHost: crisewing.com\r\n\r\n' - To do so, we will need to "install" our own package +You can **only** send bytes through a socket, **never** unicode - Make sure your ``ljenv`` virtualenv is active and then type the following: +.. code-block:: ipython - .. code-block:: bash + In [35]: cewing_socket.sendall(msg.decode('utf8')) + --------------------------------------------------------------------------- + TypeError Traceback (most recent call last) + in () + ----> 1 cewing_socket.sendall(msg.decode('utf8')) - (ljenv)$ python setup.py develop - running develop - running egg_info - creating learning_journal.egg-info - ... - Finished processing dependencies for learning-journal==0.0 + TypeError: 'str' does not support the buffer interface -.. nextslide:: -Once that is complete, all the *dependencies* listed in our ``setup.py`` will -be installed. +Receiving a Reply +----------------- -.. rst-class:: build -.. container:: +Whatever reply we get is received by the socket we created. We can read it +back out (again, **do not type this yet**): - You can also install the package using ``python setup.py install`` +.. code-block:: ipython - But using ``develop`` allows us to continue developing our package without - needing to re-install it every time we change something. + In [24]: response = cewing_socket.recv(4096) + In [25]: response[:60] + Out[25]: b'HTTP/1.1 200 OK\r\nServer: nginx\r\nDate: Sun, 20 Sep 2015 03:38' - It is very similar to using the ``-e`` option to ``pip`` +.. rst-class:: build - Now, we'll only need to re-run this command if we change ``setup.py`` - itself. +* The sole required argument is ``buffer_size`` (an integer). It should be a + power of 2 and smallish (~4096) +* It returns a byte string of ``buffer_size`` (or smaller if less data was + received) +* If the response is longer than ``buffer size``, you can call the method + repeatedly. The last bunch will be less than ``buffer size``. -.. nextslide:: -We also need to adjust our ``.gitignore`` file: +Cleaning Up +----------- -.. rst-class:: build -.. code-block:: bash +When you are finished with a connection, you should always close it:: - (ljenv)$ git status - ... - Untracked files: - (use "git add ..." to include in what will be committed) + cewing_socket.close() - learning_journal.egg-info/ -.. rst-class:: build -.. container:: +Putting it all together +----------------------- - The ``egg-info`` directory that was just created is an artifact of - installing a Python egg. +First, connect and send a message: - It should never be committed to a repository. +.. code-block:: ipython - Let's add ``*.egg-info`` to our ``.gitignore`` file and then commit that - change + In [55]: info = socket.getaddrinfo('crisewing.com', 'http') + In [56]: streams = [i for i in info if i[1] == socket.SOCK_STREAM] + In [57]: sock_info = streams[0] + In [58]: msg = "GET / HTTP/1.1\r\n" + In [59]: msg += "Host: crisewing.com\r\n\r\n" + In [60]: msg = msg.encode('utf8') + In [61]: cewing_socket = socket.socket(*sock_info[:3]) + In [62]: cewing_socket.connect(sock_info[-1]) + In [63]: cewing_socket.sendall(msg) - Remember how? -.. nextslide:: Our First Model +.. nextslide:: -Our project skeleton contains up a first, basic model created for us: +Then, receive a reply, iterating until it is complete: -.. code-block:: python +.. code-block:: ipython - # in models.py - Base = declarative_base() + In [65]: buffsize = 4096 + In [66]: response = b'' + In [67]: done = False + In [68]: while not done: + ....: msg_part = cewing_socket.recv(buffsize) + ....: if len(msg_part) < buffsize: + ....: done = True + ....: cewing_socket.close() + ....: response += msg_part + ....: + In [69]: len(response) + Out[69]: 19464 - class MyModel(Base): - __tablename__ = 'models' - id = Column(Integer, primary_key=True) - name = Column(Text) - value = Column(Integer) - Index('my_index', MyModel.name, unique=True, mysql_length=255) -.. _SQLAlchemy: http://docs.sqlalchemy.org/en/rel_0_9/ +Server Side +=========== .. rst-class:: build .. container:: - Our class inherits from ``Base`` + .. rst-class:: large - We ran into ``Base`` earlier when discussing configuration. + What about the other half of the equation? - We were binding it to the database we wanted to use (the ``engine``) + Let's build a server and see how that part works. -.. nextslide:: ``Base`` +Construct a Socket +------------------ -Any class we create that inherits from this ``Base`` becomes a *model* +**For the moment, stop typing this into your interpreter.** .. rst-class:: build .. container:: - It will be connected through the ORM to a table in our database. - - The name of the table is determined by the ``__tablename__`` special - attribute. + Again, we begin by constructing a socket. Since we are actually the server + this time, we get to choose family, type and protocol: - Other aspects of table configuration can also be controlled through special - attributes + .. code-block:: ipython - Instances of the class, once saved, will become rows in the table. + In [70]: server_socket = socket.socket( + ....: socket.AF_INET, + ....: socket.SOCK_STREAM, + ....: socket.IPPROTO_TCP) - Attributes of the model that are instances of ``Column`` will become - columns in the table. + In [71]: server_socket + Out[71]: - You can learn much more in the `Declarative`_ chapter of the SQLAlchemy docs -.. _Declarative: http://docs.sqlalchemy.org/en/rel_0_9/orm/extensions/declarative/ - -.. nextslide:: Columns +Bind the Socket +--------------- -Each attribute of your model that will be persisted must be an instance of -`Column`_. +Our server socket needs to be **bound** to an address. This is the IP Address +and Port to which clients must connect: .. rst-class:: build .. container:: - Each instance requires *at least* a specific `data type`_ (such as - Integer). + .. code-block:: ipython - Additionally, you can control other aspects of the column such as it being - a primary key. + In [72]: address = ('127.0.0.1', 50000) + In [73]: server_socket.bind(address) - In the *declarative* style we are using, the name of the column in the - database will default to the attribute name you assigned. + **Terminology Note**: In a server/client relationship, the server *binds* + to an address and port. The client *connects* - If you wish, you may provide a name specifically. It must be the first - argument and must be a string. +Listen for Connections +---------------------- -.. _Column: http://docs.sqlalchemy.org/en/rel_0_9/core/metadata.html#sqlalchemy.schema.Column -.. _data type: http://docs.sqlalchemy.org/en/rel_0_9/core/types.html +Once our socket is bound to an address, we can listen for attempted +connections: -Creating The Database ---------------------- +.. code-block:: ipython -We have a *model* which allows us to persist Python objects to an SQL database. + In [74]: server_socket.listen(1) .. rst-class:: build -.. container:: - - But we're still missing one ingredient here. - - We need to create our database, or there will be nowhere for our data to - go. - - Luckily, our ``pcreate`` scaffold also gave us a convenient way to handle - this: - .. code-block:: python +* The argument to ``listen`` is the *backlog* +* The *backlog* is the **maximum** number of connection requests that the + socket will queue +* Once the limit is reached, the socket refuses new connections. - # in setup.py - entry_points="""\ - [paste.app_factory] - main = learning_journal:main - [console_scripts] - initialize_learning_journal_db = learning_journal.scripts.initializedb:main - """, - The ``console_script`` set up as an entry point will help us. - -.. nextslide:: ``initialize_learning_journal_db`` +Accept A Connection +------------------- -Let's look at that code for a moment. +When a socket is listening, it can receive incoming connection requests: -.. code-block:: python +.. code-block:: ipython - # in scripts/intitalizedb.py - from ..models import DBSession, MyModel, Base - # ... - def main(argv=sys.argv): - if len(argv) < 2: - usage(argv) - config_uri = argv[1] - options = parse_vars(argv[2:]) - setup_logging(config_uri) - settings = get_appsettings(config_uri, options=options) - engine = engine_from_config(settings, 'sqlalchemy.') - DBSession.configure(bind=engine) - Base.metadata.create_all(engine) - with transaction.manager: - model = MyModel(name='one', value=1) - DBSession.add(model) - -.. nextslide:: Console Scripts - -By connecting this function as a ``console script``, our Python package makes -this command available to us. + In [75]: connection, client_address = server_socket.accept() .. rst-class:: build -.. container:: - - When we exectute ``initialize_learning_journal_db`` at the command line, we - will be running this function. - Let's try it out. +* The call to ``socket.accept()`` is a *blocking* call. It will not return + values until a client *connects* +* The ``connection`` returned by a call to ``accept`` is a **new socket**. + This new socket is used to communicate with the client +* The ``client_address`` is a two-tuple of IP Address and Port for the client + socket +* When a connection request is 'accepted', it is removed from the backlog + queue. - We'll need to provide a configuration file name, let's use - ``development.ini``: - .. code-block:: bash +Communicate +----------- - (ljenv)$ initialize_learning_journal_db development.ini - 2015-01-05 18:59:55,426 INFO [sqlalchemy.engine.base.Engine][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 - ... - 2015-01-05 18:59:55,434 INFO [sqlalchemy.engine.base.Engine][MainThread] COMMIT +The ``connection`` socket can now be used to receive messages from the client +which made the connection: - The ``[loggers]`` configuration in our ``.ini`` file sends a stream of - INFO-level logging to sys.stdout as the console script runs. +.. code-block:: ipython -.. nextslide:: A Bit More Cleanup + In [76]: connection.recv(buffsize) -So what was the outcome of running that script? +It may also be used to return a reply: -.. rst-class:: build -.. container:: +.. code-block:: ipython - .. code-block:: bash + In [77]: connection.sendall("message received") - (ljenv)$ ls - ... - learning_journal.sqlite - ... - We've now created an sqlite database. - - You'll need to add ``*.sqlite`` to ``.gitignore`` so you don't add that - file to your repository. - - Once you've done so, commit the change to your repository - -Interacting with SQLA Models ----------------------------- +Clean Up +-------- -It's pretty easy to play with your models from in an interpreter. +Once a transaction between the client and server is complete, the +``connection`` socket should be closed: .. rst-class:: build .. container:: - Let's try that out and see what we have. Start up an interpreter: + .. code-block:: ipython - .. code-block:: pycon + In [78]: connection.close() - >>> config = 'development.ini' - >>> from pyramid.paster import get_appsettings - >>> settings = get_appsettings(config) - >>> from sqlalchemy import engine_from_config - >>> engine = engine_from_config(settings, 'sqlalchemy.') - >>> from sqlalchemy.orm import sessionmaker - >>> Session = sessionmaker(bind=engine) - >>> session = Session() - >>> from learning_journal.models import MyModel - >>> session.query(MyModel).all() - [] + At this point, the ``server_socket`` can again accept a new client + connection. - We are basically stealing the important bits from ``initializedb.py`` + Note that the ``server_socket`` is *never* closed as long as the server + continues to run. -.. nextslide:: Basic Interactions -Any interaction with the database requires a ``session``. +Getting the Flow +================ -.. rst-class:: build +.. rst-class:: left .. container:: - This object represents the connection to the database. - - All database queries are phrased as methods of the session. + The flow of this interaction can be a bit confusing. Let's see it in + action step-by-step. + .. rst-class:: build .. container:: - .. code-block:: pycon + .. container:: - >>> query = session.query(MyModel).all() - >>> type(query) - + Open a second iPython interpreter and place it next to your first so + you can see both of them at the same time. - The ``query`` method of the session object returns a ``Query`` object - Arguments to the ``query`` method can be a *model* class or *columns* from - a model class. - -.. nextslide:: Queries are Iterators +Create a Server +--------------- -You can iterate over a query object. The result depends on the args you -passed. +In your first python interpreter, create a server socket and prepare it for +connections: .. rst-class:: build .. container:: - .. code-block:: pycon - - >>> q1 = session.query(MyModel) - >>> for row in q1: - ... print row - ... type(row) - ... - - + .. code-block:: ipython - .. code-block:: pycon + In [81]: server_socket = socket.socket( + ....: socket.AF_INET, + ....: socket.SOCK_STREAM, + ....: socket.IPPROTO_IP) + In [82]: server_socket.bind(('127.0.0.1', 50000)) + In [83]: server_socket.listen(1) + In [84]: conn, addr = server_socket.accept() - >>> q2 = session.query(MyModel.name, MyModel.id, MyModel.value) - >>> for name, id, val in q2: - ... print name, type(name) - ... print id, type(id) - ... print val, type(val) - ... - one - 1 - 1 -.. nextslide:: Queries have SQL + At this point, you should **not** get back a prompt. The server socket is + waiting for a connection to be made. -You can view the SQL that your query will use: -.. rst-class:: build -.. container:: - - .. code-block:: pycon - - >>> str(q1) - 'SELECT models.id AS models_id, models.name AS models_name, models.value AS models_value \nFROM models' - >>> str(q2) - 'SELECT models.name AS models_name, models.id AS models_id, models.value AS models_value \nFROM models' - - You can use this to check that the query the ORM is constructing looks like - you expect. - - It can be helpful in debugging. - -.. nextslide:: Methods of the Query Object +Create a Client +--------------- -The methods of the ``Query`` object fall into two rough categories +In your second interpreter, create a client socket and prepare to send a +message: .. rst-class:: build .. container:: - .. rst-class:: build - - 1. Methods that return a new ``Query`` object - 2. Methods that return *scalar* values or *model* instances - - Let's start by looking quickly at a few methods from the second category + .. code-block:: ipython -.. nextslide:: ``query.get()`` + In [1]: import socket + In [2]: client_socket = socket.socket( + ...: socket.AF_INET, + ...: socket.SOCK_STREAM, + ...: socket.IPPROTO_IP) -A good example of this category of methods is ``get``, which returns one -instance only. + Before connecting, keep your eye on the server interpreter: -.. rst-class:: build -.. container:: - - It takes a primary key as an argument: + .. code-block:: ipython - .. code-block:: pycon + In [3]: client_socket.connect(('127.0.0.1', 50000)) - >>> session.query(MyModel).get(1) - - >>> session.query(MyModel).get(10) - >>> - If no item with that primary key is present, then the method returns - ``None`` +Send a Message Client->Server +----------------------------- -.. nextslide:: ``query.all()`` - -Another example is one we've already seen. +As soon as you made the connection above, you should have seen the prompt +return in your server interpreter. The ``accept`` method finally returned a +new connection socket. .. rst-class:: build .. container:: - ``query.all()`` returns a list of all rows returned by the database: - - .. code-block:: pycon + When you're ready, type the following in the *client* interpreter: - >>> q1.all() - [] - >>> type(q1.all()) - + .. code-block:: ipython - ``query.count()`` returns the number of rows that would have been returned - by the query: + In [4]: client_socket.sendall('Hey, can you hear me?'.encode('utf8')) - .. code-block:: pycon - >>> q1.count() - 1 - -.. nextslide:: Creating New Objects +Receive and Respond +------------------- -Before getting into the other category, let's learn how to create new objects. +Back in your server interpreter, go ahead and receive the message from your +client: .. rst-class:: build .. container:: - .. container:: + .. code-block:: ipython - We can create new instances of our *model* just like normal Python - objects: + In [87]: msg = conn.recv(4096) + In [88]: msg + Out[88]: b'Hey, can you hear me?' - .. code-block:: pycon + Send a message back, and then close up your connection: - >>> new_model = MyModel(name='fred', value=3) - >>> new_model - - - .. container:: + .. code-block:: ipython - In this state, the instance is *ephemeral*, our ``session`` knows - nothing about it: + In [89]: conn.sendall('Yes, I can hear you.'.encode('utf8')) + In [90]: conn.close() - .. code-block:: pycon - - >>> session.new - IdentitySet([]) - -.. nextslide:: Adding Objects to the Session +Finish Up +--------- -For the database to know about our new object, we must ``add`` it to the -session: +Back in your client interpreter, take a look at the response to your message, +then be sure to close your client socket too: .. rst-class:: build .. container:: - .. code-block:: pycon - - >>> session.add(new_model) - >>> session.new - IdentitySet([]) - - We can even bulk-add new objects: + .. code-block:: ipython - .. code-block:: pycon + In [5]: from_server = client_socket.recv(4096) + In [6]: from_server + Out[6]: b'Yes, I can hear you.' + In [7]: client_socket.close() - >>> new = [] - >>> for name, val in [('bob', 34), ('tom', 13)]: - ... new.append(MyModel(name=name, value=val)) - ... - >>> session.add_all(new) - >>> session.new - IdentitySet([, - , - ]) + And now that we're done, we can close up the server socket too (back in the + server interpreter): -.. nextslide:: Committing Changes - -Up until now, the changes you've made are not permanent. - -.. rst-class:: build -.. container:: + .. code-block:: ipython - In order for these new objects to be saved to the database, the session - must be ``committed``: + In [91]: server_socket.close() - .. code-block:: pycon - >>> other_session = Session() - >>> other_session.query(MyModel).count() - 1 - >>> session.commit() - >>> other_session.query(MyModel).count() +.. nextslide:: Congratulations! - When you are using a ``scoped_session`` in Pyramid, this action is - automatically handled for you. +.. rst-class:: large center - The session that is bound to a particular HTTP request is committed when a - response is sent back. +You've run your first client-server interaction -.. nextslide:: Altering Objects -You can edit objects that are already part of a session, or that are fetched by -a query. +Homework +======== -.. rst-class:: build +.. rst-class:: left .. container:: - Simply change the values of a persisted attribute, the session will know - it's been updated: + Your homework assignment for this week is to take what you've learned here + and build a simple "echo" server. - .. code-block:: pycon - - >>> new_model - - >>> new_model.name - u'fred' - >>> new_model.name = 'larry' - >>> session.dirty - IdentitySet([]) - - Commit the session to persist the changes: + .. rst-class:: build + .. container:: - .. code-block:: pycon - - >>> session.commit() + The server should automatically return to any client that connects *exactly* + what it receives (it should **echo** all messages). -.. nextslide:: Methods Returning Queries + You will also write a python script that, when run, will send a message to the + server and receive the reply, printing it to ``stdout``. -Returning to queries, the second category is typified by the ``filter`` method + Finally, you'll do all of this so that it can be tested. -.. rst-class:: build -.. container:: - This method allows you to reduce the number of results, based on criteria: - - .. code-block:: pycon - - >>> for obj in session.query(MyModel).filter(MyModel.value < 20): - ... print obj.name, obj.value - ... - larry 1 - fred 3 - tom 13 - -.. nextslide:: ``order_by`` +Your Task +--------- -Another typical method in this category is ``order_by``: +In our class repository, there is a folder ``resources/session01``. .. rst-class:: build .. container:: - .. code-block:: pycon - - >>> for obj in session.query(MyModel).order_by(MyModel.value): - ... print obj.name, obj.value - ... - larry 1 - fred 3 - tom 13 - bob 34 - - .. code-block:: pycon - - >>> for obj in session.query(MyModel).order_by(MyModel.name): - ... print obj.name, obj.value - ... - bob 34 - fred 3 - larry 1 - tom 13 + Inside that folder, you should find: -.. nextslide:: Method Chaining - -Since methods in this category return ``Query`` objects, they can be safely -*chained* to build more complex queries: + .. rst-class:: build -.. rst-class:: build -.. container:: + * A file ``tasks.txt`` that contains these instructions - .. code-block:: pycon + * A skeleton for your server in ``echo_server.py`` - >>> q1 = session.query(MyModel).filter(MyModel.value < 20) - >>> q1 = q1.order_by(MyModel.name) - >>> for obj in q1: - ... print obj.name, obj.value - ... - fred 3 - larry 1 - tom 13 + * A skeleton for your client script in ``echo_client.py`` - Note that you can do this inline as well - (``s.query(Model).filter().order_by()``) + * Some simple tests in ``tests.py`` - Also note that when using chained queries like this, no query is actually - sent to the database until you require a result. + Your task is to make the tests pass. -Homework -======== -.. rst-class:: left +Running the Tests +----------------- -Okay, that's enough for the moment. +To run the tests, you'll have to set the server running in one terminal: -.. rst-class:: build left +.. rst-class:: build .. container:: - You've learned quite a bit about how *models* work in SQLAlchemy - - It's time to put that knowledge to good use. - - For the first part of your assignment this week you will begin to define - the data model for our learning journal application. - - I'll provide a specification, you define the model required to do the job. + .. code-block:: bash - I'll also ask you to define a few methods to complete the first part of our - API. + $ python echo_server.py -The Model ---------- + Then, in a second terminal, you will execute the tests: -Our model will be called an ``Entry``. Here's what you need to know: + .. code-block:: bash -* It should be stored in a database table called ``entries`` -* It should have a primary key field called ``id`` -* It should have a ``title`` field which accepts unicode text up to 255 characters in length -* The ``title`` should be unique and it should be impossible to save an - ``entry`` without a ``title``. -* It should have a ``body`` field which accepts unicode text of any length - (including none) -* It should have a ``created`` field which stores the date and time the object - was created. -* It should have an ``edited`` field which stores the date and time the object - was last edited. -* Both the ``created`` and ``edited`` field should default to ``now`` if not - provided when a new instance is constructed. -* The ``entry`` class should support a classmethod ``all`` that returns all the - entries in the database, ordered so that the most recent entry is first. -* The ``entry`` class should support a classmethod ``by_id`` that returns a - single entry, given an ``id``. + $ python tests.py -.. nextslide:: Words of Advice + You should see output like this: -Use the documentation linked in this presentation to assist you. SQLAlchemy -has fantastic documentation, but it can be a bit overwhelming. Everything you -require for this assignment is on one or more of the pages linked above. + .. code-block:: bash -As you define this new model for our application, make frequent commits to your -github repository. Remember to write meaningful commit messages. + [...] + FAILED (failures=2) -Don't be afraid to start up a Python interpreter and play with your model. Try -things out. Learn how this all works by making mistakes. -Errors at the SQL level can sometimes leave your session unusable. To restore -it, use the ``session.rollback()`` method. You'll lose uncommitted changes, -but you'll gain a session that can be used again. +Submitting Your Homework +------------------------ -.. nextslide:: Submitting Your Work +To submit your homework: -I want to be able to review your code (and you want to be able to share it). +.. rst-class:: build +.. container:: -To submit this assignment, you'll need to add this learning_journal repository -to GitHub. + .. rst-class:: build -On the GitHub website you can create a new repository. Set it up to be -completely empty. Name it ``learning_journal`` and give it any description you -like. + * Create a new repository in GitHub. Call it ``echo_sockets``. -When you've created an empty repository in GitHub, you should see a set of -directions for connecting it to a repository that you've already built. Follow -those instructions to connect your emtpy GitHub repository as the ``origin`` -remote to your ``learning_journal`` repository on your machine. + * Put the ``echo_server.py``, ``echo_client.py`` and ``tests.py`` files in + this repository. -Finally, push your ``master`` branch to your new ``origin`` remote on GitHub. + * Send us an email with a link to your repository when you are + done. -When you are done, send me an email with the URL for your new repository. + We will clone your repository and run the tests as described above. -.. nextslide:: + And we'll make comments inline on your repository. -**Our work next week will assume that you have completed this assignment** -Do not delay working on this until the last moment. +Going Further +------------- -Do not skip this assignment. +In ``resources/session01/tasks.txt`` you'll find a few extra problems to try. -Do ask questions frequently via email (use the `class google group`_). +.. rst-class:: build +.. container:: -See you next week! + If you finish the first part of the homework in less than 3-4 hours give + one or more of these a whirl. -.. _class google group: https://groups.google.com/forum/#!forum/programming-in-python + They are not required, but if you include solutions in your repository, + we'll review your work. diff --git a/source/presentations/session02.rst b/source/presentations/session02.rst index 3d1c6b8e..3dc36eac 100644 --- a/source/presentations/session02.rst +++ b/source/presentations/session02.rst @@ -1,1576 +1,1680 @@ -.. slideconf:: - :autoslides: True +.. |br| raw:: html + +
          ********** Session 02 ********** -.. image:: /_static/lj_entry.png - :width: 65% +.. figure:: /_static/protocol.png :align: center + :width: 40% + + Web Protocols + +The Languages Computers Speak +============================= + +.. rst-class:: build left +.. container:: -Interacting with Data -===================== + Programming languages like Python are the languages we speak to computers. -**Wherein we learn to display our data, and to create and edit it too!** + *Protocols* are the languages that computers speak to each-other. + + This sesson we'll look at a few of them and + + .. rst-class:: build + + * Learn what makes them similar + * Learn what makes them different + * Learn about Python's tools for speaking them + * Learn how to speak one (HTTP) ourselves But First ---------- +---------- -Last week we discussed the **model** part of the *MVC* application design -pattern. +.. rst-class:: large centered -.. rst-class:: build +Questions from the Homework? + + +.. nextslide:: + +.. rst-class:: large centered + +Examples of an echo server using ``select`` + + +What is a Protocol? +------------------- + +.. rst-class:: build large centered .. container:: - We set up a project using the `Pyramid`_ web framework and the `SQLAlchemy`_ - library for persisting our data to a database. + **a set of rules or conventions** - We looked at how to define a simple model by investigating the demo model - created on our behalf. + **governing communications** - And we went over, briefly, the way we can interact with this model at the - command line to make sure we've got it right. - Finally, we defined what attributes a learning journal entry would have, - and a pair of methods we think we will need to make the model complete. +.. nextslide:: Protocols IRL -.. _Pyramid: http://www.pylonsproject.org/projects/pyramid/about -.. _SQLAlchemy: http://docs.sqlalchemy.org/en/rel_0_9/ +Life has lots of sets of rules for how to do things. -Our Data Model --------------- +.. rst-class:: build + +* What do you say when you get on the elevator? + +* What do you do on a first date? + +* What do you wear to a job interview? + +* What do (and don't) you talk about at a dinner party? + +* ...? + + +.. nextslide:: Protocols IRL + +.. figure:: /_static/icup.png + :align: center + :width: 65% + + http://blog.xkcd.com/2009/09/02/urinal-protocol-vulnerability/ + + +.. nextslide:: Protocols In Computers -Over the last week, your assignment was to create the new model. +Digital life has lots of rules too: .. rst-class:: build -.. container:: - Did you get that done? +* how to say hello + +* how to identify yourself - If not, what stopped you? +* how to ask for information - Let's take a few minutes here to answer questions about this task so you - are more comfortable. +* how to provide answers - Questions? +* how to say goodbye -.. nextslide:: A Complete Example -I have added a new folder to our `class repository`_, ``resources``. +Real Protocol Examples +---------------------- -.. _class repository: https://github.com/UWPCE-PythonCert/training.python_web/ +What does this look like in practice? .. rst-class:: build -.. container:: - If you clone the repository to your local machine you can get to it. +* SMTP (Simple Message Transfer Protocol) |br| + http://tools.ietf.org/html/rfc5321#appendix-D - You can also just browse the repository in github to view it. +* POP3 (Post Office Protocol) |br| + http://www.faqs.org/docs/artu/ch05s03.html - In this folder, I added a ``session02`` folder that contains resources for - today. +* IMAP (Internet Message Access Protocol) |br| + http://www.faqs.org/docs/artu/ch05s03.html - Among these resources is the completed ``models.py`` file with this new - model added. +* HTTP (Hyper-Text Transfer Protocol) |br| + http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol - Let's review how it works. -.. nextslide:: Demo Interaction +.. nextslide:: A Word on Typography -Another resource I've added is the ``ljshell.py`` script. +Over the next few slides we'll be looking at server/client interactions. .. rst-class:: build .. container:: - That script will allow you to interact with a db session just like I showed - in class last week: + Each interaction is line-based, each line represents one message. - .. code-block:: python + Messages from the Server to the Client are prefaced with ``S (<--)`` - # the script - from pyramid.paster import get_appsettings, setup_logging - from sqlalchemy import engine_from_config - from sqlalchemy.orm import sessionmaker + Messages from the Client to the Server are prefaced with ``C (-->)`` - config_uri = 'development.ini' - setup_logging(config_uri) - settings = get_appsettings(config_uri) - engine = engine_from_config(settings, 'sqlalchemy.') - Session = sessionmaker(bind=engine) + **All** lines end with the character sequence ```` (``\r\n``) - Just copy the file into your learning_journal Pyramid project folder (where - ``setup.py`` is) -.. nextslide:: Using the ``ljshell.py`` script +SMTP +---- -Here's a demo interaction using the script to set up a session maker +What does SMTP look like? .. rst-class:: build .. container:: - First ``cd`` to your project code, fire up your project virtualenv and - start python: + SMTP (Say hello and identify yourself):: - .. code-block:: bash + S (<--): 220 foo.com Simple Mail Transfer Service Ready + C (-->): EHLO bar.com + S (<--): 250-foo.com greets bar.com + S (<--): 250-8BITMIME + S (<--): 250-SIZE + S (<--): 250-DSN + S (<--): 250 HELP - $ cd projects/learning-journal/learning_journal - $ source ../ljenv/bin/activate - (ljenv)$ python - >>> - Then, you can import the ``Session`` symbol from ``ljshell`` and you're off - to the races: +.. nextslide:: - .. code-block:: pycon +.. ifslides:: - >>> from ljshell import Session - >>> from learning_journal.models import MyModel - >>> session = Session() - >>> session.query(MyModel).all() - [] - ... + What does SMTP look like? - [demo] +SMTP (Ask for information, provide answers):: -The MVC Controller -================== + C (-->): MAIL FROM: + S (<--): 250 OK + C (-->): RCPT TO: + S (<--): 250 OK + C (-->): RCPT TO: + S (<--): 550 No such user here + C (-->): DATA + S (<--): 354 Start mail input; end with . + C (-->): Blah blah blah... + C (-->): ...etc. etc. etc. + C (-->): . + S (<--): 250 OK -.. rst-class:: left -.. container:: +.. nextslide:: - Let's go back to thinking for a bit about the *Model-View-Controller* - pattern. +.. ifslides:: - .. figure:: http://upload.wikimedia.org/wikipedia/commons/4/40/MVC_passive_view.png - :align: center - :width: 25% + What does SMTP look like? - By Alan Evangelista (Own work) [CC0], via Wikimedia Commons +SMTP (Say goodbye):: + + C (-->): QUIT + S (<--): 221 foo.com Service closing transmission channel - .. rst-class:: build - .. container:: - We talked last week (and today) about the *model* +.. nextslide:: SMTP Characteristics - Today, we'll dig into *controllers* and *views* +.. rst-class:: build - or as we will know them in Pyramid: *views* and *renderers* +* Interaction consists of commands and replies +* Each command or reply is *one line* terminated by |br| + (there are exceptions, see the ``250`` reply to ``EHLO`` above) +* The exception is message payload, terminated by . +* Each command has a *verb* and one or more *arguments* +* Each reply has a formal *code* and an informal *explanation* -HTTP Request/Response ---------------------- +POP3 +---- -Internet software is driven by the HTTP Request/Response cycle. +What does POP3 look like? .. rst-class:: build .. container:: - A *client* (perhaps a user with a web browser) makes a **request** + POP3 (Say hello and identify yourself):: + + C (-->): + S (<--): +OK POP3 server ready <1896.6971@mailgate.dobbs.org> + C (-->): USER bob + S (<--): +OK bob + C (-->): PASS redqueen + S (<--): +OK bob's maildrop has 2 messages (320 octets) + + +.. nextslide:: + +.. ifslides:: + + What does POP3 look like? + +POP3 (Ask for information, provide answers):: + + C (-->): STAT + S (<--): +OK 2 320 + C (-->): LIST + S (<--): +OK 1 messages (120 octets) + S (<--): 1 120 + S (<--): . - A *server* receives and handles that request and returns a **response** - The *client* receives the response and views it, perhaps making a new - **request** +.. nextslide:: + +.. ifslides:: + + What does POP3 look like? + +POP3 (Ask for information, provide answers):: + + C (-->): RETR 1 + S (<--): +OK 120 octets + S (<--): + S (<--): . + C (-->): DELE 1 + S (<--): +OK message 1 deleted - And around and around it goes. -.. nextslide:: URLs +.. nextslide:: + +.. ifslides:: + + What does POP3 look like? -An HTTP request arrives at a server through the magic of a **URL** +POP3 (Say goodbye):: -.. code-block:: bash + C (-->): QUIT + S (<--): +OK dewey POP3 server signing off (maildrop empty) + C (-->): - http://uwpce-pythoncert.github.io/training.python_web/html/index.html + +.. nextslide:: POP3 Characteristics .. rst-class:: build .. container:: - Let's break that up into its constituent parts: - .. rst-class:: build - \http://: - This part is the *protocol*, it determines how the request will be sent + * Interaction consists of commands and replies + * Each command or reply is *one line* terminated by + * The exception is message payload, terminated by . + * Each command has a *verb* and one or more *arguments* + * Each reply has a formal *code* and an informal *explanation* - uwpce-pythoncert.github.io: - This is a *domain name*. It's the human-facing address for a server - somewhere. + The codes don't really look the same, though, do they? - /training.python_web/html/index.html: - This part is the *path*. It serves as a locator for a resource *on the - server* -.. nextslide:: Paths +.. nextslide:: One Other Difference -In a static website (like our documentation) the *path* identifies a **physical -location** in the server's filesystem. +The exception to the one-line-per-message rule is *payload* .. rst-class:: build .. container:: - Some directory on the server is the *home* for the web process, and the - *path* is looked up there. + In both SMTP and POP3 this is terminated by . - Whatever resource (a file, an image, whatever) is located there is returned - to the user as a response. + In SMTP, the *client* has this ability - If the path leads to a location that doesn't exist, the server responds - with a **404 Not Found** error. + But in POP3, it belongs to the *server*. - In the golden days of yore, this was the only way content was served via - HTTP. + .. rst-class:: large centered -.. nextslide:: Paths in an MVC System + Why? -In todays world we have dynamic systems, server-side web frameworks like -Pyramid. +IMAP +---- + +What does IMAP look like? .. rst-class:: build .. container:: - The requests that you send to a server are handled by a software process - that assembles a response instead of looking up a physical location. + IMAP (Say hello and identify yourself):: - But we still have URLs, with *protocol*, *domain* and *path*. + C (-->): + S (<--): * OK example.com IMAP4rev1 v12.264 server ready + C (-->): A0001 USER "frobozz" "xyzzy" + S (<--): * OK User frobozz authenticated - What is the role for a path in a process that doesn't refer to a physical - file system? - Most web frameworks now call the *path* a **route**. +.. nextslide:: - They provide a way of matching *routes* to the code that will be run to - handle requests. +.. ifslides:: -Routes in Pyramid ------------------ + What does IMAP look like? -In Pyramid, routes are handled as *configuration* and are set up in the *main* -function in ``__init__.py``: +IMAP (Ask for information, provide answers [connect to an inbox]):: -.. code-block:: python + C (-->): A0002 SELECT INBOX + S (<--): * 1 EXISTS + S (<--): * 1 RECENT + S (<--): * FLAGS (\Answered \Flagged \Deleted \Draft \Seen) + S (<--): * OK [UNSEEN 1] first unseen message in /var/spool/mail/esr + S (<--): A0002 OK [READ-WRITE] SELECT completed - # learning_journal/__init__.py - def main(global_config, **settings): - # ... - config.add_route('home', '/') - # ... -.. rst-class:: build -.. container:: +.. nextslide:: + +.. ifslides:: - Our code template created a sample route for us, using the ``add_route`` - method of the ``Configurator`` class. + What does IMAP look like? - The ``add_route`` method has two required arguments: a *name* and a - *pattern* +IMAP (Ask for information, provide answers [Get message sizes]):: - In our sample route, the *name* is ``'home'`` + C (-->): A0003 FETCH 1 RFC822.SIZE + S (<--): * 1 FETCH (RFC822.SIZE 2545) + S (<--): A0003 OK FETCH completed - In our sample route, the *pattern* is ``'/'`` .. nextslide:: -When a request comes in to a Pyramid application, the framework looks at all -the *routes* that have been configured. +.. ifslides:: -.. rst-class:: build -.. container:: + What does IMAP look like? - One by one, in order, it tries to match the *path* of the incoming request - against the *pattern* of the route. +IMAP (Ask for information, provide answers [Get first message header]):: - As soon as a *pattern* matches the *path* from the incoming request, that - route is used and no further matching is performed. + C (-->): A0004 FETCH 1 BODY[HEADER] + S (<--): * 1 FETCH (RFC822.HEADER {1425} + + S (<--): ) + S (<--): A0004 OK FETCH completed - If no route is found that matches, then the request will automatically get - a **404 Not Found** error response. - In our sample app, we have one sample *route* named ``'home'``, with a - pattern of ``/``. +.. nextslide:: - This means that any request that comes in for ``/`` will be matched to this - route, and any other request will be **404**. +.. ifslides:: -.. nextslide:: Routes as API + What does IMAP look like? -In a very real sense, the *routes* defined in an application *are* the public -API. +IMAP (Ask for information, provide answers [Get first message body]):: -.. rst-class:: build -.. container:: + C (-->): A0005 FETCH 1 BODY[TEXT] + S (<--): * 1 FETCH (BODY[TEXT] {1120} + + S (<--): ) + S (<--): * 1 FETCH (FLAGS (\Recent \Seen)) + S (<--): A0005 OK FETCH completed - Any route that is present represents something the user can do. +.. nextslide:: - Any route that is not present is something the user cannot do. +.. ifslides:: - You can use the proper definition of routes to help conceptualize what your - app will do. + What does IMAP look like? - What routes might we want for a learning journal application? +IMAP (Say goodbye):: - What will our application do? + C (-->): A0006 LOGOUT + S (<--): * BYE example.com IMAP4rev1 server terminating connection + S (<--): A0006 OK LOGOUT completed + C (-->): -.. nextslide:: Defining our Routes -Let's add routes for our application. +.. nextslide:: IMAP Characteristics .. rst-class:: build -.. container:: - Open ``learning_journal/__init__.py``. +* Interaction consists of commands and replies +* Each command or reply is *one line* terminated by +* Each command has a *verb* and one or more *arguments* +* Each reply has a formal *code* and an informal *explanation* - For our list page, the existing ``'home'`` route will do fine, leave it. - Add the following two routes: +.. nextslide:: IMAP Differences - .. code-block:: python +.. rst-class:: build +.. container:: - config.add_route('home', '/') # already there - config.add_route('detail', '/journal/{id:\d+}') - config.add_route('action', '/journal/{action}') + .. rst-class:: build - The ``'detail'`` route will serve a single journal entry, identified by an - ``id``. + * Commands and replies are prefixed by 'sequence identifier' + * Payloads are prefixed by message size, rather than terminated by reserved + sequence - The ``action`` route will serve ``create`` and ``edit`` views, depending on - the ``action`` specified. + Compared with POP3, what do these differences suggest? - In both cases, we want to capture a portion of the matched path to use - information it provides. -.. nextslide:: Matching an ID +Using IMAP in Python +-------------------- -In a pattern, you can capture a ``path segment`` *replacement -marker*, a valid Python symbol surrounded by curly braces: +Let's try this out for ourselves! .. rst-class:: build .. container:: - :: + .. container:: + + Fire up your python interpreters and prepare to type. + - /home/{foo}/ +.. nextslide:: - If you want to match a particular pattern, like digits only, add a - *regexp*:: +Begin by importing the ``imaplib`` module from the Python Standard Library: - /journal/{id:\d+} +.. rst-class:: build +.. container:: - Matched path segments are captured in a ``matchdict``:: + .. code-block:: ipython - # pattern # actual url # matchdict - /journal/{id:\d+} /journal/27 {'id': '27'} + In [1]: import imaplib + In [2]: dir(imaplib) + Out[2]: + ['AllowedVersions', + 'CRLF', + 'Commands', + ... + 'timedelta', + 'timezone'] + In [3]: imaplib.Debug = 4 - The ``matchdict`` is made available as an attribute of the *request* + Setting ``imap.Debug`` shows us what is sent and received -.. nextslide:: Connecting Routes to Views +.. nextslide:: -In Pyramid, a *route* is connected by configuration to a *view*. +I've prepared a server for us to use, but we'll need to set up a client to +speak to it. .. rst-class:: build .. container:: - In our app, a sample view has been created for us, in ``views.py``: + Our server requires SSL (Secure Socket Layer) for connecting to IMAP + servers, so let's initialize an IMAP4_SSL client and authenticate: + + .. code-block:: ipython + + In [4]: conn = imaplib.IMAP4_SSL('mail.webfaction.com') + 22:40.32 imaplib version 2.58 + 22:40.32 new IMAP4 connection, tag=b'IMKC' + 22:40.38 < b'* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready.' + 22:40.38 > b'IMKC0 CAPABILITY' + 22:40.45 < b'* CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN' + 22:40.45 < b'IMKC0 OK Capability completed.' + 22:40.45 CAPABILITIES: ('IMAP4REV1', 'LITERAL+', 'SASL-IR', 'LOGIN-REFERRALS', 'ID', 'ENABLE', 'IDLE', 'AUTH=PLAIN') + In [5]: conn.login('crisewing_demobox', 's00p3rs3cr3t') + 22:59.92 > b'IMKC1 LOGIN crisewing_demobox "s00p3rs3cr3t"' + 23:01.79 < b'* CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS STARTTLS AUTH=PLAIN' + 23:01.79 < b'IMKC1 OK Logged in.' + Out[5]: ('OK', [b'Logged in.']) - .. code-block:: python +.. nextslide:: + +We can start by listing the mailboxes we have on the server: - @view_config(route_name='home', renderer='templates/mytemplate.pt') - def my_view(request): - # ... +.. code-block:: ipython - The order in which *routes* are configured *is important*, so that must be - done in ``__init__.py``. + In [6]: conn.list() + 26:30.64 > b'IMKC2 LIST "" *' + 26:30.72 < b'* LIST (\\HasNoChildren) "." "Trash"' + 26:30.72 < b'* LIST (\\HasNoChildren) "." "Drafts"' + 26:30.72 < b'* LIST (\\HasNoChildren) "." "Sent"' + 26:30.72 < b'* LIST (\\HasNoChildren) "." "Junk"' + 26:30.72 < b'* LIST (\\HasNoChildren) "." "INBOX"' + 26:30.72 < b'IMKC2 OK List completed.' + Out[6]: + ('OK', + [b'(\\HasNoChildren) "." "Trash"', + b'(\\HasNoChildren) "." "Drafts"', + b'(\\HasNoChildren) "." "Sent"', + b'(\\HasNoChildren) "." "Junk"', + b'(\\HasNoChildren) "." "INBOX"']) - The order in which views are connected to routes *is not important*, so the - *declarative* ``@view_config`` decorator can be used. - When ``config.scan`` is called, all files in our application are searched - for such *declarative configuration* and it is added. +.. nextslide:: -The Pyramid View ----------------- +To interact with our email, we must select a mailbox from the list we received +earlier: -Let's imagine that a *request* has come to our application for the path -``'/'``. +.. code-block:: ipython -.. rst-class:: build -.. container:: + In [7]: conn.select('INBOX') + 27:20.96 > b'IMKC3 SELECT INBOX' + 27:21.04 < b'* FLAGS (\\Answered \\Flagged \\Deleted \\Seen \\Draft)' + 27:21.04 < b'* OK [PERMANENTFLAGS (\\Answered \\Flagged \\Deleted \\Seen \\Draft \\*)] Flags permitted.' + 27:21.04 < b'* 1 EXISTS' + 27:21.04 < b'* 0 RECENT' + 27:21.04 < b'* OK [UNSEEN 1] First unseen.' + 27:21.04 < b'* OK [UIDVALIDITY 1357449499] UIDs valid' + 27:21.04 < b'* OK [UIDNEXT 24] Predicted next UID' + 27:21.04 < b'IMKC3 OK [READ-WRITE] Select completed.' + Out[7]: ('OK', [b'1']) - The framework made a match of that path to a *route* with the pattern ``'/'``. - Configuration connected that route to a *view* in our application. +.. nextslide:: - Now, the view that was connected will be *called*, which brings us to the - nature of *views* +We can search our selected mailbox for messages matching one or more criteria. - .. rst-class:: centered +.. rst-class:: build +.. container:: - --A Pyramid view is a *callable* that takes *request* as an argument-- + The return value is a list of bytestrings containing the UIDs of messages + that match our search: - Remember what a *callable* is? + .. code-block:: ipython -.. nextslide:: What the View Does + In [8]: conn.search(None, '(FROM "cris")') + 28:43.02 > b'IMKC4 SEARCH (FROM "cris")' + 28:43.09 < b'* SEARCH 1' + 28:43.09 < b'IMKC4 OK Search completed.' + Out[8]: ('OK', [b'1']) -So, a *view* is a callable that takes the *request* as an argument. +.. nextslide:: + +Once we've found a message we want to look at, we can use the ``fetch`` +command to read it from the server. .. rst-class:: build .. container:: - It can then use information from that request to build appropriate data, - perhaps using the application's *models*. + IMAP allows fetching each part of a message independently: - Then, it returns the data it assembled, passing it on to a `renderer`_. + .. code-block:: ipython - Which *renderer* to use is determined, again, by configuration: + In [9]: conn.fetch('1', 'BODY[HEADER]') + ... + Out[9]: ('OK', ...) - .. code-block:: python + In [10]: conn.fetch('1', 'FLAGS') + ... + Out[10]: ('OK', [b'1 (FLAGS (\\Seen))']) - @view_config(route_name='home', renderer='templates/mytemplate.pt') - def my_view(request): - # ... + In [11]: conn.fetch('1', 'BODY[TEXT]') + ... + Out[11]: ('OK', ...) - More about this in a moment. + What does the message say? - The *view* stands at the intersection of *input data*, the application - *model* and *renderers* that offer rendering of the results. +.. nextslide:: Batteries Included - It is the *Controller* in our MVC application. +Python even includes an *email* library that would allow us to interact with +this message in an *OO* style. -.. _renderer: http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/renderers.html +.. rst-class:: build +.. container:: -.. nextslide:: Adding Stub Views + *Neat, Huh?* -Add temporary views to our application in ``views.py`` (and comment out the -sample view): +What Have We Learned? +--------------------- -.. code-block:: python +.. rst-class:: build +.. container:: - @view_config(route_name='home', renderer='string') - def index_page(request): - return 'list page' + .. rst-class:: build - @view_config(route_name='detail', renderer='string') - def view(request): - return 'detail page' + * Protocols are just a set of rules for how to communicate - @view_config(route_name='action', match_param='action=create', renderer='string') - def create(request): - return 'create page' + * Protocols tell us how to parse and delimit messages - @view_config(route_name='action', match_param='action=edit', renderer='string') - def update(request): - return 'edit page' + * Protocols tell us what messages are valid -.. nextslide:: Testing Our Views + * If we properly format request messages to a server, we can get response + messages -Now we can verify that our view configuration has worked. + * Python supports a number of these protocols -.. rst-class:: build -.. container:: + * So we don't have to remember how to format the commands ourselves - Make sure your virtualenv is properly activated, and start the web server: + But in every case we've seen, we could do the same thing with a socket and + some strings - .. code-block:: bash - (ljenv)$ pserve development.ini - Starting server in PID 84467. - serving on http://0.0.0.0:6543 +Break Time +---------- - Then try viewing some of the expected application urls: +Let's take a few minutes here to clear our heads. - .. rst-class:: build +.. rst-class:: build +.. container:: - * http://localhost:6543/ - * http://localhost:6543/journal/1 - * http://localhost:6543/journal/create - * http://localhost:6543/journal/edit + When we return, we'll learn about the king of protocols, - What happens if you visit a URL that *isn't* in our configuration? + .. rst-class:: large centered -.. nextslide:: Interacting With the Model + HTTP -Now that we've got temporary views that work, we can fix them to get -information from our database -.. rst-class:: build +HTTP +==== + +.. rst-class:: left .. container:: - We'll begin with the list view. + HTTP is no different - We need some code that will fetch all the journal entries we've written, in - reverse order, and hand that collection back for rendering. + .. rst-class:: build + .. container:: - .. code-block:: python + HTTP is also message-centered, with two-way communications: - from .models import ( - DBSession, - MyModel, - Entry, # <- Add this import - ) + .. rst-class:: build - # and update this view function - def index_page(request): - entries = Entry.all() - return {'entries': entries} + * Requests (Asking for information) + * Responses (Providing answers) -.. nextslide:: Using the ``matchdict`` -Next, we want to write the view for a single entry. +What does HTTP look like? +------------------------- -.. rst-class:: build -.. container:: +HTTP (Ask for information): - We'll need to use the ``id`` value our route captures into the - ``matchdict``. +.. code-block:: http - Remember that the ``matchdict`` is an attribute of the request. + GET /index.html HTTP/1.1 + Host: www.example.com + - We'll get the ``id`` from there, and use it to get the correct entry. +.. ifnotslides:: - .. code-block:: python + .. note:: the ```` you see here is a visualization of the ``\r\n`` + character sequence. - # add this import at the top - from pyramid.exceptions import HTTPNotFound +.. ifslides:: - # and update this view function: - def blog_view(request): - this_id = request.matchdict.get('id', -1) - entry = Entry.by_id(this_id) - if not entry: - return HTTPNotFound() - return {'entry': entry} + **note**: the ```` you see here is a visualization of the ``\r\n`` + character sequence. -.. nextslide:: Testing Our Views -We can now verify that these views work correctly. +.. nextslide:: -.. rst-class:: build -.. container:: +HTTP (Provide answers): - Make sure your virtualenv is properly activated, and start the web server: +.. code-block:: http - .. code-block:: bash + HTTP/1.1 200 OK + Date: Mon, 23 May 2005 22:38:34 GMT + Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux) + Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT + Etag: "3f80f-1b6-3e1cb03b" + Accept-Ranges: none + Content-Length: 438 + Connection: close + Content-Type: text/html; charset=UTF-8 + + \n\n \n This is a .... </html> - (ljenv)$ pserve development.ini - Starting server in PID 84467. - serving on http://0.0.0.0:6543 +Pay particular attention to the ``<CRLF>`` on a line by itself. - Then try viewing the list page and an entry page: - * http://localhost:6543 - * http://localhost:6543/journal/1 +.. nextslide:: HTTP Core Format - What happens when you request an entry with an id that isn't in the - database? +In HTTP, both *request* and *response* share a common basic format: - * http://localhost:6543/journal/100 +.. rst-class:: build -The MVC View -============ +* Line separators are <CRLF> (familiar, no?) +* A required initial line (a command or a response code) +* A (mostly) optional set of headers, one per line +* A blank line +* An optional body -.. rst-class:: left -.. container:: - Again, back to the *Model-View-Controller* pattern. +Implementing HTTP +----------------- - .. figure:: http://upload.wikimedia.org/wikipedia/commons/4/40/MVC_passive_view.png - :align: center - :width: 25% +Let's investigate the HTTP protocol a bit in real life. - By Alan Evangelista (Own work) [CC0], via Wikimedia Commons +.. rst-class:: build +.. container:: - .. rst-class:: build - .. container:: + We'll do so by building a simplified HTTP server, one step at a time. - We've built a *model* and we've created some *controllers* that use it. + There is a copy of the echo server from last time in + ``resources/session02``. It's called ``http_server.py``. - In Pyramid, we call *controllers* **views** and they are callables that - take *request* as an argument. + In a terminal, move into that directory. We'll be doing our work here for + the rest of the session - Let's turn to the last piece of the *MVC* patter, the *view* -Presenting Data ---------------- +.. nextslide:: TDD IRL (a quick aside) -The job of the *view* in the *MVC* pattern is to present data in a format that -is readable to the user of the system. +Test Driven Development (TDD) is all the rage these days. .. rst-class:: build .. container:: - There are many ways to present data. - - Some are readable by humans (tables, charts, graphs, HTML pages, text - files). + It means that before you write code, you first write tests demonstrating + what you want your code to do. - Some are more for machines (xml files, csv, json). + When all your tests pass, you are finished. You did this for your last + assignment. - Which of these formats is the *right one* depends on your purpose. + We'll be doing it again today. - What is the purpose of our learning journal? -Pyramid Renderers ------------------ +.. nextslide:: Run the Tests -In Pyramid, the job of presenting data is performed by a *renderer*. +From inside ``resources/session02`` start a second python interpreter and run +``$ python http_server.py`` .. rst-class:: build .. container:: - So we can consider the Pyramid **renderer** to be the *view* in our *MVC* - app. + In your first interpreter run the tests. You should see similar output: - We've already seen how we can connect a *renderer* to a Pyramid *view* with - configuration. + .. code-block:: bash - In fact, we have already done so, using a built-in renderer called - ``'string'``. + $ python tests.py + [...] + Ran 10 tests in 0.054s - This renderer converts the return value of its *view* to a string and sends - that back to the client as an HTTP response. + FAILED (failures=3, errors=7) - But the result isn't so nice looking. + Let's take a few minutes here to look at these tests and understand them. -.. nextslide:: Template Renderers -The `built-in renderers` (``'string'``, ``'json'``, ``'jsonp'``) in Pyramid are -not the only ones available. +.. nextslide:: Viewing an HTTP Request -.. _built-in renderers: http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/renderers.html#built-in-renderers +Our job is to make all those tests pass. .. rst-class:: build .. container:: - There are add-ons to Pyramid that support using various *template - languages* as renderers. + First, though, let's pretend this server really is a functional HTTP + server. - In fact, one of these was installed by default when you created this - project. + This time, instead of using the echo client to make a connection to the + server, let's use a web browser! -.. nextslide:: Configuring a Template Renderer + Point your favorite browser at ``http://localhost:10000`` -.. code-block:: python - # in setup.py - requires = [ - # ... - 'pyramid_chameleon', - # ... - ] +.. nextslide:: A Bad Interaction - # in learning_journal/__init__.py - def main(global_config, **settings): - # ... - config.include('pyramid_chameleon') +First, look at the printed output from your echo server. .. rst-class:: build .. container:: - The `pyramid_chameleon` package supports using the `chameleon` template - language. + Second, note that your browser is still waiting to finish loading the page - The language is quite nice and powerful, but not so easy to learn. + Moreover, your server should also be hung, waiting for more from the + 'client' - Let's use a different one, *jinja2* + This is because the server is waiting for the browser to respond -.. nextslide:: Changing Template Renderers + And at the same time, the browser is waiting for the server to indicate it + is done. -Change ``pyramid_chameleon`` to ``pyramid_jinja2`` in both of these files: + Our server does not yet speak the HTTP protocol, but the browser is + expecting it. -.. code-block:: python +.. nextslide:: Echoing A Request + +Kill your server with ``ctrl-c`` (the keyboard interrupt) and you should see +some printed content in your browser: + +.. rst-class:: build +.. container:: - # in setup.py - requires = [ - # ... - 'pyramid_jinja2', - # ... - ] + .. code-block:: http - # in learning_journal/__init__.py - def main(global_config, **settings): - # ... - config.include('pyramid_jinja2') + GET / HTTP/1.1 + Host: localhost:10000 + User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:22.0) Gecko/20100101 Firefox/22.0 + Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 + Accept-Language: en-US,en;q=0.5 + Accept-Encoding: gzip, deflate + DNT: 1 + Cookie: __utma=111872281.383966302.1364503233.1364503233.1364503233.1; __utmz=111872281.1364503233.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); csrftoken=uiqj579iGRbReBHmJQNTH8PFfAz2qRJS + Connection: keep-alive + Cache-Control: max-age=0 -.. nextslide:: Picking up the Changes + Your server is simply echoing what it receives, so this is an *HTTP + Request* as sent by your browser. -We've changed the dependencies for our Pyramid project. +.. nextslide:: HTTP Debugging + + +When working on HTTP applications, it's nice to be able to see all this going back +and forth. .. rst-class:: build .. container:: - As a result, we will need to re-install it so the new dependencies are also - installed: + Good browsers support this with a set of developer tools built-in. - .. code-block:: bash + .. rst-class:: build - (ljenv)$ python setup.py develop - ... - Finished processing dependencies for learning-journal==0.0 - (ljenv)$ + * firefox -> ctrl-shift-K or cmd-opt-K (os X) + * safari -> enable in preferences:advanced then cmd-opt-i + * chrome -> ctrl-shift-i or cmd-opt-i (os X) + * IE (7.0+) -> F12 or tools menu -> developer tools - Now, we can use *Jinja2* templates in our project. + The 'Net(work)' pane of these tools can show you both request and response, + headers and all. Very useful. - Let's learn a bit about how `Jinja2 templates`_ work. -.. _Jinja2 templates: http://jinja.pocoo.org/docs/templates/ +.. nextslide:: Stop! Demo Time + +.. rst-class:: centered + +**Let's take a quick look** -Jinja2 Template Basics ----------------------- -We'll start with the absolute basics. +.. nextslide:: Other Debugging Options + +Sometimes you need or want to debug http requests that are not going through +your browser. .. rst-class:: build .. container:: - Fire up a Python interpreter, using your `ljenv` virtualenv: + Or perhaps you need functionality that is not supported by in-browser tools + (request munging, header mangling, decryption of https request/responses) - .. code-block:: bash + Then it might be time for an HTTP debugging proxy: - (ljenv)$ python - >>> + .. rst-class:: build - Then import the ``Template`` class from the ``jinja2`` package: + * windows: http://www.fiddler2.com/fiddler2/ + * win/osx/linux: http://www.charlesproxy.com/ - .. code-block:: pycon + We won't cover any of these tools here today. But you can check them out + when you have the time. - >>> from jinja2 import Template -.. nextslide:: Templates are Strings +Step 1: Basic HTTP Protocol +--------------------------- -A template is constructed with a simple string: +In HTTP 1.0, the only required line in an HTTP request is this: -.. code-block:: python +.. code-block:: http - >>> t1 = Template("Hello {{ name }}, how are you?") + GET /path/to/index.html HTTP/1.0<CRLF> + <CRLF> .. rst-class:: build .. container:: - Here, we've simply typed the string directly, but it is more common to - build a template from the contents of a *file*. + As virtual hosting grew more common, that was not enough, so HTTP 1.1 adds + a single required *header*, **Host**: - Notice that our string has some odd stuff in it: ``{{ name }}``. + .. code-block:: http - This is called a placeholder and when the template is *rendered* it is - replaced. + GET /path/to/index.html HTTP/1.1<CRLF> + Host: www.mysite1.com:80<CRLF> + <CRLF> -.. nextslide:: Rendering a Template -Call the ``render`` method, providing *context*: +.. nextslide:: HTTP Responses -.. code-block:: python +In both HTTP 1.0 and 1.1, a proper response consists of an intial line, +followed by optional headers, a single blank line, and then optionally a +response body: - >>> t1.render(name="Freddy") - u'Hello Freddy, how are you?' - >>> t1.render({'name': "Roberto"}) - u'Hello Roberto, how are you?' - >>> +.. rst-class:: build +.. container:: + + .. code-block:: http + + HTTP/1.1 200 OK<CRLF> + Content-Type: text/plain<CRLF> + <CRLF> + this is a pretty minimal response + + Let's update our server to return such a response. + +.. nextslide:: Returning a Canned HTTP Response + +Begin by implementing a new function in your ``http_server.py`` script called +`response_ok`. .. rst-class:: build .. container:: - *Context* can either be keyword arguments, or a dictionary + It can be super-simple for now. We'll improve it later. - Note the resemblance to something you've seen before: + .. container:: - .. code-block:: python - - >>> "This is {owner}'s string".format(owner="Cris") - 'This is Cris's string' + It needs to return our minimal response from above: + .. code-block:: http -.. nextslide:: Dictionaries in Context + HTTP/1.1 200 OK<CRLF> + Content-Type: text/plain<CRLF> + <CRLF> + this is a pretty minimal response -Dictionaries passed in as part of the *context* can be addressed with *either* -subscript or dotted notation: + **Remember, <CRLF> is a placeholder for the** ``\r\n`` **character sequence** -.. code-block:: python - >>> person = {'first_name': 'Frank', - ... 'last_name': 'Herbert'} - >>> t2 = Template("{{ person.last_name }}, {{ person['first_name'] }}") - >>> t2.render(person=person) - u'Herbert, Frank' +.. nextslide:: My Solution -.. rst-class:: build +.. code-block:: python -* Jinja2 will try the *correct* way first (attr for dotted, item for - subscript). -* If nothing is found, it will try the opposite. -* If nothing is found, it will return an *undefined* object. + def response_ok(): + """returns a basic HTTP response""" + resp = [] + resp.append(b"HTTP/1.1 200 OK") + resp.append(b"Content-Type: text/plain") + resp.append(b"") + resp.append(b"this is a pretty minimal response") + return b"\r\n".join(resp) +Did you remember that sockets only accept bytes? -.. nextslide:: Objects in Context -The exact same is true of objects passed in as part of *context*: +.. nextslide:: Run The Tests + +We've now implemented a function that is tested by our tests. Let's run them +again: .. rst-class:: build .. container:: - .. code-block:: python - - >>> t3 = Template("{{ obj.x }} + {{ obj['y'] }} = Fun!") - >>> class Game(object): - ... x = 'babies' - ... y = 'bubbles' - ... - >>> bathtime = Game() - >>> t3.render(obj=bathtime) - u'babies + bubbles = Fun!' - - This means your templates can be a bit agnostic as to the nature of the - things in *context* + .. code-block:: bash -.. nextslide:: Filtering values in Templates + $ python tests.py + [...] + ---------------------------------------------------------------------- + Ran 10 tests in 0.002s -You can apply `filters`_ to the data passed in *context* with the pipe ('|') -operator: + FAILED (failures=3, errors=3) -.. _filters: http://jinja.pocoo.org/docs/dev/templates/#filters + Great! We've now got 4 tests that pass. Good work. -.. code-block:: python +.. nextslide:: Server Modifications - t4 = Template("shouted: {{ phrase|upper }}") - >>> t4.render(phrase="this is very important") - u'shouted: THIS IS VERY IMPORTANT' +Next, we need to rebuild the server loop from our echo server for it's new +purpose: .. rst-class:: build .. container:: - You can also chain filters together: + It should now wait for an incoming request to be *finished*, *then* send a + response back to the client. - .. code-block:: python + The response it sends can be the result of calling our new ``response_ok`` + function for now. + + We could also bump up the ``recv`` buffer size to something more reasonable + for HTTP traffic, say 1024. + +.. nextslide:: My Solution + +.. code-block:: python + + # ... + try: + while True: + print('waiting for a connection', file=log_buffer) + conn, addr = sock.accept() # blocking + try: + print('connection - {0}:{1}'.format(*addr), file=log_buffer) + while True: + data = conn.recv(1024) + if len(data) < 1024: + break + print('sending response', file=log_buffer) + response = response_ok() + conn.sendall(response) + finally: + conn.close() + # ... - t5 = Template("confusing: {{ phrase|upper|reverse }}") - >>> t5.render(phrase="howdy doody") - u'confusing: YDOOD YDWOH' -.. nextslide:: Control Flow +.. nextslide:: Run The Tests -Logical `control structures`_ are also available: +Once you've got that set, restart your server:: -.. _control structures: http://jinja.pocoo.org/docs/dev/templates/#list-of-control-structures + $ python http_server.py .. rst-class:: build .. container:: - .. code-block:: python + Then you can re-run your tests: - tmpl = """ - ... {% for item in list %}{{ item }}, {% endfor %} - ... """ - >>> t6 = Template(tmpl) - >>> t6.render(list=[1,2,3,4,5,6]) - u'\n1, 2, 3, 4, 5, 6, ' + .. code-block:: bash - Any control structure introduced in a template **must** be paired with an - explicit closing tag ({% for %}...{% endfor %}) + $ python tests.py + [...] + ---------------------------------------------------------------------- + Ran 10 tests in 0.003s - Remember, although template tags like ``{% for %}`` or ``{% if %}`` look a - lot like Python, they are not. + FAILED (failures=2, errors=3) - The syntax is specific and must be followed correctly. + Five tests now pass! -.. nextslide:: Template Tests +Step 2: Handling HTTP Methods +----------------------------- -There are a number of specialized *tests* available for use with the -``if...elif...else`` control structure: +Every HTTP request **must** begin with a single line, broken by whitespace into +three parts: -.. code-block:: python +.. code-block:: http - >>> tmpl = """ - ... {% if phrase is upper %} - ... {{ phrase|lower }} - ... {% elif phrase is lower %} - ... {{ phrase|upper }} - ... {% else %}{{ phrase }}{% endif %}""" - >>> t7 = Template(tmpl) - >>> t7.render(phrase="FOO") - u'\n\n foo\n' - >>> t7.render(phrase="bar") - u'\n\n BAR\n' - >>> t7.render(phrase="This should print as-is") - u'\nThis should print as-is' + GET /path/to/index.html HTTP/1.1 +.. rst-class:: build +.. container:: -.. nextslide:: Basic Expressions + The three parts are the *method*, the *URI*, and the *protocol* -Basic `Python-like expressions`_ are also supported: + Let's look at each in turn. -.. _Python-like expressions: http://jinja.pocoo.org/docs/dev/templates/#expressions -.. code-block:: python +.. nextslide:: HTTP Methods - tmpl = """ - ... {% set sum = 0 %} - ... {% for val in values %} - ... {{ val }}: {{ sum + val }} - ... {% set sum = sum + val %} - ... {% endfor %} - ... """ - >>> t8 = Template(tmpl) - >>> t8.render(values=range(1,11)) - u'\n\n\n1: 1\n \n\n2: 3\n \n\n3: 6\n \n\n4: 10\n - \n\n5: 15\n \n\n6: 21\n \n\n7: 28\n \n\n8: 36\n - \n\n9: 45\n \n\n10: 55\n \n' +**GET** ``/path/to/index.html HTTP/1.1`` +.. rst-class:: build -Our Templates -------------- +* Every HTTP request must start with a *method* +* There are four main HTTP methods: -There's more that Jinja2 templates can do, but it will be easier to introduce -you to that in the context of a working template. So let's make some. + .. rst-class:: build -.. nextslide:: Detail Template + * GET + * POST + * PUT + * DELETE -We have a Pyramid view that returns a single entry. Let's create a template to -show it. +* There are others, notably HEAD, but you won't see them too much -.. rst-class:: build -.. container:: - In ``learning_journal/templates`` create a new file ``detail.jinja2``: +.. nextslide:: HTTP Methods - .. code-block:: jinja - - <article> - <h1>{{ entry.title }}</h1> - <hr/> - <p>{{ entry.body }}</p> - <hr/> - <p>Created <strong title="{{ entry.created }}">{{entry.created}}</strong></p> - </article> +These four methods are mapped to the four basic steps (*CRUD*) of persistent +storage: - Then wire it up to the detail view in ``views.py``: +.. rst-class:: build + +* POST = Create +* GET = Read +* PUT = Update +* DELETE = Delete - .. code-block:: python - - # views.py - @view_config(route_name='detail', renderer='templates/detail.jinja2') - def blog_view(request): - # ... -.. nextslide:: Try It Out +.. nextslide:: Methods: Safe <--> Unsafe -Now we should be able to see some rendered HTML for our journal entry details. +HTTP methods can be categorized as **safe** or **unsafe**, based on whether +they might change something on the server: .. rst-class:: build .. container:: - Start up your server: + .. rst-class:: build - .. code-block:: bash + * Safe HTTP Methods + + * GET - (ljenv)$ pserve development.ini - Starting server in PID 90536. - serving on http://0.0.0.0:6543 + * Unsafe HTTP Methods - Then try viewing an individual journal entry + * POST + * PUT + * DELETE - * http://localhost:6543/journal/1 + This is a *normative* distinction, which is to say **be careful** -.. nextslide:: Listing Page -The index page of our journal should show a list of journal entries, let's do -that next. +.. nextslide:: Methods: Idempotent <--> ??? + +HTTP methods can be categorized as **idempotent**. .. rst-class:: build .. container:: - In ``learning_journal/templates`` create a new file ``list.jinja2``: + This means that a given request will always have the same result: + + .. rst-class:: build - .. code-block:: jinja + * Idempotent HTTP Methods - {% if entries %} - <h2>Journal Entries</h2> - <ul> - {% for entry in entries %} - <li> - <a href="{{ request.route_url('detail', id=entry.id) }}">{{ entry.title }}</a> - </li> - {% endfor %} - </ul> - {% else %} - <p>This journal is empty</p> - {% endif %} + * GET + * PUT + * DELETE -.. nextslide:: + * Non-Idempotent HTTP Methods -It's worth taking a look at a few specifics of this template. + * POST -.. rst-class:: build -.. container:: + Again, *normative*. The developer is responsible for ensuring that it is true. - .. code-block:: jinja - - <a href="{{ request.route_url('detail', id=entry.id) }}">{{ entry.title }}</a> - Jinja2 templates are rendered with a *context*. +.. nextslide:: HTTP Method Handling - The return values of the Pyramid *view* for a template get included in that - context. +Let's keep things simple, our server will only respond to *GET* requests. - So does *request*, which is placed there by the framework. +.. rst-class:: build +.. container:: - Request has a method ``route_url`` that will create a URL for a named - route. + We need to create a function that parses a request and determines if we can + respond to it: ``parse_request``. - This allows you to include URLs in your template without needing to know - exactly what they will be. + If the request method is not *GET*, our method should raise an error - This process is called *reversing*, since it's a bit like a reverse phone - book lookup. + Remember, although a request is more than one line long, all we care about + here is the first line -.. nextslide:: -Finally, you'll need to connect this new renderer to your listing view: +.. nextslide:: My Solution .. code-block:: python - @view_config(route_name='home', renderer='templates/list.jinja2') - def index_page(request): - # ... + def parse_request(request): + first_line = request.split("\r\n", 1)[0] + method, uri, protocol = first_line.split() + if method != "GET": + raise NotImplementedError("We only accept GET") + print('request is okay', file=sys.stderr) + -.. nextslide:: Try It Out +.. nextslide:: Update the Server -We can now see our list page too. Let's try starting the server: +We'll also need to update the server code. It should .. rst-class:: build -.. container:: - .. code-block:: bash +* save the request as it comes in +* check the request using our new function +* send an OK response if things go well - (ljenv)$ pserve development.ini - Starting server in PID 90536. - serving on http://0.0.0.0:6543 - Then try viewing the home page of your journal: +.. nextslide:: My Solution - * http://localhost:6543/ +.. code-block:: python - Click on the link to an entry, it should work. + # ... + conn, addr = sock.accept() # blocking + try: + print('connection - {0}:{1}'.format(*addr), file=log_buffer) + request = "" + while True: + data = conn.recv(1024) + request += data.decode('utf8') + if len(data) < 1024 or not data: + break -.. nextslide:: Sharing Structure + parse_request(request) + print('sending response', file=log_buffer) + response = response_ok() + conn.sendall(response) + finally: + conn.close() + # ... -These views are reasonable, if quite plain. -.. rst-class:: build -.. container:: +.. nextslide:: Run The Tests - It'd be nice to put them into something that looks a bit more like a - website. +Quit and restart your server now that you've updated the code:: - Jinja2 allows you to combine templates using something called - `template inheritance`_. + $ python http_server.py - You can create a basic page structure, and then *inherit* that structure in - other templates. +.. rst-class:: build +.. container:: - In our class resources I've added a page template ``layout.jinja2``. Copy - that page to your templates directory + At this point, we should have seven tests passing: -.. _template inheritance: http://jinja.pocoo.org/docs/dev/templates/#template-inheritance + .. code-block:: bash -.. nextslide:: ``layout.jinja2`` + $ python tests.py + Ran 10 tests in 0.002s -.. code-block:: jinja + FAILED (failures=1, errors=2) - <!DOCTYPE html> - <html lang="en"> - <head> - <meta charset="utf-8"> - <title>Python Learning Journal - - - -
          - -
          -
          -

          My Python Journal

          -
          {% block body %}{% endblock %}
          -
          -

          Created in the UW PCE Python Certificate Program

          - - -.. nextslide:: Template Blocks +.. nextslide:: What About a Browser? -The important part here is the ``{% block body %}{% endblock %}`` expression. +The server quit during the tests, but an HTTP request from the browser should +work fine now. .. rst-class:: build .. container:: - This is a template **block** and it is a kind of placeholder. + Restart the server and reload your browser. You should see your OK + response. + + We can use the ``simple_client.py`` script in our resources to test our + error condition. In a second terminal window run the script like so:: - Other templates can inherit from this one, and fill that block with - additional HTML. + $ python simple_client.py "POST / HTTP/1.0\r\n\r\n" - Let's update our detail and list templates: + This should cause the server to crash. - .. code-block:: jinja - - {% extends "layout.jinja2" %} - {% block body %} - - {% endblock %} -.. nextslide:: Try It Out +Step 3: Error Responses +----------------------- -Let's try starting the server so we can see the result: +Okay, so the outcome there was pretty ugly. The client went off the rails, and +our server has terminated as well. .. rst-class:: build .. container:: - .. code-block:: bash + .. rst-class:: centered + + **why?** - (ljenv)$ pserve development.ini - Starting server in PID 90536. - serving on http://0.0.0.0:6543 + The HTTP protocol allows us to handle errors like this more gracefully. - Then try viewing the home page of your journal: + .. rst-class:: centered - * http://localhost:6543/ + **Enter the Response Code** - Click on the link to an entry, it should work. - And now you have shared page structure that is in both. +.. nextslide:: HTTP Response Codes -Static Assets -------------- +``HTTP/1.1`` **200 OK** -Although we have a shared structure, it isn't particularly nice to look at. +All HTTP responses must include a **response code** indicating the outcome of +the request. .. rst-class:: build .. container:: - Aspects of how a website looks are controlled by CSS (*Cascading Style - Sheets*). + .. rst-class:: build + + * 1xx (HTTP 1.1 only) - Informational message + * 2xx - Success of some kind + * 3xx - Redirection of some kind + * 4xx - Client Error of some kind + * 5xx - Server Error of some kind - Stylesheets are one of what we generally speak of as *static assets*. + The text bit makes the code more human-readable - Other static assets include *images* that are part of the look and feel of - the site (logos, button images, etc) and the *JavaScript* files that add - client-side dynamic behavior to the site. -.. nextslide:: Static Assets in Pyramid +.. nextslide:: Common Response Codes -Serving static assets in Pyramid requires a *static view* to configuration. -Luckily, ``pcreate`` already handled that for us: +There are certain HTTP response codes you are likely to see (and use) most +often: .. rst-class:: build .. container:: - .. code-block:: python - - # in learning_journal/__init__.py - def main(global_config, **settings): - # ... - config.add_static_view('static', 'static', cache_max_age=3600) - # ... + .. rst-class:: build - The first argument to ``add_static_view`` is a *name* that will need to - appear in the path of URLs requesting assets. + * ``200 OK`` - Everything is good + * ``301 Moved Permanently`` - You should update your link + * ``304 Not Modified`` - You should load this from cache + * ``404 Not Found`` - You've asked for something that doesn't exist + * ``500 Internal Server Error`` - Something bad happened - The second argument is a *path* that is relative to the package being - configured. + Do not be afraid to use other, less common codes in building good apps. + There are a lot of them for a reason. - Assets referenced by the *name* in a URL will be searched for in the - location defined by the *path* + See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html - Additional keyword arguments control other aspects of how the view works. -.. nextslide:: Static Assets in Templates +.. nextslide:: Handling our Error -Once you have a static view configured, you can use assets in that location in -templates. +Luckily, there's an error code that is tailor-made for this situation. .. rst-class:: build .. container:: - The *request* object in Pyramid provides a ``static_url`` method that - builds appropriate URLs + The client has made a request using a method we do not support - Add the following to our ``layout.jinja2`` template: + ``405 Method Not Allowed`` - .. code-block:: jinja - - - - - + Let's add a new function that returns this error code. It should be called + ``response_method_not_allowed`` - The one required argument to ``request.static_url`` is a *path* to an - asset. + Remember, it must be a complete HTTP Response with the correct *code* - Note that because any package *might* define a static view, we have to - specify which package we want to look in. - That's why we have ``learning_journal:static/styles.css`` in our call. +.. nextslide:: My Solution -.. nextslide:: Basic Styles +.. code-block:: python -I've created some very very basic styles for our learning journal. + def response_method_not_allowed(): + """returns a 405 Method Not Allowed response""" + resp = [] + resp.append(b"HTTP/1.1 405 Method Not Allowed") + resp.append(b"") + return b"\r\n".join(resp) -.. rst-class:: build -.. container:: - You can find them in ``resources/session02/styles.css``. Go ahead and copy - that file. +.. nextslide:: Server Updates - Add it to ``learning_journal/static``. +Again, we'll need to update the server to handle this error condition +correctly. It should - Then restart your web server and see what a difference a little style - makes: +.. rst-class:: build - .. code-block:: bash +* catch the exception raised by the ``parse_request`` function +* create our new error response as a result +* if no exception is raised, then create the OK response +* return the generated response to the user - (ljenv)$ pserve development.ini - Starting server in PID 90536. - serving on http://0.0.0.0:6543 +.. nextslide:: My Solution -.. nextslide:: The Outcome +.. code-block:: python -Your site should look something like this: + # ... + while True: + data = conn.recv(1024) + request += data.decode('utf8') + if len(data) < 1024: + break -.. figure:: /_static/learning_journal_styled.png - :align: center - :width: 75% + try: + parse_request(request) + except NotImplementedError: + response = response_method_not_allowed() + else: + response = response_ok() - The learning journal with basic styles applied + print('sending response', file=log_buffer) + conn.sendall(response) + # ... -Getting Interactive -=================== -.. rst-class:: left -.. container:: +.. nextslide:: Run The Tests - We have a site that allows us to view a list of journal entries. +Start your server (or restart it if by some miracle it's still going). - .. rst-class:: build - .. container:: +.. rst-class:: build +.. container:: - We can also view the details of a single entry. + Then run the tests again:: - But as yet, we don't really have any *interaction* in our site yet. + $ python tests.py + [...] + Ran 10 tests in 0.002s - We can't create new entries. + OK - Let's add that functionality next. + Wahoo! All our tests are passing. That means we are done writing code for + now. -User Input ----------- -In HTML websites, the traditional way of getting input from users is via -`HTML forms`_. +Step 4: Serving Resources +------------------------- + +We've got a very simple server that accepts a request and sends a response. +But what happens if we make a different request? .. rst-class:: build .. container:: - Forms use *input elements* to allow users to enter data, pick from - drop-down lists, or choose items via checkbox or radio button. + .. container:: + + In your web browser, enter the following URL:: - It is possible to create plain HTML forms in templates and use them with - Pyramid. + http://localhost:10000/page - It's a lot easier, however, to work with a *form library* to create forms, - render them in templates and interact with data sent by a client. + .. container:: - We'll be using a form library called `WTForms`_ in our project + What happened? What happens if you use this URL:: -.. _HTML forms: https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Forms -.. _WTForms: http://wtforms.readthedocs.org/en/latest/ + http://localhost:10000/section/page? -.. nextslide:: Installing WTForms -The first step to working with this library is to install it. +.. nextslide:: Determining a Resource + +We expect different urls to result in different responses. .. rst-class:: build .. container:: - Start by makin the library as a *dependency* of our package by adding it to - the *requires* list in ``setup.py``: + Each separate *path* provided should map to a *resource* - .. code-block:: python + But this isn't happening with our server, for obvious reasons. - requires = [ - # ... - 'wtforms', # <- add this to the list - ] + It brings us back to the second element of that first line of an HTTP + request. - Then, re-install our package to download and install the new dependency: + .. rst-class:: centered - .. code-block:: bash + **The Return of the URI** - (ljenv)$ python setup.py develop - ... - Finished processing dependencies for learning-journal==0.0 -Using WTForms -------------- +.. nextslide:: HTTP Requests: URI -We'll want a form to allow a user to create a new Journal Entry. +``GET`` **/path/to/index.html** ``HTTP/1.1`` .. rst-class:: build -.. container:: - Add a new file called ``forms.py`` in our learning_journal package, next to - ``models.py``: +* Every HTTP request must include a **URI** used to determine the **resource** to + be returned - .. code-block:: python - - from wtforms import Form, TextField, TextAreaField, validators +* URI?? + http://stackoverflow.com/questions/176264/whats-the-difference-between-a-uri-and-a-url/1984225#1984225 - strip_filter = lambda x: x.strip() if x else None +* Resource? Files (html, img, .js, .css), but also: - class EntryCreateForm(Form): - title = TextField( - 'Entry title', - [validators.Length(min=1, max=255)], - filters=[strip_filter]) - body = TextAreaField( - 'Entry body', - [validators.Length(min=1)], - filters=[strip_filter]) + .. rst-class:: build -.. nextslide:: Using a Form in a View + * Dynamic scripts + * Raw data + * API endpoints -Next, we need to add a new view that uses this form to create a new entry. +.. nextslide:: Parsing a Request + +Our ``parse_request`` method actually already finds the ``uri`` in the first +line of a request .. rst-class:: build .. container:: - Add this to ``views.py``: + All we need to do is update the method so that it *returns* that uri - .. code-block:: python + Then we can use it. - # add these imports - from pyramid.exceptions import HTTPFound - from .forms import EntryCreateForm +.. nextslide:: My Solution - # and update this view function - def create(request): - entry = Entry() - form = EntryCreateForm(request.POST) - if request.method == 'POST' and form.validate(): - form.populate_obj(entry) - DBSession.add(entry) - return HTTPFound(location=request.route_url('home')) - return {'form': form, 'action': request.matchdict.get('action')} +.. code-block:: python -.. nextslide:: Testing the Route/View Connection + def parse_request(request): + first_line = request.split("\r\n", 1)[0] + method, uri, protocol = first_line.split() + if method != "GET": + raise NotImplementedError("We only accept GET") + print >>sys.stderr, 'request is okay' + # add the following line: + return uri -We already have a route that connects here. Let's test it. +.. nextslide:: Pass It Along + +Now we can update our server code so that it uses the return value of +``parse_request``. .. rst-class:: build .. container:: - Start your server: + That's a pretty simple change: - .. code-block:: bash + .. code-block:: python + + try: + uri = parse_request(request) # update this line + except NotImplementedError: + response = response_method_not_allowed() + else: + # and modify this block + try: + content, mime_type = resolve_uri(url) + except NameError: + response = response_not_found() + else: + response = response_ok(content, mime_type) + +Homework +======== - (ljenv)$ pserve development.ini - Starting server in PID 90536. - serving on http://0.0.0.0:6543 +.. rst-class:: left +.. container:: - And then try connecting to the ``action`` route: + You may have noticed that we just added calls to functions that don't yet + exist - * http://localhost:6543/journal/create - - You should see something like this:: + .. rst-class:: build + .. container:: - {'action': u'create', 'form': } + It's a program that shows you what you want to do, but won't actually + run. -.. nextslide:: Rendering A Form + For your homework this week you will create these functions, completing + the HTTP server. -Finally, we need to create a template that will render our form. + Your starting point will be what we've made here in class. -.. rst-class:: build -.. container:: + I've added a directory to ``resources/session02`` called ``homework``. - Add a new template called ``edit.jinja2`` in - ``learning_journal/templates``: + In it, you'll find this ``http_server.py`` file we've just written in + class. - .. code-block:: jinja + That file also contains enough stub code for the missing functions to + let the server run. - {% extends "templates/layout.jinja2" %} - {% block body %} -
          - {% for field in form %} - {% if field.errors %} -
            - {% for error in field.errors %} -
          • {{ error }}
          • - {% endfor %} -
          - {% endif %} -

          {{ field.label }}: {{ field }}

          - {% endfor %} -

          -
          - {% endblock %} + And there are more tests for you to make pass! -.. nextslide:: Connecting the Renderer +One Step At A Time +------------------ -You'll need to update the view configuration to use this new renderer. +Take the following steps one at a time. Run the tests in +``assignments/session02/homework`` between to ensure that you are getting it +right. .. rst-class:: build -.. container:: - Update the configuration in ``learning_journal/views.py``: +* Complete the stub ``resolve_uri`` function so that it handles looking up + resources on disk using the URI returned by ``parse_request``. - .. code-block:: python - - @view_config(route_name='action', match_param='action=create', - renderer='templates/edit.jinja2') - def create(request): - # ... +* Make sure that if the URI does not map to a file that exists, it raises an + appropriate error for our server to handle. - And then you should be able to start your server and test: +* Complete the ``response_not_found`` function stub so that it returns a 404 + response. - .. code-block:: bash +* Update ``response_ok`` so that it uses the values returned by ``resolve_uri`` + by the URI. (these have already been added to the function signature) - (ljenv)$ pserve development.ini - Starting server in PID 90536. - serving on http://0.0.0.0:6543 +* You'll plug those values into the response you generate in the way required + by the protocol - * http://localhost:6543/create -.. nextslide:: Providing Access +HTTP Headers +------------ -Great! Now you can add new entries to your journal. +Along the way, you'll discover that simply returning the content of a file as +an HTTP response body is insufficient. Different *types* of content need to +be identified to your browser .. rst-class:: build .. container:: - But in order to do so, you have to hand-enter the url. + We can fix this by passing information about exactly what we are returning + as part of the response. - You should add a new link in the UI somewhere that helps you get there more - easily. + HTTP provides for this type of thing with the generic idea of *Headers* - Add the following to ``list.jinja2``: - .. code-block:: jinja +HTTP Headers +------------ - {% extends "layout.jinja2" %} - {% block body %} - {% if entries %} - ... - {% else %} - ... - {% endif %} - -

          New Entry

          - {% endblock %} - -Homework -======== +Both requests and responses can contain **headers** of the form ``Name: Value`` -.. rst-class:: left +.. rst-class:: build .. container:: - You have a website now that allows you to create, view and list journal - entries + .. rst-class:: build + + * HTTP 1.0 has 16 valid headers, 1.1 has 46 + * Any number of spaces or tabs may separate the *name* from the *value* + * If a header line starts with spaces or tabs, it is considered part of the + value for the previous header + * Header *names* are **not** case-sensitive, but *values* may be + + read more about HTTP headers: http://www.cs.tut.fi/~jkorpela/http.html + + +Content-Type Header +------------------- + +A very common header used in HTTP responses is ``Content-Type``. It tells the +client what to expect. + +.. rst-class:: build +.. container:: .. rst-class:: build - .. container:: - However, there are still a few flaws in this system. + * uses **mime-type** (Multi-purpose Internet Mail Extensions) + * foo.jpeg - ``Content-Type: image/jpeg`` + * foo.png - ``Content-Type: image/png`` + * bar.txt - ``Content-Type: text/plain`` + * baz.html - ``Content-Type: text/html`` - You should be able to edit a journal entry that already exists, in case - you make a spelling error. + There are *many* mime-type identifiers: + http://www.freeformatter.com/mime-types-list.html - It would also be nice to see a prettier site. - Let's handle that for homework this week. +Mapping Mime-types +------------------ -Part 1: Add Editing -------------------- +By mapping a given file to a mime-type, we can write a header. -For part one of your assignment, add editing of existing entries. You will need: +.. rst-class:: build +.. container:: -* A form that shows an existing entry (what is different about this form from - one for creating a new entry?) -* A pyramid view that handles that form. It should: + The standard lib module ``mimetypes`` does just this. - * Show the form with the requested entry when the page is first loaded - * Accept edits only on POST - * Update an existing entry with new data from the form - * Show the view of the entry after editing so that the user can see the edits - saved correctly - * Show errors from form validation, if any are present + We can guess the mime-type of a file based on the filename or map a file + extension to a type: -* A link somewhere that leads to the editing page for a single entry (probably - on the view page for a entry) + .. code-block:: pycon -You'll need to update a bit of configuration, but not much. Use the create -form we did here in class as an example. + >>> import mimetypes + >>> mimetypes.guess_type('file.txt') + ('text/plain', None) + >>> mimetypes.types_map['.txt'] + 'text/plain' -Part 2: Make it Yours ---------------------- -I've created for you a very bare-bones layout and stylesheet. +Resolving a URI +--------------- + +Your ``resolve_uri`` function will need to accomplish the following tasks: -You will certainly want to add a bit of your own style and panache. +.. rst-class:: build -Spend a few hours this week playing with the styles and getting a site that -looks more like you want it to look. +* It should take a URI as the sole argument -The Mozilla Developer Network has `some excellent resources`_ for learning CSS. +* It should map the pathname represented by the URI to a filesystem location. -In particular, the `Getting Started with CSS`_ tutorial is a thorough -introduction to the basics. +* It should have a 'home directory', and look only in that location. -You might also look at their `CSS 3 Demos`_ to help fire up your creative -juices. +* If the URI is a directory, it should return a plain-text listing of the + directory contents and the mimetype ``text/plain``. -Here are a few more resources: +* If the URI is a file, it should return the contents of that file and its + correct mimetype. -* `A List Apart `_ offers outstanding articles. Their - `Topics list `_ is worth a browse. -* `Smashing Magazine `_ is another excellent - resource for articles on design. +* If the URI does not map to a real location, it should raise an exception + that the server can catch to return a 404 response. -.. _some excellent resources: https://developer.mozilla.org/en-US/docs/Web/CSS -.. _Getting Started with CSS: https://developer.mozilla.org/en-US/docs/CSS/Getting_Started -.. _CSS 3 Demos: https://developer.mozilla.org/en-US/demos/tag/tech:css3 +Use Your Tests +-------------- + +One of the benefits of test-driven development is that the tests that are +failing should tell you what code you need to write. + +.. rst-class:: build +.. container:: + + As you work your way through the steps outlined above, look at your tests. + Write code that makes them pass. + + If all the tests in ``assignments/session02/tests.py`` are passing, you've + completed the assignment. -Part 3: User Model ------------------- -As it stands, our journal accepts entries from anyone who comes by. +Submitting Your Homework +------------------------ -Next week we will add security to allow only logged-in users to create and edit -entries. +To submit your homework: -To do so, we'll need a user model +.. rst-class:: build +.. container:: -The model should have: + .. rst-class:: build -* An ``id`` field that is a primary key -* A ``username`` field that is unicode, no more than 255 characters, not - nullable, unique and indexed. -* A ``password`` field that is unicode and not nullable + * Do your work in the ``assignments/session02`` directory of **your fork** of + the class respository -In addition, the model should have a classmethod that retrieves a specific user -when given a username. + * When you have all tests passing, push your work to **your fork** in github. -Part 4: Preparation for Deployment ----------------------------------- + * Using the github web interface, send me a pull request. -At the end of class next week we will be deploying our application to Heroku. + I will review your work when I receive your pull requests, make comments on + it there, and then close the pull request. -You will need to get a free account. -Once you have your free account set up and you have logged in, run through the -`getting started with Python`_ tutorial. +A Few Steps Further +------------------- -Be sure to at least complete the *set up* step. It will have you install the -Heroku Toolbelt, which you will need to have ready in class. +If you are able to finish the above in less than 4-6 hours, consider taking on +one or more of the following challenges: -.. _getting started with Python: https://devcenter.heroku.com/articles/getting-started-with-python#introduction +.. rst-class:: build +* Format directory listings as HTML, so you can link to files. +* Add a GMT ``Date:`` header in the proper format (RFC-1123) to responses. + *hint: see email.utils.formatdate in the python standard library* +* Add a ``Content-Length:`` header for ``OK`` responses that provides a + correct value. +* Protect your server against errors by providing, and using, a function that + returns a ``500 Internal Server Error`` response. +* Instead of returning the python script in ``webroot`` as plain text, execute + the file and return the results as HTML. diff --git a/source/presentations/session02.rst.norender b/source/presentations/session02.rst.norender deleted file mode 100644 index 1ab82949..00000000 --- a/source/presentations/session02.rst.norender +++ /dev/null @@ -1,1580 +0,0 @@ -Python Web Programming -====================== - -.. image:: img/protocol.png - :align: left - :width: 45% - -Session 2: Web Protocols - -.. class:: intro-blurb - -Wherein we learn about the languages that machines speak to each other - - -But First ---------- - -.. class:: big-centered - -Some boring business of identification - - -But Second ----------- - -.. class:: big-centered - -Questions from the Homework? - - -And Third ---------- - -.. class:: big-centered - -Examples of an echo server using ``select`` - - -What is a Protocol? -------------------- - -.. class:: incremental center - -a set of rules or conventions - -.. class:: incremental center - -governing communications - - -Protocols IRL -------------- - -Life has lots of sets of rules for how to do things. - -.. class:: incremental - -* What do you say when you get on the elevator? - -* What do you do on a first date? - -* What do you wear to a job interview? - -* What do (and don't) you talk about at a dinner party? - -* ...? - - -Protocols IRL -------------- - -.. image:: img/icup.png - :align: center - :width: 58% - -.. class:: image-credit - -http://blog.xkcd.com/2009/09/02/urinal-protocol-vulnerability/ - - -Protocols In Computers ----------------------- - -Digital life has lots of rules too: - -.. class:: incremental - -* how to say hello - -* how to identify yourself - -* how to ask for information - -* how to provide answers - -* how to say goodbye - - -Real Protocol Examples ----------------------- - -.. class:: big-centered - -What does this look like in practice? - - -Real Protocol Examples ----------------------- - -.. class:: incremental - -* SMTP (Simple Message Transfer Protocol) - http://tools.ietf.org/html/rfc5321#appendix-D - -* POP3 (Post Office Protocol) - http://www.faqs.org/docs/artu/ch05s03.html - -* IMAP (Internet Message Access Protocol) - http://www.faqs.org/docs/artu/ch05s03.html - -* HTTP (Hyper-Text Transfer Protocol) - http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol - - -What does SMTP look like? -------------------------- - -SMTP (Say hello and identify yourself):: - - S: 220 foo.com Simple Mail Transfer Service Ready - C: EHLO bar.com - S: 250-foo.com greets bar.com - S: 250-8BITMIME - S: 250-SIZE - S: 250-DSN - S: 250 HELP - - -What does SMTP look like? -------------------------- - -SMTP (Ask for information, provide answers):: - - C: MAIL FROM: - S: 250 OK - C: RCPT TO: - S: 250 OK - C: RCPT TO: - S: 550 No such user here - C: DATA - S: 354 Start mail input; end with . - C: Blah blah blah... - C: ...etc. etc. etc. - C: . - S: 250 OK - -What does SMTP look like? -------------------------- - -SMTP (Say goodbye):: - - C: QUIT - S: 221 foo.com Service closing transmission channel - - -SMTP Characteristics --------------------- - -.. class:: incremental - -* Interaction consists of commands and replies -* Each command or reply is *one line* terminated by -* The exception is message payload, terminated by . -* Each command has a *verb* and one or more *arguments* -* Each reply has a formal *code* and an informal *explanation* - - -What does POP3 look like? -------------------------- - -POP3 (Say hello and identify yourself):: - - C: - S: +OK POP3 server ready <1896.6971@mailgate.dobbs.org> - C: USER bob - S: +OK bob - C: PASS redqueen - S: +OK bob's maildrop has 2 messages (320 octets) - - -What does POP3 look like? -------------------------- - -POP3 (Ask for information, provide answers):: - - C: STAT - S: +OK 2 320 - C: LIST - S: +OK 1 messages (120 octets) - S: 1 120 - S: . - - -What does POP3 look like? -------------------------- - -POP3 (Ask for information, provide answers):: - - C: RETR 1 - S: +OK 120 octets - S: - S: . - C: DELE 1 - S: +OK message 1 deleted - - -What does POP3 look like? -------------------------- - -POP3 (Say goodbye):: - - C: QUIT - S: +OK dewey POP3 server signing off (maildrop empty) - C: - - -POP3 Characteristics --------------------- - -.. class:: incremental - -* Interaction consists of commands and replies -* Each command or reply is *one line* terminated by -* The exception is message payload, terminated by . -* Each command has a *verb* and one or more *arguments* -* Each reply has a formal *code* and an informal *explanation* - -.. class:: incremental - -The codes don't really look the same, though, do they? - - -One Other Difference --------------------- - -The exception to the one-line-per-message rule is *payload* - -.. class:: incremental - -In both SMTP and POP3 this is terminated by . - -.. class:: incremental - -In SMTP, the *client* has this ability - -.. class:: incremental - -But in POP3, it belongs to the *server*. Why? - - -What does IMAP look like? -------------------------- - -IMAP (Say hello and identify yourself):: - - C: - S: * OK example.com IMAP4rev1 v12.264 server ready - C: A0001 USER "frobozz" "xyzzy" - S: * OK User frobozz authenticated - - -What does IMAP look like? -------------------------- - -IMAP (Ask for information, provide answers [connect to an inbox]):: - - C: A0002 SELECT INBOX - S: * 1 EXISTS - S: * 1 RECENT - S: * FLAGS (\Answered \Flagged \Deleted \Draft \Seen) - S: * OK [UNSEEN 1] first unseen message in /var/spool/mail/esr - S: A0002 OK [READ-WRITE] SELECT completed - - -What does IMAP look like? -------------------------- - -IMAP (Ask for information, provide answers [Get message sizes]):: - - C: A0003 FETCH 1 RFC822.SIZE - S: * 1 FETCH (RFC822.SIZE 2545) - S: A0003 OK FETCH completed - - -What does IMAP look like? -------------------------- - -IMAP (Ask for information, provide answers [Get first message header]):: - - C: A0004 FETCH 1 BODY[HEADER] - S: * 1 FETCH (RFC822.HEADER {1425} - - S: ) - S: A0004 OK FETCH completed - - -What does IMAP look like? -------------------------- - -IMAP (Ask for information, provide answers [Get first message body]):: - - C: A0005 FETCH 1 BODY[TEXT] - S: * 1 FETCH (BODY[TEXT] {1120} - - S: ) - S: * 1 FETCH (FLAGS (\Recent \Seen)) - S: A0005 OK FETCH completed - -What does IMAP look like? -------------------------- - -IMAP (Say goodbye):: - - C: A0006 LOGOUT - S: * BYE example.com IMAP4rev1 server terminating connection - S: A0006 OK LOGOUT completed - C: - - -IMAP Characteristics --------------------- - -.. class:: incremental - -* Interaction consists of commands and replies -* Each command or reply is *one line* terminated by -* Each command has a *verb* and one or more *arguments* -* Each reply has a formal *code* and an informal *explanation* - -.. class:: incremental - - -IMAP Differences ----------------- - -.. class:: incremental - -* Commands and replies are prefixed by 'sequence identifier' -* Payloads are prefixed by message size, rather than terminated by reserved - sequence - -.. class:: incremental - -Compared with POP3, what do these differences suggest? - - -Protocols in Python -------------------- - -.. class:: big-centered - -Let's try this out for ourselves! - - -Protocols in Python -------------------- - -.. class:: big-centered - -Fire up your python interpreters and prepare to type. - - -IMAP in Python --------------- - -Begin by importing the ``imaplib`` module from the Python Standard Library:: - - >>> import imaplib - >>> dir(imaplib) - ['AllowedVersions', 'CRLF', 'Commands', - 'Continuation', 'Debug', 'Flags', 'IMAP4', - 'IMAP4_PORT', 'IMAP4_SSL', 'IMAP4_SSL_PORT', - ... - 'socket', 'ssl', 'sys', 'time'] - >>> imaplib.Debug = 4 - -.. class:: incremental - -Setting ``imap.Debug`` shows us what is sent and received - - -IMAP in Python --------------- - -I've prepared a server for us to use, we'll need to set up a client to speak -to it. Our server requires SSL for connecting to IMAP servers, so let's -initialize an IMAP4_SSL client and authenticate:: - - >>> conn = imaplib.IMAP4_SSL('mail.webfaction.com') - 57:04.83 imaplib version 2.58 - 57:04.83 new IMAP4 connection, tag=FNHG - ... - >>> conn.login(username, password) - 12:16.50 > IMAD1 LOGIN username password - 12:18.52 < IMAD1 OK Logged in. - ('OK', ['Logged in.']) - - -IMAP in Python --------------- - -We can start by listing the mailboxes we have on the server:: - - >>> conn.list() - 00:41.91 > FNHG3 LIST "" * - 00:41.99 < * LIST (\HasNoChildren) "." "INBOX" - 00:41.99 < FNHG3 OK List completed. - ('OK', ['(\\HasNoChildren) "." "INBOX"']) - - -IMAP in Python --------------- - -To interact with our email, we must select a mailbox from the list we received -earlier:: - - >>> conn.select('INBOX') - 00:00.47 > FNHG2 SELECT INBOX - 00:00.56 < * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) - 00:00.56 < * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. - 00:00.56 < * 2 EXISTS - 00:00.57 < * 0 RECENT - 00:00.57 < * OK [UNSEEN 2] First unseen. - 00:00.57 < * OK [UIDVALIDITY 1357449499] UIDs valid - 00:00.57 < * OK [UIDNEXT 3] Predicted next UID - 00:00.57 < FNHG2 OK [READ-WRITE] Select completed. - ('OK', ['2']) - - -IMAP in Python --------------- - -We can search our selected mailbox for messages matching one or more criteria. -The return value is a string list of the UIDs of messages that match our -search:: - - >>> conn.search(None, '(FROM "cris")') - 18:25.41 > FNHG5 SEARCH (FROM "cris") - 18:25.54 < * SEARCH 1 - 18:25.54 < FNHG5 OK Search completed. - ('OK', ['1']) - >>> - - -IMAP in Python --------------- - -Once we've found a message we want to look at, we can use the ``fetch`` -command to read it from the server. IMAP allows fetching each part of -a message independently:: - - >>> conn.fetch('1', '(BODY[HEADER])') - ... - >>> conn.fetch('1', '(BODY[TEXT])') - ... - >>> conn.fetch('1', '(FLAGS)') - - -Python Means Batteries Included -------------------------------- - -So we can download an entire message and then make a Python email message -object - -.. class:: small - -:: - - >>> import email - >>> typ, data = conn.fetch('1', '(RFC822)') - 28:08.40 > FNHG8 FETCH 1 (RFC822) - ... - -Parse the returned data to get to the actual message - -.. class:: small - -:: - - >>> for part in data: - ... if isinstance(part, tuple): - ... msg = email.message_from_string(part[1]) - ... - >>> - - -IMAP in Python --------------- - -Once we have that, we can play with the resulting email object: - -.. class:: small - -:: - - >>> msg.keys() - ['Return-Path', 'X-Original-To', 'Delivered-To', 'Received', - ... - 'To', 'Mime-Version', 'X-Mailer'] - >>> msg['To'] - 'demo@crisewing.com' - >>> print msg.get_payload()[0] - If you are reading this email, ... - -.. class:: incremental center - -**Neat, huh?** - - -What Have We Learned? ---------------------- - -.. class:: incremental - -* Protocols are just a set of rules for how to communicate - -* Protocols tell us how to parse and delimit messages - -* Protocols tell us what messages are valid - -* If we properly format request messages to a server, we can get response - messages - -* Python supports a number of these protocols - -* So we don't have to remember how to format the commands ourselves - -.. class:: incremental - -But in every case we've seen, we could do the same thing with a socket and -some strings - - -Break Time ----------- - -Let's take a few minutes here to clear our heads. - -.. class:: incremental - -See you back here in 10 minutes. - - -HTTP ----- - -.. class:: big-centered - -HTTP is no different - - -HTTP ----- - -HTTP is also message-centered, with two-way communications: - -.. class:: incremental - -* Requests (Asking for information) -* Responses (Providing answers) - -What does HTTP look like? -------------------------- - -HTTP (Ask for information):: - - GET /index.html HTTP/1.1 - Host: www.example.com - - -What does HTTP look like? -------------------------- - -HTTP (Provide answers):: - - HTTP/1.1 200 OK - Date: Mon, 23 May 2005 22:38:34 GMT - Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux) - Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT - Etag: "3f80f-1b6-3e1cb03b" - Accept-Ranges: none - Content-Length: 438 - Connection: close - Content-Type: text/html; charset=UTF-8 - - <438 bytes of content> - - -HTTP Req/Resp Format --------------------- - -Both share a common basic format: - -.. class:: incremental - -* Line separators are (familiar, no?) -* A required initial line (a command or a response code) -* A (mostly) optional set of headers, one per line -* A blank line -* An optional body - - -HTTP In Real Life ------------------ - -Let's investigate the HTTP protocol a bit in real life. - -.. class:: incremental - -We'll do so by building a simplified HTTP server, one step at a time. - -.. class:: incremental - -There is a copy of the echo server from last time in ``resources/session02``. -It's called ``http_server.py``. - -.. class:: incremental - -In a terminal, move into that directory. We'll be doing our work here for the -rest of the session - - -TDD IRL (a quick aside) ------------------------ - -Test Driven Development (TDD) is all the rage these days. - -.. class:: incremental - -It means that before you write code, you first write tests demonstrating what -you want your code to do. - -.. class:: incremental - -When all your tests pass, you are finished. You did this for your last -assignment. - -.. class:: incremental - -We'll be doing it again today. - - -Run the Tests -------------- - -From inside ``resources/session02`` start a second python interpreter and run -``$ python http_server.py`` - -.. container:: incremental - - In your first interpreter run the tests. You should see similar output: - - .. class:: small - - :: - - $ python tests.py - [...] - Ran 10 tests in 0.003s - - FAILED (failures=3, errors=7) - - -.. class:: incremental - -Let's take a few minutes here to look at these tests and understand them. - - -Viewing an HTTP Request ------------------------ - -Our job is to make all those tests pass. - -.. class:: incremental - -First, though, let's pretend this server really is a functional HTTP server. - -.. class:: incremental - -This time, instead of using the echo client to make a connection to the -server, let's use a web browser! - -.. class:: incremental - -Point your favorite browser at ``http://localhost:10000`` - - -A Bad Interaction ------------------ - -First, look at the printed output from your echo server. - -.. class:: incremental - -Second, note that your browser is still waiting to finish loading the page - -.. class:: incremental - -Moreover, your server should also be hung, waiting for more from the 'client' - -.. class:: incremental - -This is because we are not yet following the right protocol. - - -Echoing A Request ------------------ - -Kill your server with ``ctrl-c`` (the keyboard interrupt) and you should see -some printed content: - -.. class:: small incremental - -:: - - GET / HTTP/1.1 - Host: localhost:10000 - User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:22.0) Gecko/20100101 Firefox/22.0 - Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 - Accept-Language: en-US,en;q=0.5 - Accept-Encoding: gzip, deflate - DNT: 1 - Cookie: __utma=111872281.383966302.1364503233.1364503233.1364503233.1; __utmz=111872281.1364503233.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); csrftoken=uiqj579iGRbReBHmJQNTH8PFfAz2qRJS - Connection: keep-alive - Cache-Control: max-age=0 - -.. class:: incremental - -Your results will vary from mine. - -HTTP Debugging --------------- - -When working on applications, it's nice to be able to see all this going back -and forth. - -.. container:: incremental - - Good browsers support this with a set of developer tools built-in. - - .. class:: small incremental - - * firefox -> ctrl-shift-K or cmd-opt-K (os X) - * safari -> enable in preferences:advanced then cmd-opt-i - * chrome -> ctrl-shift-i or cmd-opt-i (os X) - * IE (7.0+) -> F12 or tools menu -> developer tools - -.. class:: incremental - -The 'Net(work)' pane of these tools can show you both request and response, -headers and all. Very useful. - - -Stop! Demo Time ---------------- - -.. class:: big-centered - -Let's take a quick look - - -Other Debugging Options ------------------------ - -Sometimes you need or want to debug http requests that are not going through -your browser. - -.. class:: incremental - -Or perhaps you need functionality that is not supported by in-browser tools -(request munging, header mangling, decryption of https request/responses) - -.. container:: incremental - - Then it might be time for an HTTP debugging proxy: - - * windows: http://www.fiddler2.com/fiddler2/ - * win/osx/linux: http://www.charlesproxy.com/ - - -HTTP Requests -------------- - -In HTTP 1.0, the only required line in an HTTP request is this:: - - GET /path/to/index.html HTTP/1.0 - - -.. class:: incremental - -As virtual hosting grew more common, that was not enough, so HTTP 1.1 adds a -single required *header*, **Host**: - -.. class:: incremental - -:: - - GET /path/to/index.html HTTP/1.1 - Host: www.mysite1.com:80 - - - -HTTP Responses --------------- - -In both HTTP 1.0 and 1.1, a proper response consists of an intial line, -followed by optional headers, a single blank line, and then optionally a -response body:: - - HTTP/1.1 200 OK - Content-Type: text/plain - - this is a pretty minimal response - -.. class:: incremental - -Let's update our server to return such a response. - - -Basic HTTP Protocol -------------------- - -Begin by implementing a new function in your ``http_server.py`` script called -`response_ok`. - -.. class:: incremental - -It can be super-simple for now. We'll improve it later. - -.. container:: incremental - - It needs to return our minimal response from above: - - .. class:: small - - :: - - HTTP/1.1 200 OK - Content-Type: text/plain - - this is a pretty minimal response - -.. class:: incremental small - -**Remember, is a placeholder for an intentionally blank line** - - -My Solution ------------ - -.. code-block:: python - :class: incremental - - def response_ok(): - """returns a basic HTTP response""" - resp = [] - resp.append("HTTP/1.1 200 OK") - resp.append("Content-Type: text/plain") - resp.append("") - resp.append("this is a pretty minimal response") - return "\r\n".join(resp) - - -Run The Tests -------------- - -We've now implemented a function that is tested by our tests. Let's run them -again: - -.. class:: incremental small - -:: - - $ python tests.py - [...] - ---------------------------------------------------------------------- - Ran 10 tests in 0.002s - - FAILED (failures=3, errors=3) - -.. class:: incremental - -Great! We've now got 4 tests that pass. Good work. - -Server Modifications --------------------- - -Next, we need to rebuild the server loop from our echo server for it's new -purpose: - -.. class:: incremental - -It should now wait for an incoming request to be *finished*, *then* send a -response back to the client. - -.. class:: incremental - -The response it sends can be the result of calling our new ``response_ok`` -function for now. - -.. class:: incremental - -We could also bump up the ``recv`` buffer size to something more reasonable -for HTTP traffic, say 1024. - -My Solution ------------ - -.. code-block:: python - :class: incremental small - - # ... - try: - while True: - print >>log_buffer, 'waiting for a connection' - conn, addr = sock.accept() # blocking - try: - print >>log_buffer, 'connection - {0}{1}'.format(*addr) - while True: - data = conn.recv(1024) - if len(data) < 1024: - break - - print >>log_buffer, 'sending response' - response = response_ok() - conn.sendall(response) - finally: - conn.close() - # ... - - -Run The Tests -------------- - -Once you've got that set, restart your server:: - - $ python http_server.py - -.. container:: incremental - - Then you can re-run your tests: - - .. class:: small - - :: - - $ python tests.py - [...] - ---------------------------------------------------------------------- - Ran 10 tests in 0.003s - - FAILED (failures=2, errors=3) - -.. class:: incremental - -Five tests now pass! - -Parts of a Request ------------------- - -Every HTTP request **must** begin with a single line, broken by whitespace into -three parts:: - - GET /path/to/index.html HTTP/1.1 - -.. class:: incremental - -The three parts are the *method*, the *URI*, and the *protocol* - -.. class:: incremental - -Let's look at each in turn. - - -HTTP Methods ------------- - -**GET** ``/path/to/index.html HTTP/1.1`` - -.. class:: incremental - -* Every HTTP request must start with a *method* -* There are four main HTTP methods: - - .. class:: incremental - - * GET - * POST - * PUT - * DELETE - -.. class:: incremental - -* There are others, notably HEAD, but you won't see them too much - - -HTTP Methods ------------- - -These four methods are mapped to the four basic steps (*CRUD*) of persistent -storage: - -.. class:: incremental - -* POST = Create -* GET = Read -* PUT = Update -* DELETE = Delete - - -Methods: Safe <--> Unsafe -------------------------- - -HTTP methods can be categorized as **safe** or **unsafe**, based on whether -they might change something on the server: - -.. class:: incremental - -* Safe HTTP Methods - * GET -* Unsafe HTTP Methods - * POST - * PUT - * DELETE - -.. class:: incremental - -This is a *normative* distinction, which is to say **be careful** - - -Methods: Idempotent <--> ??? ----------------------------- - -HTTP methods can be categorized as **idempotent**, based on whether a given -request will always have the same result: - -.. class:: incremental - -* Idempotent HTTP Methods - * GET - * PUT - * DELETE -* Non-Idempotent HTTP Methods - * POST - -.. class:: incremental - -Again, *normative*. The developer is responsible for ensuring that it is true. - - -HTTP Method Handling --------------------- - -Let's keep things simple, our server will only respond to *GET* requests. - -.. class:: incremental - -We need to create a function that parses a request and determines if we can -respond to it: ``parse_request``. - -.. class:: incremental - -If the request method is not *GET*, our method should raise an error - -.. class:: incremental - -Remember, although a request is more than one line long, all we care about -here is the first line - - -My Solution ------------ - -.. code-block:: python - :class: incremental - - def parse_request(request): - first_line = request.split("\r\n", 1)[0] - method, uri, protocol = first_line.split() - if method != "GET": - raise NotImplementedError("We only accept GET") - print >>sys.stderr, 'request is okay' - - -Update the Server ------------------ - -We'll also need to update the server code. It should - -.. class:: incremental - -* save the request as it comes in -* check the request using our new function -* send an OK response if things go well - - -My Solution ------------ - -.. code-block:: python - :class: incremental small - - # ... - conn, addr = sock.accept() # blocking - try: - print >>log_buffer, 'connection - {0}{1}'.format(*addr) - request = "" - while True: - data = conn.recv(1024) - request += data - if len(data) < 1024 or not data: - break - - parse_request(request) - print >>log_buffer, 'sending response' - response = response_ok() - conn.sendall(response) - finally: - conn.close() - # ... - - -Run The Tests -------------- - -Quit and restart your server now that you've updated the code:: - - $ python http_server.py - -.. container:: incremental - - At this point, we should have seven tests passing: - - .. class:: small - - :: - - $ python tests.py - Ran 10 tests in 0.002s - - FAILED (failures=1, errors=2) - - -What About a Browser? ---------------------- - -Quit and restart your server, now that you've updated the code. - -.. class:: incremental - -Reload your browser. It should work fine. - -.. class:: incremental - -We can use the ``simple_client.py`` script in our resources to test our error -condition. In a second terminal window run the script like so: - -.. class:: incremental - -:: - - $ python simple_client.py "POST / HTTP/1.0\r\n\r\n" - -.. class:: incremental - -You'll have to quit the client pretty quickly with ``ctrl-c`` - - -Error Responses ---------------- - -Okay, so the outcome there was pretty ugly. The client went off the rails, and -our server has terminated as well. - -.. class:: incremental - -The HTTP protocol allows us to handle errors like this more gracefully. - -.. class:: incremental center - -**Enter the Response Code** - - -HTTP Response Codes -------------------- - -``HTTP/1.1`` **200 OK** - -All HTTP responses must include a **response code** indicating the outcome of -the request. - -.. class:: incremental - -* 1xx (HTTP 1.1 only) - Informational message -* 2xx - Success of some kind -* 3xx - Redirection of some kind -* 4xx - Client Error of some kind -* 5xx - Server Error of some kind - -.. class:: incremental - -The text bit makes the code more human-readable - - -Common Response Codes ---------------------- - -There are certain HTTP response codes you are likely to see (and use) most -often: - -.. class:: incremental - -* ``200 OK`` - Everything is good -* ``301 Moved Permanently`` - You should update your link -* ``304 Not Modified`` - You should load this from cache -* ``404 Not Found`` - You've asked for something that doesn't exist -* ``500 Internal Server Error`` - Something bad happened - -.. class:: incremental - -Do not be afraid to use other, less common codes in building good apps. There -are a lot of them for a reason. See -http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html - - -Handling our Error ------------------- - -Luckily, there's an error code that is tailor-made for this situation. - -.. class:: incremental - -The client has made a request using a method we do not support - -.. class:: incremental - -``405 Method Not Allowed`` - -.. class:: incremental - -Let's add a new function that returns this error code. It should be called -``response_method_not_allowed`` - - -My Solution ------------ - -.. code-block:: python - :class: incremental - - def response_method_not_allowed(): - """returns a 405 Method Not Allowed response""" - resp = [] - resp.append("HTTP/1.1 405 Method Not Allowed") - resp.append("") - return "\r\n".join(resp) - - -Server Updates --------------- - -Again, we'll need to update the server to handle this error condition -correctly. It should - -.. class:: incremental - -* catch the exception raised by the ``parse_request`` function -* return our new error response as a result -* if no exception is raised, then return the OK response - -My Solution ------------ - -.. code-block:: python - :class: incremental small - - # ... - while True: - data = conn.recv(1024) - request += data - if len(data) < 1024 or not data: - break - - try: - parse_request(request) - except NotImplementedError: - response = response_method_not_allowed() - else: - response = response_ok() - - print >>sys.stderr, 'sending response' - conn.sendall(response) - # ... - - -Run The Tests -------------- - -Start your server (or restart it if by some miracle it's still going). - -.. container:: incremental - - Then run the tests again: - - .. class:: small - - :: - - $ python tests.py - [...] - Ran 10 tests in 0.002s - - OK - -.. class:: incremental - -Wahoo! All our tests are passing. That means we are done writing code for now. - - -HTTP - Resources ----------------- - -We've got a very simple server that accepts a request and sends a response. -But what happens if we make a different request? - -.. container:: incremental - - In your web browser, enter the following URL:: - - http://localhost:10000/page - -.. container:: incremental - - What happened? What happens if you use this URL:: - - http://localhost:10000/section/page? - - -HTTP - Resources ----------------- - -We expect different urls to result in different responses. - -.. class:: incremental - -But this isn't happening with our server, for obvious reasons. - -.. class:: incremental - -It brings us back to the second element of that first line of an HTTP request. - -.. class:: incremental center - -**The Return of the URI** - - -HTTP Requests: URI ------------------- - -``GET`` **/path/to/index.html** ``HTTP/1.1`` - -.. class:: incremental - -* Every HTTP request must include a **URI** used to determine the **resource** to - be returned - -* URI?? - http://stackoverflow.com/questions/176264/whats-the-difference-between-a-uri-and-a-url/1984225#1984225 - -* Resource? Files (html, img, .js, .css), but also: - - .. class:: incremental - - * Dynamic scripts - * Raw data - * API endpoints - - -Homework --------- - -For your homework this week you will expand your server's capabilities so that -it can make different responses to different URIs. - -.. class:: incremental - -You'll allow your server to serve up directories and files from your own -filesystem. - -.. class:: incremental - -You'll be starting from the ``http_server.py`` script that is currently in the -``assignments/session02`` directory. It should be pretty much the same as what -you've created here. - - -One Step At A Time ------------------- - -Take the following steps one at a time. Run the tests in -``assignments/session02`` between to ensure that you are getting it right. - -.. class:: incremental - -* Update ``parse_request`` to return the URI it parses from the request. - -* Update ``response_ok`` so that it uses the resource and mimetype identified - by the URI. - -* Write a new function ``resolve_uri`` that handles looking up resources on - disk using the URI. - -* Write a new function ``response_not_found`` that returns a 404 response if the - resource does not exist. - - -HTTP Headers ------------- - -Along the way, you'll discover that simply returning as the body in -response_ok is insufficient. Different *types* of content need to be -identified to your browser - -.. class:: incremental - -We can fix this by passing information about exactly what we are returning as -part of the response. - -.. class:: incremental - -HTTP provides for this type of thing with the generic idea of *Headers* - - -HTTP Headers ------------- - -Both requests and responses can contain **headers** of the form ``Name: Value`` - -.. class:: incremental - -* HTTP 1.0 has 16 valid headers, 1.1 has 46 -* Any number of spaces or tabs may separate the *name* from the *value* -* If a header line starts with spaces or tabs, it is considered part of the - value for the previous header -* Header *names* are **not** case-sensitive, but *values* may be - -.. class:: incremental - -read more about HTTP headers: http://www.cs.tut.fi/~jkorpela/http.html - - -Content-Type Header -------------------- - -A very common header used in HTTP responses is ``Content-Type``. It tells the -client what to expect. - -.. class:: incremental - -* uses **mime-type** (Multi-purpose Internet Mail Extensions) -* foo.jpeg - ``Content-Type: image/jpeg`` -* foo.png - ``Content-Type: image/png`` -* bar.txt - ``Content-Type: text/plain`` -* baz.html - ``Content-Type: text/html`` - -.. class:: incremental - -There are *many* mime-type identifiers: -http://www.webmaster-toolkit.com/mime-types.shtml - - -Mapping Mime-types ------------------- - -By mapping a given file to a mime-type, we can write a header. - -.. class:: incremental - -The standard lib module ``mimetypes`` does just this. - -.. container:: incremental - - We can guess the mime-type of a file based on the filename or map a file - extension to a type: - - .. code-block:: python - :class: small - - >>> import mimetypes - >>> mimetypes.guess_type('file.txt') - ('text/plain', None) - >>> mimetypes.types_map['.txt'] - 'text/plain' - - -Resolving a URI ---------------- - -Your ``resolve_uri`` function will need to accomplish the following tasks: - -.. class:: incremental - -* It should take a URI as the sole argument - -* It should map the pathname represented by the URI to a filesystem location. - -* It should have a 'home directory', and look only in that location. - -* If the URI is a directory, it should return a plain-text listing and the - mimetype ``text/plain``. - -* If the URI is a file, it should return the contents of that file and its - correct mimetype. - -* If the URI does not map to a real location, it should raise an exception - that the server can catch to return a 404 response. - - -Use Your Tests --------------- - -One of the benefits of test-driven development is that the tests that are -failing should tell you what code you need to write. - -.. class:: incremental - -As you work your way through the steps outlined above, look at your tests. -Write code that makes them pass. - -.. class:: incremental - -If all the tests in ``assignments/session02/tests.py`` are passing, you've -completed the assignment. - - -Submitting Your Homework ------------------------- - -To submit your homework: - -* Do your work in the ``assignments/session02`` directory of **your fork** of - the class respository - -* When you have all tests passing, push your work to **your fork** in github. - -* Using the github web interface, send me a pull request. - -.. class:: incremental - -I will review your work when I receive your pull requests, make comments on it -there, and then close the pull request. - - -A Few Steps Further -------------------- - -If you are able to finish the above in less than 4-6 hours, consider taking on -one or more of the following challenges: - -.. class:: incremental - -* Format directory listings as HTML, so you can link to files. -* Add a GMT ``Date:`` header in the proper format (RFC-1123) to responses. - *hint: see email.utils.formatdate in the python standard library* -* Add a ``Content-Length:`` header for ``OK`` responses that provides a - correct value. -* Protect your server against errors by providing, and using, a function that - returns a ``500 Internal Server Error`` response. -* Instead of returning the python script in ``webroot`` as plain text, execute - the file and return the results as HTML. diff --git a/source/presentations/session03-addenda.rst.norender b/source/presentations/session03-addenda.rst.norender deleted file mode 100644 index 09d901da..00000000 --- a/source/presentations/session03-addenda.rst.norender +++ /dev/null @@ -1,445 +0,0 @@ -Web Service API Addenda -======================= - -The following are provided as self-directed exercises. We just don't have the -time to cover them in depth in class. - - -XML-RPC -------- - -Examples of XML-RPC using the Python Standard Library - - -XML-RPC Example - Server ------------------------- - -xmlrpc_server.py: - -.. code-block:: python - :class: small - - from SimpleXMLRPCServer import SimpleXMLRPCServer - - server = SimpleXMLRPCServer(('localhost', 50000)) - - def multiply(a, b): - return a * b - server.register_function(multiply) - - try: - print "Use Ctrl-C to Exit" - server.serve_forever() - except KeyboardInterrupt: - print "Exiting" - - -XML-RPC Example - Client ------------------------- - -We can run a client from a terminal. First, open one terminal and run the -xmlrpc_server.py script: - - $ python xmlrcp_server.py - -Then, open another terminal and start up python: - -.. code-block:: python - :class: small - - >>> import xmlrpclib - >>> proxy = xmlrpclib.ServerProxy('http://localhost:50000', verbose=True) - >>> proxy.multiply(3, 24) - ... - 72 - - -XML-RPC Request ---------------- - -``verbose=True`` allows us to see the request we sent: - -.. class:: tiny - -:: - - POST /RPC2 HTTP/1.0 - Host: localhost:50000 - User-Agent: xmlrpclib.py/1.0.1 (by www.pythonware.com) - Content-Type: text/xml - Content-Length: 192 - - - - multiply - - - 3 - - - 24 - - - - - -XML-RPC Response ----------------- - -and we can see the response, too: - -.. class:: tiny - -:: - - HTTP/1.0 200 OK - Server: BaseHTTP/0.3 Python/2.6.1 - Date: Sun, 13 Jan 2013 03:38:00 GMT - Content-type: text/xml - Content-length: 121 - - - - - - 72 - - - - - -More XML-RPC ------------- - -Register an entire Python class as a service, exposing class methods:: - - server.register_instance(MyClass()) - -Keep an instance method private : - -.. code-block:: python - :class: tiny - - class MyServiceClass(object): - ... - def public_method(self, arg1, arg2): - """this method is public""" - pass - - def _private_method(self): - """this method is private because it starts with '_' - """ - pass - - -XML-RPC Introspection ---------------------- - -First, implement required methods on your service class: - -.. code-block:: python - :class: tiny - - from SimpleXMLRPCServer import list_public_methods - - class MyServiceClass(object): - ... - def _listMethods(self): - """custom logic for presenting method names to users - - list_public_methods is a convenience function from the Python - library, but you can make your own logic if you wish. - """ - return list_public_methods(self) - - def _methodHelp(self, method): - """provide help text for an individual method - """ - f = getattr(self, method) - return f.__doc__ - - -XML-RPC Introspection ---------------------- - -Then enable introspection via the server instance: - -.. code-block:: python - :class: small - - server.register_introspection_functions() - -After this, a client proxy can call pre-defined methods to learn about what -your service offers: - -.. code-block:: python - :class: small - - >>> for name in proxy.system.listMethods(): - ... help = proxy.system.methodHelp(name) - ... print name - ... print "\t%s" % help - ... - public_method - this method is public - - -Introspection Question ----------------------- - -I told you when we added the ``_private_method`` that any method that any -method whose name starts with ``_`` would be **private**. - -.. class:: incremental - -But we also added a ``_listMethods`` method and a ``_methodHelp`` method and -*those* methods are listed when you run ``proxy.system.listMethods()`` - -.. class:: incremental - -Why is this? - -.. class:: incremental - -For a complete discussion of this, read `this MOTW post -`_ - - -SOAP ----- - -Example of Using SOAP via the ``suds`` package - - -Install Suds ------------- - -* Quit your python interpreter if you have it running. -* If you see (soupenv) at your command line prompt, cool. -* If you do not, type ``source /path/to/soupenv/bin/activate`` -* Windows folks: ``> \path\to\soupenv\Scripts\activate`` -* Once activated: ``pip install suds`` - - -Creating a Suds Client ----------------------- - -Suds allows us to create a SOAP client object. SOAP uses WSDL to define a -service. All we need to do to set this up in python is load the URL of the -WSDL for the service we want to use: - -.. code-block:: python - :class: small - - (soupenv)$ python - >>> from suds.client import Client - >>> geo_client = Client('https://geoservices.tamu.edu/Services/Geocode/WebService/GeocoderService_V03_01.asmx?wsdl') - >>> geo_client - - - -Peeking at the Service ----------------------- - -Suds allows us to visually scan the service. Simply print the client object to -see what the service has to offer: - -.. code-block:: python - :class: small - - >>> print geo_client - - Suds ( https://fedorahosted.org/suds/ ) version: 0.4 GA build: R699-20100913 - - Service ( GeocoderService_V03_01 ) tns="https://geoservices.tamu.edu/" - Prefixes (1) - ns0 = "https://geoservices.tamu.edu/" - Ports (2): - (GeocoderService_V03_01Soap) - Methods (4): - ... - Types (12): - ... - - -Debugging Suds --------------- - -Suds uses python logging to deal with debug information, so if you want to see -what's going on under the hood, you configure it via the Python logging -module: - -.. code-block:: python - - >>> import logging - >>> logging.basicConfig(level=logging.INFO) - >>> logging.getLogger('suds.client').setLevel(logging.DEBUG) - -.. class:: incremental - -This will allow us to see the messages sent and received by our client. - - -Client Options --------------- - -SOAP Servers can provide more than one *service* and each *service* might have -more than one *port*. Suds provides two ways to configure which *service* and -*port* you wish to use. - -Via subscription: - -.. code-block:: python - - client.service[''][''].method(args) - -Or the way we will do it, via configuration: - -.. code-block:: python - - geo_client.set_options(service='GeocoderService_V03_01', - port='GeocoderService_V03_01Soap') - - -Providing Arguments -------------------- - -Arguments to a method are set up as a dictionary. Although some may not be -required according to api documentation, it is safest to provide them all: - -.. code-block:: python - :class: small - - apiKey = '' - args = {'apiKey': apiKey, } - args['streetAddress'] = '1325 4th Avenue' - args['city'] = 'Seattle' - args['state'] = 'WA' - args['zip'] = '98101' - args['version'] = 3.01 - args['shouldReturnReferenceGeometry'] = True - args['shouldNotStoreTransactionDetails'] = True - args['shouldCalculateCensus'] = False - args['censusYear'] = "TwoThousandTen" - - -Making the Call ---------------- - -Finally, once we've got the arguments all ready we can go ahead and make a call -to the server: - -.. code-block:: python - :class: small - - >>> res = geo_client.service.GeocodeAddressNonParsed(**args) - DEBUG:suds.client:sending to - (https://geoservices.tamu.edu/Services/Geocode/WebService/GeocoderService_V03_01.asmx) - message: - ... - - -What does it look like? ------------------------ - -.. class:: tiny - -:: - - - - - - - 1325 4th Avenue - Seattle - WA - 98101 - a450a9181f85498598e21f8a39440e9a - 3.01 - false - TwoThousandTen - true - true - - - - - -And the Reply? --------------- - -.. class:: tiny - -:: - - - - - - - 6ef9c110-994c-4142-93d5-a55173526b64 - 47.6084110119244 - -122.3351592971042 - 3.01 - QUALITY_ADDRESS_RANGE_INTERPOLATION - LOCATION_TYPE_STREET_ADDRESS - Exact - 1 - ... - 2910.69420560356 - Meters - 4269 - <?xml version="1.0" encoding="utf-8"?><LineString xmlns="http://www.opengis.net/gml"><posList>-122.334868 47.608226 -122.335777 47.609219</posList></LineString> - ... - - - - - - -And What of Our Result? ------------------------ - -The WSDL we started with should provide type definitions for both data we send -and results we receive. The ``res`` symbol we bound to our result earlier -should now be an instance of a *GeocodeAddressNonParsedResult*. Lets see what -that looks like: - -.. code-block:: python - - >>> type(res) - - >>> dir(res) - ['CensusTimeTaken', 'CensusYear', 'ErrorMessage', 'FArea', - 'FAreaType', 'FCity', 'FCounty', 'FCountySubRegion', - ...] - >>> res.Latitude, res.Longitude - (47.608411011924403, -122.3351592971042) - - -A Word on Debugging -------------------- - -.. class:: center - -**blerg** - -.. class:: incremental - -* Messages sent to the server are long XML strings -* Error messages are generally based on parsing errors in XML -* These error messages can be quite cryptic: -* "There is an error in XML document (1, 572). ---> The string '' is not a - valid Boolean value.' - -.. class:: incremental - -Try this: - -.. code-block:: python - :class: small incremental - - >>> geo_client.last_sent().str().replace(" ","")[:573] - '...\n' - diff --git a/source/presentations/session03.rst b/source/presentations/session03.rst new file mode 100644 index 00000000..6a2cdbb5 --- /dev/null +++ b/source/presentations/session03.rst @@ -0,0 +1,1400 @@ +.. |br| raw:: html + +
          + +********** +Session 03 +********** + +.. figure:: /_static/gateway.jpg + :align: center + :width: 50% + + The Wandering Angel http://www.flickr.com/photos/wandering_angel/1467802750/ - CC-BY + +CGI, WSGI and Living Online +=========================== + +Wherein we discover the gateways to dynamic processes on a server. + + +But First +--------- + +.. rst-class:: large centered + +Homework Review and Questions + + +Previously +---------- + +.. rst-class:: build + +* You've learned about passing messages back and forth with sockets +* You've created a simple HTTP server using sockets +* You may even have made your server *dynamic* by returning the output of a + python script. + +.. rst-class:: build +.. container:: + + What if you want to pass information to that script? + + How can you give the script access to information about the HTTP request + itself? + + +Stepping Away: The Environment +------------------------------ + +A computer has an *environment*: + +.. rst-class:: build +.. container:: + + in \*nix, you can see this in a shell: + + .. code-block:: bash + + $ printenv + TERM_PROGRAM=iTerm.app + ... + + or in Windows at the command prompt: + + .. code-block:: posh + + C:\> set + ALLUSERSPROFILE=C:\ProgramData + ... + + or in PowerShell: + + .. code-block:: posh + + PS C:\> Get-ChildItem Env: + ALLUSERSPROFILE C:\ProgramData + ... + + + +.. nextslide:: Setting The Environment + +.. rst-class:: build +.. container:: + + In a ``bash`` shell we can do this: + + .. code-block:: bash + + $ export VARIABLE='some value' + $ echo $VARIABLE + some value + + or at a Windows command prompt: + + .. code-block:: posh + + C:\Users\Administrator\> set VARIABLE='some value' + C:\Users\Administrator\> echo %VARIABLE% + 'some value' + + or in PowerShell: + + .. code-block:: posh + + PS C:\> $env:VARIABLE = "some value" + PS C:\> Get-ChildItem Env:VARIABLE + 'some value' + + +.. nextslide:: Viewing the Results + +These new values are now part of the *environment* + +.. rst-class:: build +.. container:: + + \*nix: + + .. code-block:: bash + + $ printenv + ... + VARIABLE=some value + + Windows: + + .. code-block:: posh + + C:\> set + ... + VARIABLE='some value' + + PowerShell: + + .. code-block:: posh + + PS C:\> Get-ChildItem Env: + ... + VARIABLE 'some value' + +.. nextslide:: Environment in Python + +We can see this *environment* in Python, too:: + + $ python + +.. code-block:: pycon + + >>> import os + >>> print(os.environ['VARIABLE']) + some_value + >>> print(os.environ.keys()) + ['VERSIONER_PYTHON_PREFER_32_BIT', 'VARIABLE', + 'LOGNAME', 'USER', 'PATH', ...] + + +.. nextslide:: Altering the Environment + +You can alter os environment values while in Python: + +.. code-block:: pycon + + >>> os.environ['VARIABLE'] = 'new_value' + >>> print(os.environ['VARIABLE']) + new_value + +.. rst-class:: build +.. container:: + + But that doesn't change the original value, *outside* Python: + + .. code-block:: bash + + >>> ^D + + $ echo this is the value: $VARIABLE + this is the value: some_value + + C:\> \Users\Administrator\> echo %VARIABLE% + 'some value' + +.. nextslide:: Lessons Learned + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * Subprocesses inherit their environment from their Parent + * Parents do not see changes to environment in subprocesses + * In Python, you can actually set the environment for a subprocess explicitly + + .. code-block:: python + + subprocess.Popen(args, bufsize=0, executable=None, + stdin=None, stdout=None, stderr=None, + preexec_fn=None, close_fds=False, + shell=False, cwd=None, env=None, # <------- + universal_newlines=False, startupinfo=None, + creationflags=0) + + +CGI - The Web Environment +========================= + +.. rst-class:: large centered + +CGI is little more than a set of standard environmental variables + + +What is CGI +----------- + +First discussed in 1993, formalized in 1997, the current version (1.1) has +been in place since 2004. + +From the preamble:: + + This memo provides information for the Internet community. It does not + specify an Internet standard of any kind. + + -- RFC 3875 - CGI Version 1.1: http://tools.ietf.org/html/rfc3875 + + +.. nextslide:: Meta-Variables + +:: + + 4. The CGI Request . . . . . . . . . . . . . . . . . . . . . . . 10 + 4.1. Request Meta-Variables . . . . . . . . . . . . . . . . . 10 + 4.1.1. AUTH_TYPE. . . . . . . . . . . . . . . . . . . . 11 + 4.1.2. CONTENT_LENGTH . . . . . . . . . . . . . . . . . 12 + 4.1.3. CONTENT_TYPE . . . . . . . . . . . . . . . . . . 12 + 4.1.4. GATEWAY_INTERFACE. . . . . . . . . . . . . . . . 13 + 4.1.5. PATH_INFO. . . . . . . . . . . . . . . . . . . . 13 + 4.1.6. PATH_TRANSLATED. . . . . . . . . . . . . . . . . 14 + 4.1.7. QUERY_STRING . . . . . . . . . . . . . . . . . . 15 + 4.1.8. REMOTE_ADDR. . . . . . . . . . . . . . . . . . . 15 + 4.1.9. REMOTE_HOST. . . . . . . . . . . . . . . . . . . 16 + 4.1.10. REMOTE_IDENT . . . . . . . . . . . . . . . . . . 16 + 4.1.11. REMOTE_USER. . . . . . . . . . . . . . . . . . . 16 + 4.1.12. REQUEST_METHOD . . . . . . . . . . . . . . . . . 17 + 4.1.13. SCRIPT_NAME. . . . . . . . . . . . . . . . . . . 17 + 4.1.14. SERVER_NAME. . . . . . . . . . . . . . . . . . . 17 + 4.1.15. SERVER_PORT. . . . . . . . . . . . . . . . . . . 18 + 4.1.16. SERVER_PROTOCOL. . . . . . . . . . . . . . . . . 18 + 4.1.17. SERVER_SOFTWARE. . . . . . . . . . . . . . . . . 19 + + +Running CGI +----------- + +You have a couple of options: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * Python Standard Library CGIHTTPServer + * Apache + * IIS (on Windows) + * Some other HTTP server that implements CGI (lighttpd, ...?) + + Let's keep it simple by using the Python module + + +.. nextslide:: Preparations + +In the class resources for this session, you'll find a directory named ``cgi``. + +.. rst-class:: build +.. container:: + + Make a copy of that folder in your class working directory. + + Windows Users, you may have to edit the first line of + ``cgi/cgi-bin/cgi_1.py`` to point to your python executable. + + .. rst-class:: build + + * Open *two* terminal windows in this ``cgi`` directory + * In the first terminal, run ``python -m http.server --cgi`` + * Open a web browser and load ``http://localhost:8000/`` + * Click on *CGI Test 1* + + +.. nextslide:: Did that work? + +.. rst-class:: build + +* Your browser might show a 404 or 403 error +* If you see something like that, check the permissions for ``cgi-bin`` *and* + ``cgi_1.py`` +* The file must be executable, the ``cgi-bin`` directory needs to be readable + *and* executable. + + +.. rst-class:: build +.. container:: + + Remember that you can use the bash ``chmod`` command to change permissions + in \*nix: ``chmod a+x cgi-bin/cgi_1.py`` + + Windows users, use the 'properties' context menu to get to permissions, + just grant 'full' + + +.. nextslide:: Break It + +Problems with permissions can lead to failure. So can scripting errors + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * Open ``cgi/cgi-bin/cgi_1.py`` in an editor + * Before where it says ``cgi.test()``, add a single line: + + .. code-block:: python + + 1 / 0 + + Reload your browser, what happens now? + + +.. nextslide:: Errors in CGI + +CGI is famously difficult to debug. There are reasons for this: + +.. rst-class:: build + +* CGI is designed to provide access to runnable processes to *the internet* +* The internet is a wretched hive of scum and villainy +* Revealing error conditions can expose data that could be exploited + + +.. nextslide:: Viewing Errors in Python CGI + +Back in your editor, add the following lines, just below ``import cgi``: + +.. rst-class:: build +.. container:: + + .. code-block:: python + + import cgitb + cgitb.enable() + + Now, reload again. + +.. nextslide:: cgitb Output + +.. figure:: /_static/cgitb_output.png + :align: center + :width: 100% + + +.. nextslide:: Repair the Error + +Let's fix the error from our traceback. Edit your ``cgi_1.py`` file to match: + +.. code-block:: python + + #!/usr/bin/env python + import cgi + import cgitb + + cgitb.enable() + + cgi.test() + +.. rst-class:: build +.. container:: + + Notice the first line of that script: ``#!/usr/bin/env python``. + + This is called a *shebang* (short for hash-bang) + + It tells the system what executable program to use when running the script. + + +CGI Process Execution +--------------------- + +Servers like ``http.server --cgi`` run CGI scripts as a system user called +``nobody``. + +.. rst-class:: build +.. container:: + + This is just like you calling:: + + $ ./cgi_bin/cgi_1.py + + In fact try that now in your second terminal (use the real path), what do + you get? + + Windows folks, you may need ``C:\>python cgi-bin/cgi_1.py`` + + Notice what is missing? + + +.. nextslide:: + +There are a couple of important facts about CGI that derive from this: + +.. rst-class:: build + +* The script **must** include a *shebang* so that the system knows how to run + it. +* The script **must** be executable. +* The *executable* named in the *shebang* will be called as the *nobody* user. +* This is a security feature to prevent CGI scripts from running as a user + with any privileges. +* This means that the *executable* from the script *shebang* must be one that + *anyone* can run. + + +.. nextslide:: The CGI Environment + +CGI is largely a set of agreed-upon environmental variables. + +.. rst-class:: build +.. container:: + + We've seen how environmental variables are found in python in + ``os.environ`` + + We've also seen that at least some of the variables in CGI are **not** part + of the system environment. + + Where do they come from? + + +.. nextslide:: CGI Servers + +Let's find 'em. In a terminal fire up python: + +.. rst-class:: build +.. container:: + + .. code-block:: ipython + + In [1]: from http import server + In [2]: server.__file__ + Out[2]: '/Users/cewing/pythons/parts/opt/lib/python3.5/http/server.py' + In [3]: !subl '/Users/cewing/pythons/parts/opt/lib/python3.5/http/server.py' + + If you don't have the ``subl`` command, or another one that starts your + editor, copy this path and open it in your text editor. + + +.. nextslide:: Environmental Set Up + +From ``http/server.py``, in the ``CGIHTTPRequestHandler`` class, in the +``run_cgi`` method: + +.. rst-class:: tiny +.. code-block:: python + + env = copy.deepcopy(os.environ) + env['SERVER_SOFTWARE'] = self.version_string() + env['SERVER_NAME'] = self.server.server_name + env['GATEWAY_INTERFACE'] = 'CGI/1.1' + ... + if self.have_fork: + # Unix -- fork as we should + ... + pid = os.fork() + ... + try: + ... + os.execve(scriptfile, args, env) + ... + else: + # Non-Unix -- use subprocess + import subprocess + ... + p = subprocess.Popen(cmdline, + ... + env = env + ) + ... + + +.. nextslide:: CGI Scripts + +And that's it, the big secret. The server takes care of setting up the +environment so it has what is needed. + +.. rst-class:: build +.. container:: + + Now, in reverse. How does the information that a script creates end up in + your browser? + + A CGI Script must print its results to stdout. + + Use the same method as above to import and open the source file for the + ``cgi`` module. Note what ``test`` does for an example of this. + + .. rst-class:: tiny + .. code-block:: python + + def test(environ=os.environ): + ... + print("Content-type: text/html") + print() + try: + form = FieldStorage() # Replace with other classes to test those + print_directory() + print_arguments() + print_form(form) + ... + except: + print_exception() + + +.. nextslide:: Recap + +What the Server Does: + +.. rst-class:: build + +* parses the request +* sets up the environment, including HTTP and SERVER variables +* sends a ``HTTP/1.1 200 OK\r\n`` first line to the client +* figures out if the URI points to a CGI script and runs it +* appends what comes from the script on stdout and sends that back + +What the Script Does: + +.. rst-class:: build + +* names appropriate *executable* in the *shebang* line +* uses os.environ to read information from the HTTP request +* builds *any and all* extra **HTTP Headers** |br| + (Content-type:, Content-length:, ...) +* prints the headers, empty line and script output (body) to stdout + + +In-Class Exercise I +------------------- + +You've seen the output from the ``cgi.test()`` method from the ``cgi`` module. +Let's make our own version of this. + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * In the directory ``cgi-bin`` you will find the file ``cgi_2.py``. + * Open that file in your editor. + * The script contains some html with text containing placeholders. + * You should use Python and the CGI environment to fill the the blanks. + * You can view the results of your work by loading + ``http://localhost:8000/`` and clicking on *Exercise One* + + **GO** + + +Getting Data from Users +----------------------- + +All this is well and good, but where's the *dynamic* stuff? + +.. rst-class:: build +.. container:: + + It'd be nice if a user could pass form data to our script for it to use. + + In HTTP, data is often passed to the server as a part of a URL called the + *query string* + + The URL query string is formatted as ``name=value`` pairs, separated by the + ampersand (``&``) character + + The entire query string is separated from other parts of the URL by a + question mark:: + + http://localhost:8000/cgi_bin/somescript.py?a=23&b=46&b=92 + + +.. nextslide:: The Query String in CGI + +In the ``cgi`` module, we get access to the query string with the +``FieldStorage`` class: + +.. code-block:: python + + import cgi + + form = cgi.FieldStorage() + stringval = form.getvalue('a', None) + listval = form.getlist('b') + +.. rst-class:: build + +* The values in the ``FieldStorage`` are *always* strings +* ``getvalue`` allows you to return a default, in case the field isn't present +* ``getlist`` always returns a list: empty, one-valued, or as many values as + are present + + +In-Class Exercise II +-------------------- + +Let's create a dynamic adding machine. + +.. rst-class:: build + +* In the ``cgi-bin`` directory you'll find ``cgi_sums.py``. +* In the ``index.html`` file in the ``cgi`` directory, the third link leads to + this file. +* You will use the structure of that link, and what you learned just now about + ``cgi.FieldStorage``. +* Complete the cgi script in ``cgi_sums.py`` so that the result of adding all + operands sent via the url query is returned. +* Return the results as plain text, with the appropriate ``Content-Type`` + header. + + +.. nextslide:: My Solution + +.. rst-class:: build + +.. code-block:: python + + form = cgi.FieldStorage() + operands = form.getlist('operand') + msg = "your total is {total}" + try: + total = sum(map(int, operands)) + msg = msg.format(total=total) + except (ValueError, TypeError): + msg = "Unable to calculate a sum, please provide integer operands" + + print("Content-Type: text/plain") + print("Content-Length: %s" % len(msg)) + print() + print(msg) + + +.. nextslide:: Break Time + +.. rst-class:: centered + +Let's take a break here, before continuing + + +WSGI +==== + +.. rst-class:: center large + +The Web Server Gateway Interface + +CGI Problems +------------ + +CGI is great, but there are problems: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * Code is executed *in a new process* + * **Every** call to a CGI script starts a new process on the server + * Starting a new process is expensive in terms of server resources + * *Especially for interpreted languages like Python* + + How do we overcome this problem? + +.. nextslide:: Alternatives to CGI + +The most popular approach is to have a long-running process *inside* the +server that handles CGI scripts. + +.. rst-class:: build +.. container:: + + FastCGI and SCGI are existing implementations of CGI in this fashion. + + The PHP scripting language works in much the same way. + + The Apache module **mod_python** offers a similar capability for Python + code. + + .. rst-class:: build + + * Each of these options has a specific API + * None are compatible with each-other + * Code written for one is **not portable** to another + + This makes it much more difficult to *share resources* + + +A Solution +---------- + +Enter WSGI, the Web Server Gateway Interface. + +.. rst-class:: build +.. container:: + + Other alternatives are specific implementations of the CGI standard. + + WSGI is itself a new standard, not an implementation. + + WSGI is generalized to describe a set of interactions. + + Developers can write WSGI-capable apps and deploy them on any WSGI server. + + Read the original WSGI spec: http://www.python.org/dev/peps/pep-0333 + + There is also an update for Python 3: |br| https://www.python.org/dev/peps/pep-3333 + + +Apps and Servers +---------------- + +WSGI consists of two parts, a *server* and an *application*. + +.. rst-class:: build +.. container:: + + .. container:: + + A WSGI Server must: + + .. rst-class:: build + + * set up an environment, much like the one in CGI + * provide a method ``start_response(status, headers, exc_info=None)`` + * build a response body by calling an *application*, passing + ``environment`` and ``start_response`` as args + * return a response with the status, headers and body + + .. container:: + + A WSGI Appliction must: + + .. rst-class:: build + + * Be a callable (function, method, class) + * Take an environment and a ``start_response`` callable as arguments + * Call the ``start_response`` method. + * Return an *iterable* of 0 or more strings, which are treated as the + body of the response. + + +.. nextslide:: Simplified WSGI Server + +.. code-block:: python + + from some_application import simple_app + + def build_env(request): + # put together some environment info from the reqeuest + return env + + def handle_request(request, app): + environ = build_env(request) + iterable = app(environ, start_response) + for data in iterable: + # send data to client here + + def start_response(status, headers): + # start an HTTP response, sending status and headers + + # listen for HTTP requests and pass on to handle_request() + serve(simple_app) + + +.. nextslide:: Simple WSGI Application + +Where the simplified server above is **not** functional, this *is* a complete +app: + +.. code-block:: python + + def application(environ, start_response) + status = "200 OK" + body = "Hello World\n" + response_headers = [('Content-type', 'text/plain'), + ('Content-length', len(body))] + start_response(status, response_headers) + return [body] + + +.. nextslide:: WSGI Middleware + +A third part of the puzzle is something called WSGI *middleware* + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * Middleware implements both the *server* and *application* interfaces + * Middleware acts as a server when viewed from an application + * Middleware acts as an application when viewed from a server + + .. figure:: /_static/wsgi_middleware_onion.png + :align: center + :width: 38% + + +.. nextslide:: WSGI Data Flow + +.. rst-class:: build +.. container:: + + .. container:: + + WSGI Servers: + + .. rst-class:: large centered + + **HTTP <---> WSGI** + + .. container:: + + WSGI Applications: + + .. rst-class:: large centered + + **WSGI <---> app code** + + +.. nextslide:: The WSGI Stack + +The WSGI *Stack* can thus be expressed like so: + +.. rst-class:: build large centered + +**HTTP <---> WSGI <---> app code** + + +.. nextslide:: Using wsgiref + +The Python standard lib provides a reference implementation of WSGI: + +.. figure:: /_static/wsgiref_flow.png + :align: center + :width: 80% + + +.. nextslide:: Apache mod_wsgi + +You can also deploy with Apache as your HTTP server, using **mod_wsgi**: + +.. figure:: /_static/mod_wsgi_flow.png + :align: center + :width: 80% + + +.. nextslide:: Proxied WSGI Servers + +Finally, it is also common to see WSGI apps deployed via a proxied WSGI +server: + +.. figure:: /_static/proxy_wsgi.png + :align: center + :width: 80% + + +The WSGI Environment +-------------------- + +REQUEST_METHOD: + The HTTP request method, such as "GET" or "POST". This cannot ever be an + empty string, and so is always required. +SCRIPT_NAME: + The initial portion of the request URL's "path" that corresponds to the + application object, so that the application knows its virtual "location". + This may be an empty string, if the application corresponds to the "root" of + the server. +PATH_INFO: + The remainder of the request URL's "path", designating the virtual + "location" of the request's target within the application. This may be an + empty string, if the request URL targets the application root and does not + have a trailing slash. +QUERY_STRING: + The portion of the request URL that follows the "?", if any. May be empty or + absent. +CONTENT_TYPE: + The contents of any Content-Type fields in the HTTP request. May be empty or + absent. + + +.. nextslide:: The WSGI Environment + +CONTENT_LENGTH: + The contents of any Content-Length fields in the HTTP request. May be empty + or absent. +SERVER_NAME, SERVER_PORT: + When combined with SCRIPT_NAME and PATH_INFO, these variables can be used to + complete the URL. Note, however, that HTTP_HOST, if present, should be used + in preference to SERVER_NAME for reconstructing the request URL. See the URL + Reconstruction section below for more detail. SERVER_NAME and SERVER_PORT + can never be empty strings, and so are always required. +SERVER_PROTOCOL: + The version of the protocol the client used to send the request. Typically + this will be something like "HTTP/1.0" or "HTTP/1.1" and may be used by the + application to determine how to treat any HTTP request headers. (This + variable should probably be called REQUEST_PROTOCOL, since it denotes the + protocol used in the request, and is not necessarily the protocol that will + be used in the server's response. However, for compatibility with CGI we + have to keep the existing name.) + + +.. nextslide:: The WSGI Environment + +HTTP\_ Variables: + Variables corresponding to the client-supplied HTTP request headers (i.e., + variables whose names begin with "HTTP\_"). The presence or absence of these + variables should correspond with the presence or absence of the appropriate + HTTP header in the request. + +.. rst-class:: build large centered + +**Seem Familiar?** + + +In-Class Exercise III +--------------------- + +Let's start simply. We'll begin by repeating our first CGI exercise in WSGI + +.. rst-class:: build + +* Find the ``wsgi`` directory in the class resources. Copy it to your working + directory. +* Open the file ``wsgi_1.py`` in your text editor. +* We will fill in the missing values using Python and the wsgi ``environ``, + just as we use ``os.environ`` in cgi + +.. rst-class:: build centered + +**But First** + + +.. nextslide:: Orientation + +.. code-block:: python + + if __name__ == '__main__': + from wsgiref.simple_server import make_server + srv = make_server('localhost', 8080, application) + srv.serve_forever() + +.. rst-class:: build +.. container:: + + Note that we pass our ``application`` function to the server factory + + We don't have to write a server, ``wsgiref`` does that for us. + + In fact, you should *never* have to write a WSGI server. + + +.. nextslide:: Orientation + +.. code-block:: python + + def application(environ, start_response): + response_body = body % ( + environ.get('SERVER_NAME', 'Unset'), # server name + ... + ) + status = '200 OK' + response_headers = [('Content-Type', 'text/html'), + ('Content-Length', str(len(response_body)))] + start_response(status, response_headers) + return [response_body.encode('utf8')] + +.. rst-class:: build +.. container:: + + We do not define ``start_response``, the application does that. + + We *are* responsible for determining the HTTP status. + + And the content we hand back *must* be ``bytes``, not unicode. + +.. nextslide:: Running a WSGI Script + +You can run this script with python:: + + $ python wsgi_1.py + +.. rst-class:: build +.. container:: + + This will start a wsgi server. What host and port will it use? + + Point your browser at ``http://localhost:8080/``. Did it work? + + Go ahead and fill in the missing bits. Use the ``environ`` passed into + ``application`` + + +.. nextslide:: Some Tips + +WSGI is a long-running process. + +.. rst-class:: build +.. container:: + + The file you are editing is *not* reloaded after you edit it. + + You'll need to quit and re-run the script between edits. + + Notice the use of ``pprint.pprint``, check your terminal for useful output. + + +A WSGI Application +------------------ + +So now we've learned a bit about the WSGI specification and how a WSGI +application can get data that comes in via an HTTP request. + +.. rst-class:: build +.. container:: + + Let's create a multi-page wsgi application. + + It will serve a small database of python books. + + The database (with a very simple api) can be found in ``wsgi/bookdb.py`` + + .. rst-class:: build + + * We'll need a listing page that shows the titles of all the books + * Each title will link to a details page for that book + * The details page for each book will display all the information and have + a link back to the list + + +.. nextslide:: Some Questions to Ponder + +When viewing our first wsgi app, do we see the name of the wsgi application +script anywhere in the URL? + +.. rst-class:: build +.. container:: + + In our wsgi application script, how many applications did we actually have? + + How are we going to serve different types of information out of a single + application? + + +.. nextslide:: Dispatch + +We have to write an app that will map our incoming request path to some code +that can handle that request. + +.. rst-class:: build +.. container:: + + This process is called ``dispatch``. There are many possible approaches. + + Let's begin by designing this piece of our app. + + Open ``bookapp.py`` from the ``wsgi`` folder. We'll do our work here. + + +.. nextslide:: PATH + +The wsgi environment gives us access to *PATH_INFO*. + +.. rst-class:: build +.. container:: + + This value is the URI from the client's HTTP request. + + We can design the URLs that our app will use to assist us in routing. + + Let's declare that any request for ``/`` will map to the list page. + + .. container:: + + We can also say that the URL for a book will look like this:: + + http://localhost:8080/book/ + +Writing ``resolve_path`` +------------------------ + +Let's write a function, called ``resolve_path`` in our application file. + +.. rst-class:: build + +* It should take the *PATH_INFO* value from environ as an argument. +* It should return the function that will be called. +* It should also return any arguments needed to call that function. +* This implies of course that the arguments should be part of the PATH + + +.. nextslide:: My Solution + +.. rst-class:: build + +.. code-block:: python + + def resolve_path(path): + urls = [(r'^$', books), + (r'^book/(id[\d]+)$', book)] + matchpath = path.lstrip('/') + for regexp, func in urls: + match = re.match(regexp, matchpath) + if match is None: + continue + args = match.groups([]) + return func, args + # we get here if no url matches + raise NameError + + +.. nextslide:: Application Updates + +We need to hook our new dispatch function into the application. + +.. rst-class:: build + +* The path should be extracted from ``environ``. +* The dispatch function should be used to get a function and arguments +* The body to return should come from calling that function with those + arguments +* If an error is raised by calling the function, an appropriate response + should be returned +* If the router raises a NameError, the application should return a 404 + response + + +.. nextslide:: My Solution + +.. rst-class:: build + +.. code-block:: python + + def application(environ, start_response): + headers = [("Content-type", "text/html")] + try: + path = environ.get('PATH_INFO', None) + if path is None: + raise NameError + func, args = resolve_path(path) + body = func(*args) + status = "200 OK" + except NameError: + status = "404 Not Found" + body = "

          Not Found

          " + except Exception: + status = "500 Internal Server Error" + body = "

          Internal Server Error

          " + finally: + headers.append(('Content-length', str(len(body)))) + start_response(status, headers) + return [body.encode('utf8')] + + +Test Your Work +-------------- + +Once you've got your script settled, run it:: + + $ python bookapp.py + +.. rst-class:: build +.. container:: + + Then point your browser at ``http://localhost:8080/`` + + .. rst-class:: build + + * ``http://localhost/book/id3`` + * ``http://localhost/book/id73/`` + * ``http://localhost/sponge/damp`` + + Did that all work as you would have expected? + + +Building the Book List +---------------------- + +The function ``books`` should return an html list of book titles where each +title is a link to the detail page for that book + +.. rst-class:: build + +* You'll need all the ids and titles from the book database. +* You'll need to build a list in HTML using this information +* Each list item should have the book title as a link +* The href for the link should be of the form ``/book/`` + + +.. nextslide:: My Solution + +.. rst-class:: build + +.. code-block:: python + + def books(): + all_books = DB.titles() + body = ['

          My Bookshelf

          ', '
            '] + item_template = '
          • {title}
          • ' + for book in all_books: + body.append(item_template.format(**book)) + body.append('
          ') + return '\n'.join(body) + + +Test Your Work +-------------- + +Quit and then restart your application script:: + + $ python bookapp.py + +.. rst-class:: build +.. container:: + + .. container:: + + Then reload the root of your application:: + + http://localhost:8080/ + + You should see a nice list of the books in the database. Do you? + + Click on a link to view the detail page. Does it load without error? + + +Showing Details +--------------- + +The next step of course is to polish up those detail pages. + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * You'll need to retrieve a single book from the database + * You'll need to format the details about that book and return them as HTML + * You'll need to guard against ids that do not map to books + + In this last case, what's the right HTTP response code to send? + + +.. nextslide:: My Solution + +.. rst-class:: build + +.. code-block:: python + + def book(book_id): + page = """ +

          {title}

          + + + + +
          Author{author}
          Publisher{publisher}
          ISBN{isbn}
          + Back to the list + """ + book = DB.title_info(book_id) + if book is None: + raise NameError + return page.format(**book) + + +.. nextslide:: Revel in Your Success + +Quit and restart your script one more time + +.. rst-class:: build +.. container:: + + Then poke around at your application and see the good you've made + + And your application is portable and sharable + + It should run equally well under any `wsgi server `_ + + +.. nextslide:: A Few Steps Further + +Next steps for an app like this might be: + +* Create a shared full page template and incorporate it into your app +* Improve the error handling by emitting error codes other than 404 and 500 +* Swap out the basic backend here with a different one, maybe a Web Service? +* Think about ways to make the application less tightly coupled to the pages + it serves + + +Homework +======== + +.. rst-class:: left +.. container:: + + For your homework this week, you'll be creating a wsgi application of your + own. + + .. rst-class:: build + .. container:: + + You'll create an online calculator that can perform several operations + + You'll need to support: + + .. rst-class:: build + + * Addition + * Subtraction + * Multiplication + * Division + + .. container:: + + Your users should be able to send appropriate requests and get back + proper responses:: + + http://localhost:8080/multiply/3/5 => 15 + http://localhost:8080/add/23/42 => 65 + http://localhost:8080/divide/6/0 => HTTP "400 Bad Request" + + +.. nextslide:: Submitting Your Homework + +.. rst-class:: left +.. container:: + + To submit your homework: + + .. rst-class:: build + + * Create a new github repository. Call it ``wsgi-calc``. + * Add a python script to it called ``calculator.py``. + * Your script should be runnable using ``$ python calculator.py`` + * When the script is running, I should be able to view your application in + my browser. + * I should be able to see a home page that explains how to perform + calculations. + + .. rst-class:: build + .. container:: + + Your repository should include a README.md file. + + Include all instructions I need to successfully run and view your + script. + + When you are done, send Maria and I an email with a link to your + repository. + +One Last Task +------------- + +Next week we will be installing Python packages that are not part of the +standard library. + +.. rst-class:: build +.. container:: + + This is a common occurence in web development. But it can be hazardous. + + In order to practice safe development I am going to ask you to read and + follow through a `brief tutorial`_ I've created on the subject. + + If you have any trouble, or if things do not work the way they are supposed + to, please reach out. We will need this to be working next week. + +.. _brief tutorial: ../../html/presentations/venv_intro.html + +Wrap-Up +------- + +For educational purposes, you might wish to take a look at the source code for +the ``wsgiref`` module. It's the canonical example of a simple wsgi server + + >>> import wsgiref + >>> wsgiref.__file__ + '/full/path/to/your/copy/of/wsgiref.py' + ... + +.. rst-class:: build centered + +**See you Next Time** diff --git a/source/presentations/session03.rst.norender b/source/presentations/session03.rst.norender deleted file mode 100644 index b1391371..00000000 --- a/source/presentations/session03.rst.norender +++ /dev/null @@ -1,1770 +0,0 @@ -Python Web Programming -====================== - -.. image:: img/granny_mashup.png - :align: left - :width: 50% - -Session 3: Scraping, APIs and Mashups - -.. class:: intro-blurb - -Wherein we learn how to make order from the chaos of the wild internet. - -.. class:: image-credit - -image: Paul Downey http://www.flickr.com/photos/psd/492139935/ - CC-BY - - -A Dilemma ---------- - -The internet makes a vast quantity of data available. - -.. class:: incremental - -But not always in the form or combination you want. - -.. class:: incremental - -It would be nice to be able to combine data from different sources to create -*meaning*. - - -The Big Question ----------------- - -.. class:: big-centered - -But How? - - -The Big Answer --------------- - -.. class:: big-centered - -Mashups - - -Mashups -------- - -A mashup is: - - a web page, or web application, that uses and combines data, presentation - or functionality from two or more sources to create new services. - -.. class:: image-credit - -definition courtsey of `wikipedia -`_ - - -Data Sources ------------- - -The key to mashups is the idea of data sources. - -.. class:: incremental - -These come in many flavors: - -.. class:: incremental - -* Simple websites with data in HTML -* Web services providing structured data -* Web services providing tranformative service (geocoding) -* Web services providing presentation (mapping) - - -HTML Sources ------------- - -It would be nice if all online data were available in well-structured formats. - -.. class:: incremental - -The reality is that much data is available only in HTML. - -.. class:: incremental - -Still we can get at it, with some effort. - -.. class:: incremental - -By scraping the data from the web pages. - - -HTML, Ideally -------------- - -:: - - - - - - -

          A nice clean paragraph

          -

          And another nice clean paragraph

          - - - - -HTML... IRL ------------ - -:: - - -
          - - ``). + +.. rst-class:: build +.. container:: + + The rows we want at this time all have two *cells* inside them. + + The first contains the *label* of the data, the second contains the *value* + + We'll need a function in ``mashup.py`` that: + + .. rst-class:: build + + * takes an HTML element as an argument + * verifies that it is a ```` element + * verifies that it has two immediate children that are ```` tag. + + We only want the ones at the top of that tag (ones nested more deeply + contain other data) + + .. code-block:: ipython + + In [13]: data_rows = rec.find('tbody').find_all(has_two_tds, recursive=False) + In [14]: len(data_rows) + Out[14]: 7 + In [15]: print(data_rows[0].prettify()) + + + + + +.. nextslide:: Extracting Labels and Values + +Now we have a list of the rows that contain our data. + +.. rst-class:: build +.. container:: + + Next we have to collect the data they contain + + The *label/value* structure of this data should suggest the right container + to store the information. + + Let's start by trying to get at the first label + + .. code-block:: ipython + + In [18]: row1 = data_rows[0] + In [19]: cells = row1.find_all('td') + In [20]: cell1 = cells[0] + In [21]: cell1.text + Out[21]: '\n - Business Name\n ' + + That works well enough, but all that extra stuff is nasty + + We need a method to clean up the text we get from these cells + + It should strip extra whitespace, and characters like ``-`` and ``:`` we + don't want. + +.. nextslide:: My Solution + +Try writing such a function for yourself now in ``mashup.py`` + +.. rst-class:: build +.. container:: + + .. code-block:: python + + def clean_data(td): + return td.text.strip(" \n:-") + + Add it to your interpreter and test it out: + + .. code-block:: ipython + + In [25]: def clean_data(td): + ....: return td.text.strip(" \n:-") + ....: + In [26]: clean_data(cell1) + Out[26]: 'Business Name' + In [27]: + + Ahhh, much better + +.. nextslide:: The Complete Function + +So we can get a list of the rows that contain label/value pairs. + +.. rst-class:: build +.. container:: + + And we can extract clean values from the cells in these rows + + Now we need a function in ``mashup.py`` that will iterate through the rows + we find and build a dictionary of the pairs. + + We have to be cautious because some rows don't have a label. + + The values in these rows should go with the label from the previous row. + +.. nextslide:: My Solution + +Here's the version I came up with: + +.. code-block:: python + + def extract_restaurant_metadata(elem): + restaurant_data_rows = elem.find('tbody').find_all( + has_two_tds, recursive=False + ) + rdata = {} + current_label = '' + for data_row in restaurant_data_rows: + key_cell, val_cell = data_row.find_all('td', recursive=False) + new_label = clean_data(key_cell) + current_label = new_label if new_label else current_label + rdata.setdefault(current_label, []).append(clean_data(val_cell)) + return rdata + + +.. nextslide:: Testing It Out + +Add it to our script: + +.. rst-class:: build +.. container:: + + .. code-block:: python + + # ... + data_list = restaurant_data_generator(content_col) + for data_div in data_list: + metadata = extract_restaurant_metadata(data_div) + print metadata + + And then try it out: + + .. code-block:: bash + + (soupenv)$ python mashup.py + ... + {u'Business Category': [u'Seating 0-12 - Risk Category III'], + u'Longitude': [u'122.3401786000'], u'Phone': [u'(206) 501-9554'], + u'Business Name': [u"ZACCAGNI'S"], u'Address': [u'97B PIKE ST', u'SEATTLE, WA 98101'], + u'Latitude': [u'47.6086651300']} + + This script is available as ``resources/session04/mashup_3.py`` + + +Extracting Inspection Data +-------------------------- + +The final step is to extract the inspection data for each restaurant. + +.. rst-class:: build +.. container:: + + We want to capture only the score from each inspection, details we can + leave behind. + + We'd like to calculate the average score for all known inspections. + + We'd also like to know how many inspections there were in total. + + Finally, we'd like to preserve the highest score of all inspections for a + restaurant. + + We'll add this information to our metadata about the restaurant. + + +.. nextslide:: Finding the Data + +Let's start by getting our bearings. Return to viewing the +``inspection_page.html`` you saved in a browser. + +.. rst-class:: build +.. container:: + + Find a restaurant that has had an inspection or two. + + What can you say about the HTML that contains the scores for these + inspections? + + I notice four characteristics that let us isolate the information we want: + + .. rst-class:: build + + * Inspection data is containd in ```` elements + * Rows with inspection data in them have four ``
          Row 1 cell 1 -
          Row 2 cell 1 - - Row 2 cell 2
          This
          sure is a long cell - - - - -FFFFFFFFFUUUUUUUUUUUUU ----------------------- - -.. image:: img/scream.jpg - :align: center - :width: 32% - -.. class:: image-credit - -Photo by Matthew via Flickr (http://www.flickr.com/photos/purplemattfish/3918004964/) - CC-BY-NC-ND - - -The Law of The Internet ------------------------ - -.. class:: big-centered - -"Be strict in what you send and tolerant in what you receive" - - -Taming the Mess ---------------- - -Luckily, there's a tool to help with this: ``BeautifulSoup``. - -.. class:: incremental - -BeautifulSoup is a great tool, but it's not in the Standard Library. We'll -need to install it. - -.. class:: incremental - -* As a real-world developer you need to do this a lot -* As a web developer you need to install *different* versions of the *same* - library -* For every non-standard library installed into a System Python, the gods kill - a kitten -* Use Virtualenv... -* **Always** - - -Getting Virtualenv ------------------- - -Three options for installing virtualenv (this is the exception to the above -rule): - -.. class:: incremental small - -* ``pip install virtualenv`` -* ``easy_install virtualenv`` - -.. class:: incremental - -You must have ``pip`` or ``easy_install`` installed. Try this: - -.. class:: incremental small - -:: - - $ curl -O \ - https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.tar.gz - $ tar -xzvf virtualenv-1.11.tar.gz - -.. class:: incremental - -* remember where it goes. You'll need it -* there is a copy in the class resources (``resources/common``) - - -Creating a Virtualenv ---------------------- - -Creating a new virtualenv is very very simple: - -.. class:: small - -:: - - $ python virtualenv.py [options] - - $ virtualenv [options] - -.. container:: incremental small - - is just the name of the environment you want to create. It's - arbitrary. Let's make one for our BeautifulSoup install:: - - $ python virtualenv.py soupenv - Using real prefix '/Users/cewing/newpythons/parts/opt' - New python executable in soupenv26/bin/python2.6 - Also creating executable in soupenv26/bin/python - Installing setuptools, pip...done. - - -What Happened? --------------- - -When you ran that file, a couple of things took place: - -.. class:: incremental - -* A new directory with your requested name was created -* A new Python executable was created in /bin (/Scripts on Windows) -* The new Python was cloned from the Python used to run the file -* The new Python was isolated from any libraries installed in the old Python -* Setuptools was installed so you have ``easy_install`` for this new python -* Pip was installed so you have ``pip`` for this new python - -.. class:: incremental - -Cool, eh? Learn more at http://www.virtualenv.org - - -Using Virtualenv ----------------- - -To install new libraries into a virtualenv, first activate the env:: - - $ source soupenv/bin/activate - (soupenv)$ which python - /path/to/soupenv/bin/python - -Or, on Windows:: - - > \path\to\soupenv\Scripts\activate - -.. class:: image-credit - -If you use Powershell, read the note here: -http://www.virtualenv.org/en/latest/virtualenv.html#activate-script - - -Install BeautifulSoup ---------------------- - -Once the virtualenv is activated, you can simply use pip or easy_install to -install the libraries you want:: - - (soupenv)$ pip install beautifulsoup4 - - -Choose a Parsing Engine ------------------------ - -BeautifulSoup is built to use the Python HTMLParser. - -.. class:: incremental - -* Batteries Included. It's already there -* It's not great, especially before Python 2.7.3 - -.. class:: incremental - -BeautifulSoup also supports using other parsers. - -.. class:: incremental - -There are two good choices: ``lxml`` and ``html5lib``. - -.. class:: incremental - -``lxml`` is better, but much harder to install. Let's use ``html5lib``. - - -Install a Parsing Engine ------------------------- - -Again, this is pretty simple:: - - (soupenv)$ pip install html5lib - -.. class:: incremental - -Once installed, BeautifulSoup will choose it automatically. - -.. class:: incremental - -BeautifulSoup will choose the "best" available. - -.. class:: incremental - -You can specify the parser if you need to for some reason. - - -Install Requests ----------------- - -Python provides tools for opening urls and communicating with servers. It's -spread across the ``urllib`` and ``urllib2`` packages. - -.. class:: incremental - -These packages have pretty unintuitive APIs. - -.. class:: incremental - -The ``requests`` library is becoming the de-facto standard for this type of -work. Let's install it too. - -.. class:: incremental - -:: - - (soupenv)$ pip install requests - - -Our Class Mashup ----------------- - -We're going to build a mashup together today. - -.. class:: incremental - -It will give us an annotated list of apartment rentals, so the next time we -have to move, we can find the exact right place. - -.. class:: incremental - -We'll start by getting a raw list of apartment rentals from today's canonical -source: - -.. class:: incremental - -Craigslist - -.. class:: incremental - -Open a new file in your editor: ``mashup.py``. - - -Examine the Source ------------------- - -Craigslist doesn't have an api, just a website, so we'll need to dig a bit - -.. class:: incremental - -By going to the website and playing with the form there, we can derive a -formula for a search URL - -.. class:: incremental - -* Base URL: ``http://seattle.craigslist.org/search/apa`` -* keywords: ``query=keyword+values+here`` -* price: ``minAsk=NNN maxAsk=NNN`` -* bedrooms: ``bedrooms=N`` (N in range 1-8) - -.. class:: incremental - -We'll make an HTTP request with these parameters - - -Opening URLs with Requests --------------------------- - -In ``requests``, each HTTP method has a module-level function: - -.. class:: incremental - -* ``GET`` == ``requests.get(url, **kwargs)`` -* ``POST`` == ``requests.post(url, **kwargs)`` -* ... - -.. class:: incremental - -``kwargs`` represent other parts of an HTTP request: - -.. class:: incremental - -* ``params``: a dict of url parameters (?foo=bar&baz=bim) -* ``headers``: a dict of headers to send with the request -* ``data``: the body of the request, if any (form data for POST goes here) -* ... - - -Getting Responses with Requests -------------------------------- - -The return value from one of these functions is a ``response`` which provides: - -.. class:: incremental - -* ``response.status_code``: see the HTTP Status Code returned -* ``response.ok``: True if ``response.status_code`` is not an error -* ``response.raise_for_status()``: call to raise a python error if it is -* ``response.headers``: The headers sent from the server -* ``response.text``: Body of the response, decoded to unicode -* ``response.encoding``: The encoding used to decode -* ``response.content``: The original encoded response body as bytes - -.. class:: incremental small - -``requests documentation``: http://docs.python-requests.org/en/latest/ - -Fetch Search Results --------------------- - -We'll start by writing a function ``fetch_search_results`` - -.. class:: incremental - -* It will accept one keyword argument for each of the possible query values -* It will build a dictionary of request query parameters from incoming keywords -* It will make a request to the craigslist server using this query -* It will return the body of the response if there is no error -* It will raise an error if there is a problem with the response - -.. class:: incremental - -Try writing this function. Put it in ``mashup.py`` - - -My Solution ------------ - -Here's the one I created: - -.. code-block:: python - :class: small incremental - - import requests - - def fetch_search_results( - query=None, minAsk=None, maxAsk=None, bedrooms=None - ): - incoming = locals().copy() - base = 'http://seattle.craigslist.org/search/apa' - search_params = dict( - [(key, val) for key, val in incoming.items() - if val is not None]) - if not search_params: - raise ValueError("No valid keywords") - - resp = requests.get(base, params=search_params, timeout=3) - resp.raise_for_status() #<- no-op if status==200 - return resp.content, resp.encoding - - -Parse the Results ------------------ - -Next, we need a function ``parse_source`` to set up HTML for scraping. It will -need to: - -.. class:: incremental - -* Take the response body from the previous method (or some other source) -* Parse it using BeautifulSoup -* Return the parsed object for further processing - -.. class:: incremental - -Before you start, a word about parsing HTML with BeautifulSoup - - -Parsing HTML with BeautifulSoup -------------------------------- - -The BeautifulSoup object can be instantiated with a string or a file-like -object as the sole argument: - -.. code-block:: python - :class: small - - from bs4 import BeautifulSoup - parsed = BeautifulSoup('

          Some HTML

          ') - - fh = open('a_page.html', 'r') - parsed = BeautifulSoup(fh) - - page = urllib2.urlopen('http://site.com/page.html') - parsed = BeautifulSoup(page) - - -.. class:: incremental - -You might want to open the documentation as reference -(http://www.crummy.com/software/BeautifulSoup/bs4/doc) - - -My Solution ------------ - -Take a shot at writing this new function in ``mashup.py`` - -.. code-block:: python - :class: incremental small - - # add this import at the top - from bs4 import BeautifulSoup - - # then add this function lower down - def parse_source(html, encoding='utf-8'): - parsed = BeautifulSoup(html, from_encoding=encoding) - return parsed - - -Put It Together ---------------- - -We'll need to make our script do something when run. - -.. code-block:: python - :class: incremental small - - if __name__ == '__main__': - # do something - -.. class:: incremental - -* Fetch a search results page -* Parse the resulting HTML -* For now, print out the results so we can see what we get - -.. container:: incremental small - - Use the ``prettify`` method on a BeautifulSoup object:: - - print parsed.prettify() - - -My Solution ------------ - -Try to come up with the proper code on your own. Add it to ``mashup.py`` - -.. code-block:: python - :class: incremental small - - if __name__ == '__main__': - html, encoding = fetch_search_results( - minAsk=500, maxAsk=1000, bedrooms=2 - ) - doc = parse_source(html, encoding) - print doc.prettify(encoding=encoding) - - -Test Your Work --------------- - -Assuming your virtualenv is still active, you should be able to execute the -script. - -.. class:: incremental - -:: - - (soupenv)$ python mashup.py - - - - - seattle apts/housing for rent classifieds - craigslist - - ... - - -Preserve the Results --------------------- - -Try it again, this time redirect the output to a local file, so we can use -it without needing to hit the craiglist servers each time:: - - (soupenv)$ python mashup.py > craigslist_results.html - - -Finding The Needle ------------------- - -Next we find the bits of this pile of HTML that matter to us. - -.. class:: incremental - -Open your html file in a browser and take a look (w/ dev tools). - -.. class:: incremental - -We'll want to find: - -.. class:: incremental - -* The HTML element that contains a single listing -* The source of location data, listings without location should be abandoned -* The description of a listing -* The link to a full listing page on craigslist -* Relevant price or size data. - - -Pulling it Out --------------- - -We can extract this information now. In BeautifulSoup: - -.. class:: incremental - -* All HTML elements (including the parsed document itself) are ``tags`` -* A ``tag`` can be searched using its ``find_all`` method -* This searches the descendents of the tag on which it is called. -* It takes arguments which act as *filters* on the search results - -.. container:: incremental - - like so: - - .. class:: small - - :: - - tag.find_all(name, attrs, recursive, text, limit, **kwargs) - - -Searching by CSS Class ----------------------- - -The items we are looking for are ``p`` tags which have the CSS class -``row``: - -.. class:: incremental - -``find_all`` supports keyword arguments. If the keyword you use isn't one of -the listed arguments, it is treated as an ``attribute`` - -.. class:: incremental - -In Python, ``class`` is a reserved word, so we can't use it as a keyword, but -you can use ``class_``! - -.. class:: incremental small - -:: - - parsed.find_all('p', class_='row') - - -Try It Out ----------- - -Let's fire up a python interpreter and get our hands dirty here:: - - (soupenv)$ python - -.. code-block:: python - :class: small incremental - - >>> html = open('craigslist_results.html', 'r').read() - >>> from bs4 import BeautifulSoup - >>> parsed = BeautifulSoup(html) - >>> listings = parsed.find_all('p', class_='row') - >>> len(listings) - 100 - - -.. class:: incremental - -That sounds about right. Let's see if we can get only those with location -data. - - -Filtering Tricks ----------------- - -Attribute filters given a ``True`` value match tags with that attribute - -.. class:: incremental - -Location data was in the ``data-latitude`` and ``data-longitude`` attributes. - -.. code-block:: python - :class: small incremental - - >>> location_attrs = { - ... 'data-longitude': True, - ... 'data-latitude': True} - >>> locatable = parsed.find_all( - ... 'p', class_='row', attrs=location_attrs) - >>> len(locatable) - 43 - -.. class:: incremental - -Great. That worked nicely - - -Parsing a Row -------------- - -Now that we have the rows we want, we need to parse them. We want to preserve: - -.. class:: incremental - -* Location data (latitude and longitude) -* Source link (to craiglist detailed listing) -* Description text -* Price and size data - -.. class:: incremental - -Which parts of a single row contain each of these elements? - - -Extracting Location -------------------- - -Location data is in the ``data-`` attributes we used to filter rows. - -.. container:: incremental - - We can read the HTML attributes of a 'tag' easily, using ``attrs``: - - .. code-block:: python - :class: small - - >>> row1 = locatable[0] - >>> row1.attrs - {u'data-pid': u'3949023084', u'data-latitude': u'35.8625743108992', - u'class': [u'row'], u'data-longitude': u'-78.6232739959049'} - >>> lat = row1.attrs.get('data-latitude', None) - >>> lon = row1.attrs.get('data-longitude', None) - >>> print lat, lon - 46.9989830869194 -122.847250593816 - - -Extracting Description and Link -------------------------------- - -Where ``find_all`` will find many elements, ``find`` will only find the first -that matches the filters you provide. - -.. container:: incremental - - Our targets are in the first ``a`` tag in the ``pl`` span inside our row: - - .. code-block:: python - :class: small - - >>> link = row1.find('span', class_='pl').find('a') - -.. container:: incremental - - The link path will be in the attrs: - - .. code-block:: python - :class: small - - >>> path = link.attrs['href'] - -.. container:: incremental - - Text contained *inside* tags is in the ``string`` property: - - .. code-block:: python - :class: small - - >>> description = link.string.strip() - - -Extracting Price and Size -------------------------- - -Both price and size are held in the ``l2`` span: - -.. code-block:: python - :class: small - - >>> l2 = row1.find('span', class_='l2') - -.. container:: incremental - - Price, conveniently, is in it's own container: - - .. code-block:: python - :class: small - - >>> price_span = l2.find('span', class_='price') - >>> price = price_span.string.strip() - -.. class:: incremental - -But the size element is not. It is a standalone *text node*. - -.. class:: incremental - -Try finding it by reading the ``string`` property of our ``l2`` tag. - - -Simple Navigation and Text --------------------------- - -We can get to a simple text node by navigating there. - -.. class:: incremental - -You can navigate up, down and across document nodes. - -.. container:: incremental - - We already have the ``price`` span, the size text node is next at the same - level: - - .. code-block:: python - :class: small - - >>> size = price_span.next_sibling.strip(' \n-/') - >>> size - u'2br - 912ft\xb2' - -.. class:: incremental - -You may have noticed that we keep using ``strip``. There are two reasons for -this. - - -The NavigableString Element ---------------------------- - -The most obvious reason is that we don't want extra whitespace. - -.. class:: incremental - -The second reason is more subtle. The values returned by ``string`` are -**not** simple unicode strings - -.. container:: incremental - - They are actually instances of a class called ``NavigableString``: - - .. code-block:: python - :class: small - - >>> price_span.next_sibling.__class__ - - -.. class:: incremental - -Calling ``strip`` or casting them to ``unicode`` converts them, saving memory - - -Put It All Together -------------------- - -Okay, a challenge. Combine everything we've done into a function that: - -.. class:: incremental - -* Extracts all the locatable listings from our html page -* Iterates over each of them, and builds a dictionary of data - - * include ``location``, ``href``, ``description``, ``price`` and ``size`` - -* Returns a list of these dictionaries - -.. class:: incremental - -Call it ``extract_listings`` - -.. class:: incremental - -Put this new function into ``mashup.py`` and call it from ``__main__``, -printing the result - - -Break Time ----------- - -Once you have this working, take a break. - -.. class:: incremental - -When we return, we'll try a saner approach to getting data from online - -.. container:: incremental - - While you have a moment, sign up for an API key from this service: - - http://www.walkscore.com/professional/api.php - - -My Solution ------------ - -.. code-block:: python - :class: small incremental - - def extract_listings(doc): - location_attrs = {'data-latitude': True, - 'data-longitude': True} - for row in doc.find_all('p', class_='row', - attrs=location_attrs): - location = dict( - [(key, row.attrs.get(key)) for key in location_attrs]) - link = row.find('span', class_='pl').find('a') - price_span = row.find('span', class_='price') - listing = { - 'location': location, - 'href': link.attrs['href'], - 'description': link.string.strip(), - 'price': price_span.string.strip(), - 'size': price_span.next_sibling.strip(' \n-/') - } - yield listing - - -My Solution ------------ - -.. code-block:: python - :class: small - - if __name__ == '__main__': - html, encoding = fetch_search_results( - minAsk=500, maxAsk=1000, bedrooms=2 - ) - doc = parse_source(html, encoding) - for listing in extract_listings(doc): - print listing - - -Another Approach ----------------- - -Scraping web pages is tedious and inherently brittle - -.. class:: incremental - -The owner of the website updates their layout, your code breaks - -.. class:: incremental - -But there is another way to get information from the web in a more normalized -fashion - -.. class:: incremental center - -**Web Services** - - -Web Services ------------- - -"a software system designed to support interoperable machine-to-machine -interaction over a network" - W3C - -.. class:: incremental - -* provides a defined set of calls -* returns structured data - - -Early Web Services ------------------- - -RSS is one of the earliest forms of Web Services - -* First known as ``RDF Site Summary`` -* Became ``Really Simple Syndication`` -* More at http://www.rss-specification.com/rss-specifications.htm - -.. class:: incremental - -A single web-based *endpoint* provides a dynamically updated listing of -content - -.. class:: incremental - -Implemented in pure HTTP. Returns XML - -.. class:: incremental - -**Atom** is a competing, but similar standard - - -RSS Document ------------- - -.. class:: tiny - -:: - - - - - RSS Title - This is an example of an RSS feed - http://www.someexamplerssdomain.com/main.html - Mon, 06 Sep 2010 00:01:00 +0000 - Mon, 06 Sep 2009 16:45:00 +0000 - 1800 - - - Example entry - Here is some text containing an interesting description. - http://www.wikipedia.org/ - unique string per item - Mon, 06 Sep 2009 16:45:00 +0000 - - ... - - - - -XML-RPC -------- - -RSS provides a pre-defined data set, can we also allow *calling procedures* to -get more dynamic data? - -.. class:: incremental - -We can! Enter XML-RPC (Remote Procedure Call) - -.. class:: incremental - -* Provides a set of defined procedures which can take arguments -* Calls are made via HTTP GET, by passing an XML document -* Returns from a call are sent to the client in XML - -.. class:: incremental - -There is an interactive example of this at the end of this session. We will -not go through it here, though. - - -Beyond XML-RPC --------------- - -.. class:: incremental - -* XML-RPC allows introspection -* XML-RPC forces you to introspect to get information -* **Wouldn't it be nice to get that automatically?** -* XML-RPC provides data types -* XML-RPC provides only *certain* data types -* **Wouldn't it be nice to have an extensible system for types?** -* XML-RPC allows calling methods with parameters -* XML-RPC only allows calling methods, nothing else -* **wouldn't it be nice to have contextual data as well?** - -.. class:: incremental center - -**Enter SOAP: Simple Object Access Protocol** - - -SOAP ----- - -SOAP extends XML-RPC in a couple of useful ways: - -.. class:: incremental - -* It uses Web Services Description Language (WSDL) to provide meta-data about - an entire service in a machine-readable format (Automatic introspection) - -* It establishes a method for extending available data types using XML - namespaces - -* It provides a wrapper around method calls called the **envelope**, which - allows the inclusion of a **header** with system meta-data that can be used - by the application - - -SOAP in Python --------------- - -There is no standard library module that supports SOAP directly. - -.. class:: incremental - -* The best-known and best-supported module available is **Suds** -* The homepage is https://fedorahosted.org/suds/ -* It can be installed using ``easy_install`` or ``pip install`` - -.. class:: incremental - -Again, there is a good example of using SOAP via the ``suds`` library at the -end of this session. - -.. class:: incremental - -But we're going to move on - - -Afterword ---------- - -SOAP (and XML-RPC) have some problems: - -.. class:: incremental - -* XML is pretty damned inefficient as a data transfer medium -* Why should I need to know method names? -* If I can discover method names at all, I have to read a WSDL to do it? - -.. class:: incremental - -Suds is the best we have, and it hasn't been updated since Sept. 2010. - -If Not XML, Then What? ----------------------- - -.. class:: big-centered incremental - -**JSON** - - -JSON ----- - -JavaScript Object Notation: - -.. class:: incremental - -* a lightweight data-interchange format -* easy for humans to read and write -* easy for machines to parse and generate - -.. class:: incremental - -Based on Two Structures: - -.. class:: incremental - -* object: ``{ string: value, ...}`` -* array: ``[value, value, ]`` - -.. class:: center incremental - -pythonic, no? - - -JSON Data Types ---------------- - -JSON provides a few basic data types (see http://json.org/): - -.. class:: incremental - -* string: unicode, anything but ", \\ and control characters -* number: any number, but json does not use octal or hexadecimal -* object, array (we've seen these above) -* true -* false -* null - -.. class:: incremental center - -**No date type? OMGWTF??!!1!1** - - -Dates in JSON -------------- - -.. class:: incremental - -Option 1 - Unix Epoch Time (number): - -.. code-block:: python - :class: small incremental - - >>> import time - >>> time.time() - 1358212616.7691269 - -.. class:: incremental - -Option 2 - ISO 8661 (string): - -.. code-block:: python - :class: small incremental - - >>> import datetime - >>> datetime.datetime.now().isoformat() - '2013-01-14T17:18:10.727240' - - -JSON in Python --------------- - -You can encode python to json, and decode json back to python: - -.. code-block:: python - :class: small - - >>> import json - >>> array = [1,2,3] - >>> json.dumps(array) - >>> '[1, 2, 3]' - >>> orig = {'foo': [1,2,3], 'bar': u'my resumé', 'baz': True} - >>> encoded = json.dumps(orig) - >>> encoded - '{"baz": true, "foo": [1, 2, 3], "bar": "my resum\\u00e9"}' - >>> decoded = json.loads(encoded) - >>> decoded == orig - True - -.. class:: incremental - -Customizing the encoder or decoder class allows for specialized serializations - - -JSON in Python --------------- - -the json module also supports reading and writing to *file-like objects* via -``json.dump(fp)`` and ``json.load(fp)`` (note the missing 's') - -.. class:: incremental - -Remember duck-typing. Anything with a ``.write`` and a ``.read`` method is -*file-like* - -.. class:: incremental - -This usage can be much more memory-friendly with large files/sources - - -What about WSDL? ----------------- - -SOAP was invented in part to provide completely machine-readable -interoperability. - -.. class:: incremental - -Does that really work in real life? - -.. class:: incremental center - -Hardly ever - - -What about WSDL? ----------------- - -Another reason was to provide extensibility via custom types - -.. class:: incremental - -Does that really work in real life? - -.. class:: incremental center - -Hardly ever - - -Why Do All The Work? --------------------- - -So, if neither of these goals is really achieved by using SOAP, why pay all -the overhead required to use the protocol? - -.. class:: incremental - -Enter REST - - -REST ----- - -.. class:: center - -Representational State Transfer - -.. class:: incremental - -* Originally described by Roy T. Fielding (worth reading) -* Use HTTP for what it can do -* Read more in `this book - `_\* - -.. class:: image-credit incremental - -\* Seriously. Buy it and read -( HTTP/1.1 -* GET /comment HTTP/1.1 -* POST /comment HTTP/1.1 -* PUT /comment/ HTTP/1.1 -* DELETE /comment/ HTTP/1.1 - - -ROA ---- - -This is **Resource Oriented Architecture** - -.. class:: incremental - -The URL represents the *resource* we are working with - -.. class:: incremental - -The HTTP Method represents the ``action`` to be taken - -.. class:: incremental - -The HTTP Code returned tells us the ``result`` (whether success or failure) - - -HTTP Codes Revisited --------------------- - -.. class:: small - -POST /comment HTTP/1.1 (creating a new comment): - -.. class:: incremental small - -* Success: ``HTTP/1.1 201 Created`` -* Failure (unauthorized): ``HTTP/1.1 401 Unauthorized`` -* Failure (NotImplemented): ``HTTP/1.1 405 Not Allowed`` -* Failure (ValueError): ``HTTP/1.1 406 Not Acceptable`` - -.. class:: small incremental - -PUT /comment/ HTTP/1.1 (edit comment): - -.. class:: incremental small - -* Success: ``HTTP/1.1 200 OK`` -* Failure: ``HTTP/1.1 409 Conflict`` - -.. class:: small incremental - -DELETE /comment/ HTTP/1.1 (delete comment): - -.. class:: incremental small - -* Success: ``HTTP/1.1 204 No Content`` - - -HTTP Is Stateless ------------------ - -No individual request may be assumed to know anything about any other request. - -.. class:: incremental - -All the required information representing the possible actions to take *should -be present in every response*. - -.. class:: incremental big-centered - -Thus: HATEOAS - - -HATEOAS -------- - -.. class:: big-centered - -Hypermedia As The Engine Of Application State - - -Applications are State Engines ------------------------------- - -A State Engine is a machine that provides *states* for a resource to be in and -*transitions* to move resources between states. A Restful api should: - -.. class:: incremental - -* provide information about the current state of a resource -* provide information about available transitions for that resource (URIs) -* provide all this in *each* HTTP response - - -Playing With REST ------------------ - -Let's take a moment to play with REST. - -.. class:: incremental - -We'll use a common, public API provided by Google. - -.. class:: incremental center - -**Geocoding** - - -Geocoding with Google APIs --------------------------- - -https://developers.google.com/maps/documentation/geocoding - -.. container:: incremental - - Open a python interpreter using our virtualenv: - - .. class:: small - - :: - - (soupenv)$ python - -.. code-block:: python - :class: small incremental - - >>> import requests - >>> import json - >>> from pprint import pprint - >>> url = 'http://maps.googleapis.com/maps/api/geocode/json' - >>> addr = '1325 4th Ave, Seattle, 98101' - >>> parameters = {'address': addr, 'sensor': 'false' } - >>> resp = requests.get(url, params=parameters) - >>> data = json.loads(resp.text) - >>> if data['status'] == 'OK': - ... pprint(data) - - - -Reverse Geocoding ------------------ - -You can do the same thing in reverse, supply latitude and longitude and get -back address information: - -.. code-block:: python - :class: small - - >>> location = data['results'][0]['geometry']['location'] - >>> latlng="{lat},{lng}".format(**location) - >>> parameters = {'latlng': latlng, 'sensor': 'false'} - >>> resp = requests.get(url, params=paramters) - >>> data = json.loads(resp.text) - >>> if data['status'] == 'OK': - ... pprint(data) - -.. class:: incremental - -Notice that there are a number of results returned, ordered from most specific -to least. - - -Mash It Up ----------- - -Let's add a new function to ``mashup.py``. It will: - -.. class:: incremental - -* take a single listing from our craiglist work -* format the location data provided in that listing properly -* make a reverse geocoding lookup using the google api above -* add the best available address to the listing -* return the updated listing - -.. class:: incremental - -Call it ``add_address`` - - -My Solution ------------ - -.. code-block:: python - :class: small incremental - - # add an import - import json - - # and a function - def add_address(listing): - api_url = 'http://maps.googleapis.com/maps/api/geocode/json' - loc = listing['location'] - latlng_tmpl = "{data-latitude},{data-longitude}" - parameters = { - 'sensor': 'false', - 'latlng': latlng_tmpl.format(**loc), - } - resp = requests.get(api_url, params=parameters) - data = json.loads(resp.text) - if data['status'] == 'OK': - best = data['results'][0] - listing['address'] = best['formatted_address'] - else: - listing['address'] = 'unavailable' - return listing - - -Add Address to Output ---------------------- - -Go ahead and bolt the new function into our ``__main__`` block: - -.. code-block:: python - :class: small incremental - - import pprint - if __name__ == '__main__': - params = {'minAsk': 500, 'maxAsk': 1000, 'bedrooms': 2} - html, encoding = fetch_search_results(**params) - doc = parse_source(html, encoding) - for listing in extract_listings(doc): - listing = add_address(listing) - pprint.pprint(listing) - -.. container:: incremental - - And give the result a whirl: - - .. class:: small - - :: - - (soupenv)$ python mashup.py - {'address': u'123 Some Street, Chapel Hill, NC ...', - 'description': u'3 bedroom 2 bathroom unit is move in ready!' - ... - } - - -One More Step -------------- - -I'm a big fan of walking places. - -.. class:: incremental - -So I'd like to find an apartment that is located somewhere 'walkable' - -.. class:: incremental - -There's an API for that! - -.. class:: incremental - -http://www.walkscore.com/professional/api.php - -.. class:: incremental - -If you haven't already, sign up for an API key now. - - -Getting a Walk Score --------------------- - -The API documentation tells us we have to provide lat, lon and address to get -a walk score, along with our API key. - -.. class:: incremental - -It also tells us we have a choice of XML or JSON output. Let's use JSON - -.. class:: incremental - -Let's poke at it and see what we get back - -.. class:: incremental - -Fire up your virtualenv Python interpreter again - - -Making an API Call ------------------- - -:: - - (soupenv)$ python - -.. code-block:: python - :class: small - - >>> import requests - >>> import json - >>> from pprint import pprint - >>> api_url = 'http://api.walkscore.com/score' - >>> lat, lon = 35.9108986, -79.053783 - >>> addr = '120 E. Cameron Avenue Chapel Hill, NC 27599' - >>> params = {'lat': lat, 'lon', lon, 'address': addr} - >>> params['wsapikey'] = '' - >>> params['format'] = 'json' - >>> resp = requests.get(api_url, params=params) - >>> data = json.loads(resp.text) - >>> if data['status'] == 1: - ... pprint(data) - - -Mash It Up ----------- - -Add a function to ``mashup.py`` that: - -.. class:: incremental - -* takes a single listing from our craigslist search -* uses the location and address to make a walkscore api call -* adds the description, walkscore and ws_link parameters to the listing -* returns the updated listing - -.. class:: incremental - -Call the function ``add_walkscore`` - -.. class:: incremental - -Bolt it into our script's ``__main__`` block where it fits best - - -My Solution ------------ - -.. code-block:: python - :class: small incremental - - def add_walkscore(listing): - api_url = 'http://api.walkscore.com/score' - apikey = '' - loc = listing['location'] - if listing['address'] == 'unavailable': - return listing - parameters = { - 'lat': loc['data-latitude'], 'lon': loc['data-longitude'], - 'address': listing['address'], 'wsapikey': apikey, - 'format': 'json' - } - resp = requests.get(api_url, params=parameters) - data = json.loads(resp.text) - if data['status'] == 1: - listing['ws_description'] = data['description'] - listing['ws_score'] = data['walkscore'] - listing['ws_link'] = data['ws_link'] - return listing - - -My Results ----------- - -.. code-block:: python - :class: small - - if __name__ == '__main__': - params = {'minAsk': 500, 'maxAsk': 1000, 'bedrooms': 2} - html, encoding = fetch_search_results(**params) - doc = parse_source(html, encoding) - for listing in extract_listings(doc): - listing = add_address(listing) - listing = add_walkscore(listing) - pprint.pprint(listing) - -.. container:: incremental - - Let's try it out:: - - (soupenv)$ python mashup.py - - -Wrap Up -------- - -We've built a simple mashup combining data from three different sources. - -.. class:: incremental - -As a result we can now make a listing of apartments ranked by the walkability -of their neighborhood. - -.. class:: incremental - -What other data sources might we use? Check out -http://www.programmableweb.com/apis/directory to see some of the possibilities - - -Addenda -------- - -Altough we do not have class time to do walkthrough examples of using XML-RPC -and SOAP, I have provided exercises in each as an addenda to this session. If -you have the time and the interest, please try them out. - -.. class:: center - -`Web Service API Addenda `_ - - -Homework --------- - -For your homework this week, you'll be creating a mashup of your own. - -.. class:: incremental - -Use the programmable web api directory from above as a source of inspiration. - -.. class:: incremental - -Your mashup should combine at least two sources of data in some way that -tickles your fancy. - -.. class:: incremental - -Your results need not look pretty. Focus on data acquisition and processing. - - -Submitting Your Homework ------------------------- - -To submit your homework: - -* Create a new python script in ``assignments/session03``. It should be - something I can run with:: - - $ python your_script.py - -* Provide me with a text file describing what you did. Tell me about the - sources you use, how you combine them, what you hoped to achieve. - -* Include any instruction I might need to successfully run your script. - -* Commit your changes to your fork of the repo in github, then open a pull - request. - - -Extra Credit ------------- - -Bonus points if you write unit tests for the elements of your mashup. - diff --git a/source/presentations/session04-old.rst.norender b/source/presentations/session04-old.rst.norender deleted file mode 100644 index f8214583..00000000 --- a/source/presentations/session04-old.rst.norender +++ /dev/null @@ -1,1435 +0,0 @@ -Python Web Programming -====================== - -.. image:: img/gateway.jpg - :align: left - :width: 50% - -Session 4: CGI, WSGI and Living Online - -.. class:: intro-blurb - -Wherein we discover the gateways to dynamic processes on a server. - -.. class:: image-credit - -image: The Wandering Angel http://www.flickr.com/photos/wandering_angel/1467802750/ - CC-BY - -But First ---------- - -.. class:: big-centered - -A look at some of the cool mashups you built over the week. - - -But First ---------- - -Clean up the git situation. - - -But First ---------- - -Before you leave the classroom today, please complete the following tasks: - -1. Create a virtualenv called ``flaskenv`` -2. Activate that virtualenv -3. ``pip install flask`` to your virtualenv - -You will need this for some of your homework this week. - -But First ---------- - -A special note to pay attention to the readings. You will be expected to have -read the basics on Jinja2, SQLite3 and Flask **before** class starts. - -Previously ----------- - -.. class:: incremental - -* You've learned about passing messages back and forth with sockets -* You've created a simple HTTP server using sockets -* You may even have made your server *dynamic* by returning the output of a - python script. - -.. class:: incremental - -What if you want to pass information to that script? - -.. class:: incremental - -How can you give the script access to information about the HTTP request -itself? - - -Stepping Away -------------- - -A computer has an *environment*: - -.. container:: incremental - - in \*nix, you can see this in a shell: - - .. class:: small - - :: - - $ printenv - TERM_PROGRAM=iTerm.app - ... - -.. container:: incremental - - or in Windows at the command prompt: - - .. class:: small - - :: - - C:\> set - ALLUSERSPROFILE=C:\ProgramData - ... - - -Setting The Environment ------------------------ - -This can be manipulated: - -.. container:: incremental - - In a ``bash`` shell we can do this: - - .. class:: small - - :: - - $ export VARIABLE='some value' - $ echo $VARIABLE - some value - -.. container:: incremental - - or at a Windows command prompt: - - .. class:: small - - :: - - C:\Users\Administrator\> set VARIABLE='some value' - C:\Users\Administrator\> echo %VARIABLE% - 'some value' - - -Viewing the Results -------------------- - -These new values are now part of the *environment* - -.. container:: incremental - - \*nix: - - .. class:: small - - :: - - $ printenv - TERM_PROGRAM=iTerm.app - ... - VARIABLE=some value - -.. container:: incremental - - Windows: - - .. class:: small - - :: - - C:\> set - ALLUSERSPROFILE=C:\ProgramData - ... - VARIABLE='some value' - -Environment in Python ---------------------- - -We can see this *environment* in Python, too:: - - $ python - -.. code-block:: python - - >>> import os - >>> print os.environ['VARIABLE'] - some_value - >>> print os.environ.keys() - ['VERSIONER_PYTHON_PREFER_32_BIT', 'VARIABLE', - 'LOGNAME', 'USER', 'PATH', ...] - -Altering the Environment ------------------------- - -You can alter os environment values while in Python: - -.. code-block:: python - :class: small - - >>> os.environ['VARIABLE'] = 'new_value' - >>> print os.environ['VARIABLE'] - new_value - -.. container:: incremental - - But that doesn't change the original value, *outside* Python: - - .. class:: small - - :: - - >>> ^D - - $ echo this is the value: $VARIABLE - this is the value: some_value - - C:\> \Users\Administrator\> echo %VARIABLE% - 'some value' - -Lessons Learned ---------------- - -.. class:: incremental - -* Subprocesses inherit their environment from their Parent -* Parents do not see changes to environment in subprocesses -* In Python, you can actually set the environment for a subprocess explicitly - -.. class:: incremental small - -:: - - subprocess.Popen(args, bufsize=0, executable=None, - stdin=None, stdout=None, stderr=None, - preexec_fn=None, close_fds=False, - shell=False, cwd=None, env=None, # <------- - universal_newlines=False, startupinfo=None, - creationflags=0) - - -Web Environment ---------------- - -.. class:: big-centered - -CGI is little more than a set of standard environmental variables - - -RFC 3875 --------- - -First discussed in 1993, formalized in 1997, the current version (1.1) has -been in place since 2004. - -From the preamble: - -.. class:: center - -*This memo provides information for the Internet community. It does not specify -an Internet standard of any kind.* - -.. class:: image-credit - -RFC 3875 - CGI Version 1.1: http://tools.ietf.org/html/rfc3875 - - -Meta-Variables --------------- - -.. class:: small - -:: - - 4. The CGI Request . . . . . . . . . . . . . . . . . . . . . . . 10 - 4.1. Request Meta-Variables . . . . . . . . . . . . . . . . . 10 - 4.1.1. AUTH_TYPE. . . . . . . . . . . . . . . . . . . . 11 - 4.1.2. CONTENT_LENGTH . . . . . . . . . . . . . . . . . 12 - 4.1.3. CONTENT_TYPE . . . . . . . . . . . . . . . . . . 12 - 4.1.4. GATEWAY_INTERFACE. . . . . . . . . . . . . . . . 13 - 4.1.5. PATH_INFO. . . . . . . . . . . . . . . . . . . . 13 - 4.1.6. PATH_TRANSLATED. . . . . . . . . . . . . . . . . 14 - 4.1.7. QUERY_STRING . . . . . . . . . . . . . . . . . . 15 - 4.1.8. REMOTE_ADDR. . . . . . . . . . . . . . . . . . . 15 - 4.1.9. REMOTE_HOST. . . . . . . . . . . . . . . . . . . 16 - 4.1.10. REMOTE_IDENT . . . . . . . . . . . . . . . . . . 16 - 4.1.11. REMOTE_USER. . . . . . . . . . . . . . . . . . . 16 - 4.1.12. REQUEST_METHOD . . . . . . . . . . . . . . . . . 17 - 4.1.13. SCRIPT_NAME. . . . . . . . . . . . . . . . . . . 17 - 4.1.14. SERVER_NAME. . . . . . . . . . . . . . . . . . . 17 - 4.1.15. SERVER_PORT. . . . . . . . . . . . . . . . . . . 18 - 4.1.16. SERVER_PROTOCOL. . . . . . . . . . . . . . . . . 18 - 4.1.17. SERVER_SOFTWARE. . . . . . . . . . . . . . . . . 19 - - -Running CGI ------------ - -You have a couple of options: - -.. class:: incremental - -* Python Standard Library CGIHTTPServer -* Apache -* IIS (on Windows) -* Some other HTTP server that implements CGI (lighttpd, ...?) - -.. class:: incremental - -Let's keep it simple by using the Python module - - -Preparations ------------- - -In the class resources, you'll find a directory named ``cgi``. Make a copy of -that folder in your class working directory. - -.. class:: incremental small red - -Windows Users, you will have to edit the first line of -``cgi/cgi-bin/cgi_1.py`` to point to your python executable. - -.. class:: incremental - -* Open *two* terminal windows in this ``cgi`` directory -* In the first terminal, run ``python -m CGIHTTPServer`` -* Open a web browser and load ``http://localhost:8000/`` -* Click on *CGI Test 1* - - -Did that work? --------------- - -* If nothing at all happens, check your terminal window -* Look for this: ``OSError: [Errno 13] Permission denied`` -* If you see something like that, check permissions for ``cgi-bin`` *and* - ``cgi_1.py`` -* The file must be executable, the directory needs to be readable *and* - executable. - - -.. class:: incremental - -Remember that you can use the bash ``chmod`` command to change permissions in -\*nix - -.. class:: incremental - -Windows users, use the 'properties' context menu to get to permissions, just -grant 'full' - -Break It --------- - -Problems with permissions can lead to failure. So can scripting errors - -.. class:: incremental - -* Open ``cgi/cgi-bin/cgi_1.py`` in an editor -* Before where it says ``cgi.test()``, add a single line: - -.. code-block:: python - :class: incremental - - 1 / 0 - -.. class:: incremental - -Reload your browser, what happens now? - - -Errors in CGI -------------- - -CGI is famously difficult to debug. There are reasons for this: - -.. class:: incremental - -* CGI is designed to provide access to runnable processes to *the internet* -* The internet is a wretched hive of scum and villainy -* Revealing error conditions can expose data that could be exploited - -Viewing Errors in Python CGI ----------------------------- - -Back in your editor, add the following lines, just below ``import cgi``: - -.. code-block:: python - :class: incremental - - import cgitb - cgitb.enable() - -.. class:: incremental - -Now, reload again. - -cgitb Output ------------- - -.. image:: img/cgitb_output.png - :align: center - :width: 100% - - -Repair the Error ----------------- - -Let's fix the error from our traceback. Edit your ``cgi_1.py`` file to match: - -.. code-block:: python - :class: small - - #!/usr/bin/python - import cgi - import cgitb - - cgitb.enable() - - cgi.test() - -.. class:: incremental - -Notice the first line of that script: ``#!/usr/bin/python``. This is called a -*shebang* (short for hash-bang) and it tells the system what executable -program to use when running the script. - - -CGI Process Execution ---------------------- - -When a web server like ``CGIHTTPServer`` or ``Apache`` runs a CGI script, it -simply attempts to run the script as if it were a normal system user. This is -just like you calling:: - - $ ./cgi_bin/cgi_1.py - -.. class:: incremental - -In fact try that now in your second terminal (use the real path), what do you -get? - -.. class:: incremental small center - -Windows folks, you may need ``C:\>python cgi_1.py`` - -.. class:: incremental - -What is missing? - - -CGI Process Execution ---------------------- - -There are a couple of important facts that are related to the way CGI -processes are run: - -.. class:: incremental - -* The script **must** include a *shebang* so that the system knows how to run - it. -* The script **must** be executable. -* The *executable* named in the *shebang* will be called as the *nobody* user. -* This is a security feature to prevent CGI scripts from running as a user - with any privileges. -* This means that the *executable* from the script *shebang* must be one that - *anyone* can run. - - -The CGI Environment -------------------- - -CGI is largely a set of agreed-upon environmental variables. - -.. class:: incremental - -We've seen how environmental variables are found in python in ``os.environ`` - -.. class:: incremental - -We've also seen that at least some of the variables in CGI are **not** in the -standard set of environment variables. - -.. class:: incremental - -Where do they come from? - - -CGI Servers ------------ - -Let's find 'em. In a terminal (on your local machine, please) fire up python: - -.. code-block:: - - >>> import CGIHTTPServer - >>> CGIHTTPServer.__file__ - '/big/giant/path/to/lib/python2.6/CGIHTTPServer.py' - -.. class:: incremental - -Copy this path and open the file it points to in your text editor - - -Environmental Set Up --------------------- - -From CGIHTTPServer.py, in the CGIHTTPServer.run_cgi method: - -.. code-block:: python - :class: tiny - - # Reference: http://hoohoo.ncsa.uiuc.edu/cgi/env.html - # XXX Much of the following could be prepared ahead of time! - env = {} - env['SERVER_SOFTWARE'] = self.version_string() - env['SERVER_NAME'] = self.server.server_name - env['GATEWAY_INTERFACE'] = 'CGI/1.1' - env['SERVER_PROTOCOL'] = self.protocol_version - env['SERVER_PORT'] = str(self.server.server_port) - env['REQUEST_METHOD'] = self.command - ... - ua = self.headers.getheader('user-agent') - if ua: - env['HTTP_USER_AGENT'] = ua - ... - os.environ.update(env) - ... - - -CGI Scripts ------------ - -And that's it, the big secret. The server takes care of setting up the -environment so it has what is needed. - -.. class:: incremental - -Now, in reverse. How does the information that a script creates end up in your -browser? - -.. class:: incremental - -A CGI Script must print its results to stdout. - -.. class:: incremental - -Use the same method as above to import and open the source file for the -``cgi`` module. Note what ``test`` does for an example of this. - - -Recap: ------- - -What the Server Does: - -.. class:: incremental small - -* parses the request -* sets up the environment, including HTTP and SERVER variables -* figures out if the URI points to a CGI script and runs it -* builds an appropriate HTTP Response first line ('HTTP/1.1 200 OK\\r\\n') -* appends what comes from the script on stdout and sends that back - -What the Script Does: - -.. class:: incremental small - -* names appropriate *executable* in it's *shebang* line -* uses os.environ to read information from the HTTP request -* builds *any and all* appropriate **HTTP Headers** (Content-type:, - Content-length:, ...) -* prints headers, empty line and script output (body) to stdout - - -In-Class Exercise ------------------ - -You've seen the output from the ``cgi.test()`` method from the ``cgi`` module. -Let's make our own version of this. - -.. class:: incremental small - -* In the directory ``cgi-bin`` you will find the file ``cgi_2.py``. -* Open that file in your editor. -* The script contains some html with text naming elements of the CGI - environment. -* You should use the values in os.environ to fill in the blanks. -* You should be able to view the results of your work by loading - ``http://localhost:8000/`` and clicking on *Exercise One* - -.. class:: incremental center - -**GO** - - -User Provided Data ------------------- - -All this is well and good, but where's the *dynamic* stuff? - -.. class:: incremental - -It'd be nice if a user could pass form data to our script for it to use. - -.. container:: incremental - - In HTTP, these types of inputs show up in the URL *query* (the part after - the ``?``):: - - http://myhost.com/script.py?a=23&b=37 - - -Form Data in CGI ----------------- - -In the ``cgi`` module, we get access to this with the ``FieldStorage`` class: - -.. code-block:: python - :class: incremental small - - import cgi - - form = cgi.FieldStorage() - stringval = form.getvalue('a', None) - listval = form.getlist('b') - -.. class:: incremental - -* The values in the ``FieldStorage`` are *always* strings -* ``getvalue`` allows you to return a default, in case the field isn't present -* ``getlist`` always returns a list: empty, one-valued, or as many values as - are present - - -In-Class Exercise ------------------ - -Let's create a dynamic adding machine. - -.. class:: incremental - -* In the ``cgi-bin`` directory you'll find ``cgi_sums.py``. -* In the ``index.html`` file in the ``cgi`` directory, the third link leads to - this file. -* You will use the structure of that link, and what you learned just now about - ``cgi.FieldStorage``. -* Complete the cgi script in ``cgi_sums.py`` so that the result of adding all - operands sent via the url query is returned. - -.. class:: incremental - -For extra fun, return the results in ``json`` format (mimetype: -'application/json'). - - -My Solution ------------ - -.. code-block:: python - :class: small incremental - - form = cgi.FieldStorage() - operands = form.getlist('operand') - total = 0 - for operand in operands: - try: - value = int(operand) - except ValueError: - value = 0 - total += value - - output = {'result': total} - json_output = json.dumps(output) - - print "Content-Type: application/json" - print "Content-Length: %s" % len(json_output) - print - print json_output - - -Stopping Point --------------- - -.. class:: big-centered - -Let's take a break here, before continuing - - -CGI Problems ------------- - -CGI is great, but there are problems: - -.. class:: incremental - -* Code is executed *in a new process* -* **Every** call to a CGI script starts a new process on the server -* Starting a new process is expensive in terms of server resources -* *Especially for interpreted languages like Python* - -.. class:: incremental - -How do we overcome this problem? - - -Alternatives to CGI -------------------- - -The most popular approach is to have a long-running process *inside* the -server that handles CGI scripts. - -.. class:: incremental - -FastCGI and SCGI are existing implementations of CGI in this fashion. The -Apache module **mod_python** offers a similar capability for Python code. - -.. class:: incremental - -* Each of these options has a specific API -* None are compatible with each-other -* Code written for one is **not portable** to another - -.. class:: incremental - -This makes it much more difficult to *share resources* - - -WSGI ----- - -Enter WSGI, the Web Server Gateway Interface. - -.. class:: incremental - -Where other alternatives are specific implementations of the CGI standard, -WSGI is itself a new standard, not an implementation. - -.. class:: incremental - -WSGI is generalized to describe a set of interactions, so that developers can -write WSGI-capable apps and deploy them on any WSGI server. - -.. class:: incremental - -Read the WSGI spec: http://www.python.org/dev/peps/pep-0333 - - -WSGI: Apps and Servers ----------------------- - -.. class:: small - -WSGI consists of two parts, a *server* and an *application*. - -.. class:: small - -A WSGI Server must: - -.. class:: incremental small - -* set up an environment, much like the one in CGI -* provide a method ``start_response(status, headers, exc_info=None)`` -* build a response body by calling an *application*, passing - ``environment`` and ``start_response`` as args -* return a response with the status, headers and body - -.. class:: small - -A WSGI Appliction must: - -.. class:: incremental small - -* Be a callable (function, method, class) -* Take an environment and a ``start_response`` callable as arguments -* Call the ``start_response`` method. -* Return an iterable of 0 or more strings, which are treated as the body of - the response. - - -Simplified WSGI Server ----------------------- - -.. code-block:: python - :class: small - - from some_application import simple_app - - def build_env(request): - # put together some environment info from the reqeuest - return env - - def handle_request(request, app): - environ = build_env(request) - iterable = app(environ, start_response) - for data in iterable: - # send data to client here - - def start_response(status, headers): - # start an HTTP response, sending status and headers - - # listen for HTTP requests and pass on to handle_request() - serve(simple_app) - - -Simple WSGI Application ------------------------ - -Where the simplified server above is **not** functional, this *is* a complete -app: - -.. code-block:: python - - def application(environ, start_response) - status = "200 OK" - body = "Hello World\n" - response_headers = [('Content-type', 'text/plain'), - ('Content-length', len(body))] - start_response(status, response_headers) - return [body] - - -WSGI Middleware ---------------- - -A third part of the puzzle is something called WSGI *middleware* - -.. class:: incremental - -* Middleware implements both the *server* and *application* interfaces -* Middleware acts as a server when viewed from an application -* Middleware acts as an application when viewed from a server - -.. image:: img/wsgi_middleware_onion.png - :align: center - :width: 38% - :class: incremental - - -Flowcharts ----------- - -WSGI Servers: - -.. class:: center incremental - -**HTTP <---> WSGI** - -.. class:: incremental - -WSGI Applications: - -.. class:: center incremental - -**WSGI <---> app code** - - -The Whole Enchilada -------------------- - -The WSGI *Stack* can thus be expressed like so: - -.. class:: incremental big-centered - -**HTTP <---> WSGI <---> app code** - - -Using wsgiref -------------- - -The Python standard lib provides a reference implementation of WSGI: - -.. image:: img/wsgiref_flow.png - :align: center - :width: 80% - :class: incremental - - -Apache mod_wsgi ---------------- - -You can also deploy with Apache as your HTTP server, using **mod_wsgi**: - -.. image:: img/mod_wsgi_flow.png - :align: center - :width: 80% - :class: incremental - - -Proxied WSGI Servers --------------------- - -Finally, it is also common to see WSGI apps deployed via a proxied WSGI -server: - -.. image:: img/proxy_wsgi.png - :align: center - :width: 80% - :class: incremental - - -The WSGI Environment --------------------- - -.. class:: small - -REQUEST_METHOD - The HTTP request method, such as "GET" or "POST". This cannot ever be an - empty string, and so is always required. -SCRIPT_NAME - The initial portion of the request URL's "path" that corresponds to the - application object, so that the application knows its virtual "location". - This may be an empty string, if the application corresponds to the "root" of - the server. -PATH_INFO - The remainder of the request URL's "path", designating the virtual - "location" of the request's target within the application. This may be an - empty string, if the request URL targets the application root and does not - have a trailing slash. -QUERY_STRING - The portion of the request URL that follows the "?", if any. May be empty or - absent. -CONTENT_TYPE - The contents of any Content-Type fields in the HTTP request. May be empty or - absent. - - -The WSGI Environment --------------------- - -.. class:: small - -CONTENT_LENGTH - The contents of any Content-Length fields in the HTTP request. May be empty - or absent. -SERVER_NAME, SERVER_PORT - When combined with SCRIPT_NAME and PATH_INFO, these variables can be used to - complete the URL. Note, however, that HTTP_HOST, if present, should be used - in preference to SERVER_NAME for reconstructing the request URL. See the URL - Reconstruction section below for more detail. SERVER_NAME and SERVER_PORT - can never be empty strings, and so are always required. -SERVER_PROTOCOL - The version of the protocol the client used to send the request. Typically - this will be something like "HTTP/1.0" or "HTTP/1.1" and may be used by the - application to determine how to treat any HTTP request headers. (This - variable should probably be called REQUEST_PROTOCOL, since it denotes the - protocol used in the request, and is not necessarily the protocol that will - be used in the server's response. However, for compatibility with CGI we - have to keep the existing name.) - - -The WSGI Environment --------------------- - -.. class:: small - -HTTP\_ Variables - Variables corresponding to the client-supplied HTTP request headers (i.e., - variables whose names begin with "HTTP\_"). The presence or absence of these - variables should correspond with the presence or absence of the appropriate - HTTP header in the request. - -.. class:: center incremental - -**Seem Familiar?** - - -A Bit of Repetition -------------------- - -Let's start simply. We'll begin by repeating our first CGI exercise in WSGI - -.. class:: incremental - -* Find the ``wsgi`` directory in the class resources. Copy it to your working - directory. -* Open the file ``wsgi_1.py`` in your text editor. -* We will fill in the missing values using the wsgi ``environ``, just as we - use ``os.environ`` in cgi - -.. class:: incremental center - -**But First** - - -Orientation ------------ - -.. code-block:: python - :class: small - - if __name__ == '__main__': - from wsgiref.simple_server import make_server - srv = make_server('localhost', 8080, application) - srv.serve_forever() - -.. class:: incremental - -Note that we pass our ``application`` function to the server factory - -.. class:: incremental - -We don't have to write a server, ``wsgiref`` does that for us. - -.. class:: incremental - -In fact, you should *never* have to write a WSGI server. - - -Orientation ------------ - -.. code-block:: python - :class: small - - def application(environ, start_response): - response_body = body % ( - environ.get('SERVER_NAME', 'Unset'), # server name - ... - ) - status = '200 OK' - response_headers = [('Content-Type', 'text/html'), - ('Content-Length', str(len(response_body)))] - start_response(status, response_headers) - return [response_body] - -.. class:: incremental - -We do not define ``start_response``, the application does that. - -.. class:: incremental - -We *are* responsible for determining the HTTP status. - - -Running a WSGI Script ---------------------- - -You can run this script with python:: - - $ python wsgi_1.py - -.. class:: incremental - -This will start a wsgi server. What host and port will it use? - -.. class:: incremental - -Point your browser at ``http://localhost:8080/``. Did it work? - -.. class:: incremental - -Go ahead and fill in the missing bits. Use the ``environ`` passed into -``application`` - - -Some Tips ---------- - -Because WSGI is a long-running process, the file you are editing is *not* -reloaded after you edit it. - -.. class:: incremental - -You'll need to quit and re-run the script between edits. - -.. class:: incremental - -You may also want to consider using ``print environ`` in your application so -you can see the dictionary. - -.. class:: incremental - -If you do that, where will the printed environment appear? - - -A More Complex Example ----------------------- - -Let's create a multi-page wsgi application. It will serve a small database of -python books. - -.. class:: incremental - -The database (with a very simple api) can be found in ``wsgi/bookdb.py`` - -.. class:: incremental - -* We'll need a listing page that shows the titles of all the books -* Each title will link to a details page for that book -* The details page for each book will display all the information and have a - link back to the list - - -Some Questions to Ponder ------------------------- - -.. class:: incremental - -When viewing our first wsgi app, do we see the name of the wsgi application -script anywhere in the URL? - -.. class:: incremental - -In our wsgi application script, how many applications did we actually have? - -.. class:: incremental - -How are we going to serve different types of information out of a single -application? - - -Dispatch --------- - -We have to write an app that will map our incoming request path to some code -that can handle that request. - -.. class:: incremental - -This process is called ``dispatch``. There are many possible approaches - -.. class:: incremental - -Let's begin by designing this piece of it. - -.. class:: incremental - -Open ``bookapp.py`` from the ``wsgi`` folder. We'll do our work here. - - -PATH ----- - -The wsgi environment gives us access to *PATH_INFO*, which maps to the URI the -user requested when they loaded the page. - -.. class:: incremental - -We can design the URLs that our app will use to assist us in routing. - -.. class:: incremental - -Let's declare that any request for ``/`` will map to the list page - -.. container:: incremental - - We can also say that the URL for a book will look like this:: - - http://localhost:8080/book/ - -Writing resolve_path --------------------- - -Let's write a function, called ``resolve_path`` in our application file. - -.. class:: incremental - -* It should take the *PATH_INFO* value from environ as an argument. -* It should return the function that will be called. -* It should also return any arguments needed to call that function. -* This implies of course that the arguments should be part of the PATH - - -My Solution ------------ - -.. code-block:: python - :class: small incremental - - def resolve_path(path): - urls = [(r'^$', books), - (r'^book/(id[\d]+)$', book)] - matchpath = path.lstrip('/') - for regexp, func in urls: - match = re.match(regexp, matchpath) - if match is None: - continue - args = match.groups([]) - return func, args - # we get here if no url matches - raise NameError - - -Application Updates -------------------- - -We need to hook our new router into the application. - -.. class:: incremental - -* The path should be extracted from ``environ``. -* The router should be used to get a function and arguments -* The body to return should come from calling that function with those - arguments -* If an error is raised by calling the function, an appropriate response - should be returned -* If the router raises a NameError, the application should return a 404 - response - - -My Solution ------------ - -.. code-block:: python - :class: small incremental - - def application(environ, start_response): - headers = [("Content-type", "text/html")] - try: - path = environ.get('PATH_INFO', None) - if path is None: - raise NameError - func, args = resolve_path(path) - body = func(*args) - status = "200 OK" - except NameError: - status = "404 Not Found" - body = "

          Not Found

          " - except Exception: - status = "500 Internal Server Error" - body = "

          Internal Server Error

          " - finally: - headers.append(('Content-length', str(len(body)))) - start_response(status, headers) - return [body] - - -Test Your Work --------------- - -Once you've got your script settled, run it:: - - $ python bookapp.py - -.. class:: incremental - -Then point your browser at ``http://localhost:8080/`` - -.. class:: incremental - -* ``http://localhost/book/id3`` -* ``http://localhost/book/id73/`` -* ``http://localhost/sponge/damp`` - -.. class:: incremental - -Did that all work as you would have expected? - - -Building the List ------------------ - -The function ``books`` should return an html list of book titles where each -title is a link to the detail page for that book - -.. class:: incremental - -* You'll need all the ids and titles from the book database. -* You'll need to build a list in HTML using this information -* Each list item should have the book title as a link -* The href for the link should be of the form ``/book/`` - - -My Solution ------------ - -.. code-block:: python - :class: incremental small - - def books(): - all_books = DB.titles() - body = ['

          My Bookshelf

          ', '
            '] - item_template = '
          • {title}
          • ' - for book in all_books: - body.append(item_template.format(**book)) - body.append('
          ') - return '\n'.join(body) - - -Test Your Work --------------- - -Quit and then restart your application script:: - - $ python bookapp.py - -.. container:: incremental - - Then reload the root of your application:: - - http://localhost:8080/ - -.. class:: incremental - -You should see a nice list of the books in the database. Do you? - -.. class:: incremental - -Click on a link to view the detail page. Does it load without error? - - -Showing Details ---------------- - -The next step of course is to polish up those detail pages. - -.. class:: incremental - -* You'll need to retrieve a single book from the database -* You'll need to format the details about that book and return them as HTML -* You'll need to guard against ids that do not map to books - -.. class:: incremental - -In this last case, what's the right HTTP response code to send? - - -My Solution ------------ - -.. code-block:: python - :class: incremental small - - def book(book_id): - page = """ -

          {title}

          - - - - -
          Author{author}
          Publisher{publisher}
          ISBN{isbn}
          - Back to the list - """ - book = DB.title_info(book_id) - if book is None: - raise NameError - return page.format(**book) - - -Revel in Your Success ---------------------- - -Quit and restart your script one more time - -.. class:: incremental - -Then poke around at your application and see the good you've made - -.. class:: incremental - -And your application is portable and sharable - -.. class:: incremental - -It should run equally well under any `wsgi server -`_ - - -A Few Steps Further -------------------- - -Next steps for an app like this might be: - -* Create a shared full page template and incorporate it into your app -* Improve the error handling by emitting error codes other than 404 and 500 -* Swap out the basic backend here with a different one, maybe a Web Service? -* Think about ways to make the application less tightly coupled to the pages - it serves - - -Homework --------- - -For your homework this week, you'll be creating a wsgi application of your -own. - -.. class:: incremental - -As the source of your data, use the mashup you created last week. - -.. class:: incremental - -Your application should have at least two separate "pages" in it. - -.. class:: incremental - -The HTML you produce does not need to be pretty, but it should be something -that shows up in a browser. - - -Submitting Your Homework ------------------------- - -To submit your homework: - -.. class:: small - -* Create a new python script in ``assignments/session04``. It should be - something I can run with: - -.. class:: small - -:: - - $ python your_script.py - -.. class:: small - -* Once your script is running, I should be able to view your application in my - browser. - -* Include all instructions I need to successfully run and view your script. - -* Add tests for your code. I should be able to run the tests like so: - -.. class:: small - -:: - - $ python tests.py - -.. class:: small - -* Commit your changes to your fork of the repo in github, then open a pull - request. - - -But Wait, There's More ----------------------- - -In addition, read and step through the quick tutorials on templates and -database persistence in the assignments directory. - -Use your flaskenv Python, it has everything you need installed. - - -Wrap-Up -------- - -For educational purposes, you might wish to take a look at the source code for -the ``wsgiref`` module. It's the canonical example of a simple wsgi server - - >>> import wsgiref - >>> wsgiref.__file__ - '/full/path/to/your/copy/of/wsgiref.py' - ... - -.. class:: incremental center - -**See you Next Time** diff --git a/source/presentations/session04.rst b/source/presentations/session04.rst new file mode 100644 index 00000000..eed0bf73 --- /dev/null +++ b/source/presentations/session04.rst @@ -0,0 +1,1889 @@ +********** +Session 04 +********** + +.. figure:: /_static/granny_mashup.png + :align: center + :width: 70% + + Paul Downey http://www.flickr.com/photos/psd/492139935/ - CC-BY + +Scraping, APIs and Mashups +========================== + +Wherein we learn how to make order from the chaos of the wild internet. + + +A Dilemma +--------- + +The internet makes a vast quantity of data available. + +.. rst-class:: build +.. container:: + + But not always in the form or combination you want. + + It would be nice to be able to combine data from different sources to + create *meaning*. + + +The Big Question +---------------- + +.. rst-class:: large centered + +But How? + + +The Big Answer +-------------- + +.. rst-class:: large centered + +Mashups + + +Mashups +------- + +A mashup is:: + + a web page, or web application, that uses and combines data, presentation + or functionality from two or more sources to create new services. + + -- wikipedia (http://en.wikipedia.org/wiki/Mashup_(web_application_hybrid)) + + +Data Sources +------------ + +The key to mashups is the idea of data sources. + +.. rst-class:: build +.. container:: + + These come in many flavors: + + .. rst-class:: build + + * Simple websites with data in HTML + * Web services providing structured data + * Web services providing tranformative service (geocoding) + * Web services providing presentation (mapping) + +Web Scraping +============ + +.. rst-class:: left +.. container:: + + It would be nice if all online data were available in well-structured formats. + + .. rst-class:: build + .. container:: + + The reality is that much data is available only in HTML. + + Still we can get at it, with some effort. + + By scraping the data from the web pages. + + +HTML +---- + +.. ifnotslides:: + + Ideally, it looks like this: + +.. code-block:: html + + + + + + +

          A nice clean paragraph

          +

          And another nice clean paragraph

          + + + + +.. nextslide:: HTML... IRL + +.. ifnotslides:: + + But in real life, it's more often like this: + +.. code-block:: html + + +
          + + + + +.. nextslide:: Filtering By Regular Expression + +The next job is to find the inspection data we can see when we click on the +restaurant names in our page. + +.. rst-class:: build +.. container:: + + Do you notice a pattern in how that data is structured? + + For each restaurant in our results, there are *two* ``
          `` tags. + + The first contains the content you see at first, the second the content + that displays when we click. + + What can you see that identifies these items? + + ``
          `` and ``
          `` + + Each pair shares an ID, and the stuff we want is in the second one + + Each number is different for each restaurant + + We can use a regular expression to help us here. + +.. nextslide:: Getting the Information Divs + +Let's write a function in ``mashup.py`` that will find all the divs in our +column with the right kind of id: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * It should match ``
          `` tags only + * It should match ids that start with ``PR`` + * It should match ids that contain some number of *digits* after that + * It should match ids that end with a *tilde* (``~``) character + + .. code-block:: python + + # add an import up top + import re + + # and add this function + def restaurant_data_generator(html): + id_finder = re.compile(r'PR[\d]+~') + return html.find_all('div', id=id_finder) + + +.. nextslide:: Verify It Works + +Let's add that step to the *main* block at the bottom of ``mashup.py`` (only +print the first of the many divs that match): + +.. rst-class:: build +.. container:: + + .. code-block:: python + + html, encoding = load_inspection_page('inspection_page.html') + parsed = parse_source(html, encoding) + content_col = parsed.find("td", id="contentcol") + data_list = restaurant_data_generator(content_col) + print data_list[0].prettify() + + + Finally, test it out: + + .. code-block:: bash + + (soupenv)$ python mashup.py +
          +
          Row 1 cell 1 +
          Row 2 cell 1 + + Row 2 cell 2
          This
          sure is a long cell + + + + +.. nextslide:: FFFFFFFFFUUUUUUUUUUUUU!!!! + +.. figure:: /_static/scream.jpg + :align: center + :width: 32% + + Photo by Matthew via Flickr (http://www.flickr.com/photos/purplemattfish/3918004964/) - CC-BY-NC-ND + + +.. nextslide:: The Law of The Internet + +.. rst-class:: large centered + +"Be strict in what you send and tolerant in what you receive" + + +Taming the Mess +--------------- + +Luckily, there are tools to help with this. + +.. rst-class:: build +.. container:: + + In python there are several candidates, but I like ``BeautifulSoup``. + + BeautifulSoup is a great tool, but it's not in the Standard Library. + + We'll need to install it. + + Create a virtualenv to do so: + + .. code-block:: bash + + $ pyvenv soupenv + ... + $ source soupenv/bin/activate + + (remember, for Windows users that should be ``soupenv/Scripts/activate.bat``) + + +.. nextslide:: Install BeautifulSoup + +Once the virtualenv is activated, you can simply use pip or easy_install to +install the libraries you want: + +.. code-block:: bash + + (soupenv)$ pip install beautifulsoup4 + + +.. nextslide:: Choose a Parsing Engine + +BeautifulSoup is built to use the Python HTMLParser. + +.. rst-class:: build + +* Batteries Included. It's already there +* It's not great, especially before Python 2.7.3 + +.. rst-class:: build +.. container:: + + BeautifulSoup also supports using other parsers. + + There are two good choices: ``lxml`` and ``html5lib``. + + ``lxml`` is better, but much harder to install. Let's use ``html5lib``. + + +.. nextslide:: Install a Parsing Engine + +Again, this is pretty simple:: + + (soupenv)$ pip install html5lib + +.. rst-class:: build +.. container:: + + Once installed, BeautifulSoup will choose it automatically. + + BeautifulSoup will choose the "best" available. + + You can specify the parser if you need to for some reason. + + In fact, in recent versions of BeautifulSoup, you'll be warned if you don't + (though you can ignore the warning). + + +.. nextslide:: Install Requests + +Python provides tools for opening urls and communicating with servers. It's +spread across the ``urllib`` and ``urllib2`` packages. + +.. rst-class:: build +.. container:: + + These packages have pretty unintuitive APIs. + + The ``requests`` library is becoming the de-facto standard for this type of + work. Let's install it too. + + .. code-block:: bash + + (soupenv)$ pip install requests + + +Our Class Mashup +---------------- + +We're going to explore some tools for making a mashup today + +.. rst-class:: build +.. container:: + + We'll be starting by scraping restaurant health code data for + a given ZIP code + + Then, we'll look up the geographic location of those zipcodes using Google + + Finally, we'll display the results of our work on a map + + Start by opening a new file in your editor: ``mashup.py``. + + +.. nextslide:: Getting Some HTML + +The source for the data we'll be displaying is a search tool provided by King +County. + +.. rst-class:: build +.. container:: + + It's supposed to have a web service, but the service is broken. + + Luckily, the HTML search works just fine. + + Open `the search form`_ in your browser. + + Fill in a ZIP code (perhaps 98101). + + Add a start and end date (perhaps about 1 or 2 years apart). + + Submit the form, and take a look at what you get. + +.. _the search form: http://info.kingcounty.gov/health/ehs/foodsafety/inspections/search.aspx + + +.. nextslide:: Repeat, But Automate + +Next we want to automate the process. + +.. rst-class:: build +.. container:: + + Copy the domain and path of the url into your new ``mashup.py`` file like + so: + + .. code-block:: python + + INSPECTION_DOMAIN = "http://info.kingcounty.gov" + INSPECTION_PATH = "/health/ehs/foodsafety/inspections/Results.aspx" + +.. nextslide:: Repeat, But Automate + +Next, copy the query parameters from the URL and convert them to a dictionary: + +.. code-block:: python + + INSPECTION_PARAMS = { + 'Output': 'W', + 'Business_Name': '', + 'Business_Address': '', + 'Longitude': '', + 'Latitude': '', + 'City': '', + 'Zip_Code': '', + 'Inspection_Type': 'All', + 'Inspection_Start': '', + 'Inspection_End': '', + 'Inspection_Closed_Business': 'A', + 'Violation_Points': '', + 'Violation_Red_Points': '', + 'Violation_Descr': '', + 'Fuzzy_Search': 'N', + 'Sort': 'H' + } + + +Fetching Search Results +----------------------- + +Next we'll use the ``requests`` library to write a function to fetch these +results on demand. + +.. rst-class:: build +.. container:: + + In ``requests``, each HTTP method has a module-level function: + + .. rst-class:: build + + * ``GET`` == ``requests.get(url, **kwargs)`` + * ``POST`` == ``requests.post(url, **kwargs)`` + * ... + + ``kwargs`` represent other parts of an HTTP request: + + .. rst-class:: build + + * ``params``: a dict of url parameters (?foo=bar&baz=bim) + * ``headers``: a dict of headers to send with the request + * ``data``: the body of the request, if any (form data for POST goes here) + * ... + + +.. nextslide:: Handling Requests Responses + +The return value from one of these functions is a ``response`` object which +provides: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * ``response.status_code``: see the HTTP Status Code returned + * ``response.ok``: True if ``response.status_code`` is not an error + * ``response.raise_for_status()``: call to raise a python error if it is + * ``response.headers``: The headers sent from the server + * ``response.text``: Body of the response, decoded to unicode + * ``response.encoding``: The encoding used to decode + * ``response.content``: The original encoded response body as bytes + + ``requests documentation``: http://docs.python-requests.org/en/latest/ + +.. nextslide:: Fetch Search Results + +We'll start by writing a function ``get_inspection_page`` + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * It will accept keyword arguments for each of the possible query values + * It will build a dictionary of request query parameters from incoming + keywords, using INSPECTION_PARAMS as a template + * It will make a request to the inspection service search page using this + query + * It will return the encoded content and the encoding used as a tuple + + Try writing this function. Put it in ``mashup.py`` + + +My Solution +----------- + +Here's the one I created: + +.. rst-class:: build + +.. code-block:: python + + import requests + + def get_inspection_page(**kwargs): + url = INSPECTION_DOMAIN + INSPECTION_PATH + params = INSPECTION_PARAMS.copy() + for key, val in kwargs.items(): + if key in INSPECTION_PARAMS: + params[key] = val + resp = requests.get(url, params=params) + resp.raise_for_status() + return resp.text + + +Parse the Results +----------------- + +Next, we'll need to parse the results we get when we call that function + +But before we start, a word about parsing HTML with BeautifulSoup + + +.. nextslide:: Parsing HTML with BeautifulSoup + +The BeautifulSoup object can be instantiated with a string or a file-like +object as the sole argument: + +.. rst-class:: build +.. container:: + + .. code-block:: python + + from bs4 import BeautifulSoup + parsed = BeautifulSoup('

          Some HTML

          ') + + fh = open('a_page.html', 'r') + parsed = BeautifulSoup(fh) + + page = urllib2.urlopen('http://site.com/page.html') + parsed = BeautifulSoup(page) + + You might want to open the documentation as reference + (http://www.crummy.com/software/BeautifulSoup/bs4/doc) + + +My Solution +----------- + +Take a shot at writing this new function in ``mashup.py`` + +.. code-block:: python + + # add this import at the top + from bs4 import BeautifulSoup + + # then add this function lower down + def parse_source(html): + parsed = BeautifulSoup(html) + return parsed + + +Put It Together +--------------- + +We'll need to make our script do something when run. + +.. code-block:: python + + if __name__ == '__main__': + # do something + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * Fetch a search results page + * Parse the resulting HTML + * For now, print out the results so we can see what we get + + .. container:: + + Use the ``prettify`` method on a BeautifulSoup object:: + + print(parsed.prettify()) + + +My Solution +----------- + +Try to come up with the proper code on your own. Add it to ``mashup.py`` + +.. rst-class:: build +.. code-block:: python + + if __name__ == '__main__': + use_params = { + 'Inspection_Start': '2/1/2013', + 'Inspection_End': '2/1/2015', + 'Zip_Code': '98101' + } + html = get_inspection_page(**use_params) + parsed = parse_source(html) + print(parsed.prettify()) + + +.. nextslide:: Test The Results + +Assuming your virtualenv is still active, you should be able to execute the +script. + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + (soupenv)$ python mashup.py + ... + + + + + + + This script is available as ``resources/session04/mashup_1.py`` + + + +.. nextslide:: Preserve the Results + +Now, let's re-run the script, saving the output to a file so we can use it +later:: + + $ python mashup.py > inspection_page.html + +.. rst-class:: build +.. container:: + + Then add a quick function to our script that will use these saved results: + + .. code-block:: python + + def load_inspection_page(name): + file_path = pathlib.Path(name) + return file_path.read_text(encoding='utf8') + + Finally, bolt that in to your script to use it: + + .. code-block:: python + + # COMMENT OUT THIS LINE AND REPLACE IT + # html = get_inspection_page(**use_params) + html = load_inspection_page('inspection_page.html') + + +Extracting Data +--------------- + +Next we find the bits of this pile of HTML that matter to us. + +.. rst-class:: build +.. container:: + + Open the page you just wrote to disk in your web browser and open the + developer tools to inspect the page source. + + You'll want to start by finding the element in the page that contains all + our search results. + + Look at the source and identify the single element we are looking for. + +.. nextslide:: Tags and Searching + +Having found it visually, we can now search for it automatically. In +BeautifulSoup: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * All HTML elements (including the parsed document itself) are ``tags`` + * A ``tag`` can be searched using its ``find`` or ``find_all`` methods + * This searches the descendents of the tag on which it is called. + * It takes arguments which act as *filters* on the search results + + .. container:: + + like so:: + + tag.find(name, attrs, recursive, text, **kwargs) + tag.find_all(name, attrs, recursive, text, limit, **kwargs) + + +.. nextslide:: Searching by Attribute + +The ``find`` method allows us to pass *kwargs*. + +.. rst-class:: build +.. container:: + + Keywords that are not among the named parameters will be considered an HTML + attribute. + + We can use this to find the column that holds our search results: + + .. code-block:: python + + content_col = parsed.find('td', id="contentcol") + + Add that line to our mashup script and try it out: + + .. code-block:: python + + #... + parsed = parse_source(html) + content_col = parsed.find("td", id="contentcol") + print content_col.prettify() + + .. code-block:: bash + + (soupenv)$ python mashup.py +
          + ... +
          + ... +
          + + + This code is available as ``/resources/session04/mashup_2.py`` + + +Parsing Restaurant Data +----------------------- + +Now that we have the records we want, we need to parse them. + +.. rst-class:: build +.. container:: + + We'll start by extracting information about the restaurants: + + .. rst-class:: build + + * Name + * Address + * Location + + How is this information contained in our records? + + +.. nextslide:: Complex Filtering + +Each record consists of a table with a series of *rows* (``
          `` elements + + My solution: + + .. code-block:: python + + def has_two_tds(elem): + is_tr = elem.name == 'tr' + td_children = elem.find_all('td', recursive=False) + has_two = len(td_children) == 2 + return is_tr and has_two + +.. nextslide:: Test It Out + +Let's try this out in an interpreter: + +.. code-block:: ipython + + In [1]: from mashup_3 import load_inspection_page, parse_source, + restaurant_data_generator, has_two_tds + In [2]: html = load_inspection_page('inspection_page.html') + In [3]: parsed = parse_source(html) + ... + In [4]: content_col = parsed.find('td', id='contentcol') + In [5]: records = restaurant_data_generator(content_col) + In [6]: rec = records[4] + +.. nextslide:: Test It Out + +We'd like to find all table rows in that div that contain *two* cells + +.. rst-class:: build +.. container:: + + The table rows are all contained in a ``
          + - Business Name + + SPICE ORIENT +
          `` children + * The text in the first cell contains the word "inspection" + * But the text does not *start* with the word "inspection" + + Let's try to write a filter function like the one above that will catch + these rows for us. + +.. nextslide:: The filter + +Add this new function ``is_inspection_data_row`` to ``mashup.py`` + +.. rst-class:: build +.. code-block:: python + + def is_inspection_data_row(elem): + is_tr = elem.name == 'tr' + if not is_tr: + return False + td_children = elem.find_all('td', recursive=False) + has_four = len(td_children) == 4 + this_text = clean_data(td_children[0]).lower() + contains_word = 'inspection' in this_text + does_not_start = not this_text.startswith('inspection') + return is_tr and has_four and contains_word and does_not_start + +.. nextslide:: Test It Out + +We can test this function by adding it into our script: + +.. code-block:: python + + for data_div in data_list: + metadata = extract_restaurant_metadata(data_div) + # UPDATE THIS BELOW HERE + inspection_rows = data_div.find_all(is_inspection_data_row) + print(metadata) + print(len(inspection_rows)) + print('*'*10) + +.. rst-class:: build +.. container:: + + And try running the script in your terminal: + + .. code-block:: bash + + (soupenv)$ python mashup.py + {u'Business Category': [u'Seating 0-12 - Risk Category III'], + u'Longitude': [u'122.3401786000'], u'Phone': [u'(206) 501-9554'], + u'Business Name': [u"ZACCAGNI'S"], u'Address': [u'97B PIKE ST', u'SEATTLE, WA 98101'], + u'Latitude': [u'47.6086651300']} + 0 + ********** + +.. nextslide:: Building Inspection Data + +Now we can isolate a list of the rows that contain inspection data. + +.. rst-class:: build +.. container:: + + Next we need to calculate the average score, total number and highest score + for each restaurant. + + Let's add a function to ``mashup.py`` that will: + + .. rst-class:: build + + * Take a div containing a restaurant record + * Extract the rows containing inspection data + * Keep track of the highest score recorded + * Sum the total of all inspections + * Count the number of inspections made + * Calculate the average score for inspections + * Return the three calculated values in a dictionary + +.. nextslide:: My Solution + +Try writing this routine yourself. + +.. code-block:: python + + def get_score_data(elem): + inspection_rows = elem.find_all(is_inspection_data_row) + samples = len(inspection_rows) + total = high_score = average = 0 + for row in inspection_rows: + strval = clean_data(row.find_all('td')[2]) + try: + intval = int(strval) + except (ValueError, TypeError): + samples -= 1 + else: + total += intval + high_score = intval if intval > high_score else high_score + if samples: + average = total/float(samples) + return {'Average Score': average, 'High Score': high_score, + 'Total Inspections': samples} + +.. nextslide:: Test It Out + +We can now incorporate this new routine into our ``mashup`` script. + +.. rst-class:: build +.. container:: + + We'll want to add the data it produces to the metadata we've already + extracted. + + .. code-block:: python + + for data_div in data_list: + metadata = extract_restaurant_metadata(data_div) + inspection_data = get_score_data(data_div) + metadata.update(inspection_data) + print metadata + + And test it out at the command line: + + .. code-block:: bash + + (soupenv)$ python mashup.py + ... + {u'Business Category': [u'Seating 0-12 - Risk Category III'], + u'Longitude': [u'122.3401786000'], u'High Score': 0, + u'Phone': [u'(206) 501-9554'], u'Business Name': [u"ZACCAGNI'S"], + u'Total Inspections': 0, u'Address': [u'97B PIKE ST', u'SEATTLE, WA 98101'], + u'Latitude': [u'47.6086651300'], u'Average Score': 0} + +Break Time +---------- + +Once you have this working, take a break. + +When we return, we'll try a saner approach to getting data from online + + + +Another Approach +================ + +.. rst-class:: left +.. container:: + + Scraping web pages is tedious and inherently brittle + + .. rst-class:: build + .. container:: + + The owner of the website updates their layout, your code breaks + + But there is another way to get information from the web in a more normalized + fashion + + .. rst-class:: centered + + **Web Services** + + +Web Services +------------ + +"a software system designed to support interoperable machine-to-machine +interaction over a network" - W3C + +.. rst-class:: build + +* provides a defined set of calls +* returns structured data + + +.. nextslide:: Early Web Services + +**RSS** is one of the earliest forms of Web Services + +.. rst-class:: build +.. container:: + + A single web-based *endpoint* provides a dynamically updated listing of + content + + Implemented in pure HTTP. Returns XML + + **Atom** is a competing, but similar standard + + There's a solid Python library for consuming RSS: `feedparser`_. + +.. _feedparser: https://pythonhosted.org/feedparser/ + +.. nextslide:: XML-RPC + +XML-RPC extended the essentially static nature of RSS by allowing users to call +procedures and pass arguments. + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * Calls are made via HTTP GET, by passing an XML document + * Returns from a call are sent to the client in XML + + In python, you can access XML-RPC services using `xmlrpc`_ from the + standard library. It has two libraries, ``xmlrpc.client`` and + ``xmlrpc.server`` + +.. _xmlrpc: https://docs.python.org/3.5/library/xmlrpc.html + +.. nextslide:: SOAP + +SOAP extends XML-RPC in a couple of useful ways: + +.. rst-class:: build + +* It uses Web Services Description Language (WSDL) to provide meta-data about + an entire service in a machine-readable format (Automatic introspection) + +* It establishes a method for extending available data types using XML + namespaces + +.. rst-class:: build +.. container:: + + There is no standard library module that supports SOAP directly. + + .. rst-class:: build + + * The best-known and best-supported module available is **Suds** + * The homepage is https://fedorahosted.org/suds/ + * It can be installed using ``easy_install`` or ``pip install`` + * A `fork of the library`_ compatible with Python 3 does exist + + **I HATE SOAP** + +.. _fork of the library: https://github.com/cackharot/suds-py3 + +.. nextslide:: What about WSDL? + +SOAP was invented in part to provide completely machine-readable +interoperability. + +.. rst-class:: build +.. container:: + + *Does that really work in real life?* + + .. rst-class:: centered + + **Hardly ever** + + Another reason was to provide extensibility via custom types + + *Does that really work in real life?* + + .. rst-class:: centered + + **Hardly ever** + +.. nextslide:: I have to write XML? + +In addition, XML is a pretty inefficient medium for transmitting data. There's +a lot of extra characters transmitted that lack any meaning. + +.. code-block:: xml + + + + + + + + IBM + + + + +.. nextslide:: Why Do All The Work? + +So, if neither of the original goals is really achieved by using SOAP + +.. rst-class:: build +.. container:: + + And if the transmission medium is too bloated to use + + why pay all the overhead required to use the protocol? + + Is there another way we could consider approaching the problem? + + .. rst-class:: centered + + **Enter REST** + + +REST +---- + +.. rst-class:: centered + +**Representational State Transfer** + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * Originally described by Roy T. Fielding (worth reading) + * Use HTTP for what it can do + * Read more in `RESTful Web Services `_\* + + \* Seriously. Buy it and read it + +.. nextslide:: A Comparison + +The XML-RCP/SOAP way: + +.. rst-class:: build + +* POST /getComment HTTP/1.1 +* POST /getComments HTTP/1.1 +* POST /addComment HTTP/1.1 +* POST /editComment HTTP/1.1 +* POST /deleteComment HTTP/1.1 + +.. rst-class:: build +.. container:: + + The RESTful way: + + .. rst-class:: build + + * GET /comment/ HTTP/1.1 + * GET /comment HTTP/1.1 + * POST /comment HTTP/1.1 + * PUT /comment/ HTTP/1.1 + * DELETE /comment/ HTTP/1.1 + + +.. nextslide:: ROA + +REST is a **Resource Oriented Architecture** + +.. rst-class:: build +.. container:: + + The URL represents the *resource* we are working with + + The HTTP Method indicates the ``action`` to be taken + + The HTTP Code returned tells us the ``result`` (whether success or failure) + +.. nextslide:: HTTP Codes Revisited + +.. rst-class:: build +.. container:: + + POST /comment HTTP/1.1 (creating a new comment): + + .. rst-class:: build + + * Success: ``HTTP/1.1 201 Created`` + * Failure (unauthorized): ``HTTP/1.1 401 Unauthorized`` + * Failure (NotImplemented): ``HTTP/1.1 405 Not Allowed`` + * Failure (ValueError): ``HTTP/1.1 406 Not Acceptable`` + + PUT /comment/ HTTP/1.1 (edit comment): + + .. rst-class:: build + + * Success: ``HTTP/1.1 200 OK`` + * Failure: ``HTTP/1.1 409 Conflict`` + + DELETE /comment/ HTTP/1.1 (delete comment): + + .. rst-class:: build + + * Success: ``HTTP/1.1 204 No Content`` + +REST uses JSON +-------------- + +JavaScript Object Notation: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * a lightweight data-interchange format + * easy for humans to read and write + * easy for machines to parse and generate + + Based on Two Structures: + + * object: ``{ string: value, ...}`` + * array: ``[value, value, ]`` + + .. rst-class:: centered + + pythonic, no? + + +.. nextslide:: JSON Data Types + +JSON provides a few basic data types (see http://json.org/): + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * string: unicode, anything but ", \\ and control characters + * number: any number, but json does not use octal or hexadecimal + * object, array (we've seen these above) + * true + * false + * null + + .. rst-class:: centered + + **No date type? OMGWTF??!!1!1** + +.. nextslide:: Dates in JSON + +You have two options: + +.. rst-class:: build +.. container:: + + .. container:: + + Option 1 - Unix Epoch Time (number): + + .. code-block:: python + + >>> import time + >>> time.time() + 1358212616.7691269 + + .. container:: + + Option 2 - ISO 8661 (string): + + .. code-block:: python + + >>> import datetime + >>> datetime.datetime.now().isoformat() + '2013-01-14T17:18:10.727240' + + +JSON in Python +-------------- + +You can encode python to json, and decode json back to python: + +.. rst-class:: build +.. container:: + + .. code-block:: python + + In [1]: import json + In [2]: array = [1, 2, 3] + In [3]: json.dumps(array) + Out[3]: '[1, 2, 3]' + In [4]: orig = {'foo': [1,2,3], 'bar': 'my resumé', 'baz': True} + In [5]: encoded = json.dumps(orig) + In [6]: encoded + Out[6]: '{"foo": [1, 2, 3], "bar": "my resum\\u00e9", "baz": true}' + In [7]: decoded = json.loads(encoded) + In [8]: decoded == orig + Out[8]: True + + Customizing the encoder or decoder class allows for specialized serializations + + +.. nextslide:: + +the json module also supports reading and writing to *file-like objects* via +``json.dump(fp)`` and ``json.load(fp)`` (note the missing 's') + +.. rst-class:: build +.. container:: + + Remember duck-typing. Anything with a ``.write`` and a ``.read`` method is + *file-like* + + This usage can be much more memory-friendly with large files/sources + + +Playing With REST +----------------- + +Let's take a moment to play with REST. + +.. rst-class:: build +.. container:: + + We'll use a common, public API provided by Google. + + .. rst-class:: centered + + **Geocoding** + +.. nextslide:: Geocoding with Google APIs + +https://developers.google.com/maps/documentation/geocoding + +.. rst-class:: build +.. container:: + + Open a python interpreter using our virtualenv:: + + (soupenv)$ python + + .. code-block:: ipython + + In [1]: import requests + In [2]: import json + In [3]: from pprint import pprint + In [4]: url = 'http://maps.googleapis.com/maps/api/geocode/json' + In [5]: addr = '1325 4th Ave, Seattle, 98101' + In [6]: parameters = {'address': addr, 'sensor': 'false'} + In [7]: resp = requests.get(url, params=parameters) + In [8]: data = resp.json() + + +.. nextslide:: Reverse Geocoding + +You can do the same thing in reverse, supply latitude and longitude and get +back address information: + +.. rst-class:: build +.. container:: + + .. code-block:: ipython + + In [15]: if data['status'] == 'OK': + ....: pprint(data['results']) + ....: + [{'address_components': [{'long_name': '1325', + 'short_name': '1325', + ... + 'types': ['street_address']}] + + Notice that there may be a number of results returned, ordered from most + specific to least. + + +Mashing It Up +------------- + +Google's geocoding data is quite nice. + +.. rst-class:: build +.. container:: + + But it's not in a format we can use directly to create a map + + For that we need `geojson` + + Moreover, formatting the data for all those requests is going to get + tedious. + + Luckily, people create *wrappers* for popular REST apis like google's + geocoding service. + + Once such wrapper is `geocoder`_, which provides not only google's service, + but many others under a single umbrella. + +.. _geocoder: http://geocoder.readthedocs.org/en/latest/ +.. _geojson: http://geojson.org + +.. nextslide:: Install ``geocoder`` + +Install geocoder into your ``soupenv`` so that it's available to use: + +.. code-block:: bash + + (soupenv)$ pip install geocoder + +.. rst-class:: build +.. container:: + + Our final step for tonight will be to geocode the results we have scraped + from the inspection site. + + We'll then convert that to ``geojson``, insert our own properties and map + the results. + + Let's begin by converting our script so that what we have so far is + contained in a generator function + + We'll eventually sort our results and generate the top 10 or so for + geocoding. + + Open up ``mashup.py`` and copy everthing in the ``main`` block. + +.. nextslide:: Make a Generator Function + +Add a new function ``result_generator`` to the ``mashup.py`` script. Paste the +code you copied from the ``main`` block and then update it a bit: + +.. rst-class:: build +.. code-block:: python + + def result_generator(count): + use_params = { + 'Inspection_Start': '2/1/2013', + 'Inspection_End': '2/1/2015', + 'Zip_Code': '98101' + } + # html, encoding = get_inspection_page(**use_params) + html, encoding = load_inspection_page('inspection_page.html') + parsed = parse_source(html, encoding) + content_col = parsed.find("td", id="contentcol") + data_list = restaurant_data_generator(content_col) + for data_div in data_list[:count]: + metadata = extract_restaurant_metadata(data_div) + inspection_data = get_score_data(data_div) + metadata.update(inspection_data) + yield metadata + + +.. nextslide:: Test It Out + +Update the ``main`` block of your ``mashup.py`` script to use the new function: + +.. rst-class:: build +.. container:: + + .. code-block:: python + + if __name__ == '__main__': + for result in result_generator(10): + print result + + Then run your script and verify that the only thing that has changed is the + number of results that print. + + .. code-block:: bash + + (soupenv)$ python mashup.py + # you should see 10 dictionaries print here. + +Add Geocoding +------------- + +The API for geocoding with ``geocoder`` is the same for all providers. + +.. rst-class:: build +.. container:: + + You give an address, it returns geocoded data. + + You provide latitude and longitude, it provides address data + + .. code-block:: ipython + + In [1]: response = geocoder.google(
          ) + In [2]: response.json + Out[2]: # json result data + In [3]: response.geojson + Out[3]: # geojson result data + +.. nextslide:: Adding The Function + +Let's add a new function ``get_geojson`` to ``mashup.py`` + +.. rst-class:: build +.. container:: + + It will + + .. rst-class:: build + + * Take a result from our search as it's input + * Get geocoding data from google using the address of the restaurant + * Return the geojson representation of that data + + Try to write this function on your own + + .. code-block:: python + + def get_geojson(result): + address = " ".join(result.get('Address', '')) + if not address: + return None + geocoded = geocoder.google(address) + return geocoded.geojson + +.. nextslide:: Testing It Out + +Next, update our ``main`` block to get the geojson for each result and print +it: + +.. rst-class:: build +.. container:: + + .. code-block:: python + + if __name__ == '__main__': + for result in result_generator(10): + geojson = get_geojson(result) + print geojson + + Then test your results by running your script: + + .. code-block:: bash + + (soupenv)$ python mashup.py + {'geometry': {'type': 'Point', 'coordinates': [-122.3393005, 47.6134378]}, + 'type': 'Feature', 'properties': {'neighborhood': 'Belltown', + 'encoding': 'utf-8', 'county': 'King County', 'city_long': 'Seattle', + 'lng': -122.3393005, 'quality': u'street_address', 'city': 'Seattle', + 'confidence': 9, 'state': 'WA', 'location': u'1933 5TH AVE SEATTLE, WA 98101', + 'provider': 'google', 'housenumber': '1933', 'accuracy': 'ROOFTOP', + 'status': 'OK', 'state_long': 'Washington', + 'address': '1933 5th Avenue, Seattle, WA 98101, USA', 'lat': 47.6134378, + 'postal': '98101', 'ok': True, 'road_long': '5th Avenue', 'country': 'US', + 'country_long': 'United States', 'street': '5th Ave'}, + 'bbox': [-122.3406494802915, 47.6120888197085, -122.3379515197085, 47.6147867802915]} + +.. nextslide:: Update Geojson Properties + +The ``properties`` of our geojson records are filled with data we don't really +care about. + +.. rst-class:: build +.. container:: + + Let's replace that information with some of the metadata from our + inspection results. + + We'll update our ``get_geojson`` function so that it: + + .. rst-class:: build + + * Builds a dictionary containing only the values we want from our + inspection record. + * Converts list values to strings (geojson requires this) + * Replaces the 'properties' of our geojson with this new data + * Returns the modified geojson record + +.. nextslide:: Write the Function + +See if you can make the updates on your own. + +.. rst-class:: build +.. code-block:: python + + def get_geojson(result): + # ... + geocoded = geocoder.google(address) + geojson = geocoded.geojson + inspection_data = {} + use_keys = ( + 'Business Name', 'Average Score', 'Total Inspections', 'High Score' + ) + for key, val in result.items(): + if key not in use_keys: + continue + if isinstance(val, list): + val = " ".join(val) + inspection_data[key] = val + geojson['properties'] = inspection_data + return geojson + +.. nextslide:: Making Mappable Data + +We are now generating a series of ``geojson`` *Feature* objects. + +.. rst-class:: build +.. container:: + + To map these objects, we'll need to create a file which contains a + ``geojson`` *FeatureCollection*. + + The structure of such a collection looks like this: + + .. code-block:: json + + {'type': 'FeatureCollection', 'features': [...]} + + Let's update our ``main`` function to append each feature to such a + structure. + + Then we can dump the structure as ``json`` to a file. + +.. nextslide:: Update the Script + +In ``mashup.py`` update the ``main`` block like so: + +.. rst-class:: build +.. container:: + + .. code-block:: python + + # add an import at the top: + import json + + if __name__ == '__main__': + total_result = {'type': 'FeatureCollection', 'features': []} + for result in result_generator(10): + geojson = get_geojson(result) + total_result['features'].append(geojson) + with open('my_map.json', 'w') as fh: + json.dump(total_result, fh) + + When you run the script nothing will print, but the new file will appear. + + .. code-block:: bash + + (soupenv)$ python mashup.py + + This script is available as ``resources/session04/mashup_5.py`` + +Display the Results +------------------- + +Once the new file is written you are ready to display your results. + +.. rst-class:: build +.. container:: + + Open your web browser and go to http://geojson.io + + Then drag and drop the new file you wrote onto the map you see there. + + .. figure:: /_static/geojson-io.png + :align: center + :width: 75% + +Wrap Up +------- + +We've built a simple mashup combining data from different sources. + +.. rst-class:: build +.. container:: + + We scraped health inspection data from the King County government site. + + We geocoded that data. + + And we've displayed the results on a map. + + What other sources of data might we choose to combine? + + Check out `programmable web `_ + to see some of the possibilities + + + + +Homework +======== + +.. rst-class:: left +.. container:: + + For your homework this week, you'll be polishing this mashup. + + .. rst-class:: build + .. container:: + + Begin by sorting the results of our search by the average score (can + you do this and still use a generator for getting the geojson?). + + Then, update your script to allow the user to choose how to sort, by + average, high score or most inspections:: + + (soupenv)$ python mashup.py highscore + + Next, allow the user to choose how many results to map:: + + (soupenv)$ python mashup.py highscore 25 + + Or allow them to reverse the results, showing the lowest scores first:: + + (soupenv)$ python mashup.py highscore 25 reverse + + If you're feeling particularly adventurous, see if you can use the + `argparse`_ module from the standard library to handle command line + arguments + +.. _argparse: https://docs.python.org/2/library/argparse.html#module-argparse + +More Fun +-------- + +Next, try adding a bit of information to your map by setting the +``marker-color`` property. This will display a marker with the provided +css-style color (``#FF0000``) + +.. rst-class:: build +.. container:: + + See if you can make the color change according to the values used for the + sorting of the list. Either vary the intensity of the color, or the hue. + + Finally, if you are feeling particularly frisky, you can update your script + to automatically open a browser window with your map loaded on + *geojson.io*. + + To do this, you'll want to read about the `webbrowser`_ module from the + standard library. + + In addition, you'll want to read up on using the URL parameters API for + *geojson.io*. Click on the **help** tab in the sidebar to view the + information. + + You will also need to learn about how to properly quote special characters + for a URL, using the `urllib.parse`_ ``quote`` function. + +.. _urllib.parse: https://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote +.. _webbrowser: https://docs.python.org/3/library/webbrowser.html + +Submitting Your Work +-------------------- + +Create a github repository to contain your mashup work. Start by populating it +with the script as we finished it today (mashup_5.py). + +As you implement the above features, commit early and commit often. + +When you're ready for us to look it over, email a link to your repository to +Maria and I. + diff --git a/source/presentations/session04.rst.norender b/source/presentations/session04.rst.norender deleted file mode 100644 index 39b36076..00000000 --- a/source/presentations/session04.rst.norender +++ /dev/null @@ -1,1127 +0,0 @@ -Python Web Programming -====================== - -.. image:: img/python.png - :align: left - :width: 33% - -Session 1: Networking and Sockets - - -Computer Communications ------------------------ - -.. image:: img/network_topology.png - :align: left - :width: 40% - -.. class:: incremental - -* processes can communicate - -* inside one machine - -* between two machines - -* among many machines - -.. class:: image-credit - -image: http://en.wikipedia.org/wiki/Internet_Protocol_Suite - - -Computer Communications ------------------------ - -.. image:: img/data_in_tcpip_stack.png - :align: left - :width: 55% - -.. class:: incremental - -* Process divided into 'layers' - -* 'Layers' are mostly arbitrary - -* Different descriptions have different layers - -* Most common is the 'TCP/IP Stack' - -.. class:: image-credit - -image: http://en.wikipedia.org/wiki/Internet_Protocol_Suite - - -The TCP/IP Stack - Link ------------------------ - -The bottom layer is the 'Link Layer' - -.. class:: incremental - -* Deals with the physical connections between machines, 'the wire' - -* Packages data for physical transport - -* Executes transmission over a physical medium - - * what that medium is is arbitrary - -* Implemented in the Network Interface Card(s) (NIC) in your computer - - -The TCP/IP Stack - Internet ---------------------------- - -Moving up, we have the 'Internet Layer' - -.. class:: incremental - -* Deals with addressing and routing - - * Where are we going and how do we get there? - -* Agnostic as to physical medium (IP over Avian Carrier - IPoAC) - -* Makes no promises of reliability - -* Two addressing systems - - .. class:: incremental - - * IPv4 (current, limited '192.168.1.100') - - * IPv6 (future, 3.4 x 10^38 addresses, '2001:0db8:85a3:0042:0000:8a2e:0370:7334') - - -The TCP/IP Stack - Internet ---------------------------- - -.. class:: big-centered - -That's 4.3 x 10^28 addresses *per person alive today* - - -The TCP/IP Stack - Transport ----------------------------- - -Next up is the 'Transport Layer' - -.. class:: incremental - -* Deals with transmission and reception of data - - * error correction, flow control, congestion management - -* Common protocols include TCP & UDP - - * TCP: Tranmission Control Protocol - - * UDP: User Datagram Protocol - -* Not all Transport Protocols are 'reliable' - - .. class:: incremental - - * TCP ensures that dropped packets are resent - - * UDP makes no such assurance - - * Reliability is slow and expensive - - -The TCP/IP Stack - Transport ----------------------------- - -The 'Transport Layer' also establishes the concept of a **port** - -.. class:: incremental - -* IP Addresses designate a specific *machine* on the network - -* A **port** provides addressing for individual *applications* in a single host - -* 192.168.1.100:80 (the *:80* part is the **port**) - -* [2001:db8:85a3:8d3:1319:8a2e:370:7348]:443 (*:443* is the **port**) - -.. class:: incremental - -This means that you don't have to worry about information intended for your -web browser being accidentally read by your email client. - - -The TCP/IP Stack - Transport ----------------------------- - -There are certain **ports** which are commonly understood to belong to given -applications or protocols: - -.. class:: incremental - -* 80/443 - HTTP/HTTPS -* 20 - FTP -* 22 - SSH -* 23 - Telnet -* 25 - SMTP -* ... - -.. class:: incremental - -These ports are often referred to as **well-known ports** - -.. class:: small - -(see http://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers) - - -The TCP/IP Stack - Transport ----------------------------- - -Ports are grouped into a few different classes - -.. class:: incremental - -* Ports numbered 0 - 1023 are *reserved* - -* Ports numbered 1024 - 65535 are *open* - -* Ports numbered 1024 - 49151 may be *registered* - -* Ports numbered 49152 - 65535 are called *ephemeral* - - -The TCP/IP Stack - Application ------------------------------- - -The topmost layer is the 'Application Layer' - -.. class:: incremental - -* Deals directly with data produced or consumed by an application - -* Reads or writes data using a set of understood, well-defined **protocols** - - * HTTP, SMTP, FTP etc. - -* Does not know (or need to know) about lower layer functionality - - * The exception to this rule is **endpoint** data (or IP:Port) - - -The TCP/IP Stack - Application ------------------------------- - -.. class:: big-centered - -this is where we live and work - - -Sockets -------- - -Think back for a second to what we just finished discussing, the TCP/IP stack. - -.. class:: incremental - -* The *Internet* layer gives us an **IP Address** - -* The *Transport* layer establishes the idea of a **port**. - -* The *Application* layer doesn't care about what happens below... - -* *Except for* **endpoint data** (IP:Port) - -.. class:: incremental - -A **Socket** is the software representation of that endpoint. - -.. class:: incremental - -Opening a **socket** creates a kind of transceiver that can send and/or -receive *bytes* at a given IP address and Port. - - -Sockets in Python ------------------ - -Python provides a standard library module which provides socket functionality. -It is called **socket**. - -.. class:: incremental - -The library is really just a very thin wrapper around the system -implementation of *BSD Sockets* - -.. class:: incremental - -Let's spend a few minutes getting to know this module. - -.. class:: incremental - -We're going to do this next part together, so open up a terminal and start a -python interpreter - - -Sockets in Python ------------------ - -The Python sockets library allows us to find out what port a *service* uses: - -.. class:: small - - >>> import socket - >>> socket.getservbyname('ssh') - 22 - -.. class:: incremental - -You can also do a *reverse lookup*, finding what service uses a given *port*: - -.. class:: incremental small - - >>> socket.getservbyport(80) - 'http' - - -Sockets in Python ------------------ - -The sockets library also provides tools for finding out information about -*hosts*. For example, you can find out about the hostname and IP address of -the machine you are currently using:: - - >>> socket.gethostname() - 'heffalump.local' - >>> socket.gethostbyname(socket.gethostname()) - '10.211.55.2' - - -Sockets in Python ------------------ - -You can also find out about machines that are located elsewhere, assuming you -know their hostname. For example:: - - >>> socket.gethostbyname('google.com') - '173.194.33.4' - >>> socket.gethostbyname('uw.edu') - '128.95.155.135' - >>> socket.gethostbyname('crisewing.com') - '108.59.11.99' - - -Sockets in Python ------------------ - -The ``gethostbyname_ex`` method of the ``socket`` library provides more -information about the machines we are exploring:: - - >>> socket.gethostbyname_ex('google.com') - ('google.com', [], ['173.194.33.9', '173.194.33.14', - ... - '173.194.33.6', '173.194.33.7', - '173.194.33.8']) - >>> socket.gethostbyname_ex('crisewing.com') - ('crisewing.com', [], ['108.59.11.99']) - >>> socket.gethostbyname_ex('www.rad.washington.edu') - ('elladan.rad.washington.edu', # <- canonical hostname - ['www.rad.washington.edu'], # <- any machine aliases - ['128.95.247.84']) # <- all active IP addresses - - -Sockets in Python ------------------ - -To create a socket, you use the **socket** method of the ``socket`` library. -It takes up to three optional positional arguments (here we use none to get -the default behavior):: - - >>> foo = socket.socket() - >>> foo - - - -Sockets in Python ------------------ - -A socket has some properties that are immediately important to us. These -include the *family*, *type* and *protocol* of the socket:: - - >>> foo.family - 2 - >>> foo.type - 1 - >>> foo.proto - 0 - -.. class:: incremental - -You might notice that the values for these properties are integers. In fact, -these integers are **constants** defined in the socket library. - - -A quick utility method ----------------------- - -Let's define a method in place to help us see these constants. It will take a -single argument, the shared prefix for a defined set of constants: - -.. class:: small - -:: - - >>> def get_constants(prefix): - ... """mapping of socket module constants to their names.""" - ... return dict( - ... (getattr(socket, n), n) - ... for n in dir(socket) - ... if n.startswith(prefix) - ... ) - ... - >>> - -.. class:: small - -(you can also find this in ``resources/session01/session1.py``) - - -Socket Families ---------------- - -Think back a moment to our discussion of the *Internet* layer of the TCP/IP -stack. There were a couple of different types of IP addresses: - -.. class:: incremental - -* IPv4 ('192.168.1.100') - -* IPv6 ('2001:0db8:85a3:0042:0000:8a2e:0370:7334') - -.. class:: incremental - -The **family** of a socket corresponds to the *addressing system* it uses for -connecting. - - -Socket Families ---------------- - -Families defined in the ``socket`` library are prefixed by ``AF_``:: - - >>> families = get_constants('AF_') - >>> families - {0: 'AF_UNSPEC', 1: 'AF_UNIX', 2: 'AF_INET', - 11: 'AF_SNA', 12: 'AF_DECnet', 16: 'AF_APPLETALK', - 17: 'AF_ROUTE', 23: 'AF_IPX', 30: 'AF_INET6'} - -.. class:: small incremental - -*Your results may vary* - -.. class:: incremental - -Of all of these, the ones we care most about are ``2`` (IPv4) and ``30`` (IPv6). - - -Unix Domain Sockets -------------------- - -When you are on a machine with an operating system that is Unix-like, you will -find another generally useful socket family: ``AF_UNIX``, or Unix Domain -Sockets. Sockets in this family: - -.. class:: incremental - -* connect processes **on the same machine** - -* are generally a bit slower than IPC connnections - -* have the benefit of allowing the same API for programs that might run on one - machine __or__ across the network - -* use an 'address' that looks like a pathname ('/tmp/foo.sock') - - -Test your skills ----------------- - -What is the *default* family for the socket we created just a moment ago? - -.. class:: incremental - -(remember we bound the socket to the symbol ``foo``) - -.. class:: incremental center - -How did you figure this out? - - -Socket Types ------------- - -The socket *type* determines the semantics of socket communications. - -Look up socket type constants with the ``SOCK_`` prefix:: - - >>> types = get_constants('SOCK_') - >>> types - {1: 'SOCK_STREAM', 2: 'SOCK_DGRAM', - ...} - -.. class:: incremental - -The most common are ``1`` (Stream communication (TCP)) and ``2`` (Datagram -communication (UDP)). - - -Test your skills ----------------- - -What is the *default* type for our generic socket, ``foo``? - - -Socket Protocols ----------------- - -A socket also has a designated *protocol*. The constants for these are -prefixed by ``IPPROTO_``:: - - >>> protocols = get_constants('IPPROTO_') - >>> protocols - {0: 'IPPROTO_IP', 1: 'IPPROTO_ICMP', - ..., - 255: 'IPPROTO_RAW'} - -.. class:: incremental - -The choice of which protocol to use for a socket is determined by the -*internet layer* protocol you intend to use. ``TCP``? ``UDP``? ``ICMP``? -``IGMP``? - - -Test your skills ----------------- - -What is the *default* protocol used by our generic socket, ``foo``? - - -Custom Sockets --------------- - -These three properties of a socket correspond to the three positional -arguments you may pass to the socket constructor. - -.. container:: incremental - - Using them allows you to create sockets with specific communications - profiles:: - - >>> bar = socket.socket(socket.AF_INET, - ... socket.SOCK_DGRAM, - ... socket.IPPROTO_UDP) - ... - >>> bar - - - -Break Time ----------- - -So far we have: - -.. class:: incremental - -* learned about the "layers" of the TCP/IP Stack -* discussed *families*, *types* and *protocols* in sockets -* learned how to create sockets with a specific communications profile. - -.. class:: incremental - -When we return we'll learn how to find the communcations profiles of remote -sockets, how to connect to them, and how to send and receive messages. - -.. class:: incremental - -Take a few minutes now to clear your head (do not quit your python -interpreter). - - -Address Information -------------------- - -When you are creating a socket to communicate with a remote service, the -remote socket will have a specific communications profile. - -.. class:: incremental - -The local socket you create must match that communications profile. - -.. class:: incremental - -How can you determine the *correct* values to use? - -.. class:: incremental center - -You ask. - - -Address Information -------------------- - -The function ``socket.getaddrinfo`` provides information about available -connections on a given host. - -.. code-block:: python - :class: small - - socket.getaddrinfo('127.0.0.1', 80) - -.. class:: incremental - -This provides all you need to make a proper connection to a socket on a remote -host. The value returned is a tuple of: - -.. class:: incremental - -* socket family -* socket type -* socket protocol -* canonical name (usually empty, unless requested by flag) -* socket address (tuple of IP and Port) - - -A quick utility method ----------------------- - -Again, let's create a utility method in-place so we can see this in action: - -.. class:: small - -:: - - >>> def get_address_info(host, port): - ... for response in socket.getaddrinfo(host, port): - ... fam, typ, pro, nam, add = response - ... print 'family: ', families[fam] - ... print 'type: ', types[typ] - ... print 'protocol: ', protocols[pro] - ... print 'canonical name: ', nam - ... print 'socket address: ', add - ... print - ... - >>> - -.. class:: small - -(you can also find this in ``resources/session01/session1.py``) - - -On Your Own Machine -------------------- - -Now, ask your own machine what possible connections are available for 'http':: - - >>> get_address_info(socket.gethostname(), 'http') - family: AF_INET - type: SOCK_DGRAM - protocol: IPPROTO_UDP - canonical name: - socket address: ('10.211.55.2', 80) - - family: AF_INET - ... - >>> - -.. class:: incremental - -What answers do you get? - - -On the Internet ---------------- - -:: - - >>> get_address_info('crisewing.com', 'http') - family: AF_INET - type: SOCK_DGRAM - ... - - family: AF_INET - type: SOCK_STREAM - ... - >>> - -.. class:: incremental - -Try a few other servers you know about. - - -First Steps ------------ - -.. class:: big-centered - -Let's put this to use - - -Construct a Socket ------------------- - -We've already made a socket ``foo`` using the generic constructor without any -arguments. We can make a better one now by using real address information from -a real server online [**do not type this yet**]: - -.. class:: small - -:: - - >>> streams = [info - ... for info in socket.getaddrinfo('crisewing.com', 'http') - ... if info[1] == socket.SOCK_STREAM] - >>> streams - [(2, 1, 6, '', ('108.59.11.99', 80))] - >>> info = streams[0] - >>> cewing_socket = socket.socket(*info[:3]) - - -Connecting a Socket -------------------- - -Once the socket is constructed with the appropriate *family*, *type* and -*protocol*, we can connect it to the address of our remote server:: - - >>> cewing_socket.connect(info[-1]) - >>> - -.. class:: incremental - -* a successful connection returns ``None`` - -* a failed connection raises an error - -* you can use the *type* of error returned to tell why the connection failed. - - -Sending a Message ------------------ - -Send a message to the server on the other end of our connection (we'll -learn in session 2 about the message we are sending):: - - >>> msg = "GET / HTTP/1.1\r\n" - >>> msg += "Host: crisewing.com\r\n\r\n" - >>> cewing_socket.sendall(msg) - >>> - -.. class:: incremental small - -* the transmission continues until all data is sent or an error occurs - -* success returns ``None`` - -* failure to send raises an error - -* you can use the type of error to figure out why the transmission failed - -* if an error occurs you **cannot** know how much, if any, of your data was - sent - - -Receiving a Reply ------------------ - -Whatever reply we get is received by the socket we created. We can read it -back out (again, **do not type this yet**):: - - >>> response = cewing_socket.recv(4096) - >>> response - 'HTTP/1.1 200 OK\r\nDate: Thu, 03 Jan 2013 05:56:53 - ... - -.. class:: incremental small - -* The sole required argument is ``buffer_size`` (an integer). It should be a - power of 2 and smallish (~4096) -* It returns a byte string of ``buffer_size`` (or smaller if less data was - received) -* If the response is longer than ``buffer size``, you can call the method - repeatedly. The last bunch will be less than ``buffer size``. - - -Cleaning Up ------------ - -When you are finished with a connection, you should always close it:: - - >>> cewing_socket.close() - - -Putting it all together ------------------------ - -First, connect and send a message: - -.. class:: small - -:: - - >>> streams = [info - ... for info in socket.getaddrinfo('crisewing.com', 'http') - ... if info[1] == socket.SOCK_STREAM] - >>> info = streams[0] - >>> cewing_socket = socket.socket(*info[:3]) - >>> cewing_socket.connect(info[-1]) - >>> msg = "GET / HTTP/1.1\r\n" - >>> msg += "Host: crisewing.com\r\n\r\n" - >>> cewing_socket.sendall(msg) - - -Putting it all together ------------------------ - -Then, receive a reply, iterating until it is complete: - -:: - - >>> buffsize = 4096 - >>> response = '' - >>> done = False - >>> while not done: - ... msg_part = cewing_socket.recv(buffsize) - ... if len(msg_part) < buffsize: - ... done = True - ... cewing_socket.close() - ... response += msg_part - ... - >>> len(response) - 19427 - - -Server Side ------------ - -.. class:: big-centered - -What about the other half of the equation? - -Construct a Socket ------------------- - -**For the moment, stop typing this into your interpreter.** - -.. container:: incremental - - Again, we begin by constructing a socket. Since we are actually the server - this time, we get to choose family, type and protocol:: - - >>> server_socket = socket.socket( - ... socket.AF_INET, - ... socket.SOCK_STREAM, - ... socket.IPPROTO_TCP) - ... - >>> server_socket - - - -Bind the Socket ---------------- - -Our server socket needs to be bound to an address. This is the IP Address and -Port to which clients must connect:: - - >>> address = ('127.0.0.1', 50000) - >>> server_socket.bind(address) - -.. class:: incremental - -**Terminology Note**: In a server/client relationship, the server *binds* to -an address and port. The client *connects* - - -Listen for Connections ----------------------- - -Once our socket is bound to an address, we can listen for attempted -connections:: - - >>> server_socket.listen(1) - -.. class:: incremental - -* The argument to ``listen`` is the *backlog* - -* The *backlog* is the **maximum** number of connection requests that the - socket will queue - -* Once the limit is reached, the socket refuses new connections. - - -Accept Incoming Messages ------------------------- - -When a socket is listening, it can receive incoming connection requests:: - - >>> connection, client_address = server_socket.accept() - ... # this blocks until a client connects - >>> connection.recv(16) - -.. class:: incremental - -* The ``connection`` returned by a call to ``accept`` is a **new socket**. - This new socket is used to communicate with the client - -* The ``client_address`` is a two-tuple of IP Address and Port for the client - socket - -* When a connection request is 'accepted', it is removed from the backlog - queue. - - -Send a Reply ------------- - -The same socket that received a message from the client may be used to return -a reply:: - - >>> connection.sendall("message received") - - -Clean Up --------- - -Once a transaction between the client and server is complete, the -``connection`` socket should be closed:: - - >>> connection.close() - -.. class:: incremental - -Note that the ``server_socket`` is *never* closed as long as the server -continues to run. - - -Getting the Flow ----------------- - -The flow of this interaction can be a bit confusing. Let's see it in action -step-by-step. - -.. class:: incremental - -Open a second python interpreter and place it next to your first so you can -see both of them at the same time. - - -Create a Server ---------------- - -In your first python interpreter, create a server socket and prepare it for -connections:: - - >>> server_socket = socket.socket( - ... socket.AF_INET, - ... socket.SOCK_STREAM, - ... socket.IPPROTO_IP) - >>> server_socket.bind(('127.0.0.1', 50000)) - >>> server_socket.listen(1) - >>> conn, addr = server_socket.accept() - -.. class:: incremental - -At this point, you should **not** get back a prompt. The server socket is -waiting for a connection to be made. - - -Create a Client ---------------- - -In your second interpreter, create a client socket and prepare to send a -message:: - - >>> import socket - >>> client_socket = socket.socket( - ... socket.AF_INET, - ... socket.SOCK_STREAM, - ... socket.IPPROTO_IP) - -.. container:: incremental - - Before connecting, keep your eye on the server interpreter:: - - >>> client_socket.connect(('127.0.0.1', 50000)) - - -Send a Message Client->Server ------------------------------ - -As soon as you made the connection above, you should have seen the prompt -return in your server interpreter. The ``accept`` method finally returned a -new connection socket. - -.. class:: incremental - -When you're ready, type the following in the *client* interpreter. - -.. class:: incremental - -:: - - >>> client_socket.sendall("Hey, can you hear me?") - - -Receive and Respond -------------------- - -Back in your server interpreter, go ahead and receive the message from your -client:: - - >>> conn.recv(32) - 'Hey, can you hear me?' - -Send a message back, and then close up your connection:: - - >>> conn.sendall("Yes, I hear you.") - >>> conn.close() - - -Finish Up ---------- - -Back in your client interpreter, take a look at the response to your message, -then be sure to close your client socket too:: - - >>> client_socket.recv(32) - 'Yes, I hear you.' - >>> client_socket.close() - -And now that we're done, we can close up the server too (back in the server -interpreter):: - - >>> server_socket.close() - - -Congratulations! ----------------- - -.. class:: big-centered - -You've run your first client-server interaction - - -Homework --------- - -Your homework assignment for this week is to take what you've learned here -and build a simple "echo" server. - -.. class:: incremental - -The server should automatically return to any client that connects *exactly* -what it receives (it should **echo** all messages). - -.. class:: incremental - -You will also write a python script that, when run, will send a message to the -server and receive the reply, printing it to ``stdout``. - -.. class:: incremental - -Finally, you'll do all of this so that it can be tested. - - -What You Have -------------- - -In our class repository, there is a folder ``assignments/session01``. - -.. class:: incremental - -Inside that folder, you should find: - -.. class:: incremental - -* A file ``tasks.txt`` that contains these instructions - -* A skeleton for your server in ``echo_server.py`` - -* A skeleton for your client script in ``echo_client.py`` - -* Some simple tests in ``tests.py`` - -.. class:: incremental - -Your task is to make the tests pass. - - -Running the tests ------------------ - -To run the tests, you'll have to set the server running in one terminal: - -.. class:: small - -:: - - $ python echo_server.py - -.. container:: incremental - - Then, in a second terminal, you will execute the tests: - - .. class:: small - - :: - - $ python tests.py - -.. container:: incremental - - You should see output like this: - - .. class:: small - - :: - - [...] - FAILED (failures=2) - - -Submitting Your Homework ------------------------- - -To submit your homework: - -.. class:: incremental - -* In github, make a fork of my repository into *your* account. - -* Clone your fork of my repository to your computer. - -* Do your work in the ``assignments/session01/`` folder on your computer and - commit your changes to your fork. - -* When you are finished and your tests are passing, you will open a pull - request in github.com from your fork to mine. - -.. class:: incremental - -I will review your work when I receive your pull requests, make comments on it -there, and then close the pull request. - - -Going Further -------------- - -In ``assignments/session01/tasks.txt`` you'll find a few extra problems to try. - -.. class:: incremental - -If you finish the first part of the homework in less than 3-4 hours give one -or more of these a whirl. - -.. class:: incremental - -They are not required, but if you include solutions in your pull request, I'll -review your work. diff --git a/source/presentations/session05.rst b/source/presentations/session05.rst new file mode 100644 index 00000000..4e153be2 --- /dev/null +++ b/source/presentations/session05.rst @@ -0,0 +1,1523 @@ +.. slideconf:: + :autoslides: True + +********** +Session 05 +********** + +.. image:: /_static/python.png + :align: center + :width: 43% + + + +MVC Applications +================ + +Wherin we learn about the Model View Controller approach to app design and +explore data persistence in Python. + +.. figure:: http://upload.wikimedia.org/wikipedia/commons/4/40/MVC_passive_view.png + :align: center + :width: 40% + + By Alan Evangelista (Own work) [CC0], via Wikimedia Commons + +Separation of Concerns +---------------------- + +.. rst-class:: build +.. container:: + + In the first part of this course, you were introduced to the concept of + *Object Oriented Programming* + + OOP was `first formalized`_ in the 1970s in *Smalltalk*, invented by Alan + Kay at *Xerox PARC* + + *Smalltalk* was also the first language which utilized the + `Model View Controller`_ design pattern. + + This pattern (like all `design patterns`_) seeks to provide a *way of + thinking* that helps to make software design easier. + + In this case, the goal is to help clarify the high-level *separation of + concerns* in a system. + +.. _first formalized: http://en.wikipedia.org/wiki/Object-oriented_programming#History +.. _Model View Controller: http://en.wikipedia.org/wiki/Model–view–controller +.. _design patterns: http://en.wikipedia.org/wiki/Software_design_pattern + +Three Components +---------------- + +The pattern divides the elements of a system into three parts: + +.. rst-class:: build + +Model: + This component represents the *data* that comprises the system, and the + *logic* used to manipulate that data. + +View: + This component can be any *representation* of the data to the outside world: + a chart, diagram, table, user interface, etc. + + It also includes representations of the *actions* available in the system. + +Controller: + This component coordinates the Model and the View in a system. + + It accepts input from a user and channels that input into the Model. + + It accepts information about the current state of the Model and transmits + that information to the View. + +On the Web +---------- + +This pattern has proven useful for thinking about the applications we build for +the web. + +.. rst-class:: build +.. container:: + + A web browser provides a convenient container for *views* of data. + + These *views* are created by *controller* software hosted on a server. + + This *controller* software accepts input from users via *HTTP requests*, + channeling it into a *data model*, often stored in some database. + + The *controller* returns information about the state of the *data model* to + the user via *HTTP responses* + +.. nextslide:: + +This approach is so common, that it has been formalized into any number of *web +frameworks* + +.. rst-class:: build +.. container:: + + *Web frameworks* abstract away the specifics of the *HTTP request/response + cycle*, leaving simple MVC components for the developer to use. + + *Web frameworks* exist in nearly all modern languages. + + Python has scores of them. + + Over the weeks to come, we'll learn about two of them, `Pyramid`_ and + `Django`_. + +.. _Pyramid: http://www.pylonsproject.org/projects/pyramid/about +.. _Django: https://www.djangoproject.com/ + +A Word About Terminology +------------------------ + +Although the MVC pattern is a useful abstraction, there are a few differences +in how things are named in Python web frameworks + +.. rst-class:: build centered +.. container:: + + model <--> model + + controller <--> view + + view <--> template (or even HTTP response) + + .. rst-class:: left + + For more on this difference, you can `read this`_ from the Pyramid design + documentation. + +.. _read this: http://docs.pylonsproject.org/projects/pyramid/en/latest/designdefense.html#pyramid-gets-its-terminology-wrong-mvc + +Our First Application +===================== + +.. rst-class:: left + +But enough abstract blabbering. + +.. rst-class:: build left +.. container:: + + There's no better way to make concepts like these concrete than to build + something using them. + + Let's make an application! + + We're going to build a Learning Journal. + + When we're done, you'll have a live, online application you can use to keep + note of the things you are learning about Python development. + + We'll use one of our Python web framework to do this: `Pyramid`_ + +Pyramid +------- + +First published in 2010, `Pyramid`_ is a powerful, flexible web framework. + +.. rst-class:: build +.. container:: + + You can create compelling one-page applications, much like in + microframeworks like Flask + + You can also create powerful, scalable applications using the full + power of Python + + Created by the combined powers of the teams behind Pylons and Zope + + It represents the first true second-generation web framework in + existence. + +Starting the Project +-------------------- + +The first step is to prepare for the project. + +.. rst-class:: build +.. container:: + + Begin by creating a location where you'll do your work. + + I generally put all my work in a folder called ``projects`` in my home + directory: + + .. code-block:: bash + + $ cd + $ mkdir projects + $ cd projects + $ mkdir learning-journal + $ cd learning-journal + $ pwd + /Users/cewing/project/learning-journal + +.. nextslide:: Creating an Environment + +We continue our preparations by creating the virtual environment we will use +for our project. + +.. rst-class:: build +.. container:: + + Again, this will help us to keep our work here isolated from anything else + we do. + + Remember how to make a new venv? + + .. code-block:: bash + + $ pyvenv ljenv + + .. code-block:: posh + + c:\Temp>python -m venv myenv + + And then, how to activate it? + + .. code-block:: bash + + $ source ljenv/bin/activate + (ljenv)$ + + .. code-block:: posh + + C:> ljenv/Scripts/activate.bat + +.. nextslide:: Installing Pyramid + +Next, we install the Pyramid web framework into our new virtualenv. + +.. rst-class:: build +.. container:: + + We can do this with the ``pip`` in our active ``ljenv``: + + .. code-block:: bash + + (ljenv)$ pip install pyramid + Collecting pyramid + Downloading pyramid-1.5.2-py2.py3-none-any.whl (545kB) + 100% |################################| 548kB 172kB/s + ... + Successfully installed PasteDeploy-1.5.2 WebOb-1.4 + pyramid-1.5.2 repoze.lru-0.6 translationstring-1.3 + venusian-1.0 zope.deprecation-4.1.1 zope.interface-4.1.2 + + Once that is complete, we are ready to create a *scaffold* for our project. + +Working with Pyramid +-------------------- + +Many web frameworks require at least a bit of *boilerplate* code to get +started. + +.. rst-class:: build +.. container:: + + Pyramid does not. + + However, our application will require a database and handling that does + require some. + + Pyramid provides a system for creating boilerplate called ``pcreate``. + + You use it to generate the skeleton for a project based on some pattern: + + .. code-block:: bash + + (ljenv)$ pcreate -s alchemy learning_journal + Creating directory /Users/cewing/projects/learning-journal/learning_journal + ... + Welcome to Pyramid. Sorry for the convenience. + =============================================================================== + + Let's take a quick look at what that did + +.. nextslide:: What You Get + +.. code-block:: bash + + ... + ├── development.ini + ├── learning_journal + │   ├── __init__.py + │   ├── models.py + │   ├── scripts + │   │   ├── __init__.py + │   │   └── initializedb.py + │   ├── static + ... + │   ├── templates + │   │   └── mytemplate.pt + │   ├── tests.py + │   └── views.py + ├── production.ini + └── setup.py + +.. nextslide:: Saving Your Work + +You've now created something worth saving. + +.. rst-class:: build +.. container:: + + Start by initializing a new git repository in the `learning_journal` folder + you just created: + + .. code-block:: bash + + (ljenv)$ cd learning_journal + (ljenv)$ git init + Initialized empty Git repository in + /Users/cewing/projects/learning-journal/learning_journal/.git/ + +.. nextslide:: Saving Your Work + +Check ``git status`` to see where things stand: + +.. code-block:: bash + + (ljenv)$ git status + On branch master + + Initial commit + + Untracked files: + (use "git add ..." to include in what will be committed) + + CHANGES.txt + MANIFEST.in + README.txt + development.ini + learning_journal/ + production.ini + setup.py + +.. nextslide:: Add the Project Code + +Add your work to this new repository: + +.. code-block:: bash + + (ljenv)$ git add . + (ljenv)$ git status + ... + Changes to be committed: + (use "git rm --cached ..." to unstage) + + new file: CHANGES.txt + new file: MANIFEST.in + ... + new file: production.ini + new file: setup.py + +.. nextslide:: Ignore Irrelevant Files + +Python creates ``.pyc`` files when it executes your code. + +.. rst-class:: build +.. container:: + + There are many other files you don't want or need in your repository + + You can ignore this in ``git`` with the ``.gitignore`` file. + + Create one now, in this same directory, and add the following basic lines:: + + *.pyc + .DS_Store + + Finally, add this new file to your repository, too. + + .. code-block:: bash + + (ljenv)$ git add .gitignore + +.. nextslide:: Make It Permanent + +To preserve all these changes, you'll need to commit what you've done: + +.. code-block:: bash + + (ljenv)$ git commit -m "initial commit of the Pyramid learning journal" + +.. rst-class:: build +.. container:: + + This will make a first commit here in this local repository. + + For homework, you'll put this into GitHub, but this is enough for now. + + Let's move on to learning about what we've built so far. + +.. nextslide:: Project Structure + +When you ran the ``pcreate`` command, a new folder was created: +``learning_journal``. + +.. rst-class:: build +.. container:: + + This folder contains your *project*. + + At the top level, you have *configuration* (.ini files) + + You also have a file called ``setup.py`` + + This file turns this collection of Python code and configuration into an + *installable Python distribution* + + Let's take a moment to look over the code in that file + +.. nextslide:: ``setup.py`` + +.. code-block:: python + + from setuptools import setup, find_packages + ... + requires = [ + 'pyramid', + ... # packages on which this software depends (dependencies) + ] + setup(name='learning_journal', + version='0.0', + ... # package metadata (used by PyPI) + install_requires=requires, + # Entry points are ways that we can run our code once installed + entry_points="""\ + [paste.app_factory] + main = learning_journal:main + [console_scripts] + initialize_learning_journal_db = learning_journal.scripts.initializedb:main + """, + ) + +Pyramid is Python +----------------- + +In the ``__init__.py`` file of your app *package*, you'll find a ``main`` +function: + +.. code-block:: python + + def main(global_config, **settings): + """ This function returns a Pyramid WSGI application. + """ + engine = engine_from_config(settings, 'sqlalchemy.') + DBSession.configure(bind=engine) + Base.metadata.bind = engine + config = Configurator(settings=settings) + config.include('pyramid_chameleon') + config.add_static_view('static', 'static', cache_max_age=3600) + config.add_route('home', '/') + config.scan() + return config.make_wsgi_app() + +Let's take a closer look at this, line by line. + +.. nextslide:: System Configuration + +.. code-block:: python + + def main(global_config, **settings): + +Configuration is passed in to an application after being read from the +``.ini`` file we saw above. + +.. rst-class:: build +.. container:: + + These files contain sections (``[app:main]``) containing ``name = value`` + pairs of *configuration data* + + This data is parsed with the Python + `ConfigParser `_ module. + + The result is a dict of values: + + .. code-block:: python + + {'app:main': {'pyramid.reload_templates': True, ...}, ...} + + The default section of the file is passed in as ``global_config``, the + section for *this app* as ``settings``. + +.. nextslide:: Database Configuration + +.. code-block:: python + + from sqlalchemy import engine_from_config + from .models import DBSession, Base + ... + engine = engine_from_config(settings, 'sqlalchemy.') + DBSession.configure(bind=engine) + Base.metadata.bind = engine + +We will use a package called ``SQLAlchemy`` to interact with our database. + +.. rst-class:: build +.. container:: + + Our connection is set up using settings read from the ``.ini`` file. + + Can you find the settings for the database? + + The ``DBSession`` ensures that each *database transaction* is tied to HTTP + requests. + + The ``Base`` provides a parent class that will hook our *models* to the + database. + +.. nextslide:: App Configuration + +.. code-block:: python + + config = Configurator(settings=settings) + config.include('pyramid_chameleon') + config.add_static_view('static', 'static', cache_max_age=3600) + config.add_route('home', '/') + config.scan() + +Pyramid controlls application-level configuration using a ``Configurator`` class. + +.. rst-class:: build +.. container:: + + It uses app-specific settings passed in from the ``.ini`` file + + We can also ``include`` configuration from other add-on packages + + Additionally, we can configure *routes* and *views* needed to connect our + application to the outside world here (more on this next week). + + Finally, the ``Configurator`` instance performs a ``scan`` to ensure there + are no problems with what we've created. + +.. nextslide:: A Last Word on Configuration + +We will return to the configuration of our application repeatedly over the next +sessions. + +.. rst-class:: build +.. container:: + + Pyramid configuration is powerful and flexible. + + We'll use a few of its features + + But there's a lot more you could (and should) learn. + + Read about it in the `configuration chapter`_ of the Pyramid documentation. + +.. _configuration chapter: http://docs.pylonsproject.org/projects/pyramid/en/latest/api/config.html + +.. nextslide:: Break Time + +Let's take a moment to rest up and absorb what we've learned. + +When we return, we'll see how we can create *models* that will embody the data +for our Learning Journal application. + +.. rst-class:: centered + +**Pyramid Models** + + +Models in Pyramid +================= + +.. rst-class:: left +.. container:: + + The central component of MVC, the model, captures the behavior of the + application in terms of its problem domain, independent of the user + interface. The model directly manages the data, logic and rules of the + application + + -- from the Wikipedia article on `Model-view-controller`_ + +.. _Model-view-controller: http://en.wikipedia.org/wiki/Model–view–controller + +Models and ORMs +--------------- + +In an MVC application, we define the *problem domain* by creating one or more +*Models*. + +.. rst-class:: build +.. container:: + + These capture relevant details about the information we want to preserve + and how we want to interact with it. + + In Python-based MVC applications, these *Models* are implemented as Python + classes. + + The individual bits of data we want to know about are *attributes* of our + classes. + + The actions we want to take using that data are *methods* of our classes. + + Together, we can refer to this as the *API* of our system. + +.. nextslide:: Persistence + +It's all well and good to have a set of Python classes that represent your +system. + +.. rst-class:: build +.. container:: + + But what happens when you want to *save* information. + + What happens to a instance of a Python class when you quit the interprer? + + When your script stops running? + + The code in a website runs when an HTTP request comes in from a client. + + It stops running when an HTTP response goes back out to the client. + + So what happens to the data in your system in-between these moments? + + The data must be *persisted* + +.. nextslide:: Alternatives + +In the last class from part one of this series, you explored a number of +alternatives for persistence + +.. rst-class:: build + +* Python Literals +* Pickle/Shelf +* Interchange Files (CSV, XML, INI) +* Object Stores (ZODB, Durus) +* NoSQL Databases (MongoDB, CouchDB) +* SQL Databases (sqlite, MySQL, PostgreSQL, Oracle, SQLServer) + +.. rst-class:: build +.. container:: + + Any of these might be useful for certain types of applications. + + On the web, you tend to see two used the most: + + .. rst-class:: build + + * NoSQL + * SQL + +.. nextslide:: Choosing One + +How do you choose one over the other? + +.. rst-class:: build +.. container:: + + In general, the telling factor is going to be how you intend to use your + data. + + In systems where the dominant feature is viewing/interacting with + individual objects, a NoSQL storage solution might be the best way to go. + + In systems with objects that are related to eachother, SQL-based Relational + Databases are a better choice. + + Our system is more like this latter type (trust me on that one for now). + + We'll be using SQL (sqlite to start with). + + +.. nextslide:: Objects and Tables + +So we have a system where our data is captured in Python *objects* + +.. rst-class:: build +.. container:: + + And a storage system where our data must be rendered as database *tables* + + Python provides a specification for interacting directly with databases: + `dbapi2`_ + + And there are multiple Python packages that implement this specification + for various databases: + + .. rst-class:: build + + * sqlite3 + * python-mysql + * psycopg2 + * ... + + With these, you can write SQL to save your Python objects into your + database. + +.. _dbapi2: https://www.python.org/dev/peps/pep-0249/ + +.. nextslide:: ORMs + +But that's a pain. + +.. rst-class:: build +.. container:: + + SQL, while not impossible, is yet another language to learn. + + And there is a viable alternative in using an *Object Relational Manager* + (ORM) + + An ORM provides a layer of *abstraction* between you and SQL + + You instantiate Python objects and set attributes on them + + The ORM handles converting data from these objects into SQL statements (and + back) + +SQLAlchemy +---------- + +In our project we will be using the `SQLAlchemy`_ ORM. + +.. rst-class:: build +.. container:: + + You can find SQLAlchemy among the packages in ``requires`` in ``setup.py`` + in our new ``learning_journal`` package. + + However, we don't yet have that code installed. + + To do so, we will need to "install" our own package + + Make sure your ``ljenv`` virtualenv is active and then type the following: + + .. code-block:: bash + + (ljenv)$ python setup.py develop + running develop + running egg_info + creating learning_journal.egg-info + ... + Finished processing dependencies for learning-journal==0.0 + +.. nextslide:: + +Once that is complete, all the *dependencies* listed in our ``setup.py`` will +be installed. + +.. rst-class:: build +.. container:: + + You can also install the package using ``python setup.py install`` + + But using ``develop`` allows us to continue developing our package without + needing to re-install it every time we change something. + + It is very similar to using the ``-e`` option to ``pip`` + + Now, we'll only need to re-run this command if we change ``setup.py`` + itself. + +.. nextslide:: + +We also need to adjust our ``.gitignore`` file: + +.. rst-class:: build +.. code-block:: bash + + (ljenv)$ git status + ... + Untracked files: + (use "git add ..." to include in what will be committed) + + learning_journal.egg-info/ + +.. rst-class:: build +.. container:: + + The ``egg-info`` directory that was just created is an artifact of + installing a Python egg. + + It should never be committed to a repository. + + Let's add ``*.egg-info`` to our ``.gitignore`` file and then commit that + change + + Remember how? + +.. nextslide:: Our First Model + +Our project skeleton contains up a first, basic model created for us: + +.. code-block:: python + + # in models.py + Base = declarative_base() + + class MyModel(Base): + __tablename__ = 'models' + id = Column(Integer, primary_key=True) + name = Column(Text) + value = Column(Integer) + Index('my_index', MyModel.name, unique=True, mysql_length=255) + +.. _SQLAlchemy: http://docs.sqlalchemy.org/en/rel_0_9/ + +.. rst-class:: build +.. container:: + + Our class inherits from ``Base`` + + We ran into ``Base`` earlier when discussing configuration. + + We were binding it to the database we wanted to use (the ``engine``) + +.. nextslide:: ``Base`` + +Any class we create that inherits from this ``Base`` becomes a *model* + +.. rst-class:: build +.. container:: + + It will be connected through the ORM to a table in our database. + + The name of the table is determined by the ``__tablename__`` special + attribute. + + Other aspects of table configuration can also be controlled through special + attributes + + Instances of the class, once saved, will become rows in the table. + + Attributes of the model that are instances of ``Column`` will become + columns in the table. + + You can learn much more in the `Declarative`_ chapter of the SQLAlchemy docs + +.. _Declarative: http://docs.sqlalchemy.org/en/rel_0_9/orm/extensions/declarative/ + +.. nextslide:: Columns + +Each attribute of your model that will be persisted must be an instance of +`Column`_. + +.. rst-class:: build +.. container:: + + Each instance requires *at least* a specific `data type`_ (such as + Integer). + + Additionally, you can control other aspects of the column such as it being + a primary key. + + In the *declarative* style we are using, the name of the column in the + database will default to the attribute name you assigned. + + If you wish, you may provide a name specifically. It must be the first + argument and must be a string. + +.. _Column: http://docs.sqlalchemy.org/en/rel_0_9/core/metadata.html#sqlalchemy.schema.Column +.. _data type: http://docs.sqlalchemy.org/en/rel_0_9/core/types.html + +Creating The Database +--------------------- + +We have a *model* which allows us to persist Python objects to an SQL database. + +.. rst-class:: build +.. container:: + + But we're still missing one ingredient here. + + We need to create our database, or there will be nowhere for our data to + go. + + Luckily, our ``pcreate`` scaffold also gave us a convenient way to handle + this: + + .. code-block:: python + + # in setup.py + entry_points="""\ + [paste.app_factory] + main = learning_journal:main + [console_scripts] + initialize_learning_journal_db = learning_journal.scripts.initializedb:main + """, + + The ``console_script`` set up as an entry point will help us. + +.. nextslide:: Initializing the Database + +Let's look at that code for a moment. + +.. code-block:: python + + # in scripts/intitalizedb.py + from ..models import DBSession, MyModel, Base + # ... + def main(argv=sys.argv): + if len(argv) < 2: + usage(argv) + config_uri = argv[1] + options = parse_vars(argv[2:]) + setup_logging(config_uri) + settings = get_appsettings(config_uri, options=options) + engine = engine_from_config(settings, 'sqlalchemy.') + DBSession.configure(bind=engine) + Base.metadata.create_all(engine) + with transaction.manager: + model = MyModel(name='one', value=1) + DBSession.add(model) + +.. nextslide:: Console Scripts + +By connecting this function as a ``console script``, our Python package makes +this command available to us when we install it. + +.. rst-class:: build +.. container:: + + When we exectute the script at the command line, we will be running this + function. + + But before we try it out, let's update the name we use so we don't have to + type that whole big mess. + + In ``setup.py`` change ``initialize_learning_journal_db`` to ``setup_db``: + + .. code-block:: python + + entry_points="""\ + [paste.app_factory] + main = learning_journal:main + [console_scripts] + setup_db = learning_journal.scripts.initializedb:main + """, + + Then, as you have changed ``setup.py``, re-install your package: + + .. code-block:: bash + + (ljenv)$ python setup.py develop + ... + +.. nextslide:: Running the Script + +Now that the script has been renamed, let's try it out. + +.. rst-class:: build +.. container:: + + We'll need to provide a configuration file name, let's use + ``development.ini``: + + .. code-block:: bash + + (ljenv)$ setup_db development.ini + 2015-01-05 18:59:55,426 INFO [sqlalchemy.engine.base.Engine][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 + ... + 2015-01-05 18:59:55,434 INFO [sqlalchemy.engine.base.Engine][MainThread] COMMIT + + The ``[loggers]`` configuration in our ``.ini`` file sends a stream of + INFO-level logging to sys.stdout as the console script runs. + +.. nextslide:: A Bit More Cleanup + +So what was the outcome of running that script? + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + (ljenv)$ ls + ... + learning_journal.sqlite + ... + + We've now created an sqlite database. + + You'll need to add ``*.sqlite`` to ``.gitignore`` so you don't + inadvertently add that file to your repository. + + Once you've done so, commit the change to your repository + +Interacting with SQLA Models +---------------------------- + +It's pretty easy to play with your models from in an interpreter. + +.. rst-class:: build +.. container:: + + But before we do so, let's make a nicer interpreter available for our + project + + You've been using iPython in class, we can use it here too. + + Just install it with ``pip``: + + .. code-block:: bash + + (ljenv)$ pip install ipython pyramid_ipython + + Once that finishes, you'll be able to use iPython as your interpreter for + this project. + + And ``Pyramid`` provides a way to connect your interpreter to the + application code you are writing: + + The ``pshell`` command + +.. nextslide:: The ``pshell`` command + +Let's fire up ``pshell`` and explore for a moment to see what we have at our +disposal: + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + (ljenv)$ pshell development.ini + Python 3.5.0 (default, Sep 16 2015, 10:42:55) + Type "copyright", "credits" or "license" for more information. + + IPython 4.0.1 -- An enhanced Interactive Python. + ? -> Introduction and overview of IPython's features. + %quickref -> Quick reference. + help -> Python's own help system. + object? -> Details about 'object', use 'object??' for extra details. + + Environment: + app The WSGI application. + registry Active Pyramid registry. + request Active request object. + root Root of the default resource tree. + root_factory Default root factory used to create `root`. + +.. nextslide:: + +The ``environment`` created by ``pshell`` provides us with a few useful tools. + +.. code-block:: bash + + app The WSGI application. + registry Active Pyramid registry. + request Active request object. + root Root of the default resource tree. + root_factory Default root factory used to create `root`. + +.. rst-class:: build + +* The ``app`` is our new learning journal application +* The ``registry`` provides us with access to settings and other useful + information +* The ``request`` is an artificial HTTP request we can use if we need to + pretend we are listening to clients +* ... + +.. nextslide:: + +Let's use this environment to build a database session and interact with our +data: + +.. code-block:: ipython + + In [1]: from sqlalchemy import engine_from_config + In [2]: engine = engine_from_config(registry.settings, 'sqlalchemy.') + In [3]: from sqlalchemy.orm import sessionmaker + In [4]: Session = sessionmaker(bind=engine) + In [5]: session = Session() + In [6]: from learning_journal.models import MyModel + In [7]: session.query(MyModel).all() + ... + 2015-12-21 18:06:05,179 INFO [sqlalchemy.engine.base.Engine][MainThread] SELECT models.id AS models_id, models.name AS models_name, models.value AS models_value + FROM models + 2015-12-21 18:06:05,179 INFO [sqlalchemy.engine.base.Engine][MainThread] () + Out[7]: [] + +We've stolen a lot of this from the ``initializedb.py`` script + +.. nextslide:: Basic Interactions + +Any interaction with the database requires a ``session``. + +.. rst-class:: build +.. container:: + + This object represents the connection to the database. + + All database queries are phrased as methods of the session. + + .. container:: + + .. code-block:: ipython + + In [8]: query = session.query(MyModel) + In [9]: type(query) + Out[9]: sqlalchemy.orm.query.Query + + The ``query`` method of the session object returns a ``Query`` object + + Arguments to the ``query`` method can be a *model* class or *columns* from + a model class. + +.. nextslide:: Queries are Iterators + +You can iterate over a query object. The result depends on the args you passed. + +.. rst-class:: build +.. container:: + + .. code-block:: ipython + + In [10]: q1 = session.query(MyModel) + In [11]: for row in q1: + ....: print(row) + ....: print(type(row)) + ....: + + + +.. nextslide:: Queries are Iterators + +You can iterate over a query object. The result depends on the args you passed. + + .. code-block:: ipython + + In [12]: q2 = session.query(MyModel.name, MyModel.id, MyModel.value) + In [13]: for name, id, val in q2: + ....: print(name) + ....: print(type(name)) + ....: print(id) + ....: print(type(id)) + ....: print(val) + ....: print(type(val)) + ....: + one + + 1 + + 1 + + +.. nextslide:: Queries have SQL + +You can view the SQL that your query will use: + +.. rst-class:: build +.. container:: + + .. code-block:: ipython + + In [14]: str(q1) + Out[14]: 'SELECT models.id AS models_id, models.name AS models_name, models.value AS models_value \nFROM models' + + In [15]: str(q2) + Out[15]: 'SELECT models.name AS models_name, models.id AS models_id, models.value AS models_value \nFROM models' + + You can use this to check that the query the ORM is constructing looks like + you expect. + + It can be helpful in debugging. + +.. nextslide:: Methods of the Query Object + +The methods of the ``Query`` object fall into two rough categories + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + 1. Methods that return a new ``Query`` object + 2. Methods that return *scalar* values or *model* instances + + Let's start by looking quickly at a few methods from the second category + +.. nextslide:: ``query.get()`` + +A good example of this category of methods is ``get``, which returns one +instance only. + +.. rst-class:: build +.. container:: + + It takes a primary key as an argument: + + .. code-block:: ipython + + In [16]: session.query(MyModel).get(1) + Out[16]: + In [17]: session.query(MyModel).get(10) + In [18]: + + + If no item with that primary key is present, then the method returns + ``None`` + +.. nextslide:: ``query.all()`` + +Another example is one we've already seen. + +.. rst-class:: build +.. container:: + + ``query.all()`` returns a list of all rows returned by the database: + + .. code-block:: ipython + + In [18]: q1.all() + Out[18]: [] + + In [19]: type(q1.all()) + Out[19]: list + + ``query.count()`` returns the number of rows that would have been returned + by the query: + + .. code-block:: ipython + + In [20]: q1.count() + Out[20]: 1 + +.. nextslide:: Creating New Objects + +Before getting into the other category, let's learn how to create new objects. + +.. rst-class:: build +.. container:: + + .. container:: + + We can create new instances of our *model* just like normal Python + objects: + + .. code-block:: ipython + + In [21]: new_model = MyModel(name='fred', value=3) + In [22]: new_model + Out[22]: + + .. container:: + + In this state, the instance is *ephemeral*, our ``session`` knows + nothing about it: + + .. code-block:: pycon + + In [23]: session.new + Out[23]: IdentitySet([]) + +.. nextslide:: Adding Objects to the Session + +For the database to know about our new object, we must ``add`` it to the +session: + +.. rst-class:: build +.. container:: + + .. code-block:: ipython + + In [24]: session.add(new_model) + In [25]: session.new + Out[25]: IdentitySet([]) + + We can even bulk-add new objects: + + .. code-block:: ipython + + In [26]: new = [] + In [27]: for name, val in [('bob', 34), ('tom', 13)]: + ....: new.append(MyModel(name=name, value=val)) + ....: + In [28]: session.add_all(new) + In [29]: session.new + Out[29]: IdentitySet([, + , + ]) + +.. nextslide:: Committing Changes + +Up until now, the changes you've made are not permanent. + +.. rst-class:: build +.. container:: + + In order for these new objects to be saved to the database, the session + must be ``committed``: + + .. code-block:: ipython + + In [30]: other_session = Session() + In [31]: other_session.query(MyModel).count() + Out[31]: 1 + In [32]: session.commit() + In [33]: other_session.query(MyModel).count() + Out[33]: 4 + + When you are using a ``scoped_session`` in Pyramid, this action is + automatically handled for you. + + The session that is bound to a particular HTTP request is committed when a + response is sent back. + + (don't worry if this seems confusing, more to come next week) + +.. nextslide:: Altering Objects + +You can edit objects that are already part of a session, or that are fetched by +a query. + +.. rst-class:: build +.. container:: + + Simply change the values of a persisted attribute, the session will know + it's been updated: + + .. code-block:: ipython + + In [34]: new_model + Out[34]: + In [35]: new_model.name + Out[35]: 'fred' + In [36]: new_model.name = 'larry' + In [37]: session.dirty + Out[37]: IdentitySet([]) + + Commit the session to persist the changes: + + .. code-block:: ipython + + In [38]: session.commit() + In [39]: [model.name for model in other_session.query(MyModel)] + Out[39]: ['one', 'larry', 'bob', 'tom'] + +.. nextslide:: Methods Returning Queries + +Returning to query methods, a good example of the second type is the ``filter`` +method. + +.. rst-class:: build +.. container:: + + This method allows you to reduce the number of results, based on criteria: + + .. code-block:: ipython + + In [40]: [(o.name, o.value) for o in session.query(MyModel).filter(MyModel.value < 20)] + Out[40]: [('one', 1), ('larry', 3), ('tom', 13)] + +.. nextslide:: ``order_by`` + +Another typical method in this category is ``order_by``: + +.. rst-class:: build +.. container:: + + .. code-block:: ipython + + In [41]: [o.value for o in session.query(MyModel).order_by(MyModel.value)] + Out[41]: [1, 3, 13, 34] + + In [42]: [o.name for o in session.query(MyModel).order_by(MyModel.name)] + Out[42]: ['bob', 'larry', 'one', 'tom'] + +.. nextslide:: Method Chaining + +Since methods in this category return ``Query`` objects, they can be safely +*chained* to build more complex queries: + +.. rst-class:: build +.. container:: + + .. code-block:: ipython + + In [43]: q1 = session.query(MyModel).filter(MyModel.value < 20) + In [44]: q1 = q1.order_by(MyModel.name) + In [45]: [(o.name, o.value) for o in q1] + Out[45]: [('larry', 3), ('one', 1), ('tom', 13)] + + Note that you can do this inline as well + (``s.query(Model).filter().order_by()``) + + Also note that when using chained queries like this, no query is actually + sent to the database until you require a result. + + +Cleaning Up After Ourselves +--------------------------- + +When you are experimenting with a new system, you often create data that is +messy or incomplete. + +.. rst-class:: build +.. container:: + + It's good to remember that none of the information we've persisted to our + database is vital to us. + + For homework this week we'll be making new models, and the data we have in + our current database will only get in the way. + + Until you have real production data it is always safe simply to delete the + database and start over: + + .. code-block:: bash + + $ rm learning_journal.sqlite + + You can always re-create it by executing ``setup_db`` + +Homework +======== + +.. rst-class:: left + +Okay, that's enough for the moment. + +.. rst-class:: build left +.. container:: + + You've learned quite a bit about how *models* work in SQLAlchemy + + It's time to put that knowledge to good use. + + For the first part of your assignment this week you will begin to define + the data model for our learning journal application. + + I'll provide a specification, you define the model required to do the job. + + I'll also ask you to define a few methods to complete the first part of our + API. + +The Model +--------- + +Our model will be called an ``Entry``. Here's what you need to know: + +* It should be stored in a database table called ``entries`` +* It should have a primary key field called ``id`` +* It should have a ``title`` field which accepts unicode text up to 255 characters in length +* The ``title`` should be unique and it should be impossible to save an + ``entry`` without a ``title``. +* It should have a ``body`` field which accepts unicode text of any length + (including none) +* It should have a ``created`` field which stores the date and time the object + was created. +* It should have an ``edited`` field which stores the date and time the object + was last edited. + +.. nextslide:: + +* Both the ``created`` and ``edited`` field should default to ``now`` if not + provided when a new instance is constructed. +* The ``entry`` class should support a classmethod ``all`` that returns all the + entries in the database, ordered so that the most recent entry is first. +* The ``entry`` class should support a classmethod ``by_id`` that returns a + single entry, given an ``id``. + +Remember that in order to have your new model table created, you will have to +re-run the ``initialize_learning_journal_db`` script after creating your model. + +.. nextslide:: Words of Advice + +Use the documentation linked in this presentation to assist you. SQLAlchemy +has fantastic documentation, but it can be a bit overwhelming. Everything you +require for this assignment is on one or more of the pages linked above. + +As you define this new model for our application, make frequent commits to your +github repository. Remember to write meaningful commit messages. + +Don't be afraid to start up a Python interpreter and play with your model. Try +things out. Learn how this all works by making mistakes. Remember the +``pshell`` command and how we set up a session once the shell is running. + +Errors at the SQL level can sometimes leave your session unusable. To restore +it, use the ``session.rollback()`` method. You'll lose uncommitted changes, +but you'll gain a session that can be used again. + +.. nextslide:: Submitting Your Work + +I want to be able to review your code (and you want to be able to share it). + +To submit this assignment, you'll need to add this learning_journal repository +to GitHub. + +On the GitHub website you can create a new repository. Set it up to be +completely empty. Name it ``learning_journal`` and give it any description you +like. + +When you've created an empty repository in GitHub, you should see a set of +directions for connecting it to a repository that you've already built. Follow +those instructions to connect your emtpy GitHub repository as the ``origin`` +remote to your ``learning_journal`` repository on your machine. + +Finally, push your ``master`` branch to your new ``origin`` remote on GitHub. + +When you are done, send me an email with the URL for your new repository. + +.. nextslide:: + +**Our work next week will assume that you have completed this assignment** + +Do not delay working on this until the last moment. + +Do not skip this assignment. + +Do ask questions frequently via email (use the `class google group`_). + +See you next week! + +.. _class google group: https://groups.google.com/forum/#!forum/programming-in-python diff --git a/source/presentations/session05.rst.norender b/source/presentations/session05.rst.norender deleted file mode 100644 index efb1775d..00000000 --- a/source/presentations/session05.rst.norender +++ /dev/null @@ -1,1653 +0,0 @@ -Python Web Programming -====================== - -.. image:: img/bike.jpg - :align: left - :width: 50% - -Session 5: Frameworks and Flask - -.. class:: intro-blurb right - -| "Reinventing the wheel is great -| if your goal is to learn more about the wheel" -| -| -- James Tauber, PyCon 2007 - -.. class:: image-credit - -image: Britanglishman http://www.flickr.com/photos/britanglishman/5999131365/ - CC-BY - - -A Moment to Reflect -------------------- - -We've been at this for a couple of days now. We've learned a great deal: - -.. class:: incremental - -* Sockets, the TCP/IP Stack and Basic Mechanics -* Web Protocols and the Importance of Clear Communication -* APIs and Consuming Data from The Web -* CGI and WSGI and Getting Information to Your Dynamic Applications - -.. class:: incremental - -Everything we do from here out will be based on tools built using these -*foundational technologies*. - - -From Now On ------------ - -Think of everything we do as sitting on top of WSGI - -.. class:: incremental - -This may not *actually* be true - -.. class:: incremental - -But we will always be working at that level of abstraction. - - -Frameworks ----------- - -From Wikipedia: - -.. class:: center incremental - -A web application framework (WAF) is a software framework that is designed to -support the development of dynamic websites, web applications and web -services. The framework aims to alleviate the overhead associated with common -activities performed in Web development. For example, many frameworks provide -libraries for database access, templating frameworks and session management, -and they often promote code reuse - - -What Does That *Mean*? ----------------------- - -You use a framework to build an application. - -.. class:: incremental - -A framework allows you to build different kinds of applications. - -.. class:: incremental - -A framework abstracts what needs to be abstracted, and allows control of the -rest. - -.. class:: incremental - -Think back over the last four sessions. What were your pain points? Which bits -do you wish you didn't have to think about? - - -Level of Abstraction --------------------- - -This last part is important when it comes to choosing a framework - -.. class:: incremental - -* abstraction ∝ 1/freedom -* The more they choose, the less you can -* *Every* framework makes choices in what to abstract -* *Every* framework makes *different* choices - - -Impedance Mismatch ------------------- - -.. class:: big-centered - -Don't Fight the Framework - - -Python Web Frameworks ---------------------- - -There are scores of 'em (this is a partial list). - -.. class:: incremental invisible small center - -========= ======== ======== ========== ============== -Django Grok Pylons TurboGears web2py -Zope CubicWeb Enamel Gizmo(QP) Glashammer -Karrigell Nagare notmm Porcupine QP -SkunkWeb Spyce Tipfy Tornado WebCore -web.py Webware Werkzeug WHIFF XPRESS -AppWsgi Bobo Bo7le CherryPy circuits.web -Paste PyWebLib WebStack Albatross Aquarium -Divmod Nevow Flask JOTWeb2 Python Servlet -Engine Pyramid Quixote Spiked weblayer -========= ======== ======== ========== ============== - - -Choosing a Framework --------------------- - -Many folks will tell you " is the **best** framework". - -.. class:: incremental - -In most cases, what they really mean is "I know how to use " - -.. class:: incremental - -In some cases, what they really mean is " fits my brain the best" - -.. class:: incremental - -What they usually forget is that everyone's brain (and everyone's use-case) is -different. - - -Cris' First Law of Frameworks ------------------------------ - -.. class:: center - -**Pick the Right Tool for the Job** - -.. class:: incremental - -First Corollary - -.. class:: incremental center - -The right tool is the tool that allows you to finish the job quickly and -correctly. - -.. class:: incremental center - -But how do you know which that one is? - - -Cris' Second Law of Frameworks ------------------------------- - -.. class:: big-centered - -You can't know unless you try - -.. class:: incremental center - -so let's try - - -From Your Homework ------------------- - -During the week, you walked through an introduction to the *Flask* web -framework. You wrote a file that looked like this: - -.. code-block:: python - :class: small - - from flask import Flask - app = Flask(__name__) - - @app.route('/') - def hello_world(): - return 'Hello World!' - - if __name__ == '__main__': - app.run() - - -The outcome ------------ - -When you ran this file, you should have seen something like this in your -browser: - -.. image:: img/flask_hello.png - :align: center - :width: 80% - - -What's Happening Here? ----------------------- - -Flask the framework provides a Python class called `Flask`. This class -functions as a single *application* in the WSGI sense. - -.. class:: incremental - -We know a WSGI application must be a *callable* that takes the arguments -*environ* and *start_response*. - -.. class:: incremental - -It has to call the *start_response* method, providing status and headers. - -.. class:: incremental - -And it has to return an *iterable* that represents the HTTP response body. - - -Under the Covers ----------------- - -In Python, an object is a *callable* if it has a ``__call__`` method. - -.. container:: incremental - - Here's the ``__call__`` method of the ``Flask`` class: - - .. code-block:: python - - def __call__(self, environ, start_response): - """Shortcut for :attr:`wsgi_app`.""" - return self.wsgi_app(environ, start_response) - -.. class:: incremental - -As you can see, it calls another method, called ``wsgi_app``. Let's follow -this down... - - -Flask.wsgi_app --------------- - -.. code-block:: python - :class: small - - def wsgi_app(self, environ, start_response): - """The actual WSGI application. - ... - """ - ctx = self.request_context(environ) - ctx.push() - error = None - try: - try: - response = self.full_dispatch_request() - except Exception as e: - error = e - response = self.make_response(self.handle_exception(e)) - return response(environ, start_response) - #... - -.. class:: incremental - -``response`` is another WSGI app. ``Flask`` is actually *middleware* - - -Abstraction Layers ------------------- - -Finally, way down in a package called *werkzeug*, we find this response object -and it's ``__call__`` method: - -.. code-block:: python - :class: small - - def __call__(self, environ, start_response): - """Process this response as WSGI application. - - :param environ: the WSGI environment. - :param start_response: the response callable provided by the WSGI - server. - :return: an application iterator - """ - app_iter, status, headers = self.get_wsgi_response(environ) - start_response(status, headers) - return app_iter - - -Common Threads --------------- - -All Python web frameworks that operate under the WSGI spec will do this same -sort of thing. - -.. class:: incremental - -They have to do it. - -.. class:: incremental - -And these layers of abstraction allow you, the developer to focus only on the -thing that really matters to you. - -.. class:: incremental - -Getting input from a request, and returning a response. - - -A Quick Reminder ----------------- - -Over the week, in addition to walking through a Flask intro you did two other -tasks: - -.. class:: incremental - -You walked through a tutorial on the Python DB API2, and learned how -to use ``sqlite3`` to store and retrieve data. - -.. class:: incremental - -You also read a bit about ``Jinja2``, the templating language Flask -uses out of the box, and ran some code to explore its abilities. - - -Moving On ---------- - -Now it is time to put all that together. - -.. class:: incremental - -We'll spend this session building a "microblog" application. - -.. class:: incremental - -Let's dive right in. - -.. class:: incremental - -Start by activating your Flask virtualenv - - -Our Database ------------- - -We need first to define what an *entry* for our microblog might look like. - -.. class:: incremental - -Let's keep it a simple as possible for now. - -.. class:: incremental - -Create a new directory ``microblog``, and open a new file in it: -``schema.sql`` - -.. code-block:: sql - :class: incremental small - - drop table if exists entries; - create table entries ( - id integer primary key autoincrement, - title string not null, - text string not null - ); - - -App Configuration ------------------ - -For any but the most trivial applications, you'll need some configuration. - -.. class:: incremental - -Flask provides a number of ways of loading configuration. We'll be using a -config file - -.. class:: incremental - -Create a new file ``microblog.cfg`` in the same directory. - -.. code-block:: python - :class: small incremental - - # application configuration for a Flask microblog - DATABASE = 'microblog.db' - - -Our App Skeleton ----------------- - -Finally, we'll need a basic app skeleton to work from. - -.. class:: incremental - -Create one more file ``microblog.py`` in the same directory, and enter the -following: - -.. code-block:: python - :class: small incremental - - from flask import Flask - - app = Flask(__name__) - - app.config.from_pyfile('microblog.cfg') - - if __name__ == '__main__': - app.run(debug=True) - - -Test Your Work --------------- - -This is enough to get us off the ground. - -.. container:: incremental - - From a terminal in the ``microblog`` directory, run the app: - - .. class:: small - - :: - - (flaskenv)$ python microblog.py - * Running on http://127.0.0.1:5000/ - * Restarting with reloader - -.. class:: incremental - -Then point your browser at http://localhost:5000/ - -.. class:: incremental - -What do you see in your browser? In the terminal? Why? - - -Creating the Database ---------------------- - -Quit the app with ``^C``. Then return to ``microblog.py`` and add the -following: - -.. code-block:: python - :class: incremental small - - # add this up at the top - import sqlite3 - - # add the rest of this below the app.config statement - def connect_db(): - return sqlite3.connect(app.config['DATABASE']) - -.. class:: incremental - -This should look familiar. What will happen? - -.. class:: incremental - -This convenience method allows us to write our very first test. - - -Tests and TDD -------------- - -.. class:: center - -**If it isn't tested, it's broken** - -.. class:: incremental - -We are going to write tests at every step of this exercise using the -``unittest`` module. - -.. class:: incremental - -In your ``microblog`` folder create a ``microblog_tests.py`` file. - -.. class:: incremental - -Open it in your editor. - - -Testing Setup -------------- - -Add the following to provide minimal test setup. - -.. code-block:: python - :class: small - - import os - import tempfile - import unittest - - import microblog - - class MicroblogTestCase(unittest.TestCase): - - def setUp(self): - db_fd = tempfile.mkstemp() - self.db_fd, microblog.app.config['DATABASE'] = db_fd - microblog.app.config['TESTING'] = True - self.client = microblog.app.test_client() - self.app = microblog.app - - -Testing Teardown ----------------- - -**Add** this method to your existing test case class to tear down after each -test: - -.. code-block:: python - - class MicroblogTestCase(unittest.TestCase): - # ... - - def tearDown(self): - os.close(self.db_fd) - os.unlink(microblog.app.config['DATABASE']) - - -Make Tests Runnable -------------------- - -Finally, we make our tests runnable by adding a ``main`` block: - -.. container:: incremental - - Add the following at the end of ``microblog_tests.py``: - - .. code-block:: python - :class: small - - if __name__ == '__main__': - unittest.main() - -.. class:: incremental - -Now, we're ready to add our first actual test.. - - -Test Database Setup -------------------- - -We'd like to test that our database is correctly initialized. The schema has -one table with three columns. Let's test that. - -.. container:: incremental - - **Add** the following method to your test class in ``microblog_tests.py``: - - .. code-block:: python - :class: small - - def test_database_setup(self): - con = microblog.connect_db() - cur = con.execute('PRAGMA table_info(entries);') - rows = cur.fetchall() - self.assertEquals(len(rows), 3) - - -Run the Tests -------------- - -We can now run our test module: - -.. class:: small - -:: - - (flaskenv)$ python microblog_tests.py - F - ====================================================================== - FAIL: test_database_setup (__main__.MicroblogTestCase) - ---------------------------------------------------------------------- - Traceback (most recent call last): - File "microblog_tests.py", line 23, in test_database_setup - self.assertEquals(len(rows) == 3) - AssertionError: 0 != 3 - - ---------------------------------------------------------------------- - Ran 1 test in 0.011s - - FAILED (failures=1) - - -Make the Test Pass ------------------- - -This is an expected failure. Why? - -.. container:: incremental - - Let's add some code to ``microblog.py`` that will actually create our - database schema: - - .. code-block:: python - :class: small - - # add this import at the top - from contextlib import closing - - # add this function after the connect_db function - def init_db(): - with closing(connect_db()) as db: - with app.open_resource('schema.sql') as f: - db.cursor().executescript(f.read()) - db.commit() - - -Initialize the DB in Tests --------------------------- - -We also need to call that function in our ``microblog_tests.py`` to set up the -database schema for each test. - -.. container:: incremental - - Add the following line at the end of that ``setUp`` method: - - .. code-block:: python - :class: small - - def setUp(self): - # ... - microblog.init_db() # <- add this at the end - -.. class:: incremental - -:: - - (flaskenv)$ python microblog_tests.py - - -Success? --------- - -.. class:: big-centered incremental - - \\o/ Wahoooo! - - -Initialize the DB IRL ---------------------- - -Our test passed, so we have confidence that ``init_db`` does what it should - -.. class:: incremental - -We'll need to have a working database for our app, so let's go ahead and do -this "in real life" - -.. class:: incremental - - (flaskenv)$ python - -.. code-block:: python - :class: incremental - - >>> import microblog - >>> microblog.init_db() - >>> ^D - - -First Break ------------ - -After you quit the interpreter, you should see ``microblog.db`` in your -directory. - -.. class:: incremental - -Let's take a few minutes here to rest and consider what we've done. - -.. class:: incremental - -When we return, we'll start writing data to our database, and reading it back -out. - - -Reading and Writing Data ------------------------- - -Before the break, we created a function that would initialize our database. - -.. class:: incremental - -It's time now to think about writing and reading data for our blog. - -.. class:: incremental - -We'll start by writing tests. - -.. class:: incremental - -But first, a word or two about the circle of life. - - -The Request/Response Cycle --------------------------- - -Every interaction in HTTP is bounded by the interchange of one request and one -response. - -.. class:: incremental - -No HTTP application can do anything until some client makes a request. - -.. class:: incremental - -And no action by an application is complete until a response has been sent -back to the client. - -.. class:: incremental - -This is the lifecycle of an http web application. - - -Managing DB Connections ------------------------ - -It makes sense to bind the lifecycle of a database connection to this same -border. - -.. class:: incremental - -Flask does not dictate that we write an application that uses a database. - -.. class:: incremental - -Because of this, managing the lifecycle of database connection so that they -are connected to the request/response cycle is up to us. - -.. class:: incremental - -Happily, Flask *does* have a way to help us. - - -Request Boundary Decorators ---------------------------- - -The Flask *app* provides decorators we can use on our database lifecycle -functions: - -.. class:: incremental - -* ``@app.before_request``: any method decorated by this will be called before - the cycle begins - -* ``@app.after_request``: any method decorated by this will be called after - the cycle is complete. If an unhandled exception occurs, these functions are - skipped. - -* ``@app.teardown_request``: any method decorated by this will be called at - the end of the cycle, *even if* an unhandled exception occurs. - - -Managing our DB ---------------- - -Consider the following functions: - -.. code-block:: python - :class: small - - def get_database_connection(): - db = connect_db() - return db - - @app.teardown_request - def teardown_request(exception): - db.close() - -.. class:: incremental - -How does the ``db`` object get from one place to the other? - - -Global Context in Flask ------------------------ - -Our flask ``app`` is only really instantiated once - -.. class:: incremental - -This means that anything we tie to it will be shared across all requests. - -.. class:: incremental - -This is what we call ``global`` context. - -.. class:: incremental - -What happens if two clients make a request at the same time? - - -Local Context in Flask ----------------------- - -Flask provides something it calls a ``local global``: "g". - -.. class:: incremental - -This is an object that *looks* global (you can import it anywhere) - -.. class:: incremental - -But in reality, it is *local* to a single request. - -.. class:: incremental - -Resources tied to this object are *not* shared among requests. Perfect for -things like a database connection. - - -Working DB Functions --------------------- - -Add the following, working methods to ``microblog.py``: - -.. code-block:: python - :class: small - - # add this import at the top: - from flask import g - - # add these function after init_db - def get_database_connection(): - db = getattr(g, 'db', None) - if db is None: - g.db = db = connect_db() - return db - - @app.teardown_request - def teardown_request(exception): - db = getattr(g, 'db', None) - if db is not None: - db.close() - - -Writing Blog Entries --------------------- - -Our microblog will have *entries*. We've set up a simple database schema to -represent them. - -.. class:: incremental - -To write an entry, what would we need to do? - -.. class:: incremental - -* Provide a title -* Provide some body text -* Write them to a row in the database - -.. class:: incremental - -Let's write a test of a function that would do that. - - -Test Writing Entries --------------------- - -The database connection is bound by a request. We'll need to mock one (in -``microblog_tests.py``) - -.. container:: incremental - - Flask provides ``app.test_request_context`` to do just that - - .. code-block:: python - :class: small - - def test_write_entry(self): - expected = ("My Title", "My Text") - with self.app.test_request_context('/'): - microblog.write_entry(*expected) - con = microblog.connect_db() - cur = con.execute("select * from entries;") - rows = cur.fetchall() - self.assertEquals(len(rows), 1) - for val in expected: - self.assertTrue(val in rows[0]) - - -Run Your Test -------------- - -.. class:: small - -:: - - (flaskenv)$ python microblog_tests.py - .E - ====================================================================== - ERROR: test_write_entry (__main__.MicroblogTestCase) - ---------------------------------------------------------------------- - Traceback (most recent call last): - File "microblog_tests.py", line 30, in test_write_entry - microblog.write_entry(*expected) - AttributeError: 'module' object has no attribute 'write_entry' - - ---------------------------------------------------------------------- - Ran 2 tests in 0.018s - - FAILED (errors=1) - -.. class:: incremental - -Great. Two tests, one passing. - - -Make It Pass ------------- - -Now we are ready to write an entry to our database. Add this function to -``microblog.py``: - -.. code-block:: python - :class: small incremental - - def write_entry(title, text): - con = get_database_connection() - con.execute('insert into entries (title, text) values (?, ?)', - [title, text]) - con.commit() - -.. class:: incremental small - -:: - - (flaskenv)$ python microblog_tests.py - .. - ---------------------------------------------------------------------- - Ran 2 tests in 0.146s - - OK - - -Reading Entries ---------------- - -We'd also like to be able to read the entries in our blog - -.. container:: incremental - - We need a method that returns all of them for a listing page - - .. class:: incremental - - * The return value should be a list of entries - * If there are none, it should return an empty list - * Each entry in the list should be a dictionary of 'title' and 'text' - -.. class:: incremental - -Let's begin by writing tests. - - -Test Reading Entries --------------------- - -In ``microblog_tests.py``: - -.. code-block:: python - :class: small - - def test_get_all_entries_empty(self): - with self.app.test_request_context('/'): - entries = microblog.get_all_entries() - self.assertEquals(len(entries), 0) - - def test_get_all_entries(self): - expected = ("My Title", "My Text") - with self.app.test_request_context('/'): - microblog.write_entry(*expected) - entries = microblog.get_all_entries() - self.assertEquals(len(entries), 1) - for entry in entries: - self.assertEquals(expected[0], entry['title']) - self.assertEquals(expected[1], entry['text']) - - -Run Your Tests --------------- - -.. class:: small - -:: - - (flaskenv)$ python microblog_tests.py - .EE. - ====================================================================== - ERROR: test_get_all_entries (__main__.MicroblogTestCase) - ---------------------------------------------------------------------- - Traceback (most recent call last): - File "microblog_tests.py", line 47, in test_get_all_entries - entries = microblog.get_all_entries() - AttributeError: 'module' object has no attribute 'get_all_entries' - - ====================================================================== - ERROR: test_get_all_entries_empty (__main__.MicroblogTestCase) - ---------------------------------------------------------------------- - Traceback (most recent call last): - File "microblog_tests.py", line 40, in test_get_all_entries_empty - entries = microblog.get_all_entries() - AttributeError: 'module' object has no attribute 'get_all_entries' - - ---------------------------------------------------------------------- - Ran 4 tests in 0.021s - - FAILED (errors=2) - -Make Them Pass --------------- - -Now we have 4 tests, and two fail. - -.. class:: incremental - -add the ``get_all_entries`` function to ``microblog.py``: - -.. code-block:: python - :class: small incremental - - def get_all_entries(): - con = get_database_connection() - cur = con.execute('SELECT title, text FROM entries ORDER BY id DESC') - return [dict(title=row[0], text=row[1]) for row in cur.fetchall()] - -.. container:: incremental - - And back in your terminal: - - .. class:: small - - :: - - (flaskenv)$ python microblog_tests.py - .... - ---------------------------------------------------------------------- - Ran 4 tests in 0.021s - - OK - - -Where We Stand --------------- - -We've moved quite a ways in implementing our microblog: - -.. class:: incremental - -* We've created code to initialize our database schema -* We've added functions to manage the lifecycle of our database connection -* We've put in place functions to write and read blog entries -* And, since it's tested, we are reasonably sure our code does what we think - it does. - -.. class:: incremental - -We're ready now to put a face on it, so we can see what we're doing! - - -Second Break ------------- - -But first, let's take a quick break to clear our heads. - - -Templates In Flask ------------------- - -We'll start with a detour into templates as they work in Flask - -.. container:: incremental - - Jinja2 templates use the concept of an *Environment* to: - - .. class:: incremental - - * Figure out where to look for templates - * Set configuration for the templating system - * Add some commonly used functionality to the template *context* - -.. class:: incremental - -Flask sets up a proper Jinja2 Environment when you instantiate your ``app``. - - -Flask Environment ------------------ - -Flask uses the value you pass to the ``app`` constructor to calculate the root -of your application on the filesystem. - -.. class:: incremental - -From that root, it expects to find templates in a directory name ``templates`` - -.. container:: incremental - - This allows you to use the ``render_template`` command from ``flask`` like - so: - - .. code-block:: python - :class: small - - from flask import render_template - page_html = render_template('hello_world.html', name="Cris") - - -Flask Context -------------- - -Keyword arguments you pass to ``render_template`` become the *context* passed -to the template for rendering. - -.. class:: incremental - -Flask will add a few things to this context. - -.. class:: incremental - -* **config**: contains the current configuration object -* **request**: contains the current request object -* **session**: any session data that might be available -* **g**: the request-local object to which global variables are bound -* **url_for**: so you can easily *reverse* urls from within your templates -* **get_flashed_messages**: a function that returns messages you flash to your - users (more on this later). - - -Setting Up Our Templates ------------------------- - -In your ``microblog`` directory, add a new ``templates`` directory - -.. container:: incremental - - In this directory create a new file ``layout.html`` - - .. code-block:: jinja - :class: small - - - - - Microblog! - - -

          My Microblog

          -
          - {% block body %}{% endblock %} -
          - - - -Template Inheritance --------------------- - -You can combine templates in a number of different ways. - -.. class:: incremental - -* you can make replaceable blocks in templates with blocks - - * ``{% block foo %}{% endblock %}`` - -* you can build on a template in a second template by extending - - * ``{% extends "layout.html" %}`` - * this *must* be the first text in the template - -* you can re-use common structure with *include*: - - * ``{% include "footer.html" %}`` - - -Displaying an Entries List --------------------------- - -Create a new file, ``show_entries.html`` in ``templates``: - -.. code-block:: jinja - :class: small - - {% extends "layout.html" %} - {% block body %} -

          Posts

          -
            - {% for entry in entries %} -
          • -

            {{ entry.title }}

            -
            - {{ entry.text|safe }} -
            -
          • - {% else %} -
          • No entries here so far
          • - {% endfor %} -
          - {% endblock %} - - -Viewing Entries ---------------- - -We just need a Python function that will: - -.. class:: incremental - -* build a list of entries -* pass the list to our template to be rendered -* return the result to a client's browser - -.. class:: incremental - -As usual, we'll start by writing tests for this new function - - -Test Viewing Entries --------------------- - -Add the following two tests to ``microblog_tests.py``: - -.. code-block:: python - :class: small - - def test_empty_listing(self): - actual = self.client.get('/').data - expected = 'No entries here so far' - self.assertTrue(expected in actual) - - def test_listing(self): - expected = ("My Title", "My Text") - with self.app.test_request_context('/'): - microblog.write_entry(*expected) - actual = self.client.get('/').data - for value in expected: - self.assertTrue(value in actual) - -.. class:: incremental - -``app.test_client()`` creates a mock http client for us. - - -Run Your Tests --------------- - -.. class:: small - -:: - - (flaskenv)$ python microblog_tests.py - .F..F. - ====================================================================== - FAIL: test_empty_listing (__main__.MicroblogTestCase) - ---------------------------------------------------------------------- - Traceback (most recent call last): - File "microblog_tests.py", line 55, in test_empty_listing - assert 'No entries here so far' in response.data - AssertionError - ====================================================================== - FAIL: test_listing (__main__.MicroblogTestCase) - ---------------------------------------------------------------------- - Traceback (most recent call last): - File "microblog_tests.py", line 63, in test_listing - assert value in response.data - AssertionError - ---------------------------------------------------------------------- - Ran 6 tests in 0.138s - - FAILED (failures=2) - - -Make Them Pass --------------- - -In ``microblog.py``: - -.. code-block:: python - :class: small - - # at the top, import - from flask import render_template - - # and after our last functions: - @app.route('/') - def show_entries(): - entries = get_all_entries() - return render_template('show_entries.html', entries=entries) - -.. class:: incremental small - -:: - - (flaskenv)$ python microblog_tests.py - ...... - ---------------------------------------------------------------------- - Ran 6 tests in 0.100s - - OK - - -Creating Entries ----------------- - -We still lack a way to add an entry. We need a view that will: - -.. class:: incremental - -* Accept incoming form data from a request -* Get the data for ``title`` and ``text`` -* Create a new entry in the database -* Throw an appropriate HTTP error if that fails -* Show the user the list of entries when done. - -.. class:: incremental - -Again, first come the tests. - - -Testing Add an Entry --------------------- - -Add this to ``microblog_tests.py``: - -.. code-block:: python - :class: small - - def test_add_entries(self): - actual = self.client.post('/add', data=dict( - title='Hello', - text='This is a post' - ), follow_redirects=True).data - self.assertFalse('No entries here so far' in actual) - self.assertTrue('Hello' in actual) - self.assertTrue('This is a post' in actual) - - -Run Your Tests --------------- - -Verify that our test fails as expected: - -.. class:: small - -:: - - (flaskenv)$ python microblog_tests.py - F...... - ====================================================================== - FAIL: test_add_entries (__main__.MicroblogTestCase) - ---------------------------------------------------------------------- - Traceback (most recent call last): - File "microblog_tests.py", line 72, in test_add_entries - self.assertTrue('Hello' in actual) - AssertionError: False is not true - - ---------------------------------------------------------------------- - Ran 7 tests in 0.050s - - FAILED (failures=1) - - -Make Them Pass --------------- - -We have all we need to write entries, all we lack is an endpoint (in -``microblog.py``): - -.. code-block:: python - :class: small - - # add imports - from flask import abort - from flask import request - from flask import url_for - from flask import redirect - - @app.route('/add', methods=['POST']) - def add_entry(): - try: - write_entry(request.form['title'], request.form['text']) - except sqlite3.Error: - abort(500) - return redirect(url_for('show_entries')) - - -And...? -------- - -.. class:: small - -:: - - (flaskenv)$ python microblog_tests.py - ....... - ---------------------------------------------------------------------- - Ran 7 tests in 0.047s - - OK - -.. class:: incremental center - -**Hooray!** - - -Where do Entries Come From --------------------------- - -Finally, we're almost done. We can add entries and view them. But look at that -last view. Do you see a call to ``render_template`` in there at all? - -.. class:: incremental - -There isn't one. That's because that view is never meant to be be visible. -Look carefully at the logic. What happens? - -.. class:: incremental - -So where do the form values come from? - -.. class:: incremental - -Let's add a form to the main view. Open ``show_entries.html`` - - -Provide a Form --------------- - -.. code-block:: jinja - :class: small - - {% block body %} -
          -
          - - -
          -
          - - -
          -
          - -
          -
          -

          Posts

          - - -All Done --------- - -Okay. That's it. We've got an app all written. - -.. class:: incremental - -So far, we haven't actually touched our browsers at all, but we have -reasonable certainty that this works because of our tests. Let's try it. - - -.. class:: incremental - -In the terminal where you've been running tests, run our microblog app: - -.. class:: incremental - -:: - - (flaskenv)$ python microblog.py - * Running on http://127.0.0.1:5000/ - * Restarting with reloader - - -The Big Payoff --------------- - -Now load ``http://localhost:5000/`` in your browser and enjoy your reward. - - -Making It Pretty ----------------- - -What we've got here is pretty ugly. - -.. class:: incremental - -If you've fallen behind, or want to start fresh, you can find the finished -``microblog`` directory in the class resources. - -.. class:: incremental - -In that directory inside the ``static`` directory you will find -``styles.css``. Open it in your editor. It contains basic CSS for this app. - -.. class:: incremental - -We'll need to include this file in our ``layout.html``. - - -Static Files ------------- - -Like page templates, Flask locates static resources like images, css and -javascript by looking for a ``static`` directory relative to the app root. - -.. class:: incremental - -You can use the special url endpoint ``static`` to build urls that point here. -Open ``layout.html`` and add the following: - -.. code-block:: jinja - :class: small incremental - - - Flaskr - - - - -Reap the Rewards ----------------- - -Make sure that your `microblog` folder has a `static` folder inside it, and -that the `styles.css` file is in it. - -.. class:: incremental - -Then, reload your web browser and see the difference a bit of style can make. - -Homework --------- - -We've built a simple microblog application in the *Flask* web framework. - -.. class:: incremental - -For your homework this week I'd like you to add two features to this app. - -.. class:: incremental - -1. Authentication -2. Flash messaging - - -Authentication Specifications ------------------------------ - -Writing new entries should be restricted to users who have logged in. This -means that: - -.. class:: incremental - -* The form to create a new entry should only be visible to logged in users -* There should be a visible link to allow a user to log in -* This link should display a login form that expects a username and password -* If the user provides incorrect login information, this form should tell her - so. -* If the user provides correct login information, she should end up at the - list page -* Once logged in, the user should see a link to log out. -* Upon clicking that link, the system should no longer show the entry form and - the log in link should re-appear. - - -Flash Messaging Specifications ------------------------------- - -A flask app provides a method called `flash` that allows passing messages from -a view function into a template context so that they can be viewed by a user. - -.. class:: incremental - -Use this method to provide the following messages to users: - -.. class:: incremental - -* Upon a successful login, display the message "You are logged in" -* Upon a successful logout, display the message "You have logged out" -* Upon posting a successful new entry, display the message "New entry posted" -* If adding an entry causes an error, instead of returning a 500 response, - alert the user to the error by displaying the error message to the user. - - -Resources to Use ----------------- - -The microblog we created today comes from the tutorial on the `flask` website. -I've modified that tutorial to omit authentication and flash messaging. You can -refer to the tutorial and to the flask api documentation to learn what you need -to accomplish these tasks. - -`The Flask Tutorial `_ - -`Flask API Documentation `_ - -Both features depend on *sessions*, so you will want to pay particular -attention to how a session is enabled and what you can do with it once it -exists. - - -Next Week ---------- - -Next week we are going to mix things up a little and do something quite -different. - -.. class:: incremental - -We'll be starting from the app you have just built (with the additional -features you complete over the week). - -.. class:: incremental - -We will divide into pairs and each pair will select one feature from a list I -will provide. - -.. class:: incremental - -We'll spend the entire class implementing this feature, and at 8:15, each pair -will show their work to the class. - - -Wrap-Up -------- - -For educational purposes you might try taking a look at the source code for -Flask and Werkzeug. Neither is too large a package. - -.. class:: incremental - -In particular seeing how Werkzeug sets up a Request and Response--and how -these relate to the WSGI specification--can be very enlightening. diff --git a/source/presentations/session06.rst b/source/presentations/session06.rst new file mode 100644 index 00000000..652e1af8 --- /dev/null +++ b/source/presentations/session06.rst @@ -0,0 +1,1588 @@ +.. slideconf:: + :autoslides: True + +********** +Session 06 +********** + +.. image:: /_static/lj_entry.png + :width: 65% + :align: center + +Interacting with Data +===================== + +**Wherein we learn to display our data, and to create and edit it too!** + + +But First +--------- + +Last week we discussed the **model** part of the *MVC* application design +pattern. + +.. rst-class:: build +.. container:: + + We set up a project using the `Pyramid`_ web framework and the `SQLAlchemy`_ + library for persisting our data to a database. + + We looked at how to define a simple model by investigating the demo model + created on our behalf. + + And we went over, briefly, the way we can interact with this model at the + command line to make sure we've got it right. + + Finally, we defined what attributes a learning journal entry would have, + and a pair of methods we think we will need to make the model complete. + +.. _Pyramid: http://www.pylonsproject.org/projects/pyramid/about +.. _SQLAlchemy: http://docs.sqlalchemy.org/en/rel_0_9/ + +Our Data Model +-------------- + +Over the last week, your assignment was to create the new model. + +.. rst-class:: build +.. container:: + + Did you get that done? + + If not, what stopped you? + + Let's take a few minutes here to answer questions about this task so you + are more comfortable. + + Questions? + +.. nextslide:: A Complete Example + +I've added a working ``models.py`` file to our `class repository`_ in the +``resources/session06/`` folder. + +Let's review how it works. + +.. _class repository: https://github.com/UWPCE-PythonCert/training.python_web/tree/master/resources/session06 + +.. nextslide:: Demo Interaction + +I've also made a few small changes to make the ``pshell`` command a bit more +helpful. + +.. rst-class:: build +.. container:: + + In ``learning_journal/__init__.py`` I added the following function: + + .. code-block:: python + + def create_session(settings): + from sqlalchemy.orm import sessionmaker + engine = engine_from_config(settings, 'sqlalchemy.') + Session = sessionmaker(bind=engine) + return Session() + + Then, in ``development.ini`` I added the following configuration: + + .. code-block:: ini + + [pshell] + create_session = learning_journal.create_session + entry = learning_journal.models.Entry + +.. nextslide:: Using the new ``pshell`` + +Here's a demo interaction using ``pshell`` with these new features: + +.. rst-class:: build +.. container:: + + First ``cd`` to your project code, fire up your project virtualenv and + start python: + + .. code-block:: bash + + $ cd projects/learning-journal/learning_journal + $ source ../ljenv/bin/activate + (ljenv)$ pshell development.ini + Python 3.5.0 (default, Sep 16 2015, 10:42:55) + ... + Environment: + app The WSGI application. + ... + Custom Variables: + create_session learning_journal.create_session + entry learning_journal.models.Entry + + In [1]: session = create_session(registry.settings) + + [demo] + +The MVC Controller +================== + +.. rst-class:: left +.. container:: + + Let's go back to thinking for a bit about the *Model-View-Controller* + pattern. + + .. figure:: http://upload.wikimedia.org/wikipedia/commons/4/40/MVC_passive_view.png + :align: center + :width: 25% + + By Alan Evangelista (Own work) [CC0], via Wikimedia Commons + + .. rst-class:: build + .. container:: + + We talked last week (and today) about the *model* + + Today, we'll dig into *controllers* and *views* + + or as we will know them in Pyramid: *views* and *renderers* + + +HTTP Request/Response +--------------------- + +Internet software is driven by the HTTP Request/Response cycle. + +.. rst-class:: build +.. container:: + + A *client* (perhaps a user with a web browser) makes a **request** + + A *server* receives and handles that request and returns a **response** + + The *client* receives the response and views it, perhaps making a new + **request** + + And around and around it goes. + +.. nextslide:: URLs + +An HTTP request arrives at a server through the magic of a **URL** + +.. code-block:: bash + + http://uwpce-pythoncert.github.io/training.python_web/html/index.html + +.. rst-class:: build +.. container:: + + Let's break that up into its constituent parts: + + .. rst-class:: build + + \http://: + This part is the *protocol*, it determines how the request will be sent + + uwpce-pythoncert.github.io: + This is a *domain name*. It's the human-facing address for a server + somewhere. + + /training.python_web/html/index.html: + This part is the *path*. It serves as a locator for a resource *on the + server* + +.. nextslide:: Paths + +In a static website (like our documentation) the *path* identifies a **physical +location** in the server's filesystem. + +.. rst-class:: build +.. container:: + + Some directory on the server is the *home* for the web process, and the + *path* is looked up there. + + Whatever resource (a file, an image, whatever) is located there is returned + to the user as a response. + + If the path leads to a location that doesn't exist, the server responds + with a **404 Not Found** error. + + In the golden days of yore, this was the only way content was served via + HTTP. + +.. nextslide:: Paths in an MVC System + +In todays world we have dynamic systems, server-side web frameworks like +Pyramid. + +.. rst-class:: build +.. container:: + + The requests that you send to a server are handled by a software process + that assembles a response instead of looking up a physical location. + + But we still have URLs, with *protocol*, *domain* and *path*. + + What is the role for a path in a process that doesn't refer to a physical + file system? + + Most web frameworks now call the *path* a **route**. + + They provide a way of matching *routes* to the code that will be run to + handle requests. + +Routes in Pyramid +----------------- + +In Pyramid, routes are handled as *configuration* and are set up in the *main* +function in ``__init__.py``: + +.. code-block:: python + + # learning_journal/__init__.py + def main(global_config, **settings): + # ... + config.add_route('home', '/') + # ... + +.. rst-class:: build +.. container:: + + Our code template created a sample route for us, using the ``add_route`` + method of the ``Configurator`` class. + + The ``add_route`` method has two required arguments: a *name* and a + *pattern* + + In our sample route, the *name* is ``'home'`` + + In our sample route, the *pattern* is ``'/'`` + +.. nextslide:: + +When a request comes in to a Pyramid application, the framework looks at all +the *routes* that have been configured. + +.. rst-class:: build +.. container:: + + One by one, in order, it tries to match the *path* of the incoming request + against the *pattern* of the route. + + As soon as a *pattern* matches the *path* from the incoming request, that + route is used and no further matching is performed. + + If no route is found that matches, then the request will automatically get + a **404 Not Found** error response. + + In our sample app, we have one sample *route* named ``'home'``, with a + pattern of ``/``. + + This means that any request that comes in for ``/`` will be matched to this + route, and any other request will be **404**. + +.. nextslide:: Routes as API + +In a very real sense, the *routes* defined in an application *are* the public +API. + +.. rst-class:: build +.. container:: + + Any route that is present represents something the user can do. + + Any route that is not present is something the user cannot do. + + You can use the proper definition of routes to help conceptualize what your + app will do. + + What routes might we want for a learning journal application? + + What will our application do? + +.. nextslide:: Defining our Routes + +Let's add routes for our application. + +.. rst-class:: build +.. container:: + + Open ``learning_journal/__init__.py``. + + For our list page, the existing ``'home'`` route will do fine, leave it. + + Add the following two routes: + + .. code-block:: python + + config.add_route('home', '/') # already there + config.add_route('detail', '/journal/{id:\d+}') + config.add_route('action', '/journal/{action}') + + The ``'detail'`` route will serve a single journal entry, identified by an + ``id``. + + The ``action`` route will serve ``create`` and ``edit`` views, depending on + the ``action`` specified. + + In both cases, we want to capture a portion of the matched path to use + information it provides. + +.. nextslide:: Matching an ID + +In a pattern, you can capture a ``path segment`` *replacement +marker*, a valid Python symbol surrounded by curly braces: + +.. rst-class:: build +.. container:: + + :: + + /home/{foo}/ + + If you want to match a particular pattern, like digits only, add a + *regular expression*:: + + /journal/{id:\d+} + + Matched path segments are captured in a ``matchdict``:: + + # pattern # actual url # matchdict + /journal/{id:\d+} /journal/27 {'id': '27'} + + The ``matchdict`` is made available as an attribute of the *request object* + + (more on that soon) + + +.. nextslide:: Connecting Routes to Views + +In Pyramid, a *route* is connected by configuration to a *view*. + +.. rst-class:: build +.. container:: + + In our app, a sample view has been created for us, in ``views.py``: + + .. code-block:: python + + @view_config(route_name='home', renderer='templates/mytemplate.pt') + def my_view(request): + # ... + + The order in which *routes* are configured *is important*, so that must be + done in ``__init__.py``. + + The order in which views are connected to routes *is not important*, so the + *declarative* ``@view_config`` decorator can be used. + + When ``config.scan`` is called, all files in our application are searched + for such *declarative configuration* and it is added. + +The Pyramid View +---------------- + +Let's imagine that a *request* has come to our application for the path +``'/'``. + +.. rst-class:: build +.. container:: + + The framework made a match of that path to a *route* with the pattern ``'/'``. + + Configuration connected that route to a *view* in our application. + + Now, the view that was connected will be *called*, which brings us to the + nature of *views* + + .. rst-class:: centered + + --A Pyramid view is a *callable* that takes *request* as an argument-- + + Remember what a *callable* is? + +.. nextslide:: What the View Does + +So, a *view* is a callable that takes the *request* as an argument. + +.. rst-class:: build +.. container:: + + It can then use information from that request to build appropriate data, + perhaps using the application's *models*. + + Then, it returns the data it assembled, passing it on to a `renderer`_. + + Which *renderer* to use is determined, again, by configuration: + + .. code-block:: python + + @view_config(route_name='home', renderer='templates/mytemplate.pt') + def my_view(request): + # ... + + More about this in a moment. + + The *view* stands at the intersection of *input data*, the application + *model* and *renderers* that offer rendering of the results. + + It is the *Controller* in our MVC application. + +.. _renderer: http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/renderers.html + + +.. nextslide:: Adding Stub Views + +Add temporary views to our application in ``views.py`` (and comment out the +sample view): + +.. code-block:: python + + @view_config(route_name='home', renderer='string') + def index_page(request): + return 'list page' + + @view_config(route_name='detail', renderer='string') + def view(request): + return 'detail page' + + @view_config(route_name='action', match_param='action=create', renderer='string') + def create(request): + return 'create page' + + @view_config(route_name='action', match_param='action=edit', renderer='string') + def update(request): + return 'edit page' + +.. nextslide:: Testing Our Views + +Now we can verify that our view configuration has worked. + +.. rst-class:: build +.. container:: + + Make sure your virtualenv is properly activated, and start the web server: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 84467. + serving on http://0.0.0.0:6543 + + Then try viewing some of the expected application urls: + + .. rst-class:: build + + * http://localhost:6543/ + * http://localhost:6543/journal/1 + * http://localhost:6543/journal/create + * http://localhost:6543/journal/edit + + What happens if you visit a URL that *isn't* in our configuration? + +.. nextslide:: Interacting With the Model + +Now that we've got temporary views that work, we can fix them to get +information from our database + +.. rst-class:: build +.. container:: + + We'll begin with the list view. + + We need some code that will fetch all the journal entries we've written, in + reverse order, and hand that collection back for rendering. + + .. code-block:: python + + from .models import ( + DBSession, + MyModel, + Entry, # <- Add this import + ) + + # and update this view function + def index_page(request): + entries = Entry.all() + return {'entries': entries} + +.. nextslide:: Using the ``matchdict`` + +Next, we want to write the view for a single entry. + +.. rst-class:: build +.. container:: + + We'll need to use the ``id`` value our route captures into the + ``matchdict``. + + Remember that the ``matchdict`` is an attribute of the request. + + We'll get the ``id`` from there, and use it to get the correct entry. + + .. code-block:: python + + # add this import at the top + from pyramid.httpexceptions import HTTPNotFound + + # and update this view function: + def view(request): + this_id = request.matchdict.get('id', -1) + entry = Entry.by_id(this_id) + if not entry: + return HTTPNotFound() + return {'entry': entry} + +.. nextslide:: Testing Our Views + +We can now verify that these views work correctly. + +.. rst-class:: build +.. container:: + + Make sure your virtualenv is properly activated, and start the web server: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 84467. + serving on http://0.0.0.0:6543 + + Then try viewing the list page and an entry page: + + * http://localhost:6543 + * http://localhost:6543/journal/1 + + What happens when you request an entry with an id that isn't in the + database? + + * http://localhost:6543/journal/100 + +The MVC View +============ + +.. rst-class:: left +.. container:: + + Again, back to the *Model-View-Controller* pattern. + + .. figure:: http://upload.wikimedia.org/wikipedia/commons/4/40/MVC_passive_view.png + :align: center + :width: 25% + + By Alan Evangelista (Own work) [CC0], via Wikimedia Commons + + .. rst-class:: build + .. container:: + + We've built a *model* and we've created some *controllers* that use it. + + In Pyramid, we call *controllers* **views** and they are callables that + take *request* as an argument. + + Let's turn to the last piece of the *MVC* patter, the *view* + +Presenting Data +--------------- + +The job of the *view* in the *MVC* pattern is to present data in a format that +is readable to the user of the system. + +.. rst-class:: build +.. container:: + + There are many ways to present data. + + Some are readable by humans (tables, charts, graphs, HTML pages, text + files). + + Some are more for machines (xml files, csv, json). + + Which of these formats is the *right one* depends on your purpose. + + What is the purpose of our learning journal? + +Pyramid Renderers +----------------- + +In Pyramid, the job of presenting data is performed by a *renderer*. + +.. rst-class:: build +.. container:: + + So we can consider the Pyramid **renderer** to be the *view* in our *MVC* + app. + + We've already seen how we can connect a *renderer* to a Pyramid *view* with + configuration. + + In fact, we have already done so, using a built-in renderer called + ``'string'``. + + This renderer converts the return value of its *view* to a string and sends + that back to the client as an HTTP response. + + But the result isn't so nice looking. + +.. nextslide:: Template Renderers + +The `built-in renderers` (``'string'``, ``'json'``, ``'jsonp'``) in Pyramid are +not the only ones available. + +.. _built-in renderers: http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/renderers.html#built-in-renderers + +.. rst-class:: build +.. container:: + + There are add-ons to Pyramid that support using various *template + languages* as renderers. + + In fact, one of these was installed by default when you created this + project. + +.. nextslide:: Configuring a Template Renderer + +.. code-block:: python + + # in setup.py + requires = [ + # ... + 'pyramid_chameleon', + # ... + ] + + # in learning_journal/__init__.py + def main(global_config, **settings): + # ... + config.include('pyramid_chameleon') + +.. rst-class:: build +.. container:: + + The `pyramid_chameleon` package supports using the `chameleon` template + language. + + The language is quite nice and powerful, but not so easy to learn. + + Let's use a different one, *jinja2* + +.. nextslide:: Changing Template Renderers + +Change ``pyramid_chameleon`` to ``pyramid_jinja2`` in both of these files: + +.. code-block:: python + + # in setup.py + requires = [ + # ... + 'pyramid_jinja2', + # ... + ] + + # in learning_journal/__init__.py + def main(global_config, **settings): + # ... + config.include('pyramid_jinja2') + +.. nextslide:: Picking up the Changes + +We've changed the dependencies for our Pyramid project. + +.. rst-class:: build +.. container:: + + As a result, we will need to re-install it so the new dependencies are also + installed: + + .. code-block:: bash + + (ljenv)$ python setup.py develop + ... + Finished processing dependencies for learning-journal==0.0 + (ljenv)$ + + Now, we can use *Jinja2* templates in our project. + + Let's learn a bit about how `Jinja2 templates`_ work. + +.. _Jinja2 templates: http://jinja.pocoo.org/docs/templates/ + +Jinja2 Template Basics +---------------------- + +We'll start with the absolute basics. + +.. rst-class:: build +.. container:: + + Fire up an iPython interpreter, using your `ljenv` virtualenv: + + .. code-block:: bash + + (ljenv)$ ipython + ... + In [1]: + + Then import the ``Template`` class from the ``jinja2`` package: + + .. code-block:: ipython + + In [1]: from jinja2 import Template + +.. nextslide:: Templates are Strings + +A template is constructed with a simple string: + +.. code-block:: ipython + + In [2]: t1 = Template("Hello {{ name }}, how are you") + +.. rst-class:: build +.. container:: + + Here, we've simply typed the string directly, but it is more common to + build a template from the contents of a *file*. + + Notice that our string has some odd stuff in it: ``{{ name }}``. + + This is called a *placeholder* and when the template is *rendered* it is + replaced. + +.. nextslide:: Rendering a Template + +Call the ``render`` method, providing *context*: + +.. code-block:: ipython + + In [3]: t1.render(name="Freddy") + Out[3]: 'Hello Freddy, how are you' + In [4]: t1.render(name="Gloria") + Out[4]: 'Hello Gloria, how are you' + +.. rst-class:: build +.. container:: + + *Context* can either be keyword arguments, or a dictionary + + Note the resemblance to something you've seen before: + + .. code-block:: python + + >>> "This is {owner}'s string".format(owner="Cris") + 'This is Cris's string' + + +.. nextslide:: Dictionaries in Context + +Dictionaries passed in as part of the *context* can be addressed with *either* +subscript or dotted notation: + +.. code-block:: ipython + + In [5]: person = {'first_name': 'Frank', + ...: 'last_name': 'Herbert'} + In [6]: t2 = Template("{{ person.last_name }}, {{ person['first_name'] }}") + In [7]: t2.render(person=person) + Out[7]: 'Herbert, Frank' + +.. rst-class:: build + +* Jinja2 will try the *correct* way first (attr for dotted, item for + subscript). +* If nothing is found, it will try the opposite. +* If nothing is found, it will return an *undefined* object. + + +.. nextslide:: Objects in Context + +The exact same is true of objects passed in as part of *context*: + +.. rst-class:: build +.. container:: + + .. code-block:: ipython + + In [8]: t3 = Template("{{ obj.x }} + {{ obj['y'] }} = Fun!") + In [9]: class Game(object): + ...: x = 'babies' + ...: y = 'bubbles' + ...: + In [10]: bathtime = Game() + In [11]: t3.render(obj=bathtime) + Out[11]: 'babies + bubbles = Fun!' + + This means your templates can be agnostic as to the nature of the + things found in *context* + +.. nextslide:: Filtering values in Templates + +You can apply `filters`_ to the data passed in *context* with the pipe ('|') +operator: + +.. _filters: http://jinja.pocoo.org/docs/dev/templates/#filters + +.. code-block:: ipython + + In [12]: t4 = Template("shouted: {{ phrase|upper }}") + In [13]: t4.render(phrase="this is very important") + Out[13]: 'shouted: THIS IS VERY IMPORTANT' + +.. rst-class:: build +.. container:: + + You can also chain filters together: + + .. code-block:: ipython + + In [14]: t5 = Template("confusing: {{ phrase|upper|reverse }}") + In [15]: t5.render(phrase="howdy doody") + Out[15]: 'confusing: YDOOD YDWOH' + +.. nextslide:: Control Flow + +Logical `control structures`_ are also available: + +.. _control structures: http://jinja.pocoo.org/docs/dev/templates/#list-of-control-structures + +.. rst-class:: build +.. container:: + + .. code-block:: ipython + + In [16]: tmpl = """ + ....: {% for item in list %}{{ item}}, {% endfor %} + ....: """ + In [17]: t6 = Template(tmpl) + In [18]: t6.render(list=['a', 'b', 'c', 'd', 'e']) + Out[18]: '\na, b, c, d, e, ' + + Any control structure introduced in a template **must** be paired with an + explicit closing tag (``{% for %}...{% endfor %}``) + + Remember, although template tags like ``{% for %}`` or ``{% if %}`` look a + lot like Python, *they are not*. + + The syntax is specific and must be followed correctly. + +.. nextslide:: Template Tests + +There are a number of specialized *tests* available for use with the +``if...elif...else`` control structure: + +.. code-block:: ipython + + In [19]: tmpl = """ + ....: {% if phrase is upper %} + ....: {{ phrase|lower }} + ....: {% elif phrase is lower %} + ....: {{ phrase|upper }} + ....: {% else %}{{ phrase }}{% endif %}""" + In [20]: t7 = Template(tmpl) + In [21]: t7.render(phrase="FOO") + Out[21]: '\n\n foo\n' + In [22]: t7.render(phrase='bar') + Out[22]: '\n\n BAR\n' + In [23]: t7.render(phrase='This should print as-is') + Out[23]: '\nThis should print as-is' + + +.. nextslide:: Basic Expressions + +Basic `Python-like expressions`_ are also supported: + +.. _Python-like expressions: http://jinja.pocoo.org/docs/dev/templates/#expressions + +.. code-block:: ipython + + In [24]: tmpl = """ + ....: {% set sum = 0 %} + ....: {% for val in values %} + ....: {{ val }}: {{ sum + val }} + ....: {% set sum = sum + val %} + ....: {% endfor %} + ....: """ + In [25]: t8 = Template(tmpl) + In [26]: t8.render(values=range(1, 11)) + Out[26]: '\n\n\n1: 1\n \n\n2: 3\n \n\n3: 6\n \n\n4: 10\n + \n\n5: 15\n \n\n6: 21\n \n\n7: 28\n \n\n8: 36\n + \n\n9: 45\n \n\n10: 55\n \n\n' + + +Our Templates +------------- + +There's more that Jinja2 templates can do, but it will be easier to introduce +you to that in the context of a working template. So let's make some. + +.. nextslide:: Detail Template + +We have a Pyramid view that returns a single entry. Let's create a template to +show it. + +.. rst-class:: build +.. container:: + + In ``learning_journal/templates`` create a new file ``detail.jinja2``: + + .. code-block:: jinja + +
          +

          {{ entry.title }}

          +
          +

          {{ entry.body }}

          +
          +

          Created {{entry.created}}

          +
          + + Then wire it up to the detail view in ``views.py``: + + .. code-block:: ipython + + # views.py + @view_config(route_name='detail', renderer='templates/detail.jinja2') + def view(request): + # ... + +.. nextslide:: Try It Out + +Now we should be able to see some rendered HTML for our journal entry details. + +.. rst-class:: build +.. container:: + + Start up your server: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 90536. + serving on http://0.0.0.0:6543 + + Then try viewing an individual journal entry + + * http://localhost:6543/journal/1 + +.. nextslide:: Listing Page + +The index page of our journal should show a list of journal entries, let's do +that next. + +.. rst-class:: build +.. container:: + + In ``learning_journal/templates`` create a new file ``list.jinja2``: + + .. code-block:: jinja + + {% if entries %} +

          Journal Entries

          + + {% else %} +

          This journal is empty

          + {% endif %} + +.. nextslide:: + +It's worth taking a look at a few specifics of this template. + +.. rst-class:: build +.. container:: + + .. code-block:: jinja + + {% for entry in entries %} + ... + {% endfor %} + + Jinja2 templates are rendered with a *context*. + + A Pyramid *view* returns a dictionary, which is passed to the renderer as + part of of that *context* + + This means we can access values we return from our *view* in the *renderer* + using the names we assigned to them. + +.. nextslide:: + +It's worth taking a look at a few specifics of this template. + + .. code-block:: jinja + + {{ entry.title }} + + The *request* object is also placed in the context by Pyramid. + + Request has a method ``route_url`` that will create a URL for a named + route. + + This allows you to include URLs in your template without needing to know + exactly what they will be. + + This process is called *reversing*, since it's a bit like a reverse phone + book lookup. + +.. nextslide:: + +Finally, you'll need to connect this new renderer to your listing view: + +.. code-block:: python + + @view_config(route_name='home', renderer='templates/list.jinja2') + def index_page(request): + # ... + +.. nextslide:: Try It Out + +We can now see our list page too. Let's try starting the server: + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 90536. + serving on http://0.0.0.0:6543 + + Then try viewing the home page of your journal: + + * http://localhost:6543/ + + Click on the link to an entry, it should work. + +.. nextslide:: Sharing Structure + +These views are reasonable, if quite plain. + +.. rst-class:: build +.. container:: + + It'd be nice to put them into something that looks a bit more like a + website. + + Jinja2 allows you to combine templates using something called + `template inheritance`_. + + You can create a basic page structure, and then *inherit* that structure in + other templates. + + In our class resources I've added a page template ``layout.jinja2``. Copy + that page to your templates directory + +.. _template inheritance: http://jinja.pocoo.org/docs/dev/templates/#template-inheritance + +.. nextslide:: ``layout.jinja2`` + +.. code-block:: jinja + + + + + + Python Learning Journal + + + +
          + +
          +
          +

          My Python Journal

          +
          {% block body %}{% endblock %}
          +
          +

          Created in the UW PCE Python Certificate Program

          + + + +.. nextslide:: Template Blocks + +The important part here is the ``{% block body %}{% endblock %}`` expression. + +.. rst-class:: build +.. container:: + + This is a template **block** and it is a kind of placeholder. + + Other templates can inherit from this one, and fill that block with + additional HTML. + + Let's update our detail and list templates: + + .. code-block:: jinja + + {% extends "layout.jinja2" %} + {% block body %} + + {% endblock %} + +.. nextslide:: Try It Out + +Let's try starting the server so we can see the result: + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 90536. + serving on http://0.0.0.0:6543 + + Then try viewing the home page of your journal: + + * http://localhost:6543/ + + Click on the link to an entry, it should work. + + And now you have shared page structure that is in both. + +Static Assets +------------- + +Although we have a shared structure, it isn't particularly nice to look at. + +.. rst-class:: build +.. container:: + + Aspects of how a website looks are controlled by CSS (*Cascading Style + Sheets*). + + Stylesheets are one of what we generally speak of as *static assets*. + + Other static assets include *images* that are part of the look and feel of + the site (logos, button images, etc) and the *JavaScript* files that add + client-side dynamic behavior to the site. + +.. nextslide:: Static Assets in Pyramid + +Serving static assets in Pyramid requires a *static view* to configuration. +Luckily, ``pcreate`` already handled that for us: + +.. rst-class:: build +.. container:: + + .. code-block:: python + + # in learning_journal/__init__.py + def main(global_config, **settings): + # ... + config.add_static_view('static', 'static', cache_max_age=3600) + # ... + + The first argument to ``add_static_view`` is a *name* that will need to + appear in the path of URLs requesting assets. + + The second argument is a *path* that is relative to the package being + configured. + + Assets referenced by the *name* in a URL will be searched for in the + location defined by the *path* + + Additional keyword arguments control other aspects of how the view works. + +.. nextslide:: Static Assets in Templates + +Once you have a static view configured, you can use assets in that location in +templates. + +.. rst-class:: build +.. container:: + + The *request* object in Pyramid provides a ``static_path`` method that + will render an appropriate asset path for us. + + Add the following to our ``layout.jinja2`` template: + + .. code-block:: jinja + + + + + + + The one required argument to ``request.static_path`` is a *path* to an + asset. + + Note that because any package *might* define a static view, we have to + specify which package we want to look in. + + That's why we have ``learning_journal:static/styles.css`` in our call. + +.. nextslide:: Basic Styles + +I've created some very very basic styles for our learning journal. + +.. rst-class:: build +.. container:: + + You can find them in ``resources/session06/styles.css``. Go ahead and copy + that file. + + Add it to ``learning_journal/static``. + + Then restart your web server and see what a difference a little style + makes: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 90536. + serving on http://0.0.0.0:6543 + +.. nextslide:: The Outcome + +Your site should look something like this: + +.. figure:: /_static/learning_journal_styled.png + :align: center + :width: 75% + + The learning journal with basic styles applied + +Getting Interactive +=================== + +.. rst-class:: left +.. container:: + + We have a site that allows us to view a list of journal entries. + + .. rst-class:: build + .. container:: + + We can also view the details of a single entry. + + But as yet, we don't really have any *interaction* in our site yet. + + We can't create new entries. + + Let's add that functionality next. + +User Input +---------- + +In HTML websites, the traditional way of getting input from users is via +`HTML forms`_. + +.. rst-class:: build +.. container:: + + Forms use *input elements* to allow users to enter data, pick from + drop-down lists, or choose items via checkbox or radio button. + + It is possible to create plain HTML forms in templates and use them with + Pyramid. + + It's a lot easier, however, to work with a *form library* to create forms, + render them in templates and interact with data sent by a client. + + We'll be using a form library called `WTForms`_ in our project + +.. _HTML forms: https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Forms +.. _WTForms: http://wtforms.readthedocs.org/en/latest/ + +.. nextslide:: Installing WTForms + +The first step to working with this library is to install it. + +.. rst-class:: build +.. container:: + + Start by makin the library as a *dependency* of our package by adding it to + the *requires* list in ``setup.py``: + + .. code-block:: python + + requires = [ + # ... + 'wtforms', # <- add this to the list + ] + + Then, re-install our package to download and install the new dependency: + + .. code-block:: bash + + (ljenv)$ python setup.py develop + ... + Finished processing dependencies for learning-journal==0.0 + +Using WTForms +------------- + +We'll want a form to allow a user to create a new Journal Entry. + +.. rst-class:: build +.. container:: + + Add a new file called ``forms.py`` in our learning_journal package, next to + ``models.py``: + + .. code-block:: python + + from wtforms import Form, TextField, TextAreaField, validators + + strip_filter = lambda x: x.strip() if x else None + + class EntryCreateForm(Form): + title = TextField( + 'Entry title', + [validators.Length(min=1, max=255)], + filters=[strip_filter]) + body = TextAreaField( + 'Entry body', + [validators.Length(min=1)], + filters=[strip_filter]) + +.. nextslide:: Using a Form in a View + +Next, we need to add a new view that uses this form to create a new entry. + +.. rst-class:: build +.. container:: + + Add this to ``views.py``: + + .. code-block:: python + + # add these imports + from pyramid.httpexceptions import HTTPFound + from .forms import EntryCreateForm + + # and update this view function + def create(request): + entry = Entry() + form = EntryCreateForm(request.POST) + if request.method == 'POST' and form.validate(): + form.populate_obj(entry) + DBSession.add(entry) + return HTTPFound(location=request.route_url('home')) + return {'form': form, 'action': request.matchdict.get('action')} + +.. nextslide:: Testing the Route/View Connection + +We already have a route that connects here. Let's test it. + +.. rst-class:: build +.. container:: + + Start your server: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 90536. + serving on http://0.0.0.0:6543 + + And then try connecting to the ``action`` route: + + * http://localhost:6543/journal/create + + You should see something like this:: + + {'action': u'create', 'form': } + +.. nextslide:: Rendering A Form + +Finally, we need to create a template that will render our form. + +.. rst-class:: build +.. container:: + + Add a new template called ``edit.jinja2`` in + ``learning_journal/templates``: + + .. code-block:: jinja + + {% extends "templates/layout.jinja2" %} + {% block body %} +
          + {% for field in form %} + {% if field.errors %} +
            + {% for error in field.errors %} +
          • {{ error }}
          • + {% endfor %} +
          + {% endif %} +

          {{ field.label }}: {{ field }}

          + {% endfor %} +

          +
          + {% endblock %} + +.. nextslide:: Connecting the Renderer + +You'll need to update the view configuration to use this new renderer. + +.. rst-class:: build +.. container:: + + Update the configuration in ``learning_journal/views.py``: + + .. code-block:: python + + @view_config(route_name='action', match_param='action=create', + renderer='templates/edit.jinja2') + def create(request): + # ... + + And then you should be able to start your server and test: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 90536. + serving on http://0.0.0.0:6543 + + * http://localhost:6543/journal/create + +.. nextslide:: Providing Access + +Great! Now you can add new entries to your journal. + +.. rst-class:: build +.. container:: + + But in order to do so, you have to hand-enter the url. + + You should add a new link in the UI somewhere that helps you get there more + easily. + + Add the following to ``list.jinja2``: + + .. code-block:: jinja + + {% extends "layout.jinja2" %} + {% block body %} + {% if entries %} + ... + {% else %} + ... + {% endif %} + +

          New Entry

          + {% endblock %} + +Homework +======== + +.. rst-class:: left +.. container:: + + You have a website now that allows you to create, view and list journal + entries + + .. rst-class:: build + .. container:: + + However, there are still a few flaws in this system. + + You should be able to edit a journal entry that already exists, in case + you make a spelling error. + + It would also be nice to see a prettier site. + + Let's handle that for homework this week. + +Part 1: Add Editing +------------------- + +For part one of your assignment, add editing of existing entries. You will need: + +* A form that shows an existing entry (what is different about this form from + one for creating a new entry?) +* A pyramid view that handles that form. It should: + + * Show the form with the requested entry when the page is first loaded + * Accept edits only on POST + * Update an existing entry with new data from the form + * Show the view of the entry after editing so that the user can see the edits + saved correctly + * Show errors from form validation, if any are present + +* A link somewhere that leads to the editing page for a single entry (probably + on the view page for a entry) + +You'll need to update a bit of configuration, but not much. Use the create +form we did here in class as an example. + +Part 2: Make it Yours +--------------------- + +I've created for you a very bare-bones layout and stylesheet. + +You will certainly want to add a bit of your own style and panache. + +Spend a few hours this week playing with the styles and getting a site that +looks more like you want it to look. + +The Mozilla Developer Network has `some excellent resources`_ for learning CSS. + +In particular, the `Getting Started with CSS`_ tutorial is a thorough +introduction to the basics. + +You might also look at their `CSS 3 Demos`_ to help fire up your creative +juices. + +Here are a few more resources: + +* `A List Apart `_ offers outstanding articles. Their + `Topics list `_ is worth a browse. +* `Smashing Magazine `_ is another excellent + resource for articles on design. + +.. _some excellent resources: https://developer.mozilla.org/en-US/docs/Web/CSS +.. _Getting Started with CSS: https://developer.mozilla.org/en-US/docs/CSS/Getting_Started +.. _CSS 3 Demos: https://developer.mozilla.org/en-US/demos/tag/tech:css3 + + +Part 3: User Model +------------------ + +As it stands, our journal accepts entries from anyone who comes by. + +Next week we will add security to allow only logged-in users to create and edit +entries. + +To do so, we'll need a user model + +The model should have: + +* An ``id`` field that is a primary key +* A ``username`` field that is unicode, no more than 255 characters, not + nullable, unique and indexed. +* A ``password`` field that is unicode and not nullable + +In addition, the model should have a classmethod that retrieves a specific user +when given a username. + +Part 4: Preparation for Deployment +---------------------------------- + +At the end of class next week we will be deploying our application to Heroku. + +You will need to get a free account. + +Once you have your free account set up and you have logged in, run through the +`getting started with Python`_ tutorial. + +Be sure to at least complete the *set up* step. It will have you install the +Heroku Toolbelt, which you will need to have ready in class. + +.. _getting started with Python: https://devcenter.heroku.com/articles/getting-started-with-python#introduction + +Submitting Your Work +-------------------- + +As usual, submit your work by committing and pushing it to your project github +repository + +Commit early and commit often. + +Write yourself good commit messages explaining what you have done and why. + +When you are ready to have your work reviewed, email the link to your +repository to us, we'll take a look and make comments. diff --git a/source/presentations/session06.rst.norender b/source/presentations/session06.rst.norender deleted file mode 100644 index 3bc546f1..00000000 --- a/source/presentations/session06.rst.norender +++ /dev/null @@ -1,179 +0,0 @@ -Python Web Programming -====================== - -.. image:: img/flask_cover.png - :align: left - :width: 50% - -Session 6: Extending Flask - -.. class:: intro-blurb right - -| "Web Development, -| one drop at a time" - -.. class:: image-credit - -image: Flask Logo (http://flask.pocoo.org/community/logos/) - - -Last Week ---------- - -Last week, we created a nice, simple flask microblog application. - -.. class:: incremental - -Over the week, as your homework, you added in authentication and flash -messaging. - -.. class:: incremental - -There's still quite a bit more we can do to improve this application. - -.. class:: incremental - -And today, that's what we are going to do. - - -Pair Programming ----------------- - -`Pair programming `_ is a -technique used in agile development. - -.. class:: incremental - -The basic idea is that two heads are better than one. - -.. class:: incremental - -A pair of developers work together at one computer. One *drives* and the other -*navigates* - -.. class:: incremental - -The driver can focus on the tactics of completing a function, while the -navigator can catch typos, think strategically, and find answers to questions -that arise. - - -Pair Up -------- - -We are going to employ this technique for todays class. - -.. class:: incremental - -So take the next few minutes to find a partner and pair up. You must end up -sitting next to your partner, so get up and move. - -.. class:: incremental - -One of you will start as the driver, the other as the observer. - -.. class:: incremental - -About every 20-30 minutes, we will switch, so that each of you can take a turn -driving. - - -Preparation ------------ - -In order for this to work properly, we'll need to have a few things in place. - -.. container:: incremental small - - First, we'll start from a canonical copy of the microblog. Make a fork of - the following repository to your github account: - - .. code-block:: - :class: small - - https://github.com/UWPCE-PythonCert/training.sample-flask-app - -.. container:: incremental small - - Then, clone that repository to your local machine: - - .. code-block:: bash - :class: small - - $ git clone https://github.com//training.sample-flask-app.git - or - $ git clone git@github.com:/training.sample-flask-app.git - -Connect to Your Partner ------------------------ - -Finally, you'll want to connect to your partner's repository, so that you can -each work on your own laptop and still share the changes you make. - -.. container:: incremental small - - First, add your partner's repository as ``upstream`` to yours: - - .. code-block:: bash - :class: small - - $ git remote add upstream https://github.com//training.sample-flask-app.git - or - $ git remote add upstream git@github.com:/training.sample-flask-app.git - -.. container:: incremental small - - Then, fetch their copy so that you can easily merge their changes later: - - .. code-block:: bash - :class: small - - $ git fetch upstream - -While You Work --------------- - -.. class:: small - -Now, when you switch roles during your work, here's the workflow you can use: - -.. class:: small - -1. The current driver commits all changes and pushes to their repository: - -.. code-block:: bash - :class: small - - $ git commit -a -m "Time to switch roles" - $ git push origin master - -.. class:: small - -2. The new driver fetches and merges changes made upstream: - -.. code-block:: bash - :class: small - - $ git fetch upstream master - $ git branch -a - * master - remotes/origin/master - remotes/upstream/master - $ git merge upstream/master - -.. class:: small - -3. The new driver continues working from where their partner left off. - - -Homework --------- - -For this week, please read and complete the Introduction to Django tutorial -linked from the class website and from the course outline. - -You will be expected to have successfully completed that tutorial upon arrival -in class for our next session. - -We will begin our work starting from where it leaves off. - diff --git a/source/presentations/session07.rst b/source/presentations/session07.rst new file mode 100644 index 00000000..787bf014 --- /dev/null +++ b/source/presentations/session07.rst @@ -0,0 +1,1889 @@ +********** +Session 07 +********** + +.. figure:: /_static/no_entry.jpg + :align: center + :width: 60% + + By `Joel Kramer via Flickr`_ + +.. _Joel Kramer via Flickr: https://www.flickr.com/photos/75001512@N00/2707796203 + +Security And Deployment +======================= + +.. rst-class:: left +.. container:: + + By the end of this session we'll have deployed our learning journal to a + public server. + + So we will need to add a bit of security to it. + + We'll get started on that in a moment + +But First +--------- + +.. rst-class:: large center + +Questions About the Homework? + +.. nextslide:: A Working Edit Form + +.. code-block:: python + + class EntryEditForm(EntryCreateForm): + id = HiddenField() + +`View the form online `_ + +.. nextslide:: A Working Edit View + +.. code-block:: python + + @view_config(route_name='action', match_param='action=edit', + renderer='templates/edit.jinja2') + def update(request): + id = int(request.params.get('id', -1)) + entry = Entry.by_id(id) + if not entry: + return HTTPNotFound() + form = EntryEditForm(request.POST, entry) + if request.method == 'POST' and form.validate(): + form.populate_obj(entry) + return HTTPFound(location=request.route_url('detail', id=entry.id)) + return {'form': form, 'action': request.matchdict.get('action')} + +`See this view online `_ + +.. nextslide:: Linking to the Edit Form + +.. code-block:: html+jinja + + {% extends "layout.jinja2" %} + {% block body %} +
          + +
          +

          + Go Back :: + + Edit Entry +

          + {% endblock %} + + +`View this template online `_ + +.. nextslide:: A Working User Model + +.. code-block:: python + + class User(Base): + __tablename__ = 'users' + id = Column(Integer, primary_key=True, autoincrement=True) + name = Column(Unicode(255), unique=True, nullable=False) + password = Column(Unicode(255), nullable=False) + + @classmethod + def by_name(cls, name): + return DBSession.query(cls).filter(cls.name == name).first() + +`View this model online `_ + +Securing An Application +======================= + +.. rst-class:: left +.. container:: + + We've got a solid start on our learning journal. + + .. rst-class:: build + .. container:: + + We can: + + .. rst-class:: build + + * view a list of entries + * view a single entry + * create a new entry + * edit existing entries + + But so can everyone who visits the journal. + + It's a recipe for **TOTAL CHAOS** + + Let's lock it down a bit. + + +AuthN and AuthZ +--------------- + +There are two aspects to the process of access control online. + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * **Authentication**: Verification of the identity of a *principal* + * **Authorization**: Enumeration of the rights of that *principal* in a + context. + + Think of them as **Who Am I** and **What Can I Do** + + All systems with access control involve both of these aspects. + + But many systems wire them together as one. + + +.. nextslide:: Pyramid Security + +In Pyramid these two aspects are handled by separate configuration settings: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * ``config.set_authentication_policy(AuthnPolicy())`` + * ``config.set_authorization_policy(AuthzPolicy())`` + + If you set one, you must set the other. + + Pyramid comes with a few policy classes included. + + You can also roll your own, so long as they fulfill the requried interface. + + You can learn about the interfaces for `authentication`_ and + `authorization`_ in the Pyramid documentation + +.. _authentication: http://docs.pylonsproject.org/projects/pyramid/en/latest/api/interfaces.html#pyramid.interfaces.IAuthenticationPolicy +.. _authorization: http://docs.pylonsproject.org/projects/pyramid/en/latest/api/interfaces.html#pyramid.interfaces.IAuthorizationPolicy + +.. nextslide:: Our Journal Security + +We'll be using two built-in policies today: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * ``AuthTktAuthenticationPolicy``: sets an expirable + `authentication ticket`_ cookie. + * ``ACLAuthorizationPolicy``: uses an `Access Control List`_ to grant + permissions to *principals* + + Our access control system will have the following properties: + + .. rst-class:: build + + * Everyone can view entries, and the list of all entries + * Users who log in may edit entries or create new ones + +.. _authentication ticket: http://docs.pylonsproject.org/docs/pyramid/en/latest/api/authentication.html#pyramid.authentication.AuthTktAuthenticationPolicy +.. _Access Control List: http://docs.pylonsproject.org/docs/pyramid/en/latest/api/authorization.html#pyramid.authorization.ACLAuthorizationPolicy + +.. nextslide:: Engaging Security + +By default, Pyramid uses no security. We enable it through configuration. + +.. rst-class:: build +.. container:: + + Open ``learning_journal/__init__.py`` and update it as follows: + + .. code-block:: python + + # add these imports + from pyramid.authentication import AuthTktAuthenticationPolicy + from pyramid.authorization import ACLAuthorizationPolicy + # and add this configuration: + def main(global_config, **settings): + # ... + # update building the configurator to pass in our policies + config = Configurator( + settings=settings, + authentication_policy=AuthTktAuthenticationPolicy('somesecret'), + authorization_policy=ACLAuthorizationPolicy(), + default_permission='view' + ) + # ... + +.. nextslide:: Verify It Worked + +We've now informed our application that we want to use security. + +.. rst-class:: build +.. container:: + + By default we require the 'view' permission to see anything. + + But we have yet to assign *any permissions to anyone* at all. + + Let's verify now that we are unable to see anything in the website. + + Start your application, and try to view any page (You should get a 403 + Forbidden error response): + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 84467. + serving on http://0.0.0.0:6543 + + .. rst-class:: build + + * http://localhost:6543/ + * http://localhost:6543/journal/1 + * http://localhost:6543/journal/create + * http://localhost:6543/journal/edit?id=1 + +Implementing Authz +------------------ + +Next we have to grant some permissions to principals. + +.. rst-class:: build +.. container:: + + Pyramid authorization relies on a concept it calls "context". + + A *principal* can be granted rights in a particular *context* + + Context can be made as specific as a single persistent object + + Or it can be generalized to a *route* or *view* + + To have a context, we need a Python object called a *factory* that must + have an ``__acl__`` special attribute. + + The framework will use this object to determine what permissions a + *principal* has + + Let's create one + +.. nextslide:: Add ``security.py`` + +In the same folder where you have ``models.py`` and ``views.py``, add a new +file ``security.py`` + +.. rst-class:: build +.. container:: + + .. code-block:: python + + from pyramid.security import Allow, Everyone, Authenticated + + class EntryFactory(object): + __acl__ = [ + (Allow, Everyone, 'view'), + (Allow, Authenticated, 'create'), + (Allow, Authenticated, 'edit'), + ] + def __init__(self, request): + pass + + The ``__acl__`` attribute of this object contains a list of *ACE*\ s + + An *ACE* combines an *action* (Allow, Deny), a *principal* and a *permission* + +.. nextslide:: Using Our Context Factory + +Now that we have a factory that will provide context for permissions to work, +we can tell our configuration to use it. + +.. rst-class:: build +.. container:: + + Open ``learning_journal/__init__.py`` and update the route configuration + for our routes: + + .. code-block:: python + + # add an import at the top: + from .security import EntryFactory + # update the route configurations: + def main(global_config, **settings): + """ This function returns a Pyramid WSGI application. + """ + # ... Add the factory keyword argument to our route configurations: + config.add_route('home', '/', factory=EntryFactory) + config.add_route('detail', '/journal/{id:\d+}', factory=EntryFactory) + config.add_route('action', '/journal/{action}', factory=EntryFactory) + +.. nextslide:: What We've Done + +We've now told our application we want a principal to have the *view* +permission by default. + +.. rst-class:: build +.. container:: + + And we've provided a factory to supply context and an ACL for each route. + + Check our ACL. Who can view the home page? The detail page? The action + pages? + + Pyramid allows us to set a *default_permission* for *all views*\ . + + But view configuration allows us to require a different permission for *a view*\ . + + Let's make our action views require appropriate permissions next + +.. nextslide:: Requiring Permissions for a View + +Open ``learning_journal/views.py``, and edit the ``@view_config`` for +``create`` and ``update``: + +.. code-block:: python + + @view_config(route_name='action', match_param='action=create', + renderer='templates/edit.jinja2', + permission='create') # <-- ADD THIS + def create(request): + # ... + + @view_config(route_name='action', match_param='action=edit', + renderer='templates/edit.jinja2', + permission='edit') # <-- ADD THIS + def update(request): + # ... + +.. nextslide:: Verify It Worked + +At this point, our "action" views should require permissions other than the +default ``view``. + +.. rst-class:: build +.. container:: + + Start your application and verify that it is true: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 84467. + serving on http://0.0.0.0:6543 + + .. rst-class:: build + + * http://localhost:6543/ + * http://localhost:6543/journal/1 + * http://localhost:6543/journal/create + * http://localhost:6543/journal/edit?id=1 + + You should get a ``403 Forbidden`` for the action pages only. + +Implement AuthN +--------------- + +Now that we have authorization implemented, we need to add authentication. + +.. rst-class:: build +.. container:: + + By providing the system with an *authenticated user*, our ACEs for + ``Authenticated`` will apply. + + We'll need to have a way for a user to prove who they are to the + satisfaction of the system. + + The most common way of handling this is through a *username* and + *password*. + + A person provides both in an html form. + + When the form is submitted, the system seeks a user with that name, and + compares the passwords. + + If there is no such user, or the password does not match, authentication + fails. + +.. nextslide:: An Example + +Let's imagine that Alice wants to authenticate with our website. + +.. rst-class:: build +.. container:: + + Her username is ``alice`` and her password is ``s3cr3t``. + + She fills these out in a form on our website and submits the form. + + Our website looks for a ``User`` object in the database with the username + ``alice``. + + Let's imagine that there is one, so our site next compares the value she + sent for her *password* to the value stored in the database. + + If her stored password is also ``s3cr3t``, then she is who she says she is. + + All set, right? + +.. nextslide:: Encryption + +The problem here is that the value we've stored for her password is in ``plain +text``. + +.. rst-class:: build +.. container:: + + This means that anyone could potentially steal our database and have access + to all our users' passwords. + + Instead, we should *encrypt* her password with a strong one-way hash. + + Then we can store the hashed value. + + When she provides the plain text password to us, we *encrypt* it the same + way, and compare the result to the stored value. + + If they match, then we know the value she provided is the same we used to + create the stored hash. + +.. nextslide:: Adding Encryption + +Python provides a number of libraries for implementing strong encryption. + +.. rst-class:: build +.. container:: + + You should always use a well-known library for encryption. + + We'll use a good one called `Passlib`_. + + This library provides a number of different algorithms and a *context* that + implements a simple interface for each. + + .. code-block:: python + + from passlib.context import CryptContext + password_context = CryptContext(schemes=['pbkdf2_sha512']) + hashed = password_context.encrypt('password') + if password_context.verify('password', hashed): + print "It matched" + +.. _Passlib: https://pythonhosted.org/passlib/ + +.. nextslide:: Install Passlib + +To install a new package as a dependency, we add the package to our list in +``setup.py``. + +``Passlib`` provides a large number of different hashing schemes. Some (like +``bcrypt``) require underlying ``C`` extensions to be compiled. If you do not +have a ``C`` compiler, these extensions will be disabled. + +.. rst-class:: build +.. container:: + + .. code-block:: python + + requires = [ + ... + 'wtforms', + 'passlib', + ] + + Then, we re-install our package to pick up the new dependency: + + .. code-block:: bash + + (ljenv)$ python setup.py develop + + *note* if you have a c compiler installed but not the Python dev headers, + this may not work. Let me know if you get errors. + +.. nextslide:: Using Passlib + +As noted above, the passlib library uses a ``context`` object to manage +passwords. + +.. rst-class:: build +.. container:: + + This object supports a lot of functionality, but the only API we care about + for this project is encrypting and verifying passwords. + + We'll create a single, global context to be used by our project. + + Since the ``User`` class is the component in our system that should have + the responsibility for password interactions, we'll create our context in + the same place it is defined. + + In ``learning_journal/models.py`` add the following code: + + .. code-block:: python + + # add an import at the top + from passlib.context import CryptContext + + # then lower down, make a context at module scope: + password_context = CryptContext(schemes=['pbkdf2_sha512']) + + +.. nextslide:: Comparing Passwords + +Now that we have a context object available, let's write an instance method for +our ``User`` class that uses it to verify a plaintext password: + +.. rst-class:: build +.. container:: + + Again, in ``learning_journal/models.py`` add the following to the ``User`` + class: + + .. code-block:: python + + # add this method to the User class: + class User(Base): + # ... + def verify_password(self, password): + return password_context.verify(password, self.password) + +.. nextslide:: Create a User + +We'll also need to have a user for our system. + +.. rst-class:: build +.. container:: + + We can use the database initialization script to create one for us. + + Open ``learning_journal/scripts/initialzedb.py``: + + .. code-block:: python + + from learning_journal.models import password_context + from learning_journal.models import User + # and update the main function like so: + def main(argv=sys.argv): + # ... + with transaction.manager: + # replace the code to create a MyModel instance + encrypted = password_context.encrypt('admin') + admin = User(name='admin', password=encrypted) + DBSession.add(admin) + +.. nextslide:: Rebuild the Database: + +In order to get our user created, we'll need to delete our database and +re-build it. + +.. rst-class:: build +.. container:: + + Make sure you are in the folder where ``setup.py`` appears. + + Then remove the sqlite database: + + .. code-block:: bash + + (ljenv)$ rm *.sqlite + + And re-initialize: + + .. code-block:: bash + + (ljenv)$ initialize_learning_journal_db development.ini + ... + 2015-01-17 16:43:55,237 INFO [sqlalchemy.engine.base.Engine][MainThread] + INSERT INTO users (name, password) VALUES (?, ?) + 2015-01-17 16:43:55,237 INFO [sqlalchemy.engine.base.Engine][MainThread] + ('admin', '$2a$10$4Z6RVNhTE21mPLJW5VeiVe0EG57gN/HOb7V7GUwIr4n1vE.wTTTzy') + +Providing Login UI +------------------ + +We now have a user in our database with a strongly encrypted password. + +.. rst-class:: build +.. container:: + + We also have a method on our user model that will verify a supplied + password against this encrypted version. + + We must now provide a view that lets us log in to our application. + + We start by adding a new *route* to our configuration in + ``learning_journal/__init__.py``: + + .. code-block:: python + + config.add_rount('action' ...) + # ADD THIS + config.add_route('auth', '/sign/{action}', factory=EntryFactory) + +.. nextslide:: A Login Form + +It would be nice to use the form library again to make a login form. + +.. rst-class:: build +.. container:: + + Open ``learning_journal/forms.py`` and add the following: + + .. code-block:: python + + # add an import: + from wtforms import PasswordField + # and a new form class + class LoginForm(Form): + username = TextField( + 'Username', [validators.Length(min=1, max=255)] + ) + password = PasswordField( + 'Password', [validators.Length(min=1, max=255)] + ) + + +.. nextslide:: Login View in ``learning_journal/views.py`` + +.. ifnotslides:: + + Next, we'll create a login view in ``learning_journal/views.py`` + +.. code-block:: python + + # new imports: + from pyramid.security import forget, remember + from .forms import LoginForm + from .models import User + # and a new view + @view_config(route_name='auth', match_param='action=in', renderer='string', + request_method='POST') + def sign_in(request): + login_form = None + if request.method == 'POST': + login_form = LoginForm(request.POST) + if login_form and login_form.validate(): + user = User.by_name(login_form.username.data) + if user and user.verify_password(login_form.password.data): + headers = remember(request, user.name) + else: + headers = forget(request) + else: + headers = forget(request) + return HTTPFound(location=request.route_url('home'), headers=headers) + +.. nextslide:: Where's the Renderer? + +Notice that this view doesn't render anything. No matter what, you end up +returning to the ``home`` route. + +.. rst-class:: build +.. container:: + + We have to incorporate our login form somewhere. + + The home page seems like a good place. + + But we don't want to show it all the time. + + Only when we aren't logged in already. + + Let's give that a whirl. + +.. nextslide:: Updating ``index_page`` + +Pyramid security provides a method that returns the id of the user who is +logged in, if any. + +.. rst-class:: build +.. container:: + + We can use that to update our home page in ``learning_journal/views.py``: + + .. code-block:: python + + # add an import: + from pyramid.security import authenticated_userid + + # and update the index_page view: + @view_config(...) + def index_page(request): + # ... get all entries here + form = None + if not authenticated_userid(request): + form = LoginForm() + return {'entries': entries, 'login_form': form} + +.. nextslide:: Update ``list.jinja2`` + +Now we have to update the template for the ``index_page`` to display the form, *if it is there* + +.. rst-class:: build +.. container:: + + .. code-block:: jinja + + {% block body %} + {% if login_form %} + + {% endif %} + {% if entries %} + ... + +.. nextslide:: Try It Out + +We should be ready at this point. + +.. rst-class:: build +.. container:: + + Fire up your application and see it in action: + + .. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 84467. + serving on http://0.0.0.0:6543 + + Load the home page and see your login form: + + * http://localhost:6543/ + + Fill it in and submit the form, verify that you can add a new entry. + +.. nextslide:: Break Time + +That's enough for now. We have a working application. + +When we return, we'll deploy it. + + +Deploying An Application +======================== + +.. rst-class:: left +.. container:: + + Now that we have a working application, our next step is to deploy it. + + .. rst-class:: build + .. container:: + + This will allow us to interact with the application in a live setting. + + We will be able to see the application from any computer, and can share + it with friends and family. + + To do this, we'll be using one of the most popular platforms for + deploying web applications today, `Heroku`_. + +.. _Heroku: http://heroku.com + +Heroku +------ + +.. figure:: /_static/heroku-logo.png + :align: center + :width: 40% + +.. rst-class:: build +.. container:: + + Heroku provides all the infrastructure needed to run many types of + applications. + + It also provides `add-on services`_ that support everything from analytics + to payment processing. + + Elaborate applications deployed on Heroku can be quite expensive. + + But for simple applications like our learning journal, the price is just + right: **free** + +.. _add-on services: https://addons.heroku.com + +.. nextslide:: How Heroku Works + +Heroku is predicated on interaction with a git repository. + +.. rst-class:: build +.. container:: + + You initialize a new Heroku app in a repository on your machine. + + This adds Heroku as a *remote* to your repository. + + When you are ready to deploy your application, you ``git push heroku + master``. + + Adding a few special files to your repository allows Heroku to tell what + kind of application you are creating. + + It responds to your push by running an appropriate build process and then + starting your app with a command you provide. + +Preparing to Run Your App +------------------------- + +In order for Heroku to deploy your application, it has to have a command it can +run from a standard shell. + +.. rst-class:: build +.. container:: + + We could use the ``pserve`` command we've been using locally, but the + server it uses is designed for development. + + It's not really suitable for a public deployment. + + Instead we'll use a more robust, production-ready server that came as one + of our dependencies: `waitress`_. + + We'll start by creating a python file that can be executed to start the + ``waitress`` server. + +.. _waitress: http://waitress.readthedocs.org/en/latest/ + +.. nextslide:: Creating ``runapp.py`` + +At the very top level of your application project, in the same folder where you +find ``setup.py``, create a new file: ``runapp.py`` + +.. code-block:: python + + import os + from paste.deploy import loadapp + from waitress import serve + + if __name__ == "__main__": + port = int(os.environ.get("PORT", 5000)) + app = loadapp('config:production.ini', relative_to='.') + + serve(app, host='0.0.0.0', port=port) + +.. rst-class:: build +.. container:: + + Once this exists, you can try running your app with it: + + .. code-block:: bash + + (ljenv)$ python runapp.py + serving on http://0.0.0.0:5000 + +.. nextslide:: Running Via Shell + +This would be enough, but we also want to *install* our application as a Python +package. + +.. rst-class:: build +.. container:: + + This will ensure that the dependencies for the application are installed. + + Add a new file called simply ``run`` in the same folder: + + .. code-block:: bash + + #!/bin/bash + python setup.py develop + python runapp.py + + The first line of this file will install our application and its + dependencies. + + The second line will execute the server script. + +.. nextslide:: Build the Database + +We'll need to do the same thing for initializing the database. + +.. rst-class:: build +.. container:: + + Create another new file called ``build_db`` in the same folder: + + .. code-block:: bash + + #!/bin/bash + python setup.py develop + initialize_learning_journal_db production.ini + + Now, add ``run``, ``build_db`` and ``runapp.py`` to your repository and + commit the changes. + +.. nextslide:: Make it Executable + +For Heroku to use them, ``run`` and ``build_db`` must be *executable* + +.. rst-class:: build +.. container:: + + For OSX and Linux users this is easy (do the same for ``run`` and + ``build_db``): + + .. code-block:: bash + + (ljenv)$ chmod 755 run + + Windows users, if you have ``git-bash``, you can do the same + + For the rest of you, try this (for both ``run`` and ``build_db``): + + .. code-block:: posh + + C:\views\myproject>git ls-tree HEAD + ... + 100644 blob 55c0287d4ef21f15b97eb1f107451b88b479bffe run + C:\views\myproject>git update-index --chmod=+x run + C:\views\myproject>git ls-tree HEAD + 100755 blob 3689ebe2a18a1c8ec858cf531d8c0ec34c8405b4 run + + Commit your changes to git to make them permanent. + + +.. nextslide:: Procfile + +Next, we have to inform Heroku that we will be using this script to run our +application online + +.. rst-class:: build +.. container:: + + Heroku uses a special file called ``Procfile`` to do this. + + Add that file now, in the same directory. + + .. code-block:: bash + + web: ./run + + This file tells Heroku that we have one ``web`` process to run, and that it + is the ``run`` script located right here. + + Providing the ``./`` at the start of the file name allows the shell to + execute scripts that are not on the system PATH. + + Add this new file to your repository and commit it. + + +.. nextslide:: Select a Python Version + +By default, Heroku uses the latest update of Python version 2.7 for any Python +app. + +.. rst-class:: build +.. container:: + + You can override this and specify any runtime version of Python + `available in Heroku`_. + + Just add a file called ``runtime.txt`` to your repository, with one line + only: + + .. code-block:: ini + + python-3.5.0 + + Create that file, add it to your repository, and commit the changes. + +.. _available in Heroku: https://devcenter.heroku.com/articles/python-runtimes#supported-python-runtimes + + +Set Up a Heroku App +------------------- + +The next step is to create a new app with heroku. + +.. rst-class:: build +.. container:: + + You installed the Heroku toolbelt prior to class. + + The toolbelt provides a command to create a new app. + + From the root of your project (where the ``setup.py`` file is) run: + + .. code-block:: bash + + (ljenv)$ heroku create + Creating rocky-atoll-9934... done, stack is cedar-14 + https://rocky-atoll-9934.herokuapp.com/ | https://git.heroku.com/rocky-atoll-9934.git + Git remote heroku added + + Note that a new *remote* called ``heroku`` has been added: + + .. code-block:: bash + + $ git remote -v + heroku https://git.heroku.com/rocky-atoll-9934.git (fetch) + heroku https://git.heroku.com/rocky-atoll-9934.git (push) + +.. nextslide:: Adding PostgreSQL + +Your application will require a database, but ``sqlite`` is not really +appropriate for production. + +.. rst-class:: build +.. container:: + + For the deployed app, you'll use `PostgreSQL`_, the best open-source + database. + + Heroku `provides an add-on`_ that supports PostgreSQL, and you'll need to + set it up. + + Again, use the Heroku Toolbelt: + + .. code-block:: bash + + $ heroku addons:create heroku-postgresql:hobby-dev + Creating postgresql-amorphous-6784... done, (free) + Adding postgresql-amorphous-6784 to rocky-atoll-9934... done + Setting DATABASE_URL and restarting rocky-atoll-9934... done, v3 + Database has been created and is available + ! This database is empty. If upgrading, you can transfer + ! data from another database with pg:copy + Use `heroku addons:docs heroku-postgresql` to view documentation. + +.. _PostgreSQL: http://www.postgresql.org +.. _provides an add-on: https://www.heroku.com/postgres + +.. nextslide:: PostgreSQL Settings + +You can get information about the status of your PostgreSQL service with the +toolbelt: + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + (ljenv)$ heroku pg + === DATABASE_URL + Plan: Hobby-dev + ... + Data Size: 6.4 MB + Tables: 0 + Rows: 0/10000 (In compliance) + + And there is also information about the configuration for the database (and + your app): + + .. code-block:: bash + + (ljenv)$ heroku config + === rocky-atoll-9934 Config Vars + DATABASE_URL: postgres://:@:/ + +Configuration for Heroku +------------------------ + +Notice that the configuration for our application on Heroku provides a specific +database URL. + +.. rst-class:: build +.. container:: + + We could copy this value and paste it into our ``production.ini`` + configuration file. + + But if we do that, then we will be storing that value in GitHub, where + anyone at all can see it. + + That's not particularly secure. + + Luckily, Heroku provides configuration like the database URL in + *environment variables* that we can read in Python. + + In fact, we've already done this with our ``runapp.py`` script: + + .. code-block:: python + + port = int(os.environ.get("PORT", 5000)) + +.. nextslide:: Adjusting Our DB Configuration + +The Python standard library provides ``os.environ`` to allow access to +*environment variables* from Python code. + +.. rst-class:: build +.. container:: + + This attribute is a dictionary keyed by the name of the variable. + + We can use it to gain access to configuration provided by Heroku. + + Update ``learning_journal/__init__.py`` like so: + + .. code-block:: python + + # import the os module: + import os + # then look up the value we need for the database url + def main(global_config, **settings): + # ... + if 'DATABASE_URL' in os.environ: + settings['sqlalchemy.url'] = os.environ['DATABASE_URL'] + engine = engine_from_config(settings, 'sqlalchemy.') + # ... + +.. nextslide:: Adjust ``initializedb.py`` + +We'll need to make the same changes to +``learning_journal/scripts/initializedb.py``: + +.. code-block:: python + + def main(argv=sys.argv): + # ... + settings = get_appsettings(config_uri, options=options) + if 'DATABASE_URL' in os.environ: + settings['sqlalchemy.url'] = os.environ['DATABASE_URL'] + engine = engine_from_config(settings, 'sqlalchemy.') + # ... + +.. nextslide:: Additional Security + +This mechanism allows us to defer other sensitive values such as the password +for our initial user: + +.. rst-class:: build +.. container:: + + .. code-block:: python + + # in learning_journal/scripts/initializedb.py + with transaction.manager: + password = os.environ.get('ADMIN_PASSWORD', 'admin') + encrypted = password_context.encrypt(password) + admin = User(name=u'admin', password=encrypted) + DBSession.add(admin) + + And for the secret value for our AuthTktAuthenticationPolicy + + .. code-block:: python + + # in learning_journal/__init__.py + def main(global_config, **settings): + # ... + secret = os.environ.get('AUTH_SECRET', 'somesecret') + ... + authentication_policy=AuthTktAuthenticationPolicy(secret) + # ... + +.. nextslide:: Heroku Config + +We will now be looking for three values from the OS environment: + +.. rst-class:: build + +* DATABASE_URL +* ADMIN_PASSWORD +* AUTH_SECRET + +.. rst-class:: build +.. container:: + + The ``DATABASE_URL`` value is set for us by the PosgreSQL add-on. + + But the other two are not. We must set them ourselves using ``heroku + config:set``: + + .. code-block:: bash + + (ljenv)$ heroku config:set ADMIN_PASSWORD= + ... + (ljenv)$ heroku config:set AUTH_SECRET= + ... + +.. nextslide:: Checking Configuration + +You can see the values that you have set at any time using ``heroku config``: + +.. code-block:: bash + + (ljenv)$ heroku config + === rocky-atoll-9934 Config Vars + ADMIN_PASSWORD: + AUTH_SECRET: + DATABASE_URL: + +.. rst-class:: build +.. container:: + + These values are sent and received using secure transport. + + You do not need to worry about them being intercepted. + + This mechanism allows you to place important configuration values outside + the code for your application. + +.. nextslide:: Installing Dependencies + +We've been handling our application's dependencies by adding them to +``setup.py``. + +.. rst-class:: build +.. container:: + + It's a good idea to install all of these before attempting to run our app. + + The ``pip`` package manager allows us to dump a list of the packages we've + installed in a virtual environment using the ``freeze`` command: + + .. code-block:: bash + + (ljenv)$ pip freeze + ... + zope.interface==4.1.3 + zope.sqlalchemy==0.7.6 + + We can tell heroku to install these dependencies by creating a file called + ``requirements.txt`` at the root of our project repository: + + .. code-block:: bash + + (ljenv)$ pip freeze > requirements.txt + + Add this file to your repository and commit the changes. + + +.. nextslide:: Heroku-specific Dependencies + +But there is also a new dependency we've added that is only needed for Heroku. + +.. rst-class:: build +.. container:: + + Because we are using a PostgreSQL database, we need to install the + ``psycopg2`` package, which handles communicating with the database. + + We don't want to install this locally, though, where we use sqlite. + + Go ahead and add one more line to ``requirements.txt`` with the latest + version of the ``pyscopg2`` package: + + .. code-block:: bash + + psycopg2==2.6.1 + + Commit the change to your repository. + +Deployment +---------- + +We are now ready to deploy our application. + +.. rst-class:: build +.. container:: + + All we need to do is push our repository to the ``heroku`` master: + + .. code-block:: bash + + (ljenv)$ git push heroku master + ... + remote: Building source: + remote: + remote: -----> Python app detected + ... + remote: Verifying deploy... done. + To https://git.heroku.com/rocky-atoll-9934.git + b59b7c3..54f7e4d master -> master + +.. nextslide:: Using ``heroku run`` + +You can use the ``run`` command to execute arbitrary commands in the Heroku +environment. + +.. rst-class:: build +.. container:: + + You can use this to initialize the database, using the shell script you + created earlier: + + .. code-block:: bash + + (ljenv)$ heroku run ./build_db + ... + + This will install our application and then run the database initialization + script. + +.. nextslide:: Test Your Results + +At this point, you should be ready to view your application online. + +.. rst-class:: build +.. container:: + + Use the ``open`` command from heroku to open your website in a browser: + + .. code-block:: bash + + (ljenv)$ heroku open + + If you don't see your application, check to see if it is running: + + .. code-block:: bash + + (ljenv)$ heroku ps + === web (1X): `./run` + web.1: up 2015/01/18 16:44:37 (~ 31m ago) + + If you get no results, use the ``scale`` command to try turning on a web + *dyno*: + + .. code-block:: bash + + (ljenv)$ heroku scale web=1 + Scaling dynos... done, now running web at 1:1X. + +.. nextslide:: A Word About Scaling + +Heroku pricing is dependent on the number of *dynos* you are running. + +.. rst-class:: build +.. container:: + + So long as you only run one dyno per application, you will remain in the + free tier. + + Scaling above one dyno will begin to incur costs. + + **Pay attention to the number of dynos you have running**. + +.. nextslide:: Troubleshooting + +Troubleshooting problems with Heroku deployment can be challenging. + +.. rst-class:: build +.. container:: + + Your most powerful tool is the ``logs`` command: + + .. code-block:: bash + + (ljenv)$ heroku logs + ... + 2015-01-19T01:17:59.443720+00:00 app[web.1]: serving on http://0.0.0.0:53843 + 2015-01-19T01:17:59.505003+00:00 heroku[web.1]: State changed from starting to update + + This command will print the last 50 or so lines of logging from your + application. + + You can use the ``-t`` flag to *tail* the logs. + + This will continually update log entries to your terminal as you interact + with the application. + +.. nextslide:: Revel In Your Glory + +Try logging in to your application with the password you set up in Heroku +configuration. + +.. rst-class:: build +.. container:: + + Once you are logged in, try adding an entry or two. + + You are now off to the races! + + .. rst-class:: center + + **Congratulations** + +Adding Polish +============= + +.. rst-class:: left +.. container:: + + So we have now deployed a running application. + + .. rst-class:: build + .. container:: + + But there are a number of things we can do to make the application + better. + + Let's start by adding a way to log out. + + +Adding Logout +------------- + +Our ``login`` view is already set up to work for logout. + +.. rst-class:: build +.. container:: + + What is the logical path taken if that view is accessed via ``GET``? + + All we need to do is add a view_config that allows that. + + Open ``learning_journal/views.py`` and make these changes: + + .. code-block:: python + + @view_config(route_name='auth', match_param='action=in', renderer='string', + request_method='POST') # <-- THIS IS ALREADY THERE + # ADD THE FOLLOWING LINE + @view_config(route_name='auth', match_param='action=out', renderer='string') + # UPDATE THE VIEW FUNCTION NAME + def sign_in_out(request): + # ... + +.. nextslide:: Re-Deploy + +The chief advantage of Heroku is that we can re-deploy with a single command. + +.. rst-class:: build +.. container:: + + Add and commit your changes to git. + + Then re-deploy by pushing to the ``heroku master``: + + .. code-block:: bash + + (ljenv)$ git push heroku master + + Once that completes, you should be able to reload your application in the + browser. + + Visit the following URL path to test log out: + + * /sign/out + +Hide UI for Anonymous +--------------------- + +Another improvement we can make is to hide UI that is not available for users +who are not logged in. + +.. rst-class:: build +.. container:: + + The first step is to update our ``detail`` view to tell us if someone is + logged in: + + .. code-block:: python + + # learning_journal/views.py + @view_config(route_name='detail', renderer='templates/detail.jinja2') + def view(request): + # ... + logged_in = authenticated_userid(request) + return {'entry': entry, 'logged_in': logged_in} + + The ``authenticated_userid`` function returns the id of the logged in user, + if there is one, and ``None`` if there is not. + + We can use that. + +.. nextslide:: Hide "Create Entry" UI + +First we can hide the UI for creating a new entry: + +.. rst-class:: build +.. container:: + + Edit ``templates/list.jinja2``: + + .. code-block:: jinja + + {% extends "layout.jinja2" %} + {% block body %} + + {% if not login_form %} +

          New Entry

          + {% endif %} + {% endblock %} + + This relies on the fact that the login form will only be present if there + is **not** an authenticated user. + +.. nextslide:: Hide "Edit Entry" UI + +Next, we can hide the UI for editing an existing entry: + +.. rst-class:: build +.. container:: + + Edit ``templates/detail.jinja2``: + + .. code-block:: jinja + + {% extends "layout.jinja2" %} + {% block body %} + +

          + Go Back + {% if logged_in %} + :: + + Edit Entry + {% endif %} +

          + {% endblock %} + +Format Entries +-------------- + +It would be nice if our journal entries could have HTML formatting. + +.. rst-class:: build +.. container:: + + We could write HTML by hand in the body field, but that'd be a pain. + + Instead, let's allow ourselves to write entries `in Markdown`_, a popular + markup syntax used by GitHub and many other websites. + + .. _in Markdown: http://daringfireball.net/projects/markdown/syntax + + Python provides several libraries that implement markdown formatting. + + They will take text that contains markdown formatting and convert it to + HTML. + + Let's use one. + +.. nextslide:: Adding the Dependency + +The first step, is to pick a package and add it to our dependencies. + +.. rst-class:: build +.. container:: + + My recommendation is the `markdown`_ python library. + + Open ``setup.py`` and add the package to the ``requires`` list: + + .. code-block:: python + + requires = [ + # ... + 'cryptacular', + 'markdown', # <-- ADD THIS + ] + + We'll test this locally first, so go ahead and re-install your app: + + .. code-block:: bash + + (ljenv)$ python setup.py develop + ... + Finished processing dependencies for learning-journal==0.0 + +.. _markdown: https://pythonhosted.org/Markdown/ + +.. nextslide:: Jinja2 Filters + +We've seen before how Jinja2 provides a number of filters for values when +rendering templates. + +.. rst-class:: build +.. container:: + + A nice feature of the templating language is that it also allows you to + `create your own filters`_. + + Remember the template syntax for a filter: + + .. code-block:: jinja + + {{ value|filter(arg1, ..., argN) }} + + A filter is simply a function that takes the value to the left of the ``|`` + character as a first argument, and any supplied arguments as the second and + beyond: + + .. code-block:: python + + def filter(value, arg1, ..., argN): + # do something to value here + +.. _create your own filters: http://jinja.pocoo.org/docs/dev/api/#custom-filters + +.. nextslide:: Our Markdown Filter + +Creating a ``markdown`` filter will allow us to convert plain text stored in +the database to HTML at template rendering time. + +.. rst-class:: build +.. container:: + + Open ``learning_journal/views.py`` and add the following: + + .. code-block:: python + + # add two imports: + from jinja2 import Markup + import markdown + # and a function + def render_markdown(content): + output = Markup(markdown.markdown(content)) + return output + + The ``Markup`` class from jinja2 marks a string with HTML tags as "safe". + + This prevents the tags from being *escaped* when they are rendered into a + page. + +.. nextslide:: Register the Filter + +In order for ``Jinja2`` to be aware that our filter exists, we need to register +it. + +.. rst-class:: build +.. container:: + + In Pyramid, we do this in configuration. + + Open ``development.ini`` and edit it as follows: + + .. code-block:: ini + + [app:main] + ... + jinja2.filters = + markdown = learning_journal.views.render_markdown + + This informs the main app that we wish to register a jinja2 filter. + + We will call it ``markdown`` and it will be embodied by the function we + just wrote. + +.. nextslide:: Use Your Filter + +To see the results of our work, we'll need to use the filter in a template +somewhere. + +.. rst-class:: build +.. container:: + + I suggest using it in the ``learning_journal/templates/detail.jinja2`` + template: + + .. code-block:: jinja + + {% extends "layout.jinja2" %} + {% block body %} +
          + +

          {{ entry.body|markdown }}

          + +
          +

          + + {% endblock %} + +.. nextslide:: Test Your Results + +Start up your application, and create an entry using valid markdown formatting: + +.. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 84331. + serving on http://0.0.0.0:6543 + +.. rst-class:: build +.. container:: + + Once you save your entry, you should be able to see it with actual + formatting: headers, bulleted lists, links, and so on. + + That makes quite a difference. + + Go ahead and add the same filter registration to ``production.ini`` + + Then commit your changes and redeploy: + + .. code-block:: bash + + (ljenv)$ git push heroku master + + +Syntax Highlighting +------------------- + +The purpose of this journal is to allow you to write entries about the things +you learn in this class and elsewhere. + +.. rst-class:: build +.. container:: + + Markdown formatting allows for "preformatted" blocks of text like code + samples. + + But there is nothing in markdown that handles *colorizing* code. + + Luckily, the markdown package allows for extensions, and one of these + supports `colorization`_. + + It requires the `pygments`_ library + + Let's set this up next. + +.. _colorization: https://pythonhosted.org/Markdown/extensions/code_hilite.html +.. _pygments: http://pygments.org + +.. nextslide:: Install the Dependency + +Again, we need to install our new dependency first. + +.. rst-class:: build +.. container:: + + Add the following to ``requires`` in ``setup.py``: + + .. code-block:: python + + requires = [ + # ... + 'markdown', + 'pygments', # <-- ADD THIS LINE + ] + + Then re-install your app to pick up the software: + + .. code-block:: bash + + (ljenv)$ python setup.py develop + ... + Finished processing dependencies for learning-journal==0.0 + +.. nextslide:: Add to Our Filter + +The next step is to extend our markdown filter in ``learning_journal/views.py`` +with this feature. + +.. rst-class:: build +.. container:: + + .. code-block:: python + + def render_markdown(content): + output = Markup( + markdown.markdown( + content, + extensions=['codehilite(pygments_style=colorful)', 'fenced_code'] + ) + ) + return output + + Now, you'll be able to make highlighted code blocks just like in GitHub: + + .. code-block:: text + + ```python + def foo(x, y): + return x**y + ``` + +.. nextslide:: Add CSS + +Code highlighting works by putting HTML ```` tags with special CSS +classes around bits of your code. + +.. rst-class:: build +.. container:: + + We need to generate and add the css to support this. + + You can use the ``pygmentize`` command from pygments to + `generate the css`_. + + Make sure you are in the directory with ``setup.py`` when you run this: + + .. code-block:: bash + + (ljenv)$ pygmentize -f html -S colorful -a .codehilite \ + >> learning_journal/static/styles.css + + The styles will be printed to standard out. + + The ``>>`` shell operator *appends* the output to the file named. + +.. _generate the css: http://pygments.org/docs/cmdline/#generating-styles + +.. nextslide:: Try It Out + +Go ahead and restart your application and see the difference a little style +makes: + +.. code-block:: bash + + (ljenv)$ pserve development.ini + Starting server in PID 84331. + serving on http://0.0.0.0:6543 + +.. rst-class:: build +.. container:: + + Try writing an entry with a little Python code in it. + + Python is not the only language available. + + Any syntax covered by `pygments lexers`_ is available, just use the + *shortname* from a lexer to get that type of style highlighting. + +.. _pygments lexers: http://pygments.org/docs/lexers/ + +.. nextslide:: Deploy Your Changes + +When you've got this working as you wish, go ahead and deploy it. + +.. rst-class:: build +.. container:: + + Add and commit all the changes you've made. + + Then push your results to the ``heroku master``: + + .. code-block:: bash + + (ljenv)$ git push heroku master + +Homework +======== + +.. rst-class:: left +.. container:: + + That's just about enough for now. + + .. rst-class:: build + .. container:: + + There's no homework for you to submit this week. You've worked hard enough. + + Take the week to review what we've done and make sure you have a solid + understanding of it. + + If you wish, play with HTML and CSS to make your journal more personalized. + + However, in preparation for our work with Django next week, I'd like you to + get started a bit ahead of time. + + Please read and follow along with this `basic intro to Django`_. + + .. rst-class:: centered + + **See You Then** + +.. _basic intro to Django: django_intro.html diff --git a/source/presentations/session07.rst.norender b/source/presentations/session07.rst.norender deleted file mode 100644 index c5a6c966..00000000 --- a/source/presentations/session07.rst.norender +++ /dev/null @@ -1,1738 +0,0 @@ -Python Web Programming -====================== - -.. image:: img/django-pony.png - :align: left - :width: 50% - -Session 7: A Django Application - -.. class:: intro-blurb right - -Wherein we build a simple blogging app. - -.. class:: image-credit - -image: http://djangopony.com/ - - -Full Stack Framework --------------------- - -Django comes with: - -.. class:: incremental - -* Persistence via the *Django ORM* -* CRUD content editing via the automatic *Django Admin* -* URL Mapping via *urlpatterns* -* Templating via the *Django Template Language* -* Caching with levels of configurability -* Internationalization via i18n hooks -* Form rendering and handling -* User authentication and authorization - -.. class:: incremental - -Pretty much everything you need to make a solid website quickly - - -What Sets it Apart? -------------------- - -Lots of frameworks offer some of these features, if not all. - -.. class:: incremental - -What is Django's *killer feature* - -.. class:: incremental center - -**The Django Admin** - - -The Django Admin ----------------- - -Works in concert with the Django ORM to provide automatic CRUD functionality - -.. class:: incremental - -You write the models, it provides the UI - -.. class:: incremental center - -You've seen this in action. Pretty neat, eh? - - -The Pareto Principle --------------------- - -The Django Admin is a great example of the Pareto Priciple, a.k.a. the 80/20 -rule: - -.. class:: incremental center - -**80% of the problems can be solved by 20% of the effort** - -.. class:: incremental - -The converse also holds true: - -.. class:: incremental center - -**Fixing the last 20% of the problems will take the remaining 80% of the -effort.** - - -Other Django Advantages ------------------------ - -Clearly the most popular full-stack Python web framework at this time - -.. class:: incremental - -Popularity translates into: - -.. class:: incremental - -* Active, present community -* Plethora of good examples to be found online -* Rich ecosystem of *apps* (encapsulated add-on functionality) - -.. class:: incremental center - -**Jobs** - - -Active Development ------------------- - -Django releases in the last 12+ months (a short list): - -.. class:: incremental - -* 1.6.2 (February 2014) -* 1.6.1 (December 2013) -* 1.6 (November 2013) -* 1.4.10 (Novermber 2013) -* 1.5.5 (October 2013) -* 1.5 (February 2013) -* 1.4 (March 2012) - - -Great Documentation -------------------- - -Thorough, readable, and discoverable. - -.. class:: incremental - -Led the way to better documentation for all Python - -.. class:: incremental - -`Read The Docs `_ - built in connection with -Django, sponsored by the Django Software Foundation. - -.. class:: incremental - -Write documentation as part of your python package, and render new versions of -that documentation for every commit - -.. class:: incremental center - -**this is awesome** - - -Where We Stand --------------- - -For your homework this week, you created a ``Post`` model to serve as the heart -of our blogging app. - -.. class:: incremental - -You also took some time to get familiar with the basic workings of the Django -ORM. - -.. class:: incremental - -You made a minor modification to our model class and wrote a test for it. - -.. class:: incremental - -And you installed the Django Admin site and added your app to it. - - -Going Further -------------- - -One of the most common features in a blog is the ability to categorize posts. - -.. class:: incremental - -Let's add this feature to our blog! - -.. class:: incremental - -To do so, we'll be adding a new model, and making some changes to existing code. - -.. class:: incremental - -This means that we'll need to *change our database schema*. - - -Changing a Database -------------------- - -You've seen how to add new tables to a database using the ``syncdb`` command. - -.. class:: incremental - -The ``syncdb`` management command only creates tables that *do not yet exist*. -It **does not update tables**. - -.. class:: incremental - -The ``sqlclear `` command will print the ``DROP TABLE`` statements to -remove the tables for your app. - -.. class:: incremental - -Or ``sql `` will show the ``CREATE TABLE`` statements, and you can work -out the differences and update manually. - -ACK!!! ------- - -That doesn't sound very nice, does it? - -.. class:: incremental - -Luckily, there is an app available for Django that helps with this: ``South`` - -.. class:: incremental - -South allows you to incrementally update your database in a simplified way. - -.. class:: incremental - -South supports forward, backward and data migrations. - -.. class:: incremental - - -Adding South ------------- - -South is so useful, that in Django 1.7 it will become part of the core -distribution of Django. - -.. class:: incremental - -But now it is not. We need to add it, and set up our project to use it. - -.. class:: incremental - -Activate your django virtualenv and install South: - -.. code-block:: bash - - $ source djagnoenv/bin/activate - (djangoenv)$ pip install south - ... - Successfully installed south - Cleaning up... - - -Installing South ----------------- - -Like other Django apps, South provides models of its own. We need to enable them. - -.. container:: incremental - - First, add ``south`` to your list of installed apps in ``settings.py``: - - .. code-block:: python - - INSTALLED_APPS = ( - ... - 'south', #< -add this line - 'myblog', - ) - - -Setting Up South ----------------- - -Then, run ``syncdb`` to pick up the tables it provides: - -.. code-block:: bash - - (djangoenv)$ python manage.py syncdb - Syncing... - Creating tables ... - Creating table south_migrationhistory - ... - - Synced: - ... - > south - > myblog - - Not synced (use migrations): - - - (use ./manage.py migrate to migrate these) - - -Hang On, What Just Happened? ----------------------------- - -You might have noticed that the output from ``syncdb`` looks a bit different -this time. - -.. class:: incremental - -This is because Django apps that use South do not use the normal ``syncdb`` -command to initialize their SQL. - -.. class:: incremental - -Instead they use a new command that South provides: ``migrate``. - -.. class:: incremental - -This command ensures that only incremental changes are made, rather than -creating all of the SQL for an app every time. - - -Adding South to an App ----------------------- - -If you notice, our ``myblog`` app is still in the ``sync`` list. We need to add -South to it. - -.. class:: incremental - -Adding South to an existing Django project is quite simple. The trick is to do -it **before** you make any new changes to your models. - -.. container:: incremental - - Simply use the ``convert_to_south`` management command, providing the name of - your app as an argument: - - .. code-block:: bash - - (djangoenv)$ python manage.py convert_to_south myblog - ... - - -What You Get ------------- - -After running this command, South will automatically create a first migration -for you that sets up tables looking exactly like what your app has now:: - - myblog/ - ├── __init__.py - ... - ├── migrations - │   ├── 0001_initial.py - │   ├── 0001_initial.pyc - │   ├── __init__.py - │   └── __init__.pyc - ├── models.py - ... - -.. class:: incremental - -South also automatically applies this first migration using the ``--fake`` -argument, since the database is already in the proposed state. - - -Adding a Model --------------- - -We want to add a new model to represent the categories our blog posts might -fall into. - -.. class:: incremental - -This model will need to have a name for the category, a longer description and -will need to be related to the Post model. - -.. code-block:: python - :class: small - - # in models.py - class Category(models.Model): - name = models.CharField(max_length=128) - description = models.TextField(blank=True) - posts = models.ManyToManyField(Post, blank=True, null=True, - related_name='categories') - - -Strange Relationships ---------------------- - -In our ``Post`` model, we used a ``ForeignKeyField`` field to match an author -to her posts. - -.. class:: incremental - -This models the situatin in which a single author can have many posts, while -each post has only one author. - -.. class:: incremental - -But any given ``Post`` might belong in more than one ``Category``. - -.. class:: incremental - -And it would be a waste to allow only one ``Post`` for each ``Category``. - -.. class:: incremental - -Enter the ManyToManyField - - -Add a Migration ---------------- - -To get these changes set up, we now have to add a migration. - -.. class:: incremental - -We use the ``schemamigration`` management command to do so: - -.. code-block:: bash - - (djangoenv)$ python manage.py schemamigration myblog --auto - + Added model myblog.Category - + Added M2M table for posts on myblog.Category - Created 0002_auto__add_category.py. You can now apply this - migration with: ./manage.py migrate myblog - - -Apply A Migration ------------------ - -And south, along with making the migration, helpfully tells us what to do next: - -.. code-block:: bash - - (djangoenv)$ python manage.py migrate myblog - Running migrations for myblog: - - Migrating forwards to 0002_auto__add_category. - > myblog:0002_auto__add_category - - Loading initial data for myblog. - Installed 0 object(s) from 0 fixture(s) - -.. class:: incremental - -You can even look at the migration file you just applied, -``myblog/migrations/0002.py`` to see what happened. - - -Make Categories Look Nice -------------------------- - -Let's make ``Category`` object look nice the same way we did with ``Post``. -Start with a test: - -.. container:: incremental - - add this to ``tests.py``: - - .. code-block:: python - :class: incremental - - # another import - from myblog.models import Category - - # and the test case and test - class CategoryTestCase(TestCase): - - def test_unicode(self): - expected = "A Category" - c1 = Category(name=expected) - actual = unicode(c1) - self.assertEqual(expected, actual) - -Make it Pass ------------- - -Do you remember how you made that change for a ``Post``? - -.. code-block:: python - :class: incremental - - class Category(models.Model): - #... - - def __unicode__(self): - return self.name - - -Admin for Categories --------------------- - -Adding our new model to the Django admin is equally simple. - -.. container:: incremental - - Simply add the following line to ``myblog/admin.py`` - - .. code-block:: python - - # a new import - from myblog.models import Category - - # and a new admin registration - admin.site.register(Category) - - -Test It Out ------------ - -Fire up the Django development server and see what you have in the admin: - -.. code-block:: bash - - (djangoenv)$ python manage.py runserver - Validating models... - ... - Starting development server at http://127.0.0.1:8000/ - Quit the server with CONTROL-C. - -.. class:: incremental - -Point your browser at ``http://localhost:8000/admin/``, log in and play. - -.. class:: incremental - -Add a few categories, put some posts in them. Visit your posts, add new ones -and then categorize them. - - -A Public Face -------------- - -Point your browser at http://localhost:8000/ - -.. class:: incremental - -What do you see? - -.. class:: incremental - -Why? - -.. class:: incremental - -We need to add some public pages for our blog. - -.. class:: incremental - -In Django, the code that builds a page that you can see is called a *view*. - -Django Views ------------- - -A *view* can be defined as a *callable* that takes a request and returns a -response. - -.. class:: incremental - -This should sound pretty familiar to you. - -.. class:: incremental - -Classically, Django views were functions. - -.. class:: incremental - -Version 1.3 added support for Class-based Views (a class with a ``__call__`` -method is a callable) - - -A Basic View ------------- - -Let's add a really simple view to our app. - -.. class:: incremental - -It will be a stub for our public UI. Add this to ``views.py`` in ``myblog`` - -.. code-block:: python - :class: small incremental - - from django.http import HttpResponse, HttpResponseRedirect, Http404 - - def stub_view(request, *args, **kwargs): - body = "Stub View\n\n" - if args: - body += "Args:\n" - body += "\n".join(["\t%s" % a for a in args]) - if kwargs: - body += "Kwargs:\n" - body += "\n".join(["\t%s: %s" % i for i in kwargs.items()]) - return HttpResponse(body, content_type="text/plain") - - -Hooking It Up -------------- - -In your homework tutorial, you learned about Django **urlconfs** - -.. class:: incremental - -We used our project urlconf to hook the Django admin into our project. - -.. class:: incremental - -We want to do the same thing for our new app. - -.. class:: incremental - -In general, an *app* that serves any sort of views should contain its own -urlconf. - -.. class:: incremental - -The project urlconf should mainly *include* these where possible. - - -Adding A Urlconf ----------------- - -Create a new file ``urls.py`` inside the ``myblog`` app package. - -.. container:: incremental - - Open it in your editor and add the following code: - - .. code-block:: python - :class: small - - from django.conf.urls import patterns, url - - urlpatterns = patterns('myblog.views', - url(r'^$', - 'stub_view', - name="blog_index"), - ) - - -A Word On Prefixes ------------------- - -The ``patterns`` function takes a first argument called the *prefix* - -.. class:: incremental - -When it is not empty, it is added to any view names in ``url()`` calls in the -same ``patterns``. - -.. class:: incremental - -In a root urlconf like the one in ``mysite``, this isn't too useful - -.. class:: incremental - -But in ``myblog.urls`` it lets us refer to views by simple function name - -.. class:: incremental - -No need to import every view. - - -Include Blog Urls ------------------ - -In order for our new urls to load, we'll need to include them in our project -urlconf - -.. container:: incremental - - Open ``urls.py`` from the ``mysite`` project package and add this: - - .. code-block:: python - :class: small - - urlpatterns = patterns('', - url(r'^', include('myblog.urls')), #<- add this - #... other included urls - ) - -.. class:: incremental - -Try reloading http://localhost:8000/ - -.. class:: incremental - -You should see some output now. - - -Project URL Space ------------------ - -A project is defined by the urls a user can visit. - -.. class:: incremental - -What should our users be able to see when they visit our blog? - -.. class:: incremental - -* A list view that shows blog posts, most recent first. -* An individual post view, showing a single post (a permalink). - -.. class:: incremental - -Let's add urls for each of these, use the stub view for now. - - -Our URLs --------- - -We've already got a good url for the list page: ``blog_index`` at '/' - -.. container:: incremental - - For the view of a single post, we'll need to capture the id of the post. - Add this to ``urlpatterns`` in ``myblog/urls.py``: - - .. code-block:: python - :class: small incremental - - url(r'^posts/(\d+)/$', - 'stub_view', - name="blog_detail"), - -.. class:: incremental - -``(\d+)`` captures one or more digits as the post_id. - -.. class:: incremental - -Load http://localhost:8000/posts/1234/ and see what you get. - - -A Word on Capture in URLs -------------------------- - -When you load the above url, you should see ``1234`` listed as an *arg* - -.. container:: incremental - - Try changing the route like so: - - .. code-block:: python - :class: small - - r'^posts/(?P\d+)/$' - -.. class:: incremental - -Reload the same url. Notice the change. - - -Regular Expression URLS ------------------------ - -Django, unlike Flask, uses Python regular expressions to build routes. - -.. class:: incremental - -When we built our WSGI book app, we did too. - -.. class:: incremental - -There we learned about regular expression *capture groups*. We just changed an -unnamed group to a named one. - -.. class:: incremental - -How you declare a capture group in your url pattern regexp influences how it -will be passed to the view callable. - - -Full Urlconf ------------- - -.. code-block:: python - :class: small - - from django.conf.urls import patterns, url - - urlpatterns = patterns('myblog.views', - url(r'^$', - 'stub_view', - name="blog_index"), - url(r'^posts/(?P\d+)/$', - 'stub_view', - name="blog_detail"), - ) - - -Testing Views -------------- - -Before we begin writing real views, we need to add some tests for the views we -are about to create. - -.. class:: incremental - -We'll need tests for a list view and a detail view - -.. container:: incremental - - add the following *imports* at the top of ``myblog/tests.py``: - - .. code-block:: python - - import datetime - from django.utils.timezone import utc - - -Add a Test Case ---------------- - -.. code-block:: python - :class: small - - class FrontEndTestCase(TestCase): - """test views provided in the front-end""" - fixtures = ['myblog_test_fixture.json', ] - - def setUp(self): - self.now = datetime.datetime.utcnow().replace(tzinfo=utc) - self.timedelta = datetime.timedelta(15) - author = User.objects.get(pk=1) - for count in range(1,11): - post = Post(title="Post %d Title" % count, - text="foo", - author=author) - if count < 6: - # publish the first five posts - pubdate = self.now - self.timedelta * count - post.published_date = pubdate - post.save() - - -Our List View -------------- - -We'd like our list view to show our posts. - -.. class:: incremental - -But in this blog, we have the ability to publish posts. - -.. class:: incremental - -Unpublished posts should not be seen in the front-end views. - -.. class:: incremental - -We set up our tests to have 5 published, and 5 unpublished posts - -.. class:: incremental - -Let's add a test to demonstrate that the right ones show up. - - -Testing the List View ---------------------- - -.. code-block:: python - - Class FrontEndTestCase(TestCase): # already here - # ... - def test_list_only_published(self): - resp = self.client.get('/') - self.assertTrue("Recent Posts" in resp.content) - for count in range(1,11): - title = "Post %d Title" % count - if count < 6: - self.assertContains(resp, title, count=1) - else: - self.assertNotContains(resp, title) - -.. class:: incremental - -Note that we also test to ensure that the unpublished posts are *not* visible. - - -Run Your Tests --------------- - -.. code-block:: bash - - (djangoenv)$ python manage.py test myblog - Creating test database for alias 'default'... - .F. - ====================================================================== - FAIL: test_list_only_published (myblog.tests.FrontEndTestCase) - ... - Ran 3 tests in 0.024s - - FAILED (failures=1) - Destroying test database for alias 'default'... - - -Now Fix That Test! ------------------- - -Add the view for listing blog posts to ``views.py``. - -.. code-block:: python - :class: small - - # add these imports - from django.template import RequestContext, loader - from myblog.models import Post - - # and this view - def list_view(request): - published = Post.objects.exclude(published_date__exact=None) - posts = published.order_by('-published_date') - template = loader.get_template('list.html') - context = RequestContext(request, { - 'posts': posts, - }) - body = template.render(context) - return HttpResponse(body, content_type="text/html") - - -Getting Posts -------------- - -.. code-block:: python - :class: small - - published = Post.objects.exclude(published_date__exact=None) - posts = published.order_by('-published_date') - -.. class:: incremental - -We begin by using the QuerySet API to fetch all the posts that have -``published_date`` set - -.. class:: incremental - -Using the chaining nature of the API we order these posts by -``published_date`` - -.. class:: incremental - -Remember, at this point, no query has actually been issued to the database. - - -Getting a Template ------------------- - -.. code-block:: python - :class: small - - template = loader.get_template('list.html') - -.. class:: incremental - -Django uses configuration to determine how to find templates. - -.. class:: incremental - -By default, Django looks in installed *apps* for a ``templates`` directory - -.. class:: incremental - -It also provides a place to list specific directories. - -.. class:: incremental - -Let's set that up in ``settings.py`` - - -Project Templates ------------------ - -In ``settings.py`` add ``TEMPLATE_DIRS`` and add the absolute path to your -``mysite`` project package: - -.. code-block:: python - :class: small - - TEMPLATE_DIRS = ('/absolute/path/to/mysite/mysite/templates', ) - -.. class:: incremental - -Then add a ``templates`` directory to your ``mysite`` project package - -.. class:: incremental - -Finally, in that directory add a new file ``base.html`` and populate it with -the following: - - -base.html ---------- - -.. code-block:: jinja - :class: small - - - - - My Django Blog - - -

          -
          - {% block content %} - [content will go here] - {% endblock %} -
          -
          - - - - -Templates in Django -------------------- - -Before we move on, a quick word about Django templates. - -.. class:: incremental - -We've seen Jinja2 which was "inspired by Django's templating system". - -.. class:: incremental - -Basically, you already know how to write Django templates. - -.. class:: incremental - -Django templates **do not** allow any python expressions. - -.. class:: incremental center small - -https://docs.djangoproject.com/en/1.5/ref/templates/builtins/ - - -Blog Templates --------------- - -Our view tries to load ``list.html``. - -.. class:: incremental - -This template is probably specific to the blog functionality of our site - -.. class:: incremental - -It is common to keep shared templates in your project directory and -specialized ones in app directories. - -.. class:: incremental - -Add a ``templates`` directory to your ``myblog`` app, too. - -.. class:: incremental - -In it, create a new file ``list.html`` and add this: - - -list.html ---------- - -.. code-block:: jinja - :class: tiny - - {% extends "base.html" %} - - {% block content %} -

          Recent Posts

          - - {% comment %} here is where the query happens {% endcomment %} - {% for post in posts %} -
          -

          {{ post }}

          - -
          - {{ post.text }} -
          -
            - {% for category in post.categories.all %} -
          • {{ category }}
          • - {% endfor %} -
          -
          - {% endfor %} - {% endblock %} - - -Template Context ----------------- - -.. code-block:: python - :class: small - - context = RequestContext(request, { - 'posts': posts, - }) - body = template.render(context) - -.. class:: incremental - -Like Jinja2, django templates are rendered by passing in a *context* - -.. class:: incremental - -Django's RequestContext provides common bits, similar to the global context in -Flask - -.. class:: incremental - -We add our posts to that context so they can be used by the template. - - -Return a Response ------------------ - -.. code-block:: python - :class: small - - return HttpResponse(body, content_type="text/html") - -.. class:: incremental - -Finally, we build an HttpResponse and return it. - -.. class:: incremental - -This is, fundamentally, no different from the ``stub_view`` just above. - - -Fix URLs --------- - -We need to fix the url for our blog index page - -.. container:: incremental - - Update ``urls.py`` in ``myblog``: - - .. code-block:: python - :class: small - - url(r'^$', - 'list_view', - name="blog_index"), - -.. class:: incremental small - -:: - - (djangoenv)$ python manage.py test myblog - ... - Ran 3 tests in 0.033s - - OK - - -Common Patterns ---------------- - -This is a common pattern in Django views: - -.. class:: incremental - -* get a template from the loader -* build a context, usually using a RequestContext -* render the template -* return an HttpResponse - -.. class:: incremental - -So common in fact that Django provides two shortcuts for us to use: - -.. class:: incremental - -* ``render(request, template[, ctx][, ctx_instance])`` -* ``render_to_response(template[, ctx][, ctx_instance])`` - - -Shorten Our View ----------------- - -Let's replace most of our view with the ``render`` shortcut - -.. code-block:: python - :class: small - - from django.shortcuts import render # <- already there - - # rewrite our view - def list_view(request): - published = Post.objects.exclude(published_date__exact=None) - posts = published.order_by('-published_date') - context = {'posts': posts} - return render(request, 'list.html', context) - -.. class:: incremental - -Remember though, all we did manually before is still happening - - -Our Detail View ---------------- - -Next, let's add a view function for the detail view of a post - -.. class:: incremental - -It will need to get the ``id`` of the post to show as an argument - -.. class:: incremental - -Like the list view, it should only show published posts - -.. class:: incremental - -But unlike the list view, it will need to return *something* if an unpublished -post is requested. - -.. class:: incremental - -Let's start with the tests in ``views.py`` - - -Testing the Details -------------------- - -Add the following test to our ``FrontEndTestCase`` in ``myblog/tests.py``: - -.. code-block:: python - :class: small incremental - - def test_details_only_published(self): - for count in range(1,11): - title = "Post %d Title" % count - post = Post.objects.get(title=title) - resp = self.client.get('/posts/%d/' % post.pk) - if count < 6: - self.assertEqual(resp.status_code, 200) - self.assertContains(resp, title) - else: - self.assertEqual(resp.status_code, 404) - - -Run Your Tests --------------- - -.. code-block:: bash - - (djangoenv)$ python manage.py test myblog - Creating test database for alias 'default'... - .F.. - ====================================================================== - FAIL: test_details_only_published (myblog.tests.FrontEndTestCase) - ... - Ran 4 tests in 0.043s - - FAILED (failures=1) - Destroying test database for alias 'default'... - - -Let's Fix That Test -------------------- - -Now, add a new view to ``myblog/views.py``: - -.. code-block:: python - :class: incremental small - - def detail_view(request, post_id): - published = Post.objects.exclude(published_date__exact=None) - try: - post = published.get(pk=post_id) - except Post.DoesNotExist: - raise Http404 - context = {'post': post} - return render(request, 'detail.html', context) - - -Missing Content ---------------- - -One of the features of the Django ORM is that all models raise a DoesNotExist -exception if ``get`` returns nothing. - -.. class:: incremental - -This exception is actually an attribute of the Model you look for. There's also -an ``ObjectDoesNotExist`` for when you don't know which model you have. - -.. class:: incremental - -We can use that fact to raise a Not Found exception. - -.. class:: incremental - -Django will handle the rest for us. - - -Add the Template ----------------- - -We also need to add ``detail.html`` to ``myblog/templates``: - -.. code-block:: jinja - :class: tiny - - {% extends "base.html" %} - - {% block content %} - Home -

          {{ post }}

          - -
          - {{ post.text }} -
          -
            - {% for category in post.categories.all %} -
          • {{ category }}
          • - {% endfor %} -
          - {% endblock %} - - -Hook it Up ----------- - -In order to view a single post, we'll need a link from the list view - -.. container:: incremental - - We can use the ``url`` template tag (like flask ``url_for``): - - .. code-block:: jinja - :class: small - - {% url '' arg1 arg2 %} - -.. class:: incremental - -In our ``list.html`` template, let's link the post titles: - -.. code-block:: jinja - :class: small incremental - - {% for post in posts %} -
          -

          - {{ post }} -

          - ... - - -Fix URLs --------- - -Again, we need to insert our new view into the existing ``myblog/urls.py`` in -``myblog``: - -.. code-block:: python - :class: small - - url(r'^posts/(?P\d+)/$', - 'detail_view', - name="blog_detail"), - -.. class:: incremental small - -:: - - (djangoenv)$ python manage.py test myblog - ... - Ran 4 tests in 0.077s - - OK - - -A Moment To Play ----------------- - -We've got some good stuff to look at now. Fire up the server - -.. class:: incremental - -Reload your blog index page and click around a bit. - -.. class:: incremental - -You can now move back and forth between list and detail view. - -.. class:: incremental - -Try loading the detail view for a post that doesn't exist - - -Congratulations ---------------- - -You've got a functional Blog - -.. class:: incremental - -It's not very pretty, though. - -.. class:: incremental - -We can fix that by adding some css - -.. class:: incremental - -This gives us a chance to learn about Django's handling of *static files* - - -Static Files ------------- - -Like templates, Django expects to find static files in particular locations - -.. class:: incremental - -It will look for them in a directory named ``static`` in any installed apps. - -.. class:: incremental - -They will be served from the url path in the STATIC_URL setting. - -.. class:: incremental - -By default, this is ``/static/`` - - -Add CSS -------- - -I've prepared a css file for us to use. You can find it in the class resources - -.. class:: incremental - -Create a new directory ``static`` in the ``myblog`` app. - -.. class:: incremental - -Copy the ``django_blog.css`` file into that new directory. - -.. container:: incremental - - Then add this link to the of ``base.html``: - - .. code-block:: html - :class: small - - My Django Blog - - - -View Your Results ------------------ - -Reload http://localhost:8000/ and view the results of your work - -.. class:: incremental - -We now have a reasonable view of the posts of our blog on the front end - -.. class:: incremental - -And we have a way to create and categorize posts using the admin - -.. class:: incremental - -However, we lack a way to move between the two. - -.. class:: incremental - -Let's add that ability next. - - -Adding A Control Bar --------------------- - -We'll start by adding a control bar to our ``base.html`` template: - -.. code-block:: jinja - :class: small - - - ... - -
          - ... - - -Request Context Revisited -------------------------- - -When we set up our views, we used the ``render`` shortcut, which provides a -``RequestContext`` - -.. class:: incremental - -This gives us access to ``user`` in our templates - -.. class:: incremental - -It provides access to methods about the state and rights of that user - -.. class:: incremental - -We can use these to conditionally display links or UI elements. Like only -showing the admin link to staff members. - - -Login/Logout ------------- - -Django also provides a reasonable set of views for login/logout. - -.. class:: incremental - -The first step to using them is to hook them into a urlconf. - -.. container:: incremental - - Add the following to ``mysite/urls.py``: - - .. code-block:: python - :class: small - - url(r'^', include('myblog.urls')), #<- already there - url(r'^login/$', - 'django.contrib.auth.views.login', - {'template_name': 'login.html'}, - name="login"), - url(r'^logout/$', - 'django.contrib.auth.views.logout', - {'next_page': '/'}, - name="logout"), - - -Login Template --------------- - -We need to create a new ``login.html`` template in ``mysite/templates``: - -.. code-block:: jinja - :class: small - - {% extends "base.html" %} - - {% block content %} -

          My Blog Login

          -
          {% csrf_token %} - {{ form.as_p }} -

          -
          - {% endblock %} - - -Submitting Forms ----------------- - -In a web application, submitting forms is potentially hazardous - -.. class:: incremental - -Data is being sent to our application from some remote place - -.. class:: incremental - -If that data is going to alter the state of our application, we **must** use -POST - -.. class:: incremental - -Even so, we are vulnerable to Cross-Site Request Forgery, a common attack -vector. - - -Danger: CSRF ------------- - -Django provides a convenient system to fight this. - -.. class:: incremental - -In fact, for POST requests, it *requires* that you use it. - -.. class:: incremental - -The Django middleware that does this is enabled by default. - -.. class:: incremental - -All you need to do is include the ``{% csrf_token %}`` tag in your form. - - -Hooking It Up -------------- - -In ``base.html`` make the following updates: - -.. code-block:: jinja - :class: small - - - admin - - logout - - login - -.. container:: incremental - - Finally, in ``settings.py`` add the following: - - .. code-block:: python - :class: small - - LOGIN_URL = '/login/' - LOGIN_REDIRECT_URL = '/' - - -Forms In Django ---------------- - -In adding a login view, we've gotten a sneak peak at how forms work in Django. - -.. class:: incremental - -However, learning more about them is beyond what we can achieve in this -session. - -.. class:: incremental - -The form system in Django is quite nice, however. I urge you to `read more about it`_ - -.. _read more about it: https://docs.djangoproject.com/en/1.6/topics/forms/ - -.. class:: incremental - -In particular, you might want to pay attention to the documentation on `Model Forms` - -.. _Model Forms: https://docs.djangoproject.com/en/1.6/topics/forms/modelforms/ - - -Ta-Daaaaaa! ------------ - -So, that's it. We've created a workable, simple blog app in Django. - -.. class:: incremental - -There's much more we could do with this app. And for homework, you'll do some -of it. - -.. class:: incremental - -Then next session, we'll work as we did in session 6. - -.. class:: incremental - -We'll divide up into pairs, and implement a simple feature to extend our blog. - - -Homework --------- - -For your homework this week, we'll fix one glaring problem with our blog admin. - -.. class:: incremental - -As you created new categories and posts, and related them to each-other, how -did you feel about that work? - -.. class:: incremental - -Although from a data perspective, the category model is the right place for the -ManytoMany relationship to posts, this leads to awkward usage in the admin. - -.. class:: incremental - -It would be much easier if we could designate a category for a post *from the -Post admin*. - - -Your Assignment ---------------- - -You'll be reversing that relationship so that you can only add categories to posts - -Take the following steps: - -1. Read the documentation about the `Django admin.`_ -2. You'll need to create a customized `ModelAdmin`_ class for the ``Post`` and - ``Category`` models. -3. And you'll need to create an `InlineModelAdmin`_ to represent Categories on - the Post admin view. -4. Finally, you'll need to `suppress the display`_ of the 'posts' field on - your ``Category`` admin view. - - -.. _Django admin.: https://docs.djangoproject.com/en/1.6/ref/contrib/admin/ -.. _ModelAdmin: https://docs.djangoproject.com/en/1.6/ref/contrib/admin/#modeladmin-objects -.. _InlineModelAdmin: https://docs.djangoproject.com/en/1.6/ref/contrib/admin/#inlinemodeladmin-objects -.. _suppress the display: https://docs.djangoproject.com/en/1.6/ref/contrib/admin/#modeladmin-options - - -Pushing Further ---------------- - -All told, those changes should not require more than about 15 total lines of -code. - -The trick of course is reading and finding out which fifteen lines to write. - -If you complete that task in less than 3-4 hours of work, consider looking into -other ways of customizing the admin. - - -Tasks you might consider ------------------------- - -* Change the admin index to say 'Categories' instead of 'Categorys'. -* Add columns for the date fields to the list display of Posts. -* Display the created and modified dates for your posts when viewing them in - the admin. -* Add a column to the list display of Posts that shows the author. For more - fun, make this a link that takes you to the admin page for that user. - -* For the biggest challenge, look into `admin actions`_ and add an action to - the Post admin that allows you to bulk publish posts from the Post list - display - -.. _admin actions: https://docs.djangoproject.com/en/1.6/ref/contrib/admin/actions/ diff --git a/source/presentations/session08.rst b/source/presentations/session08.rst new file mode 100644 index 00000000..76a08bb7 --- /dev/null +++ b/source/presentations/session08.rst @@ -0,0 +1,1522 @@ +********** +Session 08 +********** + +.. figure:: /_static/django-pony.png + :align: center + :width: 60% + + image: http://djangopony.com/ + +Building a Django Application +============================= + +.. rst-class:: large + +Wherein we build a simple blogging app. + + +A Full Stack Framework +---------------------- + +Django comes with: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * Persistence via the *Django ORM* + * CRUD content editing via the automatic *Django Admin* + * URL Mapping via *urlpatterns* + * Templating via the *Django Template Language* + * Caching with levels of configurability + * Internationalization via i18n hooks + * Form rendering and handling + * User authentication and authorization + + Pretty much everything you need to make a solid website quickly + +.. nextslide:: What Sets it Apart? + +Lots of frameworks offer some of these features, if not all. + +.. rst-class:: build +.. container:: + + What is Django's *killer feature* + + .. rst-class:: centered + + **The Django Admin** + +.. nextslide:: The Django Admin + +Works in concert with the Django ORM to provide automatic CRUD functionality + +.. rst-class:: build +.. container:: + + You write the models, it provides the UI + + You've seen this in action. Pretty neat, eh? + +.. nextslide:: The Pareto Principle + +The Django Admin is a great example of the Pareto Priciple, a.k.a. the 80/20 +rule: + +.. rst-class:: build +.. container:: + + .. rst-class:: centered + + **80% of the problems can be solved by 20% of the effort** + + The converse also holds true: + + .. rst-class:: centered + + **Fixing the last 20% of the problems will take the remaining 80% of the + effort.** + +.. nextslide:: Other Django Advantages + +.. ifnotslides:: + + **Other Django Advantages** + +Clearly the most popular full-stack Python web framework at this time + +.. rst-class:: build +.. container:: + + Popularity translates into: + + .. rst-class:: build + + * Active, present community + * Plethora of good examples to be found online + * Rich ecosystem of *apps* (encapsulated add-on functionality) + + .. rst-class:: centered + + **Jobs** + +.. nextslide:: Active Development + +Django releases in the last 12+ months (a short list): + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * 1.9 (December 2015) + * 1.8.7 (November 2015) + * 1.7.11 (November 2015) + * 1.8.5 (October 2015) + * 1.7.10 (August 2015) + * 1.8.3 (July 2015) + * 1.8 (April 2015) + * 1.7.7 (March 2015) + * 1.7.4 (January 2014) + + Django 1.8 is the second *Long Term Support* version, with a guaranteed support + period of three years. + +.. nextslide:: Great Documentation + +Thorough, readable, and discoverable. + +.. rst-class:: build +.. container:: + + Led the way to better documentation for all Python + + `Read The Docs `_ - built in connection with + Django, sponsored by the Django Software Foundation. + + Write documentation as part of your python package. + + Render new versions of that documentation for every commit. + + .. rst-class:: centered + + **this is awesome** + + +Where We Stand +-------------- + +For your homework this week, you created a ``Post`` model to serve as the heart +of our blogging app. + +.. rst-class:: build +.. container:: + + You also took some time to get familiar with the basic workings of the + Django ORM. + + You made a minor modification to our model class and wrote a test for it. + + And you installed the Django Admin site and added your app to it. + + +Going Further +------------- + +One of the most common features in a blog is the ability to categorize posts. + +.. rst-class:: build +.. container:: + + Let's add this feature to our blog! + + To do so, we'll be adding a new model, and making some changes to existing + code. + + .. rst-class:: build + + This means that we'll need to *change our database schema*. + + +.. nextslide:: Changing a Database + +You've seen how to add new tables to a database using the ``migrate`` command. + +.. rst-class:: build +.. container:: + + And you've created your first migration in setting up the ``Post`` model. + + This is an example of altering the *database schema* using Python code. + + Starting in Django 1.7, this ability is available built-in to Django. + + Before verson 1.7 it was available in an add-on called `South`_. + +.. _South: http://south.readthedocs.org/en/latest + + +.. nextslide:: Adding a Model + +We want to add a new model to represent the categories our blog posts might +fall into. + +.. rst-class:: build +.. container:: + + This model will need to have: + + .. rst-class:: build + + * a name for the category + * a longer description + * a relationship to the Post model + + .. code-block:: python + + # in models.py + class Category(models.Model): + name = models.CharField(max_length=128) + description = models.TextField(blank=True) + posts = models.ManyToManyField(Post, blank=True, + related_name='categories') + + +.. nextslide:: Strange Relationships + +In our ``Post`` model, we used a ``ForeignKeyField`` field to match an author +to her posts. + +.. rst-class:: build +.. container:: + + This models the situation in which a single author can have many posts, + while each post has only one author. + + We call this a *Many to One* relationship. + + But any given ``Post`` might belong in more than one ``Category``. + + And it would be a waste to allow only one ``Post`` for each ``Category``. + + Enter the ``ManyToManyField`` + +.. nextslide:: Add a Migration + +To get these changes set up, we now add a new migration. + +.. rst-class:: build +.. container:: + + We use the ``makemigrations`` management command to do so: + + .. code-block:: bash + + (djangoenv)$ ./manage.py makemigrations + Migrations for 'myblog': + 0002_category.py: + - Create model Category + +.. nextslide:: Apply A Migration + +Once the migration has been created, we can apply it with the ``migrate`` +management command. + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + (djangoenv)$ ./manage.py migrate + Operations to perform: + Apply all migrations: sessions, contenttypes, admin, myblog, auth + Running migrations: + Rendering model states... DONE + Applying myblog.0002_category... OK + + You can even look at the migration file you just applied, + ``myblog/migrations/0002_category.py`` to see what happened. + + +.. nextslide:: Make Categories Look Nice + +Let's make ``Category`` object look nice the same way we did with ``Post``. +Start with a test: + +.. rst-class:: build +.. container:: + + add this to ``tests.py``: + + .. code-block:: python + + # another import + from myblog.models import Category + + # and the test case and test + class CategoryTestCase(TestCase): + + def test_string_representation(self): + expected = "A Category" + c1 = Category(name=expected) + actual = str(c1) + self.assertEqual(expected, actual) + +.. nextslide:: Make it Pass + +When you run your tests, you now have two, and one is failing because the +``Category`` object doesn't look right. + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + (djangoenv)$ ./manage.py test myblog + Creating test database for alias 'default'... + ... + + Ran 2 tests in 0.011s + + FAILED (failures=1) + + Do you remember how you made that change for a ``Post``? + + .. code-block:: python + + class Category(models.Model): + #... + + def __str__(self): + return self.name + + +.. nextslide:: Admin for Categories + +Adding our new model to the Django admin is equally simple. + +.. rst-class:: build +.. container:: + + Simply add the following line to ``myblog/admin.py`` + + .. code-block:: python + + # a new import + from myblog.models import Category + + # and a new admin registration + admin.site.register(Category) + + +.. nextslide:: Test It Out + +Fire up the Django development server and see what you have in the admin: + +.. code-block:: bash + + (djangoenv)$ ./manage.py runserver + Validating models... + ... + Starting development server at http://127.0.0.1:8000/ + Quit the server with CONTROL-C. + +.. rst-class:: build +.. container:: + + Point your browser at ``http://localhost:8000/admin/``, log in and play. + + Add a few categories, put some posts in them. Visit your posts, add new + ones and then categorize them. + + +BREAK TIME +---------- + +We've completed a data model for our application. + +And thanks to Django's easy-to-use admin, we have a reasonable CRUD application +where we can manage blog posts and the categories we put them in. + +When we return, we'll put a public face on our new creation. + +If you've fallen behind, the app as it stands now is in our class resources as +``mysite_stage_1`` + + +A Public Face +============= + +.. rst-class:: left + +Point your browser at http://localhost:8000/ + +.. rst-class:: build left +.. container:: + + What do you see? + + Why? + + We need to add some public pages for our blog. + + In Django, the code that builds a page that you can see is called a *view*. + + +Django Views +------------ + +A *view* can be defined as a *callable* that takes a request and returns a +response. + +.. rst-class:: build +.. container:: + + This should sound pretty familiar to you. + + Classically, Django views were functions. + + Version 1.3 added support for Class-based Views (a class with a + ``__call__`` method is a callable) + + +.. nextslide:: A Basic View + +Let's add a really simple view to our app. + +.. rst-class:: build +.. container:: + + It will be a stub for our public UI. Add this to ``views.py`` in + ``myblog`` + + .. code-block:: python + + from django.http import HttpResponse, HttpResponseRedirect, Http404 + + def stub_view(request, *args, **kwargs): + body = "Stub View\n\n" + if args: + body += "Args:\n" + body += "\n".join(["\t%s" % a for a in args]) + if kwargs: + body += "Kwargs:\n" + body += "\n".join(["\t%s: %s" % i for i in kwargs.items()]) + return HttpResponse(body, content_type="text/plain") + +.. nextslide:: Hooking It Up + +In your homework tutorial, you learned about Django **urlconfs** + +.. rst-class:: build +.. container:: + + We used our project urlconf to hook the Django admin into our project. + + We want to do the same thing for our new app. + + In general, an *app* that serves any sort of views should contain its own + urlconf. + + The project urlconf should mainly *include* these where possible. + + +.. nextslide:: Adding A Urlconf + +Create a new file ``urls.py`` inside the ``myblog`` app package. + +.. rst-class:: build +.. container:: + + Open it in your editor and add the following code: + + .. code-block:: python + + + from django.conf.urls import url + from myblog.views import stub_view + + urlpatterns = [ + url(r'^$', + stub_view, + name="blog_index"), + ] + + +.. nextslide:: Include Blog Urls + +In order for our new urls to load, we'll need to include them in our project +urlconf + +.. rst-class:: build +.. container:: + + Open ``urls.py`` from the ``mysite`` project package and add this: + + .. code-block:: python + + # add this new import + from django.conf.urls import include + + # then modify urlpatterns as follows: + urlpatterns = [ + url(r'^', include('myblog.urls')), #<- add this + #... other included urls + ] + + Try reloading http://localhost:8000/ + + You should see some output now. + + +Project URL Space +----------------- + +A project is defined by the urls a user can visit. + +.. rst-class:: build +.. container:: + + What should our users be able to see when they visit our blog? + + .. rst-class:: build + + * A list view that shows blog posts, most recent first. + * An individual post view, showing a single post (a permalink). + + Let's add urls for each of these. + + For now, we'll use the stub view we've created so we can concentrate on the + url routing. + +.. nextslide:: Our URLs + +We've already got a good url for the list page: ``blog_index`` at '/' + +.. rst-class:: build +.. container:: + + For the view of a single post, we'll need to capture the id of the post. + Add this to ``urlpatterns`` in ``myblog/urls.py``: + + .. code-block:: python + + url(r'^posts/(\d+)/$', + stub_view, + name="blog_detail"), + + ``(\d+)`` captures one or more digits as the post_id. + + Load http://localhost:8000/posts/1234/ and see what you get. + +.. nextslide:: A Word on Capture in URLs + +When you load the above url, you should see ``1234`` listed as an *arg* + +.. rst-class:: build +.. container:: + + Try changing the route like so: + + .. code-block:: python + + r'^posts/(?P\d+)/$' + + Reload the same url. + + Notice the change. + + What's going on there? + +.. nextslide:: Regular Expression URLS + +Like Pyramid, Django uses Python regular expressions to build routes. + +.. rst-class:: build +.. container:: + + Unlike Pyramid, Django *requires* regular expressions to capture segments + in a route. + + When we built our WSGI book app, we used this same appraoch. + + There we learned about regular expression *capture groups*. We just changed + an unnamed *capture group* to a named one. + + How you declare a capture group in your url pattern regexp influences how + it will be passed to the view callable. + + +.. nextslide:: Full Urlconf + +.. code-block:: python + + + from django.conf.urls import url + from myblog.views import stub_view + + urlpatterns = [ + url(r'^$', + stub_view, + name="blog_index"), + url(r'^posts/(?P\d+)/$', + stub_view, + name="blog_detail"), + ] + + +.. nextslide:: Testing Views + +Before we begin writing real views, we need to add some tests for the views we +are about to create. + +.. rst-class:: build +.. container:: + + We'll need tests for a list view and a detail view + + add the following *imports* at the top of ``myblog/tests.py``: + + .. code-block:: python + + import datetime + from django.utils.timezone import utc + + +.. nextslide:: Add a Test Case + +.. code-block:: python + + class FrontEndTestCase(TestCase): + """test views provided in the front-end""" + fixtures = ['myblog_test_fixture.json', ] + + def setUp(self): + self.now = datetime.datetime.utcnow().replace(tzinfo=utc) + self.timedelta = datetime.timedelta(15) + author = User.objects.get(pk=1) + for count in range(1, 11): + post = Post(title="Post %d Title" % count, + text="foo", + author=author) + if count < 6: + # publish the first five posts + pubdate = self.now - self.timedelta * count + post.published_date = pubdate + post.save() + + +Our List View +------------- + +We'd like our list view to show our posts. + +.. rst-class:: build +.. container:: + + But in this blog, we have the ability to publish posts. + + Unpublished posts should not be seen in the front-end views. + + We set up our tests to have 5 published, and 5 unpublished posts + + Let's add a test to demonstrate that the right ones show up. + +.. nextslide:: Testing the List View + +.. code-block:: python + + Class FrontEndTestCase(TestCase): # already here + # ... + def test_list_only_published(self): + resp = self.client.get('/') + # the content of the rendered response is always a bytestring + resp_text = resp.content.decode(resp.charset) + self.assertTrue("Recent Posts" in resp_text) + for count in range(1, 11): + title = "Post %d Title" % count + if count < 6: + self.assertContains(resp, title, count=1) + else: + self.assertNotContains(resp, title) + +.. rst-class:: build +.. container:: + + We test first to ensure that each published post is visible in our view. + + Note that we also test to ensure that the unpublished posts are *not* visible. + + +.. nextslide:: Run Your Tests + +.. code-block:: bash + + (djangoenv)$ ./manage.py test myblog + Creating test database for alias 'default'... + .F. + ====================================================================== + FAIL: test_list_only_published (myblog.tests.FrontEndTestCase) + ... + Ran 3 tests in 0.024s + + FAILED (failures=1) + Destroying test database for alias 'default'... + + +.. nextslide:: Now Fix That Test! + +Add the view for listing blog posts to ``views.py``. + +.. code-block:: python + + # add these imports + from django.template import RequestContext, loader + from myblog.models import Post + + # and this view + def list_view(request): + published = Post.objects.exclude(published_date__exact=None) + posts = published.order_by('-published_date') + template = loader.get_template('list.html') + context = RequestContext(request, { + 'posts': posts, + }) + body = template.render(context) + return HttpResponse(body, content_type="text/html") + + +.. nextslide:: Getting Posts + +.. code-block:: python + + published = Post.objects.exclude(published_date__exact=None) + posts = published.order_by('-published_date') + +.. rst-class:: build +.. container:: + + We begin by using the QuerySet API to fetch all the posts that have + ``published_date`` set + + Using the chaining nature of the API we order these posts by + ``published_date`` + + Remember, at this point, no query has actually been issued to the database. + + +.. nextslide:: Getting a Template + +.. code-block:: python + + template = loader.get_template('list.html') + +.. rst-class:: build +.. container:: + + Django uses configuration to determine how to find templates. + + By default, Django looks in installed *apps* for a ``templates`` directory + + It also provides a place to list specific directories. + + Let's set that up in ``settings.py`` + + +.. nextslide:: Project Templates + +Notice that ``settings.py`` already contains a ``BASE_DIR`` value which points +to the root of our project (where both the project and app packages are +located). + +.. rst-class:: build +.. container:: + + In that same file, you'll find a list bound to the symbol ``TEMPLATES``. + + That list contains one dict with an empty list at the key ``DIRS``. Update + that empty list as shown here: + + .. code-block:: python + + TEMPLATES = [ + { + 'BACKEND': 'django.template.backends.django.DjangoTemplates', + 'DIRS': [os.path.join(BASE_DIR, 'mysite/templates')], + ... + }, + ] + + This will ensure that Django will look in your ``mysite`` project folder + for a directory containing templates. + +.. nextslide:: + +The ``mysite`` project folder does not contain a ``templates`` directory, add one. + +.. rst-class:: build +.. container:: + + Then, in that directory add a new file ``base.html`` and add the following: + + .. code-block:: jinja + + + + + My Django Blog + + +
          +
          + {% block content %} + [content will go here] + {% endblock %} +
          +
          + + + + +Templates in Django +------------------- + +Before we move on, a quick word about Django templates. + +.. rst-class:: build +.. container:: + + We've seen Jinja2 which was "inspired by Django's templating system". + + Basically, you already know how to write Django templates. + + Django templates **do not** allow any python expressions. + + https://docs.djangoproject.com/en/1.9/ref/templates/builtins/ + + +.. nextslide:: Blog Templates + +Our view tries to load ``list.html``. + +.. rst-class:: build +.. container:: + + This template is probably specific to the blog functionality of our site + + It is common to keep shared templates in your project directory and + specialized ones in app directories. + + Add a ``templates`` directory to your ``myblog`` app, too. + + In it, create a new file ``list.html`` and add this: + + +.. nextslide:: ``list.html`` + +.. code-block:: jinja + + {% extends "base.html" %}{% block content %} +

          Recent Posts

          + {% comment %} here is where the query happens {% endcomment %} + {% for post in posts %} +
          +

          {{ post }}

          + +
          + {{ post.text }} +
          +
            + {% for category in post.categories.all %} +
          • {{ category }}
          • + {% endfor %} +
          +
          + {% endfor %} + {% endblock %} + + +.. nextslide:: Template Context + +.. code-block:: python + + context = RequestContext(request, { + 'posts': posts, + }) + body = template.render(context) + +.. rst-class:: build +.. container:: + + Like Jinja2, django templates are rendered by passing in a *context* + + Django's RequestContext provides common bits, similar to the context + provided automatically by Pyramid + + We add our posts to that context so they can be used by the template. + + +.. nextslide:: Return a Response + +.. code-block:: python + + return HttpResponse(body, content_type="text/html") + +.. rst-class:: build +.. container:: + + Finally, we build an HttpResponse and return it. + + This is, fundamentally, no different from the ``stub_view`` just above. + +.. nextslide:: Fix URLs + +We need to fix the url for our blog index page + +.. rst-class:: build +.. container:: + + Update ``urls.py`` in ``myblog``: + + .. code-block:: python + + # import the new view + from myblog.views import list_view + + # and then update the urlconf + url(r'^$', + list_view, #<-- Change this value from stub_view + name="blog_index"), + + Then run your tests again: + + .. code-block:: bash + + (djangoenv)$ ./manage.py test myblog + ... + Ran 3 tests in 0.033s + + OK + + +.. nextslide:: Common Patterns + +This is a common pattern in Django views: + +.. rst-class:: build + +* get a template from the loader +* build a context, usually using a RequestContext +* render the template +* return an HttpResponse + +.. rst-class:: build +.. container:: + + So common in fact that Django provides a shortcut for us to use: + + ``render(request, template[, ctx][, ctx_instance])`` + + +.. nextslide:: Shorten Our View + +Let's replace most of our view with the ``render`` shortcut + +.. code-block:: python + + from django.shortcuts import render # <- already there + + # rewrite our view + def list_view(request): + published = Post.objects.exclude(published_date__exact=None) + posts = published.order_by('-published_date') + context = {'posts': posts} + return render(request, 'list.html', context) + +.. rst-class:: build + +Remember though, all we did manually before is still happening + + +BREAK TIME +---------- + +We've got the front page for our application working great. + +Next, we'll need to provide a view of a detail page for a single post. + +Then we'll provide a way to log in and to navigate between the public part of +our application and the admin behind it. + +If you've fallen behind, the app as it stands now is in our class resources as +``mysite_stage_2`` + + +Our Detail View +--------------- + +Next, let's add a view function for the detail view of a post + +.. rst-class:: build +.. container:: + + It will need to get the ``id`` of the post to show as an argument + + Like the list view, it should only show published posts + + But unlike the list view, it will need to return *something* if an + unpublished post is requested. + + Let's start with the tests in ``views.py`` + + +.. nextslide:: Testing the Details + +Add the following test to our ``FrontEndTestCase`` in ``myblog/tests.py``: + +.. code-block:: python + + def test_details_only_published(self): + for count in range(1, 11): + title = "Post %d Title" % count + post = Post.objects.get(title=title) + resp = self.client.get('/posts/%d/' % post.pk) + if count < 6: + self.assertEqual(resp.status_code, 200) + self.assertContains(resp, title) + else: + self.assertEqual(resp.status_code, 404) + + +.. nextslide:: Run Your Tests + +.. code-block:: bash + + (djangoenv)$ ./manage.py test myblog + Creating test database for alias 'default'... + .F.. + ====================================================================== + FAIL: test_details_only_published (myblog.tests.FrontEndTestCase) + ... + Ran 4 tests in 0.043s + + FAILED (failures=1) + Destroying test database for alias 'default'... + + +.. nextslide:: Let's Fix That Test + +Now, add a new view to ``myblog/views.py``: + +.. code-block:: python + + def detail_view(request, post_id): + published = Post.objects.exclude(published_date__exact=None) + try: + post = published.get(pk=post_id) + except Post.DoesNotExist: + raise Http404 + context = {'post': post} + return render(request, 'detail.html', context) + + +.. nextslide:: Missing Content + +.. code-block:: python + + try: + post = published.get(pk=post_id) + except Post.DoesNotExist: + raise Http404 + +One of the features of the Django ORM is that all models raise a DoesNotExist +exception if ``get`` returns nothing. + +.. rst-class:: build +.. container:: + + This exception is actually an attribute of the Model you look for. + + There's also an ``ObjectDoesNotExist`` for when you don't know which model + you have. + + We can use that fact to raise a Not Found exception. + + Django will handle the rest for us. + + +.. nextslide:: Add the Template + +We also need to add ``detail.html`` to ``myblog/templates``: + +.. code-block:: jinja + + {% extends "base.html" %} + + {% block content %} + Home +

          {{ post }}

          + +
          + {{ post.text }} +
          +
            + {% for category in post.categories.all %} +
          • {{ category }}
          • + {% endfor %} +
          + {% endblock %} + + +.. nextslide:: Hook it Up + +In order to view a single post, we'll need a link from the list view + +.. rst-class:: build +.. container:: + + We can use the ``url`` template tag (like Pyramid's ``request.route_url``): + + .. code-block:: jinja + + {% url '' arg1 arg2 %} + + In our ``list.html`` template, let's link the post titles: + + .. code-block:: jinja + + {% for post in posts %} +
          +

          + {{ post }} +

          + ... + + +.. nextslide:: Fix URLs + +Again, we need to insert our new view into the existing ``myblog/urls.py`` in +``myblog``: + +.. code-block:: python + + # import the view + from myblog.views import detail_view + + url(r'^posts/(?P\d+)/$', + detail_view, #<-- Change this from stub_view + name="blog_detail"), + +.. rst-class:: build small + +:: + + (djangoenv)$ ./manage.py test myblog + ... + Ran 4 tests in 0.077s + + OK + + +.. nextslide:: A Moment To Play + +We've got some good stuff to look at now. Fire up the server + +.. rst-class:: build +.. container:: + + Reload your blog index page and click around a bit. + + You can now move back and forth between list and detail view. + + Try loading the detail view for a post that doesn't exist + + +.. nextslide:: Congratulations + +You've got a functional Blog + +.. rst-class:: build +.. container:: + + It's not very pretty, though. + + We can fix that by adding some css + + This gives us a chance to learn about Django's handling of *static files* + + +Static Files +------------ + +Like templates, Django expects to find static files in particular locations + +.. rst-class:: build +.. container:: + + It will look for them in a directory named ``static`` in any installed + apps. + + They will be served from the url path in the STATIC_URL setting. + + By default, this is ``/static/`` + + To allow Django to automatically build the correct urls for your static + files, you use a special *template tag*:: + + {% static %} + + +.. nextslide:: Add CSS + +I've prepared a css file for us to use. You can find it in the class resources + +.. rst-class:: build +.. container:: + + Create a new directory ``static`` in the ``myblog`` app. + + Copy the ``django_blog.css`` file into that new directory. + + .. container:: + + Next, load the static files template tag into ``base.html`` (this + **must** be on the *first line* of the template): + + .. code-block:: jinja + + {% load staticfiles %} + + .. container:: + + Finally, add a link to the stylesheet using the special template tag: + + .. code-block:: html + + My Django Blog + + + +.. nextslide:: View Your Results + +Reload http://localhost:8000/ and view the results of your work + +.. rst-class:: build +.. container:: + + We now have a reasonable view of the posts of our blog on the front end + + And we have a way to create and categorize posts using the admin + + However, we lack a way to move between the two. + + Let's add that ability next. + + +Global Navigation +----------------- + +We'll start by adding a control bar to our ``base.html`` template: + +.. code-block:: jinja + + + ... + +
          + ... + + +.. nextslide:: Request Context Revisited + +When we set up our views, we used the ``render`` shortcut, which provides a +``RequestContext`` + +.. rst-class:: build +.. container:: + + This gives us access to ``user`` in our templates + + It provides access to methods about the state and rights of that user + + We can use these to conditionally display links or UI elements. Like only + showing the admin link to staff members. + + +.. nextslide:: Login/Logout + +Django also provides a reasonable set of views for login/logout. + +.. rst-class:: build +.. container:: + + The first step to using them is to hook them into a urlconf. + + .. container:: + + Add the following to ``mysite/urls.py``: + + .. code-block:: python + + # add an import at the top + from django.contrib.auth.views import login, logout + + # and update the list of urlconfs + url(r'^', include('myblog.urls')), #<- already there + url(r'^login/$', + login, + {'template_name': 'login.html'}, + name="login"), + url(r'^logout/$', + logout, + {'next_page': '/'}, + name="logout"), + + +.. nextslide:: Login Template + +We need to create a new ``login.html`` template in ``mysite/templates``: + +.. code-block:: jinja + + {% extends "base.html" %} + + {% block content %} +

          My Blog Login

          +
          {% csrf_token %} + {{ form.as_p }} +

          +
          + {% endblock %} + + +.. nextslide:: Submitting Forms + +In a web application, submitting forms is potentially hazardous + +.. rst-class:: build +.. container:: + + Data is being sent to our application from some remote place + + If that data is going to alter the state of our application, we **must** + use POST + + Even so, we are vulnerable to Cross-Site Request Forgery, a common attack + vector. + + +.. nextslide:: Danger: CSRF + +Django provides a convenient system to fight this. + +.. rst-class:: build +.. container:: + + In fact, for POST requests, it *requires* that you use it. + + The Django middleware that does this is enabled by default. + + All you need to do is include the ``{% csrf_token %}`` tag in your form. + + +.. nextslide:: Hooking It Up + +In ``base.html`` make the following updates: + +.. rst-class:: build +.. container:: + + .. code-block:: jinja + + + admin + + logout + + login + + .. container:: + + Finally, in ``settings.py`` add the following: + + .. code-block:: python + + + LOGIN_URL = '/login/' + LOGIN_REDIRECT_URL = '/' + + +.. nextslide:: Forms In Django + +In adding a login view, we've gotten a sneak peak at how forms work in Django. + +.. rst-class:: build +.. container:: + + However, learning more about them is beyond what we can achieve in this + session. + + The form system in Django is quite nice, however. I urge you to + `read more about it`_ + + In particular, you might want to pay attention to the documentation on + `Model Forms`_ + + +.. _read more about it: https://docs.djangoproject.com/en/1.6/topics/forms/ +.. _Model Forms: https://docs.djangoproject.com/en/1.6/topics/forms/modelforms/ + + +Ta-Daaaaaa! +----------- + +So, that's it. We've created a workable, simple blog app in Django. + +.. rst-class:: build +.. container:: + + If you fell behind at some point, the app as it now stands is in our class + resources as ``mysite_stage_3``. + + There's much more we could do with this app. And for homework, you'll do + some of it. + + Then next session, we'll work together as pairs to implement a simple + feature to extend the blog + + +Homework +======== + +.. rst-class:: left + +For your homework this week, we'll fix one glaring problem with our blog admin. + +.. rst-class:: build left +.. container:: + + As you created new categories and posts, and related them to each-other, + how did you feel about that work? + + Although from a data perspective, the category model is the right place for + the ManytoMany relationship to posts, this leads to awkward usage in the + admin. + + It would be much easier if we could designate a category for a post *from + the Post admin*. + + +Your Assignment +--------------- + +You'll be reversing that relationship so that you can only add categories to +posts + +.. rst-class:: build +.. container:: + + Take the following steps: + + 1. Read the documentation about the `Django admin.`_ + 2. You'll need to create a customized `ModelAdmin`_ class for the ``Post`` + and ``Category`` models. + 3. And you'll need to create an `InlineModelAdmin`_ to represent Categories + on the Post admin view. + 4. Finally, you'll need to `exclude`_ the 'posts' field from the form in + your ``Category`` admin. + + +.. _Django admin.: https://docs.djangoproject.com/en/1.9/ref/contrib/admin/ +.. _ModelAdmin: https://docs.djangoproject.com/en/1.9/ref/contrib/admin/#modeladmin-objects +.. _InlineModelAdmin: https://docs.djangoproject.com/en/1.9/ref/contrib/admin/#inlinemodeladmin-objects +.. _exclude: https://docs.djangoproject.com/en/1.9/ref/contrib/admin/#django.contrib.admin.ModelAdmin.exclude + + +.. nextslide:: Pushing Further + +All told, those changes should not require more than about 15 total lines of +code. + +.. rst-class:: build +.. container:: + + The trick of course is reading and finding out which fifteen lines to + write. + + If you complete that task in less than 3-4 hours of work, consider looking + into other ways of customizing the admin. + + +.. nextslide:: Tasks you might consider + +.. rst-class:: build + +* Change the admin index to say 'Categories' instead of 'Categorys'. (hint, the + way to change this has nothing to do with the admin) +* Add columns for the date fields to the list display of Posts. +* Display the created and modified dates for your posts when viewing them in + the admin. +* Add a column to the list display of Posts that shows the author. For more + fun, make this a link that takes you to the admin page for that user. +* For the biggest challenge, look into `admin actions`_ and add an action to + the Post admin that allows you to publish posts in bulk from the Post list + display + +.. _admin actions: https://docs.djangoproject.com/en/1.6/ref/contrib/admin/actions/ diff --git a/source/presentations/session08.rst.norender b/source/presentations/session08.rst.norender deleted file mode 100644 index 2fdfc768..00000000 --- a/source/presentations/session08.rst.norender +++ /dev/null @@ -1,140 +0,0 @@ -Internet Programming with Python -================================ - -.. image:: img/django-pony.png - :align: left - :width: 50% - -Session 8: Extending Django - -.. class:: intro-blurb right - -Wherein we extend our Django blog app. - -.. class:: image-credit - -image: http://djangopony.com/ - - -Last Week ---------- - -Last week, we created a nice, simple Django microblog application. - -.. class:: incremental - -Over the week, as your homework, you made some modifications to improve how it -works. - -.. class:: incremental - -There's still quite a bit more we can do to improve this application. - -.. class:: incremental - -And today, that's what we are going to do. - - -Preparation ------------ - -In order for this to work properly, we'll need to have a few things in place. - -.. container:: incremental small - - First, we'll start from a canonical copy of the microblog. Make a fork of - the following repository to your github account: - - .. code-block:: - :class: small - - https://github.com/cewing/django-microblog - -.. container:: incremental small - - Then, clone that repository to your local machine: - - .. code-block:: bash - :class: small - - $ git clone https://github.com//django-microblog.git - or - $ git clone git@github.com:/django-microblog.git - - -Connect to Your Partner ------------------------ - -Finally, you'll want to connect to your partner's repository, so that you can -each work on your own laptop and still share the changes you make. - -.. container:: incremental small - - First, add your partner's repository as ``upstream`` to yours: - - .. code-block:: bash - :class: small - - $ git remote add upstream https://github.com//django-microblog.git - or - $ git remote add upstream git@github.com:/django-microblog.git - -.. container:: incremental small - - Then, fetch their copy so that you can easily merge their changes later: - - .. code-block:: bash - :class: small - - $ git fetch upstream - - -While You Work --------------- - -.. class:: small - -Now, when you switch roles during your work, here's the workflow you can use: - -.. class:: small - -1. The current driver commits all changes and pushes to their repository: - -.. code-block:: bash - :class: small - - $ git commit -a -m "Time to switch roles" - $ git push origin master - -.. class:: small - -2. The new driver fetches and merges changes made upstream: - -.. code-block:: bash - :class: small - - $ git fetch --all - $ git branch -a - * master - remotes/origin/master - remotes/upstream/master - $ git merge upstream/master - -.. class:: small - -3. The new driver continues working from where their partner left off. - - -Homework --------- - -For this week's homework, you will need to install the Zope Object Database -(ZODB) - -Instructions for this `may be found here`_. - -.. _may be found here: https://github.com/UWPCE-PythonCert/training.python_web/blob/master/resources/common/zodb-install-instructions.rst - -This is not trivial work. Please be sure to start early in the week so if -there is trouble, you'll be able to recover. - diff --git a/source/presentations/session09.rst b/source/presentations/session09.rst new file mode 100644 index 00000000..384738ec --- /dev/null +++ b/source/presentations/session09.rst @@ -0,0 +1,178 @@ +********** +Session 09 +********** + +.. figure:: /_static/django-pony.png + :align: center + :width: 60% + + image: http://djangopony.com/ + +Extending Django +================ + +.. rst-class:: large + +Wherein we extend our Django blog app. + + +Last Week +--------- + +Last week, we created a nice, simple Django microblog application. + +.. rst-class:: build +.. container:: + + Over the week, as your homework, you made some modifications to improve how + it works. + + There's still quite a bit more we can do to improve this application. + + And today, that's what we are going to do. + + +Preparation +----------- + +In order for this to work properly, we'll need to have a few things in place. + +.. rst-class:: build +.. container:: + + **For the time being, all these actions should only be taken by one + partner**. + + First, we'll start from a canonical copy of the microblog. Make a fork of + the following repository to your github account:: + + https://github.com/cewing/djangoblog_uwpce.git + + Then, clone that repository to your local machine: + + .. code-block:: bash + + $ git clone https://github.com//djangoblog_uwpce.git + + +Connect to Your Partner +----------------------- + +Finally, you'll need to add your partner as a collaborator for your new +repository. + +.. rst-class:: build +.. container:: + + Go to the *settings* for your repository. + + Click the *collaborators* tab on the left side of the window (you'll need + to enter your github password). + + Look up your partner by email address or github username. + + Add them. + + Then your partner can clone the repository to their desktop too. + +While You Work +-------------- + +Now, when you switch roles during your work, here's the workflow you can use: + +.. rst-class:: build +.. container:: + + .. container:: + + 1. The current driver commits all changes and pushes to their repository: + + .. code-block:: bash + + $ git commit -a -m "Time to switch roles" + $ git push origin master + + .. container:: + + 2. The new driver gets the changes: + + .. code-block:: bash + + $ git pull origin master + + 3. The new driver continues working from where their partner left off. + 4. PROFIT..... + +Homework +======== + +Next week, we will deploy your Django application to a server. + +.. rst-class:: build +.. container:: + + To help illustrate the full set of tools at our disposal, we'll go a bit + overboard for this. + + We'll be setting up an HTTP server, proxying to a WSGI server serving your + Django app. + + We'll do this all "In the cloud" using Amazon's `AWS`_ service. + + Before class starts, you'll need to accomplish a few non-programming tasks + +.. _AWS: http://aws.amazon.com/free + +Sign Up For AWS +--------------- + +Begin by going to the `AWS homepage`_ and clicking on the large, yellow button +that reads "Sign In to the Console". + +.. rst-class:: build +.. container:: + + On the sign-in page that appears, click the radio button for 'I am a new + user', fill in your email address, and then click through to begin the + sign-up process. + + You will be required to provide credit card information. + + If you are still eligible for the AWS free tier, you will not incur any + charges for work you do in this class. + +.. _AWS homepage: http://aws.amazon.com + + +Set Up an IAM User +------------------ + +Once you've signed up for an account take the following actions: + +* `Create an IAM user`_ and place them in a group with Power User access. (Search for PowerUser when selecting a policy for your group). + + * Set up Security Credentials for that IAM user. + * Save these Security Credentials in a safe place so you can use them for class. + +.. _Create an IAM user: http://docs.aws.amazon.com/IAM/latest/UserGuide/IAMBestPractices.html + +Prepare for Login +----------------- + +* `Create a Keypair`_ + + * Choose the 'US West (Oregon)' region since it's geographically closest to you. + * When you download your private key, save it to ~/.ssh/pk-aws.pem + * Make sure that the private key is secure and useable by doing the following command + + * ``$ chmod 400 ~/.ssh/pk-aws.pem`` + +* `Create a custom security group`_ + + * The security group should be named 'ssh-access' + * Add one custom TCP rule + * allow port 22 + * allow addresses 0.0.0.0/0 + +.. _Create a Keypair: http://docs.aws.amazon.com/gettingstarted/latest/wah/getting-started-create-key-pair.html +.. _Create a custom security group: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html diff --git a/source/presentations/session09.rst.norender b/source/presentations/session09.rst.norender deleted file mode 100644 index b79b0e88..00000000 --- a/source/presentations/session09.rst.norender +++ /dev/null @@ -1,1712 +0,0 @@ -Internet Programming with Python -================================ - -.. image:: img/pyramid-medium.png - :align: left - :width: 50% - -Session 9: Intro To Pyramid - -.. class:: intro-blurb right - -| The flexible framework. -| Totally not built by aliens. - - -What is Pyramid? ----------------- - -A Web Framework - -.. class:: incremental - -"Its primary job is to make it easier for a developer to create an arbitrary -web application" - -.. class:: incremental - -Makes as few decisions as possible for you. - -.. class:: incremental - -Allows *you* to make decisions, and provides tools to support you when you do - -.. class:: incremental - -"Pay only for what you eat" - - -Why is Pyramid? ---------------- - -Micro-frameworks are great for lightweight apps - -.. class:: incremental - -Micro-frameworks do not scale up or change specs easily - -.. class:: incremental - -Full-stack frameworks have lots of opinions. *Bending* them can be difficult. - -.. class:: incremental - -Pyramid can build a lightweight app easily, but it can also scale and bend - - -History - Zope and Repoze -------------------------- - -Many of the core developers of Pyramid started as Zope developers. - -.. class:: incremental - -Born in 1996, Zope was the first Python web framework, and possibly the first -in any language. - -.. class:: incremental - -After 14 years, the developers of Zope had seen and learned *a lot*. - -.. class:: incremental - -Repoze was a short-lived (2008-2010) framework intended to embody the lessons -learned from Zope. - - -History - Pylons ----------------- - -Pylons was released in 2005. - -.. class:: incremental - -It was among the first frameworks to fully embrace the WSGI specification. - -.. class:: incremental - -The creators of Pylons built WebOb (abstracted HTTP request and response -objects). - -.. class:: incremental - -This forms the foundation of Pylons much as Werkzeug is the foundation of -Flask. - - -History - 2010 --------------- - -In 2010, the authors of Repoze and Pylons got together and made an unusual -decision. - -.. class:: incremental - -Why duplicate efforts when there are already so many other frameworks? - -.. class:: incremental - -Repoze was re-named 'Pyramid' and the 'Pylons Project' was born to shepherd -this new combined project. - - -Implications ------------- - -Pylons was a framework predicated largely on relational persistence and URL -Dispatch. - -.. class:: incremental - -Zope/Repoze was based on the ZODB and Object Traversal. - -.. class:: incremental - -Each of these approaches has strengths and weaknesses. - -.. class:: incremental - -Pyramid supports using neither, both and even combinations of the two. - - -Relational DB / URL Dispatch ----------------------------- - -You've seen this before, both in Flask and Django - -.. class:: incremental - -SQLite3, the Django ORM, both are examples of relational persistence models - -.. class:: incremental - -Routes/urlpatterns, both are examples of URL Dispatch - -.. class:: incremental - -Pyramid can work this way too. SQLAlchemy, Route-based views. - -.. class:: incremental - -Been there, done that. Let's see something else. - - -ZODB ----- - -ORMs allow developers to pretend that Objects are like DB Tables. - -.. class:: incremental - -But Objects are *not* tables, so there's a `conceptual mismatch -`_ between -the two. - -.. class:: incremental - -The ZODB is an *object store*, rather than a relational database. - -.. class:: incremental - -If your data is best represented by *heterogenous* objects, it's a better -persistence solution. - - -Traversal ---------- - -In URL Dispatch, the ``PATH`` is a *virtual* construct. - -.. class:: incremental - -In our Django app ``/admin/myblog/post/13/`` doesn't map to any series of -*real* locations. - -.. class:: incremental - -This is unlike a filesystem where ``/usr/local/bin/python`` points to a *real* -location. - -.. class:: incremental - -When you use the ``cd`` command to move from place to place in a filesystem, -that is *traversal* - - -Object Graphs -------------- - -In Python, objects can *contain* other objects. - -.. class:: incremental - -Using *dict*-like structures, you can build a *graph* of objects: - -.. class:: incremental - -:: - - Family - ├── Parents - │ ├── Cris - │ ├── Kristina - ├── Children - │ ├── Kieran - │ ├── Finnian - - -We Got Both Directions ----------------------- - -``__getitem__`` allows movement from *container* to *contained* - -.. container:: incremental - - What if the *contained* can keep track of its *container*? - - .. code-block:: python - :class: small - - >>> class node(dict): - ... __parent__ = None - ... def __init__(self, parent=None): - ... self.__parent__ = parent - ... - >>> x = node() - >>> x['y'] = node(x) - >>> y = x['y'] - >>> y.__parent__ == x - True - - -Traversal - Path Lookup ------------------------ - -You can *traverse* across the object graph by treating a URL as a series of -*object names* - -.. class:: incremental small - -:: - - http://family/parents/cris -> family['parents']['cris'] - -.. class:: incremental - -If you have more names than objects, the remainder can be passed to the final -object as data: - -.. class:: incremental small - -:: - - http://family/parents/cris/edit -> subpath = /edit - http://family/parents/cris/next/steps -> subpath = /next/steps - -.. class:: incremental - -The subpath can be used to find object methods or views - -Preparation ------------ - -You should at this point have a virtualenv in which you have installed the -ZODB. - -.. class:: incremental - -Now, let's install pyramid too. - -.. container:: incremental - - In your terminal, change directories to where you build that virtualenv and - activate it: - - .. class:: small - - :: - - $ cd /path/to/right/place - $ source pyramidenv/bin/activate - - C:\> pyramidenv\Scripts\activate - - -Installation ------------- - -Next, install Pyramid and the extras we'll be using: - -.. class:: incremental small - -:: - - (pyramidenv)$ pip install pyramid - ... - (pyramidenv)$ pip install docutils nose coverage - ... - (pyramidenv)$ pip install pyramid_zodbconn pyramid_tm - ... - (pyramidenv)$ pip install pyramid_debugtoolbar - -.. class:: incremental - -These tools will allow us to manage ZODB connections, debug our app, and run -cool tests. - - -Required Setup --------------- - -In Django ``startproject`` and ``startapp`` gave us the boilerplate we needed. - -.. class:: incremental - -Pyramid uses what it calls *scaffolds* for the same purpose. - -.. class:: incremental - -When you installed it, a new ``pcreate`` command was generated in your -virtualenv. - -.. container:: incremental - - Let's use it: - - .. class:: small - - :: - - (pyramidenv)$ pcreate -s zodb wikitutorial - ... - - -Scaffolds and Opinions ----------------------- - -When you ran ``pcreate -s zodb wikitutorial`` you invoked the *zodb scaffold* - -.. class:: incremental - -Pyramid the framework is highly un-opinionated. - -.. class:: incremental - -*Scaffolds*, conversely, can be quite opinionated. The one we used has chosen -our persistence mechanism (ZODB) and how we will reach our code (Traversal). - -.. class:: incremental - -You do not have to use *scaffolds* to start a project, but it can help. - - -Project Layout --------------- - -Running ``pcreate`` has set up a file structure for us: - -.. class:: small - -:: - - wikitutorial/ - CHANGES.txt - development.ini - MANIFEST.in - production.ini - README.txt - setup.cfg - setup.py - wikitutorial/ - __init__.py - models.py - static/ - templates/ - tests.py - views.py - - -Similarities to Django ----------------------- - -Our project is organized with an outer *project* folder and an inner *package* -folder (see the ``__init__.py``?) - -.. class:: incremental - -The name of that outer directory is not really important. - -.. class:: incremental - -Our inner *package* folder has a models.py, tests.py and views.py module - -.. class:: incremental - -Our inner *package* folder has a ``static/`` and ``templates/`` directory - - -Differences from Django ------------------------ - -Our *outer* module has a ``setup.py`` file, which allows it to be installed -with ``pip`` or ``easy_install`` - -.. class:: incremental - -There is no ``manage.py`` file. Pyramid commands are console scripts (look in -*pyramidenv/bin*). - -.. class:: incremental - -There is nothing magical in Pyramid about the name of the ``models.py`` -module. - -.. class:: incremental - -There is nothing magical in Pyramid about the names of the ``static/`` or -``templates/`` directories. - - -Pyramid System Configuration ----------------------------- - -Pyramid keeps configuration intended for an entire installation in ``.ini`` -files at the top of a project. - -.. class:: incremental - -When you deploy an app to some wsgi server, you'll reference one of these files - -.. class:: incremental - -Settings there affect the environment of all apps that are running in that -wsgi server. - -.. class:: incremental - -Like Django's ``settings.py``, but **not** python. - - -INI format ----------- - -INI-style files have a particular format. - -.. class:: incremental - -Individual sections are marked by ``[SECTION_NAMES]`` in square brackets - -.. class:: incremental - -Each section will contain ``name = value`` pairs of settings. - -.. class:: incremental - -INI files are parsed using the Python `ConfigParser -`_ module. - -.. code-block:: python - :class: small incremental - - {'SECTION_NAME': {'name': 'value', ...}, ...} - - -Pyramid is Python ------------------ - -Running a Pyramid application is really just like running a Python module. In -the ``__init__.py`` file of your app *package*, you'll find a ``main`` -function: - -.. code-block:: python - :class: small incremental - - def main(global_config, **settings): - """ This function returns a Pyramid WSGI application. - """ - config = Configurator(root_factory=root_factory, - settings=settings) - config.add_static_view('static', 'static', cache_max_age=3600) - config.scan() - return config.make_wsgi_app() - -.. class:: incremental - -Let's take a closer look at this, line by line. - - -INI Configuration ------------------ - -.. code-block:: python - :class: small - - def main(global_config, **settings): - -.. class:: incremental - -Arguments passed to ``main`` are configuration from ``.ini``. - -.. class:: incremental - -``global_config`` is a dictionary of settings in [DEFAULT] - -.. class:: incremental - -``settings`` will be the name-value pairs for your app. - -.. container:: incremental - - ``[app:]`` sections are mapped to apps by the ``use`` setting - - .. code-block:: ini - :class: small - - [app:main] - use = egg:wikitutorial - - -App Configuration ------------------ - -.. code-block:: python - :class: small - - config = Configurator(root_factory=root_factory, - settings=settings) - config.add_static_view('static', 'static', cache_max_age=3600) - config.scan() - -.. class:: incremental - -Pyramid configuration is done by the ``Configurator`` class. - -.. class:: incremental - -Configuration can be *imperative* (function calls) or *declarative* -(decorators) - -.. class:: incremental - -Either way, ``.scan()`` sets it all up and reports errors. - -.. class:: incremental - -Read more in `the pyramid.config documentation -`_ - - -WSGI Hookup ------------ - -.. code-block:: python - :class: small - - return config.make_wsgi_app() - -.. class:: incremental - -Like Django and Flask, Pyramid runs in a WSGI world. - -.. class:: incremental - -``.make_wsgi_app()`` returns a ``Router`` object for your app. - -.. container:: incremental - - ``Router`` has the following ``__call__`` method: - - .. code-block:: python - :class: small - - def __call__(self, environ, start_response): - request = self.request_factory(environ) - response = self.invoke_subrequest(request, use_tweens=True) - return response(request.environ, start_response) - -.. class:: incremental - -Familiar, no? - - -The Application Root --------------------- - -The ``Configurator`` constructor takes a ``root_factory`` kwarg. - -.. class:: incremental - -This *callable* returns something to handle dispatching requests. - -.. class:: incremental - -The default root factory uses URL Dispatch. - -.. class:: incremental - -We want to use Traversal for our app, so we provide one. - - -Our Root Factory ----------------- - -.. code-block:: python - :class: small - - from pyramid_zodbconn import get_connection - from .models import appmaker - - def root_factory(request): - conn = get_connection(request) - return appmaker(conn.root()) - -.. class:: incremental - -``get_connection`` returns a connection to the ZODB. - -.. class:: incremental - -The ``root`` of this connection is then passed to ``appmaker`` - -.. class:: incremental - -This is another factory method that returns the app root. - -.. class:: incremental - -So what exactly does ``appmaker`` do? - - -The appmaker ------------- - -.. code-block:: python - :class: small - - def appmaker(zodb_root): - if not 'app_root' in zodb_root: - app_root = MyModel() - zodb_root['app_root'] = app_root - import transaction - transaction.commit() - return zodb_root['app_root'] - -.. class:: incremental - -Remember, the ZODB is an *object store*, dict-like. - -.. class:: incremental - -We look for an ``app_root`` inside this *container* - -.. class:: incremental - -If there is none, we build one and put it there. - -.. class:: incremental - -This simple Python object will manage *Traversal* for our app. - - -Install Our App ---------------- - -Our app is, in fact, a Python package. - -.. class:: incremental - -In order for us to use it, we must *install* it. - -.. class:: incremental - -``setup.py`` allows us to do this: ``python setup.py install`` **BUT** - -.. class:: incremental - -Install will make a copy of our code and use that. - -.. class:: incremental - -We don't want that, since updates we make here would not be picked up. - -*Develop* Installation ----------------------- - -We can use an alternate method called ``develop``. - -.. class:: incremental - -This will install a *pointer* to our package, but leave the code here. - -.. class:: incremental - -In a terminal, move to the ``wikitutorial`` *project* folder (find -``development.ini``) and ``develop`` the app: - -.. class:: small incremental - -:: - - (pyramidenv)$ cd wikitutorial - (pyramidenv)$ python setup.py develop - - -See It Live ------------ - -Use the ``pserve`` command installed by pyramid to serve our app: - -.. class:: small - -:: - - (pyramidenv)$ pserve development.ini - Starting server in PID 16698. - serving on http://0.0.0.0:6543 - -.. class:: incremental - -This brings up a new *wsgi server* provided by ``waitress`` serving our app. - -.. class:: incremental - -Load http://localhost:6543 and view your app root. - - -Why is it Pretty? ------------------ - -We should be looking at an instance of MyModel: - -.. code-block:: python - :class: small - - class MyModel(PersistentMapping): - __parent__ = __name__ = None - -.. class:: incremental - -What makes it look like this? - -.. class:: incremental - -The secret sauce lies in *view configuration* - - -Pyramid Views -------------- - -.. code-block:: python - :class: small - - from pyramid.view import view_config - from .models import MyModel - - @view_config(context=MyModel, renderer='templates/mytemplate.pt') - def my_view(request): - return {'project': 'wikitutorial'} - -.. class:: incremental - -Pyramid views can be configured with the ``@view_config()`` decorator. - -.. class:: incremental - -Or call ``config.add_view()`` method in your app ``main``. - -.. class:: incremental - -``config.scan()`` in ``main`` picks up all config decorators. - - -View Config - Predicates ------------------------- - -View configuration takes many arguments. Here we use two. - -.. class:: incremental - -``context`` determines the *type* of object to which this view can be applied - -.. class:: incremental - -It's an example of a *predicate* argument - -.. class:: incremental - -*Predicates* place restrictions on how and when a view is used. - -.. class:: incremental - -Read more about predicates in `view configuration -`_ - - -View Config - Renderers ------------------------ - -Pyramid separates the concerns of *view* and *renderer* - -.. class:: incremental - -So far, *views* prepare a data context **and** render it - -.. class:: incremental - -In Pyramid, the *view* only prepares the data to be rendered - -.. class:: incremental - -A ``renderer`` transforms this to something suitable for an HTTP response. - -.. class:: incremental - -In this case, ``renderer`` is a template that will return HTML - - -View Config - Summary ---------------------- - -In summary, then, our view configuration: - -.. class:: incremental - -* checks to see that we have traversed to an instance of ``MyModel`` -* calls the ``my_view`` function, which returns a simple dictionary -* passes that dictionary to the ``mytemplate`` template -* the template is rendered and returned as the body of an HTTP response. - -.. class:: incremental - -And that is how we end up looking at that very pretty page. - - -Break Time ----------- - -So far: - -.. class:: incremental - -* we've taken a look at where Pyramid comes from -* we've seen how it is like and unlike other frameworks we've seen. -* we've met the ZODB *object store* and talked about how it differs from a - database -* we've learned about *traversal* and how it differs from URL dispatch -* we've set up a Pyramid app using both -* we've looked at how the example code in that application works. - -.. class:: incremental - -Next, we'll start working on building our app, starting with Models. - - -Before We Begin ---------------- - -In your *package* directory you should see a file: ``Data.fs``. - -.. class:: incremental - -We are going to be starting over, so let's clear it. - -.. class:: incremental - -Make sure Pyramid is not running. - -.. class:: incremental - -Delete Data.fs. It will be re-created as needed. - -.. class:: incremental - -You can also delete Data.fs.* (.tmp, .index, .lock) - - -Wiki Models ------------ - -First, we want a *wiki* class to serve as our app *root*. - -.. class:: incremental - -We also need a *page* class representing a wiki page. - -.. class:: incremental - -This will be the type we view when we are looking at the wiki. - -.. class:: incremental - -These two classes will need to be stored in our ZODB - -.. class:: incremental - -This means we need to talk about *persistence*. - - -Persistence Magic ------------------ - -In SQL, data *about* an object is written to tables. - -.. class:: incremental - -In the ZODB, the *object itself* is saved in the database. - -.. class:: incremental - -The ZODB provides special classes that help us with this. - -.. class:: incremental - -Instances of these classes are able to know when they've been changed. - -.. class:: incremental - -When a ZODB transaction is committed, all changed objects are saved. - - -Persistent Base Classes ------------------------ - -We'll be using two of these classes in our wiki: - -.. class:: incremental - -* **Persistent** - automatically saves changes to class attributes - -* **PersistentMapping** - like a *dictionary*, saves changes to itself *and - its keys and values*. - -.. class:: incremental small - -Other structures like lists and B-Trees are also available, but we wont use -them here. - -.. class:: incremental - -By subclassing these, we automatically gain persistence. - - -Traversal Magic ---------------- - -Our wiki system will use *traversal* dispatch - -.. class:: incremental - -Two object attributes support *traversal*: - -.. class:: incremental - -* ``__name__`` (who am I) -* ``__parent__`` (where am I) - -.. class:: incremental - -Every object in a traversal-based system **must** provide these two -attributes. - -.. class:: incremental - -The *root* object will set these to ``None``. - - -The Wiki Class --------------- - -Open ``models.py`` from our ``wikitutorial`` *package* directory. - -.. class:: incremental - -First, delete the ``MyModel`` class. We won't need it. - -.. class:: incremental - -Add the following in its place: - -.. code-block:: python - :class: incremental - - class Wiki(PersistentMapping): - __name__ = None - __parent__ = None - - -The Page Class --------------- - -To that same file (models.py) add one import and a second class definition: - -.. code-block:: python - - from persistent import Persistent - - class Page(Persistent): - def __init__(self, data): - self.data = data - -.. class:: incremental - -What about ``__name__`` and ``__parent__``? - -.. class:: incremental - -We'll add those to each instance when we create it. - - -Update Appmaker ---------------- - -Update ``appmaker`` for our new models: - -.. code-block:: python - - def appmaker(zodb_root): - if not 'app_root' in zodb_root: - app_root = Wiki() - frontpage = Page('This is the front page') - app_root['FrontPage'] = frontpage - frontpage.__name__ = 'FrontPage' - frontpage.__parent__ = app_root - zodb_root['app_root'] = app_root - import transaction - transaction.commit() - return zodb_root['app_root'] - - -A Last Bit of Cleanup ---------------------- - -We've deleted the ``MyModel`` class. - -.. class:: incremental - -But we still have *views* that reference the class. - -.. container:: incremental - - Open ``views.py`` and delete everything except the first import - - .. code-block:: python - :class: small - - from pyramid.view import view_config - -.. class:: incremental - -Next come tests for our new models. - - -Test the Wiki Model -------------------- - -Open ``tests.py`` from the *package* directory. Delete the ``ViewTests`` -class and replace it with the following: - -.. code-block:: python - :class: small - - class WikiModelTests(unittest.TestCase): - - def _getTargetClass(self): - from wikitutorial.models import Wiki - return Wiki - - def _makeOne(self): - return self._getTargetClass()() - - def test_constructor(self): - wiki = self._makeOne() - self.assertEqual(wiki.__parent__, None) - self.assertEqual(wiki.__name__, None) - - -Test the Page Model -------------------- - -Add the following test class as well: - -.. code-block:: python - :class: small - - class PageModelTests(unittest.TestCase): - - def _getTargetClass(self): - from wikitutorial.models import Page - return Page - - def _makeOne(self, data=u'some data'): - return self._getTargetClass()(data=data) - - def test_constructor(self): - instance = self._makeOne() - self.assertEqual(instance.data, u'some data') - - -Test Appmaker -------------- - -One more test class: - -.. code-block:: python - :class: small - - class AppmakerTests(unittest.TestCase): - - def _callFUT(self, zodb_root): - from wikitutorial.models import appmaker - return appmaker(zodb_root) - - def test_initialization(self): - root = {} - self._callFUT(root) - self.assertEqual(root['app_root']['FrontPage'].data, - 'This is the front page') - - -A Side Note ------------ - -Note that there are *few* module level imports in ``tests.py`` - -.. class:: incremental - -Also note that each TestCase has a helper method to import the class it will -test. - -.. class:: incremental - -This is unusual, but it reflects Pyramid `testing best practices -`_ - -.. class:: incremental - -In short, the idea is to prevent import problems from breaking *all* your -tests. - - -Run our Tests -------------- - -Finally, let's run our tests:: - - (pyramidenv)$ python setup.py test - ... - Ran 3 tests in 0.000s - - OK - -.. class:: incremental - -We can also run tests to tell us our code-coverage: - -.. class:: incremental small - -:: - - (pyramidenv)$ nosetests --cover-package=tutorial --cover-erase\ - --with-coverage - - -Preparing for Views -------------------- - -The ``data`` attribute of our ``Page`` model holds the text of the page. - -.. class:: incremental - -We'll use ReStructuredText, which can be rendered to HTML - -.. class:: incremental - -Rendering is provided by a python package called ``docutils``. - -.. class:: incremental - -Our application is a python package, and can declare its own dependencies. - -.. class:: incremental - -We need to add the ``docutils`` package to this list. - - -Package Dependencies --------------------- - -Open the ``setup.py`` file from our *project* directory. Add ``docutils`` to -the list ``requires``: - -.. code-block:: python - - requires = [ - 'pyramid', - 'pyramid_zodbconn', - 'transaction', - 'pyramid_tm', - 'pyramid_debugtoolbar', - 'ZODB3', - 'waitress', - 'docutils', # <- ADD THIS - ] - - -Complete the Change -------------------- - -Changes to ``setup.py`` always require a re-install:: - - (pyramidenv)$ python setup.py develop - -.. class:: incremental - -You'll see a whole bunch of stuff flicker by. In it will be a reference to -``Searching for docutils``. - - -Adding Views ------------- - -We are ready to add views now. We'll need: - -.. class:: incremental - -* A view of the Wiki itself, which redirects to the front page. -* A view of an existing Page -* A view that allows us to *add* a new Page -* A view that allows us to *edit* an existing Page - -.. class:: incremental - -As we move forward, we'll be writing tests first, then building the code to -pass them. - - -Testing the Wiki View ---------------------- - -We want our wiki to automaticall redirect to ``FrontPage``. - -.. container:: incremental - - Add this new TestCase to ``tests.py``: - - .. code-block:: python - :class: small - - class WikiViewTests(unittest.TestCase): - - def test_redirect(self): - from wikitutorial.views import view_wiki - context = testing.DummyResource() - request = testing.DummyRequest() - response = view_wiki(context, request) - self.assertEqual(response.location, - 'http://example.com/FrontPage') - - -Run The Tests -------------- - -.. class:: small - -:: - - (pyramidenv)$ python setup.py test - ... - - ====================================================================== - ERROR: test_redirect (wikitutorial.tests.WikiViewTests) - ---------------------------------------------------------------------- - Traceback (most recent call last): - File "/path/to/wikitutorial/wikitutorial/tests.py", line 51, in test_redirect - from wikitutorial.views import view_wiki - ImportError: cannot import name view_wiki - - ---------------------------------------------------------------------- - Ran 4 tests in 0.001s - - FAILED (errors=1) - - -Adding view_wiki ----------------- - -Open ``views.py`` again. Add the following: - -.. code-block:: python - :class: small - - from pyramid.httpexceptions import HTTPFound - from pyramid.view import view_config # <- ALREADY THERE - - @view_config(context='.models.Wiki') - def view_wiki(context, request): - return HTTPFound(location=request.resource_url(context, - 'FrontPage')) - -.. container:: incremental - - And re-run tests: - - .. class:: small - - :: - - (pyramidenv)$ python setup.py test - ... - Ran 4 tests in 0.001s - OK - - -Some Notes ----------- - -Note that ``@view_config`` has no ``renderer`` argument. - -.. class:: incremental - -It will never be shown, so there's no need - -.. class:: incremental - -Instead, it returns ``HTTPFound``, (``302 Found``), which requires a -``location`` - -.. class:: incremental - -The ``.resource_url()`` method of a ``request`` object builds a URL for us. - - -A Page View ------------ - -Our view of a page will need to accomplish a few things: - -.. class:: incremental - -* convert the Page ``data`` to HTML -* make WikiWords in the HTML into appropriate links -* provide a url for editing itself - -.. class:: incremental - -Let's test and implement these features one at a time - - -Test HTML Rendering -------------------- - -Add the following new TestCase to ``tests.py`` - -.. code-block:: python - :class: small - - class PageViewTests(unittest.TestCase): - def _callFUT(self, context, request): - from wikitutorial.views import view_page - return view_page(context, request) - - def test_it(self): - wiki = testing.DummyResource() - context = testing.DummyResource(data='Hello CruelWorld IDoExist') - context.__parent__ = wiki - context.__name__ = 'thepage' - request = testing.DummyRequest() - info = self._callFUT(context, request) - self.assertTrue('
          ' in info['content']) - for word in context.data.split(): - self.assertTrue(word in info['content']) - - -Run The Tests -------------- - -Verify that you now have five, and that one is failing - -.. class:: incremental - -Our tests is relying on an artifact of how docutils builds HTML - -.. class:: incremental - -It makes it a weak tests, but okay for illustrative purposes. - -.. class:: incremental - -Now, let's get it passing - - -Start view_page ---------------- - -Add this code to ``views.py``: - -.. code-block:: python - :class: small - - # an import - from docutils.core import publish_parts - - # and a method - @view_config(context='.models.Page', renderer='templates/view.pt') - def view_page(context, request): - content = publish_parts( - context.data, writer_name='html')['html_body'] - return dict(page=context, content=content) - -.. class:: small incremental - -:: - - (pyramidenv)$ python setup.py test - ... - Ran 5 tests in 0.143s - OK - - -Test Link Building ------------------- - -Add the following to our test: - -.. code-block:: python - :class: small - - def test_it(self): - wiki = testing.DummyResource() - wiki['IDoExist'] = testing.DummyResource() #<- add this - context = testing.DummyResource(data='Hello CruelWorld IDoExist') - #... - # Add the following loop and assertion - for url in (request.resource_url(wiki['IDoExist']), - request.resource_url(wiki, 'add_page', 'CruelWorld')): - self.assertTrue(url in info['content']) - -.. class:: small incremental - -:: - - (pyramidenv)$ python setup.py test - Ran 5 tests in 0.108s - FAILED (failures=1) - - -Finding WikiWords ------------------ - -We'll use a regular expression to find WikiWords in our page data - -.. container:: incremental - - Add the following to ``views.py``: - - .. code-block:: python - :class: small - - # one import - import re - - # and one module constant - WIKIWORDS = re.compile(r"\b([A-Z]\w+[A-Z]+\w+)") - -.. class:: incremental - -Now, we use this to build a curried function that converts WikiWords to links - - -Converting WikiWords --------------------- - -.. code-block:: python - :class: small - - # in views.py - def view_page(context, request): - wiki = context.__parent__ - - def check(match): - word = match.group(1) - if word in wiki: - page = wiki[word] - view_url = request.resource_url(page) - return '%s' % (view_url, word) - else: - add_url = request.application_url + '/add_page/' + word - return '%s' % (add_url, word) - - content = publish_parts( - context.data, writer_name='html')['html_body'] - content = WIKIWORDS.sub(check, content) #<- add this line - return #... <- this already exists - - -Check Your Progress -------------------- - -Tests should now be five for five again. - - -Test Edit Link --------------- - -Finally, we need to verify that ``view_page`` also returns a link to edit -*this* page. - -.. container:: incremental - - Add this to our test: - - .. code-block:: python - :class: small - - def test_it(self): - #... - self.assertEqual(info['edit_url'], - 'http://example.com/thepage/edit_page') - -.. class:: small incremental - -:: - - (pyramidenv)$ python setup.py test - Ran 5 tests in 0.110s - FAILED (errors=1) - - -Return Edit Link ----------------- - -Update ``view_page``: - -.. code-block:: python - :class: small - - def view_page(context, request): - #... - content = WIKIWORDS.sub(check, content) #<- already there - edit_url = request.resource_url(context, 'edit_page') #<- add - return dict(page=context, - content=content, - edit_url = edit_url) - -.. class:: small incremental - -:: - - (pyramidenv)$ python setup.py test - Ran 5 tests in 0.110s - OK - - -What's in the ZODB? -------------------- - -We can inspect the database directly. - -.. class:: incremental - -Start an interactive session with: - -:: - - (pyramidenv)$ pshell development.ini - ... - >>> root - {'FrontPage': } - -.. class:: incremental small - -:: - - >>> root['FrontPage'].data - 'This is the front page' - -.. class:: incremental small - -:: - - >>> root['FrontPage'].__dict__ - {'__name__': 'FrontPage', 'data': 'This is the front page', '__parent__': {'FrontPage': }} - - - - -Adding Templates ----------------- - -What is the page template name for ``view_page``? - -.. class:: incremental - -Create ``view.pt`` in your ``templates`` directory. - -.. class:: incremental - -Also copy the file ``base.pt`` from the class resources. - -.. class:: incremental - -Pyramid can use a number of different templating engines. - -.. class:: incremental - -We'll be using Chameleon, which also supports extending other templates. - - -The view.pt Template --------------------- - -Type this code into your ``view.pt`` file: - -.. code-block:: xml - - - - -
          - Page text goes here. -
          -

          - - Edit this page - -

          -
          -
          - - -View Your Work --------------- - -We've created the following: - -.. class:: incremental small - -* A wiki view that redirects to the automatically-created FrontPage page -* A page view that will render the ``data`` from a page, along with a url for - editing that page -* A page template to show a wiki page. - -.. class:: incremental - -That's all we need to be able to see our work. Start Pyramid: - -.. class:: incremental small - -:: - - (pyramidenv)$ pserve development.ini - Starting server in PID 43925. - serving on http://0.0.0.0:6543 - -.. class:: incremental - -Load http://localhost:6543/ - - -What You Should See -------------------- - -.. image:: img/wiki_frontpage.png - :align: center - :width: 95% - - -Page Editing ------------- - -You'll notice that the page has a link to ``Edit This Page`` - -.. class:: incremental - -If you click it, you get a 404. We haven't created that view yet. - - -Next Steps ----------- - -We've learned a great deal about Pyramid so far. - -.. class:: incremental - -We've covered *traversal* and learned about object persistence with the ZODB. - -.. class:: incremental - -Finally, we've implemented the Data model for our wiki application and begun -to implement views. - -.. class:: incremental - -In our next session, we'll complete the wiki, adding page creation and editing -and an auth mechanism. - - -Break Time ----------- - -.. class:: big-centered - -See you back soon. diff --git a/source/presentations/session10.rst b/source/presentations/session10.rst new file mode 100644 index 00000000..abdd40ea --- /dev/null +++ b/source/presentations/session10.rst @@ -0,0 +1,666 @@ +********** +Session 10 +********** + +.. figure:: /_static/django-pony.png + :align: center + :width: 60% + + image: http://djangopony.com/ + + +Deploying Django +================ + +.. rst-class:: left +.. container:: + + Over the last two sessions you've built and extended a simple Django + application. + + .. rst-class:: build + .. container:: + + Now it is time to deploy that application to a server so the world can + see it. + + Previously, we used Heroku to deploy a simple Pyramid application. + + We could do the same with Django, but we won't. + + Instead, we'll deploy to **A**\ mazon **W**\ eb **S**\ ervices (AWS) + + +Choosing a Deployment Strategy +------------------------------ + +There are many many different ways to deploy a web application. + +.. rst-class:: build +.. container:: + + And there are many many services offering platforms for deployment. + + How do you choose the right one for you? + + In general there are a few rules of thumb to consider: + + .. rst-class:: build + + * The more convenient the service, the less configurable it is. + * The less you pay for a service, the more work you have to do yourself. + * With great power comes great responsibility. + +.. nextslide:: + +In choosing a service and a strategy, you'll want to ask yourself a few +questions: + +.. rst-class:: build +.. container:: + + .. rst-class:: build + + * What are the basic software components of my project? + * How much control or customization of each component do I require? + * What service supports all of my required components? + * What service allows my required customizations? + * If no single service does everything I need, which could be wired + together? + + The answers to these questions will help to determine the correct choice + for you. + +.. nextslide:: Our Choice for Today + +We are going to ignore all these questions, and simply ask one question. + +.. rst-class:: build +.. container:: + + Which service will allow us to set up each layer in a full web application + stack so that we can learn how the stack works from front to back? + + The simplest answer to that question is **AWS**. + + Therefore, that's the service we will use today. + +Preparing for AWS Deployment +---------------------------- + +You've started out this week by signing up for AWS. + +.. rst-class:: build +.. container:: + + You've created a security group and a key pair to help with accessing any + servers we create. + + You've also set up an IAM user and configured security credentials for that + user. + + If we were to be automating our work today, we'd use those credentials to + allow the `boto`_ library to connect to AWS as that IAM user. + + Then you could `create or destroy resources`_ using that library. + + Issues surrounding using that library on Windows prevent us from trying + that path tonight. + +.. nextslide:: + +Instead we'll be making a manual deployment using AWS. + +.. rst-class:: build +.. container:: + + This is always the first step to automation anyway, so this is an important + first step. + + We'll begin by converting some aspects of our application to better provide + for security + + In preparation for that we will need to add a new package to our django + virtual environment. + + .. code-block:: bash + + (djangoenv)$ pip install dj-database-url + + + +.. _boto: https://boto.readthedocs.org/ +.. _create or destroy resources: http://codefellows.github.io/python-dev-accelerator/lectures/day11/boto.html + + +.. nextslide:: 12-Factor + +This new package is an attempt to help Django get in line with a principle +called `12-factor`_. + +.. rst-class:: build +.. container:: + + The basic idea is that any data that your app uses for configuration that + is *external* to the app itself, should be separated from the app. + + The link about contains much more effective explanations, read it. + + We've already done this to some degree with our Pyramid application, by + putting some configuration values into *environment variables* + + ``dj-database-url`` allows us to do that with the configuration for our + database. + +.. _12-factor: http://12factor.net/ + + +.. nextslide:: Updating Settings + +Open ``settings.py`` and replace the current DATABASES dictionary with this: + +.. code-block:: python + + DATABASES = { + 'default': dj_database_url.config( + default='sqlite:///' + os.path.join(BASE_DIR, 'db.sqlite3') + ) + } + +.. rst-class:: build +.. container:: + + The default behavior of ``dj-database-url`` is to look for a + ``DATABASE_URL`` variable in the environment. + + If it doesn't find that, it uses the value you provide for *default*. + + It converts a `url-style`_ database connection string to the dictionary + Django expects. + + Here, we've set the default to be the same as what we had previously. + +.. _url-style: https://github.com/kennethreitz/dj-database-url#url-schema + +.. nextslide:: Repeatable Envs + +Another principle of the 12-factor philosophy is to keep the differences +between production and development to a minimum. + +.. rst-class:: build +.. container:: + + Again, in our Pyramid app we handled this with a ``requirements.txt`` file. + + Here we will do the same. + + At your command line, with the virtualenv active, run the following + command: + + .. code-block:: bash + + (djangoenv)$ pip freeze > requirements.txt + + Then, add that file to your repository and commit the changes. + + At this point, we're about ready to begin working directly with AWS + +Setting up An EC2 Instance +-------------------------- + +Our first step is to create an EC2 (Elastic Compute Cloud) instance for our +application. + +.. rst-class:: build +.. container:: + + Begin by opening the AWS homepage (http://aws.amazon.com) + + Then click on the big yellow "Sign in to the Console" button + + Fill in your email, check "I am a returning user..." and supply your + password. + + When the page loads, you are viewing the AWS Console. + + If you don't see a big list of services in that first page, click on + 'Services' in the black header. + + From the list of services, click on ``EC2``. + +.. nextslide:: + +The page that loads is the management console for EC2 resources. You used it +to create your security group and key pair. + +.. rst-class:: build +.. container:: + + Click the large blue "Launch Instance" button to start a new instance. + + You should see a list of types of operating system listed. + + If you don't click on *quick start* at the left. + + In the list, find "Ubuntu Server 14.04 LTS". + + Click on 'Select' to begin building an instance using that operating + system. + +.. nextslide:: + +The next page of the launch wizard allows you to choose how much CPU power and +RAM your machine will have. + +.. rst-class:: build +.. container:: + + There are only two types of instance that are in the free tier, and one is + now deprecated. + + Select the *t2.micro* instance by clicking the checkbox to the left of that + row (it may already be selected for you). + + Below the table of instance types, find and click on "Next: configure + instance details" + +.. nextslide:: + +Click through the next two steps until you reach "Configure Security Group" + +.. rst-class:: build +.. container:: + + Here, click the "select an existing security group" button, and pick your + ssh-access group. + + This group acts as a control for a *firewall* which restricts network + access to your new instance. + + You've configured that firewall to allow any machine to talk to your + instance, but only on port 22 (SSH). + + Finish by clicking "Review and Launch" + + Then click on "Launch" to start the instance. + +.. nextslide:: + +When you click "Launch" you are required to choose a key pair to control ssh +access to your new machine. + +.. rst-class:: build +.. container:: + + Without this key pair, you have no way to access the server, and you must + destroy it and create a new one. + + Select your ``pk-aws`` pair from the list of existing key pairs. + + Then, check the box that indicates you have the private key and click + "Launch Instance". + + It will take a few minutes for the new machine to initialize and be ready. + +Accessing Your Instance +----------------------- + +Once the machine indicates it is "running" you are ready to access that +machine. + +.. rst-class:: build +.. container:: + + ssh into that machine: + + .. code-block:: bash + + ssh -i ~/.ssh/pk-aws.pem ubuntu@ + + You will need to indicate that you trust this connection. + + You are now logged in to the server as the default user. + + AWS sets this user up with the ability to run commands using *sudo* + + You'll begin by updating the OS package manager so you are ensured of + having the latest versions of any software you install: + + .. code-block:: bash + + sudo apt-get update + +Deployment Layer 1: Web Server +------------------------------ + +In our deployment stack, the frontmost facing layer is the Web Server. + +.. rst-class:: build +.. container:: + + This software is responsible for receiving requests from clients' browsers. + + It will also handle serving static resources in order to relieve Django of + that burden. + + If you are using ``https``, it's also a good place to handle terminating an + SSL connection. + + Begin by using the Ubuntu package manager to install ``nginx``: + + .. code-block:: bash + + sudo apt-get install nginx + +.. nextslide:: Controlling ``nginx`` + +Like many other packages installed by ``apt-get``, nginx is set up as a +*service* + +You can check the status of the service: + +.. code-block:: bash + + sudo service nginx status + +You can start and stop the server: + +.. code-block:: bash + + sudo service nginx stop + sudo service nginx start + +.. nextslide:: Configuring Nginx + +Default configuration for nginx lives in ``/etc/nginx``. Let's look at three +files there in particular: + +* /etc/nginx/nginx.conf (controls behavior of the whole server) +* /etc/nginx/sites-available/default (controls a single 'site') +* /etc/nginx/sites-enabled/default (activates a single 'site') + + +.. nextslide:: Check Your results + +Check your results by loading your public DNS name in a browser + +.. rst-class:: build +.. container:: + + you should see this, do you? + + .. figure:: /_static/nginx_hello.png + :align: center + :width: 40% + + Add port 80 to your security group. Then reload. + +Deployment Layer 3: Database +---------------------------- + +In order to deploy our database, we'll need to install some more software + +.. rst-class:: build +.. container:: + + Use ``apt-get istall`` to add each of the following packages: + + * build-essential + * python-dev + * python-pip + * python-psycopg2 + * postgresql-client + * git + +.. nextslide:: RDS + +You *can* set up postgres directly on the machine you just built, but that's no fun. + +.. rst-class:: build +.. container:: + + Let's use RDS, the AWS service for providing databases. + + From 'services' in the header, select RDS. + + In the page that appears, click on 'Launch a DB Instance' + + From the selection of database types, choose PostgreSQL. + + Click **no** to indicate that you don't need a multi-AZ database. + +.. nextslide:: + +On the database details page, You have a bit of work to do. + +.. rst-class:: build +.. container:: + + First, select ``db.t2.micro`` as the instance type. + + Then, for multi-AZ deployment, select **no** (again) + + Finally, provide values for the last four inputs + + The database identifier must be unique to your account and region, use + "uwpce". + + For the master username, use "awsuser" + + Provide a password and repeat it to prove you can + +.. nextslide:: + +For Advanced Settings, make sure your DB is in the same availability zone as +your EC2 instance. + +.. rst-class:: build +.. container:: + + Also ensure that you select the same security group you used for your EC2 + instance from the list of VPC security groups. + + Enter a database name, use "djangodb" + + Finally, click "Launch DB Instance" + + While the database launches, let's return to setting up our application on + EC2 + +Deployment Layer 2: Application +------------------------------- + +Back on the EC2 instance, in your ssh terminal, clone your django application: + +.. code-block:: bash + + git clone + +.. rst-class:: build +.. container:: + + pip install the requirements for your app:: + + $ cd djangoblog_uwpce + $ pip install -r requirements.txt + +.. nextslide:: + +Finally, export a system environment variable called DATABASE_URL with the +following format:: + + postgres://username:password@host:port/dbname + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + export DATABASE_URL= + + You can now test access with dbshell: + + .. code-block:: bash + + python manage.py dbshell + + Work through any issues in getting that to work + +.. nextslide:: Wiring It Up + +Once working, we can point nginx at the instance: + +.. rst-class:: build +.. container:: + + .. code-block:: bash + + sudo mv /etc/nginx/sites-available/default /etc/nginx/sites-available/default.bak + sudo vi /etc/nginx/sites-available/default + + Add the following content: + + .. code-block:: nginx + + server { + listen 80; + server_name ; + access_log /var/log/nginx/django.log; + + location / { + proxy_pass http://127.0.0.1:8000; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } + } + +.. nextslide:: + +Save that file and restart nginx: + +.. code-block:: bash + + sudo service nginx restart + +Then reload your aws instance in a web browser, you should see a BAD GATEWAY +error + +now start django and then reload: + +.. code-block:: bash + + python manage.py runserver + +This works, but as soon as you exit your ssh terminal, django will quit. We +want a long-running process we can leave behind. + + +Deployment Layer 4: Permanence +------------------------------ + +Install gunicorn on the server + +.. code-block:: bash + + pip install gunicorn + +Back on your own machine, create ``mysite/production.py`` and add the following +content: + +.. code-block:: python + + from settings import * + + DEBUG = False + TEMPLATE_DEBUG = False + ALLOWED_HOSTS = ['', 'localhost'] + STATIC_ROOT = os.path.join(BASE_DIR, 'static') + +Add the file to your repository and commit your changes. + +Then pull the changes back on your EC2 instance + +.. nextslide:: Configuration Changes for Nginx + +Update nginx config (/etc/nginx/sites-available/default) to serve static files: + +.. code-block:: nginx + + server { + # ... + + location /static/ { + root /home/ubuntu/djangoblog_uwpce; + } + + } + +.. nextslide:: Running with Gunicorn + +Then set an environment variable to point at production settings:: + + export DJANGO_SETTINGS_MODULE=mysite.production + +Now, run the site using gunicorn:: + + gunicorn -b 127.0.0.1:8000 -w 4 -D mysite.wsgi + +Wahooo! + +But still not great, because nothing is monitoring this process. + +There's no way to keep track of how it is doing. + +We can do better. First, let's kill the processes that spawned:: + + killall gunicorn + +.. nextslide:: Managing Gunicorn + +We can use a process manager to run the gunicorn command, and track the results. + +Using linux `upstart`_ is relatively simple. + +Put the following in ``/etc/init/djangoblog.conf`` + +.. code-block:: cfg + + description "djangoblog" + + start on (filesystem) + stop on runlevel [016] + + respawn + setuid nobody + setgid nogroup + chdir /home/ubuntu/djangoblog_uwpce + env DJANGO_SETTINGS_MODULE=mysite.production + env DATABASE_URL=postgres://:@:/djangoblog + exec gunicorn -b 127.0.0.1:8000 -w 4 mysite.wsgi + +.. _upstart: http://blog.terminal.com/getting-started-with-upstart/ + +.. nextslide:: Using Upstart + +Once you've completed that, you will find that you can use the Linux +``service`` command to control the gunicorn process. + +.. rst-class:: build +.. container:: + + Use the following commands:: + + $ sudo service djangoblog status + $ sudo service djangoblog start + $ sudo service djangoblog stop + $ sudo service djangoblog restart + + If you see an error message about an ``unknown job`` when you run one of those + commands, it means you have an error in your configuration file. + + Find the error with this command:: + + $ init-checkconf /etc/init/djangoblog.conf + + And that's it! diff --git a/source/presentations/session10.rst.norender b/source/presentations/session10.rst.norender deleted file mode 100644 index 8c3731fc..00000000 --- a/source/presentations/session10.rst.norender +++ /dev/null @@ -1,1225 +0,0 @@ -Internet Programming with Python -================================ - -.. image:: img/pyramid-medium.png - :align: left - :width: 50% - -Session 10: A Pyramid Application - -.. class:: intro-blurb right - -| The flexible framework. -| Totally not built by aliens. - - -Chameleon Templates -------------------- - -Chameleon page templates are valid XML/HTML. - -.. class:: incremental - -You can validate and view templates in browsers without the templating engine. - -.. class:: incremental - -This can be helpful in working with designers or front-end folks - -.. class:: incremental - -Instead of using special tags for processing directives, Chameleon uses XML -tag attributes. - - -TAL/METAL ---------- - -Chameleon is descended from Zope Page Templates (ZPT) - -.. class:: incremental - -It uses two XML namespaces for directives: - -.. class:: incremental - -* TAL (Template Attribute Language) -* METAL (Macro Extension to the Template Attribute Language) - -.. class:: incremental - -TAL provides basic directives for logical structures - -.. class:: incremental - -METAL provides directives for creating and using template Macros - - -TAL Statements --------------- - -TAL and METAL statements are XML tag attributes. - -.. class:: incremental - -This means they look just like ``id="foo"`` or ``class="bar"`` - -.. class:: incremental - -* ``tal:=””`` - -* The ``tal:`` is a ‘namespace identifier’ (xml) - - * Not strictly required, but helpful - - * Strongly encouraged :) - - -TAL Operators -------------- - -There are seven basic TAL operators, which are processed in this order - -.. class:: incremental - -* ``tal:define`` - set a value or values -* ``tal:condition`` - test truth value to execute -* ``tal:repeat`` - loop over sets of values -* ``tal:content`` - set the content of a tag -* ``tal:replace`` - replace an entire tag -* ``tal:attributes`` - set html/xml attributes of a tag -* ``tal:omit-tag`` - if expression is false, omit the tag - -.. class:: incremental - -``content`` and ``replace`` are mutually exclusive. - - -TAL Expressions ---------------- - -The right half of a TAL statement is an *expression* using the TAL expression -syntax (TALES): - -.. class:: incremental - -* Exists - ``exists:foo`` -* Import - ``import:foo.bar.baz`` -* Load = ``load:../other_template.pt`` -* Not - ``not: is_anon`` -* Python - ``python: here.Title()`` -* String - ``string:my ${value}`` -* Structure - ``structure:some_html`` - - -METAL Operators ---------------- - -METAL provides operators related to creating and using template macros: - -.. class:: incremental - -* ``metal:define-macro`` - designates a DOM scope as a macro -* ``metal:use-macro`` - indicates that a macro should be used -* ``metal:extend-macro`` - extend an existing macro -* ``metal:define-slot`` - designate a customization point for a macro -* ``metal:fill-slot`` - provide custom content for a macro slot - -.. class:: incremental - -Much of this will become clearer as we actually create our templates. - - -A Few Notes ------------ - -Take a look at our ``view.pt`` template again. - -.. class:: incremental - -```` and ```` tags are processed and removed by the engine. - -.. class:: incremental - -* ``use-macro="load: base.pt"``: we will be using ``base.pt`` as our main - template *macro*. -* Template *macros* define one or more *slots*. -* ``metal:fill-slot="main-content"``: everything goes in the ``main-content`` - slot. - - -More Notes ----------- - -.. code-block:: xml - -
          - Page text goes here. -
          - -The ``tal`` directive ``replace`` replaces the ``
          `` tag with ``content``. - -The ``structure`` expression ensures that the HTML is not escaped. - -.. container:: incremental - - .. code-block:: xml - - - Edit this page - - - Here, we use the ``tal`` directive ``attributes`` to set the ``href`` for - our anchor to the value passed into our template as ``edit_url``. - - -Page Editing ------------- - -You'll notice that the page has a link to ``Edit This Page`` - -.. class:: incremental - -If you click it, you get a 404. We haven't created that view yet. - -.. class:: incremental - -Let's start by adding tests to ensure: - -.. class:: incremental - -* the edit view will submit to itself -* will save page data updates -* will redirect back to the page view after saving - - -Test Page Editing ------------------ - -In ``tests.py``: - -.. code-block:: python - :class: small - - class EditPageTests(unittest.TestCase): - def _callFUT(self, context, request): - from .views import edit_page - return edit_page(context, request) - - def test_it_notsubmitted(self): - context = testing.DummyResource() - request = testing.DummyRequest() - info = self._callFUT(context, request) - self.assertEqual(info['page'], context) - self.assertEqual(info['save_url'], - request.resource_url(context, 'edit_page')) - - -One More Method ---------------- - -.. code-block:: python - :class: small - - class EditPageTests(unittest.TestCase): - # ... - - def test_it_submitted(self): - context = testing.DummyResource() - request = testing.DummyRequest({'form.submitted':True, - 'body':'Chapel Hill Rocks'}) - response = self._callFUT(context, request) - self.assertEqual(response.location, 'http://example.com/') - self.assertEqual(context.data, 'Chapel Hill Rocks') - -.. class:: small incremental - -:: - - (pyramidenv)$ python setup.py test - Ran 7 tests in 0.110s - FAILED (errors=2) - - -Editing a Page --------------- - -Back in ``views.py`` add the following: - -.. code-block:: python - :class: small - - @view_config(name='edit_page', context='.models.Page', - renderer='templates/edit.pt') - def edit_page(context, request): - if 'form.submitted' in request.params: - context.data = request.params['body'] - return HTTPFound(location = request.resource_url(context)) - - return dict(page=context, - save_url=request.resource_url(context, 'edit_page')) - -.. class:: incremental - -Note the ``name`` in ``view_config``. - -.. class:: incremental - -When traversal runs out of objects, it tries to find views by name - - -Check Your Tests ----------------- - -Even without a template we can run our tests: - -.. class:: small incremental - -:: - - (pyramidenv)$ python setup.py test - ... - ---------------------------------------------------------------------- - Ran 7 tests in 0.112s - - OK - - -The Edit Template ------------------ - -Create and fill ``edit.pt`` in ``templates``: - -.. code-block:: xml - :class: small - - - - Editing - Page Name Goes Here - - - -
          -