diff --git a/21_skillsharing.md b/21_skillsharing.md new file mode 100644 index 000000000..0a8044efc --- /dev/null +++ b/21_skillsharing.md @@ -0,0 +1,1121 @@ +{{meta {code_links: "[\"code/skillsharing.zip\"]"}}} + +# Proyecto: Sitio web para compartir habilidades + +{{quote {author: "Margaret Fuller", chapter: true} + +Si tienes conocimiento, deja a otros encender sus velas allí. + +quote}} + +{{index "proyecto para compartir habilidades", meetup, "capítulo de proyecto"}} + +{{figure {url: "img/chapter_picture_21.jpg", alt: "Dibujo de dos monociclos", chapter: "framed"}}} + +Una reunión para _((compartir habilidades))_ es un evento en donde personas +con intereses comunes se juntan para dar pequeñas presentaciones +informales acerca de cosas que saben. En un reunión de ((jardinería)) +alguien explicaría como cultivar ((apio)). O en un grupo de programación, +podrías presentarte y hablarle a las personas acerca de Node.js. + +{{index aprendizaje, "grupo de usuarios"}} + +En estas reuniones, también llamadas _grupos de usuarios_ cuando +se tratan de computadoras, son una forma genial de ampliar tus +horizontes, aprender acerca de nuevos desarrollos, o simplemente +conoce gente con intereses similares. Muchas ciudades grandes tienen +reuniones sobre JavaScript. Típicamente la entrada es gratis, y las +que he visitado han sido amistosas y cálidas. + +En este capítulo de proyecto final, nuestra meta es construir un +((sitio web)) para administrar las pláticas dadas en una reunión para +compartir habilidades. Imagina un pequeño grupo de gente que se reúne +regularmente en las oficinas de uno de ellos para hablar de ((monociclismo)). +El organizador previo se fue a otra ciudad, y nadie se postuló para +tomar esta tarea. Queremos un sistema que deje a los participantes proponer +y discutir pláticas entre ellos, sin un organizador central. + +[Justo como en el [capítulo anterior](node), algo del código está escrito +para Node.js, y poder correrlo directamente en una página HTML es poco +probable.]{if interactive} El proyecto completo puede ser ((descarga))do de +[_https://eloquentjavascript.net/code/skillsharing.zip_](https://eloquentjavascript.net/code/skillsharing.zip) (En inglés). + +## Design + +{{index "skill-sharing project", persistence}} + +There is a _((server))_ part to this project, written for ((Node.js)), +and a _((client))_ part, written for the ((browser)). The server +stores the system's data and provides it to the client. It also serves +the files that implement the client-side system. + +{{index [HTTP, client]}} + +The server keeps the list of ((talk))s proposed for the next meeting, +and the client shows this list. Each talk has a presenter name, a +title, a summary, and an array of ((comment))s associated with it. The +client allows users to propose new talks (adding them to the list), +delete talks, and comment on existing talks. Whenever the user makes +such a change, the client makes an HTTP ((request)) to tell the +server about it. + +{{figure {url: "img/skillsharing.png", alt: "Screenshot of the skill-sharing website",width: "10cm"}}} + +{{index "live view", "user experience", "pushing data", connection}} + +The ((application)) will be set up to show a _live_ view of the +current proposed talks and their comments. Whenever someone, +somewhere, submits a new talk or adds a comment, all people who have +the page open in their browsers should immediately see the change. +This poses a bit of a challenge—there is no way for a web server to +open a connection to a client, nor is there a good way to know which +clients are currently looking at a given website. + +{{index "Node.js"}} + +A common solution to this problem is called _((long polling))_, which +happens to be one of the motivations for Node's design. + +## Long polling + +{{index firewall, notification, "long polling", network, [browser, security]}} + +To be able to immediately notify a client that something changed, we +need a ((connection)) to that client. Since web browsers do not +traditionally accept connections and clients are often behind +((router))s that would block such connections anyway, having the +server initiate this connection is not practical. + +We can arrange for the client to open the connection and keep it +around so that the server can use it to send information when it needs +to do so. + +{{index socket}} + +But an ((HTTP)) request allows only a simple flow of information: the +client sends a request, the server comes back with a single response, +and that is it. There is a technology called _((WebSockets))_, +supported by modern browsers, that makes it possible to open +((connection))s for arbitrary data exchange. But using them properly +is somewhat tricky. + +In this chapter, we use a simpler technique—((long polling))—where +clients continuously ask the server for new information using regular +HTTP requests, and the server stalls its answer when it has nothing +new to report. + +{{index "live view"}} + +As long as the client makes sure it constantly has a polling request +open, it will receive information from the server quickly after it +becomes available. For example, if Fatma has our skill-sharing +application open in her browser, that browser will have made a request +for updates and will be waiting for a response to that request. When Iman +submits a talk on Extreme Downhill Unicycling, the server will notice +that Fatma is waiting for updates and send a response containing the +new talk to her pending request. Fatma's browser will receive the data +and update the screen to show the talk. + +{{index robustness, timeout}} + +To prevent connections from timing out (being aborted because of a +lack of activity), ((long polling)) techniques usually set a maximum +time for each request, after which the server will respond anyway, +even though it has nothing to report, after which the client will +start a new request. Periodically restarting the request also makes +the technique more robust, allowing clients to recover from temporary +((connection)) failures or server problems. + +{{index "Node.js"}} + +A busy server that is using long polling may have thousands of waiting +requests, and thus ((TCP)) connections, open. Node, which makes it +easy to manage many connections without creating a separate thread of +control for each one, is a good fit for such a system. + +## HTTP interface + +{{index "skill-sharing project", [interface, HTTP]}} + +Before we start designing either the server or the client, let's think +about the point where they touch: the ((HTTP)) interface over +which they communicate. + +{{index [path, URL], [method, HTTP]}} + +We will use ((JSON)) as the format of our request and response body. +Like in the file server from [Chapter ?](node#file_server), we'll try +to make good use of HTTP methods and ((header))s. The interface is +centered around the `/talks` path. Paths that do not start with +`/talks` will be used for serving ((static file))s—the HTML and +JavaScript code for the client-side system. + +{{index "GET method"}} + +A `GET` request to `/talks` returns a JSON document like this: + +```{lang: "application/json"} +[{"title": "Unituning", + "presenter": "Jamal", + "summary": "Modifying your cycle for extra style", + "comments": []}] +``` + +{{index "PUT method", URL}} + +Creating a new talk is done by making a `PUT` request to a URL like +`/talks/Unituning`, where the part after the second slash is the title +of the talk. The `PUT` request's body should contain a ((JSON)) object +that has `presenter` and `summary` properties. + +{{index "encodeURIComponent function", [escaping, "in URLs"], [whitespace, "in URLs"]}} + +Since talk titles may contain spaces and other characters that may not +appear normally in a URL, title strings must be encoded with the +`encodeURIComponent` function when building up such a URL. + +``` +console.log("/talks/" + encodeURIComponent("How to Idle")); +// → /talks/How%20to%20Idle +``` + +A request to create a talk about idling might look something like +this: + +```{lang: http} +PUT /talks/How%20to%20Idle HTTP/1.1 +Content-Type: application/json +Content-Length: 92 + +{"presenter": "Maureen", + "summary": "Standing still on a unicycle"} +``` + +Such URLs also support `GET` requests to retrieve the JSON +representation of a talk and `DELETE` requests to delete a talk. + +{{index "POST method"}} + +Adding a ((comment)) to a talk is done with a `POST` request to a URL +like `/talks/Unituning/comments`, with a JSON body that has `author` +and `message` properties. + +```{lang: http} +POST /talks/Unituning/comments HTTP/1.1 +Content-Type: application/json +Content-Length: 72 + +{"author": "Iman", + "message": "Will you talk about raising a cycle?"} +``` + +{{index "query string", timeout, "ETag header", "If-None-Match header"}} + +To support ((long polling)), `GET` requests to `/talks` may include +extra headers that inform the server to delay the response if no new +information is available. We'll use a pair of headers normally +intended to manage caching: `ETag` and `If-None-Match`. + +{{index "304 (HTTP status code)"}} + +Servers may include an `ETag` ("entity tag") header in a response. Its +value is a string that identifies the current version of the resource. +Clients, when they later request that resource again, may make a +_((conditional request))_ by including an `If-None-Match` header whose +value holds that same string. If the resource hasn't changed, the +server will respond with status code 304, which means "not modified", +telling the client that its cached version is still current. When the +tag does not match, the server responds as normal. + +{{index "Prefer header"}} + +We need something like this, where the client can tell the server +which version of the list of talks it has, and the server +responds only when that list has changed. But instead of immediately +returning a 304 response, the server should stall the response and +return only when something new is available or a given amount of time +has elapsed. To distinguish long polling requests from normal +conditional requests, we give them another header, `Prefer: wait=90`, +which tells the server that the client is willing to wait up to 90 +seconds for the response. + +The server will keep a version number that it updates every time the +talks change and will use that as the `ETag` value. Clients can make +requests like this to be notified when the talks change: + +```{lang: null} +GET /talks HTTP/1.1 +If-None-Match: "4" +Prefer: wait=90 + +(time passes) + +HTTP/1.1 200 OK +Content-Type: application/json +ETag: "5" +Content-Length: 295 + +[....] +``` + +{{index security}} + +The protocol described here does not do any ((access control)). +Everybody can comment, modify talks, and even delete them. (Since the +Internet is full of ((hooligan))s, putting such a system online +without further protection probably wouldn't end well.) + +## The server + +{{index "skill-sharing project"}} + +Let's start by building the ((server))-side part of the program. The +code in this section runs on ((Node.js)). + +### Routing + +{{index "createServer function", [path, URL], [method, HTTP]}} + +Our server will use `createServer` to start an HTTP server. In the +function that handles a new request, we must distinguish between the +various kinds of requests (as determined by the method and the +path) that we support. This can be done with a long chain of `if` +statements, but there is a nicer way. + +{{index dispatch}} + +A _((router))_ is a component that helps dispatch a request to the +function that can handle it. You can tell the router, for example, +that `PUT` requests with a path that matches the regular expression +`/^\/talks\/([^\/]+)$/` (`/talks/` followed by a talk title) can be +handled by a given function. In addition, it can help extract the +meaningful parts of the path (in this case the talk title), wrapped in +parentheses in the ((regular expression)), and pass them to the +handler function. + +There are a number of good router packages on ((NPM)), but here we'll +write one ourselves to illustrate the principle. + +{{index "require function", "Router class", module}} + +This is `router.js`, which we will later `require` from our server +module: + +```{includeCode: ">code/skillsharing/router.js"} +const {parse} = require("url"); + +module.exports = class Router { + constructor() { + this.routes = []; + } + add(method, url, handler) { + this.routes.push({method, url, handler}); + } + resolve(context, request) { + let path = parse(request.url).pathname; + + for (let {method, url, handler} of this.routes) { + let match = url.exec(path); + if (!match || request.method != method) continue; + let urlParts = match.slice(1).map(decodeURIComponent); + return handler(context, ...urlParts, request); + } + return null; + } +}; +``` + +{{index "Router class"}} + +The module exports the `Router` class. A router object allows new +handlers to be registered with the `add` method and can resolve +requests with its `resolve` method. + +{{index "some method"}} + +The latter will return a response when a handler was found, and `null` +otherwise. It tries the routes one at a time (in the order in which +they were defined) until a matching one is found. + +{{index "capture group", "decodeURIComponent function", [escaping, "in URLs"]}} + +The handler functions are called with the `context` value (which +will be the server instance in our case), match strings for any groups +they defined in their ((regular expression)), and the request object. +The strings have to be URL-decoded since the raw URL may contain +`%20`-style codes. + +### Serving files + +When a request matches none of the request types defined in our +router, the server must interpret it as a request for a file in the +`public` directory. It would be possible to use the file server +defined in [Chapter ?](node#file_server) to serve such files, but we +neither need nor want to support `PUT` and `DELETE` requests on files, +and we would like to have advanced features such as support for +caching. So let's use a solid, well-tested ((static file)) server from +((NPM)) instead. + +{{index "createServer function", "ecstatic package"}} + +I opted for `ecstatic`. This isn't the only such server on NPM, but it +works well and fits our purposes. The `ecstatic` package exports a +function that can be called with a configuration object to produce a +request handler function. We use the `root` option to tell the server +where it should look for files. The handler function accepts `request` +and `response` parameters and can be passed directly to `createServer` +to create a server that serves _only_ files. We want to first check +for requests that we should handle specially, though, so we wrap it in +another function. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +const {createServer} = require("http"); +const Router = require("./router"); +const ecstatic = require("ecstatic"); + +const router = new Router(); +const defaultHeaders = {"Content-Type": "text/plain"}; + +class SkillShareServer { + constructor(talks) { + this.talks = talks; + this.version = 0; + this.waiting = []; + + let fileServer = ecstatic({root: "./public"}); + this.server = createServer((request, response) => { + let resolved = router.resolve(this, request); + if (resolved) { + resolved.catch(error => { + if (error.status != null) return error; + return {body: String(error), status: 500}; + }).then(({body, + status = 200, + headers = defaultHeaders}) => { + response.writeHead(status, headers); + response.end(body); + }); + } else { + fileServer(request, response); + } + }); + } + start(port) { + this.server.listen(port); + } + stop() { + this.server.close(); + } +} +``` + +This uses a similar convention as the file server from the [previous +chapter](node) for responses—handlers return promises that resolve to +objects describing the response. It wraps the server in an object that +also holds its state. + +### Talks as resources + +The ((talk))s that have been proposed are stored in the `talks` +property of the server, an object whose property names are the talk +titles. These will be exposed as HTTP ((resource))s under +`/talks/[title]`, so we need to add handlers to our router that +implement the various methods that clients can use to work with them. + +{{index "GET method", "404 (HTTP status code)"}} + +The handler for requests that `GET` a single talk must look up the +talk and respond either with the talk's JSON data or with a 404 error +response. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +const talkPath = /^\/talks\/([^\/]+)$/; + +router.add("GET", talkPath, async (server, title) => { + if (title in server.talks) { + return {body: JSON.stringify(server.talks[title]), + headers: {"Content-Type": "application/json"}}; + } else { + return {status: 404, body: `No talk '${title}' found`}; + } +}); +``` + +{{index "DELETE method"}} + +Deleting a talk is done by removing it from the `talks` object. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +router.add("DELETE", talkPath, async (server, title) => { + if (title in server.talks) { + delete server.talks[title]; + server.updated(); + } + return {status: 204}; +}); +``` + +{{index "long polling", "updated method"}} + +The `updated` method, which we will define +[later](skillsharing#updated), notifies waiting long polling requests +about the change. + +{{index "readStream function", "body (HTTP)", stream}} + +To retrieve the content of a request body, we define a function called +`readStream`, which reads all content from a ((readable stream)) and +returns a promise that resolves to a string. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +function readStream(stream) { + return new Promise((resolve, reject) => { + let data = ""; + stream.on("error", reject); + stream.on("data", chunk => data += chunk.toString()); + stream.on("end", () => resolve(data)); + }); +} +``` + +{{index validation, input, "PUT method"}} + +One handler that needs to read request bodies is the `PUT` handler, +which is used to create new ((talk))s. It has to check whether the +data it was given has `presenter` and `summary` properties, which are +strings. Any data coming from outside the system might be nonsense, +and we don't want to corrupt our internal data model or ((crash)) when +bad requests come in. + +{{index "updated method"}} + +If the data looks valid, the handler stores an object that represents +the new talk in the `talks` object, possibly ((overwriting)) an +existing talk with this title, and again calls `updated`. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +router.add("PUT", talkPath, + async (server, title, request) => { + let requestBody = await readStream(request); + let talk; + try { talk = JSON.parse(requestBody); } + catch (_) { return {status: 400, body: "Invalid JSON"}; } + + if (!talk || + typeof talk.presenter != "string" || + typeof talk.summary != "string") { + return {status: 400, body: "Bad talk data"}; + } + server.talks[title] = {title, + presenter: talk.presenter, + summary: talk.summary, + comments: []}; + server.updated(); + return {status: 204}; +}); +``` + +{{index validation, "readStream function"}} + +Adding a ((comment)) to a ((talk)) works similarly. We use +`readStream` to get the content of the request, validate the resulting +data, and store it as a comment when it looks valid. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +router.add("POST", /^\/talks\/([^\/]+)\/comments$/, + async (server, title, request) => { + let requestBody = await readStream(request); + let comment; + try { comment = JSON.parse(requestBody); } + catch (_) { return {status: 400, body: "Invalid JSON"}; } + + if (!comment || + typeof comment.author != "string" || + typeof comment.message != "string") { + return {status: 400, body: "Bad comment data"}; + } else if (title in server.talks) { + server.talks[title].comments.push(comment); + server.updated(); + return {status: 204}; + } else { + return {status: 404, body: `No talk '${title}' found`}; + } +}); +``` + +{{index "404 (HTTP status code)"}} + +Trying to add a comment to a nonexistent talk returns a 404 error. + +### Long polling support + +The most interesting aspect of the server is the part that handles +((long polling)). When a `GET` request comes in for `/talks`, it may +be either a regular request or a long polling request. + +{{index "talkResponse method", "ETag header"}} + +There will be multiple places in which we have to send an array of +talks to the client, so we first define a helper method that builds up +such an array and includes an `ETag` header in the response. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +SkillShareServer.prototype.talkResponse = function() { + let talks = []; + for (let title of Object.keys(this.talks)) { + talks.push(this.talks[title]); + } + return { + body: JSON.stringify(talks), + headers: {"Content-Type": "application/json", + "ETag": `"${this.version}"`} + }; +}; +``` + +{{index "query string", "url package", parsing}} + +The handler itself needs to look at the request headers to see whether +`If-None-Match` and `Prefer` headers are present. Node stores headers, +whose names are specified to be case insensitive, under their +lowercase names. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +router.add("GET", /^\/talks$/, async (server, request) => { + let tag = /"(.*)"/.exec(request.headers["if-none-match"]); + let wait = /\bwait=(\d+)/.exec(request.headers["prefer"]); + if (!tag || tag[1] != server.version) { + return server.talkResponse(); + } else if (!wait) { + return {status: 304}; + } else { + return server.waitForChanges(Number(wait[1])); + } +}); +``` + +{{index "long polling", "waitForChanges method", "If-None-Match header", "Prefer header"}} + +If no tag was given or a tag was given that doesn't match the +server's current version, the handler responds with the list of talks. +If the request is conditional and the talks did not change, we consult +the `Prefer` header to see whether we should delay the response or respond +right away. + +{{index "304 (HTTP status code)", "setTimeout function", timeout, "callback function"}} + +Callback functions for delayed requests are stored in the server's +`waiting` array so that they can be notified when something happens. +The `waitForChanges` method also immediately sets a timer to respond +with a 304 status when the request has waited long enough. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +SkillShareServer.prototype.waitForChanges = function(time) { + return new Promise(resolve => { + this.waiting.push(resolve); + setTimeout(() => { + if (!this.waiting.includes(resolve)) return; + this.waiting = this.waiting.filter(r => r != resolve); + resolve({status: 304}); + }, time * 1000); + }); +}; +``` + +{{index "updated method"}} + +{{id updated}} + +Registering a change with `updated` increases the `version` property +and wakes up all waiting requests. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +SkillShareServer.prototype.updated = function() { + this.version++; + let response = this.talkResponse(); + this.waiting.forEach(resolve => resolve(response)); + this.waiting = []; +}; +``` + +{{index [HTTP, server]}} + +That concludes the server code. If we create an instance of +`SkillShareServer` and start it on port 8000, the resulting HTTP +server serves files from the `public` subdirectory alongside a +talk-managing interface under the `/talks` URL. + +```{includeCode: ">code/skillsharing/skillsharing_server.js"} +new SkillShareServer(Object.create(null)).start(8000); +``` + +## The client + +{{index "skill-sharing project"}} + +The ((client))-side part of the skill-sharing website consists of +three files: a tiny HTML page, a style sheet, and a JavaScript file. + +### HTML + +{{index "index.html"}} + +It is a widely used convention for web servers to try to serve a file +named `index.html` when a request is made directly to a path that +corresponds to a directory. The ((file server)) module we use, +`ecstatic`, supports this convention. When a request is made to the +path `/`, the server looks for the file `./public/index.html` +(`./public` being the root we gave it) and returns that file if found. + +Thus, if we want a page to show up when a browser is pointed at our +server, we should put it in `public/index.html`. This is our index +file: + +```{lang: "text/html", includeCode: ">code/skillsharing/public/index.html"} + + +