Reworked intro and distributed-cluster

Author: clintongormley

010_Intro/00_Intro.asciidoc

@@ -2,7 +2,7 @@ Elasticsearch is a real-time distributed search and analytics engine which
 allows you to explore your data at a speed and at a scale never before
 possible.
 
-It is used for full text search, structured search, analytics and
+It is used for full text search, structured search, analytics, and
 all three in combination.  It can run on your laptop, or scale out to hundreds
 of servers and petabytes of data.
 

010_Intro/05_What_is_it.asciidoc

@@ -1,37 +1,37 @@
 === What is Elasticsearch?
 
-Elasticsearch is a search engine built on top of Apache Lucene, a full-text
-search library.  Lucene is arguably the most advanced, performant and fully-featured
-search engine in existence today -- both open source and proprietary.
+Elasticsearch is a search engine built on top of
+https://lucene.apache.org/core/[Apache Lucene(TM)] , a full-text search engine
+library.  Lucene is arguably the most advanced, performant and fully-featured
+search engine library in existence today -- both open source and proprietary.
 
-But Lucene is just a library. To leverage it's power you need to work in Java
-and integrate it directly with your application. Worse, you will likely
+But Lucene is just a library. To leverage its power you need to work in Java
+and to integrate Lucene directly with your application. Worse, you will likely
 require a degree in Information Retrieval to understand how it works.  Lucene
 is *very* complex.
 
-Elasticsearch aims to make full text search easy by hiding the complexities of
-Lucene behind a simple, coherent API.  Elasticsearch uses Lucene internally
-for all of its indexing and search.
+Elasticsearch uses Lucene internally for all of its indexing and search, but
+it aims to make full text search easy by hiding the complexities of Lucene
+behind a simple, coherent API.
 
-However, it is much more than just Lucene and much more than ``just'' full
-text search.
+However, Elasticsearch is much more than just Lucene and much more than
+``just'' full text search. It is also:
 
-Elasticsearch is also:
-
-* a distributed document store where every field is indexed and
+* a distributed document store where *every field* is indexed and
    searchable
 * a distributed search engine with real-time analytics
 * capable of scaling to hundreds of servers and petabytes of structured
   and unstructured data.
 
-And it packages up all of this functionality into a standalone service
-that your application can talk to via a simple RESTful API.  Use
-your favorite programming language, Elasticsearch doesn't care.
+And it packages up all of this functionality into a standalone server
+that your application can talk to via a simple RESTful API, using
+a web client from your favorite programming language, or even
+from the command line.
 
-It is easy to get started with Elasticsearch. It ships with
-sensible defaults and hides complicated search theory from beginners.
-It _just works_, right out of the box. With minimal understanding,
-you can soon become productive.
+It is easy to get started with Elasticsearch. It ships with sensible defaults
+and hides complicated search theory away from beginners. It _just works_,
+right out of the box. With minimal understanding, you can soon become
+productive.
 
 As your knowledge grows, you can leverage more of Elasticsearch's
 advanced features. The entire engine is configurable and very flexible.

010_Intro/10_Installing_ES.asciidoc

@@ -23,9 +23,9 @@ with:
 
 [source,js]
 --------------------------------------------------
-./bin/elasticsearch -f
+./bin/elasticsearch <1>
 --------------------------------------------------
-
+<1> Add `-d` if you want to run it as a daemon.
 
 Test it out by opening another terminal window and running:
 
@@ -40,24 +40,25 @@ You should see a response like this:
 [source,js]
 --------------------------------------------------
 {
-  "tagline" : "You Know, for Search",
-  "ok" : true,
-  "status" : 200,
-  "name" : "Contrary",
-  "version" : {
-    "number" : "0.20.2",
-    "snapshot_build" : false
-  }
+   "status": 200,
+   "name": "Shrunken Bones",
+   "version": {
+      "number": "1.0.0",
+      "lucene_version": "4.6"
+   },
+   "tagline": "You Know, for Search"
 }
 --------------------------------------------------
 
-
 This means that your Elasticsearch _cluster_ is up and running, and we can
 start experimenting with it.
 
 .Clusters and nodes
 ****
-A _node_ is a running instance of Elasticsearch. A _cluster_ is a group
-of nodes that are working together to share data and to provide failover and
-scale, although a single node can form a cluster by itself.
+
+A _node_ is a running instance of Elasticsearch. A _cluster_ is a group of
+nodes with the same `cluster.name` that are working together to share data,
+and to provide failover and scale, although a single node can form a cluster
+all by itself.
+
 ****

010_Intro/15_API.asciidoc

@@ -19,11 +19,11 @@ Transport client::
     forwards requests to a node in the cluster.
 
 Both Java clients talk to the cluster over *port 9300*, using the native
-Elasticsearch protocol.  The nodes in the cluster also communicate
+Elasticsearch _transport_ protocol.  The nodes in the cluster also communicate
 with each other over port 9300. If this port is not open, then your nodes will
 not be able to form a cluster.
 
-[NOTE]
+[TIP]
 ====
 The Java client must be from the same version of Elasticsearch as the nodes,
 otherwise they may not be able to understand each other.
@@ -32,7 +32,7 @@ otherwise they may not be able to understand each other.
 ==== RESTful API with JSON over HTTP
 
 All other languages can communicate with Elasticsearch over *port 9200* using
-a RESTful API, accessible with your favorite HTTP library. In fact, as you have
+a RESTful API, accessible with your favorite web client. In fact, as you have
 seen above, you can even talk to Elasticsearch from the command line, using the
 `curl` command.
 
@@ -51,10 +51,13 @@ use:
 
 [source,js]
 --------------------------------------------------
-GET /_count?pretty
+curl -XGET 'http://localhost:9200/_count?pretty' -d '
 {
-    "match_all": {}
+    "query": {
+        "match_all": {}
+    }
 }
+'
 --------------------------------------------------
 
 All responses consist of:
@@ -85,18 +88,20 @@ switch:
 
 [source,js]
 --------------------------------------------------
-curl -I -XGET 'localhost:9200/'
+curl -i -XGET 'localhost:9200/'
 --------------------------------------------------
 
-
 We will use this `curl` format in all of our examples because it is easy to
 read and easy to translate into a request using the HTTP library of your
 choice.
 
-.Curl Shorthand
+.`curl` shorthand
 ****
-For the rest of the book, you'll notice curl examples in the shorthand format 
-used above.  The curl command is shortened from:
+
+For the rest of the book, we will show `curl` examples using a shorthand
+format that leaves out all of the bits that are the same in every request,
+like the hostname and port, and the `curl` command itself. Instead of showing
+a full request like:
 
 [source,js]
 --------------------------------------------------
@@ -106,17 +111,16 @@ curl -XGET 'localhost:9200/_count?pretty' -d '
 }'
 --------------------------------------------------
 
-to simply:
+we will show it in this shorthand format:
 
 [source,js]
 --------------------------------------------------
-GET /_count?pretty
+GET /_count
 {
     "match_all": {}
 }
 --------------------------------------------------
 
-This is done to space and repetitive typing.  We will show you the HTTP method
-(GET, PUT, POST, etc) and the URL path which follows `localhost:9200`.  The
-rest of the lines following the curl command are the body of the request.
+TODO: Link to the Sense plugin?
+
 ****

010_Intro/20_Document.asciidoc

@@ -2,27 +2,27 @@
 
 Objects in an application are seldom just a simple list of keys and values.
 More often than not they are complex data structures which may contain other
-objects, or arrays of values.
+dates, geo-locations, objects, or arrays of values.
 
-When using a relational database to store these objects, you flatten
-the object to fit the table schema (usually one field per column). When
-retrieving the object from the database, you have to reconstruct it
-from the flat representation.
+Sooner or later you're going to want to store these objects in a database.
+Trying to do this with the rows and columns of a relational database is the
+equivalent of trying to squeeze your rich expressive objects into a very big
+spreadsheet: you have to flatten the object to fit the table schema -- usually
+one field per column -- and then have to reconstruct it every time you
+retrieve it.
 
-Elasticsearch is _document oriented_, meaning that it stores and
-indexes entire _documents_.  In Elasticsearch, you index, search,
-sort and filter documents...not rows of data.  This is a fundamentally different
- way of thinking about data and is one of the reasons Elasticsearch can
- perform complex full text search.
+Elasticsearch is _document oriented_, meaning that it stores and indexes
+entire objects or _documents_.  In Elasticsearch, you index, search, sort and
+filter documents... not rows of columnar data.  This is a fundamentally
+different  way of thinking about data and is one of the reasons Elasticsearch
+can  perform complex full text search.
 
+==== JSON
 
-[NOTE]
-====
-Elasticsearch uses JSON as the serialization format for documents.
-
-JSON serialization is supported by most programming languages, and has become
-the standard format used by the NoSQL movement. It is simple, concise and easy
-to read.
+Elasticsearch uses _JSON_ (or Javascript Object Notation ) as the
+serialization format for documents. JSON serialization is supported by most
+programming languages, and has become the standard format used by the NoSQL
+movement. It is simple, concise and easy to read.
 
 Consider this JSON document which represents a user object:
 
@@ -41,10 +41,7 @@ Consider this JSON document which represents a user object:
 }
 --------------------------------------------------
 
-
-Although the original user object was complex, the
-structure of the object has been retained in the JSON version.
-Converting an object to JSON for indexing in Elasticsearch
-is much simpler than the equivalent process for a flat table structure.
-====
-
+Although the original `user` object was complex, the structure and meaning of
+the object has been retained in the JSON version. Converting an object to JSON
+for indexing in Elasticsearch is much simpler than the equivalent process for
+a flat table structure.

010_Intro/25_CRUD.asciidoc

@@ -1,17 +1,19 @@
 === Indexing documents
 
-Before we can _index_ (store) our user document in Elasticsearch, we need
-to decide what the document represents, and where to store it.
+Before we can _index_ (store and make searchable) our user document in
+Elasticsearch, we need to decide what the document represents, and where to
+store it.
 
 In Elasticsearch, a document belongs to a _type_, and those types live inside
 an _index_. You can draw some (rough) parallels to a traditional relational database:
 
- - RDBM         => Databases => Tables => Columns/Rows
- - Elasticsearch => Indices   => Types  => Documents with Fields
 
-An Elasticsearch cluster can contain multiple Indices (databases), which in
-turn contain multiple Types (tables). These types hold multiple Documents (rows),
-and each document has Fields (columns).
+    Relational DB  ⇒ Databases ⇒ Tables ⇒ Rows      ⇒ Columns
+    Elasticsearch  ⇒ Indices   ⇒ Types  ⇒ Documents ⇒ Fields
+
+An Elasticsearch cluster can contain multiple _indices_ (databases), which in
+turn contain multiple _types_ (tables). These types hold multiple _documents_
+(rows), and each document has multiple _fields_ (columns).
 
 ==== An example
 We are going to store a document in the `blogs` index, as type `user`, and we
@@ -21,8 +23,9 @@ than in the document itself:
 
 [source,js]
 --------------------------------------------------
-PUT /blogs/user/johnsmith?pretty
-{
+      <1>    <2>    <3>
+PUT /blogs/user/johnsmith
+{   <4>
     "email":     "john@smith.com",
     "name": {
         "first": "John",
@@ -34,19 +37,22 @@ PUT /blogs/user/johnsmith?pretty
     "interests": ["dolphins", "whales"]
 }
 --------------------------------------------------
-
+<1> Index: `blogs`
+<2> Type: `user`
+<3> ID: `johnsmith`
+<4> Document body
 
 And we receive the following response, which confirms that our document
 has been indexed correctly:
 
 [source,js]
 --------------------------------------------------
 {
-  "ok" : true,
-  "_index" : "blogs",
-  "_type" : "user",
-  "_id" : "johnsmith",
-  "_version" : 1
+   "_index": "blogs",
+   "_type": "user",
+   "_id": "johnsmith",
+   "_version": 1,
+   "created": true
 }
 --------------------------------------------------
 
@@ -55,8 +61,8 @@ Congratulations! You just indexed your first document! How easy was that?
 
 === Real-time GET
 
-Elasticsearch has _real-time GET_. In other words, as soon as the document
-has been indexed, it can be retrieved from any node in the cluster.
+Elasticsearch has _real-time GET_. In other words, as soon as a document
+has been indexed it can be retrieved from any node in the cluster.
 
 Not only that, but changes to documents are _persistent_: if the whole cluster
 were to suffer a power failure immediately after indexing a document, the
@@ -67,10 +73,9 @@ that we specified when indexing it:
 
 [source,js]
 --------------------------------------------------
-GET /blogs/user/johnsmith?pretty
+GET /blogs/user/johnsmith
 --------------------------------------------------
 
-
 The response contains the exact same JSON document that we indexed, as the
 `_source` field, plus some extra metadata:
 
@@ -81,7 +86,7 @@ The response contains the exact same JSON document that we indexed, as the
     "_type" :     "user",
     "_id" :       "johnsmith",
     "_version" :  1,
-    "exists" :    true,
+    "found" :    true,
     "_source" :   {
          "email":     "john@smith.com",
          "name": {