Skip to content
This repository was archived by the owner on Sep 21, 2021. It is now read-only.

Commit 963cbb9

Browse files
committed
Edited 510_Deployment/40_config.asciidoc with Atlas code editor
1 parent ae95630 commit 963cbb9

File tree

1 file changed

+69
-74
lines changed

1 file changed

+69
-74
lines changed

510_Deployment/40_config.asciidoc

+69-74
Original file line numberDiff line numberDiff line change
@@ -2,31 +2,31 @@
22
Elasticsearch ships with _very good_ defaults,((("deployment", "configuration changes, important")))((("configuration changes, important"))) especially when it comes to performance-
33
related settings and options. When in doubt, just leave
44
the settings alone. We have witnessed countless dozens of clusters ruined
5-
by errant settings because the administrator thought they could turn a knob
6-
and gain 100x improvement.
5+
by errant settings because the administrator thought he could turn a knob
6+
and gain 100-fold improvement.
77

88
[NOTE]
99
====
1010
Please read this entire section! All configurations presented are equally
11-
important, and are not listed in any particular "importance" order. Please read
11+
important, and are not listed in any particular order. Please read
1212
through all configuration options and apply them to your cluster.
1313
====
1414

15-
Other databases may require tuning, but by-and-far, Elasticsearch does not.
15+
Other databases may require tuning, but by and large, Elasticsearch does not.
1616
If you are hitting performance problems, the solution is usually better data
1717
layout or more nodes. There are very few "magic knobs" in Elasticsearch.
18-
If there was...we'd have turned it already!
18+
If there were, we'd have turned them already!
1919

2020
With that said, there are some _logistical_ configurations that should be changed
21-
for production. These changes are either to make your life easier, or because
22-
there is no way to set a good default (e.g. it depends on your cluster layout).
21+
for production. These changes are necessary either to make your life easier, or because
22+
there is no way to set a good default (because it depends on your cluster layout).
2323

2424

25-
==== Assign names
25+
==== Assign Names
2626

2727
Elasticseach by default starts a cluster named `elasticsearch`. ((("configuration changes, important", "assigning names"))) It is wise
2828
to rename your production cluster to something else, simply to prevent accidents
29-
where someone's laptop joins the cluster. A simple change to `elasticsearch_production`
29+
whereby someone's laptop joins the cluster. A simple change to `elasticsearch_production`
3030
can save a lot of heartache.
3131

3232
This can be changed in your `elasticsearch.yml` file:
@@ -36,17 +36,17 @@ This can be changed in your `elasticsearch.yml` file:
3636
cluster.name: elasticsearch_production
3737
----
3838

39-
Similarly, it is wise to change the names of your nodes. You've probably
40-
noticed by now, but Elasticsearch will assign a random Marvel Superhero name
41-
to your nodes at startup. This is cute in development...less cute when it is
42-
3am and you are trying to remember which physical machine was "Tagak the Leopard Lord".
39+
Similarly, it is wise to change the names of your nodes. As you've probably
40+
noticed by now, Elasticsearch assigns a random Marvel superhero name
41+
to your nodes at startup. This is cute in development--but less cute when it is
42+
3a.m. and you are trying to remember which physical machine was Tagak the Leopard Lord.
4343

44-
More importantly, since these names are generated on startup, each time you
45-
restart your node it will get a new name. This can make logs very confusing,
44+
More important, since these names are generated on startup, each time you
45+
restart your node, it will get a new name. This can make logs confusing,
4646
since the names of all the nodes are constantly changing.
4747

4848
Boring as it might be, we recommend you give each node a name that makes sense
49-
to you - a plain, descriptive name. This is also configured in your `elasticsearch.yml`:
49+
to you--a plain, descriptive name. This is also configured in your `elasticsearch.yml`:
5050

5151
[source,yaml]
5252
----
@@ -56,17 +56,16 @@ node.name: elasticsearch_005_data
5656

5757
==== Paths
5858

59-
By default, Elasticsearch will place the plugins,((("configuration changes, important", "paths")))((("paths"))) logs and --
60-
most importantly -- your data in the installation directory. This can lead to
61-
unfortunate accidents, where the installation directory is accidentally overwritten
62-
by a new installation of ES. If you aren't careful, you can erase all of your data.
59+
By default, Elasticsearch will place the plug-ins,((("configuration changes, important", "paths")))((("paths"))) logs, and--most important--your data in the installation directory. This can lead to
60+
unfortunate accidents, whereby the installation directory is accidentally overwritten
61+
by a new installation of Elasticsearch. If you aren't careful, you can erase all your data.
6362

64-
Don't laugh...we've seen it happen more than a few times.
63+
Don't laugh--we've seen it happen more than a few times.
6564

6665
The best thing to do is relocate your data directory outside the installation
67-
location. You can optionally move your plugin and log directories as well.
66+
location. You can optionally move your plug-in and log directories as well.
6867

69-
This can be changed via:
68+
This can be changed as follows:
7069

7170
[source,yaml]
7271
----
@@ -78,41 +77,38 @@ path.logs: /path/to/logs
7877
# Path to where plugins are installed:
7978
path.plugins: /path/to/plugins
8079
----
81-
<1> Notice that you can specify more than one directory for data using comma
82-
separated lists.
80+
<1> Notice that you can specify more than one directory for data by using comma-separated lists.
8381

84-
Data can be saved to multiple directories, and if each of these directories
85-
are mounted on a different hard drive, this is a simple and effective way to
86-
setup a "software RAID 0". Elasticsearch will automatically stripe
82+
Data can be saved to multiple directories, and if each directory
83+
is mounted on a different hard drive, this is a simple and effective way to
84+
set up a software RAID 0. Elasticsearch will automatically stripe
8785
data between the different directories, boosting performance
8886

8987
==== Minimum Master Nodes
9088

91-
This setting, called `minimum_master_nodes` is _extremely_ important to the
92-
stability of your cluster.((("configuration changes, important", "minimum_master_nodes setting")))((("minimum_master_nodes setting"))) This setting helps prevent "split brains", a situation
93-
where two masters exist in a single cluster.
89+
The `minimum_master_nodes` setting is _extremely_ important to the
90+
stability of your cluster.((("configuration changes, important", "minimum_master_nodes setting")))((("minimum_master_nodes setting"))) This setting helps prevent _split brains_, the existence of two masters in a single cluster.
9491

95-
When you have a split-brain, your cluster is at danger of losing data. Because
96-
the master is considered the "supreme ruler" of the cluster, it decides
97-
when new indices can be created, how shards are moved, etc. If you have _two_
98-
masters, data integrity becomes perilous, since you have two different nodes
92+
When you have a split brain, your cluster is at danger of losing data. Because
93+
the master is considered the supreme ruler of the cluster, it decides
94+
when new indices can be created, how shards are moved, and so forth. If you have _two_
95+
masters, data integrity becomes perilous, since you have two nodes
9996
that think they are in charge.
10097

10198
This setting tells Elasticsearch to not elect a master unless there are enough
10299
master-eligible nodes available. Only then will an election take place.
103100

104-
This setting should always be configured to a quorum (majority) of your master-
105-
eligible nodes.((("quorum"))) A quorum is `(number of master-eligible nodes / 2) + 1`.
106-
Some examples:
101+
This setting should always be configured to a quorum (majority) of your master-eligible nodes.((("quorum"))) A quorum is `(number of master-eligible nodes / 2) + 1`.
102+
Here are some examples:
107103

108104
- If you have ten regular nodes (can hold data, can become master), a quorum is
109-
`6`
110-
- If you have three dedicated master nodes and 100 data nodes, the quorum is `2`,
111-
since you only need to count nodes that are master-eligible
112-
- If you have two regular nodes...you are in a conundrum. A quorum would be
105+
`6`.
106+
- If you have three dedicated master nodes and a hundred data nodes, the quorum is `2`,
107+
since you need to count only nodes that are master eligible.
108+
- If you have two regular nodes, you are in a conundrum. A quorum would be
113109
`2`, but this means a loss of one node will make your cluster inoperable. A
114110
setting of `1` will allow your cluster to function, but doesn't protect against
115-
split brain. It is best to have a minimum of 3 nodes in situations like this.
111+
split brain. It is best to have a minimum of three nodes in situations like this.
116112

117113
This setting can be configured in your `elasticsearch.yml` file:
118114

@@ -122,13 +118,12 @@ discovery.zen.minimum_master_nodes: 2
122118
----
123119

124120
But because Elasticsearch clusters are dynamic, you could easily add or remove
125-
nodes which will change the quorum. It would be extremely irritating if you had
121+
nodes that will change the quorum. It would be extremely irritating if you had
126122
to push new configurations to each node and restart your whole cluster just to
127123
change the setting.
128124

129125
For this reason, `minimum_master_nodes` (and other settings) can be configured
130-
via a dynamic API call. You can change the setting while your cluster is online
131-
using:
126+
via a dynamic API call. You can change the setting while your cluster is online:
132127

133128
[source,js]
134129
----
@@ -144,49 +139,49 @@ This will become a persistent setting that takes precedence over whatever is
144139
in the static configuration. You should modify this setting whenever you add or
145140
remove master-eligible nodes.
146141

147-
==== Recovery settings
142+
==== Recovery Settings
148143

149-
There are several settings which affect the behavior of shard recovery when
144+
Several settings affect the behavior of shard recovery when
150145
your cluster restarts.((("recovery settings")))((("configuration changes, important", "recovery settings"))) First, we need to understand what happens if nothing is
151146
configured.
152147

153-
Imagine you have 10 nodes, and each node holds a single shard -- either a primary
154-
or a replica -- in a 5 primary / 1 replica index. You take your
155-
entire cluster offline for maintenance (installing new drives, etc). When you
148+
Imagine you have ten nodes, and each node holds a single shard--either a primary
149+
or a replica--in a 5 primary / 1 replica index. You take your
150+
entire cluster offline for maintenance (installing new drives, for example). When you
156151
restart your cluster, it just so happens that five nodes come online before
157152
the other five.
158153

159-
Maybe the switch to the other five is being flaky and they didn't
154+
Maybe the switch to the other five is being flaky, and they didn't
160155
receive the restart command right away. Whatever the reason, you have five nodes
161-
online. These five nodes will gossip with eachother, elect a master and form a
162-
cluster. They notice that data is no longer evenly distributed since five
156+
online. These five nodes will gossip with each other, elect a master, and form a
157+
cluster. They notice that data is no longer evenly distributed, since five
163158
nodes are missing from the cluster, and immediately start replicating new
164159
shards between each other.
165160

166161
Finally, your other five nodes turn on and join the cluster. These nodes see
167162
that _their_ data is being replicated to other nodes, so they delete their local
168-
data (since it is now redundant, and may be out-dated). Then the cluster starts
169-
to rebalance even more, since the cluster size just went from five to 10.
163+
data (since it is now redundant, and may be outdated). Then the cluster starts
164+
to rebalance even more, since the cluster size just went from five to ten.
170165

171-
During this whole process, your nodes are thrashing the disk and network moving
172-
data around...for no good reason. For large clusters with terrabytes of data,
166+
During this whole process, your nodes are thrashing the disk and network, moving
167+
data around--for no good reason. For large clusters with terabytes of data,
173168
this useless shuffling of data can take a _really long time_. If all the nodes
174169
had simply waited for the cluster to come online, all the data would have been
175170
local and nothing would need to move.
176171

177172
Now that we know the problem, we can configure a few settings to alleviate it.
178-
First, we need give Elasticsearch a hard limit:
173+
First, we need to give Elasticsearch a hard limit:
179174

180175
[source,yaml]
181176
----
182177
gateway.recover_after_nodes: 8
183178
----
184179

185-
This will prevent Elasticsearch from starting a recovery until at least 8 nodes
186-
are present. The value for this setting is up to personal preference: how
180+
This will prevent Elasticsearch from starting a recovery until at least eight nodes
181+
are present. The value for this setting is a matter of personal preference: how
187182
many nodes do you want present before you consider your cluster functional?
188-
In this case we are setting it to `8`, which means the cluster is inoperable
189-
unless there are 8 nodes.
183+
In this case, we are setting it to `8`, which means the cluster is inoperable
184+
unless there are eight nodes.
190185

191186
Then we tell Elasticsearch how many nodes _should_ be in the cluster, and how
192187
long we want to wait for all those nodes:
@@ -197,14 +192,14 @@ gateway.expected_nodes: 10
197192
gateway.recover_after_time: 5m
198193
----
199194

200-
What this means is that Elasticsearch will:
195+
What this means is that Elasticsearch will do the following:
201196

202-
- Wait for 8 nodes to be present
203-
- Begin recovering after five minutes, OR after 10 nodes have joined the cluster,
197+
- Wait for eight nodes to be present
198+
- Begin recovering after 5 minutes _or_ after ten nodes have joined the cluster,
204199
whichever comes first.
205200

206201
These three settings allow you to avoid the excessive shard swapping that can
207-
occur on cluster restarts. It can literally make recover take seconds instead
202+
occur on cluster restarts. It can literally make recovery take seconds instead
208203
of hours.
209204

210205

@@ -216,19 +211,19 @@ Elasticsearch nodes will receive these pings and respond. A cluster is formed
216211
shortly after.
217212

218213
Multicast is excellent for development, since you don't need to do anything. Turn
219-
a few nodes on and they automatically find each other and form a cluster.
214+
a few nodes on, and they automatically find each other and form a cluster.
220215

221216
This ease of use is the exact reason you should disable it in production. The
222217
last thing you want is for nodes to accidentally join your production network, simply
223218
because they received an errant multicast ping. There is nothing wrong with
224-
multicast _per-se_. Multicast simply leads to silly problems, and can be a bit
225-
more fragile (e.g. a network engineer fiddles with the network without telling
226-
you...and all of a sudden nodes can't find each other anymore).
219+
multicast _per se_. Multicast simply leads to silly problems, and can be a bit
220+
more fragile (for example, a network engineer fiddles with the network without telling
221+
you--and all of a sudden nodes can't find each other anymore).
227222

228-
In production, it is recommended to use Unicast instead of Multicast. This works
223+
In production, it is recommended to use unicast instead of multicast. This works
229224
by providing Elasticsearch a list of nodes that it should try to contact. Once
230225
the node contacts a member of the unicast list, it will receive a full cluster
231-
state which lists all nodes in the cluster. It will then proceed to contact
226+
state that lists all nodes in the cluster. It will then proceed to contact
232227
the master and join.
233228

234229
This means your unicast list does not need to hold all the nodes in your cluster.
@@ -241,7 +236,7 @@ This setting is configured in your `elasticsearch.yml`:
241236
discovery.zen.ping.multicast.enabled: false <1>
242237
discovery.zen.ping.unicast.hosts: ["host1", "host2:port"]
243238
----
244-
<1> Make sure you disable multicast, since it can operate in parallel with unicast
239+
<1> Make sure you disable multicast, since it can operate in parallel with unicast.
245240

246241

247242

0 commit comments

Comments
 (0)