You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Sep 21, 2021. It is now read-only.
Copy file name to clipboardexpand all lines: 510_Deployment/40_config.asciidoc
+69-74
Original file line number
Diff line number
Diff line change
@@ -2,31 +2,31 @@
2
2
Elasticsearch ships with _very good_ defaults,((("deployment", "configuration changes, important")))((("configuration changes, important"))) especially when it comes to performance-
3
3
related settings and options. When in doubt, just leave
4
4
the settings alone. We have witnessed countless dozens of clusters ruined
5
-
by errant settings because the administrator thought they could turn a knob
6
-
and gain 100x improvement.
5
+
by errant settings because the administrator thought he could turn a knob
6
+
and gain 100-fold improvement.
7
7
8
8
[NOTE]
9
9
====
10
10
Please read this entire section! All configurations presented are equally
11
-
important, and are not listed in any particular "importance" order. Please read
11
+
important, and are not listed in any particular order. Please read
12
12
through all configuration options and apply them to your cluster.
13
13
====
14
14
15
-
Other databases may require tuning, but by-and-far, Elasticsearch does not.
15
+
Other databases may require tuning, but byand large, Elasticsearch does not.
16
16
If you are hitting performance problems, the solution is usually better data
17
17
layout or more nodes. There are very few "magic knobs" in Elasticsearch.
18
-
If there was...we'd have turned it already!
18
+
If there were, we'd have turned them already!
19
19
20
20
With that said, there are some _logistical_ configurations that should be changed
21
-
for production. These changes are either to make your life easier, or because
22
-
there is no way to set a good default (e.g. it depends on your cluster layout).
21
+
for production. These changes are necessary either to make your life easier, or because
22
+
there is no way to set a good default (because it depends on your cluster layout).
23
23
24
24
25
-
==== Assign names
25
+
==== Assign Names
26
26
27
27
Elasticseach by default starts a cluster named `elasticsearch`. ((("configuration changes, important", "assigning names"))) It is wise
28
28
to rename your production cluster to something else, simply to prevent accidents
29
-
where someone's laptop joins the cluster. A simple change to `elasticsearch_production`
29
+
whereby someone's laptop joins the cluster. A simple change to `elasticsearch_production`
30
30
can save a lot of heartache.
31
31
32
32
This can be changed in your `elasticsearch.yml` file:
@@ -36,17 +36,17 @@ This can be changed in your `elasticsearch.yml` file:
36
36
cluster.name: elasticsearch_production
37
37
----
38
38
39
-
Similarly, it is wise to change the names of your nodes. You've probably
40
-
noticed by now, but Elasticsearch will assign a random Marvel Superhero name
41
-
to your nodes at startup. This is cute in development...less cute when it is
42
-
3am and you are trying to remember which physical machine was "Tagak the Leopard Lord".
39
+
Similarly, it is wise to change the names of your nodes. As you've probably
40
+
noticed by now, Elasticsearch assigns a random Marvel superhero name
41
+
to your nodes at startup. This is cute in development--but less cute when it is
42
+
3a.m. and you are trying to remember which physical machine was Tagak the Leopard Lord.
43
43
44
-
More importantly, since these names are generated on startup, each time you
45
-
restart your node it will get a new name. This can make logs very confusing,
44
+
More important, since these names are generated on startup, each time you
45
+
restart your node, it will get a new name. This can make logs confusing,
46
46
since the names of all the nodes are constantly changing.
47
47
48
48
Boring as it might be, we recommend you give each node a name that makes sense
49
-
to you - a plain, descriptive name. This is also configured in your `elasticsearch.yml`:
49
+
to you--a plain, descriptive name. This is also configured in your `elasticsearch.yml`:
By default, Elasticsearch will place the plugins,((("configuration changes, important", "paths")))((("paths"))) logs and --
60
-
most importantly -- your data in the installation directory. This can lead to
61
-
unfortunate accidents, where the installation directory is accidentally overwritten
62
-
by a new installation of ES. If you aren't careful, you can erase all of your data.
59
+
By default, Elasticsearch will place the plug-ins,((("configuration changes, important", "paths")))((("paths"))) logs, and--most important--your data in the installation directory. This can lead to
60
+
unfortunate accidents, whereby the installation directory is accidentally overwritten
61
+
by a new installation of Elasticsearch. If you aren't careful, you can erase all your data.
63
62
64
-
Don't laugh...we've seen it happen more than a few times.
63
+
Don't laugh--we've seen it happen more than a few times.
65
64
66
65
The best thing to do is relocate your data directory outside the installation
67
-
location. You can optionally move your plugin and log directories as well.
66
+
location. You can optionally move your plug-in and log directories as well.
68
67
69
-
This can be changed via:
68
+
This can be changed as follows:
70
69
71
70
[source,yaml]
72
71
----
@@ -78,41 +77,38 @@ path.logs: /path/to/logs
78
77
# Path to where plugins are installed:
79
78
path.plugins: /path/to/plugins
80
79
----
81
-
<1> Notice that you can specify more than one directory for data using comma
82
-
separated lists.
80
+
<1> Notice that you can specify more than one directory for data by using comma-separated lists.
83
81
84
-
Data can be saved to multiple directories, and if each of these directories
85
-
are mounted on a different hard drive, this is a simple and effective way to
86
-
setup a "software RAID 0". Elasticsearch will automatically stripe
82
+
Data can be saved to multiple directories, and if each directory
83
+
is mounted on a different hard drive, this is a simple and effective way to
84
+
set up a software RAID 0. Elasticsearch will automatically stripe
87
85
data between the different directories, boosting performance
88
86
89
87
==== Minimum Master Nodes
90
88
91
-
This setting, called `minimum_master_nodes` is _extremely_ important to the
92
-
stability of your cluster.((("configuration changes, important", "minimum_master_nodes setting")))((("minimum_master_nodes setting"))) This setting helps prevent "split brains", a situation
93
-
where two masters exist in a single cluster.
89
+
The `minimum_master_nodes` setting is _extremely_ important to the
90
+
stability of your cluster.((("configuration changes, important", "minimum_master_nodes setting")))((("minimum_master_nodes setting"))) This setting helps prevent _split brains_, the existence of two masters in a single cluster.
94
91
95
-
When you have a split-brain, your cluster is at danger of losing data. Because
96
-
the master is considered the "supreme ruler" of the cluster, it decides
97
-
when new indices can be created, how shards are moved, etc. If you have _two_
98
-
masters, data integrity becomes perilous, since you have two different nodes
92
+
When you have a splitbrain, your cluster is at danger of losing data. Because
93
+
the master is considered the supreme ruler of the cluster, it decides
94
+
when new indices can be created, how shards are moved, and so forth. If you have _two_
95
+
masters, data integrity becomes perilous, since you have two nodes
99
96
that think they are in charge.
100
97
101
98
This setting tells Elasticsearch to not elect a master unless there are enough
102
99
master-eligible nodes available. Only then will an election take place.
103
100
104
-
This setting should always be configured to a quorum (majority) of your master-
105
-
eligible nodes.((("quorum"))) A quorum is `(number of master-eligible nodes / 2) + 1`.
106
-
Some examples:
101
+
This setting should always be configured to a quorum (majority) of your master-eligible nodes.((("quorum"))) A quorum is `(number of master-eligible nodes / 2) + 1`.
102
+
Here are some examples:
107
103
108
104
- If you have ten regular nodes (can hold data, can become master), a quorum is
109
-
`6`
110
-
- If you have three dedicated master nodes and 100 data nodes, the quorum is `2`,
111
-
since you only need to count nodes that are master-eligible
112
-
- If you have two regular nodes...you are in a conundrum. A quorum would be
105
+
`6`.
106
+
- If you have three dedicated master nodes and a hundred data nodes, the quorum is `2`,
107
+
since you need to count only nodes that are mastereligible.
108
+
- If you have two regular nodes, you are in a conundrum. A quorum would be
113
109
`2`, but this means a loss of one node will make your cluster inoperable. A
114
110
setting of `1` will allow your cluster to function, but doesn't protect against
115
-
split brain. It is best to have a minimum of 3 nodes in situations like this.
111
+
split brain. It is best to have a minimum of three nodes in situations like this.
116
112
117
113
This setting can be configured in your `elasticsearch.yml` file:
But because Elasticsearch clusters are dynamic, you could easily add or remove
125
-
nodes which will change the quorum. It would be extremely irritating if you had
121
+
nodes that will change the quorum. It would be extremely irritating if you had
126
122
to push new configurations to each node and restart your whole cluster just to
127
123
change the setting.
128
124
129
125
For this reason, `minimum_master_nodes` (and other settings) can be configured
130
-
via a dynamic API call. You can change the setting while your cluster is online
131
-
using:
126
+
via a dynamic API call. You can change the setting while your cluster is online:
132
127
133
128
[source,js]
134
129
----
@@ -144,49 +139,49 @@ This will become a persistent setting that takes precedence over whatever is
144
139
in the static configuration. You should modify this setting whenever you add or
145
140
remove master-eligible nodes.
146
141
147
-
==== Recovery settings
142
+
==== Recovery Settings
148
143
149
-
There are several settings which affect the behavior of shard recovery when
144
+
Several settings affect the behavior of shard recovery when
150
145
your cluster restarts.((("recovery settings")))((("configuration changes, important", "recovery settings"))) First, we need to understand what happens if nothing is
151
146
configured.
152
147
153
-
Imagine you have 10 nodes, and each node holds a single shard -- either a primary
154
-
or a replica -- in a 5 primary / 1 replica index. You take your
155
-
entire cluster offline for maintenance (installing new drives, etc). When you
148
+
Imagine you have ten nodes, and each node holds a single shard--either a primary
149
+
or a replica--in a 5 primary / 1 replica index. You take your
150
+
entire cluster offline for maintenance (installing new drives, for example). When you
156
151
restart your cluster, it just so happens that five nodes come online before
157
152
the other five.
158
153
159
-
Maybe the switch to the other five is being flaky and they didn't
154
+
Maybe the switch to the other five is being flaky, and they didn't
160
155
receive the restart command right away. Whatever the reason, you have five nodes
161
-
online. These five nodes will gossip with eachother, elect a master and form a
162
-
cluster. They notice that data is no longer evenly distributed since five
156
+
online. These five nodes will gossip with each other, elect a master, and form a
157
+
cluster. They notice that data is no longer evenly distributed, since five
163
158
nodes are missing from the cluster, and immediately start replicating new
164
159
shards between each other.
165
160
166
161
Finally, your other five nodes turn on and join the cluster. These nodes see
167
162
that _their_ data is being replicated to other nodes, so they delete their local
168
-
data (since it is now redundant, and may be out-dated). Then the cluster starts
169
-
to rebalance even more, since the cluster size just went from five to 10.
163
+
data (since it is now redundant, and may be outdated). Then the cluster starts
164
+
to rebalance even more, since the cluster size just went from five to ten.
170
165
171
-
During this whole process, your nodes are thrashing the disk and network moving
172
-
data around...for no good reason. For large clusters with terrabytes of data,
166
+
During this whole process, your nodes are thrashing the disk and network, moving
167
+
data around--for no good reason. For large clusters with terabytes of data,
173
168
this useless shuffling of data can take a _really long time_. If all the nodes
174
169
had simply waited for the cluster to come online, all the data would have been
175
170
local and nothing would need to move.
176
171
177
172
Now that we know the problem, we can configure a few settings to alleviate it.
178
-
First, we need give Elasticsearch a hard limit:
173
+
First, we need to give Elasticsearch a hard limit:
179
174
180
175
[source,yaml]
181
176
----
182
177
gateway.recover_after_nodes: 8
183
178
----
184
179
185
-
This will prevent Elasticsearch from starting a recovery until at least 8 nodes
186
-
are present. The value for this setting is up to personal preference: how
180
+
This will prevent Elasticsearch from starting a recovery until at least eight nodes
181
+
are present. The value for this setting is a matter of personal preference: how
187
182
many nodes do you want present before you consider your cluster functional?
188
-
In this case we are setting it to `8`, which means the cluster is inoperable
189
-
unless there are 8 nodes.
183
+
In this case, we are setting it to `8`, which means the cluster is inoperable
184
+
unless there are eight nodes.
190
185
191
186
Then we tell Elasticsearch how many nodes _should_ be in the cluster, and how
192
187
long we want to wait for all those nodes:
@@ -197,14 +192,14 @@ gateway.expected_nodes: 10
197
192
gateway.recover_after_time: 5m
198
193
----
199
194
200
-
What this means is that Elasticsearch will:
195
+
What this means is that Elasticsearch will do the following:
201
196
202
-
- Wait for 8 nodes to be present
203
-
- Begin recovering after five minutes, OR after 10 nodes have joined the cluster,
197
+
- Wait for eight nodes to be present
198
+
- Begin recovering after 5 minutes _or_ after ten nodes have joined the cluster,
204
199
whichever comes first.
205
200
206
201
These three settings allow you to avoid the excessive shard swapping that can
207
-
occur on cluster restarts. It can literally make recover take seconds instead
202
+
occur on cluster restarts. It can literally make recovery take seconds instead
208
203
of hours.
209
204
210
205
@@ -216,19 +211,19 @@ Elasticsearch nodes will receive these pings and respond. A cluster is formed
216
211
shortly after.
217
212
218
213
Multicast is excellent for development, since you don't need to do anything. Turn
219
-
a few nodes on and they automatically find each other and form a cluster.
214
+
a few nodes on, and they automatically find each other and form a cluster.
220
215
221
216
This ease of use is the exact reason you should disable it in production. The
222
217
last thing you want is for nodes to accidentally join your production network, simply
223
218
because they received an errant multicast ping. There is nothing wrong with
224
-
multicast _per-se_. Multicast simply leads to silly problems, and can be a bit
225
-
more fragile (e.g. a network engineer fiddles with the network without telling
226
-
you...and all of a sudden nodes can't find each other anymore).
219
+
multicast _perse_. Multicast simply leads to silly problems, and can be a bit
220
+
more fragile (for example, a network engineer fiddles with the network without telling
221
+
you--and all of a sudden nodes can't find each other anymore).
227
222
228
-
In production, it is recommended to use Unicast instead of Multicast. This works
223
+
In production, it is recommended to use unicast instead of multicast. This works
229
224
by providing Elasticsearch a list of nodes that it should try to contact. Once
230
225
the node contacts a member of the unicast list, it will receive a full cluster
231
-
state which lists all nodes in the cluster. It will then proceed to contact
226
+
state that lists all nodes in the cluster. It will then proceed to contact
232
227
the master and join.
233
228
234
229
This means your unicast list does not need to hold all the nodes in your cluster.
@@ -241,7 +236,7 @@ This setting is configured in your `elasticsearch.yml`:
0 commit comments