Skip to content

Commit 786104e

Browse files
authored
[DOCS] Update snapshot/restore and SLM docs for data streams (#58513)
Updates the existing snapshot/restore and SLM docs to make them aware of data streams.
1 parent 82740f6 commit 786104e

11 files changed

+159
-83
lines changed

docs/reference/glossary.asciidoc

+3-2
Original file line numberDiff line numberDiff line change
@@ -480,8 +480,9 @@ See the {ref}/indices-shrink-index.html[shrink index API].
480480

481481
[[glossary-snapshot]] snapshot ::
482482
// tag::snapshot-def[]
483-
A backup taken from a running {es} cluster.
484-
You can take snapshots of individual indices or of the entire cluster.
483+
A backup taken from a running {es} cluster.
484+
A snapshot can include backups of an entire cluster or only data streams and
485+
indices you specify.
485486
// end::snapshot-def[]
486487

487488
[[glossary-snapshot-lifecycle-policy]] snapshot lifecycle policy ::

docs/reference/ilm/ilm-and-snapshots.asciidoc

+4-4
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
[role="xpack"]
22
[testenv="basic"]
33
[[index-lifecycle-and-snapshots]]
4-
== Restore a managed index
4+
== Restore a managed data stream or index
55

6-
When you restore a snapshot that contains managed indices,
6+
When you restore a managed index or a data stream with managed backing indices,
77
{ilm-init} automatically resumes executing the restored indices' policies.
88
A restored index's `min_age` is relative to when it was originally created or rolled over,
99
not its restoration time.
@@ -12,8 +12,8 @@ an index has been restored from a snapshot.
1212
If you restore an index that was accidentally deleted half way through its month long lifecycle,
1313
it proceeds normally through the last two weeks of its lifecycle.
1414

15-
In some cases, you might want to restore a managed index and
16-
prevent {ilm-init} from immediately executing its policy.
15+
In some cases, you might want to prevent {ilm-init} from immediately executing
16+
its policy on a restored index.
1717
For example, if you are restoring an older snapshot you might want to
1818
prevent it from rapidly progressing through all of its lifecycle phases.
1919
You might want to add or update documents before it's marked read-only or shrunk,

docs/reference/slm/apis/index.asciidoc

+3-3
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,12 @@
44
== Manage the snapshot lifecycle
55

66
You can set up snapshot lifecycle policies to automate the timing, frequency, and retention of snapshots.
7-
Snapshot policies can apply to multiple indices.
7+
Snapshot policies can apply to multiple data streams and indices.
88

99
The snapshot lifecycle management (SLM) <<snapshot-lifecycle-management-api, CRUD APIs>> provide
1010
the building blocks for the snapshot policy features that are part of the Management application in {kib}.
11-
The Snapshot and Restore UI makes it easy to set up policies, register snapshot repositories,
12-
view and manage snapshots, and restore indices.
11+
The Snapshot and Restore UI makes it easy to set up policies, register snapshot repositories,
12+
view and manage snapshots, and restore data streams or indices.
1313

1414
You can stop and restart SLM to temporarily pause automatic backups while performing
1515
upgrades or other maintenance.

docs/reference/slm/apis/slm-put.asciidoc

+7-6
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ Configuration for each snapshot created by the policy.
5656
====
5757
`ignore_unavailable`::
5858
(Optional, boolean)
59-
If `true`, missing indices do *not* cause snapshot creation to fail and return
59+
If `true`, missing data streams or indices do *not* cause snapshot creation to fail and return
6060
an error. Defaults to `false`.
6161
6262
`include_global_state`::
@@ -65,14 +65,15 @@ If `true`, cluster states are included in snapshots. Defaults to `false`.
6565
6666
`indices`::
6767
(Optional, array of strings)
68-
Array of index names or wildcard pattern of index names included in snapshots. It
69-
supports <<date-math-index-names,date math>> expressions.
68+
Array of data streams and indices to include in snapshots.
69+
<<date-math-index-names,Date math>> and wildcard (`*`) expressions are
70+
supported.
7071
====
7172

7273
`name`::
7374
(Required, string)
74-
Name automatically assigned to each snapshot created by the policy. This value
75-
supports the same <<date-math-index-names,date math>> supported in index names.
75+
Name automatically assigned to each snapshot created by the policy.
76+
<<date-math-index-names,Date math>> is supported.
7677
To prevent conflicting snapshot names, a UUID is automatically appended to each
7778
snapshot name.
7879

@@ -141,7 +142,7 @@ PUT /_slm/policy/daily-snapshots
141142
<2> The name each snapshot should be given
142143
<3> Which repository to take the snapshot in
143144
<4> Any extra snapshot configuration
144-
<5> Which indices the snapshot should contain
145+
<5> Data streams and indices the snapshot should contain
145146
<6> Optional retention configuration
146147
<7> Keep snapshots for 30 days
147148
<8> Always keep at least 5 successful snapshots, even if they're more than 30 days old

docs/reference/slm/apis/slm-stop.asciidoc

+1-1
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ cluster privilege to use this API. For more information, see
2727

2828
Halts all {slm} ({slm-init}) operations and stops the {slm-init} plugin.
2929
This is useful when you are performing maintenance on a cluster and need to
30-
prevent {slm-init} from performing any actions on your indices.
30+
prevent {slm-init} from performing any actions on your data streams or indices.
3131
Stopping {slm-init} does not stop any snapshots that are in progress.
3232
You can manually trigger snapshots with the <<slm-api-execute-lifecycle>> even if {slm-init} is stopped.
3333

docs/reference/slm/getting-started-slm.asciidoc

+6-6
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
[[getting-started-snapshot-lifecycle-management]]
44
=== Tutorial: Automate backups with {slm-init}
55

6-
This tutorial demonstrates how to automate daily backups of {es} indices using an {slm-init} policy.
7-
The policy takes <<modules-snapshots, snapshots>> of all indices in the cluster
6+
This tutorial demonstrates how to automate daily backups of {es} data streams and indices using an {slm-init} policy.
7+
The policy takes <<modules-snapshots, snapshots>> of all data streams and indices in the cluster
88
and stores them in a local repository.
99
It also defines a retention policy and automatically deletes snapshots
1010
when they are no longer needed.
@@ -47,7 +47,7 @@ PUT /_snapshot/my_repository
4747

4848
Once you have a repository in place,
4949
you can define an {slm-init} policy to take snapshots automatically.
50-
The policy defines when to take snapshots, which indices should be included,
50+
The policy defines when to take snapshots, which data streams or indices should be included,
5151
and what to name the snapshots.
5252
A policy can also specify a <<slm-retention,retention policy>> and
5353
automatically delete snapshots when they are no longer needed.
@@ -58,7 +58,7 @@ Snapshots are incremental and make efficient use of storage.
5858
You can define and manage policies through {kib} Management or with the put policy API.
5959

6060
For example, you could define a `nightly-snapshots` policy
61-
to back up all of your indices daily at 2:30AM UTC.
61+
to back up all of your data streams and indices daily at 2:30AM UTC.
6262

6363
A put policy request defines the policy configuration in JSON:
6464

@@ -86,13 +86,13 @@ PUT /_slm/policy/nightly-snapshots
8686
<<date-math-index-names,date math>> to include the current date in the snapshot name
8787
<3> Where to store the snapshot
8888
<4> The configuration to be used for the snapshot requests (see below)
89-
<5> Which indices to include in the snapshot: all indices
89+
<5> Which data streams or indices to include in the snapshot: all data streams and indices
9090
<6> Optional retention policy: keep snapshots for 30 days,
9191
retaining at least 5 and no more than 50 snapshots regardless of age
9292

9393
You can specify additional snapshot configuration options to customize how snapshots are taken.
9494
For example, you could configure the policy to fail the snapshot
95-
if one of the specified indices is missing.
95+
if one of the specified data streams or indices is missing.
9696
For more information about snapshot options, see <<snapshots-take-snapshot,snapshot requests>>.
9797

9898
[discrete]

docs/reference/slm/index.asciidoc

+2-2
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,12 @@
44
== {slm-init}: Manage the snapshot lifecycle
55

66
You can set up snapshot lifecycle policies to automate the timing, frequency, and retention of snapshots.
7-
Snapshot policies can apply to multiple indices.
7+
Snapshot policies can apply to multiple data streams and indices.
88

99
The {slm} ({slm-init}) <<snapshot-lifecycle-management-api, CRUD APIs>> provide
1010
the building blocks for the snapshot policy features that are part of {kib} Management.
1111
{kibana-ref}/snapshot-repositories.html[Snapshot and Restore] makes it easy to
12-
set up policies, register snapshot repositories, view and manage snapshots, and restore indices.
12+
set up policies, register snapshot repositories, view and manage snapshots, and restore data streams or indices.
1313

1414
You can stop and restart {slm-init} to temporarily pause automatic backups while performing
1515
upgrades or other maintenance.

docs/reference/snapshot-restore/index.asciidoc

+24-20
Original file line numberDiff line numberDiff line change
@@ -5,25 +5,28 @@
55
--
66

77
// tag::snapshot-intro[]
8-
A _snapshot_ is a backup taken from a running {es} cluster.
9-
You can take snapshots of individual indices or of the entire cluster.
10-
Snapshots can be stored in either local or remote repositories.
11-
Remote repositories can reside on S3, HDFS, Azure, Google Cloud Storage,
8+
A _snapshot_ is a backup taken from a running {es} cluster.
9+
You can take snapshots of an entire cluster, including all its data streams and
10+
indices. You can also take snapshots of only specific data streams or indices in
11+
the cluster.
12+
13+
Snapshots can be stored in either local or remote repositories.
14+
Remote repositories can reside on S3, HDFS, Azure, Google Cloud Storage,
1215
and other platforms supported by a repository plugin.
1316

14-
Snapshots are incremental: each snapshot of an index only stores data that
15-
is not part of an earlier snapshot.
17+
Snapshots are incremental: each snapshot only stores data that
18+
is not part of an earlier snapshot.
1619
This enables you to take frequent snapshots with minimal overhead.
17-
// end::snapshot-intro[]
20+
// end::snapshot-intro[]
1821

1922
// tag::restore-intro[]
20-
You can restore snapshots to a running cluster with the <<snapshots-restore-snapshot,restore API>>.
21-
By default, all indices in the snapshot are restored.
22-
Alternatively, you can restore specific indices or restore the cluster state from a snapshot.
23-
When restoring indices, you can modify the index name and selected index settings.
23+
You can restore snapshots to a running cluster with the <<snapshots-restore-snapshot,restore API>>.
24+
By default, all data streams and indices in the snapshot are restored.
25+
However, you can choose to restore only the cluster state or specific data
26+
streams or indices from a snapshot.
2427
// end::restore-intro[]
2528

26-
You must <<snapshots-register-repository, register a snapshot repository>>
29+
You must <<snapshots-register-repository, register a snapshot repository>>
2730
before you can <<snapshots-take-snapshot, take snapshots>>.
2831

2932
You can use <<getting-started-snapshot-lifecycle-management, snapshot lifecycle management>>
@@ -50,7 +53,7 @@ compatibility. Follow the <<setup-upgrade,Upgrade documentation>>
5053
when migrating between versions.
5154

5255
A snapshot contains a copy of the on-disk data structures that make up an
53-
index. This means that snapshots can only be restored to versions of
56+
index or a data stream's backing indices. This means that snapshots can only be restored to versions of
5457
{es} that can read the indices:
5558

5659
* A snapshot of an index created in 6.x can be restored to 7.x.
@@ -67,20 +70,21 @@ We do not recommend restoring snapshots from later {es} versions in earlier
6770
versions. In some cases, the snapshots cannot be restored. For example, a
6871
snapshot taken in 7.6.0 cannot be restored to 7.5.0.
6972

70-
Each snapshot can contain indices created in various versions of {es},
71-
and when restoring a snapshot it must be possible to restore all of the indices
72-
into the target cluster. If any indices in a snapshot were created in an
73-
incompatible version, you will not be able restore the snapshot.
73+
Each snapshot can contain indices created in various versions of {es}. This
74+
includes backing indices created for data streams. When restoring a snapshot, it
75+
must be possible to restore all of these indices into the target cluster. If any
76+
indices in a snapshot were created in an incompatible version, you will not be
77+
able restore the snapshot.
7478

7579
IMPORTANT: When backing up your data prior to an upgrade, keep in mind that you
7680
won't be able to restore snapshots after you upgrade if they contain indices
7781
created in a version that's incompatible with the upgrade version.
7882

79-
If you end up in a situation where you need to restore a snapshot of an index
83+
If you end up in a situation where you need to restore a snapshot of a data stream or index
8084
that is incompatible with the version of the cluster you are currently running,
8185
you can restore it on the latest compatible version and use
82-
<<reindex-from-remote,reindex-from-remote>> to rebuild the index on the current
83-
version. Reindexing from remote is only possible if the original index has
86+
<<reindex-from-remote,reindex-from-remote>> to rebuild the data stream or index on the current
87+
version. Reindexing from remote is only possible if the original data stream or index has
8488
source enabled. Retrieving and reindexing the data can take significantly
8589
longer than simply restoring a snapshot. If you have a large amount of data, we
8690
recommend testing the reindex from remote process with a subset of your data to

docs/reference/snapshot-restore/monitor-snapshot-restore.asciidoc

+3-3
Original file line numberDiff line numberDiff line change
@@ -76,15 +76,15 @@ DELETE /_snapshot/my_backup/snapshot_1
7676
// TEST[continued]
7777

7878
The restore operation uses the standard shard recovery mechanism. Therefore, any currently running restore operation can
79-
be canceled by deleting indices that are being restored. Please note that data for all deleted indices will be removed
79+
be canceled by deleting data streams and indices that are being restored. Please note that data for all deleted data streams and indices will be removed
8080
from the cluster as a result of this operation.
8181

8282
[float]
8383
=== Effect of cluster blocks on snapshot and restore
8484

8585
Many snapshot and restore operations are affected by cluster and index blocks. For example, registering and unregistering
86-
repositories require write global metadata access. The snapshot operation requires that all indices and their metadata as
87-
well as the global metadata were readable. The restore operation requires the global metadata to be writable, however
86+
repositories require write global metadata access. The snapshot operation requires that all indices, backing indices, and their metadata as
87+
well as the global metadata be readable. The restore operation requires the global metadata to be writable, however
8888
the index level blocks are ignored during restore because indices are essentially recreated during restore.
8989
Please note that a repository content is not part of the cluster and therefore cluster blocks don't affect internal
9090
repository operations such as listing or deleting snapshots from an already registered repository.

0 commit comments

Comments
 (0)