Ability to Read-Only a Data Stream #100439
Labels
:Data Management/Data streams
Data streams and their lifecycles
>enhancement
Team:Data Management
Meta label for data/management team
Description
Enhancement
It would be nice that a data stream could exist in a strictly historical/archival form, similar to how an index can be made read-only, but at a data stream level (data stream mappings can't be updated, data stream can't be rolled over, even if "forced").
Use Case
Because data streams mainly target timeseries data, there are cases where a data stream is no longer actively needed, but needs to be kept around because it contains historical data. An issue that I've observed is that there is no way to mark a data stream as archived and that it shouldn't receive things like mapping updates or rollovers.
An example of where this is an issue is with Elastic Fleet. If I have a data stream with a specific
namespace
value, that I eventually stop using, Elasticsearch knows not to roll it over because the leading index has 0 docs, but Fleet doesn't seem to know this, and if the Fleet integration gets an update that changes mappings, Fleet will see this data stream and force a rollover, causing 0 doc indices to exist.It would be nice to have the ability to mark a data stream in a way that things like Fleet (or other similar products), no-op changes on archived data streams.
The text was updated successfully, but these errors were encountered: