Skip to content

Adding support to DLM for the new .workflows-execution-data-stream-logs data stream #139162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
masseyke wants to merge 1 commit into
base: patch/serverless-fix
Choose a base branch
from
Open

Adding support to DLM for the new .workflows-execution-data-stream-logs data stream #139162

masseyke wants to merge 1 commit into from
+8 −2

Conversation

@masseyke
Copy link
Member

@masseyke masseyke commented Dec 5, 2025

DLM is only allowed to perform actions on "restricted" data streams that are in the whitelist in InternalUsers. A new restricted data stream, .workflows-execution-data-stream-logs , was added in elastic/kibana#242489. This causes DLM to fail because DLM attempts to manage this data stream and fails in rollover because it does not have permissions.
This PR is meant as a stopgap -- we still have the weird situation that:

  • If a new restricted data stream with a lifecycle is added, we don't know until we stand up a server
  • It seems broken that DLM makes an attempt to manage every index with a lifecycle, but causes bad failures if one of those is not in the whitelist -- DLM needs to either be able to manage all data streams that have a lifecycle, or have a way of knowing which ones to skip.

Note: the whitelist was introduced in #96253, and there is more context in the comments on that PR.

@masseyke masseyke added >non-issue :Data Management/Data streams Data streams and their lifecycles auto-backport Automatically create backport pull requests when merged v9.3.0 labels Dec 5, 2025
@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Dec 5, 2025
@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Dec 5, 2025

Pinging @elastic/es-data-management (Team:Data Management)

@masseyke masseyke changed the base branch from main to patch/serverless-fix December 5, 2025 22:59
dakrone
dakrone approved these changes Dec 5, 2025
View reviewed changes
Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@masseyke masseyke mentioned this pull request Dec 5, 2025
Closed
@masseyke masseyke added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Dec 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dakrone dakrone dakrone approved these changes

Assignees

No one assigned

Labels

auto-backport Automatically create backport pull requests when merged auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) :Data Management/Data streams Data streams and their lifecycles >non-issue Team:Data Management Meta label for data/management team v9.3.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@masseyke @elasticsearchmachine @dakrone