[tag] Force creating auto-tag even no data when calling trigger_tag_automatic_creation #7227
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Make sure the auto-tag exists as expected even when no data is provided.
The former version of procedure
trigger_tag_automatic_creationcreates the tag only if it finds the right snapshot which is aftertag.creation-delay. This is not always okay.As we know, the auto-tag would be created within the Flink CDC job. But it would fail if the Flink job:
To meet the auto-tag expectation, I enhanced the
trigger_tag_automatic_creationprocedure by adding a newforceparameter, defaulting to false. When it's true, and the auto-tag still does not exist after the first attempt at auto-creation, we make an empty commit to force the creation of the snapshot which could be used to create the auto-tag during the second run.Tests
Flink UT
TriggerTagAutomaticCreationProcedureITCaseand Spark UTTriggerTagAutomaticCreationProcedureTestwere adapted.API and Format
NO
Documentation
Doc provied for the new parameter.