Avoid MongoDB write conflicts in MongoJobRepository by replacing sequence-based ID with TSID #5144
+206
−34
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR addresses a concurrency issue in the MongoDBJobRepository.
The current implementation generates numeric identifiers for job/step executions through a sequence document.
Under concurrency, especially with transactional writes, this causes frequent WriteConflict errors from MongoDB.
This PR replaces the numeric sequence generation with application-generated, unique, time-ordered identifier.
This eliminates write conflicts caused by sequence updates.
Problem
MongoDB uses optimistic concurrency control, and concurrent writes to the same document cause conflicts.
This produces high contention on a single document under parallel workloads.
Solution
Replace sequence-based numeric ID generation with TSID.
Implements TSID algorithm(42-bit timestamp + 10-bit node ID + 12-bit sequence)
TSID characteristics
Fixes #4960