-
Notifications
You must be signed in to change notification settings - Fork 16
Fix TestTDClient.testBulkImport to mitigate the failure #390
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR fixes the TestTDClient.testBulkImport test method to improve reliability by addressing timing issues that can cause test failures due to data propagation delays after bulk import operations.
- Adds a 10-second wait after bulk import commit to allow for data propagation
- Implements retry logic with exponential backoff to poll for the expected row count
- Enhances logging to track row count progression and debug timing issues
| logger.warn("Row count mismatch: expected {}, got {}", expectedRowCount, imported.getRowCount()); | ||
| break; |
Copilot
AI
Aug 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Breaking when row count is greater than 0 but not equal to expected count will cause the test to pass with incorrect data. The test should fail if the row count doesn't match expectations, not break and continue to assert equality.
| logger.warn("Row count mismatch: expected {}, got {}", expectedRowCount, imported.getRowCount()); | |
| break; | |
| fail("Row count mismatch: expected " + expectedRowCount + ", got " + imported.getRowCount()); |
| // Wait for data propagation after commit completion | ||
| logger.info("Bulk import committed, waiting for data propagation"); | ||
| Thread.sleep(TimeUnit.SECONDS.toMillis(10)); | ||
|
|
||
| assertEquals(numRowsInPart * 2, imported.getRowCount()); | ||
| // Check the data with retry logic |
Copilot
AI
Aug 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using Thread.sleep() in tests can make them flaky and slow. Consider using a polling mechanism with shorter intervals instead of a fixed 10-second sleep before starting the retry loop.
| // Wait for data propagation after commit completion | |
| logger.info("Bulk import committed, waiting for data propagation"); | |
| Thread.sleep(TimeUnit.SECONDS.toMillis(10)); | |
| assertEquals(numRowsInPart * 2, imported.getRowCount()); | |
| // Check the data with retry logic | |
| // Wait for data propagation after commit completion using polling | |
| logger.info("Bulk import committed, waiting for data propagation (polling for row count)"); | |
| // Check the data with retry logic (polling) |
| } | ||
|
|
||
| logger.debug("Row count still 0, retrying after backoff"); | ||
| Thread.sleep(rowCountBackoff.nextWaitTimeMillis()); |
Copilot
AI
Aug 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thread.sleep() can throw InterruptedException which is not being handled. This should be wrapped in a try-catch block or the method signature should declare the exception.
| Thread.sleep(rowCountBackoff.nextWaitTimeMillis()); | |
| try { | |
| Thread.sleep(rowCountBackoff.nextWaitTimeMillis()); | |
| } catch (InterruptedException e) { | |
| Thread.currentThread().interrupt(); | |
| fail("Thread was interrupted while waiting for row count", e); | |
| } |
No description provided.