Skip to content

Comments

HDDS-14629. Code cleanup after HDDS-13906#9773

Open
Russole wants to merge 4 commits intoapache:masterfrom
Russole:HDDS-14629
Open

HDDS-14629. Code cleanup after HDDS-13906#9773
Russole wants to merge 4 commits intoapache:masterfrom
Russole:HDDS-14629

Conversation

@Russole
Copy link
Contributor

@Russole Russole commented Feb 16, 2026

What changes were proposed in this pull request?

  • Refactor OMDBArchiver to own and initialize the hardlink mapping internally, removing setHardLinkFileMap() and nullable state.
  • Normalize and convert paths to absolute form before calling relativize() in writeHardlinkFile to prevent abs/relative path mismatch exceptions.
  • Stop exposing internal hardlink map and use recordHardLinkMapping() instead.
  • Return HTTP 500 when archive streaming fails to ensure proper error handling.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14629

How was this patch tested?

path = getDbStore().getDbLocation().toPath().resolve(dbFile.getFileName()).toAbsolutePath().toString();
}
omdbArchiver.getHardLinkFileMap().put(path, fileId);
String path = dbFile.toFile().getAbsolutePath();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this one be null or empty?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dbFile comes from either Files.list(...) or from a List<Path> created in extractSSTFilesFromCompactionLog().

From directory listing:

try (Stream<Path> files = Files.list(dbDir)) {
  collectFilesFromDir(..., files, ...);
}

From compaction log (in extractSSTFilesFromCompactionLog()):

sstFiles.add(sstBackupDir.resolve(f.getFileName() + ROCKSDB_SST_SUFFIX));

In both cases, dbFile is a valid Path (no null elements). Also, Path.resolve() never returns null — it returns a Path or throws NullPointerException if the argument is null.

So dbFile.toFile().getAbsolutePath() cannot be null in this flow.

@Russole
Copy link
Contributor Author

Russole commented Feb 17, 2026

Thanks @yandrey321 for the review.

@Russole Russole requested a review from yandrey321 February 17, 2026 03:37
Copy link
Contributor

@Gargi-jais11 Gargi-jais11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Russole for the patch.
Please find inlined comments.

Comment on lines 409 to 411
Path data = Files.createTempFile(DATA_PREFIX, DATA_SUFFIX);
Path metaDirPath = OMStorage.getOmDbDir(conf).toPath();
Path metaDirPath = OMStorage.getOmDbDir(conf).toPath()
.toAbsolutePath().normalize();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The temporary file data created is never cleaned up. Consider wrapping the logic around try block and delete the temp file in a finally block. Without this fix, every checkpoint operation creates an orphned temp file. Over time, this will consume disk space and could eventually cause the system to run out of space, leading to operation failures.

Path data = Files.createTempFile(DATA_PREFIX, DATA_SUFFIX);
  try {
    Path metaDirPath = OMStorage.getOmDbDir(conf).toPath()
        .toAbsolutePath().normalize();
    StringBuilder sb = new StringBuilder();

    for (Map.Entry<String, String> entry : hardlinkFileMap.entrySet()) {
      Path p = new File(entry.getKey()).toPath();
      if (!p.isAbsolute()) {
        p = metaDirPath.resolve(p);
      }
      p = p.toAbsolutePath().normalize();
      String fileId = entry.getValue();
      Path relativePath = metaDirPath.relativize(p);
      sb.append(relativePath).append('\t').append(fileId).append('\n');
    }
    Files.write(data, sb.toString().getBytes(StandardCharsets.UTF_8), StandardOpenOption.TRUNCATE_EXISTING);
    includeFile(data.toFile(), OmSnapshotManager.OM_HARDLINK_FILE, archiveOutputStream);
  } finally {
    Files.deleteIfExists(data);
  }

@Russole
Copy link
Contributor Author

Russole commented Feb 18, 2026

Thanks for the suggestion, @Gargi-jais11. The temp file is now deleted in a finally block.

@Russole Russole requested a review from Gargi-jais11 February 18, 2026 04:58
@Russole
Copy link
Contributor Author

Russole commented Feb 19, 2026

@jojochuang Hi, this PR is currently under review.
If you have a chance, I’d really appreciate your feedback as well. Thanks!

Copy link

@yandrey321 yandrey321 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

@Gargi-jais11 Gargi-jais11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Gargi-jais11
Copy link
Contributor

@jojochuang Please review the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants