[client/flink] generate javadoc for public modules by MehulBatra · Pull Request #1724 · apache/fluss

MehulBatra · 2025-09-18T21:05:28Z

Purpose

Linked issue: close #1712

Implement javadoc generation

Brief change log

Created build_javadocs.sh script to generate javadoc
Configured profile to include only API-relevant modules (fluss-client, fluss-flink) while excluding modules (fluss-protogen, fluss-jmh, fluss-test-*, fluss-metrics, common, server etc.)
Added version-aware output directory structure (website/static/javadoc/VERSION/)
Created automatic redirect page for latest version navigation

Tests

Manual testing of javadoc generation script with different scenarios
Verified generated HTML output structure and content
Validated redirect functionality and version-specific URL routing

Run:

cd website
npm start
http://localhost:3000/javadoc/0.8-SNAPSHOT/

API and Format

No API changes - This is purely a build/documentation improvement that doesn't affect runtime APIs.

Documentation

The generated javadoc itself serves as API reference documentation.

wuchong · 2025-09-21T13:23:26Z

Thanks for the great work @MehulBatra !

Here are my thoughts about the javadoc design:

I don't want to maintain the generated javadoc in the main repo and branches, this will quickly increase our main repo size. which is very heavy.
We can maintain each generated javadocs at the apache/fluss-website in the branches javadoc-<version>.
We need to maintain javadocs for each version, including the released (not include the com.alibaba.fluss versions) and unreleased (main branch).

Here are suggestions about how to implement:

Create a new script build_javadocs.sh under website/ along with build_versioned_docs.sh

The script just generates javadocs for the current branch, the generated javadocs can be put in the folder website/genearted_javadocs/ (add this directory into .gitignore)

Add a build_javadocs.yaml in apache/fluss-website https://github.com/apache/fluss-website/tree/main/.github/workflows

it's a daily scheduled github action
it checkout apache/fluss repo and for each release-* (exclude <= release-0.7 branches) and main branch
for each branch, it runs the build_javadocs.sh script and commit the files in website/genearted_javadocs/ to the branch javadoc-<version> of apache/fluss-website
maybe we can skip branches if the branch has no git log changes in 7 days, to reduce the build cost.
I think we can get the SHORT_VERSION from the branch name easily, so no need to use mvn command to parse

Update website-deploy.yaml in apache/fluss-website, add a step Copy Javadocs before the Deploy website.

it copies javadocs files from branches javadoc-<version> to the directory ./build/javadoc/<version>
you can learn the code from build_versioned_docs.sh in apache/fluss which has similar code.

Add javadoc links in our official docs.

MehulBatra · 2025-09-22T17:12:17Z

Thanks for the feedback @wuchong , I agree on all the points will implement on these lines!

MehulBatra · 2025-09-28T14:25:18Z

Github Action PR: apache/fluss-website#1

Copilot

Pull Request Overview

This PR implements javadoc generation for Apache Fluss's public API modules. The changes add a Maven-based javadoc build system that generates comprehensive API documentation for client and Flink integration modules while excluding internal implementation details.

Added automated javadoc generation script with version-aware output structure
Configured Docusaurus navigation to include Javadocs link
Cleaned up minor formatting issues in root pom.xml

Reviewed Changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.

File	Description
website/docusaurus.config.ts	Added Javadocs navigation link to website header
website/build_javadocs.sh	New script for generating javadocs with Maven, branch-aware versioning, and redirect page creation
pom.xml	Minor formatting cleanup removing extra blank lines

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-28T14:30:34Z

website/docusaurus.config.ts

        {to: '/community/welcome', label: 'Community', position: 'left'},
        {to: '/roadmap', label: 'Roadmap', position: 'left'},
        {to: '/downloads', label: 'Downloads', position: 'left'},
+        {to: '/javadoc', label: 'Javadocs', position: 'left',  target: '_blank'},


Extra space before 'target: '_blank'' - there should be only one space after the comma for consistent formatting.

Suggested change

{to: '/javadoc', label: 'Javadocs', position: 'left', target: '_blank'},

{to: '/javadoc', label: 'Javadocs', position: 'left', target: '_blank'},

Copilot · 2025-09-28T14:30:34Z

website/build_javadocs.sh

+VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout 2>/dev/null || mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
+if [[ -z "$VERSION" ]]; then
+  echo "Error: Could not extract project version"


[nitpick] The command fallback logic could fail silently if both mvnw and mvn commands fail but return empty output. Consider adding explicit error handling to detect when both commands fail.

Suggested change

VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout 2>/dev/null || mvn help:evaluate -Dexpression=project.version -q -DforceStdout)

if [[ -z "$VERSION" ]]; then

echo "Error: Could not extract project version"

# Try to get version with ./mvnw first

VERSION=""

if [[ -x "./mvnw" ]]; then

VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout 2>/dev/null)

MVNW_STATUS=$?

else

MVNW_STATUS=127

fi

if [[ $MVNW_STATUS -ne 0 || -z "$VERSION" ]]; then

VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout 2>/dev/null)

MVN_STATUS=$?

else

MVN_STATUS=0

fi

if [[ ($MVNW_STATUS -ne 0 && $MVN_STATUS -ne 0) || -z "$VERSION" ]]; then

echo "Error: Could not extract project version using either ./mvnw or mvn."

Copilot · 2025-09-28T14:30:35Z

website/build_javadocs.sh

+if (./mvnw -f "$TEMP_POM" clean compile javadoc:aggregate -q 2>/dev/null || mvn -f "$TEMP_POM" clean compile javadoc:aggregate -q); then
+  echo "Javadoc generation completed successfully"
+else
+  echo "Error: Javadoc generation failed"


[nitpick] Similar to the version extraction, this Maven command fallback could benefit from more explicit error handling. The current approach may not distinguish between command not found vs actual build failures.

Suggested change

if (./mvnw -f "$TEMP_POM" clean compile javadoc:aggregate -q 2>/dev/null || mvn -f "$TEMP_POM" clean compile javadoc:aggregate -q); then

echo "Javadoc generation completed successfully"

else

echo "Error: Javadoc generation failed"

# Explicit Maven command fallback with error handling

if [[ -x "./mvnw" ]]; then

echo "Using ./mvnw to build and generate Javadoc..."

./mvnw -f "$TEMP_POM" clean compile javadoc:aggregate -q

MVN_EXIT_CODE=$?

if [[ $MVN_EXIT_CODE -ne 0 ]]; then

echo "Error: Javadoc generation failed using ./mvnw (exit code $MVN_EXIT_CODE)"

exit 1

fi

elif command -v mvn >/dev/null 2>&1; then

echo "Using mvn to build and generate Javadoc..."

mvn -f "$TEMP_POM" clean compile javadoc:aggregate -q

MVN_EXIT_CODE=$?

if [[ $MVN_EXIT_CODE -ne 0 ]]; then

echo "Error: Javadoc generation failed using mvn (exit code $MVN_EXIT_CODE)"

exit 1

fi

else

echo "Error: Neither ./mvnw nor mvn found. Please install Maven or ensure ./mvnw is present."

MehulBatra · 2025-09-28T14:33:37Z

wuchong · 2025-10-06T09:58:11Z

website/build_javadocs.sh

+# Create temporary POM for targeted javadoc generation
+TEMP_POM="temp-javadoc-pom.xml"
+cat > "$TEMP_POM" << EOF
+<?xml version="1.0" encoding="UTF-8"?>


I remembered in the first version, you made the javadoc plugin changes in the root pom.xml. Why using a temp pom xml here? I think it's better to keep all the pom changes in the same place, so it's easier for future developers to modify both places.

wuchong · 2025-10-06T10:00:06Z

website/build_javadocs.sh

+                <version>3.6.3</version>
+                <configuration>
+                    <windowTitle>Apache Fluss \${project.version} API</windowTitle>
+                    <doctitle>Apache Fluss \${project.version} API</doctitle>


If we move the changes to root pom, we can use build-helper-maven-plugin to parse project version into without -SNAPSHOT suffix (e.g., the following base.version property).

<build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>build-helper-maven-plugin</artifactId> <version>3.5.0</version> <executions> <execution> <id>parse-version</id> <goals> <goal>parse-version</goal> </goals> </execution> </executions> </plugin> </plugins> </build>

<properties> <base.version>${parsedVersion.majorVersion}.${parsedVersion.minorVersion}</base.version> </properties>

wuchong · 2025-10-06T10:02:14Z

website/build_javadocs.sh

+
+    <modules>
+        <module>fluss-client</module>
+        <module>fluss-flink/fluss-flink-common</module>


I think we need to include fluss-common, fluss-rpc as well, most client side API are in fluss-common.

wuchong · 2025-10-06T10:03:12Z

website/build_javadocs.sh

+cat > "$JAVADOC_INDEX" << EOF
+<!DOCTYPE html>
+<html>
+<head>
+    <meta charset="UTF-8">
+    <meta http-equiv="refresh" content="0; url=${JAVADOC_VERSION}/">
+    <title>Apache Fluss API Documentation</title>
+</head>
+<body>
+    <p>Redirecting to <a href="${JAVADOC_VERSION}/">Apache Fluss ${JAVADOC_VERSION} API</a></p>
+</body>
+</html>
+EOF


Do we really need this index.html? As it is under javadoc/index.html, so multiple javadoc version will override this file.

wuchong · 2025-10-06T10:05:03Z

Thank you @MehulBatra for the updates. The new architecture looks very good. I think there is only some minor improvements to do. Besides, I think we don't need a navbar for Javadoc, because Javadoc is also versioned, so it should be going into Docs, maybe under Docs -> Fluss APIs -> Javadocs

Mehul Batra added 2 commits September 19, 2025 02:29

added javadoc for public modules

07b8251

remove unwanted comments

4496bba

MehulBatra requested a review from wuchong September 19, 2025 12:43

Mehul Batra added 2 commits September 28, 2025 19:08

address Jark commennts

d511f90

ignore javadocs directory to get added

64c5c59

MehulBatra mentioned this pull request Sep 28, 2025

[CI/CD Javadoc] Workflow Changes apache/fluss-website#1

Open

MehulBatra requested a review from Copilot September 28, 2025 14:27

Copilot AI reviewed Sep 28, 2025

View reviewed changes

wuchong reviewed Oct 6, 2025

View reviewed changes

luoyuxia force-pushed the main branch from 0e2d71f to 74ee888 Compare February 6, 2026 12:32

	{to: '/javadoc', label: 'Javadocs', position: 'left', target: '_blank'},
	{to: '/javadoc', label: 'Javadocs', position: 'left', target: '_blank'},

-VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout 2>/dev/null || mvn help:evaluate -Dexpression=project.version -q -DforceStdout)
-if [[ -z "$VERSION" ]]; then
-  echo "Error: Could not extract project version"
+# Try to get version with ./mvnw first
+VERSION=""
+if [[ -x "./mvnw" ]]; then
+  VERSION=$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout 2>/dev/null)
+  MVNW_STATUS=$?
+else
+  MVNW_STATUS=127
+fi
+if [[ $MVNW_STATUS -ne 0 || -z "$VERSION" ]]; then
+  VERSION=$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout 2>/dev/null)
+  MVN_STATUS=$?
+else
+  MVN_STATUS=0
+fi
+if [[ ($MVNW_STATUS -ne 0 && $MVN_STATUS -ne 0) || -z "$VERSION" ]]; then
+  echo "Error: Could not extract project version using either ./mvnw or mvn."

-if (./mvnw -f "$TEMP_POM" clean compile javadoc:aggregate -q 2>/dev/null || mvn -f "$TEMP_POM" clean compile javadoc:aggregate -q); then
-  echo "Javadoc generation completed successfully"
-else
-  echo "Error: Javadoc generation failed"
+# Explicit Maven command fallback with error handling
+if [[ -x "./mvnw" ]]; then
+  echo "Using ./mvnw to build and generate Javadoc..."
+  ./mvnw -f "$TEMP_POM" clean compile javadoc:aggregate -q
+  MVN_EXIT_CODE=$?
+  if [[ $MVN_EXIT_CODE -ne 0 ]]; then
+    echo "Error: Javadoc generation failed using ./mvnw (exit code $MVN_EXIT_CODE)"
+    exit 1
+  fi
+elif command -v mvn >/dev/null 2>&1; then
+  echo "Using mvn to build and generate Javadoc..."
+  mvn -f "$TEMP_POM" clean compile javadoc:aggregate -q
+  MVN_EXIT_CODE=$?
+  if [[ $MVN_EXIT_CODE -ne 0 ]]; then
+    echo "Error: Javadoc generation failed using mvn (exit code $MVN_EXIT_CODE)"
+    exit 1
+  fi
+else
+  echo "Error: Neither ./mvnw nor mvn found. Please install Maven or ensure ./mvnw is present."

Comments

Conversation

MehulBatra commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Brief change log

Tests

API and Format

Documentation

Uh oh!

wuchong commented Sep 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MehulBatra commented Sep 22, 2025

Uh oh!

MehulBatra commented Sep 28, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 28, 2025

Choose a reason for hiding this comment

Uh oh!

MehulBatra commented Sep 28, 2025

Uh oh!

wuchong Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

wuchong Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

wuchong Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

wuchong Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

wuchong commented Oct 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MehulBatra commented Sep 18, 2025 •

edited

Loading

wuchong commented Sep 21, 2025 •

edited

Loading