Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 69 additions & 72 deletions documentation/ingestion/clients/java.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
---
title: Java Client Documentation
description: "Dive into QuestDB using the Java ingestion client for high-performance,
insert-only operations. Unlock peak time series data ingestion and analysis
efficiency."
description: "Reference for the questdb-client Maven artifact — the Java ILP ingestion client for QuestDB, covering setup, configuration, authentication, and error handling."
---

import Tabs from "@theme/Tabs"
Expand All @@ -11,8 +9,6 @@ import TabItem from "@theme/TabItem"

import CodeBlock from "@theme/CodeBlock"

import InterpolateReleaseData from "../../../src/components/InterpolateReleaseData"

import { RemoteRepoExample } from "@theme/RemoteRepoExample"

:::note
Expand All @@ -25,7 +21,8 @@ For embedded QuestDB, please check our

:::

The QuestDB Java client is baked right into the QuestDB binary.
The QuestDB Java client is distributed as a separate Maven artifact
(`org.questdb:questdb-client`).

The client provides the following benefits:

Expand All @@ -46,43 +43,33 @@ for **writing** data to QuestDB. For retrieving data, we recommend using a

:::

## Compatible JDKs

The client relies on some JDK internal libraries, which certain specialised JDK
offerings may not support.

Here is a list of known incompatible JDKs:

- Azul Zing 17
- A fix is in progress. You can use Azul Zulu 17 in the meantime.

## Quick start

Add QuestDB as a dependency in your project's build configuration file.
Add the QuestDB Java client as a dependency in your project's build configuration file.

<Tabs defaultValue="maven" values={[ { label: "Maven", value: "maven" },
{ label: "Gradle", value: "gradle" }, ]}

> <TabItem value="maven">

<InterpolateReleaseData
renderText={(release) => (
<CodeBlock className="language-xml">
{`<dependency>
<InterpolateJavaClientVersion renderText={(release) => (
<CodeBlock className="language-xml">
{`<dependency>
<groupId>org.questdb</groupId>
<artifactId>questdb</artifactId>
<artifactId>questdb-client</artifactId>
<version>${release.name}</version>
</dependency>`}
</CodeBlock>
)}
/>
</CodeBlock>
)} />
</TabItem>
<TabItem value="gradle">
<InterpolateReleaseData
renderText={(release) => (
<CodeBlock className="language-text">
{`compile group: 'org.questdb', name: 'questdb', version: '${release.name}'`}
</CodeBlock> )} /> </TabItem> </Tabs>
<InterpolateJavaClientVersion renderText={(release) => (
<CodeBlock className="language-text">
{`implementation 'org.questdb:questdb-client:${release.name}'`}
</CodeBlock>
)} />
</TabItem>
</Tabs>

The code below creates a client instance configured to use HTTP transport to
connect to a QuestDB server running on localhost, port 9000. It then sends two
Expand All @@ -92,27 +79,10 @@ time.

<RemoteRepoExample name="ilp-http" lang="java" header={false} />

Configure the client using a configuration string. It follows this general
format:

```text
<protocol>::<key>=<value>;<key>=<value>;...;
```

[Transport protocol](/docs/ingestion/ilp/overview/#transport-selection)
can be one of these:

- `http` — ILP/HTTP
- `https` — ILP/HTTP with TLS encryption
- `tcp` — ILP/TCP
- `tcps` — ILP/TCP with TLS encryption

The key `addr` sets the hostname and port of the QuestDB server. Port defaults
to 9000 for HTTP(S) and 9009 for TCP(S).

The minimum configuration includes the transport and the address. For a complete
list of options, refer to the [Configuration Options](#configuration-options)
section.
The client is configured using a configuration string. See
[Ways to create the client](#ways-to-create-the-client) for all configuration
methods, and [Configuration options](#configuration-options) for available
settings.

## Authenticate and encrypt

Expand All @@ -132,20 +102,38 @@ There are three ways to create a client instance:

1. **From a configuration string.** This is the most common way to create a
client instance. It describes the entire client configuration in a single
string. See [Configuration options](#configuration-options) for all available
options. It allows sharing the same configuration across clients in different
languages.
string, and allows sharing the same configuration across clients in different
languages. The general format is:

```text
<protocol>::<key>=<value>;<key>=<value>;...;
```

[Transport protocol](/docs/ingestion/ilp/overview/#transport-selection)
can be one of these:

- `http` — ILP/HTTP
- `https` — ILP/HTTP with TLS encryption
- `tcp` — ILP/TCP
- `tcps` — ILP/TCP with TLS encryption

The key `addr` sets the hostname and port of the QuestDB server. Port
defaults to 9000 for HTTP(S) and 9009 for TCP(S). The minimum configuration
includes the transport and the address.

```java
try (Sender sender = Sender.fromConfig("http::addr=localhost:9000;auto_flush_rows=5000;retry_timeout=10000;")) {
// ...
}
```

For all available options, see
[Configuration options](#configuration-options).

2. **From an environment variable.** The `QDB_CLIENT_CONF` environment variable
is used to set the configuration string. Moving configuration parameters to
an environment variable allows you to avoid hard-coding sensitive information
such as tokens and password in your code.
such as tokens and passwords in your code.

```bash
export QDB_CLIENT_CONF="http::addr=localhost:9000;auto_flush_rows=5000;retry_timeout=10000;"
Expand All @@ -169,16 +157,16 @@ There are three ways to create a client instance:
}
```

## Configuring multiple urls
## Configuring multiple URLs

:::note

This feature requires QuestDB OSS 9.1.0+ or Enterprise 3.0.4+.

:::

The ILP client can be configured with multiple _possible_ endpoints to send your data to. Only one will be sent to at
any one time.
The ILP client can be configured with multiple _possible_ endpoints to send your data to. Only one endpoint is used at
a time.

To configure this feature, simply provide multiple `addr` entries. For example:

Expand All @@ -193,10 +181,10 @@ On initialisation, if `protocol_version=auto`, the sender will identify the firs
any subsequent data to it.

In the event that the instance becomes unavailable for writes, the client will retry the other possible endpoints, and when it finds
a new writeable instance, will _stick_ to it instead. This unvailability is characterised by failures to connect or locate the instance,
a new writeable instance, will _stick_ to it instead. This unavailability is characterised by failures to connect or locate the instance,
or the instance returning an error code due to it being read-only.

By configuring multiple addresses, you can continue allowing you to continue to capture data if your primary instance
By configuring multiple addresses, you can continue to capture data if your primary instance
fails, without having to reconfigure the clients. This backup instance can be hot or cold, and so long as it is assigned a known address, it will be written to as soon as it is started.

Enterprise users can leverage this feature to transparently handle replication failover, without the need to introduce a load-balancer or
Expand All @@ -217,7 +205,7 @@ to `30s` or higher.
1. Create a client instance via `Sender.fromConfig()`.
2. Use `table(CharSequence)` to select a table for inserting a new row.
3. Use `symbol(CharSequence, CharSequence)` to add all symbols. You must add
symbols before adding other column type.
symbols before adding other column types.
4. Use the following options to add all the remaining columns:

- `stringColumn(CharSequence, CharSequence)`
Expand All @@ -243,7 +231,7 @@ precision and scale.
set a designated timestamp.
6. Optionally: You can use `flush()` to send locally buffered data into a
server.
7. Go to the step no. 2 to start a new row.
7. Repeat from step 2 to start a new row.
8. Use `close()` to dispose the Sender after you no longer need it.

## Ingest arrays
Expand Down Expand Up @@ -293,15 +281,15 @@ You can configure the client to not use automatic flushing, and issue explicit
flush requests by calling `sender.flush()`:

```java
try (Sender sender = Sender.fromConfig("http::addr=localhost:9000;auto_flush=off")) {
try (Sender sender = Sender.fromConfig("http::addr=localhost:9000;auto_flush=off")) {
sender.table("trades")
.symbol("symbol", "ETH-USD")
.symbol("side", "sell")
.doubleColumn("price", 2615.54)
.doubleColumn("amount", 0.00044)
.atNow();
sender.table("trades")
.symbol("symbol", "TC-USD")
.symbol("symbol", "BTC-USD")
.symbol("side", "sell")
.doubleColumn("price", 39269.98)
.doubleColumn("amount", 0.001)
Expand Down Expand Up @@ -343,17 +331,16 @@ closing the client.

## Error handling

HTTP automatically retries failed, recoverable requests: network errors, some
server errors, and timeouts. Non-recoverable errors include invalid data,
authentication errors, and other client-side errors.

:::note

If you have configured multiple addresses, retries will be run against different instances.

:::

HTTP automatically retries failed, recoverable requests: network errors, some
server errors, and timeouts. Non-recoverable errors include invalid data,
authentication errors, and other client-side errors.

Retrying is especially useful during transient network issues or when the server
goes offline for a short period. Configure the retrying behavior through the
`retry_timeout` configuration option or via the builder API with
Expand All @@ -364,9 +351,9 @@ it hits the timeout without success, the client throws a `LineSenderException`.
The client won't retry requests while it's being closed and attempting to flush
the data left over in the buffer.

The TCP transport has no mechanism to notify the client it encountered an
error; instead it just disconnects. When the client detects this, it throws a
`LineSenderException` and becomes unusable.
The TCP transport has no mechanism to notify the client it encountered an
error; instead it just disconnects. When the client detects this, it throws a
`LineSenderException` and becomes unusable.

## Recover after a client-side error

Expand All @@ -383,7 +370,7 @@ rows were accepted by the server.

Error handling behaviour changed with the release of QuestDB 9.1.0.

Previously, failing all retries would cause the code to except and release the buffered data.
Previously, failing all retries would cause an exception and release the buffered data.

Now the buffer will not be released. If you wish to re-use the same sender with fresh data, you must call the
new `reset()` function.
Expand Down Expand Up @@ -475,6 +462,16 @@ method.
For a breakdown of available options, see the
[Configuration string](/docs/ingestion/clients/configuration-string/) page.

## Compatible JDKs

The client relies on some JDK internal libraries, which certain specialised JDK
offerings may not support.

Here is a list of known incompatible JDKs:

- Azul Zing 17
- A fix is in progress. You can use Azul Zulu 17 in the meantime.

## Other considerations

- Refer to the [ILP overview](/docs/ingestion/ilp/overview) for details
Expand Down
1 change: 1 addition & 0 deletions docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ const config = {
},
}),
require.resolve("./plugins/fetch-latest-release/index"),
require.resolve("./plugins/fetch-java-client-release/index"),
require.resolve("./plugins/fetch-repo/index"),
require.resolve("./plugins/remote-repo-example/index"),
require.resolve("./plugins/raw-markdown/index"),
Expand Down
69 changes: 69 additions & 0 deletions plugins/fetch-java-client-release/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
import type { Plugin } from '@docusaurus/types'
import nodeFetch from 'node-fetch'

type Release = {
name: string
}

const DEFAULT_RELEASE: Release = {
name: '1.0.1',
}

async function fetchLatestJavaClientRelease(): Promise<Release> {
const url =
'https://api.github.com/repos/questdb/java-questdb-client/releases/latest'

if (typeof fetch === 'undefined') {
try {
const response = await nodeFetch(url, {
headers: {
'Content-Type': 'application/json',
},
})

if (!response.ok) {
console.error(`GitHub API error: ${response.status}`)
return DEFAULT_RELEASE
}

const data = await response.json()
const tagName = (data as { tag_name?: string }).tag_name ?? ''
return { name: tagName.replace(/^v/, '') || DEFAULT_RELEASE.name }
} catch (error) {
console.error('Failed to fetch latest Java client release:', error)
return DEFAULT_RELEASE
}
}

// Browser environment with native fetch
try {
const response = await fetch(url, {
next: { revalidate: 3600 },
})

if (!response.ok) {
console.error(`GitHub API error: ${response.status}`)
return DEFAULT_RELEASE
}

const data = await response.json()
const tagName = (data as { tag_name?: string }).tag_name ?? ''
return { name: tagName.replace(/^v/, '') || DEFAULT_RELEASE.name }
} catch (error) {
console.error('Failed to fetch latest Java client release:', error)
return DEFAULT_RELEASE
}
}

export default function plugin(): Plugin {
return {
name: 'fetch-java-client-release',
async loadContent() {
return fetchLatestJavaClientRelease()
},
async contentLoaded({ content, actions }) {
const { setGlobalData } = actions
setGlobalData({ release: content })
},
}
}
Loading