From 52a83bcf8c3186c527dad2cf15876b46ce06fd5d Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Tue, 16 Dec 2025 01:51:06 -0500 Subject: [PATCH 01/13] docs: add design document for JWT authentication implementation --- .../design/20251215_jwt_authentication.md | 146 ++++++++++++++++++ 1 file changed, 146 insertions(+) create mode 100644 doc/developer/design/20251215_jwt_authentication.md diff --git a/doc/developer/design/20251215_jwt_authentication.md b/doc/developer/design/20251215_jwt_authentication.md new file mode 100644 index 0000000000000..5b454d6ca1c20 --- /dev/null +++ b/doc/developer/design/20251215_jwt_authentication.md @@ -0,0 +1,146 @@ +# JWT Authentication + +- Associated: (Insert list of associated epics, issues, or PRs) + +PRD: https://www.notion.so/materialize/SSO-for-Self-Managed-2a613f48d37b806f9cb2d06914454d15 + + +## Problem + +Our goal is to enable single sign on (SSO) for our self managed product + +## Success Criteria + +- Creating a user & adding roles: An admin gives a user access to Materialize +- The user should be disabled from logging in when a user is de-provisioned. However, the database level role should still exist. +- The end user is able to create a token to connect to materialize via psql / postgres clients +- The end user is able to visit the Materialize console, and sign in with their IdP + +## Configuration + +JWT authentication can be enabled by specifying the following Materialize CRD: + +```yaml +apiVersion: materialize.cloud/v1alpha1 +kind: Materialize +metadata: + ... +spec: + ... + authenticatorKind: Jwt + jwtAuthenticationSettings: + # Must match the OIDC client ID. Required. + audience: 060a4f3d-1cac-46e4-b5a5-6b9c66cd9431 + # The claim that represents the user's name in Materialize. + # Usually either "sub" or "email". Required. + authenticationClaim: email + # The key for the `groups` claim. Optional and defaults to "groups" + groupClaim: groups + # The expected issuer URL, must correspond to the `iss` field of the JWT. + # Implicitly fetch JWKS from + # https://{ domain }/.well-known/openid-configuration and allow override + # if we need to. Required. + issuer: https://dev-123456.okta.com/oauth2/default + # Where Materialize will request tokens from the IdP using the refresh token + # if it exists. Optional. + tokenEndpoint: https://dev-123456.okta.com/oauth2/default/v1/token +``` + +Where in environmentd, it’ll look like so: + +```bash +bin/environmentd -- \ +--listeners-config-path='/listeners/jwt.json' \ +--jwt-authentication-setting="{\"audience\":\"060a4f3d-1cac-46e4-b5a5-6b9c66cd9431\",\"authentication_claim\":\"email\",\"group_claim\":\"groups\",\"issuer\":\"https://dev-123456.okta.com/oauth2/default\",\"token_endpoint\":\"https://dev-123456.okta.com/oauth2/default/v1/token\"}" +``` + +## Testing Frameworks + +- For unit testing, use cargo nextest. Similar to frontegg-mock, create a mock oidc server that exposes the correct endpoints, provides a correct JWKS, and generates a JWT with desired claims and fields. +- For end to end testing, use mzcompose and integrate an Ory hydra server docker image as the IDP. Then we can assert against an instance of Materialize created by orchestratord. + +## Phase 1: Create a JWT authenticator kind + +### Solution proposal: Creating a user & adding roles: An admin gives a user access to Materialize + +By specifying an audience, we ensure an admin must explicitly give a user in the IDP access to Materialize as long as they use a client exclusively for Materialize. Otherwise when a user first logins, we create a role for them if it does not exist. + +### Solution proposal: The user should be disabled from logging in when a user is de-provisioned. However, the database level role should still exist. + +When doing pgwire jwt authentication, we can accept a cleartext password of the form `access=&refresh=` where `&` is a delimiter and `refresh=` is optional. The JWT authenticator will then try to authenticate again and fetch a new access token using the refresh token when close to expiration (using the token API URL in the spec above). If the refresh token doesn’t exist, the session will invalidate. The implementation will be very similar to how we refresh tokens for the Frontegg authenticator. This would require users to have their IDP client generate `refresh` tokens. + +By suggesting a short time to live for access tokens, this accomplishes invalidating sessions on deprovisioning of a user. When admins deprovision a user, the next time the user tries to authenticate or refresh their access token, the token API will not allow the user to login but will keep the role in the database. + +**Alternative: Use SASL Authentication using the OAUTHBEARER mechanism rather than a cleartext password** + +This would be the most Postgres compatible way of doing this and is what it uses for its `oauth` authentication method. However, it may run into compatibility issues with clients. For example in `psql`, there’s no obvious way of sending the bearer token directly without going through libpq's device-grant flow. Furthermore, assuming access tokens are short lived, this could lead to poor UX given there’s no native way to re-authenticate a pgwire session. Finally, our HTTP endpoints wouldn’t be able to support this given they don’t support SASL auth. + +OAUTHBEARER reference: [https://www.postgresql.org/docs/18/sasl-authentication.html#SASL-OAUTHBEARER](https://www.postgresql.org/docs/18/sasl-authentication.html#SASL-OAUTHBEARER) + +### Solution proposal: The end user is able to create a token to connect to materialize via psql / postgres clients + +Unfortunately, to provide a nice flow to generate the necessary access token and refresh token, we’d need to control the client. Thus we’ll leave the retrieval of the access token/refresh token to the user, similar to CockroachDB. + +**Alternative: Revive the mz CLI** + +We have an `mz` CLI that’s catered to Cloud and no longer supported. We can potentially bring this back. + +**Open question:** Is there anything we can do on our side to easily provide access tokens / refresh tokens to the user without controlling the client? This feels like the missing piece between JWT authentication and something like `aws sso login` in the AWS CLI + +### Solution proposal: The end user is able to visit the Materialize console, and sign in with their IdP + +A generic Frontend SSO redirect flow would need to be implemented to retrieve an access token and refresh token. However once retrieved, the SQL HTTP / WS API endpoints can use bearer authorization like Cloud and accept the access token. The Console would be in charge of refreshing the access token. The Console work is out of scope for this design document. + +### Work items: + +- Create a JWT Authenticator + - Create environmentd CLI arguments to accept the JWT authentication configuration above + - Wire up the HTTP endpoints, WS endpoints, and pgwire for this authenticator kind +- Add a `jwt` authenticator kind to orchestratord and create a `jwt` listener config + - Still leave a port open for password auth for `mz_system` logins given admins can’t map a user in their IDP to `mz_system` + +An MVP of what this might look like exists here: [https://github.com/MaterializeInc/materialize/pull/34516](https://github.com/MaterializeInc/materialize/pull/34516). Some differences from the proposed design: +- Does not implement the refresh token flow +- Does not validate the psql username against the OIDC user +- Does not do `aud` validation +- Does not use a JSON map for the environmentd CLI arguments + +### Tests: + +- Successful login (e2e mzcompose) +- Invalidating the session on access token expiration and no refresh token (Rust unit test) +- A token should successfully refresh if the access token and refresh token are valid (Rust unit test) +- Session should error if access token is invalid (Rust unit test) +- Session should error if refresh token is invalid (Rust unit test) +- De-provisioning a user should invalidate the refresh token (e2e mzcompose) +- Platform-check simple login check (platform-check framework) + +## Phase 2: Map the `admin` claim to a user’s superuser attribute + +Based on the `admin` claim, we can set the `superuser` attribute we store in the catalog for password authentication. We do this by doing the following: + +- First, in our authenticator, save `admin` inside the user’s `external_metadata` +- Next, in `handle_startup_inner()` we diff them with the user’s current superuser status and if there’s a difference, apply the changes with an `ALTER` operation. We can use `catalog_transact_with_context()` for this. +- On error (e.g. if the ALTER isn’t allowed), we’ll end the connection with a descriptive error message. This is a similar pattern we use for initializing network policies. + +This is similar to how we identify superusers in Frontegg auth, except we also treat it as an operation to update the catalog + - We can keep using the session’ metadata as the source of truth to keep parity with Cloud, but eventually we’ll want to use the catalog as the source of truth for all. We can call this **out of scope.** + +Prototype: [https://github.com/MaterializeInc/materialize/pull/34372/commits](https://github.com/MaterializeInc/materialize/pull/34372/commits) + +### Tests + +- Authenticating with a varying `admin` claim should reflect when querying the superuser status of adapter +- Superuser status should reflect in `mz_roles` (Rust unit test) + + + From 3c30af8336687574756cacea745a10d7fc560708 Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Thu, 18 Dec 2025 11:57:11 -0500 Subject: [PATCH 02/13] Support passing a JWK directly --- .../design/20251215_jwt_authentication.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/doc/developer/design/20251215_jwt_authentication.md b/doc/developer/design/20251215_jwt_authentication.md index 5b454d6ca1c20..e07390780f372 100644 --- a/doc/developer/design/20251215_jwt_authentication.md +++ b/doc/developer/design/20251215_jwt_authentication.md @@ -37,10 +37,17 @@ spec: # The key for the `groups` claim. Optional and defaults to "groups" groupClaim: groups # The expected issuer URL, must correspond to the `iss` field of the JWT. - # Implicitly fetch JWKS from - # https://{ domain }/.well-known/openid-configuration and allow override - # if we need to. Required. + # Required. issuer: https://dev-123456.okta.com/oauth2/default + # The JWKS (JSON Web Key Set) used to validate JWT signatures. Optional. + # If not provided, jwksFetchFromIssuer must be true. + # Format: {"keys": [{"kty": "RSA", "kid": "...", "n": "...", "e": "..."}]} + jwks: '{"keys": [...]}' + # If true, fetches JWKS from https://{issuer}/.well-known/openid-configuration. + # Overrides jwks if both are specified. Defaults to false. + # Note: Not all JWT providers support .well-known/openid-configuration, + # so use jwks directly if the provider doesn't support it. + jwksFetchFromIssuer: true # Where Materialize will request tokens from the IdP using the refresh token # if it exists. Optional. tokenEndpoint: https://dev-123456.okta.com/oauth2/default/v1/token @@ -51,7 +58,7 @@ Where in environmentd, it’ll look like so: ```bash bin/environmentd -- \ --listeners-config-path='/listeners/jwt.json' \ ---jwt-authentication-setting="{\"audience\":\"060a4f3d-1cac-46e4-b5a5-6b9c66cd9431\",\"authentication_claim\":\"email\",\"group_claim\":\"groups\",\"issuer\":\"https://dev-123456.okta.com/oauth2/default\",\"token_endpoint\":\"https://dev-123456.okta.com/oauth2/default/v1/token\"}" +--jwt-authentication-setting="{\"audience\":\"060a4f3d-1cac-46e4-b5a5-6b9c66cd9431\",\"authentication_claim\":\"email\",\"group_claim\":\"groups\",\"issuer\":\"https://dev-123456.okta.com/oauth2/default\",\"jwks\":{\"keys\":[...]},\"jwks_fetch_from_issuer\":true,\"token_endpoint\":\"https://dev-123456.okta.com/oauth2/default/v1/token\"}" ``` ## Testing Frameworks From af72b50b3adfc2d204349af0a1c3ce35e368ef9b Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Thu, 18 Dec 2025 11:59:54 -0500 Subject: [PATCH 03/13] Add more documentation on refresh token mechanism --- doc/developer/design/20251215_jwt_authentication.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/doc/developer/design/20251215_jwt_authentication.md b/doc/developer/design/20251215_jwt_authentication.md index e07390780f372..a9adcf9dc61aa 100644 --- a/doc/developer/design/20251215_jwt_authentication.md +++ b/doc/developer/design/20251215_jwt_authentication.md @@ -48,8 +48,11 @@ spec: # Note: Not all JWT providers support .well-known/openid-configuration, # so use jwks directly if the provider doesn't support it. jwksFetchFromIssuer: true - # Where Materialize will request tokens from the IdP using the refresh token - # if it exists. Optional. + # The OAuth 2.0 token endpoint where Materialize will request new access + # tokens using a refresh token (https://www.rfc-editor.org/rfc/rfc6749.html#section-6). + # Requires `grant_type` of `refresh_token` in the client. + # Optional. If not provided, sessions will expire when the access token expires + # and refresh tokens in the password field will be ignored. tokenEndpoint: https://dev-123456.okta.com/oauth2/default/v1/token ``` From d17a9db45d7fadb46646cfc9f871fb44798f01a2 Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Thu, 18 Dec 2025 12:01:53 -0500 Subject: [PATCH 04/13] Change naming from jwt to oidc --- ...ion.md => 20251215_oidc_authentication.md} | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) rename doc/developer/design/{20251215_jwt_authentication.md => 20251215_oidc_authentication.md} (84%) diff --git a/doc/developer/design/20251215_jwt_authentication.md b/doc/developer/design/20251215_oidc_authentication.md similarity index 84% rename from doc/developer/design/20251215_jwt_authentication.md rename to doc/developer/design/20251215_oidc_authentication.md index a9adcf9dc61aa..c7b30fa4c355d 100644 --- a/doc/developer/design/20251215_jwt_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -1,4 +1,4 @@ -# JWT Authentication +# OIDC Authentication - Associated: (Insert list of associated epics, issues, or PRs) @@ -18,7 +18,7 @@ Our goal is to enable single sign on (SSO) for our self managed product ## Configuration -JWT authentication can be enabled by specifying the following Materialize CRD: +OIDC authentication can be enabled by specifying the following Materialize CRD: ```yaml apiVersion: materialize.cloud/v1alpha1 @@ -27,8 +27,8 @@ metadata: ... spec: ... - authenticatorKind: Jwt - jwtAuthenticationSettings: + authenticatorKind: Oidc + oidcAuthenticationSettings: # Must match the OIDC client ID. Required. audience: 060a4f3d-1cac-46e4-b5a5-6b9c66cd9431 # The claim that represents the user's name in Materialize. @@ -60,8 +60,8 @@ Where in environmentd, it’ll look like so: ```bash bin/environmentd -- \ ---listeners-config-path='/listeners/jwt.json' \ ---jwt-authentication-setting="{\"audience\":\"060a4f3d-1cac-46e4-b5a5-6b9c66cd9431\",\"authentication_claim\":\"email\",\"group_claim\":\"groups\",\"issuer\":\"https://dev-123456.okta.com/oauth2/default\",\"jwks\":{\"keys\":[...]},\"jwks_fetch_from_issuer\":true,\"token_endpoint\":\"https://dev-123456.okta.com/oauth2/default/v1/token\"}" +--listeners-config-path='/listeners/oidc.json' \ +--oidc-authentication-setting="{\"audience\":\"060a4f3d-1cac-46e4-b5a5-6b9c66cd9431\",\"authentication_claim\":\"email\",\"group_claim\":\"groups\",\"issuer\":\"https://dev-123456.okta.com/oauth2/default\",\"jwks\":{\"keys\":[...]},\"jwks_fetch_from_issuer\":true,\"token_endpoint\":\"https://dev-123456.okta.com/oauth2/default/v1/token\"}" ``` ## Testing Frameworks @@ -69,7 +69,7 @@ bin/environmentd -- \ - For unit testing, use cargo nextest. Similar to frontegg-mock, create a mock oidc server that exposes the correct endpoints, provides a correct JWKS, and generates a JWT with desired claims and fields. - For end to end testing, use mzcompose and integrate an Ory hydra server docker image as the IDP. Then we can assert against an instance of Materialize created by orchestratord. -## Phase 1: Create a JWT authenticator kind +## Phase 1: Create a OIDC authenticator kind ### Solution proposal: Creating a user & adding roles: An admin gives a user access to Materialize @@ -77,7 +77,7 @@ By specifying an audience, we ensure an admin must explicitly give a user in the ### Solution proposal: The user should be disabled from logging in when a user is de-provisioned. However, the database level role should still exist. -When doing pgwire jwt authentication, we can accept a cleartext password of the form `access=&refresh=` where `&` is a delimiter and `refresh=` is optional. The JWT authenticator will then try to authenticate again and fetch a new access token using the refresh token when close to expiration (using the token API URL in the spec above). If the refresh token doesn’t exist, the session will invalidate. The implementation will be very similar to how we refresh tokens for the Frontegg authenticator. This would require users to have their IDP client generate `refresh` tokens. +When doing pgwire Oidc authentication, we can accept a cleartext password of the form `access=&refresh=` where `&` is a delimiter and `refresh=` is optional. The OIDC authenticator will then try to authenticate again and fetch a new access token using the refresh token when close to expiration (using the token API URL in the spec above). If the refresh token doesn’t exist, the session will invalidate. The implementation will be very similar to how we refresh tokens for the Frontegg authenticator. This would require users to have their IDP client generate `refresh` tokens. By suggesting a short time to live for access tokens, this accomplishes invalidating sessions on deprovisioning of a user. When admins deprovision a user, the next time the user tries to authenticate or refresh their access token, the token API will not allow the user to login but will keep the role in the database. @@ -95,7 +95,7 @@ Unfortunately, to provide a nice flow to generate the necessary access token and We have an `mz` CLI that’s catered to Cloud and no longer supported. We can potentially bring this back. -**Open question:** Is there anything we can do on our side to easily provide access tokens / refresh tokens to the user without controlling the client? This feels like the missing piece between JWT authentication and something like `aws sso login` in the AWS CLI +**Open question:** Is there anything we can do on our side to easily provide access tokens / refresh tokens to the user without controlling the client? This feels like the missing piece between OIDC authentication and something like `aws sso login` in the AWS CLI ### Solution proposal: The end user is able to visit the Materialize console, and sign in with their IdP @@ -103,10 +103,10 @@ A generic Frontend SSO redirect flow would need to be implemented to retrieve an ### Work items: -- Create a JWT Authenticator - - Create environmentd CLI arguments to accept the JWT authentication configuration above +- Create a OIDC Authenticator + - Create environmentd CLI arguments to accept the OIDC authentication configuration above - Wire up the HTTP endpoints, WS endpoints, and pgwire for this authenticator kind -- Add a `jwt` authenticator kind to orchestratord and create a `jwt` listener config +- Add a `Oidc` authenticator kind to orchestratord and create a `Oidc` listener config - Still leave a port open for password auth for `mz_system` logins given admins can’t map a user in their IDP to `mz_system` An MVP of what this might look like exists here: [https://github.com/MaterializeInc/materialize/pull/34516](https://github.com/MaterializeInc/materialize/pull/34516). Some differences from the proposed design: From 8d9b7041aa7aa7ca91e1c07478f2c8a68e35a1b5 Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Thu, 18 Dec 2025 12:06:18 -0500 Subject: [PATCH 05/13] Clarify "user and adding roles" solution --- doc/developer/design/20251215_oidc_authentication.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/developer/design/20251215_oidc_authentication.md b/doc/developer/design/20251215_oidc_authentication.md index c7b30fa4c355d..2f242e57903c5 100644 --- a/doc/developer/design/20251215_oidc_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -73,7 +73,7 @@ bin/environmentd -- \ ### Solution proposal: Creating a user & adding roles: An admin gives a user access to Materialize -By specifying an audience, we ensure an admin must explicitly give a user in the IDP access to Materialize as long as they use a client exclusively for Materialize. Otherwise when a user first logins, we create a role for them if it does not exist. +By requiring the `aud` (audience) claim to match the configured client ID, we ensure that JWTs issued for other applications (e.g., Slack, internal tools) cannot be used to authenticate with Materialize. This means an admin must explicitly grant users access to the Materialize-specific OIDC client in their IdP. When a user first logs in with a valid token, we create a role for them if one does not already exist. ### Solution proposal: The user should be disabled from logging in when a user is de-provisioned. However, the database level role should still exist. From 864f61db2f6c6851180d7abf5f2af874d7f35ef3 Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Fri, 19 Dec 2025 16:04:37 -0500 Subject: [PATCH 06/13] Update doc/developer/design/20251215_oidc_authentication.md Co-authored-by: Jason Hernandez --- doc/developer/design/20251215_oidc_authentication.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/developer/design/20251215_oidc_authentication.md b/doc/developer/design/20251215_oidc_authentication.md index 2f242e57903c5..10a7fdbf260d6 100644 --- a/doc/developer/design/20251215_oidc_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -123,7 +123,8 @@ An MVP of what this might look like exists here: [https://github.com/Materialize - Session should error if access token is invalid (Rust unit test) - Session should error if refresh token is invalid (Rust unit test) - De-provisioning a user should invalidate the refresh token (e2e mzcompose) -- Platform-check simple login check (platform-check framework) +- Platform-check simple login check (platform-check framework +- JWTs should only be accepted when a valid JWK is set (we do not want to accept JWTs that are not signed with a real, cryptographically sound key) ## Phase 2: Map the `admin` claim to a user’s superuser attribute From cddf6fea1a251d9676e037d129fa471330fefd9d Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Fri, 19 Dec 2025 17:12:21 -0500 Subject: [PATCH 07/13] Mark Console login as out of scope --- doc/developer/design/20251215_oidc_authentication.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/developer/design/20251215_oidc_authentication.md b/doc/developer/design/20251215_oidc_authentication.md index 10a7fdbf260d6..cafb9724a74dd 100644 --- a/doc/developer/design/20251215_oidc_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -99,7 +99,8 @@ We have an `mz` CLI that’s catered to Cloud and no longer supported. We can po ### Solution proposal: The end user is able to visit the Materialize console, and sign in with their IdP -A generic Frontend SSO redirect flow would need to be implemented to retrieve an access token and refresh token. However once retrieved, the SQL HTTP / WS API endpoints can use bearer authorization like Cloud and accept the access token. The Console would be in charge of refreshing the access token. The Console work is out of scope for this design document. +**Out of scope**: +A generic Frontend SSO redirect flow would need to be implemented to retrieve an access token and refresh token. However once retrieved, the SQL HTTP / WS API endpoints can use bearer authorization like Cloud and accept the access token. The Console would be in charge of refreshing the access token. ### Work items: From 6832304a47f8911a9f13b247b275faae3186766d Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Mon, 5 Jan 2026 13:15:31 -0500 Subject: [PATCH 08/13] Address Justin's feedback - Specify non goals - Make `audience` optional - Remove `groupClaim` given it's not used - Define 'admin' - Describe how we'll do access token expiration - Add more detail on how we might implement SASL+OAUTH if we need to --- .../design/20251215_oidc_authentication.md | 23 ++++++++++++------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/doc/developer/design/20251215_oidc_authentication.md b/doc/developer/design/20251215_oidc_authentication.md index cafb9724a74dd..6c80a1374f6c6 100644 --- a/doc/developer/design/20251215_oidc_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -16,6 +16,11 @@ Our goal is to enable single sign on (SSO) for our self managed product - The end user is able to create a token to connect to materialize via psql / postgres clients - The end user is able to visit the Materialize console, and sign in with their IdP +## Non-goals +- SCIM +- JWK refresh +- Generically mapping JWT roles to MZ roles + ## Configuration OIDC authentication can be enabled by specifying the following Materialize CRD: @@ -29,13 +34,11 @@ spec: ... authenticatorKind: Oidc oidcAuthenticationSettings: - # Must match the OIDC client ID. Required. + # Must match the OIDC client ID. Optional. audience: 060a4f3d-1cac-46e4-b5a5-6b9c66cd9431 # The claim that represents the user's name in Materialize. # Usually either "sub" or "email". Required. authenticationClaim: email - # The key for the `groups` claim. Optional and defaults to "groups" - groupClaim: groups # The expected issuer URL, must correspond to the `iss` field of the JWT. # Required. issuer: https://dev-123456.okta.com/oauth2/default @@ -61,7 +64,7 @@ Where in environmentd, it’ll look like so: ```bash bin/environmentd -- \ --listeners-config-path='/listeners/oidc.json' \ ---oidc-authentication-setting="{\"audience\":\"060a4f3d-1cac-46e4-b5a5-6b9c66cd9431\",\"authentication_claim\":\"email\",\"group_claim\":\"groups\",\"issuer\":\"https://dev-123456.okta.com/oauth2/default\",\"jwks\":{\"keys\":[...]},\"jwks_fetch_from_issuer\":true,\"token_endpoint\":\"https://dev-123456.okta.com/oauth2/default/v1/token\"}" +--oidc-authentication-setting="{\"audience\":\"060a4f3d-1cac-46e4-b5a5-6b9c66cd9431\",\"authentication_claim\":\"email\",\"issuer\":\"https://dev-123456.okta.com/oauth2/default\",\"jwks\":{\"keys\":[...]},\"jwks_fetch_from_issuer\":true,\"token_endpoint\":\"https://dev-123456.okta.com/oauth2/default/v1/token\"}" ``` ## Testing Frameworks @@ -71,13 +74,15 @@ bin/environmentd -- \ ## Phase 1: Create a OIDC authenticator kind -### Solution proposal: Creating a user & adding roles: An admin gives a user access to Materialize +### Solution proposal: Creating a user & adding roles: An admin, someone in charge of ensuring that users can only access applications which they are authorized to, gives a user access to Materialize + +If an admin wants to forbid JWTs issued for other applications (e.g., Slack, internal tools) authenticating with Materialize, they can specify the `aud` (audience) claim to match the Materialize-specific OIDC client ID. We will keep it optional to keep parity with the Frontegg authenticator. -By requiring the `aud` (audience) claim to match the configured client ID, we ensure that JWTs issued for other applications (e.g., Slack, internal tools) cannot be used to authenticate with Materialize. This means an admin must explicitly grant users access to the Materialize-specific OIDC client in their IdP. When a user first logs in with a valid token, we create a role for them if one does not already exist. +When a user first logs in with a valid token, we create a role for them if one does not already exist. ### Solution proposal: The user should be disabled from logging in when a user is de-provisioned. However, the database level role should still exist. -When doing pgwire Oidc authentication, we can accept a cleartext password of the form `access=&refresh=` where `&` is a delimiter and `refresh=` is optional. The OIDC authenticator will then try to authenticate again and fetch a new access token using the refresh token when close to expiration (using the token API URL in the spec above). If the refresh token doesn’t exist, the session will invalidate. The implementation will be very similar to how we refresh tokens for the Frontegg authenticator. This would require users to have their IDP client generate `refresh` tokens. +When doing pgwire Oidc authentication, we can accept a cleartext password of the form `access=&refresh=` where `&` is a delimiter and `refresh=` is optional. The OIDC authenticator will then try to authenticate again and fetch a new access token using the refresh token when close to expiration (using the token API URL in the spec above). If the refresh token doesn’t exist, the session will invalidate. This would require users to have their IDP client generate `refresh` tokens. For token expiration checking, in a task, we'll repeatedly wait for `(expiration - now) * 0.8` and see if it's less than a minute. This is also how we check token expiration in the Frontegg authenticator. We'll also implement a config variable to turn off this mechanism and have it default to true. By suggesting a short time to live for access tokens, this accomplishes invalidating sessions on deprovisioning of a user. When admins deprovision a user, the next time the user tries to authenticate or refresh their access token, the token API will not allow the user to login but will keep the role in the database. @@ -85,6 +90,8 @@ By suggesting a short time to live for access tokens, this accomplishes invalida This would be the most Postgres compatible way of doing this and is what it uses for its `oauth` authentication method. However, it may run into compatibility issues with clients. For example in `psql`, there’s no obvious way of sending the bearer token directly without going through libpq's device-grant flow. Furthermore, assuming access tokens are short lived, this could lead to poor UX given there’s no native way to re-authenticate a pgwire session. Finally, our HTTP endpoints wouldn’t be able to support this given they don’t support SASL auth. +In case we need to support SASL+OAUTH in the same Materialize instance, we can create a new port for it. For now we will call it out of scope. + OAUTHBEARER reference: [https://www.postgresql.org/docs/18/sasl-authentication.html#SASL-OAUTHBEARER](https://www.postgresql.org/docs/18/sasl-authentication.html#SASL-OAUTHBEARER) ### Solution proposal: The end user is able to create a token to connect to materialize via psql / postgres clients @@ -124,7 +131,7 @@ An MVP of what this might look like exists here: [https://github.com/Materialize - Session should error if access token is invalid (Rust unit test) - Session should error if refresh token is invalid (Rust unit test) - De-provisioning a user should invalidate the refresh token (e2e mzcompose) -- Platform-check simple login check (platform-check framework +- Platform-check simple login check (platform-check framework) - JWTs should only be accepted when a valid JWK is set (we do not want to accept JWTs that are not signed with a real, cryptographically sound key) ## Phase 2: Map the `admin` claim to a user’s superuser attribute From 1c177fe4545cc590cee54b9b3f5a0487ed538cc6 Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Mon, 5 Jan 2026 13:20:35 -0500 Subject: [PATCH 09/13] Convert JWT retrieval open question to "out of scope" --- doc/developer/design/20251215_oidc_authentication.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/doc/developer/design/20251215_oidc_authentication.md b/doc/developer/design/20251215_oidc_authentication.md index 6c80a1374f6c6..7bb5aa34db510 100644 --- a/doc/developer/design/20251215_oidc_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -98,16 +98,14 @@ OAUTHBEARER reference: [https://www.postgresql.org/docs/18/sasl-authentication.h Unfortunately, to provide a nice flow to generate the necessary access token and refresh token, we’d need to control the client. Thus we’ll leave the retrieval of the access token/refresh token to the user, similar to CockroachDB. -**Alternative: Revive the mz CLI** - -We have an `mz` CLI that’s catered to Cloud and no longer supported. We can potentially bring this back. - -**Open question:** Is there anything we can do on our side to easily provide access tokens / refresh tokens to the user without controlling the client? This feels like the missing piece between OIDC authentication and something like `aws sso login` in the AWS CLI +**Out of scope**: +- Providing an easy way to gain access tokens / refresh tokens from the user's IDP. +- Controlling the client and reviving the `mz` cli ### Solution proposal: The end user is able to visit the Materialize console, and sign in with their IdP **Out of scope**: -A generic Frontend SSO redirect flow would need to be implemented to retrieve an access token and refresh token. However once retrieved, the SQL HTTP / WS API endpoints can use bearer authorization like Cloud and accept the access token. The Console would be in charge of refreshing the access token. +-A generic Frontend SSO redirect flow would need to be implemented to retrieve an access token and refresh token. However once retrieved, the SQL HTTP / WS API endpoints can use bearer authorization like Cloud and accept the access token. The Console would be in charge of refreshing the access token. ### Work items: From a7c58bf87ea72b8ffad447fad4c46410adffc2bb Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Mon, 5 Jan 2026 17:31:10 -0500 Subject: [PATCH 10/13] =?UTF-8?q?Mark=20"Map=20the=20`admin`=20claim=20to?= =?UTF-8?q?=20a=20user=E2=80=99s=20superuser=20attribute"=20as=20out=20of?= =?UTF-8?q?=20scope?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Update non-goals to include syncing IdP roles and attributes --- .../design/20251215_oidc_authentication.md | 48 +++++++------------ 1 file changed, 17 insertions(+), 31 deletions(-) diff --git a/doc/developer/design/20251215_oidc_authentication.md b/doc/developer/design/20251215_oidc_authentication.md index 7bb5aa34db510..6cc9b73ffdf29 100644 --- a/doc/developer/design/20251215_oidc_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -14,12 +14,12 @@ Our goal is to enable single sign on (SSO) for our self managed product - Creating a user & adding roles: An admin gives a user access to Materialize - The user should be disabled from logging in when a user is de-provisioned. However, the database level role should still exist. - The end user is able to create a token to connect to materialize via psql / postgres clients -- The end user is able to visit the Materialize console, and sign in with their IdP ## Non-goals -- SCIM - JWK refresh -- Generically mapping JWT roles to MZ roles +- The end user is able to visit the Materialize console, and sign in with their Identity Provider (IdP) +- SCIM +- Syncing IdP roles and attributes to Materialize roles and attributes ## Configuration @@ -70,7 +70,7 @@ bin/environmentd -- \ ## Testing Frameworks - For unit testing, use cargo nextest. Similar to frontegg-mock, create a mock oidc server that exposes the correct endpoints, provides a correct JWKS, and generates a JWT with desired claims and fields. -- For end to end testing, use mzcompose and integrate an Ory hydra server docker image as the IDP. Then we can assert against an instance of Materialize created by orchestratord. +- For end to end testing, use mzcompose and integrate an Ory hydra server docker image as the IdP. Then we can assert against an instance of Materialize created by orchestratord. ## Phase 1: Create a OIDC authenticator kind @@ -82,9 +82,9 @@ When a user first logs in with a valid token, we create a role for them if one d ### Solution proposal: The user should be disabled from logging in when a user is de-provisioned. However, the database level role should still exist. -When doing pgwire Oidc authentication, we can accept a cleartext password of the form `access=&refresh=` where `&` is a delimiter and `refresh=` is optional. The OIDC authenticator will then try to authenticate again and fetch a new access token using the refresh token when close to expiration (using the token API URL in the spec above). If the refresh token doesn’t exist, the session will invalidate. This would require users to have their IDP client generate `refresh` tokens. For token expiration checking, in a task, we'll repeatedly wait for `(expiration - now) * 0.8` and see if it's less than a minute. This is also how we check token expiration in the Frontegg authenticator. We'll also implement a config variable to turn off this mechanism and have it default to true. +When doing pgwire Oidc authentication, we can accept a cleartext password of the form `access=&refresh=` where `&` is a delimiter and `refresh=` is optional. The OIDC authenticator will then try to authenticate again and fetch a new access token using the refresh token when close to expiration (using the token API URL in the spec above). If the refresh token doesn’t exist, the session will invalidate. This would require users to have their IdP client generate `refresh` tokens. For token expiration checking, in a task, we'll repeatedly wait for `(expiration - now) * 0.8` and see if it's less than a minute. This is also how we check token expiration in the Frontegg authenticator. We'll also implement a config variable to turn off this mechanism and have it default to true. -By suggesting a short time to live for access tokens, this accomplishes invalidating sessions on deprovisioning of a user. When admins deprovision a user, the next time the user tries to authenticate or refresh their access token, the token API will not allow the user to login but will keep the role in the database. +By suggesting a short time to live for access tokens, this accomplishes invalidating sessions on de-provisioning of a user. When admins de-provision a user, the next time the user tries to authenticate or refresh their access token, the token API will not allow the user to login but will keep the role in the database. **Alternative: Use SASL Authentication using the OAUTHBEARER mechanism rather than a cleartext password** @@ -99,7 +99,7 @@ OAUTHBEARER reference: [https://www.postgresql.org/docs/18/sasl-authentication.h Unfortunately, to provide a nice flow to generate the necessary access token and refresh token, we’d need to control the client. Thus we’ll leave the retrieval of the access token/refresh token to the user, similar to CockroachDB. **Out of scope**: -- Providing an easy way to gain access tokens / refresh tokens from the user's IDP. +- Providing an easy way to gain access tokens / refresh tokens from the user's IdP. - Controlling the client and reviving the `mz` cli ### Solution proposal: The end user is able to visit the Materialize console, and sign in with their IdP @@ -113,7 +113,7 @@ Unfortunately, to provide a nice flow to generate the necessary access token and - Create environmentd CLI arguments to accept the OIDC authentication configuration above - Wire up the HTTP endpoints, WS endpoints, and pgwire for this authenticator kind - Add a `Oidc` authenticator kind to orchestratord and create a `Oidc` listener config - - Still leave a port open for password auth for `mz_system` logins given admins can’t map a user in their IDP to `mz_system` + - Still leave a port open for password auth for `mz_system` logins given admins can’t map a user in their IdP to `mz_system` An MVP of what this might look like exists here: [https://github.com/MaterializeInc/materialize/pull/34516](https://github.com/MaterializeInc/materialize/pull/34516). Some differences from the proposed design: - Does not implement the refresh token flow @@ -132,32 +132,18 @@ An MVP of what this might look like exists here: [https://github.com/Materialize - Platform-check simple login check (platform-check framework) - JWTs should only be accepted when a valid JWK is set (we do not want to accept JWTs that are not signed with a real, cryptographically sound key) -## Phase 2: Map the `admin` claim to a user’s superuser attribute +## Out of scope: Sync roles and attributes from an IdP to Materialize roles and attributes -Based on the `admin` claim, we can set the `superuser` attribute we store in the catalog for password authentication. We do this by doing the following: +This phase is out of scope, but it's still useful to describe how we can implement it. -- First, in our authenticator, save `admin` inside the user’s `external_metadata` -- Next, in `handle_startup_inner()` we diff them with the user’s current superuser status and if there’s a difference, apply the changes with an `ALTER` operation. We can use `catalog_transact_with_context()` for this. +### Potential approach: Syncing roles through the `roles` claim in a JWT +Based on the `roles` claim, we can synchronize the roles the user belongs to in the catalog. We do this by doing the following: + +- First, in our authenticator, save `roles` from the `roles` claim inside the user’s `external_metadata` +- Next, in `handle_startup_inner()` we diff them with the user’s current roles and if there’s a difference, apply the changes with `GRANT` operations. We can use `catalog_transact_with_context()` for this. Furthermore, the roles would need to exist in the system first. - On error (e.g. if the ALTER isn’t allowed), we’ll end the connection with a descriptive error message. This is a similar pattern we use for initializing network policies. -This is similar to how we identify superusers in Frontegg auth, except we also treat it as an operation to update the catalog - - We can keep using the session’ metadata as the source of truth to keep parity with Cloud, but eventually we’ll want to use the catalog as the source of truth for all. We can call this **out of scope.** +For syncing the `SUPERUSER` attribute, we can do something similar to above where we allow the user to configure a keyword that represents the "superuser role". Then in the `roles` claim, we look for that role and sync their superuser status using the same steps above, except we use `ALTER` instead of `GRANT`. This is similar to how we identify superusers in Cloud currently. The difference being in Cloud, we don't update the catalog and use `external_metadata` as the source of truth. + - We can keep using the session’ metadata as the source of truth to keep parity with Cloud, but eventually we’ll want to use the catalog as the source of truth for all. Prototype: [https://github.com/MaterializeInc/materialize/pull/34372/commits](https://github.com/MaterializeInc/materialize/pull/34372/commits) - -### Tests - -- Authenticating with a varying `admin` claim should reflect when querying the superuser status of adapter -- Superuser status should reflect in `mz_roles` (Rust unit test) - - - From bae8b33bcf290dcbede8b10e37ccd456442bbda8 Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Mon, 5 Jan 2026 17:34:40 -0500 Subject: [PATCH 11/13] Add Phase 2: Enable runtime configuration for OIDC Introduce a new section outlining the ability to update OIDC configuration at runtime through system parameter variables, allowing for JWK rotation without requiring a full rollout of the Materialize instance. --- doc/developer/design/20251215_oidc_authentication.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/developer/design/20251215_oidc_authentication.md b/doc/developer/design/20251215_oidc_authentication.md index 6cc9b73ffdf29..dbef0eda7e620 100644 --- a/doc/developer/design/20251215_oidc_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -132,6 +132,10 @@ An MVP of what this might look like exists here: [https://github.com/Materialize - Platform-check simple login check (platform-check framework) - JWTs should only be accepted when a valid JWK is set (we do not want to accept JWTs that are not signed with a real, cryptographically sound key) +## Phase 2: Make OIDC configurable on runtime + +When solely relying on flags to `environmentd` for OIDC configuation, for an admin to update the OIDC configuration (i.e. rotating the JWKs), they'd need to do an entire rollout of the Materialize instance. To prevent this, we can provide an alternative way of updating the configuration on runtime. By creating system parameter variables for each configuration variable, we can enable users to update this config through SQL or through the system parameter configmap. Orchestratord will use the Materialize CRD spec to populate the default system variables for OIDC configuration. + ## Out of scope: Sync roles and attributes from an IdP to Materialize roles and attributes This phase is out of scope, but it's still useful to describe how we can implement it. From a473c371e2900035125bdec7b65c3f917c50add8 Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Tue, 6 Jan 2026 17:19:47 -0500 Subject: [PATCH 12/13] Create another unit test in design doc --- doc/developer/design/20251215_oidc_authentication.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/developer/design/20251215_oidc_authentication.md b/doc/developer/design/20251215_oidc_authentication.md index dbef0eda7e620..518da196e29a6 100644 --- a/doc/developer/design/20251215_oidc_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -128,6 +128,7 @@ An MVP of what this might look like exists here: [https://github.com/Materialize - A token should successfully refresh if the access token and refresh token are valid (Rust unit test) - Session should error if access token is invalid (Rust unit test) - Session should error if refresh token is invalid (Rust unit test) +- A user shouldn't be able to login as another user (Rust unit test) - De-provisioning a user should invalidate the refresh token (e2e mzcompose) - Platform-check simple login check (platform-check framework) - JWTs should only be accepted when a valid JWK is set (we do not want to accept JWTs that are not signed with a real, cryptographically sound key) From 68262256e9dca3cad085dc80e236f97cdbfb11c5 Mon Sep 17 00:00:00 2001 From: Sang Jun Bak Date: Wed, 7 Jan 2026 13:21:23 -0500 Subject: [PATCH 13/13] Specify OIDC design document definitions - Add disclaimer about the audience claim - Reword "de-provisioning" to "removing a user from the upstream IDP - Fix indentation --- doc/developer/design/20251215_oidc_authentication.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/developer/design/20251215_oidc_authentication.md b/doc/developer/design/20251215_oidc_authentication.md index 518da196e29a6..abbef1786965a 100644 --- a/doc/developer/design/20251215_oidc_authentication.md +++ b/doc/developer/design/20251215_oidc_authentication.md @@ -12,7 +12,7 @@ Our goal is to enable single sign on (SSO) for our self managed product ## Success Criteria - Creating a user & adding roles: An admin gives a user access to Materialize -- The user should be disabled from logging in when a user is de-provisioned. However, the database level role should still exist. +- When the user is removed from the upstream IDP they are no longer able to login to Materialize but their role will persist in its current state. - The end user is able to create a token to connect to materialize via psql / postgres clients ## Non-goals @@ -34,7 +34,7 @@ spec: ... authenticatorKind: Oidc oidcAuthenticationSettings: - # Must match the OIDC client ID. Optional. + # Must match the OIDC client ID. Optional, but we should recommend setting this such that users can't authenticate with JWTs intended for other apps. audience: 060a4f3d-1cac-46e4-b5a5-6b9c66cd9431 # The claim that represents the user's name in Materialize. # Usually either "sub" or "email". Required. @@ -80,11 +80,11 @@ If an admin wants to forbid JWTs issued for other applications (e.g., Slack, int When a user first logs in with a valid token, we create a role for them if one does not already exist. -### Solution proposal: The user should be disabled from logging in when a user is de-provisioned. However, the database level role should still exist. +### Solution proposal: The user should be disabled from logging in when a user is removed from the upstream IDP. However, the database level role should still exist. When doing pgwire Oidc authentication, we can accept a cleartext password of the form `access=&refresh=` where `&` is a delimiter and `refresh=` is optional. The OIDC authenticator will then try to authenticate again and fetch a new access token using the refresh token when close to expiration (using the token API URL in the spec above). If the refresh token doesn’t exist, the session will invalidate. This would require users to have their IdP client generate `refresh` tokens. For token expiration checking, in a task, we'll repeatedly wait for `(expiration - now) * 0.8` and see if it's less than a minute. This is also how we check token expiration in the Frontegg authenticator. We'll also implement a config variable to turn off this mechanism and have it default to true. -By suggesting a short time to live for access tokens, this accomplishes invalidating sessions on de-provisioning of a user. When admins de-provision a user, the next time the user tries to authenticate or refresh their access token, the token API will not allow the user to login but will keep the role in the database. +By suggesting a short time to live for access tokens, this accomplishes invalidating sessions on removal of a user from an IDP. When admins remove a user from an IDP, the next time the user tries to authenticate or refresh their access token, the token API will not allow the user to login but will keep the role in the database. **Alternative: Use SASL Authentication using the OAUTHBEARER mechanism rather than a cleartext password** @@ -129,7 +129,7 @@ An MVP of what this might look like exists here: [https://github.com/Materialize - Session should error if access token is invalid (Rust unit test) - Session should error if refresh token is invalid (Rust unit test) - A user shouldn't be able to login as another user (Rust unit test) -- De-provisioning a user should invalidate the refresh token (e2e mzcompose) +- Removing a user from the upstream IDP should invalidate the refresh token (e2e mzcompose) - Platform-check simple login check (platform-check framework) - JWTs should only be accepted when a valid JWK is set (we do not want to accept JWTs that are not signed with a real, cryptographically sound key)