diff --git a/content/includes/k8s/delete-license-registry-secrets.md b/content/includes/k8s/delete-license-registry-secrets.md new file mode 100644 index 000000000..8e306e8a7 --- /dev/null +++ b/content/includes/k8s/delete-license-registry-secrets.md @@ -0,0 +1,29 @@ +--- +nd-product: MISCEL +--- + +If your deployment used NGINX Plus, you should also remove the secrets created for your license and the F5 registry. + +```shell +kubectl delete secret nplus-license +``` + +{{< details summary="Example output" >}} + +```text +secret "nplus-license" deleted +``` + +{{< /details >}} + +```shell +kubectl delete secret regcred +``` + +{{< details summary="Example output" >}} + +```text +secret "regcred" deleted +``` + +{{< /details >}} \ No newline at end of file diff --git a/content/includes/ngf/gateway-api-compat-table.md b/content/includes/ngf/gateway-api-compat-table.md new file mode 100644 index 000000000..1b96a8e97 --- /dev/null +++ b/content/includes/ngf/gateway-api-compat-table.md @@ -0,0 +1,25 @@ +--- +--- + +{{< call-out "important" >}} + +You can view the [Milestone Roadmap](https://github.com/orgs/nginx/projects/10/views/5) in the NGINX Gateway Fabric GitHub project to see what API resources will be supported in upcoming releases. + +{{< /call-out >}} + +{{< table >}} + +| Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | +|---------------------------------------|---------------------|------------------------|---------------------------------------|-------------|---------------------| +| [GatewayClass]({{< ref "/ngf/overview/gateway-api-compatibility.md#gatewayclass" >}}) | Supported | Not supported | Supported | v1 | Standard | +| [Gateway]({{< ref "/ngf/overview/gateway-api-compatibility.md#gateway" >}}) | Supported | Partially supported | Not supported | v1 | Standard | +| [HTTPRoute]({{< ref "/ngf/overview/gateway-api-compatibility.md#httproute" >}}) | Supported | Partially supported | Not supported | v1 | Standard | +| [GRPCRoute]({{< ref "/ngf/overview/gateway-api-compatibility.md#grpcroute" >}}) | Supported | Partially supported | Not supported | v1 | Standard | +| [ReferenceGrant]({{< ref "/ngf/overview/gateway-api-compatibility.md#referencegrant" >}}) | Supported | N/A | Not supported | v1beta1 | Standard | +| [TLSRoute]({{< ref "/ngf/overview/gateway-api-compatibility.md#tlsroute" >}}) | Supported | Not supported | Not supported | v1alpha2 | Experimental | +| [TCPRoute]({{< ref "/ngf/overview/gateway-api-compatibility.md#tcproute" >}}) | Not supported | Not supported | Not supported | v1alpha2 | Experimental | +| [UDPRoute]({{< ref "/ngf/overview/gateway-api-compatibility.md#udproute" >}}) | Not supported | Not supported | Not supported | v1alpha2 | Experimental | +| [BackendTLSPolicy]({{< ref "/ngf/overview/gateway-api-compatibility.md#backendtlspolicy" >}}) | Partially Supported | Supported | Partially supported | v1alpha3 | Experimental | +| [Custom policies]({{< ref "/ngf/overview/gateway-api-compatibility.md#custom-policies" >}}) | N/A | N/A | Supported | N/A | N/A | + +{{< /table >}} \ No newline at end of file diff --git a/content/includes/ngf/installation/expose-nginx-gateway-fabric.md b/content/includes/ngf/installation/expose-nginx-gateway-fabric.md index c072b0b35..9161fda86 100644 --- a/content/includes/ngf/installation/expose-nginx-gateway-fabric.md +++ b/content/includes/ngf/installation/expose-nginx-gateway-fabric.md @@ -7,4 +7,6 @@ nd-files: - content/ngf/install/manifests.md --- -The Service that is provisioned when NGINX Gateway Fabric is first installed is a ClusterIP Service used only for internal communication between the control plane and data planes. To deploy NGINX itself and get a LoadBalancer Service, you now need to [create a Gateway]({{< ref "/ngf/install/deploy-data-plane.md" >}}). +When NGINX Gateway Fabric is installed, it provisions a ClusterIP Service used only for internal communication between the control plane and data planes. + +To deploy NGINX itself and get a LoadBalancer Service, you should follow the [Deploy a Gateway for data plane instances]({{< ref "/ngf/install/deploy-data-plane.md" >}}) instructions. diff --git a/content/includes/ngf/installation/manifests/api-resources.md b/content/includes/ngf/installation/manifests/api-resources.md new file mode 100644 index 000000000..dc1e5e998 --- /dev/null +++ b/content/includes/ngf/installation/manifests/api-resources.md @@ -0,0 +1,22 @@ +--- +--- + +{{< call-out "note" >}} If you have already installed Gateway API resources in your cluster, ensure they are a version [supported by NGINX Gateway Fabric]({{< ref "/ngf/overview/technical-specifications.md" >}}) {{< /call-out >}} + +To install the Gateway API resources, use `kubectl kustomize`: + +```shell +kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v{{< version-ngf >}}" | kubectl apply -f - +``` + +{{< details summary="Example output" >}} + +```text +customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created +customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created +customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created +customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created +customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created +``` + +{{< /details >}} \ No newline at end of file diff --git a/content/includes/ngf/installation/manifests/crds.md b/content/includes/ngf/installation/manifests/crds.md new file mode 100644 index 000000000..9adbba015 --- /dev/null +++ b/content/includes/ngf/installation/manifests/crds.md @@ -0,0 +1,21 @@ +--- +--- + +Deploy the NGINX Gateway Fabric CRDs using `kubectl apply`: + +```shell +kubectl apply --server-side -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/crds.yaml +``` + +{{< details summary="Example output" >}} + +```text +customresourcedefinition.apiextensions.k8s.io/clientsettingspolicies.gateway.nginx.org serverside-applied +customresourcedefinition.apiextensions.k8s.io/nginxgateways.gateway.nginx.org serverside-applied +customresourcedefinition.apiextensions.k8s.io/nginxproxies.gateway.nginx.org serverside-applied +customresourcedefinition.apiextensions.k8s.io/observabilitypolicies.gateway.nginx.org serverside-applied +customresourcedefinition.apiextensions.k8s.io/snippetsfilters.gateway.nginx.org serverside-applied +customresourcedefinition.apiextensions.k8s.io/upstreamsettingspolicies.gateway.nginx.org serverside-applied +``` + +{{< /details >}} \ No newline at end of file diff --git a/content/includes/ngf/installation/manifests/secure-certificates.md b/content/includes/ngf/installation/manifests/secure-certificates.md new file mode 100644 index 000000000..e7b73a797 --- /dev/null +++ b/content/includes/ngf/installation/manifests/secure-certificates.md @@ -0,0 +1,127 @@ +--- +--- + +{{< call-out "note" >}} These steps use a self-signed issuer, which should not be used in production environments. For production environments, you should use a real [CA issuer](https://cert-manager.io/docs/configuration/ca/). {{< /call-out >}} + +First, create a CA (certificate authority) issuer: + +```yaml +kubectl apply -f - <}} + +```text +issuer.cert-manager.io/selfsigned-issuer created +Warning: spec.privateKey.rotationPolicy: In cert-manager >= v1.18.0, the default value changed from `Never` to `Always`. +certificate.cert-manager.io/nginx-gateway-ca created +issuer.cert-manager.io/nginx-gateway-issuer created +``` + +{{< /details >}} + +You will then need to create a server certificate for the NGINX Gateway Fabric control plane (server): + +{{< call-out "note" >}} + +The default service name is _nginx-gateway_, and the namespace is _nginx-gateway_, so the `dnsNames` value should be `nginx-gateway.nginx-gateway.svc`. + +This value becomes the name of the NGINX Gateway Fabric control plane service. + +{{< /call-out >}} + +```yaml {hl_lines=[13]} +kubectl apply -f - <}} + +```text +agent-tls kubernetes.io/tls 3 3s +nginx-gateway-ca kubernetes.io/tls 3 15s +server-tls kubernetes.io/tls 3 8s +``` + +{{< /details >}} \ No newline at end of file diff --git a/content/includes/ngf/installation/manifests/verify-deployment.md b/content/includes/ngf/installation/manifests/verify-deployment.md new file mode 100644 index 000000000..5dfabbdf3 --- /dev/null +++ b/content/includes/ngf/installation/manifests/verify-deployment.md @@ -0,0 +1,15 @@ +--- +--- + +To confirm that NGINX Gateway Fabric is running, check the pods in the `nginx-gateway` namespace: + +```shell +kubectl get pods -n nginx-gateway +``` + +The output should look similar to this (The pod name will include a unique string): + +```text +NAME READY STATUS RESTARTS AGE +nginx-gateway-694897c587-bbz62 1/1 Running 0 29s +``` \ No newline at end of file diff --git a/content/includes/ngf/installation/next-steps.md b/content/includes/ngf/installation/next-steps.md new file mode 100644 index 000000000..a566d285a --- /dev/null +++ b/content/includes/ngf/installation/next-steps.md @@ -0,0 +1,6 @@ +--- +--- + +- [Deploy a Gateway for data plane instances]({{< ref "/ngf/install/deploy-data-plane.md" >}}) +- [Routing traffic to applications]({{< ref "/ngf/traffic-management/basic-routing.md" >}}) +- [Secure traffic using Let's Encrypt]({{< ref "/ngf/traffic-security/integrate-cert-manager.md" >}}) \ No newline at end of file diff --git a/content/includes/ngf/installation/nginx-plus/docker-registry-secret.md b/content/includes/ngf/installation/nginx-plus/docker-registry-secret.md index d421f30b8..7c98e6684 100644 --- a/content/includes/ngf/installation/nginx-plus/docker-registry-secret.md +++ b/content/includes/ngf/installation/nginx-plus/docker-registry-secret.md @@ -20,4 +20,8 @@ Create a Kubernetes `docker-registry` secret type using the contents of the JWT kubectl create secret docker-registry nginx-plus-registry-secret --docker-server=private-registry.nginx.com --docker-username= --docker-password=none -n nginx-gateway ``` -It is important that the `--docker-username=` contains the contents of the token and is not pointing to the token itself. When you copy the contents of the JWT, ensure there are no additional characters such as extra whitespaces. This can invalidate the token, causing 401 errors when trying to authenticate to the registry. +It is important that the `--docker-username=` contains the contents of the token and is not pointing to the token itself. + +When you copy the contents of the JWT, ensure there are no additional characters such as extra whitespaces. + +This can invalidate the token, causing 401 errors when trying to authenticate to the registry. diff --git a/content/includes/ngf/installation/uninstall-gateway-api-resources.md b/content/includes/ngf/installation/uninstall-gateway-api-resources.md index c16b40fb6..88d84755c 100644 --- a/content/includes/ngf/installation/uninstall-gateway-api-resources.md +++ b/content/includes/ngf/installation/uninstall-gateway-api-resources.md @@ -6,7 +6,9 @@ nd-files: - content/ngf/install/manifests.md --- -{{< call-out "warning" >}} This will remove all corresponding custom resources in your entire cluster, across all namespaces. Double-check to make sure you don't have any custom resources you need to keep, and confirm that there are no other Gateway API implementations active in your cluster. {{< /call-out >}} +{{< call-out "warning" >}} This step will remove all corresponding custom resources in your entire cluster, across every namespace. + +Ensure you don't have any custom resources you need to keep, and confirm that there are no other Gateway API implementations active in your cluster. {{< /call-out >}} To uninstall the Gateway API resources, run the following: @@ -14,7 +16,19 @@ To uninstall the Gateway API resources, run the following: kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v{{< version-ngf >}}" | kubectl delete -f - ``` -Alternatively, if you installed the Gateway APIs from the experimental channel, run the following: +{{< details summary="Example output" >}} + +```text +customresourcedefinition.apiextensions.k8s.io "gatewayclasses.gateway.networking.k8s.io" deleted +customresourcedefinition.apiextensions.k8s.io "gateways.gateway.networking.k8s.io" deleted +customresourcedefinition.apiextensions.k8s.io "grpcroutes.gateway.networking.k8s.io" deleted +customresourcedefinition.apiextensions.k8s.io "httproutes.gateway.networking.k8s.io" deleted +customresourcedefinition.apiextensions.k8s.io "referencegrants.gateway.networking.k8s.io" deleted +``` + +{{< /details >}} + +If you installed the Gateway APIs from the experimental channel, run the following instead: ```shell kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v{{< version-ngf >}}" | kubectl delete -f - diff --git a/content/includes/nic/configuration/global-configuration/configmap-resource.md b/content/includes/nic/configuration/global-configuration/configmap-resource.md new file mode 100644 index 000000000..daf9b6710 --- /dev/null +++ b/content/includes/nic/configuration/global-configuration/configmap-resource.md @@ -0,0 +1,247 @@ +--- +title: ConfigMap resources +weight: 300 +toc: true +type: how-to +product: NIC +nd-docs: DOCS-586 +--- + +When using F5 NGINX Ingress Controller, you can customize or fine tune NGINX behavior using ConfigMap resources. Examples include setting the number of worker processes or customizing the access log format. + +## Using ConfigMap + +1. The [Installation with Manifests]({{< relref "installation/installing-nic/installation-with-manifests.md" >}}) documentation deploy an empty ConfigMap while the default installation manifests specify it in the command-line arguments of the Ingress Controller. However, if you customized the manifests, to use ConfigMap, make sure to specify the ConfigMap resource to use the [command-line arguments]({{< ref "/nic/configuration/global-configuration/command-line-arguments" >}}) of NGINX Ingress Controller. + +1. Create a ConfigMap file with the name *nginx-config.yaml* and set the values +that make sense for your setup: + + ```yaml + kind: ConfigMap + apiVersion: v1 + metadata: + name: nginx-config + namespace: nginx-ingress + data: + proxy-connect-timeout: "10s" + proxy-read-timeout: "10s" + client-max-body-size: "2m" + ``` + + See the section [Summary of ConfigMap Keys](#summary-of-configmap-keys) for the explanation of the available ConfigMap keys (such as `proxy-connect-timeout` in this example). + +1. Create a new (or update the existing) ConfigMap resource: + + ```shell + kubectl apply -f nginx-config.yaml + ``` + + The NGINX configuration will be updated. + +## ConfigMap and Ingress annotations + +ConfigMap applies globally, meaning that it affects every Ingress resource. In contrast, annotations always apply to their Ingress resource. Annotations can override some ConfigMap keys: an example is that the `nginx.org/proxy-connect-timeout` annotations overrides the `proxy-connect-timeout` ConfigMap key. + +For more information, view the [Advanced configuration with annotations]({{< ref "/nic/configuration/ingress-resources/advanced-configuration-with-annotations" >}}) topic. + +## ConfigMap and VirtualServer/VirtualServerRoute resources + +The ConfigMap affects every VirtualServer and VirtualServerRoute resources. However, the fields of those resources allow overriding some ConfigMap keys. For example, the `connect-timeout` field of the `upstream` overrides the `proxy-connect-timeout` ConfigMap key. + +For more information, view the [VirtualServer and VirtualServerRoute resources]({{< ref "/nic/configuration/virtualserver-and-virtualserverroute-resources" >}}) topic. + +## ConfigMap keys + +### Ingress Controller (Unrelated to NGINX Configuration) + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*external-status-address* | Sets the address to be reported in the status of Ingress resources. Requires the *-report-status* command-line argument. Overrides the *-external-service* argument. | N/A | [Reporting resource status]({{< ref "/nic/configuration/global-configuration/reporting-resources-status" >}}) | + +{{< /table >}} + +### General customization + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*proxy-connect-timeout* | Sets the value of the [proxy_connect_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) and [grpc_connect_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_connect_timeout) directive. | *60s* | | +|*proxy-read-timeout* | Sets the value of the [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) and [grpc_read_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_read_timeout) directive. | *60s* | | +|*proxy-send-timeout* | Sets the value of the [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) and [grpc_send_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_send_timeout) directive. | *60s* | | +|*client-max-body-size* | Sets the value of the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. | *1m* | | +|*client-body-buffer-size* | Sets the value of the [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. | N/A | | +|*proxy-buffering* | Enables or disables [buffering of responses](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) from the proxied server. | *True* | | +|*proxy-buffers* | Sets the value of the [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive. | Depends on the platform. | | +|*proxy-buffer-size* | Sets the value of the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) and [grpc_buffer_size](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_buffer_size) directives. | Depends on the platform. | | +|*proxy-busy-buffers-size* | Sets the value of the [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) directive. | Depends on the platform. | | +|*proxy-max-temp-file-size* | Sets the value of the [proxy_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size) directive. | *1024m* | | +|*set-real-ip-from* | Sets the value of the [set_real_ip_from](https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from) directive. | N/A | | +|*real-ip-header* | Sets the value of the [real_ip_header](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header) directive. | *X-Real-IP* | | +|*real-ip-recursive* | Enables or disables the [real_ip_recursive](https://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_recursive) directive. | *False* | | +|*default-server-return* | Configures the [return](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return) directive in the default server, which handles a client request if none of the hosts of Ingress or VirtualServer resources match. The default value configures NGINX to return a 404 error page. You can configure a fixed response or a redirect. For example, *default-server-return: 302 https://nginx.org* will redirect a client to *https://nginx.org*. | *404* | | +|*server-tokens* | Enables or disables the [server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens) directive. Additionally, with the NGINX Plus, you can specify a custom string value, including the empty string value, which disables the emission of the “Server” field. | *True* | | +|*worker-processes* | Sets the value of the [worker_processes](https://nginx.org/en/docs/ngx_core_module.html#worker_processes) directive. | *auto* | | +|*worker-rlimit-nofile* | Sets the value of the [worker_rlimit_nofile](https://nginx.org/en/docs/ngx_core_module.html#worker_rlimit_nofile) directive. | N/A | | +|*worker-connections* | Sets the value of the [worker_connections](https://nginx.org/en/docs/ngx_core_module.html#worker_connections) directive. | *1024* | | +|*worker-cpu-affinity* | Sets the value of the [worker_cpu_affinity](https://nginx.org/en/docs/ngx_core_module.html#worker_cpu_affinity) directive. | N/A | | +|*worker-shutdown-timeout* | Sets the value of the [worker_shutdown_timeout](https://nginx.org/en/docs/ngx_core_module.html#worker_shutdown_timeout) directive. | N/A | | +|*server-names-hash-bucket-size* | Sets the value of the [server_names_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_names_hash_bucket_size) directive. | *256* | | +|*server-names-hash-max-size* | Sets the value of the [server_names_hash_max_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_names_hash_max_size) directive. | *1024* | | +|*map-hash-bucket-size* | Sets the value of the [map_hash_bucket_size](http://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_bucket_size) directive.| *256* | | +|*map-hash-max-size* | Sets the value of the [map_hash_max_size](http://nginx.org/en/docs/http/ngx_http_map_module.html#map_hash_max_size) directive. | *2048* | | +|*resolver-addresses* | Sets the value of the [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) addresses. Note: If you use a DNS name (for example, *kube-dns.kube-system.svc.cluster.local* ) as a resolver address, NGINX Plus will resolve it using the system resolver during the start and on every configuration reload. If the name cannot be resolved or the DNS server doesn't respond, NGINX Plus will fail to start or reload. To avoid this, we recommend using IP addresses as resolver addresses instead of DNS names. Supported in NGINX Plus only. | N/A | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/externalname-services). | +|*resolver-ipv6* | Enables IPv6 resolution in the resolver. Supported in NGINX Plus only. | *True* | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/externalname-services). | +|*resolver-valid* | Sets the time NGINX caches the resolved DNS records. Supported in NGINX Plus only. | TTL value of a DNS record | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/externalname-services). | +|*resolver-timeout* | Sets the [resolver_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver_timeout) for name resolution. Supported in NGINX Plus only. | *30s* | [Support for Type ExternalName Services](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/externalname-services). | +|*keepalive-timeout* | Sets the value of the [keepalive_timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_timeout) directive. | *75s* | | +|*keepalive-requests* | Sets the value of the [keepalive_requests](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests) directive. | *1000* | | +|*variables-hash-bucket-size* | Sets the value of the [variables_hash_bucket_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_bucket_size) directive. | *256* | | +|*variables-hash-max-size* | Sets the value of the [variables-hash-max-size](https://nginx.org/en/docs/http/ngx_http_core_module.html#variables_hash_max_size) directive. | *1024* | | + +{{< /table >}} + +### Logging + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*error-log-level* | Sets the global [error log level](https://nginx.org/en/docs/ngx_core_module.html#error_log) for NGINX. | *notice* | | +|*access-log* | Sets the directive [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log). A syslog destination is the only valid value. The value will be set to its default in-case user tries to configure it with location other than a syslog. +| ``/dev/stdout main`` | ``syslog:server=localhost:514`` | +|*access-log-off* | Disables the [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log). | *False* | | +|*default-server-access-log-off* | Disables the [access log](https://nginx.org/en/docs/http/ngx_http_log_module.html#access_log) for the default server. If access log is disabled globally (*access-log-off: "True"*), then the default server access log is always disabled. | *False* | | +|*log-format* | Sets the custom [log format](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format) for HTTP and HTTPS traffic. For convenience, it is possible to define the log format across multiple lines (each line separated by *\n*). In that case, the Ingress Controller will replace every *\n* character with a space character. All *'* characters must be escaped. | See the [template file](https://github.com/nginx/kubernetes-ingress/blob/v{{< nic-version >}}/internal/configs/version1/nginx.tmpl) for the access log. | [Custom Log Format](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/shared-examples/custom-log-format). | +|*log-format-escaping* | Sets the characters escaping for the variables of the log format. Supported values: *json* (JSON escaping), *default* (the default escaping) *none* (disables escaping). | *default* | | +|*stream-log-format* | Sets the custom [log format](https://nginx.org/en/docs/stream/ngx_stream_log_module.html#log_format) for TCP, UDP, and TLS Passthrough traffic. For convenience, it is possible to define the log format across multiple lines (each line separated by *\n*). In that case, the Ingress Controller will replace every *\n* character with a space character. All *'* characters must be escaped. | See the [template file](https://github.com/nginx/kubernetes-ingress/blob/v{{< nic-version >}}/internal/configs/version1/nginx.tmpl). | | +|*stream-log-format-escaping* | Sets the characters escaping for the variables of the stream log format. Supported values: *json* (JSON escaping), *default* (the default escaping) *none* (disables escaping). | *default* | | + +{{< /table >}} + +### Request URI/Header manipulation + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*proxy-hide-headers* | Sets the value of one or more [proxy_hide_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header) directives. Example: *"nginx.org/proxy-hide-headers": "header-a,header-b"* | N/A | | +|*proxy-pass-headers* | Sets the value of one or more [proxy_pass_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header) directives. Example: *"nginx.org/proxy-pass-headers": "header-a,header-b"* | N/A | | + +{{< /table >}} + +### Auth and SSL/TLS + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*redirect-to-https* | Sets the 301 redirect rule based on the value of the *http_x_forwarded_proto* header on the server block to force incoming traffic to be over HTTPS. Useful when terminating SSL in a load balancer in front of the Ingress Controller — see [115](https://github.com/nginx/kubernetes-ingress/issues/115) | *False* | | +|*ssl-redirect* | Sets an unconditional 301 redirect rule for all incoming HTTP traffic to force incoming traffic over HTTPS. | *True* | | +|*hsts* | Enables [HTTP Strict Transport Security (HSTS)](https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/) : the HSTS header is added to the responses from backends. The *preload* directive is included in the header. | *False* | | +|*hsts-max-age* | Sets the value of the *max-age* directive of the HSTS header. | *2592000* (1 month) | | +|*hsts-include-subdomains* | Adds the *includeSubDomains* directive to the HSTS header. | *False* | | +|*hsts-behind-proxy* | Enables HSTS based on the value of the *http_x_forwarded_proto* request header. Should only be used when TLS termination is configured in a load balancer (proxy) in front of the Ingress Controller. Note: to control redirection from HTTP to HTTPS configure the *nginx.org/redirect-to-https* annotation. | *False* | | +|*ssl-protocols* | Sets the value of the [ssl_protocols](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols) directive. | *TLSv1 TLSv1.1 TLSv1.2* | | +|*ssl-prefer-server-ciphers* | Enables or disables the [ssl_prefer_server_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers) directive. | *False* | | +|*ssl-ciphers* | Sets the value of the [ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers) directive. | *HIGH:!aNULL:!MD5* | | +|*ssl-dhparam-file* | Sets the content of the dhparam file. The controller will create the file and set the value of the [ssl_dhparam](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_dhparam) directive with the path of the file. | N/A | | + +{{< /table >}} + +### Listeners + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*http2* | Enables HTTP/2 in servers with SSL enabled. | *False* | | +|*proxy-protocol* | Enables PROXY Protocol for incoming connections. | *False* | [Proxy Protocol](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/shared-examples/proxy-protocol). | + +{{< /table >}} + +### Backend services (Upstreams) + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*lb-method* | Sets the [load balancing method](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#choosing-a-load-balancing-method). To use the round-robin method, specify *"round_robin"*. | *"random two least_conn"* | | +|*max-fails* | Sets the value of the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) parameter of the *server* directive. | *1* | | +|*upstream-zone-size* | Sets the size of the shared memory [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) for upstreams. For NGINX, the special value 0 disables the shared memory zones. For NGINX Plus, shared memory zones are required and cannot be disabled. The special value 0 will be ignored. | *256k* for NGINX, *512k* for NGINX Plus | | +|*fail-timeout* | Sets the value of the [fail_timeout](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) parameter of the *server* directive. | *10s* | | +|*keepalive* | Sets the value of the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. Note that *proxy_set_header Connection "";* is added to the generated configuration when the value > 0. | *0* | | + +{{< /table >}} + +### Zone Sync + +Zone Sync enables the [ngx_stream_zone_sync_module](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html) in NGINX Ingress Controller when NGINX Plus is used. Multiple replicas are required to effectively utililise this functionality. More information is available in the [How NGINX Plus Performs Zone Synchronization](https://docs.nginx.com/nginx/admin-guide/high-availability/zone_sync_details/) topic. + +Zone synchronization with TLS for NGINX Ingress Controller is not yet available with ConfigMap. If you would like to enable Zone Sync with TLS, please remove `zone-sync` from ConfigMap and add Zone Sync parameters via [`stream-snippets`]({{< ref "/configuration/ingress-resources/advanced-configuration-with-snippets.md" >}}) similar to [this example](https://github.com/nginx/kubernetes-ingress/blob/v4.0.1/examples/custom-resources/oidc/nginx-config.yaml) and adding the [zone_sync_ssl directive](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_ssl) along with any other TLS parameters to the `stream-snippets`. + +You will also need to manually add the headless service, such as in [this example](https://github.com/nginx/kubernetes-ingress/blob/v4.0.1/examples/custom-resources/oidc/nginx-ingress-headless.yaml). + +{{< call-out "caution" >}} +If you previously installed OIDC or used the `zone_sync` directive with `stream-snippets` in [v4.0.1](https://github.com/nginx/kubernetes-ingress/tree/v4.0.1) or earlier, and you plan to enable the `zone-sync` ConfigMap key, the `zone_sync` directive should be removed from `stream-snippets`. + +If you encounter the error `error [emerg] 13#13: "zone_sync" directive is duplicate in /etc/nginx/nginx.conf:164` it is likely due to `zone_sync` being enabled in both `stream-snippets` and the ConfigMap. Once upgraded, remove the [old headless service](https://github.com/nginx/kubernetes-ingress/blob/v4.0.1/examples/custom-resources/oidc/nginx-ingress-headless.yaml) deployed for OIDC. +{{< /call-out >}} + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*zone-sync* | Enables zone synchronization between NGINX Ingress Controller Pods. This autogenerates a [zone_sync_server](https://nginx.org/en/docs/stream/ngx_stream_zone_sync_module.html#zone_sync_server) and a headless service using the `ReplicaSet`, `DaemonSet` or `StatefulSet` name. Please note that this headless service will be automatically cleaned up when uninstalling via Helm or by removing the value from the ConfigMap. The headless service will need to be manually removed if the `controller.customConfigMap` value is set via Helm or the deployment is uninstalled via Manifests. Each Ingress Controller manages its own headless service. NGINX Plus Required. | *False* | | +|*zone-sync-port* | Specifies the optional port on which NGINX Ingress Controller listens for zone sync traffic. NGINX Plus & `zone-sync` Required. | *12345* | | +|*zone-sync-resolver-addresses* | Configures optional addresses used in the [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive for zone-sync. This field takes a comma separated list of addresses. NGINX Plus & `zone-sync` Required | `kube-dns.kube-system.svc.cluster.local` | | +|*zone-sync-resolver-ipv6* | Configures whether the optional [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive for zone-sync will look up IPv6 addresses. NGINX Plus & `zone-sync` Required | `true` | | +|*zone-sync-resolver-valid* | Configures an [NGINX time](https://nginx.org/en/docs/syntax.html) that the optional [resolver](https://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) directive for zone-sync will override the TTL value of responses from nameservers with. NGINX Plus & `zone-sync` Required | `5s` | | + +{{< /table >}} + +### Snippets and custom templates + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*main-snippets* | Sets a custom snippet in main context. | N/A | | +|*http-snippets* | Sets a custom snippet in http context. | N/A | | +|*location-snippets* | Sets a custom snippet in location context. | N/A | | +|*server-snippets* | Sets a custom snippet in server context. | N/A | | +|*stream-snippets* | Sets a custom snippet in stream context. | N/A | [Support for TCP/UDP Load Balancing](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/tcp-udp). | +|*main-template* | Sets the main NGINX configuration template. | By default the template is read from the file in the container. | [Custom Templates](/nginx-ingress-controller/configuration/global-configuration/custom-templates). | +|*ingress-template* | Sets the NGINX configuration template for an Ingress resource. | By default the template is read from the file on the container. | [Custom Templates](/nginx-ingress-controller/configuration/global-configuration/custom-templates). | +|*virtualserver-template* | Sets the NGINX configuration template for an VirtualServer resource. | By default the template is read from the file on the container. | [Custom Templates](/nginx-ingress-controller/configuration/global-configuration/custom-templates). | +|*transportserver-template* | Sets the NGINX configuration template for a TransportServer resource. | By default the template is read from the file on the container. | [Custom Templates](/nginx-ingress-controller/configuration/global-configuration/custom-templates). | + +{{< /table >}} + +### Modules + +{{< table >}} + +|ConfigMap Key | Description | Default | Example | +| ---| ---| ---| --- | +|*otel-exporter-endpoint* | OTLP/gRPC endpoint that will accept [OpenTelemetry](https://opentelemetry.io) data. Set `otel-trace-in-http` to *"true"* to enable OpenTelemetry at the global level. | N/A | *"https://otel-collector:4317"* | +|*otel-exporter-header-name* | The name of a custom HTTP header to add to telemetry export request. `otel-exporter-endpoint` and `otel-exporter-header-value` required. | N/A | *"X-custom-header"* | +|*otel-exporter-header-value* | The value of a custom HTTP header to add to telemetry export request. `otel-exporter-endpoint` and `otel-exporter-header-name` required. | N/A | *"custom-value"* | +|*otel-service-name* | Sets the `service.name` attribute of the OTel resource. `otel-exporter-endpoint` required. | N/A | *"nginx-ingress-controller:nginx"* | +| *otel-trace-in-http* | Enables [OpenTelemetry](https://opentelemetry.io) globally (for all Ingress, VirtualServer and VirtualServerRoute resources). Set this to *"false"* to enable OpenTelemetry for individual routes with snippets. `otel-exporter-endpoint` required. | *"false"* | *"true"* | +|*opentracing* | Removed in v5.0.0. Enables [OpenTracing](https://opentracing.io) globally (for all Ingress, VirtualServer and VirtualServerRoute resources). Note: requires the Ingress Controller image with OpenTracing module and a tracer. See the [docs]({{< relref "/installation/integrations/opentracing.md" >}}) for more information. | *False* | | +|*opentracing-tracer* | Removed in v5.0.0. Sets the path to the vendor tracer binary plugin. | N/A | | +|*opentracing-tracer-config* | Removed in v5.0.0. Sets the tracer configuration in JSON format. | N/A | | +|*app-protect-compressed-requests-action* | Sets the *app_protect_compressed_requests_action* [global directive](/nginx-app-protect/configuration/#global-directives). | *drop* | | +|*app-protect-cookie-seed* | Sets the *app_protect_cookie_seed* [global directive](/nginx-app-protect/configuration/#global-directives). | Random automatically generated string | | +|*app-protect-failure-mode-action* | Sets the *app_protect_failure_mode_action* [global directive](/nginx-app-protect/configuration/#global-directives). | *pass* | | +|*app-protect-cpu-thresholds* | Sets the *app_protect_cpu_thresholds* [global directive](/nginx-app-protect/configuration/#global-directives). | *high=100 low=100* | | +|*app-protect-physical-memory-util-thresholds* | Sets the *app_protect_physical_memory_util_thresholds* [global directive](/nginx-app-protect/configuration/#global-directives). | *high=100 low=100* | | +|`app-protect-reconnect-period-seconds` | Sets the `app_protect_reconnect_period_seconds` [global directive](/nginx-app-protect/configuration/#global-directives). | `5` | | +|*app-protect-dos-log-format* | Sets the custom [log format](https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format) for Dos Access log traffic. For convenience, it is possible to define the log format across multiple lines (each line separated by *\n*). In that case, the Ingress Controller will replace every *\n* character with a space character. All *'* characters must be escaped. | `, vs_name_al=$app_protect_dos_vs_name, ip=$remote_addr, tls_fp=$app_protect_dos_tls_fp, outcome=$app_protect_dos_outcome, reason=$app_protect_dos_outcome_reason, policy_name=$app_protect_dos_policy_name, dos_version=$app_protect_dos_version, ip_tls=$remote_addr:$app_protect_dos_tls_fp,` | | +|*app-protect-dos-log-format-escaping* | Sets the characters escaping for the variables of the stream log format. Supported values: *json* (JSON escaping), *default* (the default escaping) *none* (disables escaping). | *default* | | +|*app-protect-dos-arb-fqdn* | Sets the *app-protect-dos-arb-fqdn* [directive](/nginx-app-protect-dos/directives-and-policy/learn-about-directives-and-policy/#arbitrator-fqdn-directive-app_protect_dos_arb_fqdn). | *svc-appprotect-dos-arb* | | + +{{< /table >}} \ No newline at end of file diff --git a/content/includes/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md b/content/includes/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md new file mode 100644 index 000000000..99caae078 --- /dev/null +++ b/content/includes/nic/configuration/ingress-resources/advanced-configuration-with-annotations.md @@ -0,0 +1,249 @@ +--- +title: Advanced configuration with Annotations +toc: true +weight: 200 +nd-content-type: how-to +nd-product: INGRESS +nd-docs: DOCS-591 +--- + +This topic explains how to enable advanced features in F5 NGINX Ingress Controller with Annotations. + +The Ingress resource can use basic NGINX features such as host or path-based routing and TLS termination. Advanced features like rewriting the request URI or inserting additional response headers can be enabled with Annotations. + +Outside of advanced features, Annotations are necessary for customizing NGINX behavior such as setting the value of connection timeouts. + +Customization is also available through the [ConfigMap]({{< relref "/configuration/global-configuration/configmap-resource.md" >}}) resources: Annotations take priority. + +## Using Annotations + +This example uses Annotations to customize the configuration for an Ingress resource: + +```yaml +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: cafe-ingress-with-annotations + annotations: + nginx.org/proxy-connect-timeout: "30s" + nginx.org/proxy-read-timeout: "20s" + nginx.org/client-max-body-size: "4m" + nginx.org/server-snippets: | + location / { + return 302 /coffee; + } +spec: + rules: + - host: cafe.example.com + http: + paths: + - path: /tea + pathType: Prefix + backend: + service: + name: tea-svc + port: + number: 80 + - path: /coffee + pathType: Prefix + backend: + service: + name: coffee-svc + port: + number: 80 +``` + +## Validation + +NGINX Ingress Controller validates the annotations of Ingress resources. If an Ingress is invalid, NGINX Ingress Controller will reject it: the Ingress will continue to exist in the cluster, but NGINX Ingress Controller will ignore it. + +You can check if NGINX Ingress Controller successfully applied the configuration for an Ingress resource. For the example `cafe-ingress-with-annotations` Ingress, you can run: + +```shell +kubectl describe ing cafe-ingress-with-annotations +``` + +```text +... +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal AddedOrUpdated 3s nginx-ingress-controller Configuration for default/cafe-ingress-with-annotations was added or updated +``` + +The events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied. + +If you create an invalid Ingress, NGINX Ingress Controller will reject it and emit a Rejected event. For example, if you create an Ingress `cafe-ingress-with-annotations`, with an annotation `nginx.org/redirect-to-https` set to `yes please` instead of `true`, you will get: + +```shell +kubectl describe ing cafe-ingress-with-annotations +``` + +```text +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Warning Rejected 13s nginx-ingress-controller annotations.nginx.org/redirect-to-https: Invalid value: "yes please": must be a boolean +``` + +Note how the events section includes a Warning event with the Rejected reason. + +{{< call-out "note" >}} If you make an existing Ingress invalid, NGINX Ingress Controller will reject it and remove the corresponding configuration from NGINX. {{< /call-out >}} + +The `nginx.com/jwt-token` Ingress annotation has limited validation. + +## Summary of Annotations + +The table below summarizes the available annotations. + +{{< call-out "note" >}} Annotations that start with `nginx.com` are only supported with NGINX Plus. {{< /call-out >}} + +### General customization + +{{< table >}} + +|Annotation | ConfigMap Key | Description | Default | Example | +| ---| ---| ---| ---| --- | +| *nginx.org/proxy-connect-timeout* | *proxy-connect-timeout* | Sets the value of the [proxy_connect_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) and [grpc_connect_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_connect_timeout) directive. | *60s* | | +| *nginx.org/proxy-read-timeout* | *proxy-read-timeout* | Sets the value of the [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) and [grpc_read_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_read_timeout) directive. | *60s* | | +| *nginx.org/proxy-send-timeout* | *proxy-send-timeout* | Sets the value of the [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) and [grpc_send_timeout](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_send_timeout) directive. | *60s* | | +| *nginx.org/client-max-body-size* | *client-max-body-size* | Sets the value of the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. | *1m* | | +| *nginx.org/client-body-buffer-size* | *client-body-buffer-size* | Sets the value of the [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. | N/A | | +| *nginx.org/proxy-buffering* | *proxy-buffering* | Enables or disables [buffering of responses](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) from the proxied server. | *True* | | +| *nginx.org/proxy-buffers* | *proxy-buffers* | Sets the value of the [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive. | Depends on the platform. | | +| *nginx.org/proxy-buffer-size* | *proxy-buffer-size* | Sets the value of the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) and [grpc_buffer_size](https://nginx.org/en/docs/http/ngx_http_grpc_module.html#grpc_buffer_size) directives. | Depends on the platform. | | +| *nginx.org/proxy-busy-buffers-size* | *proxy-busy-buffers-size* | Sets the value of the [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) directive. | Depends on the platform. | | +| *nginx.org/proxy-max-temp-file-size* | *proxy-max-temp-file-size* | Sets the value of the [proxy_max_temp_file_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size) directive. | *1024m* | | +| *nginx.org/server-tokens* | *server-tokens* | Enables or disables the [server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens) directive. Additionally, with the NGINX Plus, you can specify a custom string value, including the empty string value, which disables the emission of the “Server” field. | *True* | | +| *nginx.org/path-regex* | N/A | Enables regular expression modifiers for Ingress path parameter. This translates to the NGINX [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) directive. You can specify one of these values: "case_sensitive", "case_insensitive", or "exact". The annotation is applied to the entire Ingress resource and its paths. While using Master and Minion Ingresses i.e. Mergeable Ingresses, this annotation can be specified on Minion types. The `path-regex` annotation specified on Master is ignored, and has no effect on paths defined on Minions. | N/A | [path-regex](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/path-regex) | + +{{< /table >}} + +### Request URI/Header Manipulation + +{{< table >}} + +|Annotation | ConfigMap Key | Description | Default | Example | +| ---| ---| ---| ---| --- | +| *nginx.org/proxy-hide-headers* | *proxy-hide-headers* | Sets the value of one or more [proxy_hide_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header) directives. Example: ``"nginx.org/proxy-hide-headers": "header-a,header-b"* | N/A | | +| *nginx.org/proxy-pass-headers* | *proxy-pass-headers* | Sets the value of one or more [proxy_pass_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header) directives. Example: ``"nginx.org/proxy-pass-headers": "header-a,header-b"* | N/A | | +| *nginx.org/rewrites* | N/A | Configures URI rewriting using [proxy_pass](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass) directive. | N/A | [rewrites](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/rewrites) | +|*nginx.org/proxy-set-headers* | N/A | Enables customization of proxy headers and values using the [proxy_set_header](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive. Example: ``"nginx.org/proxy-set-headers": "header-a: valueA,header-b: valueB,header-c: valueC"`` | N/A | [Proxy Set Headers](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/proxy-set-headers). | + +{{< /table >}} + +### Auth and SSL/TLS + +{{< table >}} + +|Annotation | ConfigMap Key | Description | Default | Example | +| ---| ---| ---| ---| --- | +| *nginx.org/redirect-to-https* | *redirect-to-https* | Sets the 301 redirect rule based on the value of the ``http_x_forwarded_proto* header on the server block to force incoming traffic to be over HTTPS. Useful when terminating SSL in a load balancer in front of NGINX Ingress Controller — see [115](https://github.com/nginx/kubernetes-ingress/issues/115) | *False* | | +| *ingress.kubernetes.io/ssl-redirect* | *ssl-redirect* | Sets an unconditional 301 redirect rule for all incoming HTTP traffic to force incoming traffic over HTTPS. | *True* | | +| *nginx.org/hsts* | *hsts* | Enables [HTTP Strict Transport Security (HSTS)](https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/)\ : the HSTS header is added to the responses from backends. The ``preload* directive is included in the header. | *False* | | +| *nginx.org/hsts-max-age* | *hsts-max-age* | Sets the value of the ``max-age* directive of the HSTS header. | *2592000* (1 month) | | +| *nginx.org/hsts-include-subdomains* | *hsts-include-subdomains* | Adds the ``includeSubDomains* directive to the HSTS header. | *False* | | +| *nginx.org/hsts-behind-proxy* | *hsts-behind-proxy* | Enables HSTS based on the value of the ``http_x_forwarded_proto* request header. Should only be used when TLS termination is configured in a load balancer (proxy) in front of NGINX Ingress Controller. Note: to control redirection from HTTP to HTTPS configure the ``nginx.org/redirect-to-https* annotation. | *False* | | +| *nginx.org/basic-auth-secret* | N/A | Specifies a Secret resource with a user list for HTTP Basic authentication. | N/A | | +| *nginx.org/basic-auth-realm* | N/A | Specifies a realm. | N/A | | +| *nginx.org/ssl-ciphers* | *ssl-ciphers* | Sets the value of the [ssl_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ciphers) directive. | N/A | *HIGH:!aNULL:!MD5* | +| *nginx.org/ssl-prefer-server-ciphers* | *ssl-prefer-server-ciphers* | Enables or disables the [ssl_prefer_server_ciphers](https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_prefer_server_ciphers) directive. | *False* | | +| *nginx.com/jwt-key* | N/A | Specifies a Secret resource with keys for validating JSON Web Tokens (JWTs). | N/A | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/jwt). | +| *nginx.com/jwt-realm* | N/A | Specifies a realm. | N/A | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/jwt). | +| *nginx.com/jwt-token* | N/A | Specifies a variable that contains a JSON Web Token. | By default, a JWT is expected in the ``Authorization* header as a Bearer Token. | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/jwt). | +| *nginx.com/jwt-login-url* | N/A | Specifies a URL to which a client is redirected in case of an invalid or missing JWT. | N/A | [Support for JSON Web Tokens (JWTs)](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/jwt). | + +{{< /table >}} + +### Listeners + +{{< table >}} + +|Annotation | ConfigMap Key | Description | Default | Example | +| ---| ---| ---| ---| --- | +| *nginx.org/listen-ports* | N/A | Configures HTTP ports that NGINX will listen on. | *[80]* | | +| *nginx.org/listen-ports-ssl* | N/A | Configures HTTPS ports that NGINX will listen on. | *[443]* | | + +{{< /table >}} + +### Backend services (Upstreams) + +{{< table >}} + +|Annotation | ConfigMap Key | Description | Default | Example | +| ---| ---| ---| ---| --- | +| *nginx.org/lb-method* | *lb-method* | Sets the [load balancing method](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#choosing-a-load-balancing-method). To use the round-robin method, specify ``"round_robin"``. | *"random two least_conn"* | | +| *nginx.org/ssl-services* | N/A | Enables HTTPS or gRPC over SSL when connecting to the endpoints of services. | N/A | [ssl-services](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/ssl-services) | +| *nginx.org/grpc-services* | N/A | Enables gRPC for services. Note: requires HTTP/2 (see ``http2* ConfigMap key); only works for Ingresses with TLS termination enabled. | N/A | [grpc-services](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/grpc-services) | +| *nginx.org/websocket-services* | N/A | Enables WebSocket for services. | N/A | [websocket](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/websocket) | +| *nginx.org/max-fails* | *max-fails* | Sets the value of the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) parameter of the ``server* directive. | *1* | | +| *nginx.org/max-conns* | N\A | Sets the value of the [max_conns](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_conns) parameter of the ``server* directive. | *0* | | +| *nginx.org/upstream-zone-size* | *upstream-zone-size* | Sets the size of the shared memory [zone](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone) for upstreams. For NGINX, the special value 0 disables the shared memory zones. For NGINX Plus, shared memory zones are required and cannot be disabled. The special value 0 will be ignored. | *256K* | | +| *nginx.org/fail-timeout* | *fail-timeout* | Sets the value of the [fail_timeout](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) parameter of the ``server* directive. | *10s* | | +| *nginx.com/sticky-cookie-services* | N/A | Configures session persistence. | N/A | [session-persistence](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/session-persistence) | +| *nginx.org/keepalive* | *keepalive* | Sets the value of the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. Note that ``proxy_set_header Connection "";* is added to the generated configuration when the value > 0. | *0* | | +| *nginx.com/health-checks* | N/A | Enables active health checks. | *False* | [health-checks](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/health-checks) | +| *nginx.com/health-checks-mandatory* | N/A | Configures active health checks as mandatory. | *False* | [health-checks](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/health-checks) | +| *nginx.com/health-checks-mandatory-queue* | N/A | When active health checks are mandatory, creates a queue where incoming requests are temporarily stored while NGINX Plus is checking the health of the endpoints after a configuration reload. | *0* | [health-checks](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/health-checks) | +| *nginx.com/slow-start* | N/A | Sets the upstream server [slow-start period](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#server-slow-start). By default, slow-start is activated after a server becomes [available](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-health-check/#passive-health-checks) or [healthy](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-health-check/#active-health-checks). To enable slow-start for newly-added servers, configure [mandatory active health checks](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/health-checks). | *"0s"* | | +| *nginx.org/use-cluster-ip* | N/A | Enables using the Cluster IP and port of the service instead of the default behavior of using the IP and port of the pods. When this field is enabled, the fields that configure NGINX behavior related to multiple upstream servers (like ``lb-method* and ``next-upstream``) will have no effect, as NGINX Ingress Controller will configure NGINX with only one upstream server that will match the service Cluster IP. | *False* | | + +{{< /table >}} + +### Rate limiting + +{{< table >}} + +|Annotation | ConfigMap Key | Description | Default | Example | +| ---| ---| ---| ---| --- | +| *nginx.org/limit-req-rate* | N/A | Enables request-rate-limiting for this ingress by creating a [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone) and matching [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) for each location. All servers/locations of one ingress share the same zone. Must have unit r/s or r/m. | N/A | 200r/s | +| *nginx.org/limit-req-key* | N/A | The key to which the rate limit is applied. Can contain text, variables, or a combination of them. Variables must be surrounded by ${}. | ${binary_remote_addr} | ${binary_remote_addr} | +| *nginx.org/limit-req-zone-size* | N/A | Configures the size of the created [limit_req_zone](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req_zone). | 10m | 20m | +| *nginx.org/limit-req-delay* | N/A | Configures the delay-parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive. | 0 | 100 | +| *nginx.org/limit-req-no-delay* | N/A | Configures the nodelay-parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive. | false | true | +| *nginx.org/limit-req-burst* | N/A | Configures the burst-parameter of the [limit_req](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html#limit_req) directive. | N/A | 100 | +| *nginx.org/limit-req-dry-run* | N/A | Enables the dry run mode. In this mode, the rate limit is not actually applied, but the number of excessive requests is accounted as usual in the shared memory zone. | false | true | +| *nginx.org/limit-req-log-level* | N/A | Sets the desired logging level for cases when the server refuses to process requests due to rate exceeding, or delays request processing. Allowed values are info, notice, warn or error. | error | info | +| *nginx.org/limit-req-reject-code* | N/A | Sets the status code to return in response to rejected requests. Must fall into the range 400..599. | 429 | 503 | +| *nginx.org/limit-req-scale* | N/A | Enables a constant rate-limit by dividing the configured rate by the number of nginx-ingress pods currently serving traffic. This adjustment ensures that the rate-limit remains consistent, even as the number of nginx-pods fluctuates due to autoscaling. Note: This will not work properly if requests from a client are not evenly distributed accross all ingress pods (sticky sessions, long lived TCP-Connections with many requests etc.). In such cases using [zone-sync]({{< ref "/configuration/global-configuration/configmap-resource.md#zone-sync" >}}) instead would give better results. Enabling `zone-sync` will suppress this setting. | false | true | + +{{< /table >}} + +### Snippets and custom templates + +{{< table >}} + +|Annotation | ConfigMap Key | Description | Default | Example | +| ---| ---| ---| ---| --- | +| *nginx.org/location-snippets* | *location-snippets* | Sets a custom snippet in location context. | N/A | | +| *nginx.org/server-snippets* | *server-snippets* | Sets a custom snippet in server context. | N/A | | + +{{< /table >}} + +### F5 WAF for NGINX + +{{< call-out "note" >}} The App Protect annotations only work if the F5 WAF for NGINX module is [installed]({{< relref "installation/integrations/app-protect-waf/installation.md" >}}). {{< /call-out >}} + +{{< table >}} + +|Annotation | ConfigMap Key | Description | Default | Example | +| ---| ---| ---| ---| --- | +| *appprotect.f5.com/app-protect-policy* | N/A | The name of the App Protect Policy for the Ingress Resource. Format is ``namespace/name``. If no namespace is specified, the same namespace of the Ingress Resource is used. If not specified but ``appprotect.f5.com/app-protect-enable* is true, a default policy id applied. If the referenced policy resource does not exist, or policy is invalid, this annotation will be ignored, and the default policy will be applied. | N/A | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/app-protect-waf) | +| *appprotect.f5.com/app-protect-enable* | N/A | Enable App Protect for the Ingress Resource. | *False* | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/app-protect-waf) | +| *appprotect.f5.com/app-protect-security-log-enable* | N/A | Enable the [security log](/nginx-app-protect/troubleshooting/#app-protect-logging-overview) for App Protect. | *False* | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/app-protect-waf) | +| *appprotect.f5.com/app-protect-security-log* | N/A | The App Protect log configuration for the Ingress Resource. Format is ``namespace/name``. If no namespace is specified, the same namespace as the Ingress Resource is used. If not specified the default is used which is: filter: ``illegal``, format: ``default``. Multiple configurations can be specified in a comma separated list. Both log configurations and destinations list (see below) must be of equal length. Configs and destinations are paired by the list indices. | N/A | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/app-protect-waf) | +| *appprotect.f5.com/app-protect-security-log-destination* | N/A | The destination of the security log. For more information check the [DESTINATION argument](/nginx-app-protect/troubleshooting/#app-protect-logging-overview). Multiple destinations can be specified in a comma-separated list. Both log configurations and destinations list (see above) must be of equal length. Configs and destinations are paired by the list indices. | *syslog:server=localhost:514* | [app-protect-waf](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/app-protect-waf) | + +{{< /table >}} + +### F5 DoS for NGINX + +{{< call-out "note" >}} F5 DoS for NGINX annotations only work if the F5 DoS for NGINX module is [installed]({{< relref "installation/integrations/app-protect-dos/installation.md" >}}). {{< /call-out >}} + +{{< table >}} + +|Annotation | ConfigMap Key | Description | Default | Example | +| ---| ---| ---| ---| --- | +| *appprotectdos.f5.com/app-protect-dos-resource* | N/A | Enable F5 DoS for NGINX for the Ingress Resource by specifying a [DosProtectedResource]({{< relref "installation/integrations/app-protect-dos/dos-protected.md" >}}). | N/A | [app-protect-dos](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/app-protect-dos) | + +{{< /table >}} \ No newline at end of file diff --git a/content/includes/nic/configuration/virtualserver-and-virtualserverroute-resources.md b/content/includes/nic/configuration/virtualserver-and-virtualserverroute-resources.md new file mode 100644 index 000000000..c5720335f --- /dev/null +++ b/content/includes/nic/configuration/virtualserver-and-virtualserverroute-resources.md @@ -0,0 +1,1083 @@ +--- +title: VirtualServer and VirtualServerRoute resources +toc: true +weight: 700 +nd-content-type: reference +nd-product: INGRESS +nd-docs: DOCS-599 +--- + +This document is reference material for the VirtualServer and VirtualServerRoute resources used by F5 NGINX Ingress Controller. + +VirtualServer and VirtualServerRoute resources are load balancing configurations recommended as an alternative to the Ingress resource. + +They enable use cases not supported with the Ingress resource, such as traffic splitting and advanced content-based routing. The resources are implemented as [Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). + +The GitHub repository has [examples of the resources](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/custom-resources) for specific use cases. + +--- + +## VirtualServer specification + +The VirtualServer resource defines load balancing configuration for a domain name, such as `example.com`. Below is an example of such configuration: + +```yaml +apiVersion: k8s.nginx.org/v1 +kind: VirtualServer +metadata: + name: cafe +spec: + host: cafe.example.com + listener: + http: http-8083 + https: https-8443 + tls: + secret: cafe-secret + gunzip: on + upstreams: + - name: tea + service: tea-svc + port: 80 + - name: coffee + service: coffee-svc + port: 80 + routes: + - path: /tea + action: + pass: tea + - path: /coffee + action: + pass: coffee + - path: ~ ^/decaf/.*\\.jpg$ + action: + pass: coffee + - path: = /green/tea + action: + pass: tea +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``host`` | The host (domain name) of the server. Must be a valid subdomain as defined in RFC 1123, such as ``my-app`` or ``hello.example.com``. When using a wildcard domain like ``*.example.com`` the domain must be contained in double quotes. The ``host`` value needs to be unique among all Ingress and VirtualServer resources. See also [Handling Host and Listener Collisions](/nginx-ingress-controller/configuration/host-and-listener-collisions). | ``string`` | Yes | +|``listener`` | Sets a custom HTTP and/or HTTPS listener. Valid fields are `listener.http` and `listener.https`. Each field must reference the name of a valid listener defined in a GlobalConfiguration resource | [listener](#virtualserverlistener) | No | +|``tls`` | The TLS termination configuration. | [tls](#virtualservertls) | No | +|``gunzip`` | Enables or disables [decompression](https://docs.nginx.com/nginx/admin-guide/web-server/compression/) of gzipped responses for clients. Allowed values “on”/“off”, “true”/“false” or “yes”/“no”. If the ``gunzip`` value is not set, it defaults to ``off``. | ``boolean`` | No | +|``externalDNS`` | The externalDNS configuration for a VirtualServer. | [externalDNS](#virtualserverexternaldns) | No | +|``dos`` | A reference to a DosProtectedResource, setting this enables DOS protection of the VirtualServer. | ``string`` | No | +|``policies`` | A list of policies. | [[]policy](#virtualserverpolicy) | No | +|``upstreams`` | A list of upstreams. | [[]upstream](#upstream) | No | +|``routes`` | A list of routes. | [[]route](#virtualserverroute) | No | +|``ingressClassName`` | Specifies which Ingress Controller must handle the VirtualServer resource. | ``string`` | No | +|``internalRoute`` | Specifies if the VirtualServer resource is an internal route or not. | ``boolean`` | No | +|``http-snippets`` | Sets a custom snippet in the http context. | ``string`` | No | +|``server-snippets`` | Sets a custom snippet in server context. Overrides the ``server-snippets`` ConfigMap key. | ``string`` | No | + +### VirtualServer.TLS + +The tls field defines TLS configuration for a VirtualServer. For example: + +```yaml +secret: cafe-secret +redirect: + enable: true +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``secret`` | The name of a secret with a TLS certificate and key. The secret must belong to the same namespace as the VirtualServer. The secret must be of the type ``kubernetes.io/tls`` and contain keys named ``tls.crt`` and ``tls.key`` that contain the certificate and private key as described [here](https://kubernetes.io/docs/concepts/services-networking/ingress/#tls). If the secret doesn't exist or is invalid, NGINX will break any attempt to establish a TLS connection to the host of the VirtualServer. If the secret is not specified but [wildcard TLS secret](/nginx-ingress-controller/configuration/global-configuration/command-line-arguments#cmdoption-wildcard-tls-secret) is configured, NGINX will use the wildcard secret for TLS termination. | ``string`` | No | +|``redirect`` | The redirect configuration of the TLS for a VirtualServer. | [tls.redirect](#virtualservertlsredirect) | No | ### VirtualServer.TLS.Redirect | +|``cert-manager`` | The cert-manager configuration of the TLS for a VirtualServer. | [tls.cert-manager](#virtualservertlscertmanager) | No | ### VirtualServer.TLS.CertManager | + +### VirtualServer.TLS.Redirect + +The redirect field configures a TLS redirect for a VirtualServer: + +```yaml +enable: true +code: 301 +basedOn: scheme +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``enable`` | Enables a TLS redirect for a VirtualServer. The default is ``False``. | ``boolean`` | No | +|``code`` | The status code of a redirect. The allowed values are: ``301`` , ``302`` , ``307`` , ``308``. The default is ``301``. | ``int`` | No | +|``basedOn`` | The attribute of a request that NGINX will evaluate to send a redirect. The allowed values are ``scheme`` (the scheme of the request) or ``x-forwarded-proto`` (the ``X-Forwarded-Proto`` header of the request). The default is ``scheme``. | ``string`` | No | ### VirtualServer.Policy | + +### VirtualServer.TLS.CertManager + +The cert-manager field configures x509 automated Certificate management for VirtualServer resources using cert-manager (cert-manager.io). Please see the [cert-manager configuration documentation](https://cert-manager.io/docs/configuration/) for more information on deploying and configuring Issuers. Example: + +```yaml +cert-manager: + cluster-issuer: "my-issuer-name" +``` + +{{< table >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``issuer`` | the name of an Issuer. An Issuer is a cert-manager resource which describes the certificate authority capable of signing certificates. The Issuer must be in the same namespace as the VirtualServer resource. Please note that one of `issuer` and `cluster-issuer` are required, but they are mutually exclusive - one and only one must be defined. | ``string`` | No | +|``cluster-issuer`` | the name of a ClusterIssuer. A ClusterIssuer is a cert-manager resource which describes the certificate authority capable of signing certificates. It does not matter which namespace your VirtualServer resides, as ClusterIssuers are non-namespaced resources. Please note that one of `issuer` and `cluster-issuer` are required, but they are mutually exclusive - one and only one must be defined. | ``string`` | No | +|``issuer-kind`` | The kind of the external issuer resource, for example AWSPCAIssuer. This is only necessary for out-of-tree issuers. This cannot be defined if `cluster-issuer` is also defined. | ``string`` | No | +|``issuer-group`` | The API group of the external issuer controller, for example awspca.cert-manager.io. This is only necessary for out-of-tree issuers. This cannot be defined if `cluster-issuer` is also defined. | ``string`` | No | +|``common-name`` | This field allows you to configure spec.commonName for the Certificate to be generated. This configuration adds a CN to the x509 certificate. | ``string`` | No | +|``duration`` | This field allows you to configure spec.duration field for the Certificate to be generated. Must be specified using a [Go time.Duration](https://pkg.go.dev/time#ParseDuration) string format, which does not allow the d (days) suffix. You must specify these values using s, m, and h suffixes instead. | ``string`` | No | +|``renew-before`` | this annotation allows you to configure spec.renewBefore field for the Certificate to be generated. Must be specified using a [Go time.Duration](https://pkg.go.dev/time#ParseDuration) string format, which does not allow the d (days) suffix. You must specify these values using s, m, and h suffixes instead. | ``string`` | No | +|``usages`` | This field allows you to configure spec.usages field for the Certificate to be generated. Pass a string with comma-separated values i.e. ``key agreement,digital signature, server auth``. An exhaustive list of supported key usages can be found in the [the cert-manager api documentation](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.KeyUsage). | ``string`` | No | +|``issue-temp-cert`` | When ``true``, ask cert-manager for a [temporary self-signed certificate](https://cert-manager.io/docs/usage/certificate/#temporary-certificates-while-issuing) pending the issuance of the Certificate. This allows HTTPS-only servers to use ACME HTTP01 challenges when the TLS secret does not exist yet. | ``boolean`` | No | + +{{< /table >}} + +### VirtualServer.Listener + +The listener field defines a custom HTTP and/or HTTPS listener. +The respective listeners used must reference the name of a listener defined using a [GlobalConfiguration](/nginx-ingress-controller/configuration/global-configuration/globalconfiguration-resource/) resource. + +For example: + +```yaml +http: http-8083 +https: https-8443 +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``http`` | The name of am HTTP listener defined in a [GlobalConfiguration](/nginx-ingress-controller/configuration/global-configuration/globalconfiguration-resource/) resource. | ``string`` | No | +|``https`` | The name of an HTTPS listener defined in a [GlobalConfiguration](/nginx-ingress-controller/configuration/global-configuration/globalconfiguration-resource/) resource. | ``string`` | No | + +### VirtualServer.ExternalDNS + +The externalDNS field configures controlling DNS records dynamically for VirtualServer resources using [ExternalDNS](https://github.com/kubernetes-sigs/external-dns). Please see the [ExternalDNS configuration documentation](https://kubernetes-sigs.github.io/external-dns/) for more information on deploying and configuring ExternalDNS and Providers. Example: + +```yaml +enable: true +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``enable`` | Enables ExternalDNS integration for a VirtualServer resource. The default is ``false``. | ``string`` | No | +|``labels`` | Configure labels to be applied to the Endpoint resources that will be consumed by ExternalDNS. | ``map[string]string`` | No | +|``providerSpecific`` | Configure provider specific properties which holds the name and value of a configuration which is specific to individual DNS providers. | [[]ProviderSpecific](#virtualserverexternaldnsproviderspecific) | No | +|``recordTTL`` | TTL for the DNS record. This defaults to 0 if not defined. See [the ExternalDNS TTL documentation for provider-specific defaults](https://kubernetes-sigs.github.io/external-dns/v0.14.2/ttl/#providers) | ``int64`` | No | +|``recordType`` | The record Type that should be created, e.g. "A", "AAAA", "CNAME". This is automatically computed based on the external endpoints if not defined. | ``string`` | No | + +### VirtualServer.ExternalDNS.ProviderSpecific + +The providerSpecific field of the externalDNS block allows the specification of provider specific properties which is a list of key value pairs of configurations which are specific to individual DNS providers. Example: + +```yaml +- name: my-name + value: my-value +- name: my-name2 + value: my-value2 +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``name`` | The name of the key value pair. | ``string`` | Yes | +|``value`` | The value of the key value pair. | ``string`` | Yes | + +### VirtualServer.Policy + +The policy field references a [Policy resource](/nginx-ingress-controller/configuration/policy-resource/) by its name and optional namespace. For example: + +```yaml +name: access-control +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``name`` | The name of a policy. If the policy doesn't exist or invalid, NGINX will respond with an error response with the `500` status code. | ``string`` | Yes | +|``namespace`` | The namespace of a policy. If not specified, the namespace of the VirtualServer resource is used. | ``string`` | No | + +### VirtualServer.Route + +The route defines rules for matching client requests to actions like passing a request to an upstream. For example: + +```yaml + path: /tea + action: + pass: tea +``` + +{{< table >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``path`` | The path of the route. NGINX will match it against the URI of a request. Possible values are: a prefix ( ``/`` , ``/path`` ), an exact match ( ``=/exact/match`` ), a case insensitive regular expression ( ``~*^/Bar.*\.jpg`` ) or a case sensitive regular expression ( ``~^/foo.*\.jpg`` ). In the case of a prefix (must start with ``/`` ) or an exact match (must start with ``=`` ), the path must not include any whitespace characters, ``{`` , ``}`` or ``;``. In the case of the regex matches, all double quotes ``"`` must be escaped and the match can't end in an unescaped backslash ``\``. The path must be unique among the paths of all routes of the VirtualServer. Check the [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) directive for more information. | ``string`` | Yes | +|``policies`` | A list of policies. The policies override the policies of the same type defined in the ``spec`` of the VirtualServer. See [Applying Policies](/nginx-ingress-controller/configuration/policy-resource/#applying-policies) for more details. | [[]policy](#virtualserverpolicy) | No | +|``action`` | The default action to perform for a request. | [action](#action) | No | +|``dos`` | A reference to a DosProtectedResource, setting this enables DOS protection of the VirtualServer route. | ``string`` | No | +|``splits`` | The default splits configuration for traffic splitting. Must include at least 2 splits. | [[]split](#split) | No | +|``matches`` | The matching rules for advanced content-based routing. Requires the default ``action`` or ``splits``. Unmatched requests will be handled by the default ``action`` or ``splits``. | [matches](#match) | No | +|``route`` | The name of a VirtualServerRoute resource that defines this route. If the VirtualServerRoute belongs to a different namespace than the VirtualServer, you need to include the namespace. For example, ``tea-namespace/tea``. | ``string`` | No | +|``errorPages`` | The custom responses for error codes. NGINX will use those responses instead of returning the error responses from the upstream servers or the default responses generated by NGINX. A custom response can be a redirect or a canned response. For example, a redirect to another URL if an upstream server responded with a 404 status code. | [[]errorPage](#errorpage) | No | +|``location-snippets`` | Sets a custom snippet in the location context. Overrides the ``location-snippets`` ConfigMap key. | ``string`` | No | + +{{< /table >}} + +\* -- a route must include exactly one of the following: `action`, `splits`, or `route`. + +## VirtualServerRoute specification + +The VirtualServerRoute resource defines a route for a VirtualServer. It can consist of one or multiple subroutes. The VirtualServerRoute is an alternative to [Mergeable Ingress types](/nginx-ingress-controller/configuration/ingress-resources/cross-namespace-configuration). + +In the example below, the VirtualServer `cafe` from the namespace `cafe-ns` defines a route with the path `/coffee`, which is further defined in the VirtualServerRoute `coffee` from the namespace `coffee-ns`. + +VirtualServer: + +```yaml +apiVersion: k8s.nginx.org/v1 +kind: VirtualServer +metadata: + name: cafe + namespace: cafe-ns +spec: + host: cafe.example.com + upstreams: + - name: tea + service: tea-svc + port: 80 + routes: + - path: /tea + action: + pass: tea + - path: /coffee + route: coffee-ns/coffee +``` + +VirtualServerRoute: + +```yaml +apiVersion: k8s.nginx.org/v1 +kind: VirtualServerRoute +metadata: + name: coffee + namespace: coffee-ns +spec: + host: cafe.example.com + upstreams: + - name: latte + service: latte-svc + port: 80 + - name: espresso + service: espresso-svc + port: 80 + subroutes: + - path: /coffee/latte + action: + pass: latte + - path: /coffee/espresso + action: + pass: espresso +``` + +Note that each subroute must have a `path` that starts with the same prefix (here `/coffee`), which is defined in the route of the VirtualServer. Additionally, the `host` in the VirtualServerRoute must be the same as the `host` of the VirtualServer. + +{{< table >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``host`` | The host (domain name) of the server. Must be a valid subdomain as defined in RFC 1123, such as ``my-app`` or ``hello.example.com``. When using a wildcard domain like ``*.example.com`` the domain must be contained in double quotes. Must be the same as the ``host`` of the VirtualServer that references this resource. | ``string`` | Yes | +|``upstreams`` | A list of upstreams. | [[]upstream](#upstream) | No | +|``subroutes`` | A list of subroutes. | [[]subroute](#virtualserverroutesubroute) | No | +|``ingressClassName`` | Specifies which Ingress Controller must handle the VirtualServerRoute resource. Must be the same as the ``ingressClassName`` of the VirtualServer that references this resource. | ``string`` | No | + +{{< /table >}} + +### VirtualServerRoute.Subroute + +The subroute defines rules for matching client requests to actions like passing a request to an upstream. For example: + +```yaml +path: /coffee +action: + pass: coffee +``` + +{{< table >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``path`` | The path of the subroute. NGINX will match it against the URI of a request. Possible values are: a prefix ( ``/`` , ``/path`` ), an exact match ( ``=/exact/match`` ), a case insensitive regular expression ( ``~*^/Bar.*\.jpg`` ) or a case sensitive regular expression ( ``~^/foo.*\.jpg`` ). In the case of a prefix, the path must start with the same path as the path of the route of the VirtualServer that references this resource. In the case of an exact or regex match, the path must be the same as the path of the route of the VirtualServer that references this resource. A matching path of the route of the VirtualServer but in different type is not accepted, e.g. a regex path (`~/match`) cannot be used with a prefix path in VirtualServer (`/match`) In the case of a prefix or an exact match, the path must not include any whitespace characters, ``{`` , ``}`` or ``;``. In the case of the regex matches, all double quotes ``"`` must be escaped and the match can't end in an unescaped backslash ``\``. The path must be unique among the paths of all subroutes of the VirtualServerRoute. | ``string`` | Yes | +|``policies`` | A list of policies. The policies override *all* policies defined in the route of the VirtualServer that references this resource. The policies also override the policies of the same type defined in the ``spec`` of the VirtualServer. See [Applying Policies](/nginx-ingress-controller/configuration/policy-resource/#applying-policies) for more details. | [[]policy](#virtualserverpolicy) | No | +|``action`` | The default action to perform for a request. | [action](#action) | No | +|``dos`` | A reference to a DosProtectedResource, setting this enables DOS protection of the VirtualServerRoute subroute. | ``string`` | No | +|``splits`` | The default splits configuration for traffic splitting. Must include at least 2 splits. | [[]split](#split) | No | +|``matches`` | The matching rules for advanced content-based routing. Requires the default ``action`` or ``splits``. Unmatched requests will be handled by the default ``action`` or ``splits``. | [matches](#match) | No | +|``errorPages`` | The custom responses for error codes. NGINX will use those responses instead of returning the error responses from the upstream servers or the default responses generated by NGINX. A custom response can be a redirect or a canned response. For example, a redirect to another URL if an upstream server responded with a 404 status code. | [[]errorPage](#errorpage) | No | +|``location-snippets`` | Sets a custom snippet in the location context. Overrides the ``location-snippets`` of the VirtualServer (if set) or the ``location-snippets`` ConfigMap key. | ``string`` | No | + +{{< /table >}} + +\* -- a subroute must include exactly one of the following: `action` or `splits`. + +## Common VirtualServer and VirtualServerRoute specifications + +### Upstream + +The upstream defines a destination for the routing configuration. For example: + +```yaml +name: tea +service: tea-svc +subselector: + version: canary +port: 80 +lb-method: round_robin +fail-timeout: 10s +max-fails: 1 +max-conns: 32 +keepalive: 32 +connect-timeout: 30s +read-timeout: 30s +send-timeout: 30s +next-upstream: "error timeout non_idempotent" +next-upstream-timeout: 5s +next-upstream-tries: 10 +client-max-body-size: 2m +tls: + enable: true +``` + +{{< call-out "note" >}} The WebSocket protocol is supported without any additional configuration. {{< /call-out >}} + +{{< table >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``name`` | The name of the upstream. Must be a valid DNS label as defined in RFC 1035. For example, ``hello`` and ``upstream-123`` are valid. The name must be unique among all upstreams of the resource. | ``string`` | Yes | +|``service`` | The name of a [service](https://kubernetes.io/docs/concepts/services-networking/service/). The service must belong to the same namespace as the resource. If the service doesn't exist, NGINX will assume the service has zero endpoints and return a ``502`` response for requests for this upstream. For NGINX Plus only, services of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname) are also supported (check the [prerequisites](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/ingress-resources/externalname-services#prerequisites) ). | ``string`` | Yes | +|``subselector`` | Selects the pods within the service using label keys and values. By default, all pods of the service are selected. Note: the specified labels are expected to be present in the pods when they are created. If the pod labels are updated, NGINX Ingress Controller will not see that change until the number of the pods is changed. | ``map[string]string`` | No | +|``use-cluster-ip`` | Enables using the Cluster IP and port of the service instead of the default behavior of using the IP and port of the pods. When this field is enabled, the fields that configure NGINX behavior related to multiple upstream servers (like ``lb-method`` and ``next-upstream``) will have no effect, as NGINX Ingress Controller will configure NGINX with only one upstream server that will match the service Cluster IP. | ``boolean`` | No | +|``port`` | The port of the service. If the service doesn't define that port, NGINX will assume the service has zero endpoints and return a ``502`` response for requests for this upstream. The port must fall into the range ``1..65535``. | ``uint16`` | Yes | +|``lb-method`` | The load [balancing method](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#choosing-a-load-balancing-method). To use the round-robin method, specify ``round_robin``. The default is specified in the ``lb-method`` ConfigMap key. | ``string`` | No | +|``fail-timeout`` | The time during which the specified number of unsuccessful attempts to communicate with an upstream server should happen to consider the server unavailable. See the [fail_timeout](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#fail_timeout) parameter of the server directive. The default is set in the ``fail-timeout`` ConfigMap key. | ``string`` | No | +|``max-fails`` | The number of unsuccessful attempts to communicate with an upstream server that should happen in the duration set by the ``fail-timeout`` to consider the server unavailable. See the [max_fails](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails) parameter of the server directive. The default is set in the ``max-fails`` ConfigMap key. | ``int`` | No | +|``max-conns`` | The maximum number of simultaneous active connections to an upstream server. See the [max_conns](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_conns) parameter of the server directive. By default there is no limit. Note: if keepalive connections are enabled, the total number of active and idle keepalive connections to an upstream server may exceed the ``max_conns`` value. | ``int`` | No | +|``keepalive`` | Configures the cache for connections to upstream servers. The value ``0`` disables the cache. See the [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) directive. The default is set in the ``keepalive`` ConfigMap key. | ``int`` | No | +|``connect-timeout`` | The timeout for establishing a connection with an upstream server. See the [proxy_connect_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_connect_timeout) directive. The default is specified in the ``proxy-connect-timeout`` ConfigMap key. | ``string`` | No | +|``read-timeout`` | The timeout for reading a response from an upstream server. See the [proxy_read_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) directive. The default is specified in the ``proxy-read-timeout`` ConfigMap key. | ``string`` | No | +|``send-timeout`` | The timeout for transmitting a request to an upstream server. See the [proxy_send_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_send_timeout) directive. The default is specified in the ``proxy-send-timeout`` ConfigMap key. | ``string`` | No | +|``next-upstream`` | Specifies in which cases a request should be passed to the next upstream server. See the [proxy_next_upstream](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream) directive. The default is ``error timeout``. | ``string`` | No | +|``next-upstream-timeout`` | The time during which a request can be passed to the next upstream server. See the [proxy_next_upstream_timeout](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_timeout) directive. The ``0`` value turns off the time limit. The default is ``0``. | ``string`` | No | +|``next-upstream-tries`` | The number of possible tries for passing a request to the next upstream server. See the [proxy_next_upstream_tries](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_next_upstream_tries) directive. The ``0`` value turns off this limit. The default is ``0``. | ``int`` | No | +|``client-max-body-size`` | Sets the maximum allowed size of the client request body. See the [client_max_body_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size) directive. The default is set in the ``client-max-body-size`` ConfigMap key. | ``string`` | No | +|``client-body-buffer-size`` | Sets the value of the [client_body_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. | ``string`` | No | +|``tls`` | The TLS configuration for the Upstream. | [tls](#upstreamtls) | No | +|``healthCheck`` | The health check configuration for the Upstream. See the [health_check](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive. Note: this feature is supported only in NGINX Plus. | [healthcheck](#upstreamhealthcheck) | No | +|``slow-start`` | The slow start allows an upstream server to gradually recover its weight from 0 to its nominal value after it has been recovered or became available or when the server becomes available after a period of time it was considered unavailable. By default, the slow start is disabled. See the [slow_start](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start) parameter of the server directive. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods and will be ignored. | ``string`` | No | +|``queue`` | Configures a queue for an upstream. A client request will be placed into the queue if an upstream server cannot be selected immediately while processing the request. By default, no queue is configured. Note: this feature is supported only in NGINX Plus. | [queue](#upstreamqueue) | No | +|``buffering`` | Enables buffering of responses from the upstream server. See the [proxy_buffering](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering) directive. The default is set in the ``proxy-buffering`` ConfigMap key. | ``boolean`` | No | +|``buffers`` | Configures the buffers used for reading a response from the upstream server for a single connection. | [buffers](#upstreambuffers) | No | +|``buffer-size`` | Sets the size of the buffer used for reading the first part of a response received from the upstream server. See the [proxy_buffer_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size) directive. The default is set in the ``proxy-buffer-size`` ConfigMap key. | ``string`` | No | +|``busy-buffer-size`` | Sets the size of the buffer used for reading a response from the upstream server when the response is larger than the ``buffer-size``. See the [proxy_busy_buffers_size](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_busy_buffers_size) directive. The default is set in the ``proxy-busy-buffers-size`` ConfigMap key. | ``string`` | No | +|``ntlm`` | Allows proxying requests with NTLM Authentication. See the [ntlm](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#ntlm) directive. In order for NTLM authentication to work, it is necessary to enable keepalive connections to upstream servers using the ``keepalive`` field. Note: this feature is supported only in NGINX Plus.| ``boolean`` | No | +|``type`` |The type of the upstream. Supported values are ``http`` and ``grpc``. The default is ``http``. For gRPC, it is necessary to enable HTTP/2 in the [ConfigMap](/nginx-ingress-controller/configuration/global-configuration/configmap-resource/#listeners) and configure TLS termination in the VirtualServer. | ``string`` | No | +|``backup`` | The name of the backup service of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname). This will be used when the primary servers are unavailable. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods. | ``string`` | No | +|``backupPort`` | The port of the backup service. The backup port is required if the backup service name is provided. The port must fall into the range ``1..65535``. | ``uint16`` | No | + +{{< /table >}} + +### Upstream.Buffers + +The buffers field configures the buffers used for reading a response from the upstream server for a single connection: + +```yaml +number: 4 +size: 8K +``` + +See the [proxy_buffers](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffers) directive for additional information. + +{{< table >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``number`` | Configures the number of buffers. The default is set in the ``proxy-buffers`` ConfigMap key. | ``int`` | Yes | +|``size`` | Configures the size of a buffer. The default is set in the ``proxy-buffers`` ConfigMap key. | ``string`` | Yes | + +{{< /table >}} + +### Upstream.TLS + +{{< table >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``enable`` | Enables HTTPS for requests to upstream servers. The default is ``False`` , meaning that HTTP will be used. Note: by default, NGINX will not verify the upstream server certificate. To enable the verification, configure an [EgressMTLS Policy](/nginx-ingress-controller/configuration/policy-resource/#egressmtls). | ``boolean`` | No | + +{{< /table >}} + +### Upstream.Queue + +The queue field configures a queue. A client request will be placed into the queue if an upstream server cannot be selected immediately while processing the request: + +```yaml +size: 10 +timeout: 60s +``` + +See [`queue`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#queue) directive for additional information. + +{{< call-out "note" >}} This feature is only supported with NGINX Plus. {{< /call-out >}} + +{{< table >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``size`` | The size of the queue. | ``int`` | Yes | +|``timeout`` | The timeout of the queue. A request cannot be queued for a period longer than the timeout. The default is ``60s``. | ``string`` | No | + +{{< /table >}} + +### Upstream.Healthcheck + +The Healthcheck defines an [active health check](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-health-check/). In the example below we enable a health check for an upstream and configure all the available parameters, including the `slow-start` parameter combined with [`mandatory` and `persistent`](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-health-check/#mandatory-health-checks): + +```yaml +name: tea +service: tea-svc +port: 80 +slow-start: 30s +healthCheck: + enable: true + path: /healthz + interval: 20s + jitter: 3s + fails: 5 + passes: 5 + port: 8080 + tls: + enable: true + connect-timeout: 10s + read-timeout: 10s + send-timeout: 10s + headers: + - name: Host + value: my.service + statusMatch: "! 500" + mandatory: true + persistent: true + keepalive-time: 60s +``` + +{{< call-out "note" >}} This feature is only supported with NGINX Plus. {{< /call-out >}} + +{{< table >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``enable`` | Enables a health check for an upstream server. The default is ``false``. | ``boolean`` | No | +|``path`` | The path used for health check requests. The default is ``/``. This not configurable for gRPC type upstreams. | ``string`` | No | +|``interval`` | The interval between two consecutive health checks. The default is ``5s``. | ``string`` | No | +|``jitter`` | The time within which each health check will be randomly delayed. By default, there is no delay. | ``string`` | No | +|``fails`` | The number of consecutive failed health checks of a particular upstream server after which this server will be considered unhealthy. The default is ``1``. | ``integer`` | No | +|``passes`` | The number of consecutive passed health checks of a particular upstream server after which the server will be considered healthy. The default is ``1``. | ``integer`` | No | +|``port`` | The port used for health check requests. By default, the [server port is used](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check_port). Note: in contrast with the port of the upstream, this port is not a service port, but a port of a pod. | ``integer`` | No | +|``tls`` | The TLS configuration used for health check requests. By default, the ``tls`` field of the upstream is used. | [upstream.tls](#upstreamtls) | No | +|``connect-timeout`` | The timeout for establishing a connection with an upstream server. By default, the ``connect-timeout`` of the upstream is used. | ``string`` | No | +|``read-timeout`` | The timeout for reading a response from an upstream server. By default, the ``read-timeout`` of the upstream is used. | ``string`` | No | +|``send-timeout`` | The timeout for transmitting a request to an upstream server. By default, the ``send-timeout`` of the upstream is used. | ``string`` | No | +|``headers`` | The request headers used for health check requests. NGINX Plus always sets the ``Host`` , ``User-Agent`` and ``Connection`` headers for health check requests. | [[]header](#header) | No | +|``statusMatch`` | The expected response status codes of a health check. By default, the response should have status code 2xx or 3xx. Examples: ``"200"`` , ``"! 500"`` , ``"301-303 307"``. See the documentation of the [match](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html?#match) directive. This not supported for gRPC type upstreams. | ``string`` | No | +|``grpcStatus`` | The expected [gRPC status code](https://github.com/grpc/grpc/blob/master/doc/statuscodes.md#status-codes-and-their-use-in-grpc) of the upstream server response to the [Check method](https://github.com/grpc/grpc/blob/master/doc/health-checking.md). Configure this field only if your gRPC services do not implement the gRPC health checking protocol. For example, configure ``12`` if the upstream server responds with `12 (UNIMPLEMENTED)` status code. Only valid on gRPC type upstreams. | ``int`` | No | +|``grpcService`` | The gRPC service to be monitored on the upstream server. Only valid on gRPC type upstreams. | ``string`` | No | +|``mandatory`` | Require every newly added server to pass all configured health checks before NGINX Plus sends traffic to it. If this is not specified, or is set to false, the server will be initially considered healthy. When combined with [slow-start](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#slow_start), it gives a new server more time to connect to databases and “warm up” before being asked to handle their full share of traffic. | ``bool`` | No | +|``persistent`` | Set the initial “up” state for a server after reload if the server was considered healthy before reload. Enabling persistent requires that the mandatory parameter is also set to `true`. | ``bool`` | No | +|``keepalive-time`` | Enables [keepalive](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive) connections for health checks and specifies the time during which requests can be processed through one keepalive connection. The default is ``60s``. | ``string`` | No | + +{{< /table >}} + +### Upstream.SessionCookie + +The SessionCookie field configures session persistence which allows requests from the same client to be passed to the same upstream server. The information about the designated upstream server is passed in a session cookie generated by NGINX Plus. + +In the example below, we configure session persistence with a session cookie for an upstream and configure all the available parameters: + +```yaml +name: tea +service: tea-svc +port: 80 +sessionCookie: + enable: true + name: srv_id + path: / + expires: 1h + domain: .example.com + httpOnly: false + secure: true + samesite: strict +``` + +See the [`sticky`](https://nginx.org/en/docs/http/ngx_http_upstream_module.html?#sticky) directive for additional information. The session cookie corresponds to the `sticky cookie` method. + +{{< call-out "note" >}} This feature is only supported with NGINX Plus. {{< /call-out >}} + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``enable`` | Enables session persistence with a session cookie for an upstream server. The default is ``false``. | ``boolean`` | No | +|``name`` | The name of the cookie. | ``string`` | Yes | +|``path`` | The path for which the cookie is set. | ``string`` | No | +|``expires`` | The time for which a browser should keep the cookie. Can be set to the special value ``max`` , which will cause the cookie to expire on ``31 Dec 2037 23:55:55 GMT``. | ``string`` | No | +|``domain`` | The domain for which the cookie is set. | ``string`` | No | +|``httpOnly`` | Adds the ``HttpOnly`` attribute to the cookie. | ``boolean`` | No | +|``secure`` | Adds the ``Secure`` attribute to the cookie. | ``boolean`` | No | +|``samesite`` | Adds the ``SameSite`` attribute to the cookie. The allowed values are: ``strict``, ``lax``, ``none`` | ``string`` | No | + +### Header + +The header defines an HTTP Header: + +```yaml +name: Host +value: example.com +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``name`` | The name of the header. | ``string`` | Yes | +|``value`` | The value of the header. | ``string`` | No | + +### Action + +The action defines an action to perform for a request. + +In the example below, client requests are passed to an upstream `coffee`: + +```yaml + path: /coffee + action: + pass: coffee +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``pass`` | Passes requests to an upstream. The upstream with that name must be defined in the resource. | ``string`` | No | +|``redirect`` | Redirects requests to a provided URL. | [action.redirect](#actionredirect) | No | +|``return`` | Returns a preconfigured response. | [action.return](#actionreturn) | No | +|``proxy`` | Passes requests to an upstream with the ability to modify the request/response (for example, rewrite the URI or modify the headers). | [action.proxy](#actionproxy) | No | + +\* -- an action must include exactly one of the following: `pass`, `redirect`, `return` or `proxy`. + +### Action.Redirect + +The redirect action defines a redirect to return for a request. + +In the example below, client requests are passed to a url `http://www.nginx.com`: + +```yaml +redirect: + url: http://www.nginx.com + code: 301 +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``url`` | The URL to redirect the request to. Supported NGINX variables: ``$scheme`` , ``$http_x_forwarded_proto`` , ``$request_uri`` , ``$host``. Variables must be enclosed in curly braces. For example: ``${host}${request_uri}``. | ``string`` | Yes | +|``code`` | The status code of a redirect. The allowed values are: ``301`` , ``302`` , ``307`` , ``308``. The default is ``301``. | ``int`` | No | + +### Action.Return + +The return action defines a preconfigured response for a request. + +In the example below, NGINX will respond with the preconfigured response for every request: + +```yaml +return: + code: 200 + type: text/plain + body: "Hello World\n" + headers: + - name: x-coffee + value: espresso +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``code`` | The status code of the response. The allowed values are: ``2XX``, ``4XX`` or ``5XX``. The default is ``200``. | ``int`` | No | +|``type`` | The MIME type of the response. The default is ``text/plain``. | ``string`` | No | +|``body`` | The body of the response. Supports NGINX variables*. Variables must be enclosed in curly brackets. For example: ``Request is ${request_uri}\n``. | ``string`` | Yes | +|``headers`` | The custom headers of the response. | [[]Action.Return.Header](#actionreturnheader) | No | + +\* -- Supported NGINX variables: `$request_uri`, `$request_method`, `$request_body`, `$scheme`, `$http_`, `$args`, `$arg_`, `$cookie_`, `$host`, `$request_time`, `$request_length`, `$nginx_version`, `$pid`, `$connection`, `$remote_addr`, `$remote_port`, `$time_iso8601`, `$time_local`, `$server_addr`, `$server_port`, `$server_name`, `$server_protocol`, `$connections_active`, `$connections_reading`, `$connections_writing` and `$connections_waiting`. + +### Action.Return.Header + +The header defines an HTTP Header for a canned response in an actionReturn: + +```yaml +name: x-coffee +value: espresso +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``name`` | The name of the header. | ``string`` | Yes | +|``value`` | The value of the header. | ``string`` | Yes | + +### Action.Proxy + +The proxy action passes requests to an upstream with the ability to modify the request/response (for example, rewrite the URI or modify the headers). + +In the example below, the request URI is rewritten to `/`, and the request and the response headers are modified: + +```yaml +proxy: + upstream: coffee + requestHeaders: + pass: true + set: + - name: My-Header + value: Value + - name: Client-Cert + value: ${ssl_client_escaped_cert} + responseHeaders: + add: + - name: My-Header + value: Value + - name: IC-Nginx-Version + value: ${nginx_version} + always: true + hide: + - x-internal-version + ignore: + - Expires + - Set-Cookie + pass: + - Server + rewritePath: / +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``upstream`` | The name of the upstream which the requests will be proxied to. The upstream with that name must be defined in the resource. | ``string`` | Yes | +|``requestHeaders`` | The request headers modifications. | [action.Proxy.RequestHeaders](#actionproxyrequestheaders) | No | +|``responseHeaders`` | The response headers modifications. | [action.Proxy.ResponseHeaders](#actionproxyresponseheaders) | No | +|``rewritePath`` | The rewritten URI. If the route path is a regular expression -- starts with `~` -- the `rewritePath` can include capture groups with ``$1-9``. For example `$1` for the first group, and so on. For more information, check the [rewrite](https://github.com/nginx/kubernetes-ingress/tree/v{{< nic-version >}}/examples/custom-resources/rewrites) example. | ``string`` | No | + +### Action.Proxy.RequestHeaders + +The RequestHeaders field modifies the headers of the request to the proxied upstream server. + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``pass`` | Passes the original request headers to the proxied upstream server. See the [proxy_pass_request_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_headers) directive for more information. Default is true. | ``bool`` | No | +|``set`` | Allows redefining or appending fields to present request headers passed to the proxied upstream servers. See the [proxy_set_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header) directive for more information. | [[]header](#actionproxyrequestheaderssetheader) | No | + +### Action.Proxy.RequestHeaders.Set.Header + +The header defines an HTTP Header: + +```yaml +name: My-Header +value: My-Value +``` + +It is possible to override the default value of the `Host` header, which NGINX Ingress Controller sets to [`$host`](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_host): + +```yaml +name: Host +value: example.com +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``name`` | The name of the header. | ``string`` | Yes | +|``value`` | The value of the header. Supports NGINX variables*. Variables must be enclosed in curly brackets. For example: ``${scheme}``. | ``string`` | No | + +\* -- Supported NGINX variables: `$request_uri`, `$request_method`, `$request_body`, `$scheme`, `$http_`, `$args`, `$arg_`, `$cookie_`, `$host`, `$request_time`, `$request_length`, `$nginx_version`, `$pid`, `$connection`, `$remote_addr`, `$remote_port`, `$time_iso8601`, `$time_local`, `$server_addr`, `$server_port`, `$server_name`, `$server_protocol`, `$connections_active`, `$connections_reading`, `$connections_writing`, `$connections_waiting`, `$ssl_cipher`, `$ssl_ciphers`, `$ssl_client_cert`, `$ssl_client_escaped_cert`, `$ssl_client_fingerprint`, `$ssl_client_i_dn`, `$ssl_client_i_dn_legacy`, `$ssl_client_raw_cert`, `$ssl_client_s_dn`, `$ssl_client_s_dn_legacy`, `$ssl_client_serial`, `$ssl_client_v_end`, `$ssl_client_v_remain`, `$ssl_client_v_start`, `$ssl_client_verify`, `$ssl_curves`, `$ssl_early_data`, `$ssl_protocol`, `$ssl_server_name`, `$ssl_session_id`, `$ssl_session_reused`, `$jwt_claim_` (NGINX Plus only) and `$jwt_header_` (NGINX Plus only). + +### Action.Proxy.ResponseHeaders + +The ResponseHeaders field modifies the headers of the response to the client. + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``hide`` | The headers that will not be passed* in the response to the client from a proxied upstream server. See the [proxy_hide_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header) directive for more information. | ``[]string`` | No | +|``pass`` | Allows passing the hidden header fields* to the client from a proxied upstream server. See the [proxy_pass_header](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_header) directive for more information. | ``[]string`` | No | +|``ignore`` | Disables processing of certain headers** to the client from a proxied upstream server. See the [proxy_ignore_headers](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_headers) directive for more information. | ``[]string`` | No | +|``add`` | Adds headers to the response to the client. | [[]addHeader](#addheader) | No | + +\* -- Default hidden headers are: `Date`, `Server`, `X-Pad` and `X-Accel-...`. + +\** -- The following fields can be ignored: `X-Accel-Redirect`, `X-Accel-Expires`, `X-Accel-Limit-Rate`, `X-Accel-Buffering`, `X-Accel-Charset`, `Expires`, `Cache-Control`, `Set-Cookie` and `Vary`. + +### AddHeader + +The addHeader defines an HTTP Header with an optional `always` field: + +```yaml +name: My-Header +value: My-Value +always: true +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``name`` | The name of the header. | ``string`` | Yes | +|``value`` | The value of the header. Supports NGINX variables*. Variables must be enclosed in curly brackets. For example: ``${scheme}``. | ``string`` | No | +|``always`` | If set to true, add the header regardless of the response status code**. Default is false. See the [add_header](http://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header) directive for more information. | ``bool`` | No | + +\* -- Supported NGINX variables: `$request_uri`, `$request_method`, `$request_body`, `$scheme`, `$http_`, `$args`, `$arg_`, `$cookie_`, `$host`, `$request_time`, `$request_length`, `$nginx_version`, `$pid`, `$connection`, `$remote_addr`, `$remote_port`, `$time_iso8601`, `$time_local`, `$server_addr`, `$server_port`, `$server_name`, `$server_protocol`, `$connections_active`, `$connections_reading`, `$connections_writing`, `$connections_waiting`, `$ssl_cipher`, `$ssl_ciphers`, `$ssl_client_cert`, `$ssl_client_escaped_cert`, `$ssl_client_fingerprint`, `$ssl_client_i_dn`, `$ssl_client_i_dn_legacy`, `$ssl_client_raw_cert`, `$ssl_client_s_dn`, `$ssl_client_s_dn_legacy`, `$ssl_client_serial`, `$ssl_client_v_end`, `$ssl_client_v_remain`, `$ssl_client_v_start`, `$ssl_client_verify`, `$ssl_curves`, `$ssl_early_data`, `$ssl_protocol`, `$ssl_server_name`, `$ssl_session_id`, `$ssl_session_reused`, `$jwt_claim_` (NGINX Plus only) and `$jwt_header_` (NGINX Plus only). + +{{< call-out "note" >}} If `always` is false, the response header is added only if the response status code is any of `200`, `201`, `204`, `206`, `301`, `302`, `303`, `304`, `307` or `308`. {{< /call-out >}} + +### Split + +The split defines a weight for an action as part of the splits configuration. + +In the example below NGINX passes 80% of requests to the upstream `coffee-v1` and the remaining 20% to `coffee-v2`: + +```yaml +splits: +- weight: 80 + action: + pass: coffee-v1 +- weight: 20 + action: + pass: coffee-v2 +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``weight`` | The weight of an action. Must fall into the range ``0..100``. The sum of the weights of all splits must be equal to ``100``. | ``int`` | Yes | +|``action`` | The action to perform for a request. | [action](#action) | Yes | + +### Match + +The match defines a match between conditions and an action or splits. + +In the example below, NGINX routes requests with the path `/coffee` to different upstreams based on the value of the cookie `user`: + +- `user=john` -> `coffee-future` +- `user=bob` -> `coffee-deprecated` +- If the cookie is not set or not equal to either `john` or `bob`, NGINX routes to `coffee-stable` + +```yaml +path: /coffee +matches: +- conditions: + - cookie: user + value: john + action: + pass: coffee-future +- conditions: + - cookie: user + value: bob + action: + pass: coffee-deprecated +action: + pass: coffee-stable +``` + +In the next example, NGINX routes requests based on the value of the built-in [`$request_method` variable](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_method), which represents the HTTP method of a request: + +- all POST requests -> `coffee-post` +- all non-POST requests -> `coffee` + +```yaml +path: /coffee +matches: +- conditions: + - variable: $request_method + value: POST + action: + pass: coffee-post +action: + pass: coffee +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``conditions`` | A list of conditions. Must include at least 1 condition. | [[]condition](#condition) | Yes | +|``action`` | The action to perform for a request. | [action](#action) | No | +|``splits`` | The splits configuration for traffic splitting. Must include at least 2 splits. | [[]split](#split) | No | + +{{< call-out "note" >}} A match must include exactly one of the following: `action` or `splits`. {{< /call-out >}} + +### Condition + +The condition defines a condition in a match. + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``header`` | The name of a header. Must consist of alphanumeric characters or ``-``. | ``string`` | No | +|``cookie`` | The name of a cookie. Must consist of alphanumeric characters or ``_``. | ``string`` | No | +|``argument`` | The name of an argument. Must consist of alphanumeric characters or ``_``. | ``string`` | No | +|``variable`` | The name of an NGINX variable. Must start with ``$``. See the list of the supported variables below the table. | ``string`` | No | +|``value`` | The value to match the condition against. How to define a value is shown below the table. | ``string`` | Yes | + +{{< call-out "note" >}} a condition must include exactly one of the following: `header`, `cookie`, `argument` or `variable`. {{< /call-out >}} + +Supported NGINX variables: `$args`, `$http2`, `$https`, `$remote_addr`, `$remote_port`, `$query_string`, `$request`, `$request_body`, `$request_uri`, `$request_method`, `$scheme`. Find the documentation for each variable [here](https://nginx.org/en/docs/varindex.html). + +The value supports two kinds of matching: + +- *Case-insensitive string comparison*. For example: + - `john` -- case-insensitive matching that succeeds for strings, such as `john`, `John`, `JOHN`. + - `!john` -- negation of the case-insensitive matching for john that succeeds for strings, such as `bob`, `anything`, `''` (empty string). +- *Matching with a regular expression*. Note that NGINX supports regular expressions compatible with those used by the Perl programming language (PCRE). For example: + - `~^yes` -- a case-sensitive regular expression that matches any string that starts with `yes`. For example: `yes`, `yes123`. + - `!~^yes` -- negation of the previous regular expression that succeeds for strings like `YES`, `Yes123`, `noyes`. (The negation mechanism is not part of the PCRE syntax). + - `~*no$` -- a case-insensitive regular expression that matches any string that ends with `no`. For example: `no`, `123no`, `123NO`. + +{{< call-out "note" >}} A value must not include any unescaped double quotes (`"`) and must not end with an unescaped backslash (`\`). For example, the following are invalid values: `some"value`, `somevalue\`. {{< /call-out >}} + +### ErrorPage + +The errorPage defines a custom response for a route for the case when either an upstream server responds with (or NGINX generates) an error status code. The custom response can be a redirect or a canned response. See the [error_page](https://nginx.org/en/docs/http/ngx_http_core_module.html#error_page) directive for more information. + +```yaml +path: /coffee +errorPages: +- codes: [502, 503] + redirect: + code: 301 + url: https://nginx.org +- codes: [404] + return: + code: 200 + body: "Original resource not found, but success!" +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``codes`` | A list of error status codes. | ``[]int`` | Yes | +|``redirect`` | The redirect action for the given status codes. | [errorPage.Redirect](#errorpageredirect) | No | +|``return`` | The canned response action for the given status codes. | [errorPage.Return](#errorpagereturn) | No | + +{{< call-out "note" >}} An errorPage must include exactly one of the following: `return` or `redirect`. {{< /call-out >}} + +### ErrorPage.Redirect + +The redirect defines a redirect for an errorPage. + +In the example below, NGINX responds with a redirect when a response from an upstream server has a 404 status code. + +```yaml +codes: [404] +redirect: + code: 301 + url: ${scheme}://cafe.example.com/error.html +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``code`` | The status code of a redirect. The allowed values are: ``301`` , ``302`` , ``307`` , ``308``. The default is ``301``. | ``int`` | No | +|``url`` | The URL to redirect the request to. Supported NGINX variables: ``$scheme`` and ``$http_x_forwarded_proto``. Variables must be enclosed in curly braces. For example: ``${scheme}``. | ``string`` | Yes | + +### ErrorPage.Return + +The return defines a canned response for an errorPage. + +In the example below, NGINX responds with a canned response when a response from an upstream server has either 401 or 403 status code. + +```yaml +codes: [401, 403] +return: + code: 200 + type: application/json + body: | + {\"msg\": \"You don't have permission to do this\"} + headers: + - name: x-debug-original-statuses + value: ${upstream_status} +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``code`` | The status code of the response. The default is the status code of the original response. | ``int`` | No | +|``type`` | The MIME type of the response. The default is ``text/html``. | ``string`` | No | +|``body`` | The body of the response. Supported NGINX variable: ``$upstream_status`` . Variables must be enclosed in curly braces. For example: ``${upstream_status}``. | ``string`` | Yes | +|``headers`` | The custom headers of the response. | [[]errorPage.Return.Header](#errorpagereturnheader) | No | + +### ErrorPage.Return.Header + +The header defines an HTTP Header for a canned response in an errorPage: + +```yaml +name: x-debug-original-statuses +value: ${upstream_status} +``` + +|Field | Description | Type | Required | +| ---| ---| ---| --- | +|``name`` | The name of the header. | ``string`` | Yes | +|``value`` | The value of the header. Supported NGINX variable: ``$upstream_status`` . Variables must be enclosed in curly braces. For example: ``${upstream_status}``. | ``string`` | No | + +## Using VirtualServer and VirtualServerRoute + +You can use the usual `kubectl` commands to work with VirtualServer and VirtualServerRoute resources, similar to Ingress resources. + +For example, the following command creates a VirtualServer resource defined in `cafe-virtual-server.yaml` with the name `cafe`: + +```shell +kubectl apply -f cafe-virtual-server.yaml +``` + +```text +virtualserver.k8s.nginx.org "cafe" created +``` + +You can get the resource by running: + +```shell +kubectl get virtualserver cafe +``` + +```text +NAME STATE HOST IP PORTS AGE +cafe Valid cafe.example.com 12.13.23.123 [80,443] 3m +``` + +In `kubectl get` and similar commands, you can use the short name `vs` instead of `virtualserver`. + +Similarly, for VirtualServerRoute you can use `virtualserverroute` or the short name `vsr`. + +### Using Snippets + +Snippets allow you to insert raw NGINX config into different contexts of NGINX configuration. In the example below, we use snippets to configure several NGINX features in a VirtualServer: + +```yaml +apiVersion: k8s.nginx.org/v1 +kind: VirtualServer +metadata: + name: cafe + namespace: cafe +spec: + http-snippets: | + limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s; + proxy_cache_path /tmp keys_zone=one:10m; + host: cafe.example.com + tls: + secret: cafe-secret + server-snippets: | + limit_req zone=mylimit burst=20; + upstreams: + - name: tea + service: tea-svc + port: 80 + - name: coffee + service: coffee-svc + port: 80 + routes: + - path: /tea + location-snippets: | + proxy_cache one; + proxy_cache_valid 200 10m; + action: + pass: tea + - path: /coffee + action: + pass: coffee +``` + +For additional information, view the [Advanced configuration with Snippets]({{< ref "/nic/configuration/ingress-resources/advanced-configuration-with-snippets.md" >}}) topic. + +### Validation + +Two types of validation are available for VirtualServer and VirtualServerRoute resources: + +- *Structural validation* by the `kubectl` and Kubernetes API server. +- *Comprehensive validation* by NGINX Ingress Controller. + +#### Structural Validation + +The custom resource definitions for VirtualServer and VirtualServerRoute include structural OpenAPI schema which describes the type of every field of those resources. + +If you try to create (or update) a resource that violates the structural schema (for example, you use a string value for the port field of an upstream), `kubectl` and Kubernetes API server will reject such a resource: + +- Example of `kubectl` validation: + + ```shell + kubectl apply -f cafe-virtual-server.yaml + ``` + + ```text + error: error validating "cafe-virtual-server.yaml": error validating data: ValidationError(VirtualServer.spec.upstreams[0].port): invalid type for org.nginx.k8s.v1.VirtualServer.spec.upstreams.port: got "string", expected "integer"; if you choose to ignore these errors, turn validation off with --validate=false + ``` + +- Example of Kubernetes API server validation: + + ```shell + kubectl apply -f cafe-virtual-server.yaml --validate=false + ``` + + ```text + The VirtualServer "cafe" is invalid: []: Invalid value: map[string]interface {}{ ... }: validation failure list: + spec.upstreams.port in body must be of type integer: "string" + ``` + +If a resource is not rejected (it doesn't violate the structural schema), NGINX Ingress Controller will validate it further. + +#### Comprehensive Validation + +NGINX Ingress Controller validates the fields of the VirtualServer and VirtualServerRoute resources. If a resource is invalid, NGINX Ingress Controller will reject it: the resource will continue to exist in the cluster, but NGINX Ingress Controller will ignore it. + +You can check if NGINX Ingress Controller successfully applied the configuration for a VirtualServer. For our example `cafe` VirtualServer, we can run: + +```shell +kubectl describe vs cafe +``` + +```text +... +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal AddedOrUpdated 16s nginx-ingress-controller Configuration for default/cafe was added or updated +``` + +Note how the events section includes a Normal event with the AddedOrUpdated reason that informs us that the configuration was successfully applied. + +If you create an invalid resource, NGINX Ingress Controller will reject it and emit a Rejected event. For example, if you create a VirtualServer `cafe` with two upstream with the same name `tea`, you will get: + +```shell +kubectl describe vs cafe +``` + +```text +... +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Warning Rejected 12s nginx-ingress-controller VirtualServer default/cafe is invalid and was rejected: spec.upstreams[1].name: Duplicate value: "tea" +``` + +Note how the events section includes a Warning event with the Rejected reason. + +Additionally, this information is also available in the `status` field of the VirtualServer resource. Note the Status section of the VirtualServer: + +```shell +kubectl describe vs cafe +``` + +```text +... +Status: + External Endpoints: + Ip: 12.13.23.123 + Ports: [80,443] + Message: VirtualServer default/cafe is invalid and was rejected: spec.upstreams[1].name: Duplicate value: "tea" + Reason: Rejected + State: Invalid +``` + +NGINX Ingress Controller validates VirtualServerRoute resources in a similar way. + +**Note**: If you make an existing resource invalid, NGINX Ingress Controller will reject it and remove the corresponding configuration from NGINX. + +## Customization using ConfigMap + +You can customize the NGINX configuration for VirtualServer and VirtualServerRoutes resources using the [ConfigMap](/nginx-ingress-controller/configuration/global-configuration/configmap-resource). Most of the ConfigMap keys are supported, with the following exceptions: + +- `proxy-hide-headers` +- `proxy-pass-headers` +- `hsts` +- `hsts-max-age` +- `hsts-include-subdomains` +- `hsts-behind-proxy` +- `redirect-to-https` +- `ssl-redirect` diff --git a/content/ngf/_index.md b/content/ngf/_index.md index 67b974f39..f91d3e15b 100644 --- a/content/ngf/_index.md +++ b/content/ngf/_index.md @@ -27,8 +27,6 @@ nd-product: FABRIC NGINX Gateway Fabric provides an implementation of the [Gateway API](https://gateway-api.sigs.k8s.io/) using [NGINX](https://nginx.org/) as the data plane. The goal of the project is to implement the core Gateway APIs needed to configure an HTTP or TCP/UDP load balancer, reverse proxy, or API gateway for Kubernetes applications. -For a list of supported Gateway API resources and features, see the [Gateway API Compatibility]({{< ref "/ngf/overview/gateway-api-compatibility.md" >}}) documentation. - ## Featured content {{}} @@ -38,8 +36,8 @@ For a list of supported Gateway API resources and features, see the [Gateway API {{}} Choose how to deploy NGINX Gateway Fabric in production. {{}} - {{}} - Review the changes from the latest releases. + {{}} + View how much of the Gateway API NGINX Gateway Fabric supports. {{}} {{}} @@ -58,8 +56,8 @@ For more information, see the [Gateway architecture]({{< ref "/ngf/overview/gate ## More information {{}} - {{}} - View how much of the Gateway API NGINX Gateway Fabric supports. + {{}} + Review the changes from the latest releases. {{}} {{}} Check which versions of NGINX Gateway Fabric match the API. @@ -67,7 +65,7 @@ For more information, see the [Gateway architecture]({{< ref "/ngf/overview/gate {{}} Create simple rules for directing network traffic with HTTPRoute resources. {{}} - {{}} + {{}} Implement HTTPS with Let's Encrypt to secure client-server communication. {{}} {{}} \ No newline at end of file diff --git a/content/ngf/install/manifests.md b/content/ngf/install/manifests.md deleted file mode 100644 index ef1fb6665..000000000 --- a/content/ngf/install/manifests.md +++ /dev/null @@ -1,215 +0,0 @@ ---- -title: Install NGINX Gateway Fabric with Manifests -weight: 200 -toc: true -nd-content-type: how-to -nd-product: FABRIC -nd-docs: DOCS-1429 ---- - -## Overview - -Learn how to install, upgrade, and uninstall NGINX Gateway Fabric using Kubernetes manifests. - -## Before you begin - -To complete this guide, you'll need to install: - -- [kubectl](https://kubernetes.io/docs/tasks/tools/), a command-line interface for managing Kubernetes clusters. -- [Add certificates for secure authentication]({{< ref "/ngf/install/secure-certificates.md" >}}) in a production environment. - -{{< call-out "important" >}} If you’d like to use NGINX Plus, some additional setup is also required: {{< /call-out >}} - -{{< details summary="NGINX Plus JWT setup" >}} - -{{< include "/ngf/installation/jwt-password-note.md" >}} - -### Download the JWT from MyF5 - -{{< include "/ngf/installation/nginx-plus/download-jwt.md" >}} - -### Create the Docker Registry Secret - -{{< include "/ngf/installation/nginx-plus/docker-registry-secret.md" >}} - -### Create the NGINX Plus Secret - -{{< include "/ngf/installation/nginx-plus/nginx-plus-secret.md" >}} - -{{< call-out "note" >}} For more information on why this is needed and additional configuration options, including how to report to NGINX Instance Manager instead, see the [NGINX Plus Image and JWT Requirement]({{< ref "/ngf/install/nginx-plus.md" >}}) document. {{< /call-out >}} - -{{< /details >}} - -## Deploy NGINX Gateway Fabric - -Deploying NGINX Gateway Fabric with Kubernetes manifests takes only a few steps. With manifests, you can configure your deployment exactly how you want. Manifests also make it easy to replicate deployments across environments or clusters, ensuring consistency. - -### Install the Gateway API resources - -{{< include "/ngf/installation/install-gateway-api-resources.md" >}} - -### Deploy the NGINX Gateway Fabric CRDs - -#### Stable release - -```shell -kubectl apply --server-side -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/crds.yaml -``` - -#### Edge version - -```shell -kubectl apply --server-side -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/main/deploy/crds.yaml -``` - -### Deploy NGINX Gateway Fabric - -{{< call-out "note" >}} By default, NGINX Gateway Fabric is installed in the **nginx-gateway** namespace. You can deploy in another namespace by modifying the manifest files. {{< /call-out >}} - -{{}} - -{{%tab name="Default"%}} - -Deploys NGINX Gateway Fabric with NGINX OSS. - -```shell -kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/default/deploy.yaml -``` - -{{% /tab %}} - -{{%tab name="AWS NLB"%}} - -Deploys NGINX Gateway Fabric with NGINX OSS. - -```shell -kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/default/deploy.yaml -``` - -To set up an AWS Network Load Balancer service, add these annotations to your Gateway infrastructure field: - -```yaml -spec: - infrastructure: - annotations: - service.beta.kubernetes.io/aws-load-balancer-type: "external" - service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip" -``` - -{{% /tab %}} - -{{%tab name="Azure"%}} - -Deploys NGINX Gateway Fabric with NGINX OSS and `nodeSelector` to deploy on Linux nodes. - -```shell -kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/azure/deploy.yaml -``` - -{{% /tab %}} - -{{%tab name="NGINX Plus"%}} - -Deploys NGINX Gateway Fabric with NGINX Plus. The image is pulled from the -NGINX Plus Docker registry, and the `imagePullSecretName` is the name of the Secret to use to pull the image. -The NGINX Plus JWT Secret used to run NGINX Plus is also specified in a volume mount and the `--usage-report-secret` parameter. These Secrets are created as part of the [Before you begin](#before-you-begin) section. - -```shell -kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/nginx-plus/deploy.yaml -``` - -{{% /tab %}} - -{{%tab name="Experimental"%}} - -Deploys NGINX Gateway Fabric with NGINX OSS and experimental features. - -```shell -kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/experimental/deploy.yaml -``` - -{{< call-out "note" >}} Requires the Gateway APIs installed from the experimental channel. {{< /call-out >}} - -{{% /tab %}} - -{{%tab name="NGINX Plus Experimental"%}} - -Deploys NGINX Gateway Fabric with NGINX Plus and experimental features. The image is pulled from the -NGINX Plus Docker registry, and the `imagePullSecretName` is the name of the Secret to use to pull the image. -The NGINX Plus JWT Secret used to run NGINX Plus is also specified in a volume mount and the `--usage-report-secret` parameter. These Secrets are created as part of the [Before you begin](#before-you-begin) section. - -```shell -kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/nginx-plus-experimental/deploy.yaml -``` - -{{< call-out "note" >}} Requires the Gateway APIs installed from the experimental channel. {{< /call-out >}} - -{{% /tab %}} - -{{%tab name="NodePort"%}} - -Deploys NGINX Gateway Fabric with NGINX OSS using a Service type of `NodePort`. - -```shell -kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/nodeport/deploy.yaml -``` - -{{% /tab %}} - -{{%tab name="OpenShift"%}} - -Deploys NGINX Gateway Fabric with NGINX OSS on OpenShift. - -```shell -kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/openshift/deploy.yaml -``` - -{{% /tab %}} - -{{}} - -### Verify the Deployment - -To confirm that NGINX Gateway Fabric is running, check the pods in the `nginx-gateway` namespace: - -```shell -kubectl get pods -n nginx-gateway -``` - -The output should look similar to this (note that the pod name will include a unique string): - -```text -NAME READY STATUS RESTARTS AGE -nginx-gateway-5d4f4c7db7-xk2kq 1/1 Running 0 112s -``` - -### Access NGINX Gateway Fabric - -{{< include "/ngf/installation/expose-nginx-gateway-fabric.md" >}} - -## Uninstall NGINX Gateway Fabric - -Follow these steps to uninstall NGINX Gateway Fabric and Gateway API from your Kubernetes cluster: - -1. **Uninstall NGINX Gateway Fabric:** - - - To remove NGINX Gateway Fabric and its custom resource definitions (CRDs), run: - - ```shell - kubectl delete namespace nginx-gateway - kubectl delete cluterrole nginx-gateway - kubectl delete clusterrolebinding nginx-gateway - ``` - - ```shell - kubectl delete -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/crds.yaml - ``` - -1. **Remove the Gateway API resources:** - - - {{< include "/ngf/installation/uninstall-gateway-api-resources.md" >}} - -## Next steps - -- [Deploy a Gateway for data plane instances]({{< ref "/ngf/install/deploy-data-plane.md" >}}) -- [Routing traffic to applications]({{< ref "/ngf/traffic-management/basic-routing.md" >}}) \ No newline at end of file diff --git a/content/ngf/install/manifests/_index.md b/content/ngf/install/manifests/_index.md new file mode 100644 index 000000000..c3d20a585 --- /dev/null +++ b/content/ngf/install/manifests/_index.md @@ -0,0 +1,5 @@ +--- +title: "Manifests" +url: /nginx-gateway-fabric/install/manifests +weight: 200 +--- diff --git a/content/ngf/install/manifests/experimental.md b/content/ngf/install/manifests/experimental.md new file mode 100644 index 000000000..1f35b7926 --- /dev/null +++ b/content/ngf/install/manifests/experimental.md @@ -0,0 +1,68 @@ +--- +title: Use Manifests to install experimental NGINX Gateway Fabric versions +linkTitle: Experimental +weight: 300 +toc: true +nd-content-type: reference +nd-product: FABRIC +--- + +This page describes how to use Manifests to install experimental NGINX Gateway Fabric versions. + +The deployment process is similar to other installation procedures, and can be done with NGINX Open Source or NGINX Plus. + +{{< call-out "note" >}} + +Using experimental NGINX Gateway Fabric versions can allow you to test API resources from upcoming releases as outlined by the [Milestone Roadmap](https://github.com/orgs/nginx/projects/10/views/5). + +For information on API resource support, view the [Gateway API compatibility]({{< ref "/ngf/overview/gateway-api-compatibility.md" >}}) topic. + +{{< /call-out >}} + +The main requirement for installing experimental NGINX Gateway Fabric versions is to change the URL for each resource during the installation steps. + +Each of the following sections provides the necessary replacement URL and any other necessary information. + +## API resources + +The Gateway API resources from the experimental channel include everything in the standard release channel. + +To install API resources from the experimental channel, run the following command: + +```shell +kubectl kustomize "https://github.com/nginx/nginx-gateway-fabric/config/crd/gateway-api/experimental?ref=v{{< version-ngf >}}" | kubectl apply -f - +``` + +## Custom resource definitions + +The custom resource definitions (CRDs) for an experimental NGINX Gateway Fabric release are the same as a regular deployment, and do not require replacement. + +## NGINX Gateway Fabric deployment + +{{< call-out "warning" >}} + +To install an experimental NGINX Gateway Fabric version, you **must** also install [experimental API resources](#api-resources). They cannot be mis-matched. + +{{< /call-out >}} + +To deploy an experimental NGINX Gateway Fabric instance, use the command based on your NGINX type: + +{{< tabs name="nginx-version" >}} + +{{% tab name="NGINX Open Source" %}} + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/experimental/deploy.yaml +``` + +{{% /tab %}} + +{{% tab name="NGINX Plus" %}} + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/nginx-plus-experimental/deploy.yaml +``` + +{{% /tab %}} + +{{< /tabs >}} \ No newline at end of file diff --git a/content/ngf/install/manifests/open-source.md b/content/ngf/install/manifests/open-source.md new file mode 100644 index 000000000..bc69cc97d --- /dev/null +++ b/content/ngf/install/manifests/open-source.md @@ -0,0 +1,147 @@ +--- +title: Use Manifests to install NGINX Gateway Fabric with NGINX Open Source +linkTitle: NGINX Open Source +weight: 100 +toc: true +nd-content-type: how-to +nd-product: FABRIC +nd-docs: DOCS-1429 +--- + +This page describes how to use Manifests to install NGINX Gateway Fabric with NGINX Open Source. + +It explains how to install the Gateway API resources and add certificates for secure authentication, then deploy NGINX Gateway Fabric and its CRDs (Custom resource definitions). + +By following these instructions, you will finish with a functional NGINX Gateway Fabric instance for your Kubernetes cluster. + +{{< call-out "note" >}} To learn which Gateway API resources NGINX Gateway Fabric currently supports, view the [Gateway API Compatibility]({{< ref "/ngf/overview/gateway-api-compatibility.md" >}}) topic. {{< /call-out >}} + +## Before you begin + +To complete this guide, you will need the following pre-requisites: + +- [A supported Kubernetes version]({{< ref "/ngf/overview/technical-specifications.md" >}}) +- A functional Kubernetes cluster +- [cert-manager](https://cert-manager.io/docs/installation/) + +## Install the Gateway API resources + +{{< include "/ngf/installation/manifests/api-resources.md" >}} + +You should also create the _nginx-gateway_ namespace, which is used by the Manifest files by default: + +```shell +kubectl create namespace nginx-gateway +``` + +## Add certificates for secure authentication + +{{< include "/ngf/installation/manifests/secure-certificates.md" >}} + +## Deploy the NGINX Gateway Fabric CRDs + +{{< include "/ngf/installation/manifests/crds.md" >}} + +## Deploy NGINX Gateway Fabric + +By default, NGINX Gateway Fabric is installed in the **nginx-gateway** namespace. + +If you want to deploy it in another namespace, you must modify the Manifest files. + +Your next step is dependent on how you intend to expose NGINX Gateway Fabric: + +{{< tabs name="install-manifests" >}} + +{{% tab name="Default" %}} + +To deploy NGINX Gateway Fabric with NGINX Open Source, use this `kubectl` command: + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/default/deploy.yaml +``` + +{{% /tab %}} + +{{% tab name="AWS NLB" %}} + +To deploy NGINX Gateway Fabric with NGINX Open Source, use this `kubectl` command: + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/default/deploy.yaml +``` + +To set up an AWS Network Load Balancer service, add these annotations to your Gateway infrastructure field: + +```yaml +spec: + infrastructure: + annotations: + service.beta.kubernetes.io/aws-load-balancer-type: "external" + service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip" +``` + +{{% /tab %}} + +{{% tab name="Azure" %}} + +To deploy NGINX Gateway Fabric with NGINX Open Source and `nodeSelector`, use this `kubectl` command: + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/azure/deploy.yaml +``` + +{{% /tab %}} + +{{% tab name="NodePort "%}} + +To deploy NGINX Gateway Fabric with NGINX Open Source and a `NodePort` service, use this `kubectl` command: + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/nodeport/deploy.yaml +``` + +{{% /tab %}} + +{{% tab name="OpenShift "%}} + +To deploy NGINX Gateway Fabric with NGINX Open Source on OpenShift, use this `kubectl` command: + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/openshift/deploy.yaml +``` + +{{% /tab %}} + +{{< /tabs >}} + +{{< details summary="Example output" >}} + +```text +namespace/nginx-gateway configured +serviceaccount/nginx-gateway created +serviceaccount/nginx-gateway-cert-generator created +role.rbac.authorization.k8s.io/nginx-gateway-cert-generator created +clusterrole.rbac.authorization.k8s.io/nginx-gateway created +rolebinding.rbac.authorization.k8s.io/nginx-gateway-cert-generator created +clusterrolebinding.rbac.authorization.k8s.io/nginx-gateway created +service/nginx-gateway created +deployment.apps/nginx-gateway created +job.batch/nginx-gateway-cert-generator created +gatewayclass.gateway.networking.k8s.io/nginx created +nginxgateway.gateway.nginx.org/nginx-gateway-config created +nginxproxy.gateway.nginx.org/nginx-gateway-proxy-config created +``` + +{{< /details >}} + +## Verify the Deployment + +{{< include "/ngf/installation/manifests/verify-deployment.md" >}} + +## Access NGINX Gateway Fabric + +{{< include "/ngf/installation/expose-nginx-gateway-fabric.md" >}} + +## Next steps + +{{< include "/ngf/installation/next-steps.md" >}} \ No newline at end of file diff --git a/content/ngf/install/manifests/plus.md b/content/ngf/install/manifests/plus.md new file mode 100644 index 000000000..32c58f1c7 --- /dev/null +++ b/content/ngf/install/manifests/plus.md @@ -0,0 +1,128 @@ +--- +title: Use Manifests to install NGINX Gateway Fabric with NGINX Plus +linkTitle: NGINX Plus +weight: 200 +toc: true +nd-content-type: how-to +nd-product: FABRIC +--- + +This page describes how to use Manifests to install NGINX Gateway Fabric with NGINX Plus. + +It explains the requirements for NGINX Gateway Fabric, how to deploy NGINX Gateway Fabric, its custom resource definitions (CRDs) and the Gateway API resources. + +By following these instructions, you will finish with a functional NGINX Gateway Fabric instance for your Kubernetes cluster. + +## Before you begin + +To complete this guide, you will need the following pre-requisites: + +- An active NGINX Plus subscription (Purchased or trial) +- [A supported Kubernetes version]({{< ref "/ngf/overview/technical-specifications.md" >}}) +- A functional Kubernetes cluster +- [cert-manager](https://cert-manager.io/docs/installation/) + +## Download your JSON web token + +{{< include "/ngf/installation/nginx-plus/download-jwt.md" >}} + +## Create license and registry secrets + +First, create the _nginx-gateway_ namespace, which is used by the Manifest files by default: + +```shell +kubectl create namespace nginx-gateway +``` + +{{< call-out "note" >}} + +The commands in the rest of this document should be run in the same directory as your **license.jwt** file. + +JWTs are sensitive information and should be stored securely. Delete them after use to prevent unauthorized access. + +{{< /call-out >}} + +Once you have obtained your license JWT, create a Kubernetes secret using `kubectl create`: + +```shell +kubectl create -n nginx-gateway secret generic nplus-license --from-file license.jwt +``` + +Then create another Kubernetes secret to allow interactions with the F5 registry: + +```shell +kubectl create -n nginx-gateway secret docker-registry nginx-plus-registry-secret \ + --docker-server=private-registry.nginx.com \ + --docker-username=$(cat license.jwt) \ + --docker-password=none +``` + +You can verify the creation of the secrets using `kubectl get`: + +```shell +kubectl get -n nginx-gateway secrets +``` + +{{< details summary="Example output" >}} + +```text +NAME TYPE DATA AGE +nplus-license Opaque 1 31s +regcred kubernetes.io/dockerconfigjson 1 22s +``` + +{{< /details >}} + +## Install the Gateway API resources + +{{< include "/ngf/installation/manifests/api-resources" >}} + +## Add certificates for secure authentication + +{{< include "/ngf/installation/manifests/secure-certificates.md" >}} + +## Deploy the NGINX Gateway Fabric CRDs + +{{< include "/ngf/installation/manifests/crds.md" >}} + +## Deploy NGINX Gateway Fabric + +By default, NGINX Gateway Fabric is installed in the **nginx-gateway** namespace. + +If you want to deploy it in another namespace, you must modify the Manifest files + +```shell +kubectl apply -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/nginx-plus/deploy.yaml +``` + +{{< details summary="Example output" >}} + +```text +namespace/nginx-gateway configured +serviceaccount/nginx-gateway created +serviceaccount/nginx-gateway-cert-generator created +role.rbac.authorization.k8s.io/nginx-gateway-cert-generator created +clusterrole.rbac.authorization.k8s.io/nginx-gateway created +rolebinding.rbac.authorization.k8s.io/nginx-gateway-cert-generator created +clusterrolebinding.rbac.authorization.k8s.io/nginx-gateway created +service/nginx-gateway created +deployment.apps/nginx-gateway created +job.batch/nginx-gateway-cert-generator created +gatewayclass.gateway.networking.k8s.io/nginx created +nginxgateway.gateway.nginx.org/nginx-gateway-config created +nginxproxy.gateway.nginx.org/nginx-gateway-proxy-config created +``` + +{{< /details >}} + +## Verify the Deployment + +{{< include "/ngf/installation/manifests/verify-deployment.md" >}} + +## Access NGINX Gateway Fabric + +{{< include "/ngf/installation/expose-nginx-gateway-fabric.md" >}} + +## Next steps + +{{< include "/ngf/installation/next-steps.md" >}} \ No newline at end of file diff --git a/content/ngf/install/manifests/uninstall.md b/content/ngf/install/manifests/uninstall.md new file mode 100644 index 000000000..3bc9e4916 --- /dev/null +++ b/content/ngf/install/manifests/uninstall.md @@ -0,0 +1,47 @@ +--- +title: Use Manifests to uninstall NGINX Gateway Fabric +linkTitle: Uninstall +weight: 400 +toc: true +nd-content-type: how-to +nd-product: FABRIC +--- + +This page describes how to uninstall NGINX Gateway Fabric, its custom resource definitions (CRDs) and the Gateway API resources from a Kubernetes cluster. + +These instructions are for a Manifest deployment: for a Helm deployment, view the [Use Helm to uninstall NGINX Gateway Fabric]({{< ref "/ngf/install/helm.md#uninstall-nginx-gateway-fabric" >}}) topic. + +## Remove NGINX Gateway Fabric + +To remove NGINX Gateway Fabric run: + +```shell +kubectl delete namespace nginx-gateway +kubectl delete clusterrole nginx-gateway +kubectl delete clusterrolebinding nginx-gateway +``` + +## Remove the CRDs + +To remove the NGINX Gateway Fabric CRDs, run: + +```shell +kubectl delete -f https://raw.githubusercontent.com/nginx/nginx-gateway-fabric/v{{< version-ngf >}}/deploy/crds.yaml +``` + +{{< details summary="Example output" >}} + +```text +customresourcedefinition.apiextensions.k8s.io "clientsettingspolicies.gateway.nginx.org" deleted +customresourcedefinition.apiextensions.k8s.io "nginxgateways.gateway.nginx.org" deleted +customresourcedefinition.apiextensions.k8s.io "nginxproxies.gateway.nginx.org" deleted +customresourcedefinition.apiextensions.k8s.io "observabilitypolicies.gateway.nginx.org" deleted +customresourcedefinition.apiextensions.k8s.io "snippetsfilters.gateway.nginx.org" deleted +customresourcedefinition.apiextensions.k8s.io "upstreamsettingspolicies.gateway.nginx.org" deleted +``` + +{{< /details >}} + +## Remove the Gateway API resources + +{{< include "/ngf/installation/uninstall-gateway-api-resources.md" >}} \ No newline at end of file diff --git a/content/ngf/overview/custom-policies.md b/content/ngf/overview/custom-policies.md index 7d904fdaf..51ce6da7f 100644 --- a/content/ngf/overview/custom-policies.md +++ b/content/ngf/overview/custom-policies.md @@ -1,21 +1,19 @@ --- title: Custom policies -weight: 600 +weight: 700 toc: true nd-content-type: reference nd-product: FABRIC nd-docs: DOCS-1835 --- -## Overview +This topic explains the custom policies for NGINX Gateway Fabric CRDs (Custom Resource Definitions), which allow users to configure NGINX data plane features that are unavailable in the Gateway API. -Custom policies are NGINX Gateway Fabric CRDs (Custom Resource Definitions) that allow users to configure NGINX data plane features that are unavailable in the Gateway API. These custom policies follow the Gateway API [Policy Attachment](https://gateway-api.sigs.k8s.io/reference/policy-attachment/) pattern, which allows users to extend the Gateway API functionality by creating implementation-specific policies and attaching them to Kubernetes objects such as HTTPRoutes, Gateways, and Services. Policies are a Kubernetes object that augments the behavior of an object in a standard way. Policies can be attached to one object ([Direct Policy Attachment](#direct-policy-attachment)) or objects in a hierarchy ([Inherited Policy Attachment](#inherited-policy-attachment)). The following table summarizes NGINX Gateway Fabric custom policies: - | Policy | Description | Attachment Type | Supported Target Object(s) | Supports Multiple Target Refs | Mergeable | API Version | |---------------------------------------------------------------------------------------------|---------------------------------------------------------|-----------------|-------------------------------|-------------------------------|-----------|-------------| | [ClientSettingsPolicy]({{< ref "/ngf/traffic-management/client-settings.md" >}}) | Configure connection behavior between client and NGINX | Inherited | Gateway, HTTPRoute, GRPCRoute | No | Yes | v1alpha1 | @@ -90,7 +88,6 @@ spec: retries: 10 ``` - ```yaml apiVersion: gateway.nginx.org/v1alpha1 kind: ExamplePolicy diff --git a/content/ngf/overview/gateway-api-compatibility.md b/content/ngf/overview/gateway-api-compatibility.md index 79a5b503e..cbe75b629 100644 --- a/content/ngf/overview/gateway-api-compatibility.md +++ b/content/ngf/overview/gateway-api-compatibility.md @@ -1,5 +1,5 @@ --- -title: Gateway API Compatibility +title: Gateway API compatibility weight: 200 toc: true nd-content-type: reference @@ -7,24 +7,13 @@ nd-product: FABRIC nd-docs: DOCS-1412 --- -Learn which Gateway API resources NGINX Gateway Fabric supports and to which level. +This page describes which Gateway API resources NGINX Gateway Fabric supports and to which level. + +For information about NGINX Gateway Fabric and Gateway API version compatibility, view the [Technical specifications]({{< ref "/ngf/overview/technical-specifications.md" >}}) topic. ## Summary -{{< table >}} -| Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | -|---------------------------------------|---------------------|------------------------|---------------------------------------|-------------|---------------------| -| [GatewayClass](#gatewayclass) | Supported | Not supported | Supported | v1 | Standard | -| [Gateway](#gateway) | Supported | Partially supported | Not supported | v1 | Standard | -| [HTTPRoute](#httproute) | Supported | Partially supported | Not supported | v1 | Standard | -| [GRPCRoute](#grpcroute) | Supported | Partially supported | Not supported | v1 | Standard | -| [ReferenceGrant](#referencegrant) | Supported | N/A | Not supported | v1beta1 | Standard | -| [TLSRoute](#tlsroute) | Supported | Not supported | Not supported | v1alpha2 | Experimental | -| [TCPRoute](#tcproute) | Not supported | Not supported | Not supported | v1alpha2 | Experimental | -| [UDPRoute](#udproute) | Not supported | Not supported | Not supported | v1alpha2 | Experimental | -| [BackendTLSPolicy](#backendtlspolicy) | Partially Supported | Supported | Partially supported | v1alpha3 | Experimental | -| [Custom policies](#custom-policies) | N/A | N/A | Supported | N/A | N/A | -{{< /table >}} +{{< include "ngf/gateway-api-compat-table.md" >}} ## Terminology @@ -41,7 +30,6 @@ Gateway API features has three [support levels](https://gateway-api.sigs.k8s.io/ Please note that while we make every effort to reflect the support status of experimental fields in our code and documentation, there may be instances where this is not explicitly indicated. Support for such fields is provided on a best-effort basis.{{< /call-out >}} - ## Resources Each resource below includes the support status of their corresponding fields. @@ -51,9 +39,11 @@ For a description of each field, visit the [Gateway API documentation](https://g ### GatewayClass {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |--------------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | GatewayClass | Supported | Not supported | Supported | v1 | Standard | + {{< /table >}} NGINX Gateway Fabric supports a single GatewayClass resource configured with the `--gatewayclass` flag of the [controller]({{< ref "/ngf/reference/cli-help.md#controller">}}) command. @@ -77,9 +67,11 @@ NGINX Gateway Fabric supports a single GatewayClass resource configured with the ### Gateway {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | Gateway | Supported | Partially supported | Not supported | v1 | Standard | + {{< /table >}} NGINX Gateway Fabric supports multiple Gateway resources. The Gateway resources must reference NGINX Gateway Fabric's corresponding GatewayClass. @@ -145,9 +137,11 @@ See the [controller]({{< ref "/ngf/reference/cli-help.md#controller">}}) command ### HTTPRoute {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |-----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | HTTPRoute | Supported | Partially supported | Not supported | v1 | Standard | + {{< /table >}} **Fields**: @@ -200,9 +194,11 @@ See the [controller]({{< ref "/ngf/reference/cli-help.md#controller">}}) command ### GRPCRoute {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |-----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | GRPCRoute | Supported | Partially supported | Not supported | v1 | Standard | + {{< /table >}} **Fields**: @@ -247,9 +243,11 @@ See the [controller]({{< ref "/ngf/reference/cli-help.md#controller">}}) command ### ReferenceGrant {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | ReferenceGrant | Supported | N/A | Not supported | v1beta1 | Standard | + {{< /table >}} Fields: @@ -267,9 +265,11 @@ Fields: ### TLSRoute {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | TLSRoute | Supported | Not supported | Not supported | v1alpha2 | Experimental | + {{< /table >}} **Fields**: @@ -299,30 +299,34 @@ Fields: - `ResolvedRefs/False/UnsupportedValue`: Custom reason for when one of the TLSRoute rules has a backendRef with an unsupported value. - `PartiallyInvalid/True/UnsupportedValue` ---- - ### TCPRoute {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | TCPRoute | Not supported | Not supported | Not supported | v1alpha2 | Experimental | + {{< /table >}} ### UDPRoute {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |----------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | UDPRoute | Not supported | Not supported | Not supported | v1alpha2 | Experimental | + {{< /table >}} ### BackendTLSPolicy {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |------------------|---------------------|------------------------|---------------------------------------|-------------|---------------------| | BackendTLSPolicy | Partially Supported | Supported | Partially Supported | v1alpha3 | Experimental | + {{< /table >}} Fields: @@ -354,9 +358,11 @@ Fields: ### Custom Policies {{< table >}} + | Resource | Core Support Level | Extended Support Level | Implementation-Specific Support Level | API Version | API Release Channel | |-----------------|--------------------|------------------------|---------------------------------------|-------------|---------------------| | Custom policies | N/A | N/A | Supported | N/A | N/A | + {{< /table >}} Custom policies are NGINX Gateway Fabric-specific CRDs (Custom Resource Definitions) that support features such as tracing, and client connection settings. These important data-plane features are not part of the Gateway API specifications. diff --git a/content/ngf/overview/product-telemetry.md b/content/ngf/overview/product-telemetry.md index 714d83ea2..594f9d06e 100644 --- a/content/ngf/overview/product-telemetry.md +++ b/content/ngf/overview/product-telemetry.md @@ -1,6 +1,6 @@ --- title: Product telemetry -weight: 500 +weight: 600 nd-content-type: reference nd-product: FABRIC nd-docs: DOCS-1836 diff --git a/content/ngf/overview/resource-validation.md b/content/ngf/overview/resource-validation.md index f4200948e..75bdd5d26 100644 --- a/content/ngf/overview/resource-validation.md +++ b/content/ngf/overview/resource-validation.md @@ -1,6 +1,6 @@ --- title: Resource validation -weight: 400 +weight: 500 toc: true nd-content-type: reference nd-product: FABRIC @@ -63,6 +63,7 @@ Below is an example of how NGINX Gateway Fabric rejects an invalid resource. The ```shell kubectl describe httproutes.gateway.networking.k8s.io coffee ``` + ```text . . . Status: diff --git a/content/ngf/overview/technical-specifications.md b/content/ngf/overview/technical-specifications.md new file mode 100644 index 000000000..51ddee62d --- /dev/null +++ b/content/ngf/overview/technical-specifications.md @@ -0,0 +1,46 @@ +--- +title: Technical specifications +weight: 300 +nd-content-type: reference +nd-product: FABRIC +--- + +This page describes the technical specifications for NGINX Gateway Fabric. + +The information included covers version compatibility between NGINX Gateway Fabric and the Gateway API, as well as other NGINX products. + +## NGINX Gateway Fabric versions + +The following table lists the software versions NGINX Gateway Fabric supports. + +| NGINX Gateway Fabric | Gateway API | Kubernetes | NGINX OSS | NGINX Plus | NGINX Agent | +|----------------------|-------------|------------|-----------|------------|-------------| +| Edge | 1.4.0 | 1.25+ | 1.29.3 | R35 | v3.5.0 | +| 2.2.1 | 1.3.0 | 1.25+ | 1.29.2 | R35 | v3.5.0 | +| 2.2.0 | 1.3.0 | 1.25+ | 1.29.2 | R35 | v3.3.2 | +| 2.1.4 | 1.3.0 | 1.25+ | 1.29.1 | R35 | v3.3.1 | +| 2.1.3 | 1.3.0 | 1.25+ | 1.29.1 | R35 | v3.3.1 | +| 2.1.2 | 1.3.0 | 1.25+ | 1.29.1 | R35 | v3.3.1 | +| 2.1.1 | 1.3.0 | 1.25+ | 1.29.1 | R35 | v3.2.1 | +| 2.1.0 | 1.3.0 | 1.25+ | 1.29.1 | R35 | v3.2.1 | +| 2.0.2 | 1.3.0 | 1.25+ | 1.28.0 | R34 | v3.0.1 | +| 2.0.1 | 1.3.0 | 1.25+ | 1.28.0 | R34 | v3.0.1 | +| 2.0.0 | 1.3.0 | 1.25+ | 1.28.0 | R34 | v3.0.0 | +| 1.6.2 | 1.2.1 | 1.25+ | 1.27.4 | R33 | --- | +| 1.6.1 | 1.2.1 | 1.25+ | 1.27.4 | R33 | --- | +| 1.6.0 | 1.2.1 | 1.25+ | 1.27.3 | R33 | --- | +| 1.5.1 | 1.2.0 | 1.25+ | 1.27.2 | R33 | --- | +| 1.5.0 | 1.2.0 | 1.25+ | 1.27.2 | R33 | --- | +| 1.4.0 | 1.1.0 | 1.25+ | 1.27.1 | R32 | --- | +| 1.3.0 | 1.1.0 | 1.25+ | 1.27.0 | R32 | --- | +| 1.2.0 | 1.0.0 | 1.23+ | 1.25.4 | R31 | --- | +| 1.1.0 | 1.0.0 | 1.23+ | 1.25.3 | n/a | --- | +| 1.0.0 | 0.8.1 | 1.23+ | 1.25.2 | n/a | --- | + +## Gateway API compatibility + +The following tables summarizes which Gateway API resources NGINX Gateway Fabric supports and to which level. + +You can read more information by viewing the [Gateway API compatibility]({{< ref "/ngf/overview/gateway-api-compatibility.md" >}}) topic, or by selecting the resource name to go directly to the full details. + +{{< include "ngf/gateway-api-compat-table.md" >}} \ No newline at end of file diff --git a/content/nic/configuration/virtualserver-and-virtualserverroute-resources.md b/content/nic/configuration/virtualserver-and-virtualserverroute-resources.md index f5009bc7f..d7dc3088b 100644 --- a/content/nic/configuration/virtualserver-and-virtualserverroute-resources.md +++ b/content/nic/configuration/virtualserver-and-virtualserverroute-resources.md @@ -126,6 +126,7 @@ cert-manager: |``issue-temp-cert`` | When ``true``, ask cert-manager for a [temporary self-signed certificate](https://cert-manager.io/docs/usage/certificate/#temporary-certificates-while-issuing) pending the issuance of the Certificate. This allows HTTPS-only servers to use ACME HTTP01 challenges when the TLS secret does not exist yet. | ``boolean`` | No | ### VirtualServer.Listener + The listener field defines a custom HTTP and/or HTTPS listener. The respective listeners used must reference the name of a listener defined using a [GlobalConfiguration]({{< ref "/nic/configuration/global-configuration/globalconfiguration-resource.md" >}}) resource. For example: @@ -196,6 +197,7 @@ The route defines rules for matching client requests to actions like passing a r ``` {{< table >}} + |Field | Description | Type | Required | | ---| ---| ---| --- | |``path`` | The path of the route. NGINX will match it against the URI of a request. Possible values are: a prefix ( ``/`` , ``/path`` ), an exact match ( ``=/exact/match`` ), a case insensitive regular expression ( ``~*^/Bar.*\.jpg`` ) or a case sensitive regular expression ( ``~^/foo.*\.jpg`` ). In the case of a prefix (must start with ``/`` ) or an exact match (must start with ``=`` ), the path must not include any whitespace characters, ``{`` , ``}`` or ``;``. In the case of the regex matches, all double quotes ``"`` must be escaped and the match can't end in an unescaped backslash ``\``. The path must be unique among the paths of all routes of the VirtualServer. Check the [location](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) directive for more information. | ``string`` | Yes | @@ -207,6 +209,7 @@ The route defines rules for matching client requests to actions like passing a r |``route`` | The name of a VirtualServerRoute resource that defines this route. If the VirtualServerRoute belongs to a different namespace than the VirtualServer, you need to include the namespace. For example, ``tea-namespace/tea``. | ``string`` | No | |``errorPages`` | The custom responses for error codes. NGINX will use those responses instead of returning the error responses from the upstream servers or the default responses generated by NGINX. A custom response can be a redirect or a canned response. For example, a redirect to another URL if an upstream server responded with a 404 status code. | [[]errorPage](#errorpage) | No | |``location-snippets`` | Sets a custom snippet in the location context. Overrides the ``location-snippets`` ConfigMap key. | ``string`` | No | + {{< /table >}} \* -- a route must include exactly one of the following: `action`, `splits`, or `route`. @@ -285,6 +288,7 @@ action: ``` {{< table >}} + |Field | Description | Type | Required | | ---| ---| ---| --- | |``path`` | The path of the subroute. NGINX will match it against the URI of a request. Possible values are: a prefix ( ``/`` , ``/path`` ), an exact match ( ``=/exact/match`` ), a case insensitive regular expression ( ``~*^/Bar.*\.jpg`` ) or a case sensitive regular expression ( ``~^/foo.*\.jpg`` ). In the case of a prefix, the path must start with the same path as the path of the route of the VirtualServer that references this resource. In the case of an exact or regex match, the path must be the same as the path of the route of the VirtualServer that references this resource. A matching path of the route of the VirtualServer but in different type is not accepted, e.g. a regex path (`~/match`) cannot be used with a prefix path in VirtualServer (`/match`) In the case of a prefix or an exact match, the path must not include any whitespace characters, ``{`` , ``}`` or ``;``. In the case of the regex matches, all double quotes ``"`` must be escaped and the match can't end in an unescaped backslash ``\``. The path must be unique among the paths of all subroutes of the VirtualServerRoute. | ``string`` | Yes | @@ -295,6 +299,7 @@ action: |``matches`` | The matching rules for advanced content-based routing. Requires the default ``action`` or ``splits``. Unmatched requests will be handled by the default ``action`` or ``splits``. | [matches](#match) | No | |``errorPages`` | The custom responses for error codes. NGINX will use those responses instead of returning the error responses from the upstream servers or the default responses generated by NGINX. A custom response can be a redirect or a canned response. For example, a redirect to another URL if an upstream server responded with a 404 status code. | [[]errorPage](#errorpage) | No | |``location-snippets`` | Sets a custom snippet in the location context. Overrides the ``location-snippets`` of the VirtualServer (if set) or the ``location-snippets`` ConfigMap key. | ``string`` | No | + {{< /table >}} \* -- a subroute must include exactly one of the following: `action` or `splits`. @@ -330,6 +335,7 @@ tls: **Note**: The WebSocket protocol is supported without any additional configuration. {{< table >}} + |Field | Description | Type | Required | | ---| ---| ---| --- | |``name`` | The name of the upstream. Must be a valid DNS label as defined in RFC 1035. For example, ``hello`` and ``upstream-123`` are valid. The name must be unique among all upstreams of the resource. | ``string`` | Yes | @@ -362,6 +368,7 @@ tls: |``type`` |The type of the upstream. Supported values are ``http`` and ``grpc``. The default is ``http``. For gRPC, it is necessary to enable HTTP/2 in the [ConfigMap]({{< ref "/nic/configuration/global-configuration/configmap-resource.md#listeners" >}}) and configure TLS termination in the VirtualServer. | ``string`` | No | |``backup`` | The name of the backup service of type [ExternalName](https://kubernetes.io/docs/concepts/services-networking/service/#externalname). This will be used when the primary servers are unavailable. Note: The parameter cannot be used along with the ``random`` , ``hash`` or ``ip_hash`` load balancing methods. | ``string`` | No | |``backupPort`` | The port of the backup service. The backup port is required if the backup service name is provided. The port must fall into the range ``1..65535``. | ``uint16`` | No | + {{< /table >}} ### Upstream.Buffers @@ -870,6 +877,7 @@ For example, the following command creates a VirtualServer resource defined in ` ```shell kubectl apply -f cafe-virtual-server.yaml ``` + ```text virtualserver.k8s.nginx.org "cafe" created ``` @@ -879,6 +887,7 @@ You can get the resource by running: ```shell kubectl get virtualserver cafe ``` + ```text NAME STATE HOST IP PORTS AGE cafe Valid cafe.example.com 12.13.23.123 [80,443] 3m @@ -946,6 +955,7 @@ If you try to create (or update) a resource that violates the structural schema ```shell kubectl apply -f cafe-virtual-server.yaml ``` + ```text error: error validating "cafe-virtual-server.yaml": error validating data: ValidationError(VirtualServer.spec.upstreams[0].port): invalid type for org.nginx.k8s.v1.VirtualServer.spec.upstreams.port: got "string", expected "integer"; if you choose to ignore these errors, turn validation off with --validate=false ``` @@ -955,6 +965,7 @@ If you try to create (or update) a resource that violates the structural schema ```shell kubectl apply -f cafe-virtual-server.yaml --validate=false ``` + ```text The VirtualServer "cafe" is invalid: []: Invalid value: map[string]interface {}{ ... }: validation failure list: spec.upstreams.port in body must be of type integer: "string" @@ -971,6 +982,7 @@ You can check if NGINX Ingress Controller successfully applied the configuration ```shell kubectl describe vs cafe ``` + ```text ... Events: @@ -986,6 +998,7 @@ If you create an invalid resource, NGINX Ingress Controller will reject it and e ```shell kubectl describe vs cafe ``` + ```text ... Events: @@ -1001,6 +1014,7 @@ Additionally, this information is also available in the `status` field of the Vi ```shell kubectl describe vs cafe ``` + ```text ... Status: diff --git a/content/nic/install/helm/plus.md b/content/nic/install/helm/plus.md index aec295cde..0e393b548 100644 --- a/content/nic/install/helm/plus.md +++ b/content/nic/install/helm/plus.md @@ -91,6 +91,8 @@ helm install oci://ghcr.io/nginx/charts/nginx-ingress \ The commands in the rest of this document should be run in the same directory as your **license.jwt** file. +JWTs are sensitive information and should be stored securely. Delete them after use to prevent unauthorized access. + {{< /call-out >}} Once you have obtained your license JWT, create a Kubernetes secret using `kubectl create`: diff --git a/content/nic/install/helm/uninstall.md b/content/nic/install/helm/uninstall.md index 4fb883e02..cb2f06a2c 100644 --- a/content/nic/install/helm/uninstall.md +++ b/content/nic/install/helm/uninstall.md @@ -78,28 +78,4 @@ The [Run multiple NGINX Ingress Controllers]({{< ref "/nic/install/multiple-cont ## Remove secrets -If your deployment used NGINX Plus, you should also remove the secrets created for your license and the F5 registry. - -```shell -kubectl delete secret nplus-license -``` - -{{< details summary="Example output" >}} - -```text -secret "nplus-license" deleted -``` - -{{< /details >}} - -```shell -kubectl delete secret regcred -``` - -{{< details summary="Example output" >}} - -```text -secret "regcred" deleted -``` - -{{< /details >}} \ No newline at end of file +{{< include "/k8s/delete-license-registry-secrets.md" >}} \ No newline at end of file