Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 112 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Enable Kubernetes `cert-manager` to sync TLS certificates to AWS ACM, GCP, Hashi
- [Google Cloud](#google-cloud)
- [HashiCorp Vault](#hashicorp-vault)
- [Heroku](#heroku)
- [Hetzner Cloud](#hetzner-cloud)
- [Incapsula](#incapsula)
- [ThreatX](#threatx)
- [Multiple Sync Destinations](#multiple-sync-destinations)
Expand All @@ -26,6 +27,11 @@ Enable Kubernetes `cert-manager` to sync TLS certificates to AWS ACM, GCP, Hashi
- [Monitoring](#monitoring)
- [Prometheus Metrics](#prometheus-metrics)
- [Error Logging](#error-logging)
- [PKCS#12 Support for HashiCorp Vault](#pkcs12-support-for-hashicorp-vault)
- [Configuration](#configuration-1)
- [Password Management](#password-management)
- [Storage in Vault](#storage-in-vault)
- [Example](#example)

## Architecture

Expand Down Expand Up @@ -201,6 +207,108 @@ Annotations:
cert-manager-sync.lestak.sh/heroku-cert-name: "" # will be auto-filled by operator for in-place renewals
```

### Hetzner Cloud

Create a Hetzner Cloud API Token with `Read & Write` permissions for Certificates. The token can be created in the [Hetzner Cloud Console](https://console.hetzner.cloud) under Security → API Tokens.

**Project Scope**: Hetzner Cloud organizes resources into projects, and API tokens are scoped to a specific project. When you use cert-manager-sync with a Hetzner Cloud API token:
- Certificates will be uploaded to the **same project** that issued the API token
- You cannot cross project boundaries with a single token
- Ensure you're using an API token from the correct project where you want the certificates to be available for your Load Balancers

```bash
kubectl -n cert-manager \
create secret generic example-hetzner-secret \
--from-literal api_token=XXXXX
```

You will then annotate your k8s TLS secret with this secret name to tell the operator to retrieve the Hetzner Cloud API secret from this location.

Annotations:

```yaml
cert-manager-sync.lestak.sh/hetznercloud-enabled: "true" # sync certificate to Hetzner Cloud
cert-manager-sync.lestak.sh/hetznercloud-secret-name: "example-hetzner-secret" # secret in same namespace which contains the hetzner cloud api token. If provided in format "namespace/secret-name", will look in that namespace for the secret
cert-manager-sync.lestak.sh/hetznercloud-cert-name: "my-cert" # unique name to give your cert in Hetzner Cloud (optional, defaults to secret name)
cert-manager-sync.lestak.sh/hetznercloud-cert-id: "" # will be auto-filled by operator for in-place renewals
cert-manager-sync.lestak.sh/hetznercloud-label-environment: "production" # (optional) add labels to the certificate in Hetzner Cloud
cert-manager-sync.lestak.sh/hetznercloud-label-team: "devops" # (optional) add more labels as needed
```

**Certificate Updates and Load Balancer Integration:**

When cert-manager-sync updates a certificate in Hetzner Cloud:

1. **Certificate NOT in use**: The old certificate is deleted and a new one is created with the same name. No action required.

2. **Certificate in use by Load Balancer**: Since Hetzner Cloud prevents deletion of certificates that are in use:
- A new certificate is created with a modified name: `original-name-{old-cert-id}`
- The old certificate remains attached to the Load Balancer
- **Manual action required**: Update your Load Balancer to use the new certificate

**Important for Hetzner Cloud Controller Manager users:**

The Hetzner Cloud Controller Manager (for Kubernetes LoadBalancer services) identifies certificates by their exact name or ID specified in the annotation:
```yaml
load-balancer.hetzner.cloud/http-certificates: "my-cert-name"
```

When cert-manager-sync creates a new certificate with a modified name (due to the old one being in use), you must:
1. Update the annotation to the new certificate name
2. Apply the service changes to trigger the Load Balancer update

**Example workflow for certificate renewal when in use:**
1. cert-manager renews the certificate
2. cert-manager-sync attempts to update in Hetzner Cloud
3. If the old certificate is in use, a new one is created as `my-cert-name-12345`
4. Update your service annotation: `load-balancer.hetzner.cloud/http-certificates: "my-cert-name-12345"`
5. Apply the service to update the Load Balancer
6. Once updated, the old certificate can be manually deleted from Hetzner Cloud Console

**Labels Support:**
Certificates can be labeled for organization and tracking purposes. Use annotations like:
- `cert-manager-sync.lestak.sh/hetznercloud-label-environment: "production"`
- `cert-manager-sync.lestak.sh/hetznercloud-label-managed-by: "cert-manager-sync"`

**Automated Certificate Rotation Monitoring:**

For production environments, you can automate the certificate rotation process using monitoring:

1. **Use Prometheus Blackbox Exporter** to monitor SSL certificate expiry:
- The `probe_ssl_earliest_cert_expiry` metric provides Unix timestamp of certificate expiration
- Calculate days until expiry: `(probe_ssl_earliest_cert_expiry - time()) / 86400`

2. **Configure cert-manager to renew certificates early** (before your monitoring alerts):
```yaml
# In your Certificate resource
spec:
renewBefore: 720h # 30 days - renew well before the 7-day alert
```

3. **Set up alerts** for certificates expiring soon (example for 7 days):
```yaml
- alert: HetznerCertificateExpiringSoon
expr: (probe_ssl_earliest_cert_expiry{job="blackbox"} - time()) < 86400 * 7
annotations:
summary: "Certificate expiring soon for {{ $labels.instance }}"
description: "Update load-balancer.hetzner.cloud/http-certificates annotation"
```

4. **Automate the update process** (via CI/CD or operators):
- When alert triggers, the new certificate should already exist in Hetzner Cloud (thanks to early renewal)
- Update the Service annotation to the new certificate name: `original-name-{old-cert-id}`
- The Hetzner Cloud Controller Manager will switch to the new certificate on next reconciliation
- Old certificates can be manually deleted after the switch is confirmed

Hetzner Cloud store supports optional integration testing with a real API. To run:

```bash
export HETZNER_TEST_TOKEN="your-hetzner-api-token"
go test ./stores/hetznercloud/... -v
```

**Note**: Use a test project token to avoid affecting production resources. The test will create and clean up test certificates in the project associated with the API token.

### Incapsula

Create an Incapsula API Key and create a kube secret containing this key.
Expand Down Expand Up @@ -342,6 +450,10 @@ metadata:
cert-manager-sync.lestak.sh/heroku-app: "example-app" # heroku app to attach cert
cert-manager-sync.lestak.sh/heroku-secret-name: "example-heroku-secret" # secret in same namespace which contains heroku api key
cert-manager-sync.lestak.sh/heroku-cert-name: "" # will be auto-filled by operator for in-place renewals
cert-manager-sync.lestak.sh/hetznercloud-enabled: "true" # sync certificate to Hetzner Cloud
cert-manager-sync.lestak.sh/hetznercloud-secret-name: "example-hetzner-secret" # secret in same namespace which contains hetzner cloud api token
cert-manager-sync.lestak.sh/hetznercloud-cert-name: "my-cert" # unique name to give your cert in Hetzner Cloud
cert-manager-sync.lestak.sh/hetznercloud-cert-id: "" # will be auto-filled by operator for in-place renewals
cert-manager-sync.lestak.sh/incapsula-site-id: "12345" # incapsula site to attach cert
cert-manager-sync.lestak.sh/incapsula-secret-name: "cert-manager-sync-poc" # secret in same namespace which contains incapsula api key
cert-manager-sync.lestak.sh/threatx-hostname: "example.com" # threatx hostname to attach cert
Expand Down
15 changes: 8 additions & 7 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/robertlestak/cert-manager-sync

go 1.24.3
go 1.24.5

require (
cloud.google.com/go/certificatemanager v1.9.5
Expand All @@ -10,6 +10,7 @@ require (
github.com/google/uuid v1.6.0
github.com/hashicorp/vault/api v1.20.0
github.com/heroku/heroku-go/v5 v5.5.0
github.com/hetznercloud/hcloud-go/v2 v2.22.0
github.com/prometheus/client_golang v1.22.0
github.com/sirupsen/logrus v1.9.3
github.com/stretchr/testify v1.10.0
Expand Down Expand Up @@ -83,12 +84,12 @@ require (
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
golang.org/x/crypto v0.38.0 // indirect
golang.org/x/net v0.40.0 // indirect
golang.org/x/sync v0.14.0 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/term v0.32.0 // indirect
golang.org/x/text v0.25.0 // indirect
golang.org/x/crypto v0.40.0 // indirect
golang.org/x/net v0.42.0 // indirect
golang.org/x/sync v0.16.0 // indirect
golang.org/x/sys v0.34.0 // indirect
golang.org/x/term v0.33.0 // indirect
golang.org/x/text v0.27.0 // indirect
golang.org/x/time v0.11.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250528174236-200df99c418a // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250528174236-200df99c418a // indirect
Expand Down
30 changes: 16 additions & 14 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,8 @@ github.com/hashicorp/vault/api v1.20.0 h1:KQMHElgudOsr+IbJgmbjHnCTxEpKs9LnozA1D3
github.com/hashicorp/vault/api v1.20.0/go.mod h1:GZ4pcjfzoOWpkJ3ijHNpEoAxKEsBJnVljyTe3jM2Sms=
github.com/heroku/heroku-go/v5 v5.5.0 h1:+pKHpiPskqkkarrPHF7RpeUveXl+mAsKLAEI/ZIY9uA=
github.com/heroku/heroku-go/v5 v5.5.0/go.mod h1:Uo3XhGPwaTpniR4X1e50BDjg4SzdFk2Bd2mgYZVkfHo=
github.com/hetznercloud/hcloud-go/v2 v2.22.0 h1:RwcOkgB5y7kvi9Nxt40lHej8HjaS/P+9Yjfs4Glcds0=
github.com/hetznercloud/hcloud-go/v2 v2.22.0/go.mod h1:t14Logj+iLXyS03DGwEyrN+y7/C9243CJt3IArTHbyM=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
Expand Down Expand Up @@ -220,44 +222,44 @@ go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8=
golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw=
golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.40.0 h1:79Xs7wF06Gbdcg4kdCCIQArK11Z1hr5POQ6+fIYHNuY=
golang.org/x/net v0.40.0/go.mod h1:y0hY0exeL2Pku80/zKK7tpntoX23cqL3Oa6njdgRtds=
golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ=
golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.32.0 h1:DR4lr0TjUs3epypdhTOkMmuF5CDFJ/8pOnbzMZPQ7bg=
golang.org/x/term v0.32.0/go.mod h1:uZG1FhGx848Sqfsq4/DlJr3xGGsYMu/L5GW4abiaEPQ=
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
golang.org/x/time v0.11.0 h1:/bpjEDfN9tkoN/ryeYHnv5hcMlc8ncjMcM4XBk5NWV0=
golang.org/x/time v0.11.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.26.0 h1:v/60pFQmzmT9ExmjDv2gGIfi3OqfKoEP6I5+umXlbnQ=
golang.org/x/tools v0.26.0/go.mod h1:TPVVj70c7JJ3WCazhD8OdXcZg/og+b9+tH/KxylGwH0=
golang.org/x/tools v0.34.0 h1:qIpSLOxeCYGg9TrcJokLBG4KFA6d795g0xkBkiESGlo=
golang.org/x/tools v0.34.0/go.mod h1:pAP9OwEaY1CAW3HOmg3hLZC5Z0CCmzjAF2UQMSqNARg=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
Expand Down
2 changes: 2 additions & 0 deletions internal/types/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ const (
FilepathStoreType StoreType = "filepath"
GCPStoreType StoreType = "gcp"
HerokuStoreType StoreType = "heroku"
HetznerCloudStoreType StoreType = "hetznercloud"
IncapsulaStoreType StoreType = "incapsula"
ThreatxStoreType StoreType = "threatx"
VaultStoreType StoreType = "vault"
Expand All @@ -27,6 +28,7 @@ var EnabledStores = []StoreType{
FilepathStoreType,
GCPStoreType,
HerokuStoreType,
HetznerCloudStoreType,
IncapsulaStoreType,
ThreatxStoreType,
VaultStoreType,
Expand Down
3 changes: 3 additions & 0 deletions pkg/certmanagersync/certmanagersync.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
"github.com/robertlestak/cert-manager-sync/stores/filepath"
"github.com/robertlestak/cert-manager-sync/stores/gcpcm"
"github.com/robertlestak/cert-manager-sync/stores/heroku"
"github.com/robertlestak/cert-manager-sync/stores/hetznercloud"
"github.com/robertlestak/cert-manager-sync/stores/incapsula"
"github.com/robertlestak/cert-manager-sync/stores/threatx"
"github.com/robertlestak/cert-manager-sync/stores/vault"
Expand Down Expand Up @@ -50,6 +51,8 @@ func NewStore(storeType cmtypes.StoreType) (RemoteStore, error) {
store = &gcpcm.GCPStore{}
case cmtypes.HerokuStoreType:
store = &heroku.HerokuStore{}
case cmtypes.HetznerCloudStoreType:
store = &hetznercloud.HetznerCloudStore{}
case cmtypes.IncapsulaStoreType:
store = &incapsula.IncapsulaStore{}
case cmtypes.ThreatxStoreType:
Expand Down
Loading