feat: Support for fault-tolerant execution

[dervoeti]

Description

Implementation for #576

When providing credentials for S3, Azure or GCS we currently need a SecretClass. It would be easier if the user could just provide a Secret as an alternative, we had that discussion recently. To be consistent with the current state, I used SecretClass exclusively for now.

Things I did not do in this PR to ease the review process (we could tackle those as a separate PR):

• Combine redundant logic from FTE and Catalog modules
• Use config-utils instead of doing the load_env_from_files stunt

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

• Changes are OpenShift compatible
• CRD changes approved
• CRD documentation for all fields, following the style guide.
• Helm chart can be installed and deployed operator works
• Integration tests passed (for non trivial changes)
• Changes need to be "offline" compatible
• Links to generated (nightly) docs added
• Release note snippet added

Reviewer

• Code contains useful comments
• Code contains useful logging statements
• (Integration-)Test cases added
• Documentation added or updated. Follows the style guide.
• Changelog updated
• Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

• Feature Tracker has been updated
• Proper release label has been added
• Links to generated (nightly) docs added
• Release note snippet added
• Add type/deprecation label & add to the deprecation schedule
• Add type/experimental label & add to the experimental features tracker

[sbernauer]

I dropped some comments on the CRD code and YAML.
As an possible alternative I can propose a complex enum style, where we can together think about it, just pasting it here for reference for now

spec:
  clusterConfig:
    faultTolerantExecution:
      query:
        retryAttempts: 4 # maps to query-retry-attempts
        retryInitialDelay: 10s
        retryMaxDelay: 60s
        retryDelayScaleFactor: 2.0 # f32
        exchangeManager: # Optional
          deduplicationBufferSize: 64Mi # Quantity
          encryptionEnabled: true
          sinkBufferPoolMinSize: 20
          sinkBuffersPerPartition: 4
          sinkMaxFileSize: 2Gi # Quantity
          sourceConcurrentReaders: 8
          s3:
            baseDirectories:
              - s3://trino-exchange-bucket/spooling
            connection: # Mandatory
              reference: minio-connection
            maxErrorRetries: 10
            uploadPartSize: 10Mi # Quantity
      # OR
      task:
        retryAttemptsPerTask: 4 # maps to task-retry-attempts-per-task
        retryInitialDelay: 10s
        retryMaxDelay: 60s
        retryDelayScaleFactor: 2.0 # f32
        exchangeManager: # Mandatory
          # ... same struct as above

[maltesander]

LGTM!

[dervoeti]

Release note

SDP now supports configuring fault-tolerant execution for Trino via the TrinoCluster CRD

[lfrancke]

Are there any docs for this we can link to?

[sbernauer]

Yes, https://docs.stackable.tech/home/nightly/trino/usage-guide/fault-tolerant-execution/