Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ include::_attributes/common-attributes.adoc[]
toc::[]

[role="_abstract"]
You can override the default values of Kopia hashing, encryption, and splitter algorithms by using specific environment variables in the Data Protection Application (DPA).
Override the default values of Kopia hashing, encryption, and splitter algorithms by using specific environment variables in the Data Protection Application (DPA).

// configuring the DPA with Kopia environment variables
include::modules/oadp-kopia-configuring-algorithms.adoc[leveloffset=+1]
Expand Down
112 changes: 42 additions & 70 deletions modules/oadp-kopia-algorithms-benchmarking.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,24 @@
= Benchmarking Kopia hashing, encryption, and splitter algorithms

[role="_abstract"]
You can run Kopia commands to benchmark the hashing, encryption, and splitter algorithms. Based on the benchmarking results, you can select the most suitable algorithm for your workload. In this procedure, you run the Kopia benchmarking commands from a pod on the cluster. The benchmarking results can vary depending on CPU speed, available RAM, disk speed, current I/O load, and so on.
Run Kopia commands to benchmark the hashing, encryption, and splitter algorithms. Based on the benchmarking results, you can select the most suitable algorithm for your workload. You run the Kopia benchmarking commands from a pod on the cluster. The benchmarking results can vary depending on CPU speed, available RAM, disk speed, current I/O load, and so on.

include::snippets/kopia-repo-config.adoc[]

.Prerequisites

* You have installed the {oadp-short} Operator.
* You have an application with persistent volumes running in a separate namespace.
* You have run a backup of the application with Container Storage Interface (CSI) snapshots.

include::snippets/kopia-repo-config.adoc[]


.Procedure

. Configure the `must-gather` pod as shown in the following example. Make sure you are using the `oadp-mustgather` image for {oadp-short} version 1.3 and later.
+
.Example pod configuration
+

[source,yaml]
----
apiVersion: v1
Expand All @@ -39,19 +41,18 @@ spec:
args: ["infinity"]
----
+
[NOTE]
=====
The Kopia client is available in the `oadp-mustgather` image.
=====
+


. Create the pod by running the following command:
+
[source,terminal]
----
$ oc apply -f <pod_config_file_name> # <1>
$ oc apply -f <pod_config_file_name>
----
<1> Specify the name of the YAML file for the pod configuration.
+
Replace `<pod_config_file_name>` with the name of the YAML file for the pod configuration.


. Verify that the Security Context Constraints (SCC) on the pod is `anyuid`, so that Kopia can connect to the repository.
+
[source,terminal]
Expand All @@ -60,43 +61,46 @@ $ oc describe pod/oadp-mustgather-pod | grep scc
----
+
.Example output
+

[source,terminal]
----
openshift.io/scc: anyuid
----
+

. Connect to the pod via SSH by running the following command:
+
[source,terminal]
----
$ oc -n openshift-adp rsh pod/oadp-mustgather-pod
----
+

. Connect to the Kopia repository by running the following command:
+
[source,terminal]
----
sh-5.1# kopia repository connect s3 \
--bucket=<bucket_name> \ # <1>
--prefix=velero/kopia/<application_namespace> \ # <2>
--password=static-passw0rd \ # <3>
--access-key="<access_key>" \ # <4>
--secret-access-key="<secret_access_key>" \ # <5>
--endpoint=<bucket_endpoint> \ # <6>
----
<1> Specify the object storage provider bucket name.
<2> Specify the namespace for the application.
<3> This is the Kopia password to connect to the repository.
<4> Specify the object storage provider access key.
<5> Specify the object storage provider secret access key.
<6> Specify the bucket endpoint. You do not need to specify the bucket endpoint, if you are using {aws-short} S3 as the storage provider.
--bucket=<bucket_name> \
--prefix=velero/kopia/<application_namespace> \
--password=static-passw0rd \
--access-key="<access_key>" \
--secret-access-key="<secret_access_key>" \
--endpoint=<bucket_endpoint>
----
+
where:
+
`<bucket_name>`:: Specifies the object storage provider bucket name.
`<application_namespace>`:: Specifies the namespace for the application.
`static-passw0rd`:: This is the Kopia password to connect to the repository.
`<access_key>`:: Specifies the object storage provider access key.
`<secret_access_key>`:: Specifies the object storage provider secret access key.
`<bucket_endpoint>`:: Specifies the bucket endpoint. You do not need to specify the bucket endpoint, if you are using {aws-short} S3 as the storage provider.

+
[NOTE]
====

This is an example command. The command can vary based on the object storage provider.
====
+


. To benchmark the hashing algorithm, run the following command:
+
[source,terminal]
Expand All @@ -109,32 +113,11 @@ sh-5.1# kopia benchmark hashing
----
Benchmarking hash 'BLAKE2B-256' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE2B-256-128' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE2S-128' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE2S-256' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE3-256' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE3-256-128' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA224' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA256' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA256-128' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA3-224' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA3-256' (100 x 1048576 bytes, parallelism 1)
Hash Throughput
-----------------------------------------------------------------
0. BLAKE3-256 15.3 GB / second
1. BLAKE3-256-128 15.2 GB / second
2. HMAC-SHA256-128 6.4 GB / second
3. HMAC-SHA256 6.4 GB / second
4. HMAC-SHA224 6.4 GB / second
5. BLAKE2B-256-128 4.2 GB / second
6. BLAKE2B-256 4.1 GB / second
7. BLAKE2S-256 2.9 GB / second
8. BLAKE2S-128 2.9 GB / second
9. HMAC-SHA3-224 1.6 GB / second
10. HMAC-SHA3-256 1.5 GB / second
-----------------------------------------------------------------


Fastest option for this machine is: --block-hash=BLAKE3-256
----
+

. To benchmark the encryption algorithm, run the following command:
+
[source,terminal]
Expand All @@ -145,16 +128,12 @@ sh-5.1# kopia benchmark encryption
.Example output
[source,terminal]
----
Benchmarking encryption 'AES256-GCM-HMAC-SHA256'... (1000 x 1048576 bytes, parallelism 1)
Benchmarking encryption 'CHACHA20-POLY1305-HMAC-SHA256'... (1000 x 1048576 bytes, parallelism 1)
Encryption Throughput
-----------------------------------------------------------------
0. AES256-GCM-HMAC-SHA256 2.2 GB / second
1. CHACHA20-POLY1305-HMAC-SHA256 1.8 GB / second
-----------------------------------------------------------------
Benchmarking encryption 'AES256-GCM-HMAC-SHA256'
Benchmarking encryption 'CHACHA20-POLY1305-HMAC-SHA256'

Fastest option for this machine is: --encryption=AES256-GCM-HMAC-SHA256
----
+

. To benchmark the splitter algorithm, run the following command:
+
[source,terminal]
Expand All @@ -169,13 +148,6 @@ splitting 16 blocks of 32MiB each, parallelism 1
DYNAMIC 747.6 MB/s count:107 min:9467 10th:2277562 25th:2971794 50th:4747177 75th:7603998 90th:8388608 max:8388608
DYNAMIC-128K-BUZHASH 718.5 MB/s count:3183 min:3076 10th:80896 25th:104312 50th:157621 75th:249115 90th:262144 max:262144
DYNAMIC-128K-RABINKARP 164.4 MB/s count:3160 min:9667 10th:80098 25th:106626 50th:162269 75th:250655 90th:262144 max:262144
# ...
FIXED-512K 102.9 TB/s count:1024 min:524288 10th:524288 25th:524288 50th:524288 75th:524288 90th:524288 max:524288
FIXED-8M 566.3 TB/s count:64 min:8388608 10th:8388608 25th:8388608 50th:8388608 75th:8388608 90th:8388608 max:8388608
-----------------------------------------------------------------
0. FIXED-8M 566.3 TB/s count:64 min:8388608 10th:8388608 25th:8388608 50th:8388608 75th:8388608 90th:8388608 max:8388608
1. FIXED-4M 425.8 TB/s count:128 min:4194304 10th:4194304 25th:4194304 50th:4194304 75th:4194304 90th:4194304 max:4194304
# ...
22. DYNAMIC-128K-RABINKARP 164.4 MB/s count:3160 min:9667 10th:80098 25th:106626 50th:162269 75th:250655 90th:262144 max:262144

----

38 changes: 19 additions & 19 deletions modules/oadp-kopia-configuring-algorithms.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,51 +7,51 @@
= Configuring the DPA to override Kopia hashing, encryption, and splitter algorithms

[role="_abstract"]
You can use an {oadp-first} option to override the default Kopia algorithms for hashing, encryption, and splitter to improve Kopia performance or to compare performance metrics. You can set the following environment variables in the `spec.configuration.velero.podConfig.env` section of the DPA:
Configure the Data Protection Application (DPA) to override the default Kopia hashing, encryption, and splitter algorithms by setting environment variables in the Velero pod configuration. This helps you improve Kopia performance and compare performance metrics for your backup operations.

* `KOPIA_HASHING_ALGORITHM`
* `KOPIA_ENCRYPTION_ALGORITHM`
* `KOPIA_SPLITTER_ALGORITHM`
include::snippets/kopia-repo-config.adoc[]

.Prerequisites

* You have installed the {oadp-short} Operator.
* You have created the secret by using the credentials provided by the cloud provider.

include::snippets/kopia-repo-config.adoc[]

.Procedure

* Configure the DPA with the environment variables for hashing, encryption, and splitter as shown in the following example.

+
.Example DPA

[source,yaml]
----
apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
#...
configuration:
nodeAgent:
enable: true # <1>
uploaderType: kopia # <2>
enable: true
uploaderType: kopia
velero:
defaultPlugins:
- openshift
- aws
- csi #<3>
- csi
defaultSnapshotMoveData: true
podConfig:
env:
- name: KOPIA_HASHING_ALGORITHM
value: <hashing_algorithm_name> # <4>
value: <hashing_algorithm_name>
- name: KOPIA_ENCRYPTION_ALGORITHM
value: <encryption_algorithm_name> # <5>
value: <encryption_algorithm_name>
- name: KOPIA_SPLITTER_ALGORITHM
value: <splitter_algorithm_name> # <6>
value: <splitter_algorithm_name>
----
<1> Enable the `nodeAgent`.
<2> Specify the `uploaderType` as `kopia`.
<3> Include the `csi` plugin.
<4> Specify a hashing algorithm. For example, `BLAKE3-256`.
<5> Specify an encryption algorithm. For example, `CHACHA20-POLY1305-HMAC-SHA256`.
<6> Specify a splitter algorithm. For example, `DYNAMIC-8M-RABINKARP`.
+
where:

`enable`:: Set to `true` to enable the `nodeAgent`.
`uploaderType`:: Specifies the uploader type as `kopia`.
`csi`:: Include the `csi` plugin.
`<hashing_algorithm_name>`:: Specifies a hashing algorithm. For example, `BLAKE3-256`.
`<encryption_algorithm_name>`:: Specifies an encryption algorithm. For example, `CHACHA20-POLY1305-HMAC-SHA256`.
`<splitter_algorithm_name>`:: Specifies a splitter algorithm. For example, `DYNAMIC-8M-RABINKARP`.
Loading