-
Notifications
You must be signed in to change notification settings - Fork 1.3k
engine/schema: fix cluster/zone settings with encrypted values #12626
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
engine/schema: fix cluster/zone settings with encrypted values #12626
Conversation
|
@blueorangutan package |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## 4.20 #12626 +/- ##
============================================
- Coverage 16.27% 16.26% -0.01%
+ Complexity 13433 13431 -2
============================================
Files 5662 5662
Lines 500033 500033
Branches 60717 60717
============================================
- Hits 81368 81355 -13
- Misses 409591 409604 +13
Partials 9074 9074
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
DaanHoogland
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clgtm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
testing still in progress
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
sureshanaparti
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clgtm
|
@blueorangutan package |
|
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16797 |
|
@blueorangutan test |
|
@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
|
[SF] Trillian test result (tid-15434)
|
abh1sar
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code LGTM
RosiKyu
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
| # | Test Case | Scope | Result |
|---|---|---|---|
| 1 | Encrypted setting decryption (backup.plugin.veeam.password) | Zone | PASS |
| 2 | Encrypted setting decryption (backup.plugin.networker.password) | Zone | PASS |
| 3 | Non-encrypted setting read/update (cpu.overprovisioning.factor) | Cluster | PASS |
| 4 | Non-encrypted setting read/update regression (backup.framework.enabled) | Zone | PASS |
Detailed Test Report:
TC1: Zone-scoped encrypted setting decryption (backup.plugin.veeam.password)
Objective
Verify that DataCenterDetailsDaoImpl.valueIn() properly handles encrypted zone-scoped settings by storing them encrypted in the database and processing them correctly when read via API.
Test Steps
- Set
backup.plugin.veeam.passwordat zone scope via CloudMonkey - Verify the value is stored encrypted in
data_center_detailstable - Read the value back via
list configurationsAPI with zoneid
Expected Result:
- Value should be stored encrypted (not plaintext) in
data_center_detailstable - API should return a processed value (not the raw DB encrypted blob) indicating the DAO decryption path is functioning
- No errors during update or read operations
Actual Result:
- Value stored encrypted in DB:
GU2+EWLmSQOFD3nSgYNhg1+JCwAMydQNdPJhKinZcoJt2mbfC7xl54F//4Y= - API returned a different encrypted representation:
BzqFxon8fPyN1R4Lq4wyn4W/S48S84PuqNsQJHmHXjIjDepNoijJccCJM1U=- the difference confirms the value is being processed through the decryption path (decrypted by DAO, then re-encrypted for Secure display) - No errors during update or read
- Note:
Securecategory settings are always masked/encrypted in API responses by design, so plaintextTestVeeamPass123is not expected in API output. Full end-to-end decryption verification would require a functional Veeam integration test or management server log analysis.
Result: PASS
Test Evidence:
Step 1: Set password at zone scope:
(localcloud) 🐱 > update configuration name=backup.plugin.veeam.password value=TestVeeamPass123 zoneid=36eb15de-1ce1-4924-8ef3-327430e490a7
{
"configuration": {
"category": "Secure",
"component": "BackupService",
"defaultvalue": "",
"description": "The Veeam backup and recovery password.",
"displaytext": "Backup plugin veeam password",
"group": "Miscellaneous",
"isdynamic": true,
"name": "backup.plugin.veeam.password",
"scope": "zone",
"subgroup": "Backup & Recovery",
"type": "String",
"value": "AG8j56WkkRj9hvLXEjz05m3To/xslx9KVoNSGko4H5LH5GZV8WXov95PeDE="
}
}
Step 2: Verify encrypted storage in DB:
mysql> SELECT * FROM data_center_details;
+----+-------+------------------------------+--------------------------------------------------------------+---------+
| id | dc_id | name | value | display |
+----+-------+------------------------------+--------------------------------------------------------------+---------+
| 1 | 1 | backup.plugin.veeam.password | GU2+EWLmSQOFD3nSgYNhg1+JCwAMydQNdPJhKinZcoJt2mbfC7xl54F//4Y= | 1 |
+----+-------+------------------------------+--------------------------------------------------------------+---------+
1 row in set (0.00 sec)
Step 3: Read back via API:
(localcloud) 🐱 > list configurations name=backup.plugin.veeam.password zoneid=36eb15de-1ce1-4924-8ef3-327430e490a7
{
"configuration": [
{
"category": "Secure",
"component": "BackupService",
"defaultvalue": "",
"description": "The Veeam backup and recovery password.",
"displaytext": "Backup plugin veeam password",
"group": "Miscellaneous",
"isdynamic": true,
"name": "backup.plugin.veeam.password",
"scope": "zone",
"subgroup": "Backup & Recovery",
"type": "String",
"value": "BzqFxon8fPyN1R4Lq4wyn4W/S48S84PuqNsQJHmHXjIjDepNoijJccCJM1U="
}
],
"count": 1
}
TC2: Zone-scoped encrypted setting decryption (backup.plugin.networker.password)
Objective
Verify that DataCenterDetailsDaoImpl.valueIn() properly handles a second encrypted zone-scoped setting, confirming the fix is consistent across all Secure-category zone configurations.
Test Steps
- Set
backup.plugin.networker.passwordat zone scope via CloudMonkey - Verify the value is stored encrypted in
data_center_detailstable - Read the value back via
list configurationsAPI with zoneid
Expected Result:
- Value should be stored encrypted in
data_center_detailstable - API should return a different encrypted representation (confirming decrypt-then-re-encrypt path)
- No errors during update or read operations
Actual Result:
- Value stored encrypted in DB:
/9LVnL6NuUJlhxFmdnys6uaf8qGYKudpTU6hFf9aKOK9aDRHBNpREqGiXXoKgMiW - API returned different encrypted representation:
diOAUnHQmvuLrluJqKb4qRkY4xa6iqJ8hufz3uKKjRSknQTyRYVOByJhGenOW3CS - DB value ≠ API value confirms value is processed through
getActualValue()decryption path - No errors during update or read
Result: PASS
Test Evidence:
Step 1: Set password at zone scope:
(localcloud) 🐱 > update configuration name=backup.plugin.networker.password value=TestNetworkerPass456 zoneid=36eb15de-1ce1-4924-8ef3-327430e490a7
{
"configuration": {
"category": "Secure",
"component": "BackupService",
"defaultvalue": "password",
"description": "The EMC Networker API password.",
"displaytext": "Backup plugin networker password",
"group": "Miscellaneous",
"isdynamic": true,
"name": "backup.plugin.networker.password",
"scope": "zone",
"subgroup": "Backup & Recovery",
"type": "String",
"value": "3EPCT3SKiY+kfzPZNKV8MnZmct79TYsZZ8WIemNjQC2SLNiUQaFzS7Cqg54suTnV"
}
}
Step 2: Verify encrypted storage in DB:
mysql> SELECT * FROM data_center_details;
+----+-------+----------------------------------+------------------------------------------------------------------+---------+
| id | dc_id | name | value | display |
+----+-------+----------------------------------+------------------------------------------------------------------+---------+
| 1 | 1 | backup.plugin.veeam.password | GU2+EWLmSQOFD3nSgYNhg1+JCwAMydQNdPJhKinZcoJt2mbfC7xl54F//4Y= | 1 |
| 2 | 1 | backup.plugin.networker.password | /9LVnL6NuUJlhxFmdnys6uaf8qGYKudpTU6hFf9aKOK9aDRHBNpREqGiXXoKgMiW | 1 |
+----+-------+----------------------------------+------------------------------------------------------------------+---------+
2 rows in set (0.00 sec)
Step 3: Read back via API:
(localcloud) 🐱 > list configurations name=backup.plugin.networker.password zoneid=36eb15de-1ce1-4924-8ef3-327430e490a7
{
"configuration": [
{
"category": "Secure",
"component": "BackupService",
"defaultvalue": "password",
"description": "The EMC Networker API password.",
"displaytext": "Backup plugin networker password",
"group": "Miscellaneous",
"isdynamic": true,
"name": "backup.plugin.networker.password",
"scope": "zone",
"subgroup": "Backup & Recovery",
"type": "String",
"value": "diOAUnHQmvuLrluJqKb4qRkY4xa6iqJ8hufz3uKKjRSknQTyRYVOByJhGenOW3CS"
}
],
"count": 1
}
TC3: Cluster-scoped setting lookup via ClusterDetailsDaoImpl (regression)
Objective
Verify that the ClusterDetailsDaoImpl search builder fix (getClusterId() → getResourceId()) correctly handles cluster-scoped configuration lookups. No cluster-scoped Secure/Hidden settings exist in CloudStack, so this test validates the search builder change doesn't break normal cluster detail operations.
Test Steps
- Read existing cluster-scoped setting
cpu.overprovisioning.factorvia API - Update the setting from
2.0to3.0at cluster scope - Read back via API to confirm update
- Verify in
cluster_detailsDB table
Expected Result:
- Initial read returns current value (
2.0) - Update succeeds without error
- Subsequent read returns new value (
3.0) - DB reflects updated value
Actual Result:
- Initial read returned
2.0 - Update to
3.0succeeded - Subsequent read returned
3.0 - DB confirmed
cpuOvercommitRatio = 3.0
Result: PASS
Test Evidence:
Step 1: Read existing cluster setting:
(localcloud) 🐱 > list configurations name=cpu.overprovisioning.factor clusterid=a86089d6-73d4-439c-b7e1-0c433709fbf4
{
"configuration": [
{
"category": "Advanced",
"component": "CapacityManager",
"defaultvalue": "1.0",
"description": "Used for CPU overprovisioning calculation; available CPU will be (actualCpuCapacity * cpu.overprovisioning.factor)",
"displaytext": "Cpu overprovisioning factor",
"group": "Compute",
"isdynamic": true,
"name": "cpu.overprovisioning.factor",
"scope": "cluster",
"subgroup": "Virtual Machine",
"type": "Decimal",
"value": "2.0"
}
],
"count": 2
}
Step 2: Update cluster setting:
(localcloud) 🐱 > update configuration name=cpu.overprovisioning.factor value=3.0 clusterid=a86089d6-73d4-439c-b7e1-0c433709fbf4
{
"configuration": {
"category": "Advanced",
"component": "CapacityManager",
"defaultvalue": "1.0",
"description": "Used for CPU overprovisioning calculation; available CPU will be (actualCpuCapacity * cpu.overprovisioning.factor)",
"displaytext": "Cpu overprovisioning factor",
"group": "Compute",
"isdynamic": true,
"name": "cpu.overprovisioning.factor",
"scope": "cluster",
"subgroup": "Virtual Machine",
"type": "Decimal",
"value": "3.0"
}
}
Step 3: Read back via API:
(localcloud) 🐱 > list configurations name=cpu.overprovisioning.factor clusterid=a86089d6-73d4-439c-b7e1-0c433709fbf4
{
"configuration": [
{
"category": "Advanced",
"component": "CapacityManager",
"defaultvalue": "1.0",
"name": "cpu.overprovisioning.factor",
"scope": "cluster",
"type": "Decimal",
"value": "3.0"
}
],
"count": 2
}
Step 4: Verify in DB:
mysql> SELECT * FROM cluster_details;
+----+------------+-----------------------+-------+
| id | cluster_id | name | value |
+----+------------+-----------------------+-------+
| 6 | 2 | memoryOvercommitRatio | 1.0 |
| 7 | 2 | cpuOvercommitRatio | 3.0 |
+----+------------+-----------------------+-------+
2 rows in set (0.00 sec)
TC4: Non-encrypted zone-scoped setting (regression)
Objective
Verify that the DataCenterDetailsDaoImpl fix (using getActualValue() instead of getValue()) does not break non-encrypted zone-scoped settings. Non-encrypted values should pass through unchanged.
Test Steps
- Read existing zone-scoped setting
backup.framework.enabled(currentlytrue) - Update to
falseat zone scope - Read back via API to confirm
- Verify in DB that value is stored in plaintext (not encrypted)
- Restore original value to
true
Expected Result:
- Update succeeds without error
- API returns
falsein plaintext - DB stores
falsein plaintext (not encrypted) - Restore to
truesucceeds
Actual Result:
- Update to
falsesucceeded - API returned
falsein plaintext - DB stored
falsein plaintext, confirming non-encrypted values are not affected by the fix - Restore to
truesucceeded
Result: PASS
Test Evidence:
Step 1: Read existing setting:
(localcloud) 🐱 > list configurations name=backup.framework.enabled zoneid=36eb15de-1ce1-4924-8ef3-327430e490a7
{
"configuration": [
{
"category": "Advanced",
"component": "BackupService",
"defaultvalue": "false",
"description": "Is backup and recovery framework enabled.",
"displaytext": "Backup framework enabled",
"group": "Miscellaneous",
"isdynamic": false,
"name": "backup.framework.enabled",
"scope": "zone",
"subgroup": "Backup & Recovery",
"type": "Boolean",
"value": "true"
}
],
"count": 1
}
Step 2: Update to false:
(localcloud) 🐱 > update configuration name=backup.framework.enabled value=false zoneid=36eb15de-1ce1-4924-8ef3-327430e490a7
{
"configuration": {
"category": "Advanced",
"component": "BackupService",
"defaultvalue": "false",
"description": "Is backup and recovery framework enabled.",
"displaytext": "Backup framework enabled",
"group": "Miscellaneous",
"isdynamic": false,
"name": "backup.framework.enabled",
"scope": "zone",
"subgroup": "Backup & Recovery",
"type": "Boolean",
"value": "false"
}
}
Step 3: Read back via API:
(localcloud) 🐱 > list configurations name=backup.framework.enabled zoneid=36eb15de-1ce1-4924-8ef3-327430e490a7
{
"configuration": [
{
"category": "Advanced",
"component": "BackupService",
"defaultvalue": "false",
"description": "Is backup and recovery framework enabled.",
"displaytext": "Backup framework enabled",
"group": "Miscellaneous",
"isdynamic": false,
"name": "backup.framework.enabled",
"scope": "zone",
"subgroup": "Backup & Recovery",
"type": "Boolean",
"value": "false"
}
],
"count": 1
}
Step 4: Verify plaintext storage in DB:
mysql> SELECT * FROM data_center_details;
+----+-------+----------------------------------+------------------------------------------------------------------+---------+
| id | dc_id | name | value | display |
+----+-------+----------------------------------+------------------------------------------------------------------+---------+
| 1 | 1 | backup.plugin.veeam.password | GU2+EWLmSQOFD3nSgYNhg1+JCwAMydQNdPJhKinZcoJt2mbfC7xl54F//4Y= | 1 |
| 2 | 1 | backup.plugin.networker.password | /9LVnL6NuUJlhxFmdnys6uaf8qGYKudpTU6hFf9aKOK9aDRHBNpREqGiXXoKgMiW | 1 |
| 3 | 1 | backup.framework.enabled | false | 1 |
+----+-------+----------------------------------+------------------------------------------------------------------+---------+
3 rows in set (0.00 sec)
Step 5: Restore original value:
(localcloud) 🐱 > update configuration name=backup.framework.enabled value=true zoneid=36eb15de-1ce1-4924-8ef3-327430e490a7
{
"configuration": {
"category": "Advanced",
"component": "BackupService",
"defaultvalue": "false",
"description": "Is backup and recovery framework enabled.",
"displaytext": "Backup framework enabled",
"group": "Miscellaneous",
"isdynamic": false,
"name": "backup.framework.enabled",
"scope": "zone",
"subgroup": "Backup & Recovery",
"type": "Boolean",
"value": "true"
}
}
|
thanks @RosiKyu for the testing, amazing I double checked the encrypted values in cmk response and database, and confirm the original values are the same. |
Description
This PR fixes the issue that zone/cluster encrypted settings are not decrypted when use
valueIn(resourceId)The issue should have been fixed in 4.19 by #10237 which has changes with
However, the changes are missing in 4.20 in two files
other files are good.
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
How did you try to break this feature and the system with this change?