forked from apache/cloudstack
-
Notifications
You must be signed in to change notification settings - Fork 0
feature/CSTACKEX-46: Create, Delete iSCSI type Cloudstack volumes, Enter, Cancel Maintenance mode #27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sandeeplocharla
wants to merge
28
commits into
main
Choose a base branch
from
feature/CSTACKEX-46-new
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,430
−187
Open
feature/CSTACKEX-46: Create, Delete iSCSI type Cloudstack volumes, Enter, Cancel Maintenance mode #27
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
9c85c6b
CSTACKEX-46: Disable, Re-Enable, Delete Storage pool and Enter, Exit …
41e8324
CSTACKEX-50: Fixed some issues seen while testing
7f47d93
CSTACKEX-46 Create Async, Attach Cluster/Zone and Grant/Revoke Access
d6e1d70
CSTACKEX-46 Added Logging
eff6d97
CSTACKEX-46 Resolve Copilot review comments
faf8dbf
CSTACKEX-46 Resolve review comments
8852bb4
CSTACKEX-46 Resolve review bot comments
dc5f01a
CSTACKEX-46 Change method name
14ec034
CSTACKEX-46 Resolve check styling issues
423aee2
CSTACKEX-46 Testing: Resolve fetch Storage details issue
905ae4a
CSTACKEX-46 Testing: Protocol Type fetch issue
3ee5ead
CSTACKEX-46 Testing: Added debug log
dc0e84e
CSTACKEX-46 Testing: Added debug log
b2056ff
CSTACKEX-46 Testing: Corrected the URL for Create Lun and Igroup
67bf958
CSTACKEX-46 Resolve bot review comments
8732f7f
CSTACKEX-46 Resolve review comments
f1b7dbd
CSTACKEX-46 Resolve review comments again
e9ba8b3
CSTACKEX-46 Change log level
f067b50
CSTACKEX-46 added valiodation for lun name
dac753f
CSTACKEX-46 resolve check style issues
716fcaf
CSTACKEX-46 Resolve import issues
e6ece89
CSTACKEX-46 Resolve copilot comments
d111436
CSTACKEX-46: Create, Delete iSCSI type Cloudstack volumes, Enter, Can…
a5fa9d2
CSTACKEX-46: Fixed a check style issue which got introduced during re…
3eea5c4
CSTACKEX-46: Fixed a check style issue which got introduced during re…
63addcd
CSTACKEX-46: Fixed a couple of issues observed while testing NFS3 sto…
dfc1ee9
CSTACKEX-46: Refactored driver class
5895ae3
CSTACKEX-46: Fixed code as per comments received
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,6 +19,8 @@ | |
| import java.util.HashMap; | ||
| import java.util.List; | ||
| import java.util.Map; | ||
| import java.nio.file.Files; | ||
| import java.nio.file.Paths; | ||
|
|
||
| import org.apache.cloudstack.utils.qemu.QemuImg; | ||
| import org.apache.cloudstack.utils.qemu.QemuImg.PhysicalDiskFormat; | ||
|
|
@@ -96,10 +98,15 @@ public boolean connectPhysicalDisk(String volumeUuid, KVMStoragePool pool, Map<S | |
| String result = iScsiAdmCmd.execute(); | ||
|
|
||
| if (result != null) { | ||
| logger.debug("Failed to add iSCSI target " + volumeUuid); | ||
| System.out.println("Failed to add iSCSI target " + volumeUuid); | ||
| // Node record may already exist from a previous run; accept and proceed | ||
| if (isNonFatalNodeCreate(result)) { | ||
| logger.debug("iSCSI node already exists for {}@{}:{}, proceeding", getIqn(volumeUuid), pool.getSourceHost(), pool.getSourcePort()); | ||
| } else { | ||
| logger.debug("Failed to add iSCSI target " + volumeUuid); | ||
| System.out.println("Failed to add iSCSI target " + volumeUuid); | ||
|
|
||
| return false; | ||
| return false; | ||
| } | ||
| } else { | ||
| logger.debug("Successfully added iSCSI target " + volumeUuid); | ||
| System.out.println("Successfully added to iSCSI target " + volumeUuid); | ||
|
|
@@ -123,21 +130,28 @@ public boolean connectPhysicalDisk(String volumeUuid, KVMStoragePool pool, Map<S | |
| } | ||
| } | ||
|
|
||
| // ex. sudo iscsiadm -m node -T iqn.2012-03.com.test:volume1 -p 192.168.233.10:3260 --login | ||
| iScsiAdmCmd = new Script(true, "iscsiadm", 0, logger); | ||
| final String host = pool.getSourceHost(); | ||
| final int port = pool.getSourcePort(); | ||
| final String iqn = getIqn(volumeUuid); | ||
|
|
||
| // Always try to login; treat benign outcomes as success (idempotent) | ||
| iScsiAdmCmd = new Script(true, "iscsiadm", 0, logger); | ||
| iScsiAdmCmd.add("-m", "node"); | ||
| iScsiAdmCmd.add("-T", getIqn(volumeUuid)); | ||
| iScsiAdmCmd.add("-p", pool.getSourceHost() + ":" + pool.getSourcePort()); | ||
| iScsiAdmCmd.add("-T", iqn); | ||
| iScsiAdmCmd.add("-p", host + ":" + port); | ||
| iScsiAdmCmd.add("--login"); | ||
|
|
||
| result = iScsiAdmCmd.execute(); | ||
|
|
||
| if (result != null) { | ||
| logger.debug("Failed to log in to iSCSI target " + volumeUuid); | ||
| System.out.println("Failed to log in to iSCSI target " + volumeUuid); | ||
| if (isNonFatalLogin(result)) { | ||
| logger.debug("iSCSI login returned benign message for {}@{}:{}: {}", iqn, host, port, result); | ||
| } else { | ||
| logger.debug("Failed to log in to iSCSI target " + volumeUuid + ": " + result); | ||
| System.out.println("Failed to log in to iSCSI target " + volumeUuid); | ||
|
|
||
| return false; | ||
| return false; | ||
| } | ||
| } else { | ||
| logger.debug("Successfully logged in to iSCSI target " + volumeUuid); | ||
| System.out.println("Successfully logged in to iSCSI target " + volumeUuid); | ||
|
|
@@ -158,8 +172,23 @@ public boolean connectPhysicalDisk(String volumeUuid, KVMStoragePool pool, Map<S | |
| return true; | ||
| } | ||
|
|
||
| // Removed sessionExists() call to avoid noisy sudo/iscsiadm session queries that may fail on some setups | ||
|
|
||
| private boolean isNonFatalLogin(String result) { | ||
| if (result == null) return true; | ||
| String msg = result.toLowerCase(); | ||
| // Accept messages where the session already exists | ||
| return msg.contains("already present") || msg.contains("already logged in") || msg.contains("session exists"); | ||
| } | ||
|
|
||
| private boolean isNonFatalNodeCreate(String result) { | ||
| if (result == null) return true; | ||
| String msg = result.toLowerCase(); | ||
| return msg.contains("already exists") || msg.contains("database exists") || msg.contains("exists"); | ||
| } | ||
|
|
||
| private void waitForDiskToBecomeAvailable(String volumeUuid, KVMStoragePool pool) { | ||
| int numberOfTries = 10; | ||
| int numberOfTries = 30; | ||
| int timeBetweenTries = 1000; | ||
|
|
||
| while (getPhysicalDisk(volumeUuid, pool).getSize() == 0 && numberOfTries > 0) { | ||
|
|
@@ -238,6 +267,15 @@ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid, KVMStoragePool pool) { | |
| } | ||
|
|
||
| private long getDeviceSize(String deviceByPath) { | ||
| try { | ||
| if (!Files.exists(Paths.get(deviceByPath))) { | ||
| logger.debug("Device by-path does not exist yet: " + deviceByPath); | ||
| return 0L; | ||
| } | ||
| } catch (Exception ignore) { | ||
| // If FS check fails for any reason, fall back to blockdev call | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. lets add a logger here |
||
| } | ||
|
|
||
| Script iScsiAdmCmd = new Script(true, "blockdev", 0, logger); | ||
|
|
||
| iScsiAdmCmd.add("--getsize64", deviceByPath); | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| <!-- | ||
| Licensed to the Apache Software Foundation (ASF) under one | ||
| or more contributor license agreements. See the NOTICE file | ||
| distributed with this work for additional information | ||
| regarding copyright ownership. The ASF licenses this file | ||
| to you under the Apache License, Version 2.0 (the | ||
| "License"); you may not use this file except in compliance | ||
| with the License. You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, | ||
| software distributed under the License is distributed on an | ||
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| KIND, either express or implied. See the License for the | ||
| specific language governing permissions and limitations | ||
| under the License. | ||
| --> | ||
|
|
||
| # Apache CloudStack - NetApp ONTAP Storage Plugin | ||
|
|
||
| ## Overview | ||
|
|
||
| The NetApp ONTAP Storage Plugin provides integration between Apache CloudStack and NetApp ONTAP storage systems. This plugin enables CloudStack to provision and manage primary storage on ONTAP clusters, supporting both NAS (NFS) and SAN (iSCSI) protocols. | ||
|
|
||
| ## Features | ||
|
|
||
| - **Primary Storage Support**: Provision and manage primary storage pools on NetApp ONTAP | ||
| - **Multiple Protocols**: Support for NFS 3.0 and iSCSI protocols | ||
| - **Unified Storage**: Integration with traditional ONTAP unified storage architecture | ||
| - **KVM Hypervisor Support**: Supports KVM hypervisor environments | ||
| - **Managed Storage**: Operates as managed storage with full lifecycle management | ||
| - **Flexible Scoping**: Support for Zone-wide and Cluster-scoped storage pools | ||
|
|
||
| ## Architecture | ||
|
|
||
| ### Component Structure | ||
|
|
||
| | Package | Description | | ||
| |---------|-------------------------------------------------------| | ||
| | `driver` | Primary datastore driver implementation | | ||
| | `feign` | REST API clients and data models for ONTAP operations | | ||
| | `lifecycle` | Storage pool lifecycle management | | ||
| | `listener` | Host connection event handlers | | ||
| | `provider` | Main provider and strategy factory | | ||
| | `service` | ONTAP Storage strategy implementations (NAS/SAN) | | ||
| | `utils` | Constants and helper utilities | | ||
|
|
||
| ## Requirements | ||
|
|
||
| ### ONTAP Requirements | ||
|
|
||
| - NetApp ONTAP 9.15.1 or higher | ||
| - Storage Virtual Machine (SVM) configured with appropriate protocols enabled | ||
| - Management LIF accessible from CloudStack management server | ||
| - Data LIF(s) accessible from hypervisor hosts and are of IPv4 type | ||
| - Aggregates assigned to the SVM with sufficient capacity | ||
|
|
||
| ### CloudStack Requirements | ||
|
|
||
| - Apache CloudStack current version or higher | ||
| - KVM hypervisor hosts | ||
| - For iSCSI: Hosts must have iSCSI initiator configured with valid IQN | ||
| - For NFS: Hosts must have NFS client packages installed | ||
|
|
||
| ### Minimum Volume Size | ||
|
|
||
| ONTAP requires a minimum volume size of **1.56 GB** (1,677,721,600 bytes). The plugin will automatically adjust requested sizes below this threshold. | ||
|
|
||
| ## Configuration | ||
|
|
||
| ### Storage Pool Creation Parameters | ||
|
|
||
| When creating an ONTAP primary storage pool, provide the following details in the URL field (semicolon-separated key=value pairs): | ||
|
|
||
| | Parameter | Required | Description | | ||
| |-----------|----------|-------------| | ||
| | `username` | Yes | ONTAP cluster admin username | | ||
| | `password` | Yes | ONTAP cluster admin password | | ||
| | `svmName` | Yes | Storage Virtual Machine name | | ||
| | `protocol` | Yes | Storage protocol (`NFS3` or `ISCSI`) | | ||
| | `managementLIF` | Yes | ONTAP cluster management LIF IP address | | ||
|
|
||
| ### Example URL Format | ||
|
|
||
| ``` | ||
| username=admin;password=secretpass;svmName=svm1;protocol=ISCSI;managementLIF=192.168.1.100 | ||
| ``` | ||
|
|
||
| ## Port Configuration | ||
|
|
||
| | Protocol | Default Port | | ||
| |----------|--------------| | ||
| | NFS | 2049 | | ||
| | iSCSI | 3260 | | ||
| | ONTAP Management API | 443 (HTTPS) | | ||
|
|
||
| ## Limitations | ||
|
|
||
| - Supports only **KVM** hypervisor | ||
| - Supports only **Unified ONTAP** storage (disaggregated not supported) | ||
| - Supports only **NFS3** and **iSCSI** protocols | ||
| - IPv6 type and FQDN LIFs are not supported | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Common Issues | ||
|
|
||
| 1. **Connection Failures** | ||
| - Verify management LIF is reachable from CloudStack management server | ||
| - Check firewall rules for port 443 | ||
|
|
||
| 2. **Protocol Errors** | ||
| - Ensure the protocol (NFS/iSCSI) is enabled on the SVM | ||
| - Verify Data LIFs are configured for the protocol | ||
|
|
||
| 3. **Capacity Errors** | ||
| - Check aggregate space availability | ||
| - Ensure requested volume size meets minimum requirements (1.56 GB) | ||
|
|
||
| 4. **Host Connection Issues** | ||
| - For iSCSI: Verify host IQN is properly configured in host's storage URL | ||
| - For NFS: Ensure NFS client is installed and running |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.