Major refactoring of switch-images support

Assisted-By: Claude Code/claude-4.5-sonnet
Signed-off-by: Harald Jensås <hjensas@redhat.com>

Author: hjensas

docs/virtual_switches.md

@@ -1,9 +1,207 @@
 # Using virtual switches with Hotstack
 
+## Creating a switch image
+
 ```bash
 openstack image create hotstack-switch \
   --disk-format qcow2 \
   --file <switch-image-file> \
   --property hw_firmware_type=uefi  \
   --property hw_machine_type=q35 --public
 ```
+
+## Network wiring: OpenStack networks as bridges
+
+Hotstack uses a special network architecture to connect VMs (like baremetal
+nodes managed by Ironic) to ports on the virtual switch. This approach uses
+**OpenStack Neutron networks as L2 bridges** between VMs and switch ports.
+
+### Architecture overview
+
+The architecture consists of three key components:
+
+1. **Trunk port on the switch**: A Neutron trunk port that carries multiple
+   VLANs to the switch
+2. **Bridge networks**: Small point-to-point Neutron networks connecting each
+   VM to the switch
+3. **VLAN configuration inside the switch**: The switch's internal
+   configuration determines which VLAN each port belongs to
+
+### How it works
+
+The key insight is that **the switch's internal configuration determines
+VLAN membership**, not OpenStack:
+
+- The bridge network (e.g. `ironic0-br-net`) provides L2 connectivity
+  between the VM and the switch
+- Inside the switch, you configure which VLAN the port (connected to the
+  bridge network) belongs to
+- When you change the VLAN configuration of the switch port, the VM
+  effectively "moves" to a different VLAN
+
+**Example flow:**
+
+1. VM `ironic0` is connected to `ironic0-br-net`
+2. The switch port `ethernet1/2` is also connected to `ironic0-br-net`
+3. Inside the switch, configure `ethernet1/2` to be an access port on
+   VLAN 101
+4. Traffic from `ironic0` now flows through the bridge network to the
+   switch, where it enters VLAN 101
+5. VLAN 101 traffic exits the switch through the trunk port (tagged with
+   VLAN 101)
+6. OpenStack Neutron receives this as traffic on the `ironic-net` network
+   (which is VLAN 101 on the trunk)
+
+**To move the VM to a different VLAN:**
+
+1. Reconfigure switch port `ethernet1/2` to be on VLAN 103 instead
+2. Now traffic from `ironic0` enters VLAN 103 and exits as traffic on the
+   `tenant-vlan103` network
+3. **No changes needed in OpenStack** - the bridge network stays the same
+
+### Benefits of this approach
+
+- **Flexibility**: VMs can be moved between VLANs by reconfiguring the
+  switch, not by reconfiguring OpenStack
+- **Realistic testing**: Mimics how physical networks work with switches
+  controlling VLAN membership
+- **Simplified OpenStack config**: Each VM just needs one static connection
+  (the bridge network)
+- **Switch-driven networking**: The switch becomes the central point of
+  control for network topology, just like in a physical datacenter
+
+### Implementation in Heat templates
+
+#### 1. Switch trunk port setup
+
+The switch has a trunk port that connects to multiple VLAN networks as
+subports:
+
+```yaml
+switch-trunk-parent-port:
+  type: OS::Neutron::Port
+  properties:
+    network: {get_resource: trunk-net}
+    port_security_enabled: false
+
+switch-trunk-ironic-port:
+  type: OS::Neutron::Port
+  properties:
+    network: {get_resource: ironic-net}
+    port_security_enabled: false
+
+switch-trunk-tenant-vlan103-port:
+  type: OS::Neutron::Port
+  properties:
+    network: {get_resource: tenant-vlan103}
+    port_security_enabled: false
+
+switch-trunk-tenant-vlan104-port:
+  type: OS::Neutron::Port
+  properties:
+    network: {get_resource: tenant-vlan104}
+    port_security_enabled: false
+
+switch-trunk-tenant-vlan105-port:
+  type: OS::Neutron::Port
+  properties:
+    network: {get_resource: tenant-vlan105}
+    port_security_enabled: false
+
+switch-trunk:
+  type: OS::Neutron::Trunk
+  properties:
+    port: {get_resource: switch-trunk-parent-port}
+    sub_ports:
+      # Ironic VLAN
+      - port: {get_resource: switch-trunk-ironic-port}
+        segmentation_id: 101
+        segmentation_type: vlan
+      # Tenant VLANs
+      - port: {get_resource: switch-trunk-tenant-vlan103-port}
+        segmentation_id: 103
+        segmentation_type: vlan
+      - port: {get_resource: switch-trunk-tenant-vlan104-port}
+        segmentation_id: 104
+        segmentation_type: vlan
+      - port: {get_resource: switch-trunk-tenant-vlan105-port}
+        segmentation_id: 105
+        segmentation_type: vlan
+```
+
+This trunk port presents multiple VLANs to the switch VM, just like a
+physical trunk port would.
+
+#### 2. Bridge networks connecting VMs to the switch
+
+Each VM that needs to connect to the switch gets a dedicated "bridge
+network" - a small Neutron network that acts as a point-to-point L2
+connection:
+
+```yaml
+ironic0-br-net:
+  type: OS::Neutron::Net
+  properties:
+    port_security_enabled: false
+
+ironic1-br-net:
+  type: OS::Neutron::Net
+  properties:
+    port_security_enabled: false
+```
+
+Both the VM and the switch get a port on this bridge network:
+
+```yaml
+switch-ironic0-br-port:
+  type: OS::Neutron::Port
+  properties:
+    network: {get_resource: ironic0-br-net}
+    port_security_enabled: false
+
+switch-ironic1-br-port:
+  type: OS::Neutron::Port
+  properties:
+    network: {get_resource: ironic1-br-net}
+    port_security_enabled: false
+
+switch:
+  type: OS::Nova::Server
+  properties:
+    # ... image, flavor, and disk configuration ...
+    networks:
+      - port: {get_resource: switch-machine-port}
+      - port: {get_attr: [switch-trunk, port_id]}
+      - port: {get_resource: switch-ironic0-br-port}
+      - port: {get_resource: switch-ironic1-br-port}
+```
+
+The VMs also connect to their respective bridge networks:
+
+```yaml
+ironic0-port:
+  type: OS::Neutron::Port
+  properties:
+    network: {get_resource: ironic0-br-net}
+    port_security_enabled: false
+
+ironic0:
+  type: OS::Nova::Server
+  properties:
+    # ... image, flavor, and disk configuration ...
+    networks:
+      - port: {get_resource: ironic0-port}
+
+ironic1-port:
+  type: OS::Neutron::Port
+  properties:
+    network: {get_resource: ironic1-br-net}
+    port_security_enabled: false
+
+ironic1:
+  type: OS::Nova::Server
+  properties:
+    # ... image, flavor, and disk configuration ...
+    networks:
+      - port: {get_resource: ironic1-port}
+```

images/.gitignore

@@ -1,7 +1,11 @@
+# Built images
 controller.qcow2
 blank-image.qcow2
 nat64-appliance.qcow2
-switch-host.qcow2
-switch-host-base.qcow2
+
+# Temporary files
+*.tmp
+
+# NAT64 build artifacts
 .ci-framework/
 .nat64-build/

images/Makefile

@@ -9,18 +9,6 @@ NAT64_IMAGE_NAME             ?= nat64-appliance.qcow2
 NAT64_IMAGE_FORMAT           ?= raw
 NAT64_CIFMW_DIR              ?= $(CURDIR)/.ci-framework
 NAT64_BASEDIR                ?= $(CURDIR)/.nat64-build
-SWITCH_HOST_IMAGE_URL        ?= https://cloud.centos.org/centos/9-stream/x86_64/images/CentOS-Stream-GenericCloud-x86_64-9-latest.x86_64.qcow2
-SWITCH_HOST_BASE_IMAGE       ?= switch-host-base.qcow2
-SWITCH_HOST_IMAGE_NAME       ?= switch-host.qcow2
-SWITCH_HOST_IMAGE_FORMAT     ?= raw
-SWITCH_HOST_INSTALL_PACKAGES ?= libvirt,qemu-kvm,qemu-img,expect,unzip,jq,iproute,nmap-ncat,telnet,git,vim-enhanced,tmux,bind-utils,bash-completion,nmstate,tcpdump,python3-jinja2
-
-# Switch vendor image locations (optional, leave empty if not available)
-# These will be copied into the image at /opt/<model>/ during build
-FORCE10_10_IMAGE            ?=
-FORCE10_9_IMAGE             ?=
-NXOS_IMAGE                  ?=
-SONIC_IMAGE                 ?=
 
 all: controller blank nat64
 
@@ -86,85 +74,3 @@ nat64_clean:
 	rm -rf $(NAT64_CIFMW_DIR)
 	sudo rm -rf $(NAT64_BASEDIR)
 	rm -rf $(HOME)/test-python
-
-switch-host: switch-host_download switch-host_copy switch-host_customize switch-host_convert
-
-switch-host_download:
-	@if [ ! -f $(SWITCH_HOST_BASE_IMAGE) ]; then \
-		echo "Downloading base image to $(SWITCH_HOST_BASE_IMAGE)..."; \
-		curl -L -o $(SWITCH_HOST_BASE_IMAGE) $(SWITCH_HOST_IMAGE_URL); \
-	else \
-		echo "Base image $(SWITCH_HOST_BASE_IMAGE) already exists, skipping download."; \
-	fi
-
-switch-host_copy:
-	@echo "Creating working copy: $(SWITCH_HOST_BASE_IMAGE) -> $(SWITCH_HOST_IMAGE_NAME)"
-	@cp $(SWITCH_HOST_BASE_IMAGE) $(SWITCH_HOST_IMAGE_NAME)
-
-switch-host_customize:
-	@echo "Customizing switch-host image..."
-ifneq ($(FORCE10_10_IMAGE),)
-	@test -f $(FORCE10_10_IMAGE) || (echo "ERROR: Force10 OS10 image not found: $(FORCE10_10_IMAGE)" && exit 1)
-endif
-ifneq ($(FORCE10_9_IMAGE),)
-	@test -f $(FORCE10_9_IMAGE) || (echo "ERROR: Force10 OS9 image not found: $(FORCE10_9_IMAGE)" && exit 1)
-endif
-ifneq ($(NXOS_IMAGE),)
-	@test -f $(NXOS_IMAGE) || (echo "ERROR: NXOS image not found: $(NXOS_IMAGE)" && exit 1)
-endif
-ifneq ($(SONIC_IMAGE),)
-	@test -f $(SONIC_IMAGE) || (echo "ERROR: SONiC image not found: $(SONIC_IMAGE)" && exit 1)
-endif
-	virt-customize -a $(SWITCH_HOST_IMAGE_NAME) \
-		--install $(SWITCH_HOST_INSTALL_PACKAGES) \
-		--timezone UTC \
-		--copy-in switch-host-scripts/start-switch-vm.sh:/usr/local/bin \
-		--chmod 0755:/usr/local/bin/start-switch-vm.sh \
-		--run-command 'mkdir -p /usr/local/lib/hotstack-switch-vm' \
-		--copy-in switch-host-scripts/common.sh:/usr/local/lib/hotstack-switch-vm \
-		--copy-in switch-host-scripts/bridges.nmstate.yaml.j2:/usr/local/lib/hotstack-switch-vm \
-		--chmod 0644:/usr/local/lib/hotstack-switch-vm/common.sh \
-		--chmod 0644:/usr/local/lib/hotstack-switch-vm/bridges.nmstate.yaml.j2 \
-		--run-command 'mkdir -p /etc/hotstack-switch-vm' \
-		--run-command 'mkdir -p /var/lib/hotstack-switch-vm' \
-		$(if $(FORCE10_10_IMAGE), \
-			--copy-in switch-host-scripts/force10_10:/usr/local/lib/hotstack-switch-vm \
-			--chmod 0755:/usr/local/lib/hotstack-switch-vm/force10_10/setup.sh \
-			--chmod 0755:/usr/local/lib/hotstack-switch-vm/force10_10/wait.sh \
-			--chmod 0755:/usr/local/lib/hotstack-switch-vm/force10_10/configure.sh \
-			--chmod 0644:/usr/local/lib/hotstack-switch-vm/force10_10/domain.xml.j2 \
-			--run-command 'mkdir -p /opt/force10_10' \
-			--copy-in $(FORCE10_10_IMAGE):/opt/force10_10 \
-			--run-command 'basename $(FORCE10_10_IMAGE) > /opt/force10_10/image-info.txt',) \
-		$(if $(FORCE10_9_IMAGE), \
-			--run-command 'mkdir -p /opt/force10_9' \
-			--copy-in $(FORCE10_9_IMAGE):/opt/force10_9,) \
-		$(if $(NXOS_IMAGE), \
-			--copy-in switch-host-scripts/nxos:/usr/local/lib/hotstack-switch-vm \
-			--chmod 0755:/usr/local/lib/hotstack-switch-vm/nxos/setup.sh \
-			--chmod 0755:/usr/local/lib/hotstack-switch-vm/nxos/wait.sh \
-			--chmod 0755:/usr/local/lib/hotstack-switch-vm/nxos/configure.sh \
-			--chmod 0644:/usr/local/lib/hotstack-switch-vm/nxos/domain.xml.j2 \
-			--run-command 'mkdir -p /opt/nxos' \
-			--copy-in $(NXOS_IMAGE):/opt/nxos \
-			--run-command 'basename $(NXOS_IMAGE) > /opt/nxos/image-info.txt',) \
-		$(if $(SONIC_IMAGE), \
-			--run-command 'mkdir -p /opt/sonic' \
-			--copy-in $(SONIC_IMAGE):/opt/sonic,) \
-		--selinux-relabel
-	@echo "Switch-host image customization complete"
-
-switch-host_convert:
-ifeq ($(SWITCH_HOST_IMAGE_FORMAT),raw)
-	@echo "Converting switch-host image to raw format (in-place)..."
-	qemu-img convert -p -f qcow2 -O raw $(SWITCH_HOST_IMAGE_NAME) $(SWITCH_HOST_IMAGE_NAME).tmp
-	mv $(SWITCH_HOST_IMAGE_NAME).tmp $(SWITCH_HOST_IMAGE_NAME)
-	@echo "Switch-host image converted to raw format: $(SWITCH_HOST_IMAGE_NAME)"
-endif
-
-switch-host_clean:
-	rm -f $(SWITCH_HOST_IMAGE_NAME)
-	rm -f $(SWITCH_HOST_IMAGE_NAME).tmp
-
-switch-host_clean_all: switch-host_clean
-	rm -f $(SWITCH_HOST_BASE_IMAGE)