feat: add VZVmnetNetworkDeviceAttachment support (macOS 26.0)

[norio-nomura]

feat: add VZVmnetNetworkDeviceAttachment support (macOS 26.0)

VZVmnetNetworkDeviceAttachment is an API that creates vmnet devices on VMs added in macOS 26.

see: https://developer.apple.com/documentation/virtualization/vzvmnetnetworkdeviceattachment?language=objc

It does not require the com.apple.vm.networking entitlement nor root privileges.
HostMode and SharedMode are supported.
In order for multiple VMs to communicate with each other in SharedMode, they must be started in the same executable and the same VmnetNetwork must be passed to NewVmnetNetworkDeviceAttachment() to create an attachment.

This change adds:

• vz.VmnetNetworkDeviceAttachment represents VZVmnetNetworkDeviceAttachment in Go

• vmnet package to use vmnet APIs that added on macOS 26.0

  • Return represents vmnet_return_t as error
    • ErrSuccess, ErrFailure, ...
  • Mode represents operating_modes_t
    • HostMode, SharedMode
  • NetworkConfiguration represents vmnet_network_configuration_t
  • Network represents vmnet_network_ref
  • Interface represents interface_ref
  • *FileAdaptorForInterfaces support File Handle based network device APIs on QEMU, krunkit, and vz.NewFileHandleNetworkDeviceAttachment

• xpc package that providing <xpc/xpc.h> APIs to support implementing Mach service server/client to sharing serializations of vmnet.Network and file descriptors of vmnet.*FileAdaptorForInterfaces

• vz_test.TestVmnetSharedModeAllowsCommunicationBetweenMultipleVMs

• vz_test.TestVmnetSharedModeWithConfiguringIPv4

• vz_test.TestVmnetNetworkShareModeSharingOverXpc
TestVmnetNetworkShareModeSharingOverXpc tests sharing vmnet.Network in SharedMode over XPC communication.
  This test registers test executable as an Mach service and launches it using launchctl.
  The launched Mach service provides vmnet.Network serialization to clients upon request, after booting
  a VM using the provided vmnet.Network to ensure the network is functional on the server side.
  The client boots VM using the provided vmnet.Network serialization.

Edit: on macOS 26.2, "the same executable" restriction seems to be relaxed.

• VZVmnetNetworkDeviceAttachment seems to allow connecting to subnet created by a different executable.
• vmnet_interface_start_with_network seems to allow connecting to subnet created by an executable at same path? (may changing CDHash not affect?)
• I don't know how far the restrictions have been relaxed.

Which issue(s) this PR fixes:

Mentioned in #198 (comment)

[nirs]

This can be used by multiple processes like this:

1. Start a network process create the vmnet_network_ref, starting a xpc listener
2. Start vm process, obtaining the vmnet_network_ref from the xpc server
   • server create a xpc_object_t using https://developer.apple.com/documentation/vmnet/vmnet_network_copy_serialization(_:_:)?language=objc
   • client get the xpc_oject_t via https://developer.apple.com/documentation/xpc/xpc_connection_send_message_with_reply(_:_:_:_:)
   • client create new vment_netwrok_ref via https://developer.apple.com/documentation/vmnet/vmnet_network_create_with_serialization(_:_:)?language=objc
3. Start more vms using same vmnet_network_ref...
4. Wait until vms exit
5. Terminate network process

[norio-nomura]

This can be used by multiple processes like this:

In this procedure, I confirmed that VMs launched from multiple processes can share networks with each other. 👍🏻
It seems that it can be reproduced in the unit test, so I will try to make a unit test.

[norio-nomura]

It seems that it can be reproduced in the unit test, so I will try to make a unit test.

Added unit test and pkg/xpc.

[norio-nomura]

Added unit test and pkg/xpc.

I'll try this added xpc package with lima to make it work. Until then, it's a draft.

[nirs]

I did not review most of this change, just the stange part of about marking bridged mode as depracated.

[norio-nomura]

I think the approximately necessary APIs have been added.
Do I need to add tests to the xpc package?

[Code-Hex]

@norio-nomura Does the xpc package need to be public? If it fits within internal, I don't think we need to add tests.

[norio-nomura]

@norio-nomura Does the xpc package need to be public? If it fits within internal, I don't think we need to add tests.

It depends on the xpc package to make MachService at lima-vm/lima#4394.
The xpc package realizes the sharing of vmnet.Network serialization and the sharing of File Descriptor used to communicate with vmnet.Interface.

[nirs]

@norio-nomura Does the xpc package need to be public? If it fits within internal, I don't think we need to add tests.

We need the xpc.NewObject() for creating a vment_network_ref with serialization. Here is an example usage in vmnet-broker test vm:
https://github.com/nirs/vmnet-broker/blob/6ba05b6ba38b15524c5cb833f223e0b91aa70376/go/cmd/test.go#L197

If vmnet.NewNetworkWithSerialization() will accept xpc_object_t (unsafe.Pointer), the xpc package is not needed in this project and can be part of Lima or available as separate package.

[norio-nomura]

Unit tests added to xpc.
Previously, when I tried vmnet_network_configuration_add_port_forwarding_rule, it didn't work well, so vmnet_interface_add_ip_port_forwarding_rule related to Interface has not been implemented, but I want to make it a separate PR to support them in the future.

[Code-Hex]

It's almost LGTM!

[Code-Hex]

If ctx.Done() fires before the reply, SendMessageWithReply returns and deletes the cgoReplyHandler, but the XPC runtime may call the callback later. In that case, callReplyHandler will reference the deleted cgo.Handle and panic. This only occurs when a timeout/cancellation occurs before the reply, so you may need to keep the handle alive until the callback is executed, or explicitly cancel the session/request.

Example code below:

type ReplyHandler func(*Dictionary, *RichError)

var replyHandlerRegistry struct {
	mu       sync.Mutex
	next     uint64
	handlers map[uintptr]ReplyHandler
}

func registerReplyHandler(handler ReplyHandler) uintptr {
	if handler == nil {
		return 0
	}
	key := uintptr(atomic.AddUint64(&replyHandlerRegistry.next, 1))
	replyHandlerRegistry.mu.Lock()
	if replyHandlerRegistry.handlers == nil {
		replyHandlerRegistry.handlers = make(map[uintptr]ReplyHandler)
	}
	replyHandlerRegistry.handlers[key] = handler
	replyHandlerRegistry.mu.Unlock()
	return key
}

func popReplyHandler(handle uintptr) ReplyHandler {
	if handle == 0 {
		return nil
	}
	replyHandlerRegistry.mu.Lock()
	handler := replyHandlerRegistry.handlers[handle]
	delete(replyHandlerRegistry.handlers, handle)
	replyHandlerRegistry.mu.Unlock()
	return handler
}

func deleteReplyHandler(handle uintptr) {
	if handle == 0 {
		return
	}
	replyHandlerRegistry.mu.Lock()
	delete(replyHandlerRegistry.handlers, handle)
	replyHandlerRegistry.mu.Unlock()
}

// callReplyHandler is called from C to handle reply messages.
//
//export callReplyHandler
func callReplyHandler(cgoReplyHandler uintptr, cgoReply, cgoError uintptr) {
	reply := unwrapObject[*Dictionary](uintptr(cgoReply))
	err := unwrapObject[*RichError](cgoError)
	handler := popReplyHandler(cgoReplyHandler)
	if handler == nil {
		if reply != nil {
			ReleaseOnCleanup(reply)
		}
		if err != nil {
			ReleaseOnCleanup(err)
		}
		return
	}
	handler(reply, err)
}

// SendMessageWithReply sends a message *[Dictionary] to the [Session] and waits for a reply *[Dictionary]. (macOS 13.0+)
//
// Use [context.Context] to control cancellation and timeouts.
//   - https://developer.apple.com/documentation/xpc/xpc_session_send_message_with_reply_async
func (s *Session) SendMessageWithReply(ctx context.Context, message *Dictionary) (*Dictionary, error) {
	replyCh := make(chan *Dictionary, 1)
	errCh := make(chan *RichError, 1)
	replyHandler := (ReplyHandler)(func(reply *Dictionary, err *RichError) {
		defer close(replyCh)
		defer close(errCh)
		if err != nil {
			errCh <- ReleaseOnCleanup(Retain(err))
		} else {
			replyCh <- ReleaseOnCleanup(Retain(reply))
		}
	})
	handle := registerReplyHandler(replyHandler)
	C.xpcSessionSendMessageWithReplyAsync(objc.Ptr(s), objc.Ptr(message), C.uintptr_t(handle))
	select {
	case reply := <-replyCh:
		return reply, nil
	case err := <-errCh:
		return nil, err
	case <-ctx.Done():
		deleteReplyHandler(handle)
		return nil, ctx.Err()
	}
}

There's probably a better way...

[Code-Hex]

A stream may return less than 4 bytes without error, you should read the full 4 bytes with io.ReadFull.

Suggested change

	// Wait until 4-byte header is read
	if _, err := conn.Read(v.backingBuffers[packetCount][:headerSize]); err != nil {
	return 0, fmt.Errorf("conn.Read failed: %w", err)
	}
	// Wait until 4-byte header is read
	if _, err := io.ReadFull(conn, v.backingBuffers[packetCount][:headerSize]); err != nil {
	return 0, fmt.Errorf("io.ReadFull failed: %w", err)
	}

[norio-nomura]

The conn here was created with net.FileConn(), and the Read implementation uses syscall.Read.
man 2 read has the following description.

read() and pread() will fail if the parameter nbyte exceeds INT_MAX, and they do not attempt a partial read.

[Code-Hex]

Sorry, please add this here 🙏

case 15.4:
		target = 150400 // __MAC_15_4
	case 26:
		target = 260000 // __MAC_26_0
	}

[norio-nomura]

The performance of FileAdaptorForInterface is not satisfactory, so I'm trying to rewrite it.