Add `--network-id` and related host address options by tofugarden · Pull Request #258 · nirs/vmnet-helper

tofugarden · 2026-06-21T19:45:32Z

These changes only apply to the host operation mode.

--network-id sets vmnet_network_identifier_key. This disables DHCP, and only allows the interface to communicate with other interfaces if they have the same network id. The host address and subnet mask can be set using --host-ip-address and --host-subnet-mask, which sets vmnet_host_ip_address_key and vmnet_host_subnet_mask_key respectively.

run was updated to accept a static IP address if all three options are set together, which allows us to determine the prefix length and validate the assigned address.

Fixes #120

nirs

@tofugarden Thanks! Let's try to simplify and minimize change in existing code.

Can you add more details on what does not work with ipv6? We can add a test mark as xfail to we know that does not work.

nirs · 2026-06-21T21:13:13Z

To simplify the change, I would start with adding --network-id in all layers without the options to set the host ip/ipv6/subnet. When this works locally and in the CI we can add the ipv4 options. Finally add the ipv6 options.

tofugarden · 2026-06-22T10:50:06Z

To simplify the change, I would start with adding --network-id in all layers without the options to set the host ip/ipv6/subnet. When this works locally and in the CI we can add the ipv4 options. Finally add the ipv6 options.

Thank you so much, this is great! I should have time to have the first of these smaller patches in by Sunday.

tofugarden · 2026-06-27T16:00:10Z

Comparing test results between macOS <26 & 26, and there seems to be inconsistent behavior that isn't documented as far as I can find.

Local testing on macOS 15.6.1 shows:

setting vmnet_network_identifier_key without vmnet_host_ip_address_key results in no address being assigned to the host interface
vmnet reports 0.0.0.0/0 as the interface subnet
Tests fail trying to get an ARP response from 0.0.0.0
setting vmnet_host_ip_address_key and vmnet_host_subnet_mask works as expected, and tests that set those keys pass

On macOS 26, a /24 subnet is selected by vmnet and the host gets the first address.

There is nothing explicitly incorrect about having a bridge with no host address I think? It's just a shame that users on <26 will need to manually deconflict subnets if --network-id is set. Docs will note this.

I think I'll write a test that starts two helpers without --host-ip-address (won't be on the next commit anyway) on the same --network-id, and has them ping each other to pass.

nirs · 2026-06-27T17:06:38Z

@tofugarden Don't worry about macOS < 26. macOS 27 will be released this year, and macOS 15 will become unsupported next year with macOS 28.

We can add a warning about using --network-id requires without --host-ip-address on macOS < 26. I would not enforce this since letting the system assign the network is the best way - it ensures you do't conflict with other application creating networks.

Let's focus now on adding --network-id without anything else. This can be nice little feature for next release.

nirs

@tofugarden Thanks! Looks good in quick look, I'll do a more detailed review later.

Please squash all commits and force push, and update the PR message to reflect the current change - adding --network-id option.

tofugarden · 2026-06-27T19:23:00Z

@nirs couple small fixes, getting on it now!

nirs

Check the CI failures:
https://github.com/nirs/vmnet-helper/actions/runs/28299280398

If you enable the actions on you fork you can run the tests by creating a pull request to your fork. In this repo I need to approve the workflow manually.

nirs

Nice to see green tests!

nirs · 2026-06-27T21:15:55Z

Tested locally with:

% pytest testing/helper_test.py --log-cli-level=debug -k network_id
====================================================================== test session starts =======================================================================
platform darwin -- Python 3.14.6, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/nir/src/vmnet-helper
configfile: pyproject.toml
collected 27 items / 25 deselected / 2 selected                                                                                                                  

testing/helper_test.py::TestStart::test_host_mode_network_id 
------------------------------------------------------------------------- live log call --------------------------------------------------------------------------
root: Starting vmnet-helper for 'test' with interface id '83fa1a6e-13ec-408f-ae44-2c26bc317160'
root: interface: {'vmnet_write_max_packets': 256, 'vmnet_read_max_packets': 256, 'vmnet_subnet_mask': '255.255.255.0', 'vmnet_mtu': 1500, 'vmnet_end_address': '192.168.129.254', 'vmnet_start_address': '192.168.129.1', 'vmnet_interface_id': '83FA1A6E-13EC-408F-AE44-2C26BC317160', 'vmnet_max_packet_size': 1514, 'vmnet_mac_address': '1a:ad:75:f7:ca:a2'}
PASSED                                                                                                                                                     [ 50%]
testing/helper_test.py::TestConnectivity::test_ping_host_network_id 
------------------------------------------------------------------------- live log call --------------------------------------------------------------------------
root: Starting vmnet-helper for 'test' with interface id '83fa1a6e-13ec-408f-ae44-2c26bc317160'
root: interface: {'vmnet_write_max_packets': 256, 'vmnet_read_max_packets': 256, 'vmnet_subnet_mask': '255.255.255.0', 'vmnet_mtu': 1500, 'vmnet_end_address': '192.168.129.254', 'vmnet_start_address': '192.168.129.1', 'vmnet_interface_id': '83FA1A6E-13EC-408F-AE44-2C26BC317160', 'vmnet_max_packet_size': 1514, 'vmnet_mac_address': '1a:ad:75:f7:ca:a2'}
test: Sent ARP request to 192.168.129.1
test: Got ARP reply in 0.003647 seconds
test: Sent ICMP echo request to 192.168.129.1
test: Got ICMP reply in 0.000493 seconds
PASSED                                                                                                                                                     [100%]

I ran this after the full tests, so the host network (192.168.128.0/24) was still used by vmnet-broker. We got a new network (192.168.129.0/24) and ping works.

nirs · 2026-06-27T21:24:42Z

The missing part to complete this work: learn how to use this with the run script to create 3 vms using host mode and 2 network ids:

vm1 network-id=C2C82CCF-F735-4060-9BCF-A68694158764
vm2 network-id=C2C82CCF-F735-4060-9BCF-A68694158764
vm3 network-id=0CA57A54-4A1C-4125-A4A7-56B87F53C240

vm1 and vm2 will be on same bridge (bridge100) and network (192.168.128.0/24) and can communicate.
vm3 will will be on another bridge (bridge101) and network (192.168.129.0/24) and it should be able to communicate with vm1 and vm2.

You may need to tweak the cloud-init configuration to assign static ip to the vms, or find how to add dhcpd to the network.

Or maybe you have another way to test this mode?

Testing this without --host-ip-address and --host-subnet-mask will be tricky, since you must provide static ip to the vm, but you cannot do without knowing which network you get from vmnet.

tofugarden · 2026-06-30T14:32:18Z

@nirs do you want a CI workflow for this, or just a working run? I'm finding it hard to reuse most of integration.yaml at first glance, but happy to duplicate some of it into a separate job in the same workflow.

tofugarden · 2026-06-30T20:24:03Z

Testing --network-id should now work in run:

% shared_id=$(uuidgen)
% for vm in test0 test1; do ./run $vm --operation-mode=host --network-id=$shared_id & done
[ ... ] INFO VM is ready at 192.168.128.209
[ ... ] INFO VM is ready at 192.168.128.184

% ./run test2 --operation-mode=host --network-id=$(uuidgen) &
[ ... ] INFO VM is ready at 192.168.129.111

% ssh ubuntu@192.168.209 ping -c 5 -t 10 192.168.128.184                                             
PING 192.168.128.184 (192.168.128.184) 56(84) bytes of data.                                              
64 bytes from 192.168.128.184: icmp_seq=1 ttl=64 time=0.319 ms                                            
64 bytes from 192.168.128.184: icmp_seq=2 ttl=64 time=0.469 ms                                            
64 bytes from 192.168.128.184: icmp_seq=3 ttl=64 time=0.610 ms                                            
64 bytes from 192.168.128.184: icmp_seq=4 ttl=64 time=1.02 ms                                             
64 bytes from 192.168.128.184: icmp_seq=5 ttl=64 time=0.659 ms

% ssh ubuntu@192.168.128.209 ping -c 5 -t 10 192.168.129.111
ping: connect: Network is unreachable

Beware of flakiness due to address conflicts, my IPAM strategy without --host-ip-address is akin to rolling a dice. Also note that VMs declared this way are essentially ephemeral, since changes to their interface's subnet and resulting IP address are not propagated across reboot.

I tried writing a workflow, but I think I kept running out of memory, many runs hung indefinitely or timed out despite the artifacts looking correct. I was able to get things to work once on macos-26-intel, but never again. Haven't tried the large runners yet, they may work.

nirs · 2026-06-30T20:54:21Z

@nirs do you want a CI workflow for this, or just a working run? I'm finding it hard to reuse most of integration.yaml at first glance, but happy to duplicate some of it into a separate job in the same workflow.

No need to update or add a workflow.

The first step is testing this manually, maybe modding the run script to support static ip. This is good enough to prove that this feature works. This is enough to review this change.

The next step is to document how to test this feature in docs/development.md. This should allow developers to test this feature manually. This is enough o merge this change.

The next step is to modify the run script to support static ip. The vm should annoys the mDNS name and the ip address, allowing access to the vm using mDNS. If mDNS does not work, the user can use the static ip. It can work like this:

./run ubuntu --static-ip=192.168.200.2 --network-id FD1D6DEF-CD09-436A-AA29-FDD23960B1AB

This will not work before we implement --host-ip-address and --host-subnet-mask, and optional, since this is an esoteric feature.

The last step is adding CI for this, but this will be very hard. We cannot create vms in host mode since we depend on installing packages. So this means creating vm in shared mode, stopping it and starting it in host mode. This is very slow and we are limited to 5 concurrent runners. I don't want to make the slow CI run 2-3 times slower for testing esoteric feature.

If we will have faster runners that can create a vm in 15 seconds and start it in 5 seconds, it can become feasible to test this in the CI.

nirs · 2026-06-30T21:02:25Z

Testing --network-id should now work in run:

% shared_id=$(uuidgen)
% for vm in test0 test1; do ./run $vm --operation-mode=host --network-id=$shared_id & done
[ ... ] INFO VM is ready at 192.168.128.209
[ ... ] INFO VM is ready at 192.168.128.184

% ./run test2 --operation-mode=host --network-id=$(uuidgen) &
[ ... ] INFO VM is ready at 192.168.129.111

% ssh ubuntu@192.168.209 ping -c 5 -t 10 192.168.128.184                                             
PING 192.168.128.184 (192.168.128.184) 56(84) bytes of data.                                              
64 bytes from 192.168.128.184: icmp_seq=1 ttl=64 time=0.319 ms                                            
64 bytes from 192.168.128.184: icmp_seq=2 ttl=64 time=0.469 ms                                            
64 bytes from 192.168.128.184: icmp_seq=3 ttl=64 time=0.610 ms                                            
64 bytes from 192.168.128.184: icmp_seq=4 ttl=64 time=1.02 ms                                             
64 bytes from 192.168.128.184: icmp_seq=5 ttl=64 time=0.659 ms

% ssh ubuntu@192.168.128.209 ping -c 5 -t 10 192.168.129.111
ping: connect: Network is unreachable

Nice!

But assigning random tips is not needed - this will be much simpler:

./run vm1 --operation-mode=host --network-id=$net1 --ip-address 192.168.128.2
./run vm2 --operation-mode=host --network-id=$net1 --ip-address 192.168.128.3
./run vm2 --operation-mode=host --network-id=$net2 --ip-address 192.168.129.2

It cannot be reliable before we support --host-ip-address and --host-subnet-mask, depending on getting networks 192.168.128.0/24 and 192.168.129.0/24.

Beware of flakiness due to address conflicts, my IPAM strategy without --host-ip-address is akin to rolling a dice. Also note that VMs declared this way are essentially ephemeral, since changes to their interface's subnet and resulting IP address are not propagated across reboot.

If we set the static ip using cloud-init the ip address will remain when stating the vm.

I tried writing a workflow, but I think I kept running out of memory, many runs hung indefinitely or timed out despite the artifacts looking correct.

Don't waste time on this, this will be very hard, very slow, and not reliable enough on the old GitHub intel runners.

I was able to get things to work once on macos-26-intel, but never again. Haven't tried the large runners yet, they may work.

macos-26-intel sounds interesting - if they are available we should switch to these runners.

nirs

Let's amend the first commit with the validation fixes.

tofugarden · 2026-06-30T21:50:01Z

It cannot be reliable before we support --host-ip-address and --host-subnet-mask, depending on getting networks 192.168.128.0/24 and 192.168.129.0/24.

Ok I misunderstood, I thought we wanted a working run with --network-id before we add --host-ip-address. Setting --network-id without --host-ip-address is valid and desirable for the helper, so I wanted that to be available in run. If that's not needed, I'll get started on host IP address assignment and implement that instead.

If we set the static ip using cloud-init the ip address will remain when stating the vm.

This was causing flakiness because vmnet sometimes selects a new subnet upon restart, and then the guest tries to use an address from a different subnet. But if we don't want automatic subnet selection in run, this won't be a problem.

Creating vm requires installing packages so it cannot work in host mode. We can create a vm in shared mode, stop it , and start it again in host mode.

I think systemd-resolved ships with images for fedora & ubuntu, and has mDNS support. I'll try it out. Alpine doesn't, but that's a great workaround!

nirs · 2026-07-01T09:37:15Z

Creating vm requires installing packages so it cannot work in host mode. We can create a vm in shared mode, stop it , and start it again in host mode.

I think systemd-resolved ships with images for fedora & ubuntu, and has mDNS support. I'll try it out. Alpine doesn't, but that's a great workaround!

One issue - when restarting a vm cloud-init will not reconfigure the vm unless we generate new cloud-init disk (with new metadata instance-id). So to reconfigure the ip address in host mode we must:

delete current cidata.iso, or force creating new one - this can be useful option regardless of --network-id. I think deleting the cidata.iso is good enough, but we can also add a flag like --reconfigure. We can use --host-ip-address to force reconfiguration instead of adding a new flag.
set static ip and disable DHCP in network-config (what you did in the last commit)
we may need to disable packages and runcmd sections since in host mode we don't have access to the package manager repository. In the last commit you disabled all cloud int configuration which works with the current code. This should be triggered by --operation-id=host - not the --network-id option.

Example flow:

# Create a new vm in shared mode, Control+C to stop
./run vm1 --distro ubuntu

# vm starts in host mode with the requested ip
NET1_UUID=D637004F-D2FC-4C44-940A-12FB093D92F1
NET1_IP1=192.168.200.2
NET1_MASK=255.255.255.0
./run vm1 --operation-mode host --network-id $NET1_UUID --host-ip-address $NET1_IP1 --host-subnet-mask $NET1_MASK

We can document this flow in docs/developement.md

I suggest you start with the docs changes before spending time on the implementation. We should agree on the direction we go first.

tofugarden · 2026-07-01T11:04:31Z

I suggest you start with the docs changes before spending time on the implementation. We should agree on the direction we go first.

Ok! What should the scope of this PR be? Do you plan to merge with --host-ip-address in run but not in the helper, and add it in a future PR? Or is proper --host-ip-address support required work to merge? I was under the impression we just wanted --network-id for now.

nirs · 2026-07-01T18:48:28Z

I suggest you start with the docs changes before spending time on the implementation. We should agree on the direction we go first.

Ok! What should the scope of this PR be? Do you plan to merge with --host-ip-address in run but not in the helper, and add it in a future PR? Or is proper --host-ip-address support required work to merge? I was under the impression we just wanted --network-id for now.

I think we will have to release --network-id with --host-ip-address and --host-subnet-mask. There is no way to work with static ip if you don't control the network.

You can simply add the next commit with the docs. When we agree on how it should work we can implement it. We can squash the docs commit and implementation later if needed.

tofugarden · 2026-07-02T01:48:50Z

Also it might make sense for me to do two different PRs related to ./run first:

mDNS with --operation-mode=host
static --guest-ip-address outside the DHCP range

Neither of these are strictly dependent on each other, or --network-id, but we need them to test it.

nirs · 2026-07-02T23:25:11Z

Also it might make sense for me to do two different PRs related to ./run first:

mDNS with --operation-mode=host

It should work now - can you share your plan on this change? You can open another issue to describe the change.

static --guest-ip-address outside the DHCP range

Interesting, can be useful feature to use shared network with dhcp but use static ip. Open another issue for this? Show example usage and explain the use case.

nirs · 2026-07-29T20:24:51Z

@tofugarden We can finish this change now.

These changes only apply to the host operation mode. --network-id sets vmnet_network_identifier_key. This disables DHCP, and only allows the interface to communicate with other interfaces if they have the same network id. The host address and subnet mask can be set using --host-ip-address and --host-subnet-mask, which sets vmnet_host_ip_address_key and vmnet_host_subnet_mask_key respectively.

tofugarden · 2026-07-31T16:17:52Z

Still WIP due to missing IPv6, but changes so far are ready for review :)

nirs

Partial review. I'll do a full review later.

nirs · 2026-07-31T18:20:56Z

            xpc_dictionary_set_string(desc, vmnet_subnet_mask_key, options.subnet_mask);
        }
        xpc_dictionary_set_bool(desc, vmnet_enable_isolation_key, options.enable_isolation);
+        if (!uuid_is_null(options.network_id)) {


We cannot have start_address and network_id, host_ip_* at the same time, right?

The code try to handle both at the same time. We can use this to make the intent more clear:

if (options.start_address != NULL) { add address options... } else if (!uuid_is_null(options.network_id)) { add optional host_ip_* otions... }

nirs · 2026-07-31T18:21:38Z

 "                 [--operation-mode shared|bridged|host] [--shared-interface NAME]\n"
 "                 [--start-address ADDR] [--end-address ADDR] [--subnet-mask MASK]\n"
+"                 [--network-id UUID] [--host-ip-address ADDR]\n"
+"                 [--host-subnet-mask MASK]\n"


Less keep these in the same line - they are releated.

tofugarden marked this pull request as draft June 21, 2026 19:45

tofugarden mentioned this pull request Jun 21, 2026

Consider adding --network-identifier option #120

Open

nirs reviewed Jun 21, 2026

View reviewed changes

Comment thread testing/helper_test.py Outdated

Comment thread testing/helper_test.py Outdated

Comment thread testing/helper_test.py Outdated

nirs reviewed Jun 21, 2026

View reviewed changes

Comment thread testing/helper.py Outdated

nirs reviewed Jun 27, 2026

View reviewed changes

Comment thread run Outdated

Comment thread docs/integration.md Outdated

nirs reviewed Jun 27, 2026

View reviewed changes

Comment thread programs/helper.c

Comment thread testing/helper_test.py

Comment thread programs/helper.c Outdated

tofugarden force-pushed the feat/network-id branch from 18f8b50 to a212756 Compare June 27, 2026 19:58

tofugarden changed the title ~~WIP: Add --network-id and related host address options~~ Add --network-id option Jun 27, 2026

tofugarden marked this pull request as ready for review June 27, 2026 20:01

nirs requested changes Jun 27, 2026

View reviewed changes

tofugarden commented Jun 30, 2026

View reviewed changes

Comment thread run Outdated

tofugarden marked this pull request as draft June 30, 2026 20:32

nirs reviewed Jun 30, 2026

View reviewed changes

Comment thread testing/cidata.py Outdated

Comment thread testing/vm.py Outdated

Comment thread testing/vm.py Outdated

Comment thread run Outdated

Comment thread run Outdated

Comment thread run Outdated

Comment thread run Outdated

tofugarden commented Jul 2, 2026

View reviewed changes

Comment thread docs/development.md Outdated

tofugarden commented Jul 2, 2026

View reviewed changes

Comment thread docs/development.md Outdated

This was referenced Jul 3, 2026

run fails to discover guests via mDNS when created in host mode #259

Closed

run doesn't support static IP address assignment #260

Closed

tofugarden mentioned this pull request Jul 16, 2026

run: add --ip-address option #261

Merged

tofugarden force-pushed the feat/network-id branch from 93b63aa to 0c5a387 Compare July 31, 2026 16:13

tofugarden changed the title ~~Add --network-id option~~ Add --network-id and related host address options Jul 31, 2026

nirs reviewed Jul 31, 2026

View reviewed changes

Conversation

tofugarden commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nirs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nirs commented Jun 21, 2026

Uh oh!

tofugarden commented Jun 22, 2026

Uh oh!

tofugarden commented Jun 27, 2026

Uh oh!

nirs commented Jun 27, 2026

Uh oh!

nirs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tofugarden commented Jun 27, 2026

Uh oh!

nirs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nirs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nirs commented Jun 27, 2026

Uh oh!

nirs commented Jun 27, 2026

Uh oh!

tofugarden commented Jun 30, 2026

Uh oh!

tofugarden commented Jun 30, 2026

Uh oh!

Uh oh!

nirs commented Jun 30, 2026

Uh oh!

nirs commented Jun 30, 2026

Uh oh!

nirs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tofugarden commented Jun 30, 2026

Uh oh!

tofugarden commented Jun 21, 2026 •

edited

Loading