`run` fails to discover guests via mDNS when created in host mode

[tofugarden]

Description

The run script uses avahi as a mDNS responder on guests, but installing it requires network access. When operating in host mode, package install fails. The user needs to manually track down the guest's address.

% ./run fedora-host --operation-mode=host --distro=fedora                                                                        
[   0.032] INFO Starting vmnet-helper for 'fedora-host' with interface id '21e0b888-67c1-499e-b9c8-578be0806c66'                                                
[   0.105] INFO Creating image '/Users/tofugarden/.vmnet-helper/vms/fedora-host/disk.img'                                                                       
[   0.112] INFO Creating cloud-init iso '/Users/tofugarden/.vmnet-helper/vms/fedora-host/cidata.iso'                                                            
[   0.120] INFO Starting 'vfkit' virtual machine 'fedora-host' with mac address 'fe:84:f3:1c:ab:b5'                                                             
[   0.120] INFO Creating ssh config '/Users/tofugarden/.vmnet-helper/vms/fedora-host/ssh.config'                                                                
[  60.155] WARNING Timeout waiting for vm to become reachable  

Proposed solution

On fedora, debian and ubuntu, systemd-resolved is included in cloud images, and starts by default. Using cloud-init, we can install a simple configuration file to /etc/systemd/resolved.conf:

#cloud-config
write_files:
- path: /etc/systemd/resolved.conf
  content: |
    [Resolve]
    MulticastDNS=yes

This also speeds up cloud-init a bit on resource-constrained systems like github runners, since we don't have to wait for package installation.

Potential issues

This doesn't work on alpine, since they don't ship a mDNS responder in cloud images.

As @nirs suggested in a comment on #258, we could internally bring up the VM in shared mode, install the package, then restart it in host mode.

We could also simply issue a warning that says something along the lines of "no mDNS in host mode on alpine," and let the user find the address through leases, the serial.log file, or arp tables. Kind of a shame, but this is still better than timing out like we do now.

[nirs]

Using packages and network access is the easy way and it works with any distro supporting cloud-init. I would not spend time on limited solution that works only with some distros to support host mode. We want easy way to test host mode, but it does not mean that we need to create a vm in host mode.

I think the most practical way is to create the vm using shared mode (default), and then stop and start it in host mode. This should work now without any change, unless I broke it when adding mDNS. I don't really use host mode for anything.

[nirs]

Example host mode - vm was created in shared mode:

% ./run ubuntu --operation-mode host
[   0.048] INFO Starting vmnet-helper for 'ubuntu' with interface id '6bbc6ce9-5e26-4b39-bc9c-28c88776d519'
[   0.302] INFO Starting 'vfkit' virtual machine 'ubuntu' with mac address '82:e9:dd:3d:68:1f'
[   4.940] INFO VM is ready at ubuntu-vmnet-helper.local

Accessing the vm using mDNS:

% ping ubuntu-vmnet-helper.local
PING ubuntu-vmnet-helper.local (192.168.128.2): 56 data bytes
64 bytes from 192.168.128.2: icmp_seq=0 ttl=64 time=0.412 ms
64 bytes from 192.168.128.2: icmp_seq=1 ttl=64 time=0.394 ms
64 bytes from 192.168.128.2: icmp_seq=2 ttl=64 time=0.385 ms
^C
--- ubuntu-vmnet-helper.local ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.385/0.397/0.412/0.011 ms

Using ssh:

% ssh ubuntu
Welcome to Ubuntu 26.04 LTS (GNU/Linux 7.0.0-14-generic aarch64)

 * Documentation:  https://docs.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/pro

 System information as of Fri Jul  3 00:18:34 UTC 2026

  System load: 0.0                Memory usage: 27%   Processes:       119
  Usage of /:  13.5% of 18.33GB   Swap usage:   0%    Users logged in: 0


Expanded Security Maintenance for Applications is not enabled.

27 updates can be applied immediately.
10 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable

Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status


The list of available updates is more than a week old.
To check for new updates run: sudo apt update

Last login: Fri May  1 20:30:09 2026 from 192.168.64.1
ubuntu@ubuntu-vmnet-helper:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: enp0s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 82:e9:dd:3d:68:1f brd ff:ff:ff:ff:ff:ff
    altname enx82e9dd3d681f
    inet 192.168.128.2/24 metric 100 brd 192.168.128.255 scope global dynamic enp0s1
       valid_lft 460sec preferred_lft 460sec
    inet6 fdd2:db97:8516:6e6a:80e9:ddff:fe3d:681f/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 2591988sec preferred_lft 604788sec
    inet6 fe80::80e9:ddff:fe3d:681f/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever

[nirs]

@tofugarden So this issue is about not being to create a new vm in host mode, right?

[tofugarden]

@tofugarden So this issue is about not being to create a new vm in host mode, right?

Yes, correct. The VM creation workaround is okay, it does the job. I think it's a bit time consuming if someone wants to use this on i.e. a gh runner where it can take 2+ minutes to get a VM booted and avahid responding. But, not the end of the world if you want to say "not planned" :)

[nirs]

When #258 will be complete, we can use the static ip address to access the vm - so you will be able to create a vm in host mode. When you have ip address you don't really need mDNS. The ssh.config we generate can make login the same:

% cat ~/.vmnet-helper/vms/static-ip/ssh.config 

Host ubuntu
  StrictHostKeyChecking accept-new
  User ubuntu
  Hostname 192.168.200.128
  SetEnv LC_CTYPE=C.UTF-8

[tofugarden]

Yes I agree, this works.

[nirs]

@tofugarden another way: support customizable cloud-init, so other developers can use the suggest systemd-resolved for mDNS without changing the code.

It can be another option --mdns=avhai|systemd or --cloud-init=myconfig that will read the config from ~/.config/vment-helper/cloud-init/myconfig.yaml instead of using the builtin DISTROS. Not that this is really needed - choosing mdns implementation sounds easier to use and implement reliably.

BTW did you see https://github.com/nirs/vm? If we spend time on a tool, it should be the real tool, not the run script.

[tofugarden]

It can be another option --mdns=avhai|systemd

If you're open to this I would love to implement it real quick!

BTW did you see https://github.com/nirs/vm? If we spend time on a tool, it should be the real tool, not the run script.

I had not, I like the idea though! I would love this feature here to help me work on --network-id, but happy to help contribute there later. I'm not well-versed in Go, but I'm too deep in k8s stuff not to learn at some point!

I don't really use host mode for anything.

I'm using host mode to test an image builder for offline k8s clusters. Not using run, but anything to help test is valuable, I promise!

[nirs]

@tofugarden I think this should be resolved by documenting the limit in docs/development.md

[tofugarden]

@tofugarden I think this should be resolved by documenting the limit in docs/development.md

Ok fair enough! I can add it to the docs commit in #261.

[nirs]

@tofugarden I think this should be resolved by documenting the limit in docs/development.md

Ok fair enough! I can add it to the docs commit in #261.

Sure, or send another PR. Would be easier to focus on the ip address change and improve the docs later.