Realtek 5 GbE USB Network Dongles (WisdPi, but generic issue with 8157) cause NFS hangs and kernel crashes

[jerryabramson]

About 8 months ago I started playing around with the 8157 USB 5 GbE dongles from WisdPi and Proxmox.

A few other people have tried these dongles on Proxmox, with forum posts on both proxmox.com and servethehome.com

The first requirement was to be on Kernel 6.11 or newer, and to compile the source code from Realtek.

At the time I tried the 2.19.2 drivers that were released last fall.

For typical Networking issues, I had no real issues, and saw about 4.7 Gbs upload/downloads, and there were no stability issues.

I then tried to move over my Proxmox server that uses NFS to host many of the QCOW2 files for my cluster.

• Note I had been hosting these using 2.5 GbE USB dongles for several years without any issues.

I then move over my cluster to 5 GbE dongles and a quality Microtik 10 GbE switch.

With the 5 GbE dongles, and the driver built from Realtek source (2.19.2) the systems then became incredibly unstable.

There were many kernel hangs, both on the nodes acting as NFS clients, as well as the node acting as an NFS server.

Many QCOW2 files became corrupted (I had plenty of backups, so no loss except for downtime).

I am hesitant to try out either the 2.20.1 driver from Realtek, or either your driver, or the forked wget driver.

There was a hint in one of the forum posts upload NFS fragmentation, and hinted it might be an NFS issue, and not related to the 5 GbE dongles.

I am suspicious of this theory, since I moved over the cluster to OWC 10 GbE thunderbolt dongles, and the system has been very stable for 9+ months.

It definitely has something to do with an NFS interaction with the Realtek drivers.

Any ideas??

• https://forums.servethehome.com/index.php?threads/realtek-rtl8157-5-gbe-adapters-with-10-gb-usb-interface-wavlink-wisdpi.46098/
• https://forums.servethehome.com/index.php?threads/realtek-rtl8157-5-gbe-adapters-with-10-gb-usb-interface-wavlink-wisdpi.46098/post-467917
• https://forum.proxmox.com/threads/opt-in-linux-6-11-kernel-for-proxmox-ve-8-available-on-test-no-subscription.156818/post-780133

[hcartiaux]

Hey, first, I'm not a kernel/network driver programmer myself, this repository was made initially to host a copy a the upstream realtek driver sources for packaging it on Archlinux User Repositories. It also contains occasional fixes, when it does not build on a recent kernel.

I personally have no stability issue with my use cases, and can't reproduce yours. If you want to have a small chance to have an answer, specify:

• your dongle model and chipset, with the version of firmware
• your r8152 module version (version and commit hash)
• your distribution and kernel version (uname -r)

I would personally try to reproduce on the latest version before investigating ;)

That being said, have you read/tried this pull request ?

[ristomatti]

@jerryabramson

I am hesitant to try out either the 2.20.1 driver from Realtek, or either your driver, or the forked wget driver.

I've tried with the patched version 2.20.1 from https://github.com/awesometic/realtek-r8152-dkms. Based on a quick diff I didn't spot any difference in what was patched, except for the patch for kernel 6.16.0 only found in this repository.

I also have both 2.5Gbit rtl8156 and 5Gbit rtl8157 variants from Ugreen. I was luckily just experimeting when I ran into this issue. I had one Proxmox node (pve1) with the 5Gbit adapter and another (pve2) with the 2.5Gbit adapter. Both had the patched driver version 2.20.1 installed and were running kernel 6.14.11-2-pve.

My test was to live migrate a VM running Router OS back and forth between two the two cluster nodes, while checking if the Router OS web UI stays responsive.

Test 1: pve1 as NFS server, pve2 as client -> CPU soft lock on pve1, pve2 survives
Test 2: pve2 as NFS server, pve1 as client -> CPU soft lock on pve2, pve1 survives

After seeing this, I uninstalled driver v2.20.1 from pve2, leaving it with v1.12.13 packaged with the kernel. I was then able to repeat test 2. without issues.

I admit the test was rather brief and I might have run into issues running it longer. Regardless, the test showed both hosts experience a CPU soft lock when running NFS server with driver v2.20.1 installed. This was regardless of which adapter was in use. Both crashes occurred after or during a single back and forth live migration cycle.

Edit: Driver v2.20.1 does indeed cause instability with NFS client traffic as well.

[j-hudecek]

Same problem here: CPU Soft Lockup with Sitecom and Ugreen 2.5Gbit RTL8156 and Ubiquiti 5gbps RTL8157 connected to TL-SG105PP-M2 on Ubuntu 24.04 6.14.0-33 using v2.19 or v2.20 (b4effa4)
echo disable > /sys/class/net/enx..../rtl_adv/sg_en seems to fix it (at least for ssh, iperf and web traffic)

[ristomatti]

@j-hudecek I could run iperf3 just fine without the patch. NFS was the only consistent way to run into the problem.

[ristomatti]

Realtek has released a new driver compatible with kernel versions up to 6.17. I'm now running it on 3 hosts, two of which have the 5Gbit version. I've ran some file transfers over NFS as a quick test. So far no errors/warnings of any kind shown on dmesg. I'll try to remember to report back after running it for a while.

[popic74]

I'm also testing the new version of the 2.21.4 driver. It works great. No more kernel crashes on Debian 12 with kernel 6.12.

[sgueko]

Hello,

I also experienced crashes with the R8157, AMD chipset. I made a driver modification that resolved my issue.

My first attempt was to disable scatter gather (tp->sg_use = false) everywhere. Disabling it at the system level didn't work.

This modification was a bit too drastic, and after a Chih-Kai exchange with Realtek, the problem stemmed from RTL_MAX_SG_NUM and RTL8152_MAX_RX.

I set RTL_MAX_SG_NUM to 16 and RTL8152_MAX_RX to 8.

Now everything is stable. I've run several throughput tests and haven't had any crashes.