[GH-ISSUE #51] kernel: non-blocking send sleeps waiting for mbufs under exhaustion, wedging the system #43

New issue

Closed

opened 2026-11-03 22:03:09 -07:00 by gitea-mirror · 2 comments

gitea-mirror commented

2026-11-03 22:03:09 -07:00

Owner

Originally created by @raynorpat on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/51

Symptom

stress-net flood (opt-in level 2) opens ~3500 unix and loopback TCP socket pairs until ENFILE, sets every writer O_NONBLOCK and writes until EWOULDBLOCK/ENOBUFS. The process never gets past the fill phase (the stress watchdog times it out at -t secs), and sshd stops answering (connection accepted by the QEMU forward but no SSH banner) for as long as the process lives; once it hung for more than 10 minutes until the guest was cycled. After the watchdog kill the guest recovers; netstat -m then showed "requests for memory delayed" in the hundreds. The first 6 pairs filled normally (unix 4-8 KB, TCP ~64 KB each), so write itself returns EWOULDBLOCK correctly until the mbuf pool runs dry.

Reproduce

On the dev guest (python vm/stress_dev.py up/sync, STRESS_DEV_PORTS=2563,4563,1263), in src/Commands/stress after make:
STRESS_EXHAUST=2 ./net/stress-net -t 90 flood (run it detached; ssh is dead while it runs). Level 1 (STRESS_EXHAUST=1) uses a bounded fill and passes in 1.4 s.

Where to look

src/kernel-7/bsd/kern/uipc_socket.c sosend(): the data loop allocates with MGETHDR(m, M_WAIT, ...) (line ~673), MGET(m, M_WAIT, ...) (~678) and MCLGET(m, M_WAIT) (~682) regardless of SS_NBIO; SS_NBIO is only consulted for the sockbuf-space wait (~651, 880), not for mbuf allocation. Also m_get(M_WAIT) at ~802.
src/kernel-7/bsd/kern/uipc_mbuf.c m_retry() (~215-250): with canwait == M_WAIT it loops forever, calling m_expand/m_reclaim and then tsleep("m_retry", hz) until an mbuf is freed; nothing frees one when the sleepers are the only consumers.
The pool limit is nmbclusters (NMBCLUSTERS, uipc_mbuf.c:106, mbuf.h:406); a few hundred full TCP socket buffers exhaust it.

Impact

Any process, even one using non-blocking sockets, can sleep indefinitely in sosend when the mbuf pool is exhausted, and because the pool is global the whole network stack (sshd, new connections) stalls with it. A non-blocking send should fail with EWOULDBLOCK/ENOBUFS (M_DONTWAIT when SS_NBIO or MSG_DONTWAIT is set) instead of sleeping.

Found by the stress suite (src/Commands/stress, branch stress-phase3: net flood, phase 3). Related: #43, #45, also found by the stress suite.

Originally created by @raynorpat on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/51 ## Symptom `stress-net flood` (opt-in level 2) opens ~3500 unix and loopback TCP socket pairs until ENFILE, sets every writer O_NONBLOCK and writes until EWOULDBLOCK/ENOBUFS. The process never gets past the fill phase (the stress watchdog times it out at -t secs), and sshd stops answering (connection accepted by the QEMU forward but no SSH banner) for as long as the process lives; once it hung for more than 10 minutes until the guest was cycled. After the watchdog kill the guest recovers; netstat -m then showed "requests for memory delayed" in the hundreds. The first 6 pairs filled normally (unix 4-8 KB, TCP ~64 KB each), so write itself returns EWOULDBLOCK correctly until the mbuf pool runs dry. ## Reproduce On the dev guest (python vm/stress_dev.py up/sync, STRESS_DEV_PORTS=2563,4563,1263), in src/Commands/stress after `make`: `STRESS_EXHAUST=2 ./net/stress-net -t 90 flood` (run it detached; ssh is dead while it runs). Level 1 (`STRESS_EXHAUST=1`) uses a bounded fill and passes in 1.4 s. ## Where to look - src/kernel-7/bsd/kern/uipc_socket.c sosend(): the data loop allocates with `MGETHDR(m, M_WAIT, ...)` (line ~673), `MGET(m, M_WAIT, ...)` (~678) and `MCLGET(m, M_WAIT)` (~682) regardless of SS_NBIO; SS_NBIO is only consulted for the sockbuf-space wait (~651, 880), not for mbuf allocation. Also m_get(M_WAIT) at ~802. - src/kernel-7/bsd/kern/uipc_mbuf.c m_retry() (~215-250): with canwait == M_WAIT it loops forever, calling m_expand/m_reclaim and then tsleep("m_retry", hz) until an mbuf is freed; nothing frees one when the sleepers are the only consumers. - The pool limit is nmbclusters (NMBCLUSTERS, uipc_mbuf.c:106, mbuf.h:406); a few hundred full TCP socket buffers exhaust it. ## Impact Any process, even one using non-blocking sockets, can sleep indefinitely in sosend when the mbuf pool is exhausted, and because the pool is global the whole network stack (sshd, new connections) stalls with it. A non-blocking send should fail with EWOULDBLOCK/ENOBUFS (M_DONTWAIT when SS_NBIO or MSG_DONTWAIT is set) instead of sleeping. Found by the stress suite (`src/Commands/stress`, branch `stress-phase3`: `net flood`, phase 3). Related: #43, #45, also found by the stress suite.

gitea-mirror closed this issue

2026-11-03 22:03:09 -07:00

gitea-mirror commented

2026-11-03 22:03:10 -07:00

Author

Owner

@raynorpat commented on GitHub (Oct 15, 2026):

Fix

sosend() now asks for mbufs with M_DONTWAIT when the socket is non-blocking (SS_NBIO or MSG_DONTWAIT), and handles a failed allocation instead of sleeping in m_retry:

Atomic (datagram) sockets: return ENOBUFS. Nothing is sent.
Stream sockets: any bytes already copied into the mbuf chain are passed to the protocol first, then the call returns EWOULDBLOCK. write() therefore reports a short count that matches exactly what was sent: no lost or duplicated bytes. If nothing was copied, it returns -1/EWOULDBLOCK.
soreceive()'s MSG_OOB path gets the same non-blocking rule and returns ENOBUFS.

Blocking sockets are unchanged (M_WAIT). MCLGET already falls back to a plain mbuf when no cluster is free, so it needed no change.

Commit: kernel: non-blocking sends fail with EWOULDBLOCK/ENOBUFS instead of sleeping for mbufs (#51), a single-file change to src/kernel-7/bsd/kern/uipc_socket.c. It is merged into ffs-perf locally. Neither branch is pushed yet, and it is not on master.

Verification

Tested on a -snapshot guest of rhap-i386-bootstrapped-v2.img, i386 RELEASE kernel built with rbuild kernel. The stress suite comes from stress-phase3.

	`STRESS_EXHAUST=2 stress-net -t 90 flood`	sshd during run
stock v2 kernel	`FAIL reason=timeout secs=90`	no banner for ~9 min
patched	`PASS secs=8.9` (pairs=3496, 13.9 MB), then `PASS secs=8.2`	banner every probe

Also on the patched kernel:

STRESS_EXHAUST=1 flood passes.
-x fails as intended (the byte checker works).
tcp, udp, unix, select, accept, fdpass and storm all pass.
No new compiler warnings in uipc_socket.c.

Not covered: no ppc build was done. The kernel tested was master-based (d03ecae8f). The uipc_socket.c merged into ffs-perf is byte-identical, but the ffs-perf kernel as a whole has not been boot-tested with it.

Notes for the stress-phase3 branch:

Its README and the t_flood comment still say level 2 is expected to wedge current kernels.
After a -x (early-exit) run, ~2800 loopback TCP connections sit in FIN_WAIT_1 holding ~940 clusters, because that path skips the SO_LINGER 0 close. This is classic BSD zero-window close behaviour and is unrelated to this fix.

@raynorpat commented on GitHub (Oct 15, 2026): ## Fix `sosend()` now asks for mbufs with `M_DONTWAIT` when the socket is non-blocking (`SS_NBIO` or `MSG_DONTWAIT`), and handles a failed allocation instead of sleeping in `m_retry`: - **Atomic (datagram) sockets:** return `ENOBUFS`. Nothing is sent. - **Stream sockets:** any bytes already copied into the mbuf chain are passed to the protocol first, then the call returns `EWOULDBLOCK`. `write()` therefore reports a short count that matches exactly what was sent: no lost or duplicated bytes. If nothing was copied, it returns -1/`EWOULDBLOCK`. - `soreceive()`'s `MSG_OOB` path gets the same non-blocking rule and returns `ENOBUFS`. Blocking sockets are unchanged (`M_WAIT`). `MCLGET` already falls back to a plain mbuf when no cluster is free, so it needed no change. Commit: `kernel: non-blocking sends fail with EWOULDBLOCK/ENOBUFS instead of sleeping for mbufs (#51)`, a single-file change to `src/kernel-7/bsd/kern/uipc_socket.c`. It is merged into `ffs-perf` locally. Neither branch is pushed yet, and it is not on master. ## Verification Tested on a `-snapshot` guest of `rhap-i386-bootstrapped-v2.img`, i386 RELEASE kernel built with `rbuild kernel`. The stress suite comes from `stress-phase3`. | | `STRESS_EXHAUST=2 stress-net -t 90 flood` | sshd during run | |---|---|---| | stock v2 kernel | `FAIL reason=timeout secs=90` | no banner for ~9 min | | patched | `PASS secs=8.9` (pairs=3496, 13.9 MB), then `PASS secs=8.2` | banner every probe | Also on the patched kernel: - `STRESS_EXHAUST=1` flood passes. - `-x` fails as intended (the byte checker works). - tcp, udp, unix, select, accept, fdpass and storm all pass. - No new compiler warnings in `uipc_socket.c`. Not covered: no ppc build was done. The kernel tested was master-based (d03ecae8f). The `uipc_socket.c` merged into `ffs-perf` is byte-identical, but the ffs-perf kernel as a whole has not been boot-tested with it. Notes for the stress-phase3 branch: - Its README and the `t_flood` comment still say level 2 is expected to wedge current kernels. - After a `-x` (early-exit) run, ~2800 loopback TCP connections sit in FIN_WAIT_1 holding ~940 clusters, because that path skips the `SO_LINGER 0` close. This is classic BSD zero-window close behaviour and is unrelated to this fix.

gitea-mirror commented

2026-11-03 22:03:11 -07:00

Author

Owner

@raynorpat commented on GitHub (Oct 16, 2026):

Fixed in ffa97ff591 (Merge branch 'issue-51' into ffs-perf), commit a482d9573: non-blocking sends fail with EWOULDBLOCK/ENOBUFS instead of sleeping for mbufs.

In sosend() (bsd/kern/uipc_socket.c):

Header mbuf: a socket with SS_NBIO, or a send with MSG_DONTWAIT, now allocates with M_DONTWAIT. If none is available, the send fails with ENOBUFS for atomic sends and EWOULDBLOCK otherwise.
Later mbufs: if one of the following data mbufs can't be allocated, a non-atomic send delivers the bytes already copied and returns EWOULDBLOCK.
Out-of-band: soreceive()'s out-of-band m_get follows the same rule.

The regression check is the stress suite's STRESS_EXHAUST=2 net/stress-net flood (on ffs-perf). Before the fix it wedged the guest. After the fix it should either PASS or FAIL cleanly with the system still answering ssh. I haven't run that check against a kernel containing this fix in this session.

@raynorpat commented on GitHub (Oct 16, 2026): Fixed in ffa97ff59161295df4a489e1639d16e8ea481be1 (Merge branch 'issue-51' into ffs-perf), commit a482d9573: non-blocking sends fail with EWOULDBLOCK/ENOBUFS instead of sleeping for mbufs. In `sosend()` (`bsd/kern/uipc_socket.c`): - **Header mbuf:** a socket with `SS_NBIO`, or a send with `MSG_DONTWAIT`, now allocates with `M_DONTWAIT`. If none is available, the send fails with ENOBUFS for atomic sends and EWOULDBLOCK otherwise. - **Later mbufs:** if one of the following data mbufs can't be allocated, a non-atomic send delivers the bytes already copied and returns EWOULDBLOCK. - **Out-of-band:** `soreceive()`'s out-of-band `m_get` follows the same rule. The regression check is the stress suite's `STRESS_EXHAUST=2 net/stress-net flood` (on ffs-perf). Before the fix it wedged the guest. After the fix it should either PASS or FAIL cleanly with the system still answering ssh. I haven't run that check against a kernel containing this fix in this session.

gitea-mirror referenced this issue

2026-11-03 22:03:16 -07:00

[GH-ISSUE #60] kernel: data written through a shared mapping and never msync'd is lost at unmount (UFS, msdosfs) #53