[GH-ISSUE #85] kernel: vnode_pageout reads v_vm_info->vnode_size without a NULL check after a forced reclaim #77

New issue

Closed

opened 2026-11-03 22:04:13 -07:00 by gitea-mirror · 1 comment

gitea-mirror commented

2026-11-03 22:04:13 -07:00

Owner

Originally created by @raynorpat on GitHub (Oct 30, 2026).
Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/85

Left open by #84 (and #81).

vnode_pageout (vm/vnode_pager.c ~863-867, ffs-perf a2a72069f) clamps the pageout size against vp->v_vm_info->vnode_size before it locks the vnode, with no check that v_vm_info is still there.

A user mmap object (or exec text) can outlive its vnode's vm_info: a forced unmount's vclean runs vm_info_free and sets v_vm_info to NULL while the VM object still has dirty pages. A later pageout of such a page (pageout daemon, object terminate, msync) then dereferences NULL.

#81 already guards the error store after VOP_PAGEOUT (~927), and #84 guards both reads in vnode_pagein (~757/788). This is the remaining unguarded read in the pager.

Expected: with v_vm_info NULL, skip the clamp and let the write fail through deadfs, so the page is dropped and the pageout returns PAGER_ERROR, as vnode_pagein now does. No panic.

Reproduce: mmap a file shared and writable from a forced-unmountable mount (loopback NFS or a scratch UFS), dirty pages, umount -f, then force a pageout (memory pressure or msync on the stale mapping).

Originally created by @raynorpat on GitHub (Oct 30, 2026). Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/85 Left open by #84 (and #81). `vnode_pageout` (`vm/vnode_pager.c` ~863-867, ffs-perf a2a72069f) clamps the pageout size against `vp->v_vm_info->vnode_size` before it locks the vnode, with no check that `v_vm_info` is still there. A user mmap object (or exec text) can outlive its vnode's vm_info: a forced unmount's vclean runs `vm_info_free` and sets `v_vm_info` to NULL while the VM object still has dirty pages. A later pageout of such a page (pageout daemon, object terminate, msync) then dereferences NULL. #81 already guards the error store after `VOP_PAGEOUT` (~927), and #84 guards both reads in `vnode_pagein` (~757/788). This is the remaining unguarded read in the pager. **Expected:** with `v_vm_info` NULL, skip the clamp and let the write fail through deadfs, so the page is dropped and the pageout returns PAGER_ERROR, as `vnode_pagein` now does. No panic. **Reproduce:** mmap a file shared and writable from a forced-unmountable mount (loopback NFS or a scratch UFS), dirty pages, `umount -f`, then force a pageout (memory pressure or `msync` on the stale mapping).

gitea-mirror closed this issue

2026-11-03 22:04:14 -07:00

gitea-mirror commented

2026-11-03 22:04:15 -07:00

Author

Owner

@raynorpat commented on GitHub (Oct 31, 2026):

Fixed by 278d00c75, merged into ffs-perf in 23c1dd5f6.

Root cause. vnode_pageout's MACH_NBC size clamp (vm/vnode_pager.c :855-867) read vp->v_vm_info->vnode_size after a forced reclaim had set v_vm_info to NULL. That happens on every pageout of a stale page, not just in a race.

On i386, address 0 is the real-mode IVT. A probe kernel read 0xf000ff54 for all 16 dirty pages, once at msync and once at munmap.
Because that garbage is larger than the file, the clamp did nothing and there was no panic. Other low-memory contents would clip or drop the write.

Fix. Skip the clamp when v_vm_info is NULL, as vnode_pagein does, and let deadfs fail the write.

Also guarded. The audit of every v_vm_info use left in vnode_pager.c found three more reads that matter only after a forced unmount of a volume holding a swap file (untested):

swap-file vnode_size growth (:883-884);
vnode_pager_truncate (:1335, :1345). The vn_lock there can wait out a reclaim.

The rest are already safe.

What the user sees (same before and after; there was no panic before either):

msync returns EIO;
dirty resident pages stay usable;
touching a never-resident page gets SIGBUS;
munmap and exit are clean.

Verified on the kernel built from 1d3cb1f80 (cksum 151843857):

Reproducer (mx.sh: dirty a shared mapping, umount -f, then msync/munmap): msync EIO, SIGBUS, no panic. The probe kernel no longer shows the NULL read.
Earlier recipes: #84's s4x.sh 5/5. rp.sh and rp2.sh 3/3 each.
Checks: check-proc, check-mmap, check-vm, check-fs pass on UFS and /mnt/nfs. The NFS plain unmount is clean.
stress-vm pressure (640m, swap growing to about 420 MB): 2/3 pass. 1 run hit the 600 s timeout with the build guest running at the same time; base timed out the same way.

Left open: the pageout daemon retries a dirty page of a reclaimed vnode until it is unmapped. Each retry is a cheap deadfs EIO.

@raynorpat commented on GitHub (Oct 31, 2026): Fixed by 278d00c75, merged into ffs-perf in 23c1dd5f6. **Root cause.** `vnode_pageout`'s MACH_NBC size clamp (`vm/vnode_pager.c` :855-867) read `vp->v_vm_info->vnode_size` after a forced reclaim had set `v_vm_info` to NULL. That happens on every pageout of a stale page, not just in a race. - On i386, address 0 is the real-mode IVT. A probe kernel read `0xf000ff54` for all 16 dirty pages, once at msync and once at munmap. - Because that garbage is larger than the file, the clamp did nothing and there was no panic. Other low-memory contents would clip or drop the write. **Fix.** Skip the clamp when `v_vm_info` is NULL, as `vnode_pagein` does, and let deadfs fail the write. **Also guarded.** The audit of every `v_vm_info` use left in `vnode_pager.c` found three more reads that matter only after a forced unmount of a volume holding a swap file (untested): - swap-file `vnode_size` growth (:883-884); - `vnode_pager_truncate` (:1335, :1345). The `vn_lock` there can wait out a reclaim. The rest are already safe. **What the user sees** (same before and after; there was no panic before either): - `msync` returns EIO; - dirty resident pages stay usable; - touching a never-resident page gets SIGBUS; - munmap and exit are clean. **Verified** on the kernel built from 1d3cb1f80 (cksum 151843857): - **Reproducer** (`mx.sh`: dirty a shared mapping, `umount -f`, then msync/munmap): msync EIO, SIGBUS, no panic. The probe kernel no longer shows the NULL read. - **Earlier recipes:** #84's `s4x.sh` 5/5. `rp.sh` and `rp2.sh` 3/3 each. - **Checks:** `check-proc`, `check-mmap`, `check-vm`, `check-fs` pass on UFS and `/mnt/nfs`. The NFS plain unmount is clean. - **`stress-vm pressure`** (640m, swap growing to about 420 MB): 2/3 pass. 1 run hit the 600 s timeout with the build guest running at the same time; base timed out the same way. **Left open:** the pageout daemon retries a dirty page of a reclaimed vnode until it is unmapped. Each retry is a cheap deadfs EIO.