rmdir deadlocks two processes on hfsnode vnode locks, then hangs the system #48

New issue

Closed

opened 2026-11-03 22:03:11 -07:00 by gitea-mirror · 1 comment

gitea-mirror commented

2026-11-03 22:03:11 -07:00

Owner

Originally created by @raynorpat on GitHub (Oct 15, 2026).
Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/53

Symptom

stress-fs on an HFS+ volume hangs: every worker sits in state U with 0 CPU
and never finishes. The volume is unusable (ls -la /mnt/h never returns), and
soon after new ssh logins hang too, while sshd still sends its banner. The
console shows no panic. The test's own 600 s watchdog cannot kill the workers.

Seen twice in 2 hours with kernel mach_kernel.issue43 (cksum 1562297808,
built Oct 15):

stress-fs -n 4 -s 1 -d /mnt/h rw append tree fsync rename link: hung in
rename (default 500 iters), after rw/append/tree/fsync had finished.
stress-fs -n 4 -s 10 -d /mnt/h tree on a fresh volume: hung. The 19 other tree
seeds finished, each with an EBUSY FAIL (see below).

QEMU gdbstub dump of the four hung tree workers (pids 1166-1169). All four sleep in
tsleep("hfsnode") <- lockmgr <- vn_lock <- vget <- hfs_lookup
(directly or through hfs_vhashget) <- lookup <- namei <- the syscall.
That is rename for 1167, 1168 and 1169, and open for 1166. The lockmgr
locks they wait on:

waiter	lock	exclusive holder
1166	2031d38c	1168
1167	20317e8c	1169
1168	2029370c	1166
1169	2029870c	1168

1166 and 1168 are in a lock-order cycle. Each holds an hfsnode lock the other
needs, and the other two workers queue behind them.

The volume itself is consistent afterwards: vm/hfs_guest.py check reports
only "volume was not cleanly unmounted".

Reproduce

i386 QEMU guest, HFS+ volume built on the host with
tools/hfsimg builder.BUILDERS["hfsplus"](scenario.base_manifest(), 256 MiB)
behind the 512-byte NeXT label (hfs_guest.wrap_label), written to partition a of
the 4th IDE disk and mounted with mount_hfs /dev/hd3a /mnt/h. mount_hfs was built on the
guest from src/hfs-1/hfs_mount with /build/src/kernel-7 HFS headers. Then:

stress-fs -n 4 -s 10 -d /mnt/h tree      # or: -s 1 ... rename

tree has 4 processes doing random mkdir/rmdir/create/unlink/rename/directory
rename in one shared 3-level tree, so lookups of a parent and its child, and
.. lookups, race with directory renames.

Artifacts in vm/work/fsrepro/: hfs-10-hang-threads.txt and
hfs-10-hang-locks.txt (thread and lock dump), hfs-10-hang.{ppm,png,log},
hfs-1-hang.{ppm,png,log}, the walker gdbwalk.py/gdblocks.py (QEMU -gdb tcp:127.0.0.1:1583), and the volume images hfs-after-hang*.img.

Where to look

bsd/hfs/hfs_lookup.c hfs_lookup(): the vget / hfs_vhashget calls at
hfs_lookup+0x2bf, +0x9f9 and +0xd75 in this kernel. Look at how it
locks the child while holding the parent (the .. / ISDOTDOT case must
unlock the parent first), and at hfs_rename's locking of the source and
target directories. UFS (ufs_lookup/ufs_rename) runs the same tree
and rename loads cleanly on this kernel, so compare against it.

Related, probably expected behaviour rather than a bug: on HFS every tree run
FAILs early with rmdir: EBUSY or rename: EBUSY (hfs_rmdir: v_usecount > 2;
hfs_remove: "HFS does not have unlinking"), and every FAIL reports
leak=kalloc.256:+100..+380. Worth checking whether that kalloc.256 growth
comes back once the vnodes are recycled.

Found by the stress suite (src/Commands/stress, branch stress-phase3) during the phase 3 filesystem run. Related: #45 (rename races, fixed on fs-races).

Originally created by @raynorpat on GitHub (Oct 15, 2026). Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/53 ## Symptom `stress-fs` on an HFS+ volume hangs: every worker sits in state `U` with 0 CPU and never finishes. The volume is unusable (`ls -la /mnt/h` never returns), and soon after new ssh logins hang too, while sshd still sends its banner. The console shows no panic. The test's own 600 s watchdog cannot kill the workers. Seen twice in 2 hours with kernel `mach_kernel.issue43` (cksum 1562297808, built Oct 15): - `stress-fs -n 4 -s 1 -d /mnt/h rw append tree fsync rename link`: hung in `rename` (default 500 iters), after rw/append/tree/fsync had finished. - `stress-fs -n 4 -s 10 -d /mnt/h tree` on a fresh volume: hung. The 19 other `tree` seeds finished, each with an EBUSY FAIL (see below). QEMU gdbstub dump of the four hung `tree` workers (pids 1166-1169). All four sleep in `tsleep("hfsnode")` <- `lockmgr` <- `vn_lock` <- `vget` <- `hfs_lookup` (directly or through `hfs_vhashget`) <- `lookup` <- `namei` <- the syscall. That is `rename` for 1167, 1168 and 1169, and `open` for 1166. The lockmgr locks they wait on: | waiter | lock | exclusive holder | |--------|-----------|------------------| | 1166 | 2031d38c | 1168 | | 1167 | 20317e8c | 1169 | | 1168 | 2029370c | 1166 | | 1169 | 2029870c | 1168 | 1166 and 1168 are in a lock-order cycle. Each holds an hfsnode lock the other needs, and the other two workers queue behind them. The volume itself is consistent afterwards: `vm/hfs_guest.py check` reports only "volume was not cleanly unmounted". ## Reproduce i386 QEMU guest, HFS+ volume built on the host with `tools/hfsimg` `builder.BUILDERS["hfsplus"](scenario.base_manifest(), 256 MiB)` behind the 512-byte NeXT label (`hfs_guest.wrap_label`), written to partition a of the 4th IDE disk and mounted with `mount_hfs /dev/hd3a /mnt/h`. `mount_hfs` was built on the guest from `src/hfs-1/hfs_mount` with `/build/src/kernel-7` HFS headers. Then: ``` stress-fs -n 4 -s 10 -d /mnt/h tree # or: -s 1 ... rename ``` `tree` has 4 processes doing random mkdir/rmdir/create/unlink/rename/directory rename in one shared 3-level tree, so lookups of a parent and its child, and `..` lookups, race with directory renames. Artifacts in `vm/work/fsrepro/`: `hfs-10-hang-threads.txt` and `hfs-10-hang-locks.txt` (thread and lock dump), `hfs-10-hang.{ppm,png,log}`, `hfs-1-hang.{ppm,png,log}`, the walker `gdbwalk.py`/`gdblocks.py` (QEMU `-gdb tcp:127.0.0.1:1583`), and the volume images `hfs-after-hang*.img`. ## Where to look `bsd/hfs/hfs_lookup.c` `hfs_lookup()`: the `vget` / `hfs_vhashget` calls at `hfs_lookup+0x2bf`, `+0x9f9` and `+0xd75` in this kernel. Look at how it locks the child while holding the parent (the `..` / ISDOTDOT case must unlock the parent first), and at `hfs_rename`'s locking of the source and target directories. UFS (`ufs_lookup`/`ufs_rename`) runs the same `tree` and `rename` loads cleanly on this kernel, so compare against it. Related, probably expected behaviour rather than a bug: on HFS every `tree` run FAILs early with `rmdir: EBUSY` or `rename: EBUSY` (`hfs_rmdir`: `v_usecount > 2`; `hfs_remove`: "HFS does not have unlinking"), and every FAIL reports `leak=kalloc.256:+100..+380`. Worth checking whether that kalloc.256 growth comes back once the vnodes are recycled. Found by the stress suite (`src/Commands/stress`, branch `stress-phase3`) during the phase 3 filesystem run. Related: #45 (rename races, fixed on fs-races).

gitea-mirror closed this issue

2026-11-03 22:03:12 -07:00

gitea-mirror commented

2026-11-03 22:03:12 -07:00

Author

Owner

@raynorpat commented on GitHub (Oct 16, 2026):

Fixed in 260033acbb (kernel: Merge filesystem repairs for issues 52-55), on ffs-perf; the HFS part is in hfs_vnodeops.c.

Verified on the repair kernel:

The new focused test vm/hfs_rename_test.c passes. Its lock test completed 4,000 moves against 10,000 concurrent descendant lookups with no deadlock, and its cache checks (stale case alias, cached .., busy-replacement cleanup) pass.
stress-fs tree and rename (4 workers, seeds 1, 3 and 10) now end with EBUSY instead of hanging. HFS's refusal to remove in-use objects is a remaining filesystem limitation, not this deadlock.
The host consistency check reports zero problems.

Test notes and focused regressions: vm/filesystem-issues-52-55.md in that commit. Repair kernel cksum 3500498149 (i386, QEMU snapshot); PowerPC not tested.

@raynorpat commented on GitHub (Oct 16, 2026): Fixed in 260033acbb4c29cf90781f9b16cc028a79785e69 (kernel: Merge filesystem repairs for issues 52-55), on ffs-perf; the HFS part is in `hfs_vnodeops.c`. Verified on the repair kernel: - The new focused test `vm/hfs_rename_test.c` passes. Its lock test completed 4,000 moves against 10,000 concurrent descendant lookups with no deadlock, and its cache checks (stale case alias, cached `..`, busy-replacement cleanup) pass. - `stress-fs tree` and `rename` (4 workers, seeds 1, 3 and 10) now end with `EBUSY` instead of hanging. HFS's refusal to remove in-use objects is a remaining filesystem limitation, not this deadlock. - The host consistency check reports zero problems. Test notes and focused regressions: `vm/filesystem-issues-52-55.md` in that commit. Repair kernel cksum 3500498149 (i386, QEMU snapshot); PowerPC not tested.