[GH-ISSUE #73] kernel: i386 trap.c turns a kernel page fault on a user address into an endless, unkillable loop #65

New issue

Closed

opened 2026-11-03 22:03:37 -07:00 by gitea-mirror · 1 comment

gitea-mirror commented

2026-11-03 22:03:37 -07:00

Owner

Originally created by @raynorpat on GitHub (Oct 20, 2026).
Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/73

Symptom

A kernel-mode page fault at an address below KERNEL_LINEAR_BASE (for example a NULL or garbage pointer dereference inside a system call) never panics. The thread loops forever taking the same fault; the process leaves allproc on exit (P_WEXIT) so ps cannot see it, and it can't be killed. In #67 a bad socket pointer in sosend produced exactly this, and the stress watchdog was the only sign.

Cause

machdep/i386/trap.c ~243-270, T_PAGE_FAULT from kernel mode: when vm_fault on the user map fails and kernel_try_recover finds no recovery point (no copyin/copyout in progress), the code assumes the fault came from returning to user mode with a bad LDT entry. It calls kernel_recover_thread and exception_from_kernel, then thread_exception_return, which re-runs the kernel instruction that faulted.

Fix direction

Take that path only for the case it was written for (the fault is on the return-to-user path / segment load, which the code can recognise from the faulting EIP), and panic with the trap frame for any other kernel fault on a user address, as other kernel faults already do.

Found by the #67 investigation.

Originally created by @raynorpat on GitHub (Oct 20, 2026). Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/73 ## Symptom A kernel-mode page fault at an address below `KERNEL_LINEAR_BASE` (for example a NULL or garbage pointer dereference inside a system call) never panics. The thread loops forever taking the same fault; the process leaves `allproc` on exit (P_WEXIT) so `ps` cannot see it, and it can't be killed. In #67 a bad socket pointer in `sosend` produced exactly this, and the stress watchdog was the only sign. ## Cause `machdep/i386/trap.c` ~243-270, `T_PAGE_FAULT` from kernel mode: when `vm_fault` on the user map fails and `kernel_try_recover` finds no recovery point (no copyin/copyout in progress), the code assumes the fault came from returning to user mode with a bad LDT entry. It calls `kernel_recover_thread` and `exception_from_kernel`, then `thread_exception_return`, which re-runs the kernel instruction that faulted. ## Fix direction Take that path only for the case it was written for (the fault is on the return-to-user path / segment load, which the code can recognise from the faulting EIP), and panic with the trap frame for any other kernel fault on a user address, as other kernel faults already do. Found by the #67 investigation.

gitea-mirror closed this issue

2026-11-03 22:03:38 -07:00

gitea-mirror referenced this issue

2026-11-03 22:03:38 -07:00

[GH-ISSUE #71] QEMU TCG: x87 FSAVE/FRSTOR across a page boundary writes mantissas into the next physical page (kernel pcb workaround done; user programs still affected) #66

gitea-mirror commented

2026-11-03 22:03:39 -07:00

Author

Owner

@raynorpat commented on GitHub (Oct 20, 2026):

Fixed by 1978123ac, merged into ffs-perf in f5d9d290f.

What the branch was for. The kernel page-fault path that sends the thread back to user mode exists for one case only: RETURN in locore.s reloading user segment registers (popl %gs/%fs/%es/%ds … iret) from a private LDT that lives in user memory. That code runs from __return_with_state and from the user-mode return of interrupt, with the stack inside the thread's save area, which is what kernel_recover_thread relies on.

Fix. A new on_user_return(state) is TRUE only when the trap frame's stack pointer is inside the user save area, between regs.gs and frame.eip. The recovery runs only then; any other kernel fault on a user address goes to the existing "unexpected kernel trap" path (kdp_raise_exception or panic).

Verified on the base kernel (#67 fix, cksum 606865771) and the fixed kernel (cksum 1518819907):

Before: a garbage pointer injected with the gdbstub into fdfree during exit gives the endless loop: the process vanishes from ps, its parent never returns, and the load average stays near 2. The same injection in getpgrp turned into a user SIGSEGV.
After: both stop with "unexpected kernel trap" (eip _fdfree+0xc / _getpgrp+9) in the kernel debugger.
The real case still works: a process that loads %fs from a private LDT in a page it then deallocates faults at __return_with_state+0x12 (popl %fs) and dies with SIGSEGV on both kernels.
check-vm, check-proc, check-misc, check-thread pass; soak -t 600 -j 4 has 0 failures.

Left open: the general-protection / segment-not-present / stack-fault branch for ERR_LDT errors doesn't check that it came from the return to user mode either.

@raynorpat commented on GitHub (Oct 20, 2026): Fixed by 1978123ac, merged into ffs-perf in f5d9d290f. **What the branch was for.** The kernel page-fault path that sends the thread back to user mode exists for one case only: `RETURN` in `locore.s` reloading user segment registers (`popl %gs/%fs/%es/%ds … iret`) from a private LDT that lives in user memory. That code runs from `__return_with_state` and from the user-mode return of `interrupt`, with the stack inside the thread's save area, which is what `kernel_recover_thread` relies on. **Fix.** A new `on_user_return(state)` is TRUE only when the trap frame's stack pointer is inside the user save area, between `regs.gs` and `frame.eip`. The recovery runs only then; any other kernel fault on a user address goes to the existing "unexpected kernel trap" path (`kdp_raise_exception` or `panic`). **Verified** on the base kernel (#67 fix, cksum 606865771) and the fixed kernel (cksum 1518819907): - **Before:** a garbage pointer injected with the gdbstub into `fdfree` during exit gives the endless loop: the process vanishes from `ps`, its parent never returns, and the load average stays near 2. The same injection in `getpgrp` turned into a user SIGSEGV. - **After:** both stop with "unexpected kernel trap" (eip `_fdfree+0xc` / `_getpgrp+9`) in the kernel debugger. - **The real case still works:** a process that loads `%fs` from a private LDT in a page it then deallocates faults at `__return_with_state+0x12` (`popl %fs`) and dies with SIGSEGV on both kernels. - `check-vm`, `check-proc`, `check-misc`, `check-thread` pass; `soak -t 600 -j 4` has 0 failures. **Left open:** the general-protection / segment-not-present / stack-fault branch for `ERR_LDT` errors doesn't check that it came from the return to user mode either.