[GH-ISSUE #89] stress: vm pressure always times out on a 512 MB guest #82

New issue

Closed

opened 2026-11-03 22:04:20 -07:00 by gitea-mirror · 1 comment

gitea-mirror commented

2026-11-03 22:04:20 -07:00

Owner

Originally created by @raynorpat on GitHub (Oct 31, 2026).
Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/89

Originally assigned to: @raynorpat on GitHub.

Final retest of ffs-perf 23c1dd5f6 (vm/work/final2-retest/).

stress-vm pressure (src/Commands/stress/vm/vm.c ~32-36: every worker holds its share of 1.5× RAM, or -m, and keeps touching it) times out at the 600 s watchdog every time on a 512 MB guest: check-run, run 1, and 3 of 3 solo reruns (seed 1793415145). On a 128 MB guest it finishes in 150-180 s. It has failed this way in every 512 MB run since phase 4.

Paging 1.5× of 512 MB (768 MB) through swap on QEMU TCG simply takes longer than 600 s. The test isn't finding a kernel bug here: the swap file grows normally (to about 420-615 MB in the #85 runs) and nothing panics or corrupts.

Expected

vm pressure passes on any guest size within the watchdog. Options:

cap the working set (for example 1.5× RAM but at most some fixed size, with the pressure ratio kept by lowering the free-page target);
or make the iteration count / -T budget scale with the working set so the test ends PASS once it has paged the whole set N times;
or derive its watchdog from RAM.

It must still force real paging (working set > RAM) and still detect data corruption. Record the chosen size in the README.

Originally created by @raynorpat on GitHub (Oct 31, 2026). Original GitHub issue: https://github.com/RhapsodiOS/RhapsodiOS/issues/89 Originally assigned to: @raynorpat on GitHub. Final retest of ffs-perf 23c1dd5f6 (`vm/work/final2-retest/`). `stress-vm pressure` (`src/Commands/stress/vm/vm.c` ~32-36: every worker holds its share of 1.5× RAM, or `-m`, and keeps touching it) times out at the 600 s watchdog every time on a 512 MB guest: `check-run`, `run 1`, and 3 of 3 solo reruns (seed 1793415145). On a 128 MB guest it finishes in 150-180 s. It has failed this way in every 512 MB run since phase 4. Paging 1.5× of 512 MB (768 MB) through swap on QEMU TCG simply takes longer than 600 s. The test isn't finding a kernel bug here: the swap file grows normally (to about 420-615 MB in the #85 runs) and nothing panics or corrupts. ## Expected `vm pressure` passes on any guest size within the watchdog. Options: - cap the working set (for example 1.5× RAM but at most some fixed size, with the pressure ratio kept by lowering the free-page target); - or make the iteration count / `-T` budget scale with the working set so the test ends PASS once it has paged the whole set N times; - or derive its watchdog from RAM. It must still force real paging (working set > RAM) and still detect data corruption. Record the chosen size in the README.

gitea-mirror

2026-11-03 22:04:20 -07:00

closed this issue
added the
enhancement
label

gitea-mirror commented

2026-11-03 22:04:23 -07:00

Author

Owner

@raynorpat commented on GitHub (Oct 31, 2026):

Fixed by b19dd8e1b and 63e9efc43, merged into ffs-perf in 6314afce8, by scaling the time budget with RAM. The working set is unchanged: still 1.5× RAM, or -m.

Measured first on the base code with -t 6000, 4 procs:

128 MB guest: 213 s and 143 s, swap growing to about 203 MB.
512 MB guest: 816 s and 860 s, swap growing to about 764-790 MB.

Rule: pressure's watchdog is the applicable -t (default 600) × ⌈working set / 192 MB⌉.

192 MB is the 128 MB guest's working set, so that guest keeps 600 s and a 512 MB guest gets 2400 s.
The factor multiplies whatever -t applies, so run -t and stress_fs.sh's msdos -t 1500 scale too.
How it reaches the watchdog: the watchdog's alarm() runs in run() (lib/main.c). It learns the scale from a new optional scale hook in struct st_test. Pressure sets it to s_pressure, which uses the same pressure_bytes() that sizes the test's buffer. All other tests leave it NULL.
A timeout line reports the scaled seconds.
The README is updated, and check-vm checks that -t 1 -m 400m pressure times out at secs=3.

Harness: STRESS_RUN_LIMIT_S is now stress_run_limit(MEM_MB) = 4 h × ⌈MEM_MB / 128⌉ (16 h on 512 MB). It's only a backstop: a dead guest is still caught after 5 failed polls. A unit test was added, and test_ffsbench.py passes.

Verified on the base kernel (cksum 2686302031):

128 MB, pressure alone ×3: 235, 240, 229 s, all PASS, watchdog 600 s.
512 MB, pressure alone ×3: 870, 800, 1087 s, all PASS, watchdog 2400 s. All three are over the old 600 s limit.
The scale reaches the watchdog: -t 1 reports secs=1 on 128 MB and secs=4 on 512 MB.
Selfcheck: the -x run with the full working set still FAILs on the injected offset on both guests.
Make targets: make check, check-vm, check-run rc=0 on both. On 512 MB, check-run's vm summary has FAIL=0.

Left open:

No full five-target 512 MB harness run, so the 16 h limit is reasoned, not measured.
The worst 512 MB time (1087 s, two guests sharing the host) leaves about 2.2× margin.
t_pressure's 1.5× RAM sum would overflow on a guest with about 2.7 GB of RAM or more. That predates this change.

@raynorpat commented on GitHub (Oct 31, 2026): Fixed by b19dd8e1b and 63e9efc43, merged into ffs-perf in 6314afce8, by scaling the time budget with RAM. The working set is unchanged: still 1.5× RAM, or `-m`. **Measured first** on the base code with `-t 6000`, 4 procs: - **128 MB guest:** 213 s and 143 s, swap growing to about 203 MB. - **512 MB guest:** 816 s and 860 s, swap growing to about 764-790 MB. **Rule:** pressure's watchdog is the applicable `-t` (default 600) × ⌈working set / 192 MB⌉. - 192 MB is the 128 MB guest's working set, so that guest keeps 600 s and a 512 MB guest gets 2400 s. - The factor multiplies whatever `-t` applies, so `run -t` and `stress_fs.sh`'s msdos `-t 1500` scale too. - **How it reaches the watchdog:** the watchdog's `alarm()` runs in `run()` (`lib/main.c`). It learns the scale from a new optional `scale` hook in `struct st_test`. Pressure sets it to `s_pressure`, which uses the same `pressure_bytes()` that sizes the test's buffer. All other tests leave it NULL. - A timeout line reports the scaled seconds. - The README is updated, and `check-vm` checks that `-t 1 -m 400m pressure` times out at `secs=3`. **Harness:** `STRESS_RUN_LIMIT_S` is now `stress_run_limit(MEM_MB)` = 4 h × ⌈MEM_MB / 128⌉ (16 h on 512 MB). It's only a backstop: a dead guest is still caught after 5 failed polls. A unit test was added, and `test_ffsbench.py` passes. **Verified** on the base kernel (cksum 2686302031): - **128 MB, pressure alone ×3:** 235, 240, 229 s, all PASS, watchdog 600 s. - **512 MB, pressure alone ×3:** 870, 800, 1087 s, all PASS, watchdog 2400 s. All three are over the old 600 s limit. - **The scale reaches the watchdog:** `-t 1` reports `secs=1` on 128 MB and `secs=4` on 512 MB. - **Selfcheck:** the `-x` run with the full working set still FAILs on the injected offset on both guests. - **Make targets:** `make check`, `check-vm`, `check-run` rc=0 on both. On 512 MB, check-run's vm summary has FAIL=0. **Left open:** - No full five-target 512 MB harness run, so the 16 h limit is reasoned, not measured. - The worst 512 MB time (1087 s, two guests sharing the host) leaves about 2.2× margin. - `t_pressure`'s 1.5× RAM sum would overflow on a guest with about 2.7 GB of RAM or more. That predates this change.