diff --git a/changelog.md b/changelog.md index ce104ad..79f6c5c 100755 --- a/changelog.md +++ b/changelog.md @@ -128,6 +128,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Fixed +- An allocation failure inside a C parse or validate crashed the process on Windows instead of returning NULL, on any binary built with mingw at `-O1`, `-O2` or `-Os`. The recovery unwinds through SEH there, and it was reading off the top of the stack on the way. The whole point of the recovery is that a config problem does not take the application down with it, so on Windows it had been doing the opposite of what it promised. + - A line whose indent matches no open level (`E012`) and a `*` line with no space after it (`E013`) now hold their indent level, so what is written under them is skipped with them (`E018`) instead of attaching one level up, a fence line at a bad indent takes its whole body with it instead of parsing it as top-level bindings, and a second line at the same bad indent is refused the same way rather than binding. - A value after an index selector on the last segment (`a[0]: 2`) was dropped with no diagnostic and no lost count, so an in-place write deleted it at exit 0. It is reported (`E002`) and counted as lost now, as a value after a value selector always was. diff --git a/cicd/utility/win-runners.bash b/cicd/utility/win-runners.bash index 125fb1a..be5173e 100755 --- a/cicd/utility/win-runners.bash +++ b/cicd/utility/win-runners.bash @@ -78,9 +78,21 @@ fRunOom() { source/c/tests/oom_hook.c -o "${work}/oom_hook${exe}" -lm \ && "${work}/oom_hook${exe}" } +## The recovery point unwinds through SEH on this host and through nothing much +## on linux or under wine, so windows is the only place the arrival can be +## judged. It also has to hold at every optimization level: the shape that broke +## it needs both a frame pointer and saved xmm registers, which is a decision +## gcc makes per level, so -O2 alone would have missed -O1. +fRunOomRecover() { + local opt + for opt in -O0 -O1 -O2 -O3 -Os; do + "${cc}" -std=c11 "${opt}" -Wall -Wextra -Werror -Isource/c \ + source/c/tests/oom_recover.c -o "${work}/oom_recover${exe}" -lm || return 1 + "${work}/oom_recover${exe}" || { echo "win-runners: oom_recover: ${opt}: exit $?" >&2; return 1; } + done +} ## The allocation bounds move with the allocator, so they belong here rather -## than on linux alone. The unwind test is deliberately not here: it crashes on -## a real windows host and passes everywhere else (20260901b item 48). +## than on linux alone. fRunMemBounds() { "${cc}" -std=c11 -O2 -Wall -Wextra -Werror -Isource/c \ source/c/tests/mem_bounds.c -o "${work}/mem_bounds${exe}" -lm \ @@ -144,6 +156,7 @@ fRun "python" "${py}" source/python/tests/conformance.py fRun "c" fRunC fRun "c++ veneer" fRunCxx fRun "c oom hook" fRunOom +fRun "c oom recover" fRunOomRecover fRun "c mem bounds" fRunMemBounds fRun "c cli argv" fRunCcli fRun "closed stdin" fRunClosedStdin @@ -170,3 +183,5 @@ fi ## where the file tier's publish step is a different code path in all four. ## - 20260901: The installers' PATH handling joins, windows hosts only - it ## needs a real registry. +## - 20260903: The allocation-failure recovery joins, swept across five +## optimization levels. Its unwind is a real SEH unwind only here. diff --git a/project/backlog.md b/project/backlog.md index d2814ef..2a84e6d 100644 --- a/project/backlog.md +++ b/project/backlog.md @@ -605,10 +605,16 @@ Every item carries the date it was opened and, once settled, the date it closed. - Opened: 20260904-025000 - Closed: 20260904-034000 - - 🔘 Item 48: the C allocation-failure unwind crashes on a real windows host. + - ✅ Item 48: the C allocation-failure unwind crashes on a real windows host. - `oom_recover.c` segfaults on the hosted windows runner (mingw gcc, `-O2`), and passes on linux and under wine with the same source and the same compiler family. The recovery point is a `setjmp`, and on x86_64 mingw a `longjmp` unwinds through SEH, which wine's runtime and the real one do not implement alike - so the two disagree about a stack the library builds identically. - Until it is understood, the test runs on linux (in the test stage, the compiler sweep and under the sanitizers) and not on the windows job; `mem_bounds.c` does run there. A consumer building the header with mingw is the one this would reach. + - Reproduced on a real windows host, first run and every run. The fault is a read past the top of the stack from inside `RtlVirtualUnwind`, on the first budget tight enough to reach the `longjmp` at all - so every recovery was crashing, not an edge case among them. + - Cause: mingw's `setjmp` hands `longjmp` a target frame, so `longjmp` runs a full `RtlUnwindEx` to get there. Taking that frame address forces a frame pointer, and gcc then puts a vectorized function's xmm save slots at offsets from the post-prologue rsp while `UWOP_SET_FPREG` in the same unwind info tells the unwinder to take them from rbp, which sits above the whole frame. The difference is the frame's own size and it lands off the end of the stack. Wine derives rsp differently and never reads there. + - Fixed: both recovery points arm with `_setjmp(buf, NULL)` on mingw x86_64, which makes `longjmp` restore the context without unwinding at all. That is all a C recovery point needs - nothing in between has a destructor or a `__finally`. Recorded as a C deviation in `style-guide.md`, and the `SHCL_OOM` docs now point an embedder whose hook longjmps at the same thing, since that unwind crosses these frames too. + - Pinned by a `win-runners.bash` step that builds and runs the test at `-O0`, `-O1`, `-O2`, `-O3` and `-Os`. The shape needs both a frame pointer and saved xmm registers, which gcc decides per level: backed out, three of the five crash and two do not, so a single-level gate could have gone either way. + - Fixed on the way: the load half wrote its fixture to a hardcoded `/tmp`, which a mingw binary does not translate. The new gate would have rested on the runner happening to have a `C:\tmp`. - Opened: 20260904-052000 + - Closed: 20260904-070000 - Code review 20260901b: diff --git a/source/c/shcl.h b/source/c/shcl.h index 9b8a72f..8c54c31 100644 --- a/source/c/shcl.h +++ b/source/c/shcl.h @@ -488,11 +488,28 @@ static const char *dec_point(void) { // process that is not the library's to end. An embedder defines SHCL_OOM before // the implementation to longjmp out, log, or abort on its own terms. Nothing is // unwound first, so a hook that returns leaks whatever was being built - and -// then aborts, because the allocation it was called for still failed. +// then aborts, because the allocation it was called for still failed. A hook +// that longjmps wants the NULL-frame setjmp below, for the reason given there: +// the unwind it starts crosses this library's frames, not just its own. #ifndef SHCL_OOM #define SHCL_OOM() do { fprintf(stderr, "shcl: out of memory\n"); exit(70); } while (0) #endif +// mingw's setjmp hands longjmp a target frame, and longjmp then unwinds +// through SEH to reach it. gcc's unwind info for a function that has both a +// frame pointer and saved xmm registers puts those save slots at offsets the +// real unwinder resolves past the top of the stack, and the read faults. Wine +// resolves them from a different base and never sees it, which is why the same +// binary passes there. A NULL frame makes longjmp restore the context without +// unwinding at all, and a C recovery point needs nothing more - there is no +// destructor and no __finally between the failed allocation and the arrival. +// The full shape is in style-guide.md under the C deviations. +#if defined(__MINGW32__) && defined(__x86_64__) && defined(__SEH__) + #define SHCL_SETJMP(buf) _setjmp((buf), NULL) +#else + #define SHCL_SETJMP(buf) setjmp(buf) +#endif + /* Unwind to the recovery point `panic` names, or fall back to the macro when nothing armed one. Keeping on with a failed allocation is not an option: a bump arena holds its vectors' bookkeeping, so a request served out of @@ -2819,7 +2836,7 @@ static shcl_doc *do_parse(const char *text, size_t len, shcl_strictness strict, ShclParseOwn *volatile owned = (ShclParseOwn *)calloc(1, sizeof *owned); if (!owned) { free(doc); return NULL; } jmp_buf panic; - if (setjmp(panic)) { + if (SHCL_SETJMP(panic)) { /* An allocation failed somewhere below. Nothing built so far can be trusted and there is no way to finish, so the whole document goes and the caller gets NULL - with the process still standing, which is the @@ -5379,7 +5396,7 @@ shcl_validation *shcl_validate(shcl_doc *d, shcl_doc *schema) { memset(val, 0, sizeof *val); ShclArena *volatile levels = NULL; jmp_buf panic; - if (setjmp(panic)) { + if (SHCL_SETJMP(panic)) { shcl_validation *bad = val; ShclArena *badLevels = levels; if (badLevels) { for (size_t i = 0; i <= SHCL_MAX_DEPTH; i++) arena_free(&badLevels[i]); free(badLevels); } /* The name index is the only thing on the document this call builds, diff --git a/source/c/tests/oom_recover.c b/source/c/tests/oom_recover.c index 290e2d9..a8cdd84 100644 --- a/source/c/tests/oom_recover.c +++ b/source/c/tests/oom_recover.c @@ -65,7 +65,14 @@ int main(void) { // A load is the same call plus a read, so it reports the same way. { - const char *path = "/tmp/shcl-oom-recover.shcl"; + // A mingw binary's fopen does not translate /tmp, and windows hosts do + // not all have a C:\tmp for it to land in. + const char *dir = getenv("TMPDIR"); + if (!dir) dir = getenv("TMP"); + if (!dir) dir = getenv("TEMP"); + if (!dir) dir = "/tmp"; + char path[512]; + snprintf(path, sizeof path, "%s/shcl-oom-recover.shcl", dir); FILE *f = fopen(path, "wb"); if (!f) fail("could not write the fixture"); else { diff --git a/style-guide.md b/style-guide.md index 5fff677..3787568 100644 --- a/style-guide.md +++ b/style-guide.md @@ -113,6 +113,8 @@ New bindings (Tier 3) follow the same recipe: port the reference function-for-fu - Deliberate deviation: an allocation failure inside a parse or a validate unwinds and the call returns NULL, where the other three abort. Their languages abort on allocation failure and there is nothing there to mirror, while a C consumer embedding the header has a process that is not the library's to end. Everywhere else - a read, a write, a merge on a document already built - the `SHCL_OOM()` hook is still the answer. +- Deliberate deviation: on mingw x86_64 the two recovery points are armed with `_setjmp(buf, NULL)` rather than plain `setjmp`. mingw's `setjmp` stores `__builtin_frame_address(0)` as an SEH target frame, and `longjmp` then runs a full `RtlUnwindEx` to reach it. Taking that address forces a frame pointer, and gcc encodes a vectorized function's xmm save slots as offsets from the post-prologue rsp while `UWOP_SET_FPREG` in the same unwind info tells the unwinder to take them from rbp, which sits above the whole frame. Windows resolves the difference past the top of the stack and the read faults; wine derives rsp differently and never sees it, so the identical binary passes there. A NULL frame makes `longjmp` restore the context without unwinding, which is all a C recovery point needs, since nothing between the failed allocation and the arrival has a destructor or a `__finally`. The shape needs both a frame pointer and saved xmm registers, which is a decision gcc makes per optimization level: `-O1`, `-O2` and `-Os` crashed where `-O0` and `-O3` did not, so the gate sweeps all five. + - Deliberate deviation: the float formatter decides whether a spelling reads back with its own integer arithmetic, not `strtod`. The other three have a shortest-digits formatter in their runtime; C has `printf` and `strtod`, and more than one C runtime (msvcrt, wine's) parses some 15-digit spellings one ulp off, which made the C spelling of a value depend on the libc it was built against. The parser's float reads still go through `strtod`, so a value read on such a runtime can be one ulp off; the header cannot fix a libc. - Deliberate deviation: `shcl_compact` has no counterpart in the other three. A write lands in the document's bump arena and the value it replaced stays there until `shcl_free`, so a process rewriting one field in a loop grows by a few dozen bytes per write; the other three reclaim the old value through their runtimes. Compaction rebuilds the document into fresh arenas, carrying the diagnostics, the lost count and the strictness with it, so a save or a strict gate afterwards reads the same. The C++ veneer exposes it as `compact()`.