Code review 20260902: bugs 1-28

README.md

@@ -502,10 +502,11 @@ line 3: Error: E014 malformed line skipped: unexpected '4' after field
 failed: 1 diagnostic(s), 1 error(s)
 
 $ shcl get server.shcl log-level     # the rest of the file loaded fine
+line 3: Error: E014 malformed line skipped: unexpected '4' after field
 warn
 ```
 
-Both streams carry the code; only stdout is the contract. The stdout line is `line N: Severity: CODE` and nothing else, so a script can match on `E014` without parsing English, while the stderr line adds the prose for a person reading along. `check` exits 6 when it found errors - enough to gate a build.
+Every subcommand that loads the file says what the load dropped, once, on stderr - a read that came back fine is still a file with a line missing. Both streams carry the code; only stdout is the contract. The stdout line is `line N: Severity: CODE` and nothing else, so a script can match on `E014` without parsing English, while the stderr line adds the prose for a person reading along. `check` exits 6 when it found errors - enough to gate a build.
 
 Hand it a schema and it validates against that too. A schema is an ordinary `.shcl` file: one `field:` instance per path, constraints written as its children ([the spec](project/spec.md#schema-validation) has the full vocabulary).
 
@@ -827,7 +828,7 @@ shcl set --write server.shcl \
 
 The two spellings differ in how the value is read. `--set` takes **data**: its type follows the text, so `workers=8` writes an integer, but a comma in it is content - `hosts=a, b` would store one quoted string. `--set-literal` takes **value syntax**, the way a file spells it, so that same text writes a two-element array. Reach for it whenever the value is not a plain scalar.
 
-Raw blocks, set-only-if-absent and removal have no option form; those go in as a write-ops script on stdin, one op per line, fields separated by a literal tab:
+Removal and set-only-if-absent have option forms too - `--remove=PATH`, `--set-default=PATH=VALUE`, `--set-literal-default=PATH=TEXT` - and all five share one ordered list, so two options touching the same path resolve in the order given. Raw blocks are the one edit with no option form; those go in as a write-ops script on stdin, one op per line, fields separated by a literal tab:
 
 ```bash
 shcl set --write server.shcl <<OPS

changelog.md

@@ -36,6 +36,32 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ### Changed
 
+- On Windows a save keeps the file's hidden and system attributes. `ReplaceFile`'s documented preserve list stops at security attributes and named streams, and the fallback rename carries nothing, so a hidden config came back visible. They are re-applied after the publish now, the way read-only already was. The `REPLACEFILE_WRITE_THROUGH` flag Microsoft documents as unsupported is no longer described as what makes the write durable; the file's own flush before the publish is.
+
+- An array read of a one-element cell reports the element's quoting, like the scalar read of the same node. `read_int("h")` on `h: "5"` said quoted and `read_int_array("h")` said not, because the array path always answered false. More than one element still reports false: there is no single element to report.
+
+- At Loose, a space after a currency symbol no longer decides whether the value reads. `$ 1200` read as 1200 while `$ 3.14` was `BadType`, because the int path reached a branch that trims and the float path tested the shape on the untrimmed remainder. The space comes off once, for both.
+
+- `shcl_generate` keeps nothing when it refuses, and what it returns can be given back. The output was copied into the schema's own arena before the self-check, so a call that failed kept text it never returned, and a call that succeeded left a copy no `shcl_reads_release` could reclaim - 21.9 KB per call in a loop. The bytes live in the read arena now, and generation faults from an earlier call are dropped rather than stacked up. The C++ veneer's `generate()` releases first, like every other copying wrapper.
+
+- `init` names the path it cannot generate. A required path with a `[#N]` selector, or one past the nesting cap, went to the trailing comment block and then failed the self-check with "required path missing", which points at the generated config rather than at the schema line nothing can satisfy. It is a `V097` fault naming the path now. A name carrying a newline is generated rather than refused: names have been stored escape-resolved since 2.0 and the name escaper spells one.
+
+- A blank line before the first thing canonical output prints is dropped at load. Canonical output never starts with a blank, so a document that kept the flag did not survive its own canonical form: merging a layer gave a different result from merging its `fmt`, and the fold placed a blank line the author never wrote. Three shapes did it - a file starting with a blank line, a blank after a leading line the load dropped, and a blank on a later instance that merged into the first.
+
+- A refused `--set` or a failing ops line no longer swallows the load's diagnostics. The edit was applied before anything was printed, so a `get --set` on a file with a dropped line reported the refusal and said nothing about the damage. The diagnostics belong to the load and now go out before any edit runs.
+
+- Go's `LoadError` and Python's `diagnostics()` and `LoadError` hand back a copy. Each returned the document's own list, so a caller sorting or clearing what it was given silently changed what the document reported, and the document's next append landed in the caller's slot. Go's `Diagnostics()` was fixed for this in 2.0; these were the ones it missed.
+
+- The named-month date forms hold the day to `DD`. `Jul +12 2026`, `Jul 0012 2026` and `+12 Jul 2026` read as 12 July, because the space-separated spellings parsed the day as a plain integer where every delimited spelling holds it to one or two digits. The spec calls the format list a closed whitelist and spells the day `DD`.
+
+- A stdout that cannot be written exits 8 instead of reporting success. `shcl fmt f.shcl > /dev/full` exited 0 with an empty stderr in three of the four CLIs and killed the Python one with an interpreter message; the help and the man page have said 8 for a stream that could not be written all along. A reader that closed early is still the quiet exit, since nobody is there to read a complaint.
+
+- A stderr that cannot be written no longer costs the document. The reference aborted with nothing on stdout at all when a diagnostic could not be printed, which turned an unwritable log into a lost `fmt`. Diagnostics are best-effort now; the exit code still carries the outcome.
+
+- A wildcard read whose parent does not exist reports `NotFound` instead of `Empty`. `x[*]` on a document with no `x` said the path was there and empty, which is the answer for a field written with nothing after the colon; `x` on its own said `NotFound`. The two agree now.
+
+- A wildcard after a wildcard flattens instead of answering `Multiple` for every slot. `server[*].*` reported one unreadable slot per instance, `count` counted instances rather than leaves, and `Remove` on such a path removed nothing. The inner slots now join the outer run, so the result is one slot per resolved leaf and the two wildcards compose the way the spec says they do.
+
 - A float literal past the double range (`1e400`) reads as `BadType` instead of an infinity at `Good`. No double holds the value, and the infinity could not be written back, so a read-modify-write left a field the reader then refused. A literal below the range still reads as zero.
 
 - `get`, `count` and `instances` print the load's diagnostics to stderr, the way `fmt` and `set` already did. Below strict a damaged file used to read back a correct value at exit 0 with nothing said, so the only way to learn a line had been dropped was a separate `check` run. One report per run; stdout is unchanged.

cicd/utility/check-docs.bash

@@ -155,6 +155,65 @@ while IFS= read -r hit; do
 	fBad "backlog.md: says how it was found rather than what changed: ${hit}"
 done < <(grep -nE '^[[:space:]]*- Found (by|while) ' "${backlog}" || true)
 
+##	V096 and V097 come only from generation, so validating anything against the
+##	schema cannot reproduce them - the veneer header used to send a reader that
+##	way for the fault list, which returns the validated document's own V002 and
+##	V007 instead. The C CLI made the same mistake and was fixed in 20260830b.
+while IFS= read -r hit; do
+	fBad "shcl.hpp: sends a reader to validate() for generation faults: ${hit}"
+done < <(grep -nE 'for the fault list, validate\(\)' "${repoDir}/source/c/shcl.hpp" || true)
+
+##	A bare `Mon DD, YYYY` is two array elements, not a date: the comma splits
+##	first. The bullet listing that spelling has to say so, or a reader copies it
+##	unquoted out of the spec and gets a BadType.
+grep -q 'in the space form a comma may follow the day .*only inside quotes' "${repoDir}/project/spec.md" \
+	|| fBad "spec.md: the Mon DD, YYYY bullet does not say the comma spelling needs quotes"
+
+##	Prose that names the CLI's edit options, or the subcommands that take a
+##	layer, goes stale the moment one is added. Each claim is checked against the
+##	shipped help text rather than against a copy of the list.
+help="$("${repoDir}/source/rust/target/debug/shcl" help 2>/dev/null || true)"
+if [[ -n "${help}" ]]; then
+	for opt in --remove --set-default --set-literal-default; do
+		grep -q -- "${opt}" <<<"${help}" || continue
+		grep -q -- "\`${opt}" "${repoDir}/README.md" \
+			|| fBad "README.md: ${opt} is in the help and not in the edit-options paragraph"
+		grep -qF -- "${opt//-/\\-}" "${repoDir}/source/man/shcl.1" \
+			|| fBad "shcl.1: ${opt} is in the help and not in the man page"
+	done
+	##	The man page's WRITE OPS sentence lists the options that stop stdin
+	##	being read; the CLI reads it only when none of the five is given.
+	writeops="$(sed -n '/^\.SH WRITE OPS/,/One op per line/p' "${repoDir}/source/man/shcl.1")"
+	for opt in '\-\-set' '\-\-set\-literal' '\-\-set\-default' '\-\-set\-literal\-default' '\-\-remove'; do
+		grep -qF -- "${opt}" <<<"${writeops}" \
+			|| fBad "shcl.1: WRITE OPS does not name ${opt//\\/} among the options that carry the edits"
+	done
+fi
+
+##	Every subcommand that takes a layer has to be in the spec's list of them.
+##	Driven off the CLI rather than off a copy: the list went stale twice.
+if [[ -n "${help}" ]]; then
+	#  shellcheck disable=2016  ## the backticks are the document's own markdown.
+	layerLine="$(grep -n 'takes repeated `--layer=FILE`' "${repoDir}/project/spec.md" | head -1 || true)"
+	[[ -n "${layerLine}" ]] || fBad "spec.md: no sentence listing the subcommands that take --layer"
+	tmpErr="$(mktemp)"
+	for cmd in get fmt count instances children paths set check; do
+		"${repoDir}/source/rust/target/debug/shcl" "${cmd}" --layer=/dev/null /dev/null a < /dev/null > /dev/null 2>"${tmpErr}" || true
+		grep -qE 'unknown option|not valid for' "${tmpErr}" && continue
+		grep -qF -- "\`${cmd}\`" <<<"${layerLine}" \
+			|| fBad "spec.md: ${cmd} takes --layer and is not in the list of subcommands that do"
+	done
+	rm -f "${tmpErr}"
+fi
+
+##	Every subcommand that loads a document prints the load's diagnostics, so a
+##	README transcript reading a damaged file has to show them - the get example
+##	sat under a check example that showed the same file's diagnostic and said
+##	nothing itself.
+readmeGet="$(sed -n '/shcl get server.shcl log-level/,/^```$/p' "${repoDir}/README.md")"
+grep -q 'E014' <<<"${readmeGet}" \
+	|| fBad "README.md: the get transcript on the damaged file shows no load diagnostic"
+
 if ((nBad)); then
 	echo "check-docs: ${nBad} check(s) failed" >&2
 	exit 1

cicd/utility/cli-regress.bash

@@ -55,6 +55,15 @@ printf 'field: server.port\n\ttype: int\n\trequired: yes\n\tmin: 1\n\tmax: 10\n\
 ## documented shortfall and generates.
 printf 'field: "*"\n\ttype: int\n\trepeat: 1\n' > "${tmpDir}/star1.shcl"
 printf 'field: "*"\n\ttype: int\n\trepeat: 2\n' > "${tmpDir}/star2.shcl"
+## The generation field ceiling, either side of it: the cap used to fire AT the
+## limit while its message said past it.
+awk 'BEGIN{ for (i = 0; i < 10000; i++) printf "field: f%d\n", i }' > "${tmpDir}/cap10000.shcl"
+awk 'BEGIN{ for (i = 0; i < 10001; i++) printf "field: f%d\n", i }' > "${tmpDir}/cap10001.shcl"
+## A must-exist path with nothing to generate from: an index selector needs an
+## instance that is not there, and a path past the nesting cap would draw E016
+## on the way back in. Either way the fault names the path rather than reporting
+## the generated config as missing it.
+printf 'field: "srv[#1].port"\n\trequired: yes\n' > "${tmpDir}/idxreq.shcl"
 ## A schema that does not build: the report is the build faults alone, not the
 ## faults plus what an empty document would owe the schema.
 printf 'field: a\n\ttype: int\n\trequired: yes\nfield: b\n\ttype: nope\n' > "${tmpDir}/nobuild.shcl"
@@ -72,11 +81,13 @@ printf 'a: 1\nb: 2\n' > "${tmpDir}/two.shcl"
 ##	argv placeholders: %F% the good file, %B% the two-error file, %D% a directory,
 ##	%P% the deepest legal document, %S% the self-contradicting schema, %S1%/%S2%
 ##	a nameless must-exist path at repeat 1 and 2, %S3% a schema that does not
-##	build, %X% an
+##	build, %S4% a required path with an index selector, %S5%/%S6% a schema at and
+##	one past the generation field ceiling, %X% an
 ##	instance whose discriminator holds an '=', %T% a document with a name that
 ##	needs quoting in a path, %F2% a two-key file for the edit options, %M% a
 ##	path with no file at it.
-##	stdin: printf %b text, '-' none, '@closedin' / '@closedout' close that stream.
+##	stdin: printf %b text, '-' none, '@closedin' / '@closedout' close that
+##	stream, '@fullout' / '@fullerr' point it at a device that is always full.
 ##	stdout and stderr: '-' means unchecked; an empty stdout field means exactly empty.
 ##	A stderr regex starting with '!' must match NO line.
 ##	Each row names the round and item it pins.
@@ -106,8 +117,16 @@ rows=(
 	## 20260901 item 5: the self-check waved every V007 through, so a repeat
 	## lower bound of 1 - a must-exist path - went out as a config that fails
 	## its own schema at exit 0.
-	'init-star-repeat1|init --schema=%S1%|-|6||V097 .*not in 1\.\.1'
+	'init-star-repeat1|init --schema=%S1%|-|6||V097 required path cannot be generated'
 	'init-star-repeat2|init --schema=%S2%|-|0|-|^$'
+	## 20260902 item 19: an index selector or a path past the cap got the
+	## self-check's "required path missing", which points at the config rather
+	## than at the schema line nothing can generate.
+	'init-index-required|init --schema=%S4%|-|6||V097 required path cannot be generated: srv\[#1\].port'
+	## 20260902 item 20: V096 fired at exactly the ceiling, on a schema with no
+	## fragments, saying the schema expands past it.
+	'init-cap-at-limit|init --no-banner --schema=%S5%|-|0|-|^$'
+	'init-cap-over|init --no-banner --schema=%S6%|-|6||V096 schema expands past 10000 fields'
 	'init-build-fault|init --schema=%S3%|-|6||V091 unknown schema type'
 	'init-build-fault-only|init --schema=%S3%|-|6||!V002'
 	## 20260830 item 35: -h and --help after FILE were an unknown option, though
@@ -153,6 +172,20 @@ rows=(
 	'children-quoted|children %T% db|-|0|host\n"odd.key"|-'
 	'children-missing|children %T% nope|-|0||-'
 	'paths-all|paths %T%|-|0|db\ndb.host\ndb."odd.key"\nweb\nweb.port|-'
+	## 20260902 item 15: a refused edit returned before the load's diagnostics
+	## were printed, so a damaged file said nothing about the damage.
+	'refused-set-still-reports|get --set=a[*]=1 %B% a|-|1|-|E015 missing colon'
+	'refused-op-still-reports|set %B%|int\ta[*]\t1\n|1|-|E015 missing colon'
+	## 20260902 item 14: Go read a non-UTF-8 ops script as a usage error.
+	'ops-not-utf8|set %F%|\xff\n|8|-|-'
+	## 20260902 items 8 and 9: a stdout that could not be written was reported
+	## as success by three CLIs, and a stderr that could not be written aborted
+	## the reference with nothing on stdout at all.
+	'full-stdout-fmt|fmt %F%|@fullout|8|-|[Nn]o space left'
+	'full-stdout-check|check %F%|@fullout|8|-|-'
+	'full-stdout-get|get %F% a|@fullout|8|-|-'
+	'full-stdout-set|set --set=a=2 %F%|@fullout|8|-|-'
+	'full-stderr-keeps-stdout|fmt %B%|@fullerr|0|a: 1\n\tbad:\nb 2\n|-'
 	## Found working 20260830b item 18: a merge does not carry diagnostics, so
 	## reading them off the merged doc reported the lowest layer and stayed
 	## silent about FILE - the one file the caller actually named.
@@ -172,17 +205,28 @@ for row in "${rows[@]}"; do
 	argv="${argv//%S1%/${tmpDir}/star1.shcl}"
 	argv="${argv//%S2%/${tmpDir}/star2.shcl}"
 	argv="${argv//%S3%/${tmpDir}/nobuild.shcl}"
+	argv="${argv//%S4%/${tmpDir}/idxreq.shcl}"
+	argv="${argv//%S5%/${tmpDir}/cap10000.shcl}"
+	argv="${argv//%S6%/${tmpDir}/cap10001.shcl}"
 	argv="${argv//%X%/${tmpDir}/sel.shcl}"
 	argv="${argv//%T%/${tmpDir}/tree.shcl}"
 	argv="${argv//%F2%/${tmpDir}/two.shcl}"
 	argv="${argv//%M%/${tmpDir}/not-there.shcl}"
+	## A device that is always full exists on linux and not on windows; the
+	## rows that need one are skipped out loud rather than passing vacuously.
+	if [[ "${stdinSpec}" == @full* && ! -w /dev/full ]]; then
+		echo "cli-regress: skipping ${id} (no /dev/full here)"
+		continue
+	fi
 	read -r -a args <<<"${argv}"
 	for b in "${bindings[@]}"; do
 		name="${b%%|*}"; cli="${b#*|}"
 		rc=0
 		case "${stdinSpec}" in
 			@closedin)  "${cli}" "${args[@]}" >"${tmpDir}/out" 2>"${tmpDir}/err" 0<&- || rc=$? ;;
 			@closedout) "${cli}" "${args[@]}" 2>"${tmpDir}/err" >&- || rc=$?; : >"${tmpDir}/out" ;;
+			@fullout)   "${cli}" "${args[@]}" 2>"${tmpDir}/err" >/dev/full || rc=$?; : >"${tmpDir}/out" ;;
+			@fullerr)   "${cli}" "${args[@]}" >"${tmpDir}/out" 2>/dev/full || rc=$?; : >"${tmpDir}/err" ;;
 			-)          "${cli}" "${args[@]}" >"${tmpDir}/out" 2>"${tmpDir}/err" </dev/null || rc=$? ;;
 			*)          printf '%b' "${stdinSpec}" | "${cli}" "${args[@]}" >"${tmpDir}/out" 2>"${tmpDir}/err" || rc=$? ;;
 		esac