fix round 20260901b

changelog.md

@@ -72,6 +72,32 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ### Fixed
 
+- A line whose indent matches no open level (`E012`) and a `*` line with no space after it (`E013`) now hold their indent level, so what is written under them is skipped with them (`E018`) instead of attaching one level up, a fence line at a bad indent takes its whole body with it instead of parsing it as top-level bindings, and a second line at the same bad indent is refused the same way rather than binding.
+
+- A value after an index selector on the last segment (`a[0]: 2`) was dropped with no diagnostic and no lost count, so an in-place write deleted it at exit 0. It is reported (`E002`) and counted as lost now, as a value after a value selector always was.
+
+- A fragment mounted at one node by two schema paths reported every fault under it twice.
+
+- The `H001`/`H002` hints a schema disavows were matched on the schema's raw text, so a field path with an escaped quote in it kept its hint, and a `repeat` or `reopen` that faulted (`repeat: 0x2`, three elements) still silenced it. Both now go by the built schema.
+
+- A merge appended a layer's unmatched nodes grouped by name instead of in that file's order, and dropped a footer comment the layer repeated itself. Merging onto an empty document is the identity again.
+
+- The did-you-mean suggestion cost a full edit-distance table per name pair, so a schema and document with long field names took seconds to minutes to check. The distance is capped at the threshold and computed within that band, so it is linear in the name length.
+
+- Every binding spells a float the same way. C printed 17 digits for 46 exact powers of two where 16 read back, and the reference rounded an exact tie between two shortest spellings away from zero where Go, Python and C round to even (`2.9802322387695312e-08` came back as `...313` from one and `...312` from the other three). Ties round to even everywhere now.
+
+- On Windows the CLI aborted when the program reading its output closed early (`fmt` piped into `more`); it exits quietly now, as it dies quietly of SIGPIPE elsewhere.
+
+- The C CLI on Windows took its arguments in the active code page, so a path outside it was refused and a name the page best-fits (`ā.shcl` to `a.shcl`) reached the wrong file, `--write` included. It reads the wide command line now, and the header says paths are UTF-8 on every platform. A failed publish on Windows also left errno at 0, so the CLI printed `Success` beside its failure exit; the Win32 error is mapped onto errno now.
+
+- The `.deb` and `.rpm` declared no dependencies, so they installed on a system whose glibc is older than the binary needs and the binary then failed to load. They declare the glibc floor and libgcc read off the binary, and the deb carries its copyright and changelog files.
+
+- A system install under a tight umask left a bin or man1 directory the installer had to create root-only; the "not on your PATH" note fired when the directory was on PATH with a trailing slash.
+
+- The C++ veneer's `to_canonical()` never gave the read memory back, so a save loop grew without bound.
+
+- Go's `Diagnostics()` and both suppress filters could hand back a slice sharing the document's own backing array.
+
 - A file of lines with no colon at a constant indent parsed in quadratic time - a 1 MB plain text file took half a minute, and neither `ParseLimited` cap could stop it because no nodes or elements were built. Each refused line is kept as trivia, and every following line rewalked the whole retained list. The list is walked only as far as an incoming line could change it now, so the parse is linear again.
 
 - `ParseLimited`'s element cap bounded nothing for an inline array: the line was built in full and refused afterwards, so 9 MB of input peaked at the same 256 MB with the cap as without, and in C the refused array stayed held for the document's lifetime. The count is taken before anything splits the value now, so a refused line costs its text and no more.

cicd/config.bash

@@ -245,7 +245,9 @@ LARGEDOC_MIB=100
 ## enough to sample meaningfully: the large-document generator at a few MiB. It
 ## used to be forty copies of the corpus concatenated, which made four lines in
 ## five merge into an existing node, so the graph measured merging and hint
-## formatting rather than parsing.
+## formatting rather than parsing. stderr is closed off on every run: a hint
+## per unit once put a million lines in the log and a third of the samples in
+## the write path.
 ## Kernel perf is locked down on this box (perf_event_paranoid=3), hence the
 ## in-process sampler. PROFILE_WORKLOAD_GEN / PROFILE_RUN are eval'd by the engine
 ## with PROFILE_WORKLOAD / PROFILE_OUT / PROFILE_SECS exported.
@@ -255,18 +257,18 @@ PROFILE_BUILD_CMD=(cargo build --profile profiling --features profiling -j "${CP
 PROFILE_BIN="source/rust/target/profiling/${EXE_NAME}"
 PROFILE_OUT_DIR="cicd/artifacts/profiling"   ## relative to repo root; gitignored
 PROFILE_WORKLOAD_GEN='source cicd/utility/include/largedoc-gen.bash; largedoc_gen 4 > "${PROFILE_WORKLOAD}"'
-PROFILE_RUN='SHCL_PROFILE_OUT="${PROFILE_OUT}" SHCL_PROFILE_SECS="${PROFILE_SECS}" "${PROFILE_BIN}" fmt "${PROFILE_WORKLOAD}" >/dev/null'
+PROFILE_RUN='SHCL_PROFILE_OUT="${PROFILE_OUT}" SHCL_PROFILE_SECS="${PROFILE_SECS}" "${PROFILE_BIN}" fmt "${PROFILE_WORKLOAD}" >/dev/null 2>&1'
 ## Wall-clock per surface, logged after the flamegraph: the graph shows where
 ## time goes inside fmt, these catch merge/validate/generate/set/read going
 ## quadratic without moving a sample. "name|command"; nonzero exit = FAILED
 ## (append `|| [ $? -eq N ]` where a nonzero exit is the expected outcome).
 PROFILE_TIMED=(
-	'fmt|"${PROFILE_BIN}" fmt "${PROFILE_WORKLOAD}" >/dev/null'
-	'merge|"${PROFILE_BIN}" fmt --layer="${PROFILE_WORKLOAD}" "${PROFILE_WORKLOAD}" >/dev/null'
-	'reads|"${PROFILE_BIN}" instances "${PROFILE_WORKLOAD}" service >/dev/null && "${PROFILE_BIN}" count "${PROFILE_WORKLOAD}" service >/dev/null'
+	'fmt|"${PROFILE_BIN}" fmt "${PROFILE_WORKLOAD}" >/dev/null 2>&1'
+	'merge|"${PROFILE_BIN}" fmt --layer="${PROFILE_WORKLOAD}" "${PROFILE_WORKLOAD}" >/dev/null 2>&1'
+	'reads|"${PROFILE_BIN}" instances "${PROFILE_WORKLOAD}" service >/dev/null 2>&1 && "${PROFILE_BIN}" count "${PROFILE_WORKLOAD}" service >/dev/null 2>&1'
 	'validate|"${PROFILE_BIN}" check --schema=project/conformance/021-schema-valid/schema.shcl "${PROFILE_WORKLOAD}" >/dev/null 2>&1 || [ $? -eq 6 ]'
 	'generate|"${PROFILE_BIN}" init --schema=project/conformance/026-init-schema/init-schema.shcl >/dev/null'
-	'set|printf "int\tprofile.k\t1\nstring\tprofile.s\tv\nremove\tprofile.k\n" | "${PROFILE_BIN}" set "${PROFILE_WORKLOAD}" >/dev/null'
+	'set|printf "int\tprofile.k\t1\nstring\tprofile.s\tv\nremove\tprofile.k\n" | "${PROFILE_BIN}" set "${PROFILE_WORKLOAD}" >/dev/null 2>&1'
 )
 
 ## Stage 6: native release + cross targets. One per line:

cicd/packaging/nfpm.yaml

@@ -15,6 +15,17 @@ description: |
   bindings (Rust, Go, Python, C/C++) and Bash/PowerShell wrappers.
 homepage: https://github.com/jim-collier/shcl
 license: MIT
+# What the binary links against, read off the binary itself by package.bash:
+# the glibc floor is its newest GLIBC_ version, and libgcc joins only when
+# libgcc_s is in NEEDED. Without these a package installs on a system where
+# the binary then dies in the loader.
+overrides:
+  deb:
+    depends:
+      - libc6 (>= ${SHCL_GLIBC})${SHCL_DEB_LIBGCC}
+  rpm:
+    depends:
+      - glibc >= ${SHCL_GLIBC}${SHCL_RPM_LIBGCC}
 rpm:
   # rpm stamps the building machine's hostname into the package; a fixed name
   # keeps two builds of one commit byte-identical (the build date comes from
@@ -35,6 +46,15 @@ contents:
     dst: /usr/share/man/man1/shcl.1.gz
     file_info:
       mode: 0644
+  # Debian policy wants both under /usr/share/doc/<package>.
+  - src: ${SHCL_PAYLOAD}/doc/copyright
+    dst: /usr/share/doc/shcl/copyright
+    file_info:
+      mode: 0644
+  - src: ${SHCL_PAYLOAD}/doc/changelog.gz
+    dst: /usr/share/doc/shcl/changelog.gz
+    file_info:
+      mode: 0644
   # Completions go to the shell's own directory, not under /usr/share/shcl:
   # bash-completion and zsh only autoload from theirs.
   - src: ${SHCL_PAYLOAD}/completions/shcl.bash

cicd/utility/crosscheck.bash

@@ -377,6 +377,21 @@ fCompare "fmt --write rejects --set" fmt --write --set a=1 missing.shcl
 fCompareWrite "set --write creates a missing file" fFixAbsent set --write --set a=1
 fCompareWrite "fmt --write still refuses a missing file" fFixAbsent fmt --write
 
+# Float spelling: shortest-round-trip formatters may lawfully differ at a
+# power of two (a lopsided rounding interval) and on an exact tie between two
+# spellings of the shortest length. Every power of two, plus a fixed set of
+# random doubles built as exact m * 2^e so the text reads back to the double
+# it names, through a float write in each binding.
+awk 'BEGIN{
+	for (e = -1074; e <= 1023; e++) printf "float\tp%d\t%.17g\n", e + 1074, 2 ^ e;
+	srand(20260902);
+	for (i = 0; i < 3000; i++) {
+		m = int(rand() * 9007199254740992); e = int(rand() * 1900) - 1000;
+		printf "float\tr%d\t%.17g\n", i, m * 2 ^ e;
+	}
+}' > "${tmpDir}/floats.ops"
+fCompareStdin "float spelling" "${tmpDir}/floats.ops" set -
+
 # `set -` follows stdin, so the same spelling means two things and both are
 # pinned: the piped document when an option holds the edits, an empty base when
 # stdin is the ops script instead.

cicd/utility/flame-report.py

@@ -25,7 +25,7 @@
 from pathlib import Path
 from typing import NoReturn
 
-STEP      = 16              # flamegraph row height in the SVG, px (a child sits at parent_y - STEP)
+STEP      = 16.0            # flamegraph row height in the SVG, px, until fParse reads the real one off the rows
 SELF_TOP  = 22             	# self-time leaders to list
 INCL_TOP  = 14             	# inclusive-time buckets to list
 CHAIN     = 4              	# leaves whose caller chain we walk up to the root
@@ -64,7 +64,14 @@ def fAttr(attrs: str, key: str) -> float | None:
 
 
 def fParse(path: Path) -> tuple[int, list[Frame]]:
-	text = path.read_text(encoding="utf-8")
+	##	Skip rather than trust anything that is not a whole graph: a file cut
+	##	off mid-write, or one whose rows are not where the row height says,
+	##	still yields a sample count and some frames, and the report it gave
+	##	summed to a fraction of the samples with the marker written. The row
+	##	height comes off the rows themselves; the root and the self-time sum
+	##	are the checks that a graph is all there.
+	global STEP
+	text = path.read_text(encoding="utf-8", errors="replace")
 	m = re.search(r'total_samples="(\d+)"', text)
 	total = int(m.group(1)) if m else 0
 	frames: list[Frame] = []
@@ -77,6 +84,17 @@ def fParse(path: Path) -> tuple[int, list[Frame]]:
 		frames.append((name, x, y, w))
 	if not total or not frames:
 		fSkip(f"could not parse a flamegraph out of {path}")
+	if not text.rstrip().endswith("</svg>"):
+		fSkip(f"{path} does not end in </svg>; cut off mid-write")
+	ys = sorted({fr[2] for fr in frames})
+	gaps = {round(b - a, 6) for a, b in zip(ys, ys[1:])}
+	if len(gaps) > 1:
+		fSkip(f"rows are not evenly spaced in {path} (gaps {sorted(gaps)}); not a whole flamegraph")
+	if gaps:
+		STEP = gaps.pop()
+	roots = [fr for fr in frames if fr[2] == ys[-1]]
+	if len(roots) != 1 or abs(roots[0][3] - total) > 1e-6:
+		fSkip(f"no single root frame spanning all {total} samples in {path}; not a whole flamegraph")
 	return total, frames
 
 
@@ -116,6 +134,8 @@ def parent(fr: Frame) -> Frame | None:
 		return None
 
 	selfOf = {fr: fr[3] - sum(c[3] for c in kids(fr)) for fr in frames}
+	if abs(sum(selfOf.values()) - total) > 1 or min(selfOf.values()) < -1e-6:
+		fSkip(f"self time sums to {sum(selfOf.values()):.0f} of {total} samples; frames are missing or misplaced, not a whole flamegraph")
 
 	selfBy: dict[str, float] = {}
 	inclBy: dict[str, float] = {}
@@ -238,3 +258,5 @@ def main() -> None:
 ##	History:
 ##		- 20260712: Created from the SilkTerm sibling; attribution buckets redone for shcl (parse / emit / reads).
 ##		- 20260829: Regexes compiled once; rows indexed and bisected instead of scanned per frame.
+##		- 2026-09-02 JC: A truncated or reshaped graph skips at exit 2 instead of
+##		  reporting a fraction of it; the row height is read off the rows.

cicd/utility/include/largedoc-gen.bash

@@ -9,7 +9,9 @@
 ##		one name, nesting, inline and bullet arrays, quoted values holding the
 ##		separator, raw blocks, comments, blank lines, non-ASCII, and one array
 ##		long enough to walk past any fixed element buffer. Nothing in it merges
-##		into an earlier line, so a profile of it measures parsing, not merging.
+##		into an earlier line, so a profile of it measures parsing, not merging,
+##		and it loads with no diagnostics at all (the large-document gate
+##		checks that; a hint per unit once made the profile measure stderr).
 ##	History: At bottom of script.
 
 ##	Copyright © 2026 Bubbles (ID: XଌฅრX۳ᛟԃლፀƅꓩหδლც)
@@ -38,7 +40,7 @@ largedoc_gen(){
 			    "\t\tburst:\n"
 			for (j = 0; j < 4; j++) s = s "\t\t\t* " ((j*i)%1000) "\n"
 			s = s "\tnotes:\n\t\t~~~\n\t\tgenerated entry " i "\n\t\tsecond line\n\t\t~~~\n" \
-			      "service: svc" i "\n\tport: " (9000 + i%1000) "\n\n"
+			      "service: svc" i "-b\n\tport: " (9000 + i%1000) "\n\n"
 			printf "%s", s
 			bytes += length(s)
 		}
@@ -56,3 +58,5 @@ declare -i isSourced_ldg7c=0; [[ "${BASH_SOURCE[0]}" == "${0}" ]] || isSourced_l
 ##	History:
 ##		- 2026-08-29 JC: Lifted out of largedoc.bash so the profiler runs the
 ##		  same document instead of forty concatenated copies of the corpus.
+##		- 2026-09-02 JC: The second instance of each service carries its own value;
+##		  it used to reopen the first, so half the nodes merged and every unit hinted.

cicd/utility/largedoc.bash

@@ -155,11 +155,20 @@ refOut="${work}/out-${refName}.shcl"
 if [[ -s "${refOut}" ]]; then
 	refCli="${bindings[0]#*|}"
 	echo
-	if "${refCli}" fmt "${refOut}" | cmp -s - "${refOut}"; then
+	if "${refCli}" fmt "${refOut}" 2>/dev/null | cmp -s - "${refOut}"; then
 		echo "largedoc: fmt is a fixpoint at ${actualMib} MiB"
 	else
 		echo "largedoc: FAILED: fmt is not a fixpoint at ${actualMib} MiB" >&2; rc=1
 	fi
+	## The generator promises a document nothing in it merges into and that
+	## loads clean; the profiler's numbers depend on it. A hint per unit once
+	## made the profile measure stderr.
+	summary="$("${refCli}" check "${doc}" 2>/dev/null | tail -1 || true)"
+	if [[ "${summary}" == "ok (0 diagnostic(s))" ]]; then
+		echo "largedoc: the generated document loads with no diagnostics"
+	else
+		echo "largedoc: FAILED: the generated document does not load clean: ${summary}" >&2; rc=1
+	fi
 	wideCount="$("${refCli}" get --int --array "${refOut}" wide | grep -c '' || true)"
 	if ((wideCount == 20000)); then
 		echo "largedoc: long array read back whole (${wideCount} elements)"

cicd/utility/lint-report.bash

@@ -85,10 +85,12 @@ fi
 ##	run's log (a passing run aborts on the first error). Drop the "0 warnings" noise,
 ##	govulncheck's clean-result prose (its "found N vulnerabilities in modules you
 ##	require, but your code doesn't appear to call" block is informational and prints
-##	on every run), and the echoed cppcheck command line, which carries the word.
+##	on every run), and the echoed command lines that carry the word: cppcheck's
+##	`--enable=warning`, and clippy's `-D warnings`, which the pre-push gate's
+##	nested run echoes during publish and which used to read as a finding.
 warns="$(grep -inE 'warning|rustsec-|vulnerab|unmaintained|yanked|error\[' "$log" 2>/dev/null \
 	| grep -viE 'generated 0 warnings|: 0 warnings|no warnings|0 warnings emitted' \
-	| grep -viE 'no vulnerabilities found|affected by 0 vulnerabilities|this scan also found|appear to call|^[0-9]+:these vulnerabilities\.$|enable=warning' || true)"
+	| grep -viE 'no vulnerabilities found|affected by 0 vulnerabilities|this scan also found|appear to call|^[0-9]+:these vulnerabilities\.$|enable=warning|-D warnings' || true)"
 if [[ -n "$warns" ]]; then n=$(printf '%s\n' "$warns" | grep -c .); else n=0; fi
 
 tag="FLAG"; ((check)) && tag="NEW"
@@ -103,3 +105,4 @@ fi
 
 ##	Script history:
 ##		- 20260709: Created.
+##		- 20260902: The echoed `-D warnings` of a nested clippy run is not a finding.