Repository navigation
KBFuncs and UtilFuncs no longer wrap a computed strpiece position - #247
Merged
ddehilster merged 1 commit intoOct 19, 2026
Merged
Conversation
Since engine 4.2.15 a negative strpiece position counts from the end, so
a position computed as strlength(s) - k wraps round instead of running
off the start when s is short. Fixed in the shared library:
KBFuncs.nlp
- StripEndDigits started its scan one character early: a one-letter name
only came back right because position -1 wrapped, and a name with no
trailing digits lost its last letter ("item" -> "ite"). Rewritten as a
scan that only counts up from 0.
- SeparateConCount computed the base as strlength(name) -
strlength(num) - 1, and never stopped at a letter (its else branch was
the no-op `L("stop");`), so "a1b2" was base "a1", count "12". It now
takes only the trailing digits, with positions counted up from 0.
- JsonKBRecurse wrote each separator after a child, so an array followed
by a plain sibling was not JSON (stray "}", no comma after "]"), two
arrays side by side merged, and an element with no attributes ended in
a trailing comma. Separators are now written before each child. It
also wrote a hier.txt debug file on every call; removed. Every shape
that was valid JSON is written byte for byte as before. The jsonwrite
builtin had the same bugs and is fixed to match (nlp-engine 4.2.37,
VisualText/nlp-engine#796), so JsonKB and jsonwrite still agree.
UtilFuncs.nlp
- prosifyrec trimmed with strpiece(txt, 0, del - 1) in two branches
without the del > 0 guard its third branch got in 2011; at del 0 the
end is -1 and took the whole text instead of none of it.
The other computed positions in spec/, misc/ and languages/ subtract
only 1 or sit behind a length test, and are left alone.
Help: strpiece.md explains what changed for computed positions, with the
"," example and the -k idiom, and that out-of-range positions log an
error; jsonwrite.md and New_in_Current_Version.md (4.2.15, 4.2.37)
describe both fixes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since engine 4.2.15 a negative
strpieceposition counts from the end. Code that computes a position asstrlength(s) - know goes wrong whensis shorter thank: the position comes out negative and wraps round to the end, where it used to run off the start.strpiece(",", 0, strlength(",") - 2)gives","instead of"", and a trim loop written that way hung a production analyzer on a one-character value.What changed
spec/KBFuncs.nlpStripEndDigitsstarted its scan one character early. A one-letter name only came back right because -1 wrapped, and a name with no trailing digits lost its last letter. Rewritten so positions only count up from 0.SeparateConCountcomputed the base asstrlength(name) - strlength(num) - 1. Its else branch was the no-opL("stop");, so it never stopped at a letter. It now takes only the trailing digits, with no subtraction that can go negative.JsonKBRecursewrote each separator after a child:}, and no comma after the]).Separators now go out before each child. Every shape that was already valid JSON is written byte for byte as before. The debug write of
hier.txton every call is gone. New helper:JsonHasAttributes.The
jsonwritebuiltin copied these bugs on purpose. It's fixed the same way in jsonwrite writes valid, separate arrays (4.2.37) nlp-engine#796 (4.2.37), soJsonKBandjsonwritestill give identical output.spec/UtilFuncs.nlpprosifyrectrimmed withstrpiece(txt, 0, del - 1)in two branches without thedel > 0guard its third branch got in 2011. Atdel0 the end is -1, which took the whole text instead of none of it.spec/,misc/andlanguages/subtracts only 1 or sits behind a length test, so it can't wrap. Checked by hand: themorphhelpers (behindlen > 4), the suffix confidence code (behindlen > 2, and unreachable),numth, the percentage formatter, and the word remover with its# Sanity.checks.Help
strpiece.md: what changed for computed positions, with the","example and the-kidiom. Also that out-of-range positions return""but writestrpiece: Error.to err.log, except a start past the end, which doesn't log.jsonwrite.mdandNew_in_Current_Version.md(new 4.2.15 and 4.2.37 entries) describe both fixes.Before and after
Run with a one-pass test analyzer calling the functions; old = current main, new = this branch.
StripEndDigitsold → newSeparateConCountold → new""0→00→0aa→a(old was right only because -1 wrapped)0→0120→00→0abc12abc→abcabc, 12 →abc, 12item3item→itemitem, 3 →item, 3itemite→item0→0a1b2a1b→a1ba1, 12 →a1b, 2The JSON shapes below were written by
JsonKBRecurseand byjsonwrite. Old = engine 4.2.36 with current KBFuncs; new = the #796 engine with this KBFuncs.item1 item2 nameitem1 item2 thing1 thing2a1b1 a1b2a1a1bname item1 item2row1(bare),row2→child,tailNo output change in real analyzers. I patched the KBFuncs copies in VisualText/analyzer-templates, analyzers, nlp-tutorials, nlpfix-analyzers and parse-en-us (companion PRs). Then I ran each analyzer that carries a copy on its inputs before and after: 18 analyzers, plus parse-en-us. All output was unchanged; only the analyzer's own path in
init.logdiffered. With UtilFuncs,prosifygives the same result for every limit from 1 to 30.Help/check-help-docs.ps1passes; its one orphan,New_in_Current_Version, was already missing fromindex.md.🤖 Generated with Claude Code