Skip to content

jsonwrite writes valid, separate arrays (4.2.37) - #796

Merged
ddehilster merged 2 commits into
VisualText:masterfrom
ddehilster:fix/jsonwrite-arrays
Oct 19, 2026
Merged

jsonwrite writes valid, separate arrays (4.2.37)#796
ddehilster merged 2 commits into
VisualText:masterfrom
ddehilster:fix/jsonwrite-arrays

Conversation

@ddehilster

@ddehilster ddehilster commented Oct 19, 2026

Copy link
Copy Markdown
Member

jsonwrite turns counted concepts (item1, item2) into a JSON array. It wrote each separator after a child, before it knew what came next. Four shapes came out wrong:

Children Before After
item1 item2 name …}, } ] "name": {…}, not JSON "item": [ … ], "name": {…}
item1 item2 thing1 thing2 one "item" array with four elements "item" and "thing" arrays
a1b1 a1b2 array "a1", ids "11" and "12" array "a1b", ids "1" and "2"
row1 (no attributes, no children) "id": "1", followed by }, not JSON "id": "1" }

The first row is the common one: a list followed by a plain field.

These bugs were copied on purpose from JsonKBRecurse and SeparateConCount in KBFuncs.nlp, so the builtin would stay byte-identical; the comment in json.cpp asked for the two to be fixed together. A companion visualtext-files PR applies the same fix to KBFuncs.nlp. With both, builtin and NLP++ output still match byte for byte (checked on all six test shapes).

Already-valid shapes are unchanged. A plain sibling followed by an array, and arrays at the end, are written byte for byte as before, including the array that opens on the same line as the preceding },.

Test. The new fixture tests/json-arrays pins five shapes to expected/*.json and round-trips each through jsonparse. 4.2.36 fails all five; this build passes all five, along with the existing JSON builtins step. Built and run locally on Windows (x64 Release). Linux runs only in CI.

🤖 Generated with Claude Code

@ddehilster @claude
jsonwrite writes valid, separate arrays (4.2.37)
1b6e92e 
jsonwrite grouped counted concepts (item1, item2) into a JSON array but
wrote each separator after a child instead of before it. An array
followed by a plain sibling -- a list of items, then a field -- came out
with a stray "}" and no comma after the "]", so the file was not JSON.
Two arrays side by side (item1 item2 thing1 thing2) merged into one,
and an element with no attributes or children ended in a trailing
comma. separate_con_count also kept collecting digits past a letter,
so "a1b2" became array "a1" with id "12".

All of this was copied on purpose from JsonKBRecurse and
SeparateConCount in KBFuncs.nlp, to stay byte-identical; the matching
KBFuncs fix goes to visualtext-files and its copies, so the two still
agree. Every shape that was already valid JSON is written byte for byte
as before.

New fixture tests/json-arrays pins five shapes to expected files and
round-trips each through jsonparse; the 4.2.36 engine fails all five.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 19, 2026
Merged
Merged
Merged
Merged
Merged
Merged
@ddehilster @claude
JSON arrays CI step: skip ubuntu-20.04, ignore CR in expected files
bd9d38c 
The Windows checkout converts expected/*.json to CRLF, and the step
stripped CR from the output only, so every line differed. On Linux the
step lacked the Non-20.04 guard every other test step has; that job
builds in Docker and its binary cannot run on the host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ddehilster
ddehilster merged commit db7e163 into VisualText:master Oct 19, 2026
7 checks passed
ddehilster added a commit to VisualText/visualtext-files that referenced this pull request Oct 19, 2026
@ddehilster @claude
KBFuncs and UtilFuncs no longer wrap a computed strpiece position
39e533b 
Since engine 4.2.15 a negative strpiece position counts from the end, so
a position computed as strlength(s) - k wraps round instead of running
off the start when s is short. Fixed in the shared library:

KBFuncs.nlp
- StripEndDigits started its scan one character early: a one-letter name
  only came back right because position -1 wrapped, and a name with no
  trailing digits lost its last letter ("item" -> "ite"). Rewritten as a
  scan that only counts up from 0.
- SeparateConCount computed the base as strlength(name) -
  strlength(num) - 1, and never stopped at a letter (its else branch was
  the no-op `L("stop");`), so "a1b2" was base "a1", count "12". It now
  takes only the trailing digits, with positions counted up from 0.
- JsonKBRecurse wrote each separator after a child, so an array followed
  by a plain sibling was not JSON (stray "}", no comma after "]"), two
  arrays side by side merged, and an element with no attributes ended in
  a trailing comma. Separators are now written before each child. It
  also wrote a hier.txt debug file on every call; removed. Every shape
  that was valid JSON is written byte for byte as before. The jsonwrite
  builtin had the same bugs and is fixed to match (nlp-engine 4.2.37,
  VisualText/nlp-engine#796), so JsonKB and jsonwrite still agree.

UtilFuncs.nlp
- prosifyrec trimmed with strpiece(txt, 0, del - 1) in two branches
  without the del > 0 guard its third branch got in 2011; at del 0 the
  end is -1 and took the whole text instead of none of it.

The other computed positions in spec/, misc/ and languages/ subtract
only 1 or sit behind a length test, and are left alone.

Help: strpiece.md explains what changed for computed positions, with the
"," example and the -k idiom, and that out-of-range positions log an
error; jsonwrite.md and New_in_Current_Version.md (4.2.15, 4.2.37)
describe both fixes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ddehilster added a commit to VisualText/analyzer-templates that referenced this pull request Oct 19, 2026
@ddehilster @claude
KBFuncs: no wrapped strpiece positions, valid JSON arrays
ad3b330 
Brings this repo's KBFuncs.nlp copies in step with the canonical
visualText/spec/KBFuncs.nlp fix (VisualText/visualtext-files, same
branch name). The functions were replaced by one script, and only where
a copy held the known old text, so every copy got the same code:

- StripEndDigits: a one-letter name only came back right because
  position -1 wraps since engine 4.2.15, and a name with no trailing
  digits lost its last letter ("item" -> "ite"). It now counts positions
  up from 0.
- SeparateConCount (where present): only the trailing digits count
  ("a1b2" is "a1b" + "2", not "a1" + "12"); no position arithmetic that
  can go negative.
- JsonKBRecurse (where present): an array followed by a plain sibling is
  now valid JSON, adjacent arrays with different names stay separate,
  and the stray hier.txt debug file is gone. Shapes that were valid JSON
  are written byte for byte as before; matches the jsonwrite builtin fix
  in VisualText/nlp-engine#796.

Each analyzer was run on its inputs before and after with engine 4.2.36:
output unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ddehilster added a commit to VisualText/analyzers that referenced this pull request Oct 19, 2026
@ddehilster @claude
KBFuncs: no wrapped strpiece positions, valid JSON arrays
5940917 
Brings this repo's KBFuncs.nlp copies in step with the canonical
visualText/spec/KBFuncs.nlp fix (VisualText/visualtext-files, same
branch name). The functions were replaced by one script, and only where
a copy held the known old text, so every copy got the same code:

- StripEndDigits: a one-letter name only came back right because
  position -1 wraps since engine 4.2.15, and a name with no trailing
  digits lost its last letter ("item" -> "ite"). It now counts positions
  up from 0.
- SeparateConCount (where present): only the trailing digits count
  ("a1b2" is "a1b" + "2", not "a1" + "12"); no position arithmetic that
  can go negative.
- JsonKBRecurse (where present): an array followed by a plain sibling is
  now valid JSON, adjacent arrays with different names stay separate,
  and the stray hier.txt debug file is gone. Shapes that were valid JSON
  are written byte for byte as before; matches the jsonwrite builtin fix
  in VisualText/nlp-engine#796.

Each analyzer was run on its inputs before and after with engine 4.2.36:
output unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ddehilster added a commit to VisualText/nlpfix-analyzers that referenced this pull request Oct 19, 2026
@ddehilster @claude
KBFuncs: no wrapped strpiece positions, valid JSON arrays
65a45e4 
Brings this repo's KBFuncs.nlp copies in step with the canonical
visualText/spec/KBFuncs.nlp fix (VisualText/visualtext-files, same
branch name). The functions were replaced by one script, and only where
a copy held the known old text, so every copy got the same code:

- StripEndDigits: a one-letter name only came back right because
  position -1 wraps since engine 4.2.15, and a name with no trailing
  digits lost its last letter ("item" -> "ite"). It now counts positions
  up from 0.
- SeparateConCount (where present): only the trailing digits count
  ("a1b2" is "a1b" + "2", not "a1" + "12"); no position arithmetic that
  can go negative.
- JsonKBRecurse (where present): an array followed by a plain sibling is
  now valid JSON, adjacent arrays with different names stay separate,
  and the stray hier.txt debug file is gone. Shapes that were valid JSON
  are written byte for byte as before; matches the jsonwrite builtin fix
  in VisualText/nlp-engine#796.

Each analyzer was run on its inputs before and after with engine 4.2.36:
output unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ddehilster added a commit to VisualText/nlp-tutorials that referenced this pull request Oct 19, 2026
@ddehilster @claude
KBFuncs: no wrapped strpiece positions, valid JSON arrays
28458cd 
Brings this repo's KBFuncs.nlp copies in step with the canonical
visualText/spec/KBFuncs.nlp fix (VisualText/visualtext-files, same
branch name). The functions were replaced by one script, and only where
a copy held the known old text, so every copy got the same code:

- StripEndDigits: a one-letter name only came back right because
  position -1 wraps since engine 4.2.15, and a name with no trailing
  digits lost its last letter ("item" -> "ite"). It now counts positions
  up from 0.
- SeparateConCount (where present): only the trailing digits count
  ("a1b2" is "a1b" + "2", not "a1" + "12"); no position arithmetic that
  can go negative.
- JsonKBRecurse (where present): an array followed by a plain sibling is
  now valid JSON, adjacent arrays with different names stay separate,
  and the stray hier.txt debug file is gone. Shapes that were valid JSON
  are written byte for byte as before; matches the jsonwrite builtin fix
  in VisualText/nlp-engine#796.

Each analyzer was run on its inputs before and after with engine 4.2.36:
output unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ddehilster added a commit to VisualText/parse-en-us that referenced this pull request Oct 19, 2026
@ddehilster @claude
KBFuncs and funs: no wrapped strpiece positions
abc1bbf 
Brings this repo's KBFuncs.nlp copies in step with the canonical
visualText/spec/KBFuncs.nlp fix (VisualText/visualtext-files, same
branch name). The functions were replaced by one script, and only where
a copy held the known old text, so every copy got the same code:

- StripEndDigits: a one-letter name only came back right because
  position -1 wraps since engine 4.2.15, and a name with no trailing
  digits lost its last letter ("item" -> "ite"). It now counts positions
  up from 0.
- SeparateConCount (where present): only the trailing digits count
  ("a1b2" is "a1b" + "2", not "a1" + "12"); no position arithmetic that
  can go negative.
- JsonKBRecurse (where present): an array followed by a plain sibling is
  now valid JSON, adjacent arrays with different names stay separate,
  and the stray hier.txt debug file is gone. Shapes that were valid JSON
  are written byte for byte as before; matches the jsonwrite builtin fix
  in VisualText/nlp-engine#796.

funs.nlp: prosifyrec trimmed with strpiece(txt, 0, del - 1) in two
branches without the del > 0 guard its third branch got in 2011; at del 0
the end is -1 and took the whole text instead of none of it. Same fix as
visualText/spec/UtilFuncs.nlp.

Each analyzer was run on its inputs before and after with engine 4.2.36:
output unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ddehilster