docs(help): jsonwrite and jsonparse, and rewire JsonKB to the builtin

Help/markdown/New_in_Current_Version.md

@@ -20,6 +20,8 @@ The ANSI version shares the same source code with the Unicode version, but repli
 
 ## Miscellaneous
 
+(3.8.2) **JSON in and out as built-in functions.** [jsonwrite](jsonwrite.md) replaces the `JsonKB` family in `KBFuncs.nlp` with a built-in that produces byte-identical output. Serializing the knowledge base is where a knowledge-base analyzer spends most of its time -- on the date-time analyzer the thirteen-line output pass that calls `JsonKB` and `SaveKB` was 43.9% of total runtime -- and the built-in is about 3x faster on that pass. `JsonKB` is now a one-line wrapper, so an analyzer picks up the speedup by refreshing its copy of `KBFuncs.nlp`; newly created analyzers get it automatically. [jsonparse](jsonparse.md) is the inverse and reads a JSON document into concepts, which previously required the `json2kbb` Python library pass -- that needs Python on the machine, which the npm and pypi distributions do not provide, and it silently skips rebuilding when a stale `.kbb` is present. **FIXED:** a python pass shelled out to the literal command `python`, absent on macOS 12.3+ and most Linux distributions, and reported failure only when `system()` returned a negative value -- which never happens for command-not-found, so a failing pass was silent and its analyzer ran on with no data (3.8.1).
+
 (3.8.0) **User functions may now shadow built-in functions.** Defining a function in a [@DECL](DECL.md) region whose name is also a built-in used to be a hard error, so adding any new built-in broke every existing analyzer that had defined a function of that name. Now the user's definition wins, the analyzer builds, and a warning names the shadowed function. This is what lets [arrayreverse](arrayreverse.md) and [push](push.md) become built-ins without disturbing the hand-written versions that have long been in `visualText/spec/UtilFuncs.nlp`, and it means future built-ins can be added safely.
 
 (3.8.0) **String and array utilities.** [split](split.md) had no inverse, so building a delimited string meant a loop and a "first time through" flag: [strjoin](strjoin.md) now does it in one call. There was also no way to get the position of a substring — [strchr](strchr.md) takes a single character and returns a tail rather than an index, and [strcontains](strcontains.md) answers only yes or no — so [strindexof](strindexof.md) and [strlastindexof](strlastindexof.md) have been added. Every existing sort was knowledge-base or parse-tree oriented, leaving plain arrays with none, so there are now [arraysort](arraysort.md) (lexical or numeric, ascending or descending, stable), [arrayunique](arrayunique.md), [arrayreverse](arrayreverse.md) and [arrayslice](arrayslice.md). [push](push.md), previously listed as UNIMPLEMENTED, now works. All of these leave the source array alone and return a new one. **FIXED:** [str](str.md) converted signed values with an unsigned format, so every negative number printed as roughly 1.8e19 — `str(-1)` gave "18446744073709551615", which also made the documented -1 returns of [filesize](filesize.md) and the new index functions unreadable. The same fault was in the compiled-analyzer path and in the code generator, so compiled analyzers emitted the wrong numeric literals too.

Help/markdown/Table_of_File_Functions.md

@@ -14,6 +14,8 @@ The return types include: **STR** (string), **INT** (integer), and **BOOL** (Boo
 | [**fileexists(path_str)**](fileexists.md) | **BOOL(1,0)** | True if the path is an existing regular file. |
 | [**filesize(path_str)**](filesize.md) | **INT** | Size of a regular file in bytes, or -1 if not a readable file. |
 | [**mkdir(dir_str)**](mkdir.md) | **BOOL(1,0)** | Create a directory. Operating system-dependent. |
+| [**jsonparse(json_str, parent_con)**](jsonparse.md) | **BOOL(1,0)** | Parse a JSON document into knowledge base concepts. |
+| [**jsonwrite(path_str, concept)**](jsonwrite.md) | **BOOL(1,0)** | Write a concept tree to a file as JSON. |
 | [**readfile(path_str)**](readfile.md) | **STR** | Read a whole text file into a string. |
 | [**readlines(path_str)**](readlines.md) | **STR ARRAY** | Read a text file as an array of its lines, terminators stripped. |
 

Help/markdown/index.md

@@ -269,6 +269,8 @@ to a topic.
         - [dirlist](dirlist.md)
         - [fileexists](fileexists.md)
         - [filesize](filesize.md)
+        - [jsonparse](jsonparse.md)
+        - [jsonwrite](jsonwrite.md)
         - [readfile](readfile.md)
         - [readlines](readlines.md)
       - [Formatting and I/O Functions](Table_of_Formatting_and_I_O_Functions.md)

Help/markdown/jsonparse.md

@@ -0,0 +1,70 @@
+[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md)
+
+# jsonparse
+
+## Purpose
+
+Parse a JSON document into knowledge base concepts.
+
+## Syntax
+
+```
+returnedBool = jsonparse(jsonStr, parentConcept)
+```
+
+```
+returnedBool - type: bool (1,0)
+
+jsonStr - type: str
+
+parentConcept - type: concept
+```
+
+## Returns
+
+1 if the document parsed, else 0.
+
+## Remarks
+
+The inverse of [jsonwrite](jsonwrite.md). The members of the JSON object become children and attributes of parentConcept.
+
+Pair it with [readfile](readfile.md) to load a file:
+
+```
+jsonparse(readfile(G("$apppath") + "/data/config.json"), G("kb"));
+```
+
+Before this existed, the way to get JSON into an analyzer was the `json2kbb` Python library pass, which converts `kb/user/*.json` to `.kbb` before the tokenizer. That still works, but it requires Python on the machine — which the npm and pypi distributions of the engine do not provide — and it will not overwrite an existing `.kbb`, so editing the `.json` appears to do nothing until you delete the `.kbb`. `jsonparse` has neither problem.
+
+The mapping is the one `json2kbb` documents:
+
+| JSON | knowledge base |
+| --- | --- |
+| an object | a concept |
+| `"key": <primitive>` | an attribute `key=value` |
+| `"key": { ... }` | a child concept named `key` |
+| `"key": [ ... ]` | counted children `key1`, `key2`, ... |
+| a primitive array element | `keyN` with a `value` attribute |
+
+An integer becomes a num, a number containing `.`, `e` or `E` becomes a float, `true` and `false` become 1 and 0, and `null` becomes an empty string. String escapes are resolved, including `\uXXXX` with surrogate pairs combined.
+
+A top-level object is required, and trailing content after it is an error, so a truncated or doubled document is reported rather than half-loaded. Inside an array element, an `"id"` member is skipped: [jsonwrite](jsonwrite.md) generates it from the concept's name, so keeping it would duplicate it on the next write.
+
+## Example
+
+```
+@CODE
+
+L("text") = readfile(G("$apppath") + "/data/settings.json");
+G("cfg")  = makeconcept(findroot(),"cfg");
+
+if (jsonparse(L("text"),G("cfg"))) {
+    G("threshold") = numval(findvals(findconcept(G("cfg"),"tuning"),"threshold"));
+}
+
+@@CODE
+```
+
+## See Also
+
+[jsonwrite](jsonwrite.md), [readfile](readfile.md), [makeconcept](makeconcept.md), [Knowledge Base Functions](Table_of_Knowledge_Base_Functions.md)

Help/markdown/jsonwrite.md

@@ -0,0 +1,60 @@
+[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md)
+
+# jsonwrite
+
+## Purpose
+
+Write a knowledge base concept tree to a file as JSON.
+
+## Syntax
+
+```
+returnedBool = jsonwrite(fileNameStr, concept)
+```
+
+```
+returnedBool - type: bool (1,0)
+
+fileNameStr - type: str
+
+concept - type: concept
+```
+
+## Returns
+
+1 if the file was written, else 0.
+
+## Remarks
+
+This is the built-in form of the `JsonKB` function in `KBFuncs.nlp`, with the same argument order and **byte-identical output**, so it is a drop-in replacement. `JsonKB` is now a one-line wrapper around it.
+
+Serializing the knowledge base is where a knowledge-base analyzer spends most of its time. Measured on the date-time analyzer, its `output.nlp` — thirteen lines calling `JsonKB` and `SaveKB` — was **43.9%** of total runtime, all of it inside interpreted NLP++. The built-in is roughly **3x faster** on that pass.
+
+A bare filename is written into the current input's log directory, the same place the [output operator](NLP_PP_Stuff/Operators_and_Expressions.md) `<<` writes. An absolute path is used as given.
+
+The serialization is:
+
+| knowledge base | JSON |
+| --- | --- |
+| a concept | an object |
+| an attribute | a `"name": "value"` member |
+| counted sibling concepts (`item1`, `item2`, ...) | an array under the shared base name, each element carrying an `"id"` |
+
+Two long-standing quirks of the NLP++ original are reproduced exactly, because existing analyzers' output files depend on them: indentation for level *n* is `max(1,n)` steps rather than *n*, and a concept name containing interior digits is split at the wrong place when deciding whether it is a counted sibling.
+
+## Example
+
+```
+@CODE
+
+G("results") = makeconcept(findroot(),"results");
+# ... passes accumulate into G("results") ...
+
+jsonwrite("output.json",G("results"));
+
+@@CODE
+```
+
+## See Also
+
+[jsonparse](jsonparse.md), [readfile](readfile.md), [makeconcept](makeconcept.md), [Knowledge Base Functions](Table_of_Knowledge_Base_Functions.md)

spec/KBFuncs.nlp

@@ -399,20 +399,26 @@ PadStr(L("num str"),L("pad str"),L("pad len")) {
 ###############################################
 
 JsonKB(L("file"),L("con")) {
-	L("file") << "{\n" << SpacesStr(1) << "\"" << conceptname(L("con")) << "\": {\n";
-	# Emit the top concept's OWN attributes before its child concepts.
-	# JsonKBRecurse only descends into children, so any attribute on the
-	# top concept was silently dropped. Emitting them here lets a top
-	# concept carry scalar fields alongside its nested objects (e.g. a
-	# "document_type" next to nested "data" objects). When the top concept
-	# has no attributes, output is unchanged.
-	L("hasattrs") = JsonAttributes(L("file"),L("con"),1);
-	if (down(L("con")) && L("hasattrs"))
-		L("file") << ",\n";
-	JsonKBRecurse(L("file"),L("con"),1);
-	L("file") << "\n}\n";
-}
-
+	# Delegates to the jsonwrite builtin (engine 3.8.2 and later).
+	#
+	# This used to be ~130 lines of NLP++ across JsonKB, JsonKBRecurse,
+	# JsonAttributes and JsonStr.  Serializing the KB is where a knowledge
+	# base analyzer spends most of its time -- measured on the date-time
+	# analyzer, the 13-line output pass that calls JsonKB and SaveKB was
+	# 43.9% of total runtime.  The builtin produces byte-identical output
+	# and is about 3x faster on that pass.
+	#
+	# Kept as a wrapper rather than changing every analyzer's output.nlp,
+	# so existing analyzers pick up the speedup unchanged.  An analyzer
+	# that defines its own JsonKB still wins: a user function shadows a
+	# builtin, and this is an ordinary NLP++ function like any other.
+	jsonwrite(L("file"),L("con"));
+}
+# JsonKBRecurse, JsonAttributes and JsonStr below are no longer reached from
+# JsonKB, which now calls the jsonwrite builtin.  They are kept because an
+# analyzer may call them directly -- JsonStr in particular is useful on its
+# own for escaping a string as a JSON literal.  They also document exactly
+# what jsonwrite reproduces, byte for byte.
 JsonKBRecurse(L("file"),L("parent"),L("level")) {
 	L("con") = down(L("parent"));
 	L("first") = 1;