Help: rmchars, $os, negative strpiece, split into characters, Unicode num, names, dict case (engine 4.2.15)

Help/markdown/$os.md

@@ -0,0 +1,44 @@
+[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md)
+
+# $os
+
+## Purpose
+
+Find out which operating system the engine is running on.
+
+## Syntax
+
+```
+returnedString = variableType("$os")
+```
+
+```
+returnedString - type: str
+
+variableType - type: G
+```
+
+## Returns
+
+`windows`, `linux` or `macos`.
+
+## Remarks
+
+Use it where an analyzer has to do something differently per platform, such as building a file path or choosing the command for [system](system.md).
+
+**NEW 4.2.15**
+
+## Example
+
+```
+@CODE
+if (G("$os") == "windows")
+    system("dir > listing.txt");
+else
+    system("ls > listing.txt");
+@@CODE
+```
+
+## See Also
+
+[$dev]($dev.md), [system](system.md), [Special Variables](NLP_PP_Stuff/Special_Variables.md#table)

Help/markdown/NLP_PP_Stuff/Pass_Files.md

@@ -146,7 +146,7 @@ Optionally, each element can be followed by a list of key=values *pairs* enclose
 | **#** | Comment. Text till the end of the line is ignored. |
 | **/\* \*/** | Block comment. Everything between `/*` and the next `*/` is ignored, including line breaks. Block comments do not nest: the first `*/` closes the comment. Delimiters inside a string or inside a `#` comment are ordinary text. |
 | **AAA or NNN** | Literal tokens. Completely alphabetic or completely numeric. |
-| **_AAA** | Nonliteral tokens must begin with underscore. All the remaining letters MUST be alphabetic. (May expand to underscore,num,dash someday.) |
+| **_AAA** | Nonliteral tokens must begin with underscore, followed by letters, digits and underscores: `_noun`, `_B2`, `_phone_num`. Underscores after the first are **NEW 4.2.15**. Dashes are not allowed. |
 | **"XXX"** | Strings get double quoted. Everything within string taken literally (except backslashed characters). |
 | **\X** | Backslash any nonalphanumeric char you want taken literally. |
 | **\\** | Literal backslash character. |

Help/markdown/NLP_PP_Stuff/Special_Variables.md

@@ -50,6 +50,7 @@ A consolidated table of NLP++ special variables is provided below. Each item in
 | **$passnum** | G | Get current pass number. **NEW 2.0.2.4** |
 | **$rulenum** | G | Get current rule number within current pass. **NEW 2.0.2.4** |
 | [**$dev**](../$dev.md) | G | Check if the engine was run with **-DEV**. Returns 1 under -DEV, else 0. Use it to gate an analyzer's own diagnostic passes on what the caller asked for. **NEW 3.8.4** |
+| [**$os**](../$os.md) | G | Operating system the engine runs on: `windows`, `linux` or `macos`. **NEW 4.2.15** |
 | [**$silent**](../$silent.md) | G | Check if the engine was run with **-SILENT**. Returns 1 under -SILENT, else 0. Note -SILENT also suppresses the analyzer's own output files. **NEW 3.8.4** |
 | [**$isdirrun**](../$isdirrun.md) | G | Check if the engine is processing a directory of files instead of an individual file. Returns 1 if it is analyzing a directory, else 0. **NEW 1.26.0** |
 | [**$isfirstfile**](../$isfirstfile.md) | G | Check if is first file processed in a directory. Returns 1 if it is the first file, else 0. **NEW 1.24.0** |

Help/markdown/NLP_PP_Stuff/Tokens.md

@@ -9,7 +9,7 @@ Following are some token types that occur in NLP++ code.
 | **#** | Comment. Text till the end of the line is ignored. |
 | **/\* \*/** | Block comment. Everything between `/*` and the next `*/` is ignored, including line breaks. Block comments do not nest: the first `*/` closes the comment. Delimiters inside a string or inside a `#` comment are ordinary text. |
 | **AAA or NNN** | Literal tokens. Completely alphabetic or completely numeric. |
-| **_AAA** | Nonliteral tokens must begin with underscore. All the remaining letters MUST be alphabetic. Currently, only alphabetics are allowed after the initial underscore. |
+| **_AAA** | Nonliteral tokens must begin with underscore, followed by letters, digits and underscores: `_noun`, `_B2`, `_phone_num`. Underscores after the first are **NEW 4.2.15**. |
 | **"XXX"** | Strings get double quoted. Everything within string taken literally (except backslashed characters). |
 | **\X** | Backslash any nonalphanumeric char you want taken literally. If X = one of a b n r f t v, these are special escapes analogous to C/C++. |
 | **\\** | Literal backslash character. |

Help/markdown/Table_of_String_Functions.md

@@ -16,6 +16,7 @@ The return types include:  **STR** (string), **STR ARRAY** (string array), **IN
 | [**refind(str, pattern_str [, flags_str])**](refind.md) | **STR ARRAY** | Regular expression match: element 0 is the whole match, 1..n the capture groups. |
 | [**rematch(str, pattern_str [, flags_str])**](rematch.md) | **BOOL** | True if the regular expression matches the string. |
 | [**resubst(str, pattern_str, replacement_str [, flags_str])**](resubst.md) | **STR** | Replace every regular expression match, with `$1` backreferences. |
+| [**rmchars(str, chars_str)**](rmchars.md) | **STR** | Remove every character of chars_str from str. **NEW 4.2.15** |
 | [**split(str, char_str)**](split.md) | **STR ARRAY** | Split a string using the given char as separator. |
 | [**stem(str)**](stem.md) | **STR** | Compute stem of noun and verb. Return lowercase string. |
 | [**strchar(str, num)**](strchar.md) | **STR** | Index to numth char of string |

Help/markdown/VisualText_Basics/About_the_analyzer_sequence.md

@@ -18,6 +18,8 @@ User passes employ either the **pattern** (or **pat**) and the **recursive** (or
 
 If you need a customized tokenization pass to handle OCR output (already exists, commercially available in conjunction with http://www.XIEO.info)), convert HTML to plain text, or another specialized input format, TAI can help.  Contact us at support@textanalysis.com, 1-877-235-6259 USA toll free.
 
+**Dictionary matching and case.** **dicttok** looks each word up as written, then in lowercase, then Capitalized (first letter uppercase). So a **.dict** entry written in lowercase (`jumps`) or Capitalized (`Jumps`) matches the word in any capitalization: `jumps`, `Jumps`, `JUMPS`. An all-caps entry (`US`) matches only that spelling, which keeps it apart from `us`. Multi-word phrases are matched in lowercase, so write phrase entries in lowercase. Capitalized entries matching other capitalizations is **NEW 4.2.15**.
+
 **dicttok** also puts a **CTRL** attribute on control characters to help manage control characters (e.g., using feature-based matching), as in the code snippet below.
 
 **@PRE** <2,2> **vareq**(**"CTRL"**,128);  # Match ASCII 128 = €** @POST** "output.txt" << "Found ctrl = " << **N**("$text",2) << "\n";** @RULES**** _xNIL** <- â** _xCTRL**** _xALPHA**** @@**

Help/markdown/index.md

@@ -96,6 +96,7 @@ to a topic.
         - [$rulenum]($rulenum.md)
       - **Run Mode Variables**
         - [$dev]($dev.md)
+        - [$os]($os.md)
         - [$silent]($silent.md)
       - **Test Variables**
         - [$end]($end.md)
@@ -464,6 +465,7 @@ to a topic.
         - [refind](refind.md)
         - [rematch](rematch.md)
         - [resubst](resubst.md)
+        - [rmchars](rmchars.md)
         - [split](split.md)
         - [sqlstr](sqlstr.md)
         - [stem](stem.md)
@@ -602,7 +604,7 @@ above. They are grouped here so that every page remains reachable from this inde
 
 ### Run Mode Variables
 
-[$dev]($dev.md)  |  [$silent]($silent.md)
+[$dev]($dev.md)  |  [$os]($os.md)  |  [$silent]($silent.md)
 
 ### Control Flow Keywords
 

Help/markdown/num.md

@@ -26,6 +26,8 @@ Returns the integer value of the string if it can be converted; a float argument
 
 This function also accepts a num arg.
 
+Digits from other scripts count too: `num("１２")` (full-width) is 12 and `num("٣")` (Arabic-Indic) is 3. Any Unicode decimal digit is read as its value. **NEW 4.2.15**
+
 ## Example
 
 ```

Help/markdown/rmchars.md

@@ -0,0 +1,46 @@
+[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md)
+
+# rmchars
+
+## Purpose
+
+Remove every occurrence of a set of characters from a string.
+
+## Syntax
+
+```
+returnedString = rmchars(string, charsString)
+```
+
+```
+returnedString - type: str
+
+string - type: str
+
+charsString - type: str
+```
+
+## Returns
+
+*string* with every character that appears in *charsString* removed. If *charsString* is empty, *string* is returned unchanged.
+
+## Remarks
+
+Each character of *charsString* is removed wherever it occurs; order doesn't matter. Characters are whole characters, not bytes: `rmchars("héllo", "é")` is `hllo`.
+
+To remove a multi-character substring instead, use [strsubst](strsubst.md) with an empty replacement.
+
+**NEW 4.2.15**
+
+## Example
+
+```
+@CODE
+"output.txt" << rmchars("(555) 867-5309", "()- ") << "\n";
+# Writes 5558675309
+@@CODE
+```
+
+## See Also
+
+[strsubst](strsubst.md), [split](split.md), [strclean](strclean.md), [String Functions](Table_of_String_Functions.md)

Help/markdown/split.md

@@ -9,7 +9,7 @@ Split a string into pieces using a separator character.
 ## Syntax
 
 ```
-returnedArray = str(str, charString)
+returnedArray = split(str, charString)
 ```
 
 ```
@@ -30,7 +30,9 @@ An array of strings, each of which is a component of the given string.
 
 ## Remarks
 
-Similar to the Perl *split* function.
+Similar to the Perl *split* function. Each character in *charString* is a delimiter.
+
+With an empty *charString*, `split` returns the string's characters: `split("héllo", "")` is `h`, `é`, `l`, `l`, `o`. Characters are whole characters, not bytes, so an accented letter or emoji stays one element. **NEW 4.2.15** (it used to be an error).
 
 ## Example
 

Help/markdown/strpiece.md

@@ -28,6 +28,10 @@ The substring of `string` from the zero-based index startNumber to endNumber, in
 
 ## Remarks
 
+Positions count characters, not bytes, so accented letters and emoji count as one each.
+
+A negative position counts from the end: `-1` is the last character, `-2` the one before it. `strpiece("hello", 1, -1)` is `ello`, and `strpiece("hello", 0, -2)` is `hell`. **NEW 4.2.15**
+
 ## Example
 
 ```