From 190dfcd10c4832803c8d517f1dd49daaaefce155 Mon Sep 17 00:00:00 2001 From: David de Hilster Date: Mon, 12 Oct 2026 19:53:46 -0400 Subject: [PATCH] Help: non-ASCII punctuation is literal in rules (engine 4.2.7) Adds a row to the rule-file token tables in Tokens.md and Pass_Files.md: curly quotes, guillemets, inverted marks and dashes are literal as written, with or without a backslash (VisualText/nlp-engine#467). Before 4.2.7 any of them in a rule file was a syntax error. Co-Authored-By: Claude Opus 5.5 --- Help/markdown/NLP_PP_Stuff/Pass_Files.md | 1 + Help/markdown/NLP_PP_Stuff/Tokens.md | 1 + 2 files changed, 2 insertions(+) diff --git a/Help/markdown/NLP_PP_Stuff/Pass_Files.md b/Help/markdown/NLP_PP_Stuff/Pass_Files.md index 1575e2c..942ca38 100644 --- a/Help/markdown/NLP_PP_Stuff/Pass_Files.md +++ b/Help/markdown/NLP_PP_Stuff/Pass_Files.md @@ -150,6 +150,7 @@ Optionally, each element can be followed by a list of key=values *pairs* enclose | **"XXX"** | Strings get double quoted. Everything within string taken literally (except backslashed characters). | | **\X** | Backslash any nonalphanumeric char you want taken literally. | | **\\** | Literal backslash character. | +| **“ ” « » ¿ ¡ —** | Non-ASCII punctuation is literal as written, with or without a backslash: `_quote <- “ _xWILD [fail=(”)] ” @@`. Only ASCII punctuation has a role in rule syntax and needs the backslash. **NEW 4.2.7** -- before this, any non-ASCII punctuation in a rule file was a syntax error. | | **\** | At end of line, escapes end of line to NOTHING. [Unimplemented] | | **\X** | where X=abnrftv special escapes in C/C++. | | **@AAA** | Reserved start-of-region markers in the pass file (e.g., @RULES). | diff --git a/Help/markdown/NLP_PP_Stuff/Tokens.md b/Help/markdown/NLP_PP_Stuff/Tokens.md index 2300f9a..22734ed 100644 --- a/Help/markdown/NLP_PP_Stuff/Tokens.md +++ b/Help/markdown/NLP_PP_Stuff/Tokens.md @@ -13,6 +13,7 @@ Following are some token types that occur in NLP++ code. | **"XXX"** | Strings get double quoted. Everything within string taken literally (except backslashed characters). | | **\X** | Backslash any nonalphanumeric char you want taken literally. If X = one of a b n r f t v, these are special escapes analogous to C/C++. | | **\\** | Literal backslash character. | +| **“ ” « » ¿ ¡ —** | Non-ASCII punctuation is literal as written, with or without a backslash: `_quote <- “ _xWILD [fail=(”)] ” @@`. Only ASCII punctuation has a role in rule syntax and needs the backslash. **NEW 4.2.7** -- before this, any non-ASCII punctuation in a rule file was a syntax error. | ## See Also