From 3d5d6017e7a6ca54bb6e236ca1306f764c927f34 Mon Sep 17 00:00:00 2001 From: David de Hilster Date: Mon, 3 Aug 2026 14:47:22 -0400 Subject: [PATCH] docs(help): document the 3.8.0 builtins, shadowing, and two fixes Fourteen new function pages, plus the table, index and behavior updates that go with the 3.8.0 engine work. New pages readfile, readlines, dirlist rematch, refind, resubst strjoin, strindexof, strlastindexof arraysort, arrayunique, arrayreverse, arrayslice Rewritten push.md -- was documented as UNIMPLEMENTED. Corrected to the real signature, push(value, array), matching the hand-written push() in visualText/spec/UtilFuncs.nlp. Table_of_Array_Functions.md -- the page held a single row, the unimplemented push. Now lists the whole array family, and explains how to test for an empty result: assigning an empty array collapses it to no value and arraylength reports 1 for that, so a loop must be guarded on the first element rather than on the count. DECL.md -- a user function may now shadow a builtin of the same name. regexp.md, regexpi.md -- cross-referenced to the real regular expression functions, and the whole-token matching rule is now written down: regexp("run") does NOT match "running". That was undocumented and is an easy trap. Table_of_File_Functions, Table_of_String_Functions and index.md pick up the new entries. New_in_Current_Version records the three 3.8.0 items: the new builtins, user functions shadowing builtins, and the str() fix for negative numbers. Engine side: VisualText/nlp-engine PRs #704 through #708. Co-Authored-By: Claude Opus 5 (1M context) --- Help/markdown/DECL.md | 12 ++++ Help/markdown/New_in_Current_Version.md | 6 ++ Help/markdown/Table_of_Array_Functions.md | 23 ++++++- Help/markdown/Table_of_File_Functions.md | 5 +- Help/markdown/Table_of_String_Functions.md | 6 ++ Help/markdown/arrayreverse.md | 44 +++++++++++++ Help/markdown/arrayslice.md | 58 +++++++++++++++++ Help/markdown/arraysort.md | 60 +++++++++++++++++ Help/markdown/arrayunique.md | 49 ++++++++++++++ Help/markdown/dirlist.md | 75 ++++++++++++++++++++++ Help/markdown/index.md | 17 +++++ Help/markdown/push.md | 31 +++++++-- Help/markdown/readfile.md | 54 ++++++++++++++++ Help/markdown/readlines.md | 63 ++++++++++++++++++ Help/markdown/refind.md | 70 ++++++++++++++++++++ Help/markdown/regexp.md | 6 +- Help/markdown/regexpi.md | 8 ++- Help/markdown/rematch.md | 62 ++++++++++++++++++ Help/markdown/resubst.md | 68 ++++++++++++++++++++ Help/markdown/strindexof.md | 50 +++++++++++++++ Help/markdown/strjoin.md | 51 +++++++++++++++ Help/markdown/strlastindexof.md | 49 ++++++++++++++ 22 files changed, 854 insertions(+), 13 deletions(-) create mode 100644 Help/markdown/arrayreverse.md create mode 100644 Help/markdown/arrayslice.md create mode 100644 Help/markdown/arraysort.md create mode 100644 Help/markdown/arrayunique.md create mode 100644 Help/markdown/dirlist.md create mode 100644 Help/markdown/readfile.md create mode 100644 Help/markdown/readlines.md create mode 100644 Help/markdown/refind.md create mode 100644 Help/markdown/rematch.md create mode 100644 Help/markdown/resubst.md create mode 100644 Help/markdown/strindexof.md create mode 100644 Help/markdown/strjoin.md create mode 100644 Help/markdown/strlastindexof.md diff --git a/Help/markdown/DECL.md b/Help/markdown/DECL.md index 9fe1c48..bf56750 100644 --- a/Help/markdown/DECL.md +++ b/Help/markdown/DECL.md @@ -20,6 +20,18 @@ An example follows. This function call returns and outputs the number "6" to the file "output.txt". +## Shadowing a built-in function + +If you define a function whose name is also that of a built-in NLP++ function, **your definition wins**. The analyzer still builds, and a warning is written to the build log: + +``` +[Warning: user function shadows built-in NLP++ function name='push'. The user function will be used.] +``` + +This used to be a hard error, which meant that adding a new built-in function could break any existing analyzer that happened to have defined a function of that name. Shadowing makes the language safe to extend: your analyzer keeps doing exactly what it did before, whatever gets added to the engine later. + +If you want the built-in behavior instead, remove or rename your own definition. Check the warning if a function does not behave the way its help page describes — a shadowing definition elsewhere in the analyzer is the usual reason. + ## See Also [Declare Zone](Declare.md) diff --git a/Help/markdown/New_in_Current_Version.md b/Help/markdown/New_in_Current_Version.md index 597cc3a..f3b5cd4 100644 --- a/Help/markdown/New_in_Current_Version.md +++ b/Help/markdown/New_in_Current_Version.md @@ -20,6 +20,12 @@ The ANSI version shares the same source code with the Unicode version, but repli ## Miscellaneous +(3.8.0) **User functions may now shadow built-in functions.** Defining a function in a [@DECL](DECL.md) region whose name is also a built-in used to be a hard error, so adding any new built-in broke every existing analyzer that had defined a function of that name. Now the user's definition wins, the analyzer builds, and a warning names the shadowed function. This is what lets [arrayreverse](arrayreverse.md) and [push](push.md) become built-ins without disturbing the hand-written versions that have long been in `visualText/spec/UtilFuncs.nlp`, and it means future built-ins can be added safely. + +(3.8.0) **String and array utilities.** [split](split.md) had no inverse, so building a delimited string meant a loop and a "first time through" flag: [strjoin](strjoin.md) now does it in one call. There was also no way to get the position of a substring — [strchr](strchr.md) takes a single character and returns a tail rather than an index, and [strcontains](strcontains.md) answers only yes or no — so [strindexof](strindexof.md) and [strlastindexof](strlastindexof.md) have been added. Every existing sort was knowledge-base or parse-tree oriented, leaving plain arrays with none, so there are now [arraysort](arraysort.md) (lexical or numeric, ascending or descending, stable), [arrayunique](arrayunique.md), [arrayreverse](arrayreverse.md) and [arrayslice](arrayslice.md). [push](push.md), previously listed as UNIMPLEMENTED, now works. All of these leave the source array alone and return a new one. **FIXED:** [str](str.md) converted signed values with an unsigned format, so every negative number printed as roughly 1.8e19 — `str(-1)` gave "18446744073709551615", which also made the documented -1 returns of [filesize](filesize.md) and the new index functions unreadable. The same fault was in the compiled-analyzer path and in the code generator, so compiled analyzers emitted the wrong numeric literals too. + +(3.8.0) **Reading files, listing directories, and real regular expressions.** NLP++ could write files but had no way to read one, so a pass could not load a config file, a lookup table, or its own earlier output. New: [readfile](readfile.md) reads a whole text file into a string, [readlines](readlines.md) reads one as an array of lines, and [dirlist](dirlist.md) lists a directory as full forward-slash paths, sorted, with an optional `*`/`?` glob. Also new are three true regular-expression functions using ECMAScript syntax — [rematch](rematch.md), [refind](refind.md) (with capture groups) and [resubst](resubst.md) (replaces every match, with `$1` backreferences) — which unlike the older [regexp](regexp.md) support character classes, anchors, quantifiers, alternation and groups, and can be called from anywhere rather than only from a `@PRE` region. **FIXED:** [regexp](regexp.md) and [regexpi](regexpi.md) had been returning false for every input since 2.x — the "no literal yet" marker in the pattern compiler was changed from a null pointer to an empty string, so the pattern's literal never bound and nothing ever matched. [regexpi](regexpi.md) additionally lost its case-insensitivity after the first pattern element. Both are fixed, and the help now records that these patterns must match the whole string. + (2.7.2.0) Rework of crawling functions eg [urltofile](urltofile.md), to handle HTTPS as well as HTTP. (2.7.0.9) New function [pnmove](pnmove.md) to enable moving nodes within the parse tree. (2.7.0.0) Overhauls, optimizations, and bug fixes focusing on the recursive NLP++ pass algorithm. (2.6.0.3) New function [dejunk](dejunk.md) to clean out non-ASCII and extended ASCII characters (sometimes replacing with a question mark character). (2.6.0.0) Important fix to NLP++ pattern matcher re feature-based match with wildcards.  **dicttokz** pass, a variant of dicttok that removes whitespace from parse tree at the outset, puts in attributes NOSP (no space before), NL (new line before).  Adds capitalization attributes to parse tree as well.  (This will be an ongoing development in 2.6.x.x). (2.5.0.0) New [dicttok black-box tokenization pass](VisualText_Basics/About_the_analyzer_sequence.md) that uses KB from TAIParse to put part-of-speech tags on parse tree automatically. See [DictTok_Sample](DictTok_Sample.md) analyzer. (2.4.x.x) Maintenance releases, various fixes in feature-based matching, compiled analyzers, etc. (2.3.2.0) In text windows ctrl-A now is Select All. ctrl-G is Find Again. (2.3.1.9) Maintenance release. Fixes to error reporting and recovery, compiled vs interpreted analyzers.. [group](group.md) now returns the new "reduce" node that it creates, an important convenience.  [group](group.md) also reports when it is including the [lookahead](lookahead.md) node -- an error.  [listadd](listadd.md) no longer treats the list node as the "reduce" node. (2.3.1.8) [xmlstr](xmlstr.md) now converts characters to { where 123 is a character number, rather than to strings such as ¼. (2.3.1.6) Added special rule elements **[_xLET](_xLET.md)** and **[_xCAPLET](_xCAPLET.md)** to conveniently match single-character words. (2.3.1.0) VISUAL BASIC .NET API for calling VisualText analyzers.  See the sample program called [VB_to_VT](VB_to_VT.md), which also lists the API functions. (2.3.0.0) Compiling analyzers now generates separate code files for each pass, enabling large analyzers to be compiled into C++.  For example, the latest versions of TAIParse should now compile.  Note that a Microsoft Visual Studio project may now need to have files added to it, within an analyzer's RUN folder. (2.2.2.13) New PRE actions called [regexp](regexp.md) for matching using regular expressions, with [regexpi](regexpi.md) case insensitive version. (2.2.2.11) New function for sorting a knowledge base concept's hierarchy and children: [sorthier](sorthier.md) and [sortchilds](sortchilds.md). (2.2.2.8) Compiling analyzers once again generates PRETTY, or human-readable, code.  We tried very long lines to circumvent the 65K file line limit of some compilers, but too-long lines are an issue as well.  In subsequent versions, we'll be splitting files as needed.  For now, huge analyzers may need their generated C++ code (in the RUN folder) to be tweaked manually in order to compile. (2.2.2.5) EXITPASS, SUCCEED, and FAIL.  These functions have been restored in that they are allowed in practically any context without triggering error messages.  The compiled analyzer version uses C++ exception handling mechanisms in order to instantly handle these in deeply embedded contexts (function within a function, etc.)  Parse tree display ("yellow box") now places node variables in two columns for convenient viewing of long variable lists. Parse tree nodes now point back to rules that renamed them via [xrename](xrename.md) and [pnrename](pnrename.md).  Compile-during-analyzer-load popup of MS DOS window has been removed.  Interpreted user functions improperly returned a non-zero value in the absence of a return statement. (2.2.2.4) Incorrect VisualText TAB SIZE in Vista has been fixed. SAVE KB and COMPILE KB default popup answer is now NO.  (Unless you are explicitly trying to build a KB, then you don't want to save the KB.)  Parse tree for lone underscore character is now a terminal punctuation char.  Listadd in recursive passes when building a list right-to-left now fixed (compiled version hang fixed). [excise](excise.md) forbids deleting a lookahead node. (2.2.2.2) The functions [exitpass](exitpass.md), [succeed](succeed.md), [fail](fail.md), can appear only at the top level of code regions in compiled analyzers (**FIXED**. See 2.2.2.5 above).  Fixes to NLP++ syntax relating to negation.  A fix to float conversion.  Further fixes to optimized recursive pass algorithm. (2.2.2.1) Further fixes to align compiled and interpreted analyzers (fixes to zeroed items in relational operators and others).  Fix of an annoying VisualText crash when brought into focus from the background.  A substantial fix to the optimized recursive NLP++ pass algorithm in both the interpreted and compiled analyzer frameworks. (2.2.1.1) New function [pninsert](pninsert.md) for programmatically inserting nonliteral nodes into the parse tree. (2.2.1.0) NEW. A parse tree view mode that reflects items removed from the parse tree, such as HTML tags.  See Tree Text Mode in the View Menu. Fix to some NLP++ compiled functions to enable zero (0) as a null string.  E.g., strsubst("ab,c",",",0); will replace commas by empty string in a compiled analyzer. Note that this behavior already works as expected in interpreted analyzers, e.g., those that run inside VisualText. (2.2.0.0) A substantially cleaned up VisualText release (see 2.1.0.9 notes below).  Tree Text view does not leave everything expanded on reentering VisualText, but remembers what folders have been closed up in the view. The RUN NEXT button behaves better. (2.1.0.9) A CLEANUP RELEASE.  ALIGNED INTERPRETED AND COMPILED ANALYZERS, FIXED MISC BUGS AND LEAKS.  Fixed a substantial bug that hobbled compiled analyzers.  Memory leaks plugged in eltnode and lasteltnode.  The PROG project or sample driver for analyzers now sets the BATCHSTART flag at the start of a run, to enable a compiled analyzer to test the NLP++ batchstart() function/flag. (So that an analyzer can perform initializations when it runs the first file of a batch of files.) DEFAULT configuration of VisualText is now full screen, kb safe mode off, toolbars in a standard configuration.  TIP OF THE DAY offers real suggestions. Fixed bugs in the PROG driver of each analyzer, for the analysis of multiple files and folders. (2.1.0.8) Fix to functions like resolveurl, strchr, strrchr. Fix to parse tree nodes with large string-valued variables.  Fixes to Tab Window and Find Window show/hide bugs.  Change to VisualText naming in the Windows registry.  Fixed a memory leak in rmattrs. (2.1.0.6) Fixes to file and directory handling.  (See the PROG project of a newly created analyzer project.) (2.1.0.3) Misc fixes, including parse tree display and return of arrays in NLP++ user-defined functions. (2.1.0.0) In order to support huge knowledge bases, a VisualTextQ version is provided that utilizes the QDBM database management freeware under the hood.  For backward compatibility, one can treat the knowledge base built in this way as the "compiled" knowledge base.  Loading the interpreted knowledge base will read kb command files into a QDBM file system.  "Compiling" the kb will save the latest version of the QDBM files -- not compiling the changes will discard the latest modifications to the kb. Other notes: Removed the Windows MS-DOS Command Prompt popups from most system calls. (2.0.2.9) The [levenshtein](levenshtein.md) edit-distance computation is provided. (2.0.2.8) Optimizations to string manipulation using L (local) variables.  These are not interned, as with S,X,G,N variables.  Locally assigned strings are completely freed when a scope (function, @CODE, @CHECK) is exited, while globally assigned strings are interned and persist throughout a text analysis.  Analyzers can be optimized by using local L variables wherever feasible. (2.0.2.7) [fltval](fltval.md) knowledge base value fetch. (2.0.2.5) Linux upgrade. (2.0.2.4) New variables **[G**("$passnum")](%24passnum.md) and **[G**("$rulenum")](%24rulenum.md) e.g. to track rule context and performance. (2.0.2.3) New NLP++ function [eltnode](eltnode.md), analogous to lasteltnode. (2.0.2.2) Recovery from very long tokens. (2.0.2.1) Refix of an infrequent crash in using $treetext. (2.0.2.0) Optimization to recursive passes. (2.0.1.1) Fixes to the **Find in Files** popup menu.  Bugs were introduced in VisualText 2.0: (a) missing patterns that start at the beginning of a line, (b) incorrect display of capitalization in found patterns. # NEW IN CURRENT VERSION (2.0 NON-UNICODE BETA) diff --git a/Help/markdown/Table_of_Array_Functions.md b/Help/markdown/Table_of_Array_Functions.md index 1298683..9b7f818 100644 --- a/Help/markdown/Table_of_Array_Functions.md +++ b/Help/markdown/Table_of_Array_Functions.md @@ -10,7 +10,26 @@ Arrays can also be assigned and manipulated with parse tree functions and kb fun | **FUNCTION NAME** | **RETURNS** | **DESCRIPTION** | | --- | --- | --- | -| **push(var)** | ARRAY | Push a value to the front of a variable's value(s). Allows mixed types (string, num, flt, concept...), allows redundant values, does not check uniqueness. NOTE: This will alter the array indexing, pushed item will be the 0th element of array index. **UNIMPLEMENTED** | -| | | | +| [**arraylength(array)**](arraylength.md) | **INT** | Number of elements in an array. | +| [**arrayreverse(array)**](arrayreverse.md) | **ARRAY** | The same elements, last to first. | +| [**arrayslice(array, start_num [, end_num])**](arrayslice.md) | **ARRAY** | Elements from start up to but not including end. Zero based; omit end to run to the end. | +| [**arraysort(array [, numeric_bool [, descending_bool]])**](arraysort.md) | **ARRAY** | Sort an array. Lexical and ascending by default; stable. | +| [**arrayunique(array)**](arrayunique.md) | **ARRAY** | Keep the first of each distinct element, in the original order. | +| [**push(value, array)**](push.md) | **ARRAY** | Push a value to the front of an array. Allows mixed types (string, num, flt, concept...), allows redundant values, does not check uniqueness. NOTE: This alters the array indexing; the pushed item becomes the 0th element. | +| [**split(str, char_str)**](split.md) | **ARRAY** | Split a string into an array on a separator character. | +| [**strjoin(array, separator_str)**](strjoin.md) | **STR** | Join an array's elements into one string. The inverse of split. | + +These functions never modify the array given to them; each returns a new array. + +### Testing for an empty result + +The functions that can find nothing — `arrayslice`, `arrayunique`, [refind](refind.md), [dirlist](dirlist.md), [readlines](readlines.md) — return an **empty array**. Note that assigning an empty array to a variable collapses it back to no value, and `arraylength` reports **1** for a variable holding no value. So guard a loop by testing the first element, not the count: + +``` +L("hits") = arrayslice(L("rows"),1); +if (strlength(L("hits")[0])) { + # ... safe to loop over L("hits") ... +} +``` ## See Also diff --git a/Help/markdown/Table_of_File_Functions.md b/Help/markdown/Table_of_File_Functions.md index 8ae60b7..cff6887 100644 --- a/Help/markdown/Table_of_File_Functions.md +++ b/Help/markdown/Table_of_File_Functions.md @@ -10,11 +10,14 @@ The return types include: **STR** (string), **INT** (integer), and **BOOL** (Boo | --- | --- | --- | | [**deletefile(path_str)**](deletefile.md) | **BOOL(1,0)** | Delete a regular file. Never removes directories. | | [**direxists(path_str)**](direxists.md) | **BOOL(1,0)** | True if the path is an existing directory. | +| [**dirlist(dir_str [, pattern_str])**](dirlist.md) | **STR ARRAY** | Full paths of the entries of a directory, sorted, optionally filtered by a `*`/`?` glob. | | [**fileexists(path_str)**](fileexists.md) | **BOOL(1,0)** | True if the path is an existing regular file. | | [**filesize(path_str)**](filesize.md) | **INT** | Size of a regular file in bytes, or -1 if not a readable file. | | [**mkdir(dir_str)**](mkdir.md) | **BOOL(1,0)** | Create a directory. Operating system-dependent. | +| [**readfile(path_str)**](readfile.md) | **STR** | Read a whole text file into a string. | +| [**readlines(path_str)**](readlines.md) | **STR ARRAY** | Read a text file as an array of its lines, terminators stripped. | -For reading and writing file contents, see the file-stream functions [openfile](openfile.md), [closefile](closefile.md), and [fileout](fileout.md) in the [Formatting and I/O Functions](Table_of_Formatting_and_I_O_Functions.md) table, and [urltofile](urltofile.md) in the [Web Functions](Table_of_Web_Functions.md) table. +For writing file contents, see the file-stream functions [openfile](openfile.md), [closefile](closefile.md), and [fileout](fileout.md) in the [Formatting and I/O Functions](Table_of_Formatting_and_I_O_Functions.md) table, and [urltofile](urltofile.md) in the [Web Functions](Table_of_Web_Functions.md) table. ## See Also diff --git a/Help/markdown/Table_of_String_Functions.md b/Help/markdown/Table_of_String_Functions.md index f3ac456..8499a78 100644 --- a/Help/markdown/Table_of_String_Functions.md +++ b/Help/markdown/Table_of_String_Functions.md @@ -13,6 +13,9 @@ The return types include:  **STR** (string), **STR ARRAY** (string array), **IN | [**flt(str)**](flt.md) | **FLOAT** | Convert string or int to float, if possible. | | [**levenshtein(str1,str2)**](levenshtein.md) | **INT** | Levenshtein edit-distance between two strings.** New in 2.0.2.9** | | [**num(str)**](num.md) | **INT** | Convert string to number, if possible. (Also accepts num arg). | +| [**refind(str, pattern_str [, flags_str])**](refind.md) | **STR ARRAY** | Regular expression match: element 0 is the whole match, 1..n the capture groups. | +| [**rematch(str, pattern_str [, flags_str])**](rematch.md) | **BOOL** | True if the regular expression matches the string. | +| [**resubst(str, pattern_str, replacement_str [, flags_str])**](resubst.md) | **STR** | Replace every regular expression match, with `$1` backreferences. | | [**split(str, char_str)**](split.md) | **STR ARRAY** | Split a string using the given char as separator. | | [**stem(str)**](stem.md) | **STR** | Compute stem of noun and verb. Return lowercase string. | | [**strchar(str, num)**](strchar.md) | **STR** | Index to numth char of string | @@ -23,6 +26,9 @@ The return types include:  **STR** (string), **STR ARRAY** (string array), **IN | [**strcontainsnocase(str1,str2)**](strcontainsnocase.md) | **BOOL** | Check if str1 is contained in str2, ignoring letter case. | | [**strendswith(str, suffix_str)**](strendswith.md) | **BOOL** | Does given str ends with given suffix_str | | [**strhaspunct(str)**](strhaspunct.md) | **BOOL** | Check if str contains a punctuation character. | +| [**strindexof(str, substr)**](strindexof.md) | **INT** | Zero-based position of the first occurrence of substr, or -1. | +| [**strlastindexof(str, substr)**](strlastindexof.md) | **INT** | Zero-based position of the last occurrence of substr, or -1. | +| [**strjoin(array, separator_str)**](strjoin.md) | **STR** | Join an array's elements into one string. The inverse of split. | | [**strstartswith(str, suffix_str)**](strstartswith.md) | **BOOL** | Does given str starts with given suffix_str | | [**strequal(str1,str2)**](strequal.md) | **BOOL** | Check if str1 is identical to str2. | | [**strisalpha(str)**](strisalpha.md) | **BOOL** | Check if str is all alphabetic chars. | diff --git a/Help/markdown/arrayreverse.md b/Help/markdown/arrayreverse.md new file mode 100644 index 0000000..d900d2c --- /dev/null +++ b/Help/markdown/arrayreverse.md @@ -0,0 +1,44 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# arrayreverse + +## Purpose + +Reverse the order of an array's elements. + +## Syntax + +``` +returnedArray = arrayreverse(array) +``` + +``` +returnedArray - type: array + +array - type: array +``` + +## Returns + +A new array holding the same elements last to first. Returns an empty array if the argument is not an array. + +## Remarks + +The source array is not modified; a new array comes back. + +Combine with [arraysort](arraysort.md) only when you need something other than a plain descending sort — for a descending sort, pass 1 as arraysort's third argument instead of reversing afterwards. + +## Example + +``` +@CODE + +L("abc") = split("a b c"," "); +"out.txt" << strjoin(arrayreverse(L("abc")),",") << "\n"; # c,b,a + +@@CODE +``` + +## See Also + +[arraysort](arraysort.md), [arrayunique](arrayunique.md), [arrayslice](arrayslice.md), [Array Functions](Table_of_Array_Functions.md) diff --git a/Help/markdown/arrayslice.md b/Help/markdown/arrayslice.md new file mode 100644 index 0000000..cac6016 --- /dev/null +++ b/Help/markdown/arrayslice.md @@ -0,0 +1,58 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# arrayslice + +## Purpose + +Take a run of elements out of an array. + +## Syntax + +``` +returnedArray = arrayslice(array, startNum) +returnedArray = arrayslice(array, startNum, endNum) +``` + +``` +returnedArray - type: array + +array - type: array + +startNum - type: int + +endNum - type: int +``` + +## Returns + +A new array holding the elements from startNum up to but not including endNum. Returns an empty array if the argument is not an array or if the range selects nothing. + +## Remarks + +Indexes are zero based. **start is included and end is not**, so the number of elements taken is end - start. Omit end to run to the end of the array. + +Out-of-range values are clamped rather than reported as errors, so slicing past the end simply yields fewer elements and an empty range yields an empty array. + +The source array is not modified; a new array comes back. + +## Example + +``` +@CODE + +L("abcd") = split("a b c d"," "); + +"out.txt" << strjoin(arrayslice(L("abcd"),1,3),",") << "\n"; # b,c +"out.txt" << strjoin(arrayslice(L("abcd"),2),",") << "\n"; # c,d +"out.txt" << strjoin(arrayslice(L("abcd"),3,99),",") << "\n"; # d +"out.txt" << strjoin(arrayslice(L("abcd"),2,2),",") << "\n"; # (empty) + +# Drop a header line off a data file. +L("rows") = arrayslice(readlines(L("path")),1); + +@@CODE +``` + +## See Also + +[arraysort](arraysort.md), [arrayunique](arrayunique.md), [arrayreverse](arrayreverse.md), [arraylength](arraylength.md), [Array Functions](Table_of_Array_Functions.md) diff --git a/Help/markdown/arraysort.md b/Help/markdown/arraysort.md new file mode 100644 index 0000000..1ce2f2e --- /dev/null +++ b/Help/markdown/arraysort.md @@ -0,0 +1,60 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# arraysort + +## Purpose + +Sort the elements of an array. + +## Syntax + +``` +returnedArray = arraysort(array) +returnedArray = arraysort(array, numericBool) +returnedArray = arraysort(array, numericBool, descendingBool) +``` + +``` +returnedArray - type: array + +array - type: array + +numericBool - type: bool (1,0) + +descendingBool - type: bool (1,0) +``` + +## Returns + +A new array holding the same elements in sorted order. Returns an empty array if the argument is not an array. + +## Remarks + +The other sorts — [sortvals](sortvals.md), [sortchilds](sortchilds.md), [sorthier](sorthier.md), [sortphrase](sortphrase.md), [sortconsbyattr](sortconsbyattr.md) — are all knowledge-base or parse-tree oriented. `arraysort` sorts a plain array, such as what [split](split.md), [readlines](readlines.md) or [dirlist](dirlist.md) returned. The argument order follows [sortconsbyattr](sortconsbyattr.md). + +By default the sort is lexical and ascending. Lexical order is **byte order**, so all uppercase letters sort ahead of all lowercase ones: `Banana` comes before `apple`. Lowercase the array first with a loop and [strtolower](strtolower.md) if you want dictionary order. + +Pass 1 for numericBool to compare the elements as numbers instead, which is what you want for an array of digits — lexically, `100` sorts before `2`. + +The sort is stable, so elements that compare equal keep their original relative order. + +The source array is not modified; a new array comes back. + +## Example + +``` +@CODE + +L("words") = split("cherry apple banana"," "); +"out.txt" << strjoin(arraysort(L("words")),",") << "\n"; # apple,banana,cherry + +L("nums") = split("10 2 100"," "); +"out.txt" << strjoin(arraysort(L("nums"),1),",") << "\n"; # 2,10,100 +"out.txt" << strjoin(arraysort(L("nums"),1,1),",") << "\n"; # 100,10,2 + +@@CODE +``` + +## See Also + +[arrayunique](arrayunique.md), [arrayreverse](arrayreverse.md), [arrayslice](arrayslice.md), [sortconsbyattr](sortconsbyattr.md), [Array Functions](Table_of_Array_Functions.md) diff --git a/Help/markdown/arrayunique.md b/Help/markdown/arrayunique.md new file mode 100644 index 0000000..09b1d66 --- /dev/null +++ b/Help/markdown/arrayunique.md @@ -0,0 +1,49 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# arrayunique + +## Purpose + +Remove duplicate elements from an array. + +## Syntax + +``` +returnedArray = arrayunique(array) +``` + +``` +returnedArray - type: array + +array - type: array +``` + +## Returns + +A new array holding the first occurrence of each distinct element. Returns an empty array if the argument is not an array. + +## Remarks + +The array does **not** have to be sorted first. `arrayunique` keeps the first of each set of equal elements and preserves the original order, so `a b a c b` gives `a b c`. + +Elements are compared by their text, so a numeric 2 and a string `"2"` count as the same value. + +The source array is not modified; a new array comes back. + +## Example + +``` +@CODE + +L("seen") = split("a b a c b"," "); +"out.txt" << strjoin(arrayunique(L("seen")),",") << "\n"; # a,b,c + +# Compose: distinct values, sorted. +"out.txt" << strjoin(arraysort(arrayunique(L("seen"))),",") << "\n"; + +@@CODE +``` + +## See Also + +[arraysort](arraysort.md), [arrayreverse](arrayreverse.md), [arrayslice](arrayslice.md), [Array Functions](Table_of_Array_Functions.md) diff --git a/Help/markdown/dirlist.md b/Help/markdown/dirlist.md new file mode 100644 index 0000000..932fcd7 --- /dev/null +++ b/Help/markdown/dirlist.md @@ -0,0 +1,75 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# dirlist + +## Purpose + +List the entries of a directory. + +## Syntax + +``` +returnedArray = dirlist(dirStr) +returnedArray = dirlist(dirStr, patternStr) +``` + +``` +returnedArray - type: str array + +dirStr - type: str + +patternStr - type: str +``` + +## Returns + +An array of the full paths of the entries in the directory, sorted. Returns an empty array if the directory is empty, if nothing matches the pattern, or if the path is not a directory; a warning is written to the log in that last case. + +## Remarks + +The results are **full paths**, not bare names, so they feed straight into [readfile](readfile.md), [readlines](readlines.md), [fileexists](fileexists.md) and [filesize](filesize.md) with no path-joining by the analyzer. The paths always use forward slashes, on every platform, so analyzer code that builds or compares them stays portable. + +Results are sorted, so a run over a directory is reproducible; the order in which a file system hands back directory entries is not otherwise defined. + +The listing is **not** recursive, and it includes subdirectories alongside files. Call [fileexists](fileexists.md) or [direxists](direxists.md) on an entry to tell the two apart. + +The optional pattern is a glob matched case-insensitively against the entry **name** only, never against the rest of the path. `*` matches any run of characters, including none, and `?` matches exactly one. A pattern must match the whole name, so `"*.dict"` matches `terms.dict` but `"dict"` matches nothing. + +When nothing matches, the result is an empty array. Guard a loop over it by testing the **first element** rather than the count: assigning an empty array to a variable collapses it back to no value, and [arraylength](arraylength.md) reports 1 for a variable holding no value, so a count test would run the loop once over a phantom entry. + +``` +if (strlength(L("files")[0])) { + # ... safe to loop ... +} +``` + +To recover the bare name of an entry, split on `/` and take the last piece, or match it with [refind](refind.md): + +``` +L("name") = refind(L("path"),"[^/]+$")[0]; +``` + +## Example + +``` +@CODE + +# Load every dictionary sitting in the analyzer's data folder. +L("files") = dirlist(G("$apppath") + "/data", "*.dict"); + +# Guard on the first element, not the count -- see Remarks. +if (strlength(L("files")[0])) { + L("len") = arraylength(L("files")); + L("i") = 0; + while (L("i") < L("len")) { + "out.txt" << "loading " << L("files")[L("i")] << "\n"; + L("i")++; + } +} + +@@CODE +``` + +## See Also + +[readfile](readfile.md), [readlines](readlines.md), [direxists](direxists.md), [fileexists](fileexists.md), [File Functions](Table_of_File_Functions.md) diff --git a/Help/markdown/index.md b/Help/markdown/index.md index e86cbec..9877908 100644 --- a/Help/markdown/index.md +++ b/Help/markdown/index.md @@ -244,6 +244,14 @@ to a topic. - [Functions](NLP_PP_Stuff/Functions.md) - **Array Functions** - [Table of Array Functions](Table_of_Array_Functions.md) + - [arraylength](arraylength.md) + - [arrayreverse](arrayreverse.md) + - [arrayslice](arrayslice.md) + - [arraysort](arraysort.md) + - [arrayunique](arrayunique.md) + - [push](push.md) + - [split](split.md) + - [strjoin](strjoin.md) - **Database Functions** - [Table of Database Functions](Table_of_Database_Functions.md) - [dballocstmt](dballocstmt.md) @@ -258,8 +266,11 @@ to a topic. - [Table of File Functions](Table_of_File_Functions.md) - [deletefile](deletefile.md) - [direxists](direxists.md) + - [dirlist](dirlist.md) - [fileexists](fileexists.md) - [filesize](filesize.md) + - [readfile](readfile.md) + - [readlines](readlines.md) - [Formatting and I/O Functions](Table_of_Formatting_and_I_O_Functions.md) - [Table of Formatting and I/O Functions](Table_of_Formatting_and_I_O_Functions.md) - [cbuf](cbuf.md) @@ -442,6 +453,9 @@ to a topic. - [flt](flt.md) - [levenshtein](levenshtein.md) - [num](num.md) + - [refind](refind.md) + - [rematch](rematch.md) + - [resubst](resubst.md) - [split](split.md) - [sqlstr](sqlstr.md) - [stem](stem.md) @@ -459,6 +473,9 @@ to a topic. - [strescape](strescape.md) - [strgreaterthan](strgreaterthan.md) - [strhaspunct](strhaspunct.md) + - [strindexof](strindexof.md) + - [strjoin](strjoin.md) + - [strlastindexof](strlastindexof.md) - [strisalpha](strisalpha.md) - [strisdigit](strisdigit.md) - [strislower](strislower.md) diff --git a/Help/markdown/push.md b/Help/markdown/push.md index 28e0bc0..9bdf6e3 100644 --- a/Help/markdown/push.md +++ b/Help/markdown/push.md @@ -4,30 +4,47 @@ ## Purpose -Push a value to the front of a variable's value(s). +Add a value to the front of an array. ## Syntax ``` -ARRAY = push(var); +returnedArray = push(value, array) ``` ``` -var - type: array +returnedArray - type: array + +array - type: array + +value - type: str, num, flt or concept ``` ## Returns -ARRAY +A new array with value as element 0, followed by every element of the original array. Returns an empty array if the first argument is not an array. ## Remarks -Allows mixed types (string, num, flt, concept...), allows redundant values, and does not check uniqueness. NOTE: This will alter the array indexing — the pushed item will be the 0th element of the array index. +Up to now arrays were grown mainly by indexing past the last element. `push` adds at the **front** instead. + +The pushed value keeps its own type, so strings, numbers, floats and concepts can all be pushed, and mixed-type arrays are allowed. Uniqueness is not checked; pushing a value already in the array simply adds it again. -**UNIMPLEMENTED** +Note that this changes the array's indexing: the pushed value becomes element 0 and every existing element shifts up by one. + +The source array is not modified; a new array comes back. ## Example +``` +@CODE + +L("abc") = split("a b c"," "); +"out.txt" << strjoin(push("z",L("abc")),",") << "\n"; # z,a,b,c + +@@CODE +``` + ## See Also -[Array Functions](Table_of_Array_Functions.md) +[arrayslice](arrayslice.md), [arrayreverse](arrayreverse.md), [arraylength](arraylength.md), [Array Functions](Table_of_Array_Functions.md) diff --git a/Help/markdown/readfile.md b/Help/markdown/readfile.md new file mode 100644 index 0000000..4a9c1a3 --- /dev/null +++ b/Help/markdown/readfile.md @@ -0,0 +1,54 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# readfile + +## Purpose + +Read an entire text file into a string. + +## Syntax + +``` +returnedStr = readfile(pathStr) +``` + +``` +returnedStr - type: str + +pathStr - type: str +``` + +## Returns + +The full contents of the file as a single string. Returns no value if the path is not a readable regular file, if the file is empty, or if the file is larger than the 16MB limit; a warning is written to the log in the failure cases. + +## Remarks + +This is the counterpart to the output functions [openfile](openfile.md) and [fileout](fileout.md). It lets a pass load data that did not arrive as the analyzer's input text — a configuration file, a lookup table, or the output of an earlier run. + +A leading UTF-8 byte order mark is stripped, so a file saved by a Windows editor does not poison its own first token. A file containing NUL bytes is rejected as binary rather than being silently truncated at the first NUL. + +The contents are interned, so they stay allocated for the remainder of the parse. For large files prefer [readlines](readlines.md), which has no size limit because it interns one line at a time. + +Line terminators are returned as they appear in the file, so a file written on Windows yields `\r\n` and one written on Linux yields `\n`. If that distinction does not matter, normalize with [resubst](resubst.md) or read the file with [readlines](readlines.md), which strips terminators for you. + +Paths may use forward slashes or backslashes on Windows. Use [fileexists](fileexists.md) to test for the file first if a missing file is an expected condition rather than an error. + +## Example + +``` +@CODE + +L("path") = G("$apppath") + "/data/config.txt"; + +if (fileexists(L("path"))) { + L("text") = readfile(L("path")); + "out.txt" << "config is " << str(strlength(L("text"))) << " bytes\n"; +} + +@@CODE +``` + +## See Also + +[readlines](readlines.md), [dirlist](dirlist.md), [fileexists](fileexists.md), [filesize](filesize.md), [File Functions](Table_of_File_Functions.md) diff --git a/Help/markdown/readlines.md b/Help/markdown/readlines.md new file mode 100644 index 0000000..ba0985d --- /dev/null +++ b/Help/markdown/readlines.md @@ -0,0 +1,63 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# readlines + +## Purpose + +Read a text file as an array of its lines. + +## Syntax + +``` +returnedArray = readlines(pathStr) +``` + +``` +returnedArray - type: str array + +pathStr - type: str +``` + +## Returns + +An array holding one element per line of the file, with line terminators stripped. Returns an empty array if the file is empty or is not a readable regular file; a warning is written to the log in the latter case. + +## Remarks + +`readlines` streams the file, so unlike [readfile](readfile.md) it has no size limit and is the right choice for large data files. + +CRLF and LF are treated alike, so a file written on Windows reads the same on Linux. A leading UTF-8 byte order mark is stripped from the first line. + +A trailing newline at the end of the file does **not** produce a final empty element: a file holding `a\nb\n` yields two elements, not three. Blank lines in the middle of the file do come back as empty elements, so the element numbering always lines up with the line numbering of the file. + +Use [arraylength](arraylength.md) to get the count and index the result with square brackets, exactly as with [split](split.md). + +An empty or unreadable file gives an empty array. Guard a loop over it by testing the **first element** rather than the count: assigning an empty array to a variable collapses it back to no value, and `arraylength` reports 1 for a variable holding no value, so a count test would run the loop once over a phantom line. + +## Example + +``` +@CODE + +L("lines") = readlines(G("$apppath") + "/data/terms.txt"); + +# Guard on the first element, not the count -- see Remarks. +if (strlength(L("lines")[0])) { + L("len") = arraylength(L("lines")); + L("i") = 0; + while (L("i") < L("len")) { + L("line") = L("lines")[L("i")]; + # Skip blank lines and "#" comments. + if (strlength(L("line")) && !strstartswith(L("line"),"#")) { + addword(L("line")); + } + L("i")++; + } +} + +@@CODE +``` + +## See Also + +[readfile](readfile.md), [dirlist](dirlist.md), [split](split.md), [arraylength](arraylength.md), [File Functions](Table_of_File_Functions.md) diff --git a/Help/markdown/refind.md b/Help/markdown/refind.md new file mode 100644 index 0000000..0e4d009 --- /dev/null +++ b/Help/markdown/refind.md @@ -0,0 +1,70 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# refind + +## Purpose + +Find a regular expression match and its capture groups. + +## Syntax + +``` +returnedArray = refind(str, patternStr) +returnedArray = refind(str, patternStr, flagsStr) +``` + +``` +returnedArray - type: str array + +str - type: str + +patternStr - type: str + +flagsStr - type: str +``` + +## Returns + +An array in which element 0 is the whole match and elements 1..n are the capture groups, in the order their opening parentheses appear in the pattern. Returns an empty array if the pattern does not match. + +## Remarks + +This is how a pass pulls pieces **out** of a string. Before `refind`, extracting something like the digits after a dash meant walking the string a character at a time with [strchar](strchar.md). + +Only the **first** match is returned. A group that took part in no match, such as an unmatched alternative or an optional group that did not fire, comes back as an empty element. + +Full ECMAScript syntax, as with [rematch](rematch.md). Pass `"i"` as the flags argument to ignore case. + +Test the result before indexing it. Check the **first element**, not the count: assigning an empty array to a variable collapses it back to no value, and [arraylength](arraylength.md) reports 1 for a variable holding no value, so a count test would read 1 on a failed match. + +``` +L("g") = refind(L("s"),"([a-z]+)-([0-9]+)"); +if (strlength(L("g")[0])) { + L("word") = L("g")[1]; + L("num") = num(L("g")[2]); +} +``` + +Calling [rematch](rematch.md) with the same pattern is an equally good guard, and costs little because the compiled pattern is cached and shared between the two calls. + +Patterns are compiled once and cached. A malformed pattern writes a warning to the log and returns no value. + +## Example + +``` +@CODE + +# Pull the key and value out of a "name = widget" configuration line. +L("g") = refind(L("line"),"^([a-z]+) *= *(.+)$"); + +if (strlength(L("g")[0])) { + "out.txt" << "key=" << L("g")[1] + << " value=" << L("g")[2] << "\n"; +} + +@@CODE +``` + +## See Also + +[rematch](rematch.md), [resubst](resubst.md), [split](split.md), [strpiece](strpiece.md), [String Functions](Table_of_String_Functions.md) diff --git a/Help/markdown/regexp.md b/Help/markdown/regexp.md index a94aa4f..d561033 100644 --- a/Help/markdown/regexp.md +++ b/Help/markdown/regexp.md @@ -28,10 +28,14 @@ Succeeds if the pattern matches, else fails. A pattern consists of text and the special characters ? and *. ? matches any single character, and * matches any zero or more characters.  (There is no way to currently escape ? and *).  Matching is case sensitive. +The pattern must match the **whole** string, not a part of it. `regexp("run")` does not match `running`, and `regexp("at")` does not match `cat`. Wrap the pattern in stars to search inside a word: `regexp("*at*")` matches `cat`. + ## Example # This will match words such as junk, junks, junky, junkyard. @PRE <1,1> regexp("junk*"); @POST group(1,1,"_junkword"); @RULES _xNIL <- _xALPHA @@ +For a full regular expression — character classes, anchors, quantifiers, alternation and capture groups — and one that can be called from anywhere rather than only from a `@PRE` region, see [rematch](rematch.md), [refind](refind.md) and [resubst](resubst.md). + ## See Also -[regexpi](regexpi.md), [PRE Actions](NLP_PP_Stuff/AT-PRE_Actions.md) +[regexpi](regexpi.md), [rematch](rematch.md), [refind](refind.md), [resubst](resubst.md), [PRE Actions](NLP_PP_Stuff/AT-PRE_Actions.md) diff --git a/Help/markdown/regexpi.md b/Help/markdown/regexpi.md index 1ff2d8b..e3924bc 100644 --- a/Help/markdown/regexpi.md +++ b/Help/markdown/regexpi.md @@ -26,12 +26,16 @@ Succeeds if the pattern matches, else fails. ## Remarks -A pattern consists of text and the special characters ? and *. ? matches any single character, and * matches any zero or more characters.  (There is no way to currently escape ? and *).  Matching is case insensitive. +A pattern consists of text and the special characters ? and *. ? matches any single character, and * matches any zero or more characters.  (There is no way to currently escape ? and *).  Matching is case insensitive, for every element of the pattern. + +The pattern must match the **whole** string, not a part of it. `regexpi("RUN")` does not match `running`, and `regexpi("AT")` does not match `cat`. Wrap the pattern in stars to search inside a word: `regexpi("*AT*")` matches `cat`. ## Example # This will match words such as JUNK, junks, junky, JunkYard. @PRE <1,1> regexpi("junk*"); @POST group(1,1,"_junkword"); @RULES _xNIL <- _xALPHA @@ +For a full regular expression — character classes, anchors, quantifiers, alternation and capture groups — and one that can be called from anywhere rather than only from a `@PRE` region, see [rematch](rematch.md) with the `"i"` flag, plus [refind](refind.md) and [resubst](resubst.md). + ## See Also -[regexp](regexp.md), [PRE Actions](NLP_PP_Stuff/AT-PRE_Actions.md) +[regexp](regexp.md), [rematch](rematch.md), [refind](refind.md), [resubst](resubst.md), [PRE Actions](NLP_PP_Stuff/AT-PRE_Actions.md) diff --git a/Help/markdown/rematch.md b/Help/markdown/rematch.md new file mode 100644 index 0000000..58c9566 --- /dev/null +++ b/Help/markdown/rematch.md @@ -0,0 +1,62 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# rematch + +## Purpose + +Test a string against a regular expression. + +## Syntax + +``` +returnedBool = rematch(str, patternStr) +returnedBool = rematch(str, patternStr, flagsStr) +``` + +``` +returnedBool - type: bool (1,0) + +str - type: str + +patternStr - type: str + +flagsStr - type: str +``` + +## Returns + +1 if the pattern matches, else 0. + +## Remarks + +`rematch` uses full ECMAScript regular expression syntax, so character classes, anchors, quantifiers, alternation and groups all work. This is not the case for the older [regexp](regexp.md), which is a `*`/`?` glob, understands no other metacharacter, and can only be written in a `@PRE` region. `rematch` is an ordinary function and can be called anywhere NLP++ code can be written. + +`rematch` **searches** the string: the pattern may match anywhere inside it. Anchor with `^` and `$` to require the whole string to match. + +Pass `"i"` as the flags argument to ignore case. It is the only flag defined; anything else is an error. + +Patterns are compiled once and cached, so calling `rematch` from inside a rule loop does not recompile the pattern on every node. + +A malformed pattern writes a warning to the log and yields 0 rather than aborting the pass. + +## Example + +``` +@CODE + +# Anchored, so the whole token must be an ISBN-looking string. +if (rematch(N("$text"),"^[0-9]{3}-[0-9]{10}$")) { + N("isbn") = 1; +} + +# Unanchored search, ignoring case. +if (rematch(L("line"),"error|warning","i")) { + G("problems")++; +} + +@@CODE +``` + +## See Also + +[refind](refind.md), [resubst](resubst.md), [regexp](regexp.md), [strcontains](strcontains.md), [String Functions](Table_of_String_Functions.md) diff --git a/Help/markdown/resubst.md b/Help/markdown/resubst.md new file mode 100644 index 0000000..4b7e67d --- /dev/null +++ b/Help/markdown/resubst.md @@ -0,0 +1,68 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# resubst + +## Purpose + +Replace regular expression matches in a string. + +## Syntax + +``` +returnedStr = resubst(str, patternStr, replacementStr) +returnedStr = resubst(str, patternStr, replacementStr, flagsStr) +``` + +``` +returnedStr - type: str + +str - type: str + +patternStr - type: str + +replacementStr - type: str + +flagsStr - type: str +``` + +## Returns + +The string with every match replaced. Returns no value if the input string is empty or if the result of the replacement is empty. + +## Remarks + +`resubst` replaces **every** match, where [strsubst](strsubst.md) replaces a fixed substring. Together with the pattern syntax that means one call can do work that previously took a loop. + +The replacement string may refer back to the match: + +| **FORM** | **MEANING** | +| --- | --- | +| **$1**, **$2** ... | The text of capture group 1, 2 ... | +| **$&** | The whole match | +| **$$** | A literal dollar sign | + +Full ECMAScript syntax, as with [rematch](rematch.md). Pass `"i"` as the flags argument to ignore case. + +Patterns are compiled once and cached. A malformed pattern or replacement writes a warning to the log and returns no value. + +## Example + +``` +@CODE + +# Collapse any run of whitespace to a single blank. +L("clean") = resubst(L("text")," +"," "); + +# Normalize line endings so LF and CRLF files compare equal. +L("flat") = resubst(readfile(L("path")),"\r?\n","|"); + +# Swap the two halves of each token, using backreferences. +# "a1 b2 c3" becomes "1a 2b 3c". +L("swapped") = resubst("a1 b2 c3","([a-z])([0-9])","$2$1"); + +@@CODE +``` + +## See Also + +[rematch](rematch.md), [refind](refind.md), [strsubst](strsubst.md), [strclean](strclean.md), [String Functions](Table_of_String_Functions.md) diff --git a/Help/markdown/strindexof.md b/Help/markdown/strindexof.md new file mode 100644 index 0000000..9656d40 --- /dev/null +++ b/Help/markdown/strindexof.md @@ -0,0 +1,50 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# strindexof + +## Purpose + +Find the position of the first occurrence of a substring. + +## Syntax + +``` +returnedInt = strindexof(str, substr) +``` + +``` +returnedInt - type: int + +str - type: str + +substr - type: str +``` + +## Returns + +The zero-based index at which substr first appears in str, or -1 if it does not appear. + +## Remarks + +[strchr](strchr.md) takes only a single character and returns the tail of the string rather than a position, so getting an index used to mean subtracting two [strlength](strlength.md) calls. [strcontains](strcontains.md) answers only yes or no. `strindexof` takes a whole substring and gives the position directly. + +Matching is case sensitive. + +Because "not found" is -1 and 0 is a legitimate result meaning "at the very start", test the result against -1 rather than relying on truthiness. + +## Example + +``` +@CODE + +L("at") = strindexof("hello world","world"); # 6 +if (L("at") >= 0) { + L("tail") = strpiece(L("s"),L("at"),strlength(L("s")) - 1); +} + +@@CODE +``` + +## See Also + +[strlastindexof](strlastindexof.md), [strchr](strchr.md), [strcontains](strcontains.md), [strpiece](strpiece.md), [String Functions](Table_of_String_Functions.md) diff --git a/Help/markdown/strjoin.md b/Help/markdown/strjoin.md new file mode 100644 index 0000000..3d3e1f6 --- /dev/null +++ b/Help/markdown/strjoin.md @@ -0,0 +1,51 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# strjoin + +## Purpose + +Join the elements of an array into a single string. + +## Syntax + +``` +returnedStr = strjoin(array, separatorStr) +``` + +``` +returnedStr - type: str + +array - type: array + +separatorStr - type: str +``` + +## Returns + +The elements run together with the separator between each pair. Returns no value if the array is empty or every element is empty. + +## Remarks + +This is the inverse of [split](split.md). Without it, building a delimited string meant a loop plus a "first time through" flag to decide whether to emit a separator. + +The separator goes only **between** elements, never before the first or after the last. An array of one element comes back unchanged, with no separator at all. + +Numeric and float elements are rendered as text, so an array of mixed types still joins sensibly. Empty elements contribute nothing but are still separated, so joining `a`, ``, `c` with `,` gives `a,,c`. + +## Example + +``` +@CODE + +L("parts") = split("a b c"," "); +"out.txt" << strjoin(L("parts"),"-") << "\n"; # a-b-c + +# Round trip: re-join what split took apart, with a new separator. +L("csv") = strjoin(readlines(L("path")),","); + +@@CODE +``` + +## See Also + +[split](split.md), [arraylength](arraylength.md), [arraysort](arraysort.md), [String Functions](Table_of_String_Functions.md) diff --git a/Help/markdown/strlastindexof.md b/Help/markdown/strlastindexof.md new file mode 100644 index 0000000..eaba7b8 --- /dev/null +++ b/Help/markdown/strlastindexof.md @@ -0,0 +1,49 @@ +[← Help Contents](index.md) | [📘 NLP++ Textbook](NLP++_Textbook.md) + +# strlastindexof + +## Purpose + +Find the position of the last occurrence of a substring. + +## Syntax + +``` +returnedInt = strlastindexof(str, substr) +``` + +``` +returnedInt - type: int + +str - type: str + +substr - type: str +``` + +## Returns + +The zero-based index at which substr last appears in str, or -1 if it does not appear. + +## Remarks + +The mirror of [strindexof](strindexof.md). Useful for splitting on the final separator, such as taking the extension off a filename or the last segment off a path. + +Matching is case sensitive. As with [strindexof](strindexof.md), test against -1 rather than relying on truthiness, since 0 is a valid position. + +## Example + +``` +@CODE + +# Take the basename off a path returned by dirlist. +L("cut") = strlastindexof(L("path"),"/"); +if (L("cut") >= 0) { + L("name") = strpiece(L("path"),L("cut") + 1,strlength(L("path")) - 1); +} + +@@CODE +``` + +## See Also + +[strindexof](strindexof.md), [strrchr](strrchr.md), [dirlist](dirlist.md), [String Functions](Table_of_String_Functions.md)