Repository navigation
-DICTSOURCE: nodes record the .dict file and line that built them (#432 #475) (4.2.33) - #789
Merged
Merged
Conversation
…he sequence (#323) (4.2.32) Stage 2 of #323. - copy_file: std::filesystem::copy_file, byte for byte, instead of a character loop through text-mode streams. - resolve_file (KB "take" paths): one std::filesystem version for every platform in place of the Windows and Unix copies; the caller now checks the joined path fits its buffer (an unbounded sprintf could overflow it). - Ana::writeSeqfile copied tmp.seq to the sequence file while still open for writing, so its last unflushed lines could be lost; close it first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#475) (4.2.33) With -DICTSOURCE, each .dict entry gets dictfile and dictline attributes when it is read, so the nodes dicttok builds from it carry them. Tokens made by join= or split= get that entry's source unless their own word has an entry. Off by default: no memory or tree change otherwise. Set through a static (CG::setDictSource / NLP_ENGINE::setDictSource), so no API or class layout changes. README now lists -STDIN and -DICTSOURCE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #788. This branch contains #788's commit; merge #788 first and this diff shrinks to its own commit.
Closes #432 and gives #475 the engine side it needs. The design follows the review choice: an opt-in flag that records
dictfileanddictline.nlp ... -DICTSOURCE: while.dictfiles load, each entry getsdictfile(file name) anddictlineattributes. They travel onto the nodes dicttok builds the same way as any other dictionary attribute.join=1orsplit=gets that entry's source, unless the word it spells has its own entry.*full.dictentries get onlydictfile, because there's no line number.NLP_ENGINE::setDictSource→CG::setDictSource), soNLP_ENGINE::init, the Python binding and class layouts are untouched. The Python binding can call the setter later if it wants the feature.-STDIN(missing until now) and-DICTSOURCE.Example:
_bridge [...] ("type" "location") ("kind" "raised") ("dictfile" "words.dict") ("dictline" 14), where the entry's own line wins over the shared_bridgeline.For #475 (VSCode): the extension can pass
-DICTSOURCEon debug runs and readdictfile/dictlinefromfinal.treeto jump to or highlight the entry. That part is extension work and isn't in this PR.Tests:
tests/dict-source, reusing the dict-shape dictionary. Its CI step runs it twice: without the flag (nothing printed) and with it (14 nodes with the right file:line, including join/split tokens).$GITHUB_WORKSPACEand always fails locally.Docs: VisualText/visualtext-files PR follows. Batched: don't tag yet.
🤖 Generated with Claude Code