Repository navigation
fix: bundle rfb/spec/xvar.nlp by re-syncing vendored NLPPlus/data with engine - #33
Merged
ddehilster merged 2 commits intoJun 20, 2026
Merged
Conversation
NLPPlus/data is a committed copy of the engine's data/ tree that the wheel
ships and __init__.py copies into the working folder at runtime. There was no
build-time sync from the nlp-engine submodule, so when the engine added the
_xVAR("attr") compiler pass the vendored copy never picked it up: rfb/spec/
xvar.nlp was missing entirely, and analyzer.seq/bigtok.nlp were stale. With no
xvar pass the rule compiler has no handler for _xVAR(...), so any analyzer using
it fails to build ("Couldn't build analyzer.") even though the engine source is
correct.
Re-mirror nlp-engine/data -> NLPPlus/data (adds xvar.nlp, updates analyzer.seq
and bigtok.nlp) and prevent future drift:
- update-nlp-engine.yml: rsync --delete the engine data into NLPPlus/data on
every engine bump and stage it, keeping the vendored copy in lockstep.
- test.yml: add a diff -rq guard that fails any PR where the two trees drift.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The drift guard caught a real mismatch: this branch pinned the engine at 2b156ba (engine 3.6.x without #677), but main pins ed06820, which is a descendant of 2b156ba that adds "accept unquoted _xVAR(attr)" (#677) and bumps NLP_ENGINE_VERSION to 3.6.2. The published wheel builds from main's pin, so the vendored data must match ed06820, not the older commit this branch happened to have checked out. Fast-forward the nlp-engine submodule 2b156ba -> ed06820 (no divergence; 2b156ba is an ancestor) and re-sync NLPPlus/data from it. The only data delta is xvar.nlp, which now also carries the unquoted _xVAR(attr)/_xVAR(123) rules (_LIT/_NUM). diff -rq nlp-engine/data NLPPlus/data is clean again. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The NLPPlus 2.0.20 wheel fails to build any analyzer that uses
_xVAR(...)(Couldn't build analyzer.), even though the engine source onmasterhas the fix (PRs #647 and #677) and the 2.0.20 submodule includes them.Root cause is packaging, not engine source:
NLPPlus/data/is a hand-vendored copy of the engine'sdata/tree, committed in this repo. The wheel ships it and__init__.pycopies it into the working folder at runtime. There is no build-time sync from thenlp-enginesubmodule, so when the engine added the_xVAR("attr")rule-compiler pass, the vendored copy never picked it up.Drift was exactly 3 files under
data/rfb/spec/:xvar.nlp— missing entirely (the pass that teaches the compiler_xVAR("attr"))analyzer.seq— missing thepat xvarline that runs the passbigtok.nlp— missing the_xNUM-in-names fixWith no
xvarpass, the rule compiler has no handler for_xVAR(...), so any analyzer using it fails to build and returns null.Fix
NLPPlus/datawithnlp-engine/data(addsxvar.nlp, updatesanalyzer.seq+bigtok.nlp).diff -rq nlp-engine/data NLPPlus/datais now clean.update-nlp-engine.yml:rsync -a --delete nlp-engine/data/ NLPPlus/data/on every engine bump, staged with the submodule.test.yml: adiff -rqguard that fails any PR where the two trees drift.Release note
This is a data-only fix; the engine SHA is unchanged from 2.0.20, so the automated bump workflow won't ship it. After merge, push a
v2.0.21tag to triggerpublish.yml.Acceptance
NLPPlus/data/rfb/spec/xvar.nlp._xVAR(...)analyzer builds and returns output.🤖 Generated with Claude Code