Dictionary join=, split=, _name lines; bare entries load (#433 #434 #426 #417) (4.2.30)

[ddehilster]

Four dictionary-tokenizer (dicttok) issues. The syntax follows the choices made in review.

#433: bare entries (bug). I couldn't reproduce the issue's original symptoms on master: hot - rod s=hotrod gives _hotrod, HOT-3 doesn't match, and \- works for unknown words. The real bug was different. A .dict line with no = (license plate) was a "missing equal sign" error, readDict stopped there, and every entry after it was dropped. The error only went to cgerr, so nothing looked wrong. Now:

• a line with no attributes is an entry, and a multi-word one is a _phrase;
• any bad line is reported and skipped, and the rest of the file is read.

#434: join=1. hot - rod join=1 turns HOT-ROD into one token HOTROD, spanning the same text. Whitespace and punctuation are dropped. The token is then looked up itself, so hotrod s=vehicle builds _vehicle over it.

#426: split="words".

• licenseplate split="license plate" replaces the token with adjacent tokens license (its own offsets 42-48) and plate (49-53).
• If the words don't spell the token (weathervconditions), each word spans the whole token.
• Split runs before join and phrases, so the pieces can match weather conditions s=weather.
• Pieces aren't split again.

#417: _name lines. _bridge type=location kind=generic puts those attributes on every _bridge the dictionary builds. An entry's own attributes win: viaduct s=bridge kind=raised gives kind=raised.

Implementation: lite/dicttok.cpp. It adds splitToken, joinPhrase, makeLitTok (a literal-flagged token, as makeliteral makes) and replaceRange. The attribute loop moves from findAttrs into applyAttrs, and join/split aren't copied onto nodes as attributes. The readDict/parseDictLine change is in cs/libconsh/cg.cpp.

Impact on existing dictionaries: I scanned 110 .dict files across analyzers, visualtext-files and package-analyzers. Three have a stray bare word on their last line. It now loads as a plain word, and nothing came after it to lose.

Tests:

• New fixture tests/dict-shape, with CI steps on Windows and Linux. The input is one line so the offsets hold on CRLF checkouts.
• Local: 45/46 CI steps pass. The compiled-bridge step needs $GITHUB_WORKSPACE and always fails locally.
• Full English Test: unchanged.
• 20 analyzers × 21 inputs against 4.2.18: 0 differences.
• The compiled analyzer gives the same output as the interpreter.

Docs: VisualText/visualtext-files PR to follow. Batched: don't tag yet.

🤖 Generated with Claude Code