Summary
The compression pipeline and entity extractor produce incorrect/garbage output that pollutes storage and the entity graph.
Findings
Compression can output MORE than the input (high)
hexus/pipeline/router.py:111-118 — _compress_text splits on \n\n; for single-paragraph text (the common shape of a long memory entry) paragraphs[0] is the whole text, and it returns it verbatim plus "\n... [Truncated Text]".
- Scenario: a 50 KB single-paragraph payload passes through "compression" completely uncompressed, with a misleading truncation marker appended.
_compress_json (router.py:57-68) is likewise unbounded: ', '.join(keys) for a 10k-key object, or a 3-key sample whose values are megabyte strings, exceeds the input.
Entity extractor floods the graph with garbage (high)
hexus/entity_extractor.py
:12 hostname pattern \b[a-zA-Z0-9]+-[a-zA-Z0-9-]+\b matches every hyphenated word ("well-known", "multi-agent", "re-run" → hostnames). docker_image's \b\w+:\w+\b matches timestamps / key:value prose.version matches inside IP octets (10.0.0.1 → version 10.0.0).:39,56-58 seen is shared across entity types, so a value matching two patterns is classified by whichever regex runs first (dict order) — results depend on insertion order.- Scenario: ordinary prose floods the entities table and co-occurrence graph (
store.py:776,1410), degrading entity-based recall.
Suggested direction
Make _compress_text/_compress_json actually bound their output (truncate to a target size) and only emit the marker when real truncation happened; tighten the entity regexes (anchor hostnames to a TLD/.; require a registry/tag shape for docker images; exclude IPs from version matches) and make seen per-type; add unit tests (currently none — see the tests issue).
Filed from the 2026-07 codebase review.
Summary
The compression pipeline and entity extractor produce incorrect/garbage output that pollutes storage and the entity graph.
Findings
Compression can output MORE than the input (high)
hexus/pipeline/router.py:111-118—_compress_textsplits on\n\n; for single-paragraph text (the common shape of a long memory entry)paragraphs[0]is the whole text, and it returns it verbatim plus"\n... [Truncated Text]"._compress_json(router.py:57-68) is likewise unbounded:', '.join(keys)for a 10k-key object, or a 3-key sample whose values are megabyte strings, exceeds the input.Entity extractor floods the graph with garbage (high)
hexus/entity_extractor.py:12hostname pattern\b[a-zA-Z0-9]+-[a-zA-Z0-9-]+\bmatches every hyphenated word ("well-known", "multi-agent", "re-run" → hostnames).docker_image's\b\w+:\w+\bmatches timestamps /key:valueprose.versionmatches inside IP octets (10.0.0.1→ version10.0.0).:39,56-58seenis shared across entity types, so a value matching two patterns is classified by whichever regex runs first (dict order) — results depend on insertion order.store.py:776,1410), degrading entity-based recall.Suggested direction
Make
_compress_text/_compress_jsonactually bound their output (truncate to a target size) and only emit the marker when real truncation happened; tighten the entity regexes (anchor hostnames to a TLD/.; require a registry/tag shape for docker images; exclude IPs from version matches) and makeseenper-type; add unit tests (currently none — see the tests issue).Filed from the 2026-07 codebase review.