Background
NetBud's backend contains several pieces of subtle, failure-prone logic that run on every design request:
backend/app/jsonblock.py — extracting a JSON object from streamed LLM output, including a brace-counting truncation-recovery path for when the model runs out of tokens mid-object.
backend/app/rag/chroma_client.py — _sources_match / _source_tokens, a 60%-token-overlap heuristic that decides whether a citation is verified (the "don't let the LLM grade its own homework" keystone).
backend/app/intake/extractor.py — customer-id regex + section normalization.
Problem / Goal
There is no automated test suite anywhere in the repo — no tests/ directory, no pytest (or any test dependency) in backend/requirements.txt, and no test files under frontend/. The only hint tests were ever intended is .pytest_cache/ in .gitignore.
That means the trickiest code (truncation recovery, fuzzy citation matching, JSON extraction fallbacks) has zero regression protection, and the SQLite-migration work in the sibling parent issue will land without a safety net.
Goal: stand up a backend test suite (pytest) and cover the highest-risk pure-logic modules first.
Where to look
backend/requirements.txt — no test deps today.
backend/app/jsonblock.py — pure functions, no I/O, ideal first target.
backend/app/rag/chroma_client.py:106-133 — _source_tokens, _sources_match are pure and testable without a live Chroma.
backend/app/intake/extractor.py:38-46, 113-128 — extract_customer_id_regex (pure) and extract_brief (needs the LLM router stubbed).
.gitignore — already ignores .pytest_cache/.
Suggested approach
- Add
pytest (and pytest-asyncio for the async router paths) to a dev-dependency list, create a backend/tests/ package, and document cd backend && pytest in the README.
- Start with pure functions (no mocking):
jsonblock, chroma_client._sources_match, extractor.extract_customer_id_regex.
- For anything that calls the LLM router or Chroma, inject/monkeypatch a fake so tests stay offline and deterministic.
- Keep tests fast and hermetic — no network, no real SQLite file (use
sqlite:///:memory: where a DB is needed).
Sub-issues break this into: scaffolding, jsonblock tests, citation-matching tests, extractor tests.
Acceptance criteria
cd backend && pytest runs green with no network access.
- The four modules above have meaningful coverage of their edge cases (not just happy path).
- README documents how to run the tests.
Testing
cd backend && pytest -q from a clean checkout with .env absent should pass (tests must not require API keys).
Out of scope
- Frontend (Vitest/Jest) tests — separate effort.
- A CI pipeline / GitHub Actions workflow — can follow once tests exist.
- Integration tests against a live Chroma, Ollama, or GNS3 server.
Background
NetBud's backend contains several pieces of subtle, failure-prone logic that run on every design request:
backend/app/jsonblock.py— extracting a JSON object from streamed LLM output, including a brace-counting truncation-recovery path for when the model runs out of tokens mid-object.backend/app/rag/chroma_client.py—_sources_match/_source_tokens, a 60%-token-overlap heuristic that decides whether a citation is verified (the "don't let the LLM grade its own homework" keystone).backend/app/intake/extractor.py— customer-id regex + section normalization.Problem / Goal
There is no automated test suite anywhere in the repo — no
tests/directory, nopytest(or any test dependency) inbackend/requirements.txt, and no test files underfrontend/. The only hint tests were ever intended is.pytest_cache/in.gitignore.That means the trickiest code (truncation recovery, fuzzy citation matching, JSON extraction fallbacks) has zero regression protection, and the SQLite-migration work in the sibling parent issue will land without a safety net.
Goal: stand up a backend test suite (pytest) and cover the highest-risk pure-logic modules first.
Where to look
backend/requirements.txt— no test deps today.backend/app/jsonblock.py— pure functions, no I/O, ideal first target.backend/app/rag/chroma_client.py:106-133—_source_tokens,_sources_matchare pure and testable without a live Chroma.backend/app/intake/extractor.py:38-46, 113-128—extract_customer_id_regex(pure) andextract_brief(needs the LLM router stubbed)..gitignore— already ignores.pytest_cache/.Suggested approach
pytest(andpytest-asynciofor the async router paths) to a dev-dependency list, create abackend/tests/package, and documentcd backend && pytestin the README.jsonblock,chroma_client._sources_match,extractor.extract_customer_id_regex.sqlite:///:memory:where a DB is needed).Sub-issues break this into: scaffolding, jsonblock tests, citation-matching tests, extractor tests.
Acceptance criteria
cd backend && pytestruns green with no network access.Testing
cd backend && pytest -qfrom a clean checkout with.envabsent should pass (tests must not require API keys).Out of scope