Skip to content

[Bug] Build journal writes/reads JSONL instead of the BuildJournalEntry table #6

Description

@sjohnston1972

Part of the "SQLite migration is half-done" parent issue.

Background

The build journal is an append-only log of build / dry-run / push / GNS3-deploy events. The README says it now lives in SQLite:

"Every dry-run and push is appended to the SQLite build_journal table."

The BuildJournalEntry table (backend/app/db/models.py:127) exists for exactly this.

Problem / Goal

The runtime journal is still a JSONL file:

  • backend/app/agents/build.py:97 append_journal writes data/build_journal.jsonl.
  • build.py:105 list_journal reads that file.
  • backend/app/api/journal.py:13 serves list_journal verbatim.

So the build_journal table is empty at runtime and the README claim is false.

Goal: append to and read from BuildJournalEntry.

Where to look

  • backend/app/agents/build.py:93-117_journal_path, append_journal, list_journal.
  • Call sites of append_journal: build.py:302, 318, 331 and api/build.py:297, 383, 544.
  • backend/app/db/models.py:127-141BuildJournalEntry columns: build_id, event, node, status, reason, ts (indexed).
  • backend/app/db/legacy_import.py:204-248import_journal maps a JSONL entry onto the table (note the extra ad-hoc keys like host/port/via/project_id some call sites pass, which the table has no column for).

Suggested approach

  1. Rewrite append_journal(entry: dict) to insert a BuildJournalEntry. The table has columns for build_id, event, node, status, reason, ts. Decide what to do with extra keys some callers pass (host, port, via, project_id, node_count): either drop them, or add a small JSON extra column to the model and stash them there. Pick one and be consistent.
  2. Rewrite list_journal(limit) to query the table ordered by ts desc, limited.
  3. api/journal.py needs no shape change if list_journal still returns a list of dicts.

Acceptance criteria

  • Every existing append_journal call site persists a row to build_journal.
  • GET /api/journal/list?limit=N returns the N most recent rows, newest first, from SQLite.
  • No data/build_journal.jsonl is written at runtime.
  • Extra per-event keys are either intentionally dropped or preserved in an extra column (documented in the PR).

Testing

Local backend. Trigger a dry-run (POST /api/build/{id}/dry-run/{node}) and a build:

  • sqlite3 data/netbud.sqlite "select event, build_id, node, status from build_journal order by ts desc limit 5;"
  • GET /api/journal/list returns those events newest-first.
    Add a unit test: append three events, assert list_journal returns them in reverse-chronological order.

Out of scope

  • Backfilling the old JSONL into the table (that's what legacy_import.py import_journal is for; leave it).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions