Skip to content

rebuild_audit_projections wraps each group in its own transaction, so a mid-run failure leaves a silently half-rebuilt corpus and prints no summary #300

Description

@erskingardner

Summary

rebuild_audit_projections rebuilds each group inside its own transaction.atomic() block, with no outer transaction and no error handling around the loop. If any group raises, the groups already processed are committed, the remaining groups keep their stale projections, and the SUCCESS summary never prints — the operator is left with a partially rebuilt corpus and no report of how far the run got.

Code

forensics/management/commands/rebuild_audit_projections.py:53-65:

def handle(self, *args, **options):
    groups = self.selected_groups(options["groups"], options["audit_file_ids"])
    for group in groups:
        with transaction.atomic():        # per-group atomic; no outer txn, no try/except
            rebuild_group_projections(group)
    after_counts = projection_counts()
    self.stdout.write(self.style.SUCCESS(
        "Rebuilt audit projections for "
        f"{len(groups)} group(s): {format_projection_counts(after_counts)}"
    ))

Concrete failure scenario

A full rebuild processes groups 1..n. Group k raises during rebuild_group_projections — e.g. one of the already-filed projection-overflow conditions (#144 / #160) or a non-unique group-selector condition (#175). Groups 1..k-1 are already committed; groups k..n retain their stale projections; the exception propagates and the SUCCESS line at the end never runs. The operator sees a traceback but no indication that the corpus is now in a mixed rebuilt/stale state, or which groups were completed. A --group-scoped rebuild that half-fails is especially misleading because the command is the documented remediation tool.

Why not a duplicate

#252 is about the count numbers being whole-corpus for a scoped run (misreporting scope). #175/#186 are about group selection (get() on a non-unique column / --audit-file-id ignored). #174 is purge_audit_data doing everything in one in-memory transaction (OOM). None cover rebuild_audit_projections's partial-commit-on-error behavior or its lack of a progress/failure report.

Suggested fix

Either wrap the whole run in one transaction so a mid-run failure rolls back to a consistent pre-run state, or (better for large corpora) keep per-group commits but catch and report per-group failures, continue where safe, and emit a summary of succeeded/failed/remaining groups so the operator knows the exact post-run state.

Severity: LOW — operational robustness / observability of a maintenance command.

Metadata

Metadata

Assignees

No one assigned

    Labels

    LOWSeverity: minor correctness, polish, or maintainability issuebugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions