Skip to content

Overhauled staging tables with schemas and centralised specifiers#22

Open
nicoloesch wants to merge 2 commits into
mainfrom
15-staging
Open

Overhauled staging tables with schemas and centralised specifiers#22
nicoloesch wants to merge 2 commits into
mainfrom
15-staging

Conversation

@nicoloesch

Copy link
Copy Markdown
Collaborator

Summary

  • Fixes Make staging table location schema-aware instead of relying on the _staging_ prefix #15

  • Backend now owns staging table naming. staging_name_for_table(tablename) is a @staticmethod @abstractmethod on DatabaseBackend

    • each concrete backend declares its own naming strategy
    • PostgresBackend and SQLiteBackend both implement f"_staging_{tablename}" (Postgres relies on schema isolation; SQLite needs the prefix for collision avoidance).
  • staging_name removed as a parameter from all backend abstract methods

    • create_staging_table,
    • drop_staging_table,
    • merge_replace,
    • merge_upsert,
    • merge_insert,
    • load_staging_fast
  • Backends derive the name from table_cls.__tablename__ internally via staging_name_for_table.

  • qualify_identifier(name, schema) extracted to helpers/sql.py and exported from helpers/__init__.

    • Used by both backends/base.py and loaders/loading_helpers.py, replacing two independent inline f-strings.
  • Circular import (backends/postgres -> loaders/loading_helpers -> helpers -> helpers/bulk -> backends/resolve -> backends/postgres) fixed by making resolve_backend imports in helpers/bulk.py lazy (inside function bodies).

  • STAGING_SCHEMA: str = "staging" convention constant introduced in backends/base.py and exported from backends/__init__

    • Named as a convention (not a constructor default)
    • The base class __init__ defaults staging_schema to None (schema-agnostic).
    • PostgresBackend.__init__ opts in by defaulting to STAGING_SCHEMA.
    • SQLite always passes None.
    • Future schema-capable backends (e.g. DuckDB) follow the Postgres pattern.
  • Dead _staging_tablename: ClassVar[Optional[str]] = None on CSVLoadableTableInterface removed

    • staging_tablename() classmethod removed from model and from CSVTableProtocol.
  • FakeBackend in tests updated to match current abstract interface (stale staging_name positional params removed; staging_name_for_table implemented).

@nicoloesch nicoloesch added the fix Bug fix, backwards-compatible. PATCH: x.y.z+1 label Jul 23, 2026
@nicoloesch
nicoloesch marked this pull request as ready for review July 23, 2026 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Bug fix, backwards-compatible. PATCH: x.y.z+1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make staging table location schema-aware instead of relying on the _staging_ prefix

1 participant