Skip to content

ConfigResolver, SIGHUP handling, add table - #45

Merged
serprex merged 6 commits into
mainfrom
config
Jul 7, 2026
Merged

ConfigResolver, SIGHUP handling, add table#45
serprex merged 6 commits into
mainfrom
config

Conversation

@serprex

@serprex serprex commented Jul 1, 2026

Copy link
Copy Markdown
Member

flesh out initial load options

@serprex
serprex requested a review from harshil-goel July 1, 2026 06:23
@serprex

serprex commented Jul 1, 2026

Copy link
Copy Markdown
Member Author

plan here is to start with getting toml config stable, then build source config / logical signals on top as optional functionality

plan has also been more closely reviewed / brought up to date, & extended with a mechanism to limit logical signals to a database so that on ubicloud we could limit it to requiring ubi_admin access, so that only control plane can send logical signals

@serprex
serprex force-pushed the config branch 6 times, most recently from c824f2b to 078d05d Compare July 3, 2026 02:07
@serprex
serprex marked this pull request as ready for review July 3, 2026 04:59
@serprex serprex changed the title ConfigResolver, SIGHUP handling ConfigResolver, SIGHUP handling, add table Jul 3, 2026
@serprex
serprex force-pushed the config branch 2 times, most recently from 786fe62 to 1df1812 Compare July 4, 2026 00:45
Comment thread plans/future/dependencies.md Outdated
Comment thread plans/add_table.md Outdated
Comment thread plans/future/pipeline_backpressure_and_scaling.md Outdated
serprex added 4 commits July 6, 2026 17:07
When tables join a stream already grown,
we read their past from backups, page by page;
through one decoder every row is known,
no second codec drifting with its age

The gate admits what truly was committed:
the dead stay dead, the half-written must wait,
and writers still in flight are here omitted;
the live stream brings their changes, never late

We pour the rows into a hidden twin,
and only when the whole of it is sound
we swap the names (one motion) new rows in;
a failure leaves no phantom to be found

The ledger writes each step before we act,
so any crash resumes with truth intact
…o ClickHouse

Fixes bookkeeping flaw where config reload could wipe out routing state walshadow had learned (auto-created tables, columns added by schema changes)

Support recreation during create table, drop table, create table
@serprex
serprex merged commit 5e83ce2 into main Jul 7, 2026
5 checks passed
@serprex
serprex deleted the config branch July 7, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants