Skip to content

I34 controlled startup - #36

Merged
QSparks merged 6 commits into
i33-lighthousefrom
i34-controlled-startup
Aug 4, 2026
Merged

I34 controlled startup#36
QSparks merged 6 commits into
i33-lighthousefrom
i34-controlled-startup

Conversation

@QSparks

@QSparks QSparks commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Previously, a pull or redeploy could start all Swarm services at roughly the same time because Swarm does not use depends_on for readiness.
The importer can take time to populate the DB, so some downstream containers would fail to start and exhaust their max_attempts before upstream services were ready.
This would often require a full stop/pull/start cycle.

The new flow makes startup and redeployment deterministic:

  • Redeploying with the same CHYP_IMPORT_ID skips the import if it already completed.
  • A new import builds separate staging tables while the current dataset stays available.
  • Once the import finishes, the new dataset replaces the old one in a single database transaction.
  • BBOX waits for the matching import to finish before starting.
  • The app and Varnish wait for BBOX to serve that exact import.
  • Varnish starts with an empty tile cache only after the new dataset is ready.
  • Health checks keep the existing reader tasks serving until the replacements are ready.
  • If an import is interrupted, the next importer removes the incomplete staging tables and retries without affecting the live dataset.
  • After a Swarm or node restart, each service performs its own readiness checks instead of relying on container startup order.

Normal Portainer Pull and redeploy should now be able to replace services without manually stopping the whole stack first.

PostGIS uses stop-first when it is replaced, but the database remains on shared persistent storage.
If PostgreSQL crashes during the final swap, the database should contain either the previous dataset or the fully committed new dataset, not a partially published state.

Resolves #34

@QSparks
QSparks requested a review from Nospamas July 21, 2026 16:32
@QSparks QSparks self-assigned this Jul 21, 2026
@QSparks
QSparks marked this pull request as ready for review July 27, 2026 21:07

@Nospamas Nospamas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, but comment.

Comment thread docker/app/server.js
const delay = (milliseconds) =>
new Promise((resolve) => setTimeout(resolve, milliseconds));

const waitForStartupUrl = async (url) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency with other parts of this stack is it worth considering moving this statup logic into bash and running it before starting the server proper?

@QSparks
QSparks merged commit a0bf4ee into i33-lighthouse Aug 4, 2026
1 check passed
QSparks added a commit that referenced this pull request Aug 4, 2026
* I31 peace columbia (#32)
* I33 lighthouse (#35), 
* I34 controlled startup (#36)
* I37 bulk download (#39)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants