Skip to content

fix(volume-backups): restart services after backup failure - #5082

Open
bestmaa wants to merge 2 commits into
Dokploy:canaryfrom
bestmaa:fix/volume-backup-restart
Open

fix(volume-backups): restart services after backup failure#5082
bestmaa wants to merge 2 commits into
Dokploy:canaryfrom
bestmaa:fix/volume-backup-restart

Conversation

@bestmaa

@bestmaa bestmaa commented Aug 14, 2026

Copy link
Copy Markdown

Problem

When Turn Off Container During Backup is enabled, the generated volume-backup script stops the application or Compose service before creating the archive. Because the script runs with set -e, a backup failure exits immediately and skips the restart command, leaving the service offline until someone restarts it manually.

This is especially disruptive for transient failures such as an unavailable image registry or a failed ubuntu image pull.

What changed

  • Run the backup step in a subshell and capture its exit status.
  • Always execute the service restart after the backup attempt.
  • Preserve and return the original backup failure status after the restart, so the deployment remains marked as failed and the upload is skipped.
  • Use the same restart-safe command flow for applications, Compose stacks, and regular Compose containers.
  • Add regression tests covering both failure and success command sequences.

Impact

Services that are intentionally stopped for a consistent volume backup are brought back online even when archive creation fails. Successful backup and upload behavior remains unchanged.

Validation

  • pnpm --filter=dokploy exec vitest --config __test__/vitest.config.ts run __test__/backups — 23 tests passed
  • pnpm --filter=@dokploy/server typecheck
  • pnpm --filter=dokploy typecheck
  • Biome check on the changed files

Fixes #4263

Greptile Summary

This PR makes volume backups restart stopped services even when backup creation fails, while preserving the appropriate failure status.

  • Runs backup and restart operations in separately controlled shell contexts.
  • Preserves the original backup status when both backup and restart fail.
  • Skips upload unless backup and restart both succeed.
  • Adds regression coverage for all backup/restart success and failure combinations.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported failure is fixed: a failed restart can no longer terminate the outer shell before the saved backup status is restored, and no blocking failure remains.

Reviews (2): Last reviewed commit: "fix(volume-backups): preserve backup fai..." | Re-trigger Greptile

Context used:

@bestmaa
bestmaa marked this pull request as ready for review August 14, 2026 11:45
@bestmaa
bestmaa requested a review from Siumauricio as a code owner August 14, 2026 11:45
@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Aug 14, 2026
Comment on lines +31 to +35
set -e
${startCommand}
if [ "$DOKPLOY_VOLUME_BACKUP_STATUS" -ne 0 ]; then
exit "$DOKPLOY_VOLUME_BACKUP_STATUS"
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Restart masks backup status

When both the backup and subsequent restart fail, set -e exits during startCommand before the saved backup status is restored, so logs and failure notifications report the restart exit code instead of the original backup failure code.

Knowledge Base Used: Backups and Schedules

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Container not restarted after volume backup fails mid-run when "Turn Off Container During Backup" is enabled

1 participant