Skip to content

Optimize beat reloads for far-future clocked tasks#1041

Open
diaxoaine wants to merge 5 commits into
celery:mainfrom
diaxoaine:optimize
Open

Optimize beat reloads for far-future clocked tasks#1041
diaxoaine wants to merge 5 commits into
celery:mainfrom
diaxoaine:optimize

Conversation

@diaxoaine

@diaxoaine diaxoaine commented May 24, 2026

Copy link
Copy Markdown
Contributor

Description

ClockedSchedule is often used for dynamically created one-off tasks. In high-load apps, these schedules may be created very frequently, which can lead to frequent PeriodicTasks updates and cause beat to reload from the database more often than necessary, potentially as often as every 5 seconds.

This PR avoids unnecessary PeriodicTasks change updates for newly created clocked schedules/tasks that are scheduled far enough in the future.

@codecov

codecov Bot commented May 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.95%. Comparing base (89f4062) to head (f1cd98e).
⚠️ Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1041      +/-   ##
==========================================
+ Coverage   87.68%   87.95%   +0.27%     
==========================================
  Files          32       32              
  Lines        1015     1038      +23     
  Branches       81       85       +4     
==========================================
+ Hits          890      913      +23     
  Misses        107      107              
  Partials       18       18              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces unnecessary PeriodicTasks “last_change” updates (and thus DatabaseScheduler reloads) when creating new clocked schedules/tasks that are sufficiently far in the future, relying on the scheduler’s periodic full sync instead.

Changes:

  • Add sync-deadline helpers (next_schedule_sync_at/by) and expose scheduler sync interval constants via utils.
  • Skip PeriodicTasks.update_changed() on creation of far-future ClockedSchedule rows and far-future created PeriodicTask rows with a clocked schedule.
  • Add unit tests covering in-window vs out-of-window change tracking, including configured beat_max_loop_interval.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
t/unit/test_schedulers.py Adds tests validating when clocked creations/updates should (or should not) bump PeriodicTasks.last_change().
django_celery_beat/utils.py Centralizes scheduler interval constants and adds helpers to compute next sync timing/deadline.
django_celery_beat/signals.py Replaces ClockedSchedule post-save handler with a conditional version that may skip forced reloads.
django_celery_beat/schedulers.py Uses the new helper for computing the next full-sync cutoff for excluding far-future clocked tasks.
django_celery_beat/models.py Passes a created flag into PeriodicTasks.changed() and conditionally skips change tracking for far-future created clocked tasks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread django_celery_beat/signals.py Outdated
Comment thread django_celery_beat/models.py
Comment thread django_celery_beat/utils.py Outdated
auvipy and others added 3 commits May 25, 2026 13:39
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants