django-celery/docs.txt at main · cseshahriar/django-celery · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
3. Why Celery?

Without Celery:
    * A single job application takes ~10 seconds (PDF + Upload + Email).
    * Users leave frustrated.
    * Django request threads are blocked, reducing server availability.

With Celery:
* Django returns immediately.
* Heavy lifting is done in the background.

process you’re setting up requires at least three services to run at the same time:
    * Producer: Your Django app
    * Message Broker: The Redis server
    * Consumer: Your Celery app

# Full Example of Task Lifecycle (with these enabled):
Task Queued → SENT → STARTED → SUCCESS
                        ↳ FAILURE

# celery start
python -m celery -A django_celery worker -E
python -m celery -A django_celery flower

Flower is a real-time web-based monitoring tool for Celery tasks. It helps you:

✅ See all running, pending, successful, and failed tasks
✅ Track task progress, retries, and execution time
✅ Monitor active workers
✅ Manually revoke, retry, or inspect tasks
✅ Visualize queues and workload
✅ View task arguments, results, and exceptions

It's like a dashboard for Celery — very helpful in both development and production.

http://localhost:5555/


# environment variables
.env
EMAIL_HOST_PASSWORD="xxx"
EMAIL_HOST_USER="xxx@gmail.com"

User clicks "Apply Now" → Celery Task Queue:
   ├─ ✅ Generate PDF from HTML template (5s)
   ├─ ✅ Store PDF in S3 (3s)
   └─ ✅ Send Email to HR + applicant (2s)

Celery for Django?
------------------
Celery is a 'distributed task queue' that allows you to run 'time-intensive tasks' in the 'background',
ensuring your Django app remains responsive.

Radis ?
--------------------
Redis acts as a message broker and can also serve as a results back end,
facilitating 'communication' between your 'Django app' and the 'Celery task queue'.

Celery Task Do?
-------------------
A Celery task performs operations asynchronously, allowing you to handle time-consuming processes in the background without blocking your Django app


What Celery Can Do ?
--------------------
Asynchronous Task Processing
Scheduled Tasks (Periodic)
Retry Failed Tasks
Chained or Grouped Tasks
Real-Time Processing
Task Monitoring


Celery Tasks
----------------------------
* Asynchronous Task Execution
* Distributed Task Queue
* Task Scheduling and Periodic Tasks
* Result Handling
* Error Handling and Retry Mechanism
* Monitoring and Management

Message Provider --> Message Broker --> Celery Worker --> Result Backend
Django            --> Redis             --> Celery         --> DB
Message Provider  --> Message Broker    --> Celery Worker  --> Result Backend
Django --> Redis --> Celery --> DB

Examples of Tasks
--------------------
Email & Notifications
File & Image Processing

⚙️ How Celery Works (Basic Flow)
--------------------------------
* You define a task (a Python function decorated with @shared_task).
* You call the task asynchronously using .delay() or .apply_async().
* A worker process (started via celery -A your_project worker) picks up the task from the broker.
* The broker (like Redis or RabbitMQ) is the message queue that passes tasks from Django to the worker.
* The worker executes the task, and optionally stores the result in a result backend (like Redis, DB).
* (Optional) You use Flower to monitor the task execution.

Common Celery Setup in Django
-----------------------------
    Broker: Redis or RabbitMQ
    Result Backend: Redis, Database, or other cache
    Task Queue: Celery
    Scheduler (optional): Celery Beat
    Monitoring (optional): Flower

What You Now Know:
--------------------
    How Celery works with Django using task queues
    How to send emails/SMS/WhatsApp asynchronously
    How to handle bulk messaging (like to 1000+ users)
    How to use .delay() and .apply_async()
    Basics of error handling, batching, and throttling


Step-by-Step in Django Admin:
* Go to Admin Panel → Interval Schedule → Add
* Set:
    Every: 2
    Period: Minutes
Save

* Go to Periodic Tasks → Add
* Set:
    Name: Print every 2 minutes
    Task: myapp.tasks.send_reminder ← replace with your actual task
    Interval: select the one you just created
    Enabled: ✅ Checked

Now send_reminder will run every 2 minutes.

# https://testdriven.io/blog/django-celery-periodic-tasks/
https://redis.io/docs/latest/operate/oss_and_stack/install/archive/install-redis/install-redis-on-linux/


=========================== docker ============================================