Fix worker Docker builds for dnspython and py3.9 numpy pin

[RishabhJain2018]

Summary

Fixes Docker image build failures for worker_py3_7, remote-worker, and worker_py3_9 during staging/production packaging.

Problem

1. dnspython==2.6.1 on Python 3.7 workers — dnspython 2.4+ requires Python 3.8+. The worker_py3_7 and remote-worker images (both built from the py3.7 Dockerfile) fail with:

   ERROR: No matching distribution found for dnspython==2.6.1
   

   Available versions on Python 3.7 stop at 2.3.0.

2. numpy==1.18.1 pre-install on prod worker_py3_9 — The production py3.9 worker Dockerfile pre-installs numpy==1.18.1 before worker_py3_9.txt, which pins numpy==1.26.4 and pulls in scipy==1.11.4 / scikit-learn==1.4.2. The stale numpy pin causes metadata-generation-failed during pip resolution. The dev py3.9 Dockerfile does not pre-install numpy and avoids this issue.

Solution

• Override dnspython to 2.3.0 in py3.7 worker Docker builds, following the same sed pattern already used for PyJWT version overrides.
• Remove the numpy==1.18.1 pre-install from the prod worker_py3_9 Dockerfile; cython==0.29.36 remains as the compile prerequisite.

Notes

• Django/celery (Python 3.9) continue to use dnspython==2.6.1 from requirements/common.txt for the CVE-2023-29483 fix.
• Python 3.7 workers only need dnspython transitively via prod.txt; they do not run account email validation, but the dependency must install cleanly.
• dns.resolver.resolve() used in the Django app is available in dnspython 2.3.0.

Testing

• CI worker image builds (worker_py3_7, worker_py3_8, worker_py3_9)
• Staging/production docker compose build succeeds

  

Summary by CodeRabbit

• Bug Fixes
  • Improved Python worker build reliability by updating dependency version handling for compatibility.
  • Ensured older Python environments continue to install supported versions of key packages.
  • Removed an overly strict early version pin so package resolution can use the most appropriate compatible release.

[coderabbitai]

Walkthrough

Updates Docker build files: worker_py3_7 (dev and prod) Dockerfiles now pin dnspython to 2.3.0 alongside the existing PyJWT 2.8.0 pin via combined sed commands, while the worker_py3_9 Dockerfile removes an explicit numpy==1.18.1 pin from its builder-stage pip install.

Changes

Docker worker Dockerfile dependency updates

Layer / File(s)	Summary
Pin dnspython alongside PyJWT docker/dev/worker_py3_7/Dockerfile, docker/prod/worker_py3_7/Dockerfile	Combined sed-based RUN steps now pin both PyJWT (2.8.0) and dnspython (2.3.0) in requirements/common.txt during the builder stage.
Remove explicit numpy pin docker/prod/worker_py3_9/Dockerfile	Removes numpy==1.18.1 from the builder-stage pip install command, leaving numpy version resolution to requirements files/transitive dependencies.

Estimated code review effort: 1 (Trivial) | ~3 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main changes: fixing worker Docker builds by adjusting dnspython for py3.7 and removing the py3.9 numpy pin.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cursor/fix-dnspython-py37-worker-build-942d

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

docker/prod/worker_py3_9/Dockerfile (1)

82-83: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Consider adding numpy/scipy to the import verification step.

The runtime import check on line 83 covers core dependencies but not the scientific computing stack (numpy, scipy, scikit-learn). Since this PR changes how numpy is installed, adding these to the verification would catch build or resolution failures early rather than at worker runtime.

💡 Suggested addition to import verification

 # Verify worker runtime imports required by submission_worker.py.
-RUN python -c "import boto3, botocore, django, requests, yaml"
+RUN python -c "import boto3, botocore, django, requests, yaml, numpy, scipy, sklearn"

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docker/prod/worker_py3_9/Dockerfile` around lines 82 - 83, The worker runtime
import verification currently checks only core packages and can miss failures in
the scientific stack. Update the Dockerfile’s import check that runs python to
also import numpy, scipy, and scikit-learn so build or dependency resolution
issues are caught during image निर्माण instead of at worker runtime. Keep the
check in the same verification step used for submission_worker.py so the runtime
dependency set is validated consistently.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@docker/prod/worker_py3_9/Dockerfile`:
- Around line 82-83: The worker runtime import verification currently checks
only core packages and can miss failures in the scientific stack. Update the
Dockerfile’s import check that runs python to also import numpy, scipy, and
scikit-learn so build or dependency resolution issues are caught during image
निर्माण instead of at worker runtime. Keep the check in the same verification
step used for submission_worker.py so the runtime dependency set is validated
consistently.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 2cc8b329-1e6c-4c3f-8827-cd94981f0619

📥 Commits

Reviewing files that changed from the base of the PR and between d5c841f and 25ca52c.

📒 Files selected for processing (3)

• docker/dev/worker_py3_7/Dockerfile
• docker/prod/worker_py3_7/Dockerfile
• docker/prod/worker_py3_9/Dockerfile

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.