Skip to content

validator: make bg-download pending-eval cap configurable - #120

Open
isabella618033 wants to merge 1 commit into
masterfrom
move-pending-eval-cap-to-config
Open

validator: make bg-download pending-eval cap configurable#120
isabella618033 wants to merge 1 commit into
masterfrom
move-pending-eval-cap-to-config

Conversation

@isabella618033

Copy link
Copy Markdown
Contributor

Summary

  • Move DOWNLOAD_PENDING_EVAL_CAP from a module-level constant in connito/validator/background_download_worker.py to ValidatorCheckpointCfg.download_pending_eval_cap in connito/shared/config.py.
  • Default unchanged (10); PositiveInt validation rejects 0/negative values.
  • The worker now reads self.config.ckpt.download_pending_eval_cap once per loop iteration, so an operator can override the cap from validator.yaml (under ckpt:) without rebuilding the image.

Why

Each pending submission in Round.downloaded_pool is ~3.6 GB on disk. With the cap hardcoded at 10, the staging area can grow to ≈ 40 GB before backpressure kicks in. Operators on smaller /data volumes had no way to tune this without forking. Moving it to config matches how miner_submission_max_age_cycles, miner_submission_archive_max_files, and download_concurrency are already exposed on the same CheckpointCfg.

Test plan

  • from connito.shared.config import ValidatorCheckpointCfg — default is 10.
  • Override accepted: ValidatorCheckpointCfg(download_pending_eval_cap=4).download_pending_eval_cap == 4.
  • Invalid values rejected: ValidatorCheckpointCfg(download_pending_eval_cap=0) raises ValidationError (PositiveInt).
  • ast.parse clean on the worker module.
  • Run a validator with ckpt.download_pending_eval_cap: 4 in validator.yaml and confirm the bg-download: pausing — eval backlog above cap cap=4 pending_eval=5 log appears at the new threshold.

Move DOWNLOAD_PENDING_EVAL_CAP from a module-level constant in
background_download_worker.py to ValidatorCheckpointCfg.download_pending_eval_cap
so operators can tune the on-disk staging backlog (each pending
submission is ~3.6 GB) without forking the image. Default unchanged
(10).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant