Skip to content

Condense commits to bot-data repo? #6482

Description

@h-vetinari

In various bots, we're committing individual files to the bot-data repo. This creates a high degree of contention, which causes lots of pushes to fail and have to be retried

Details
pushing file 'version_pr_info/2/b/b/c/9/kubeseal.json' to the graph via the GitHub API
2026-08-12 21:44:34,636 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 16 more times
2026-08-12 21:44:38,886 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 15 more times
2026-08-12 21:44:43,327 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 14 more times
2026-08-12 21:44:48,727 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 13 more times
2026-08-12 21:44:53,956 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 12 more times
2026-08-12 21:45:02,207 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 11 more times
2026-08-12 21:45:11,380 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 10 more times
2026-08-12 21:45:21,499 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 9 more times
2026-08-12 21:45:38,493 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 8 more times
2026-08-12 21:46:04,781 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/2/b/b/c/9/kubeseal.json' - trying 7 more times
pushing file 'version_pr_info/8/a/c/b/c/bioio-conversion.json' to the graph via the GitHub API
2026-08-12 21:46:42,201 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/8/a/c/b/c/bioio-conversion.json' - trying 16 more times
pushing file 'pr_info/9/9/d/b/b/miller.json' to the graph via the GitHub API
2026-08-12 21:46:50,367 WARNING  conda_forge_tick.git_utils || failed to push 'pr_info/9/9/d/b/b/miller.json' - trying 16 more times
2026-08-12 21:46:54,703 WARNING  conda_forge_tick.git_utils || failed to push 'pr_info/9/9/d/b/b/miller.json' - trying 15 more times
2026-08-12 21:46:59,281 WARNING  conda_forge_tick.git_utils || failed to push 'pr_info/9/9/d/b/b/miller.json' - trying 14 more times
2026-08-12 21:47:04,516 WARNING  conda_forge_tick.git_utils || failed to push 'pr_info/9/9/d/b/b/miller.json' - trying 13 more times
2026-08-12 21:47:11,454 WARNING  conda_forge_tick.git_utils || failed to push 'pr_info/9/9/d/b/b/miller.json' - trying 12 more times
pushing file 'version_pr_info/7/2/7/c/4/lazygit.json' to the graph via the GitHub API
2026-08-12 21:47:24,551 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/7/2/7/c/4/lazygit.json' - trying 16 more times
pushing file 'version_pr_info/b/f/4/f/9/functorch.json' to the graph via the GitHub API
2026-08-12 21:47:33,787 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/b/f/4/f/9/functorch.json' - trying 16 more times
pushing file 'version_pr_info/c/1/b/0/a/gz-msgs.json' to the graph via the GitHub API
2026-08-12 21:47:43,107 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/c/1/b/0/a/gz-msgs.json' - trying 16 more times
pushing file 'version_pr_info/d/7/e/b/f/vim.json' to the graph via the GitHub API
2026-08-12 21:47:51,866 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/7/e/b/f/vim.json' - trying 16 more times
2026-08-12 21:47:55,868 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/7/e/b/f/vim.json' - trying 15 more times
2026-08-12 21:47:59,968 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/7/e/b/f/vim.json' - trying 14 more times
2026-08-12 21:48:04,311 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/7/e/b/f/vim.json' - trying 13 more times
2026-08-12 21:48:09,775 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/7/e/b/f/vim.json' - trying 12 more times
pushing file 'version_pr_info/c/e/3/6/d/hatchet-lite.json' to the graph via the GitHub API
2026-08-12 21:48:20,678 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/c/e/3/6/d/hatchet-lite.json' - trying 16 more times
pushing file 'version_pr_info/4/6/5/a/9/jkqtplotter.json' to the graph via the GitHub API
2026-08-12 21:48:30,256 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 16 more times
2026-08-12 21:48:35,243 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 15 more times
2026-08-12 21:48:40,180 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 14 more times
2026-08-12 21:48:45,148 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 13 more times
2026-08-12 21:48:51,459 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 12 more times
2026-08-12 21:48:59,229 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 11 more times
2026-08-12 21:49:07,407 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 10 more times
2026-08-12 21:49:19,879 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 9 more times
2026-08-12 21:49:35,381 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 8 more times
2026-08-12 21:49:52,603 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/4/6/5/a/9/jkqtplotter.json' - trying 7 more times
pushing file 'version_pr_info/5/d/6/e/5/libignition-common.json' to the graph via the GitHub API
2026-08-12 21:50:25,912 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/5/d/6/e/5/libignition-common.json' - trying 16 more times
pushing file 'version_pr_info/3/7/9/5/1/go-gvisor.json' to the graph via the GitHub API
2026-08-12 21:50:34,197 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/3/7/9/5/1/go-gvisor.json' - trying 16 more times
2026-08-12 21:50:38,367 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/3/7/9/5/1/go-gvisor.json' - trying 15 more times
pushing file 'version_pr_info/9/f/5/d/c/nvtx.json' to the graph via the GitHub API
2026-08-12 21:50:47,862 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 16 more times
2026-08-12 21:50:52,338 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 15 more times
2026-08-12 21:50:57,217 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 14 more times
2026-08-12 21:51:03,018 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 13 more times
2026-08-12 21:51:09,615 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 12 more times
2026-08-12 21:51:16,675 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 11 more times
2026-08-12 21:51:27,272 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 10 more times
2026-08-12 21:51:42,025 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 9 more times
2026-08-12 21:51:59,902 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 8 more times
2026-08-12 21:52:18,627 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 7 more times
2026-08-12 21:52:43,817 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/9/f/5/d/c/nvtx.json' - trying 6 more times
pushing file 'version_pr_info/e/3/d/4/a/entire.json' to the graph via the GitHub API
2026-08-12 21:53:48,789 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/e/3/d/4/a/entire.json' - trying 16 more times
pushing file 'version_pr_info/d/e/b/6/5/pyramids.json' to the graph via the GitHub API
2026-08-12 21:53:57,505 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/e/b/6/5/pyramids.json' - trying 16 more times
2026-08-12 21:54:01,259 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/e/b/6/5/pyramids.json' - trying 15 more times
pushing file 'version_pr_info/5/1/7/1/9/pyiron_workflow.json' to the graph via the GitHub API
2026-08-12 21:54:09,925 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/5/1/7/1/9/pyiron_workflow.json' - trying 16 more times
pushing file 'version_pr_info/8/2/7/f/4/hypothesis.json' to the graph via the GitHub API
2026-08-12 21:54:18,118 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/8/2/7/f/4/hypothesis.json' - trying 16 more times
2026-08-12 21:54:22,605 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/8/2/7/f/4/hypothesis.json' - trying 15 more times
2026-08-12 21:54:26,505 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/8/2/7/f/4/hypothesis.json' - trying 14 more times
2026-08-12 21:54:31,296 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/8/2/7/f/4/hypothesis.json' - trying 13 more times
pushing file 'version_pr_info/8/6/0/d/7/paper-qa.json' to the graph via the GitHub API
2026-08-12 21:54:41,994 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/8/6/0/d/7/paper-qa.json' - trying 16 more times
2026-08-12 21:54:46,293 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/8/6/0/d/7/paper-qa.json' - trying 15 more times
2026-08-12 21:54:50,933 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/8/6/0/d/7/paper-qa.json' - trying 14 more times
2026-08-12 21:54:55,417 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/8/6/0/d/7/paper-qa.json' - trying 13 more times
pushing file 'version_pr_info/d/7/d/2/d/pytorch-cpu.json' to the graph via the GitHub API
2026-08-12 21:55:07,216 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/7/d/2/d/pytorch-cpu.json' - trying 16 more times
2026-08-12 21:55:12,177 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/7/d/2/d/pytorch-cpu.json' - trying 15 more times
2026-08-12 21:55:17,853 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/7/d/2/d/pytorch-cpu.json' - trying 14 more times
2026-08-12 21:55:23,689 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/d/7/d/2/d/pytorch-cpu.json' - trying 13 more times
pushing file 'version_pr_info/1/a/5/2/8/mako-rust.json' to the graph via the GitHub API
2026-08-12 21:55:34,617 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/1/a/5/2/8/mako-rust.json' - trying 16 more times
2026-08-12 21:55:39,390 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/1/a/5/2/8/mako-rust.json' - trying 15 more times
pushing file 'version_pr_info/a/c/4/8/d/gz-tools2.json' to the graph via the GitHub API
2026-08-12 21:55:49,199 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/a/c/4/8/d/gz-tools2.json' - trying 16 more times
2026-08-12 21:55:53,411 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/a/c/4/8/d/gz-tools2.json' - trying 15 more times
2026-08-12 21:55:57,500 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/a/c/4/8/d/gz-tools2.json' - trying 14 more times
2026-08-12 21:56:02,363 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/a/c/4/8/d/gz-tools2.json' - trying 13 more times
pushing file 'version_pr_info/c/8/8/f/1/gridded.json' to the graph via the GitHub API
pushing file 'version_pr_info/5/c/5/d/6/sqlglot-mypy.json' to the graph via the GitHub API
2026-08-12 21:56:16,966 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/5/c/5/d/6/sqlglot-mypy.json' - trying 16 more times
2026-08-12 21:56:21,454 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/5/c/5/d/6/sqlglot-mypy.json' - trying 15 more times
2026-08-12 21:56:25,965 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/5/c/5/d/6/sqlglot-mypy.json' - trying 14 more times
2026-08-12 21:56:31,746 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/5/c/5/d/6/sqlglot-mypy.json' - trying 13 more times
2026-08-12 21:56:37,982 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/5/c/5/d/6/sqlglot-mypy.json' - trying 12 more times
2026-08-12 21:56:44,596 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/5/c/5/d/6/sqlglot-mypy.json' - trying 11 more times

Sometimes this runs into a pretty severe cooldown on the side of github, where we lose roughly half an hour of doing absolutely nothing

2026-08-12 21:56:52,832 WARNING  conda_forge_tick.git_utils || failed to push 'version_pr_info/5/c/5/d/6/sqlglot-mypy.json' - trying 10 more times
Request PUT /repos/conda-forge/conda-forge-bot-data/contents/version_pr_info/5/c/5/d/6/sqlglot-mypy.json failed with 403: Forbidden
2026-08-12 21:57:02,973 INFO     github.GithubRetry || Request PUT /repos/conda-forge/conda-forge-bot-data/contents/version_pr_info/5/c/5/d/6/sqlglot-mypy.json failed with 403: Forbidden
Setting next backoff to 1992.026148s
2026-08-12 21:57:02,973 INFO     github.GithubRetry || Setting next backoff to 1992.026148s

My question is: why not commit all the changes that a given bot run produces in one commit? Why one commit per file? The former might have been infeasible once upon a time due to many multi-MB files, but with #6435 fixed (reduction of the size of the migrator files by a factor ~3000!), that should now perhaps be possible?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions