bandersnatch repeatedly executes synchronously but keeps getting stuck on the same package（this time is mpf）

[89ao]

2023-12-21 20:58:01,011 bandersnatch: INFO Selected storage backend: filesystem
2023-12-21 20:58:01,011 bandersnatch: INFO Selected compare method: hash
2023-12-21 20:58:01,012 bandersnatch: INFO Selected alternative download mirror https://mirrors.tuna.tsinghua.edu.cn/
2023-12-21 20:58:01,055 bandersnatch: INFO Initialized project plugin blocklist_project, filtering ['tensorflow-io-nightly', 'tf-nightly-cpu', 'pyagrum-nightly', 'tf-nightly-gpu', 'uselesscapitalquiz', 'tf-nightly', 'appium']
2023-12-21 20:58:01,058 bandersnatch: INFO Initialized regex release plugin with [re.compile('.+-nightly.*')]
2023-12-21 20:58:01,067 bandersnatch: INFO Initialized release plugin blocklist_release, filtering [<Requirement('tensorflow-io-nightly')>, <Requirement('pyagrum-nightly')>, <Requirement('appium')>, <Requirement('tf-nightly-gpu')>, <Requirement('tf-nightly-cpu')>, <Requirement('tf-nightly')>, <Requirement('uselesscapitalquiz')>]
2023-12-21 20:58:01,076 bandersnatch.mirror: INFO Syncing with https://pypi.org.
2023-12-21 20:58:01,076 bandersnatch.mirror: INFO Current mirror serial: 20459298
2023-12-21 20:58:01,076 bandersnatch.mirror: INFO Resuming interrupted sync from local todo list.
2023-12-21 20:58:01,077 bandersnatch.mirror: INFO Trying to reach serial: 20510755
2023-12-21 20:58:01,077 bandersnatch.mirror: INFO 1 packages to sync.
2023-12-21 20:58:01,077 bandersnatch.mirror: INFO No metadata filters are enabled. Skipping metadata filtering
2023-12-21 20:58:01,077 bandersnatch.mirror: INFO No release file filters are enabled. Skipping release file filtering
2023-12-21 20:58:01,077 bandersnatch.package: INFO Fetching metadata for package: mpf (serial 20485085)
2023-12-21 20:58:01,242 bandersnatch.package: ERROR Stale serial for package mpf - Attempt 1
2023-12-21 20:58:02,243 bandersnatch.package: INFO Fetching metadata for package: mpf (serial 20485085)
2023-12-21 20:58:02,374 bandersnatch.package: ERROR Stale serial for package mpf - Attempt 2
2023-12-21 20:58:04,376 bandersnatch.package: INFO Fetching metadata for package: mpf (serial 20485085)
2023-12-21 20:58:04,505 bandersnatch.package: ERROR Stale serial for package mpf - Attempt 3
2023-12-21 20:58:04,506 bandersnatch.package: ERROR Stale serial for mpf (20485085) not updating. Giving up.
2023-12-21 20:58:04,506 bandersnatch.mirror: ERROR Error syncing package: mpf@20485085
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/bandersnatch/package.py", line 61, in update_metadata
    self._metadata = await master.get_package_metadata(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bandersnatch/master.py", line 220, in get_package_metadata
    metadata_response = await metadata_generator.asend(None)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bandersnatch/master.py", line 138, in get
    await self.check_for_stale_cache(path, required_serial, got_serial)
  File "/usr/local/lib/python3.11/site-packages/bandersnatch/master.py", line 117, in check_for_stale_cache
    raise StalePage(
bandersnatch.master.StalePage: Expected PyPI serial 20485085 for request https://pypi.org/pypi/mpf/json but got 20485084. We can no longer issue a PURGE. Report issue to PyPA Warehouse GitHub if it persists ...

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/bandersnatch/mirror.py", line 129, in package_syncer
    await package.update_metadata(self.master, attempts=3)
  File "/usr/local/lib/python3.11/site-packages/bandersnatch/package.py", line 86, in update_metadata
    raise error_class(package_name=self.name, attempts=attempts)
bandersnatch.errors.StaleMetadata: Stale serial for mpf after 3 attempts
2023-12-21 20:58:04,507 bandersnatch.simple: INFO Generating global index page.

We are using bandersnatch 6.2.0.
As shown in the log, bandersnatch repeatedly initiates synchronization, but each time it fails to synchronize at the mpf file. After the failure, the todo file is not updated, causing the next update to start again from mpf, resulting in an infinite loop.

content of todo file :

[root@VM-54-11-centos /yum/pip]# cat todo
20510755
mpf 20485085

[cooperlees]

This has been a long term CDN problem with warehouse. There is an issue open you could comment on, both for us and warehouse.

bandersnatch.master.StalePage: Expected PyPI serial 20485085 for request https://pypi.org/pypi/mpf/json but got 20485084. We can no longer issue a PURGE. Report issue to PyPA Warehouse GitHub if it persists ...

I would try delete the todo file and bandersnatch sync mpf and then run a long mirror job watching (or enable config to) fail on errors to see if you get an error free run

• bandersnatch mirror --force-check
  (Only do force check argument once)

[89ao]

thks a lot @cooperlees ,cdn issue seems to more likely a warehouse problem.

[89ao]

Can we temporarily skip this file and proceed with other files during the next synchronization? The key point is to ensure that this package doesn't hinder the synchronization of other packages.

[cooperlees]

Yup. Use the blocklist - https://bandersnatch.readthedocs.io/en/latest/filtering_configuration.html#allowlist-blocklist-filtering-settings

P.s. it is Definately a CDN issue, but the main warehouse contributors admin that CDN and I am sure want to know where and how it's failing people, so I urge you to report it.

[89ao]

Yup. Use the blocklist - https://bandersnatch.readthedocs.io/en/latest/filtering_configuration.html#allowlist-blocklist-filtering-settings

P.s. it is Definately a CDN issue, but the main warehouse contributors admin that CDN and I am sure want to know where and how it's failing people, so I urge you to report it.

thanks cooper, Can you please told me which place I can report this issue?

[cooperlees]

pypi/warehouse#4892