Skip to content

fix: do not read a four-digit issue number as a publication year - #541

Merged
allaboutduncan merged 3 commits into
allaboutduncan:mainfrom
andreasorbelli:fix/issue-number-read-as-year
Sep 4, 2026
Merged

fix: do not read a four-digit issue number as a publication year#541
allaboutduncan merged 3 commits into
allaboutduncan:mainfrom
andreasorbelli:fix/issue-number-read-as-year

Conversation

@andreasorbelli

@andreasorbelli andreasorbelli commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

📝 Description

Closes #540

Updated 2026-09-03 in response to review. Two corrections from
allaboutduncan's review below:

  1. The issue-number candidate is now discarded after the "exactly one
    year" count, not before it. Discarding first can promote an ambiguous
    filename into a confident-but-wrong one whenever the filename carries a
    second year that isn't the publication year — a scan or reprint year,
    common in exactly the Italian scene releases this PR targets (e.g.
    Topolino 1904 (c2c) (2008).cbz, issue #1904 published 1992, scanned
    2008). That means the "ambiguous → checked" improvement this description
    originally claimed for Topolino 1904 (1992).cbz is no longer part of
    this PR — that file abstains exactly as it always did. What Date check reads a four-digit issue number as a year, rejecting correct matches #540 actually
    needed, Topolino 1904.cbz abstaining instead of rejecting, is
    unaffected. The bullet list and invariant paragraph below are corrected
    inline; the stats table further down was measured against the original
    (before-the-count) implementation and has not been re-run against the
    corrected one, so treat its exact figures as historical.
  2. _is_the_issue_number now compares the issue number's leading integer
    run
    instead of requiring the whole string to be digits, so decimal and
    lettered point issues (1904.1, 1904.MU) are recognised as the issue
    number too — previously they kept the exact bug this PR exists to fix.
    This also closes a second, smaller hole: str.isdigit() is True for
    digit characters int() rejects (e.g. '²', '①'), which the module
    docstring's "pure and never raises" promised these functions wouldn't
    hit; the regex form can't match those characters, so it never reaches
    int() on them.

The date check reads a standalone four-digit number in the 1900–2099 range as a publication year.
Topolino 1904.cbz is issue #1904 of a run that has passed #3,600, so the check compares 1904
against the issue's real date of 1992-05-24, calls the 88-year gap a conflict, and under enforce
drops a match that was correct. The log then blames a date conflict, which points the
investigation the wrong way.

Nothing in the string separates the two cases. Topolino 1904.cbz and Batman 001 (2016).cbz
both put four digits between a non-alphanumeric on each side, which is exactly what _YEAR looks
for. The existing guards already handle everything that can be told apart from the filename
alone — Hal2008 has a letter before, 1920px a letter after, v2004 a letter before. The one
thing distinguishing a year from an issue number here is that the caller has already read those
digits as the issue number, and the parser was never told.

So tell it. issue_year_from_filename and evaluate take an optional issue_number, and a
candidate year equal to it is discarded. The three call sites that run the check —
accept_match and the search-metadata path in routes/metadata.py, and _date_conflicted in
core/bulk_metadata.py — all have the number in scope already. A caller that passes nothing gets
exactly the old behaviour, which is what keeps this safe to land.

The candidate is discarded before the "exactly one year" test rather than after, and that
placement earns its keep in both directions:

Corrected: the candidate is discarded after the "exactly one year" test, not before it —
so it can only ever turn a single confident-but-wrong candidate into abstention, never manufacture
a new conflict out of an ambiguous filename:

  • Topolino 1904.cbz names one candidate, 1904, which is the issue number, so the check abstains
    instead of rejecting. This is the whole fix for Date check reads a four-digit issue number as a year, rejecting correct matches #540.
  • Topolino 1904 (1992).cbz goes from two candidates to one, so a file the check used to skip as
    ambiguous is now checked against the year it actually states.
    No longer true, and
    deliberately given up
    : a filename naming two years stays ambiguous and the check still
    abstains, whether or not one of the two happens to equal the issue number — see the review
    discussion on Topolino 1904 (c2c) (2008).cbz for why promoting it would create a false
    rejection instead.

The change can only turn a conflict into no opinion, or no opinion into a real comparison. It
cannot produce a new false rejection, because the only year it removes is one the caller has
already committed to reading as the issue number, and it is only ever removed once that year is
already the sole candidate.

What it costs. A publication numbered by its year — a yearbook such as L'economia di Zio
Paperone 1992
, issue 1992 published in 1992 — loses the check rather than passing it. Nothing is
written differently for those files, since a skipped check and a passed check both write the
metadata, and the filename cannot tell that case from the broken one. There is a test pinning it,
so it stays a recorded decision rather than a later surprise.

This is not specific to any provider: core/metadata_dates.py is shared, so GCD, ComicVine and
Metron are affected equally. It stays invisible in a library of American comics and is unavoidable
in one holding Topolino, Diabolik, Tex or Dylan Dog.

🛠️ Changes Made

  • Added new feature logic
  • Updated Docker/Config if necessary — nothing to update; no new settings and no new dependencies
  • Verified build locally (docker build -t dev .)

Four files plus tests: the parser and evaluate in core/metadata_dates.py, the two call sites in
routes/metadata.py (now guarded by a wiring regression test), one in core/bulk_metadata.py
(now passing the filename-derived issue number instead of the provider's own copy of it).

🧪 Testing Performed

  • Manual test in dev container
  • Linting/Unit tests pass (see review-response commit for the updated count)

Before and after, in two containers with identical inputs: a local ComicVine SQLite database
holding volume "Topolino" with an issue numbered 1904 dated 1992-05-24, one file
/data/Topolino/Topolino 1904.cbz, a cvinfo naming the volume, and date_check_mode set to
enforce.

On main at 7e0f867 the correct match is thrown away:

Date conflict for Topolino 1904.cbz: filename says 1904, matched issue is dated 1992-05
Rejected ComicVine (Local DB) match for Topolino 1904.cbz: issue date contradicts the filename year 1904
{"processed": 0, "errors": 1, "details": [
  {"file": "Topolino 1904.cbz", "status": "error", "reason": "date conflict", "date_conflict": true}]}

On this branch, same database, same file, same enforce setting:

{"processed": 1, "errors": 0, "details": [
  {"file": "Topolino 1904.cbz", "status": "success", "source": "ComicVine (Local DB)"}]}
update_file_index_from_comicinfo: series=Topolino, number=1904, title=Topolino 1904,
                                  year=1992, publisher=Mondadori, writer=Romano Scarpa

Every pre-existing test calls issue_year_from_filename(name) with no number, so it takes the
default and hits the old path unchanged. Backwards compatibility is demonstrated by the untouched
suite rather than asserted.

The table below predates the review fixes and was measured against the original
before-the-count discard. It has not been re-run against the corrected after-the-count version,
so the exact counts (in particular "gains a year" and "was rejected, now accepted") are no
longer accurate for this branch — the corrected version discards strictly fewer candidates, so
real counts here would be lower. Left in place as a record of the methodology and of the one
real bug it caught (the [c2c Hal 2008] bracket-parsing flip below), which is unaffected by the
review fixes.

Beyond that, the old and new parsers were run side by side over a real 5,369-file library, and
every file whose parse changed was scored against the matched issue's actual publication date:

Files
Parses identically 4,406
Loses a year 49
Gains a year the check could not use before 58
Verdict change Files
Was rejected, now accepted 36
Was accepted, now rejected 1

That single flip is correct, and worth spelling out because it looks like a regression:
Topolino 0330 (Mondadori 1962-03-25) [c2c Hal 2008 & Bibbo64].cbr. extract_issue_number
returns 2008 rather than 330 here, because it strips parentheses but not square brackets, so
the file matches issue #2008 — published 1994 — when it is issue #330 from 1962. Previously the
filename named two years, the check abstained, and the wrong metadata was written silently. Now
2008 is discarded as the issue number, 1962 remains, and the mismatch is caught.

That bracket-parsing detail is a separate bug and this PR does not touch it.


🤖 Generated with Claude Code

https://claude.ai/code/session_01C2zch6Fz6z2WxRnVNCa2XZ
https://claude.ai/code/session_01Q8MJHCSmP9Sx7GARqxLShc

The date check scans a filename for a standalone four-digit number in the
1900-2099 range and treats it as the year the issue was published. "Topolino
1904.cbz" is issue #1904 of a run that has passed allaboutduncan#3,600, so the check compares
1904 against the issue's real date of 1992-05-24, calls an 88-year gap a
conflict, and under `enforce` drops a match that was correct. The log then
blames a date conflict, which points the investigation the wrong way.

Nothing in the string separates the two cases. "Topolino 1904.cbz" and "Batman
001 (2016).cbz" both put four digits between a non-alphanumeric on each side,
which is exactly what _YEAR looks for, and the existing guards already handle
everything that can be told apart from the filename alone -- "Hal2008" has a
letter before, "1920px" a letter after, "v2004" a letter before. The only thing
that distinguishes a year from an issue number here is that the caller has
already read those digits as the issue number, and the parser was never told.

So tell it. issue_year_from_filename and evaluate take an optional issue_number,
and a candidate year equal to it is discarded. The three call sites that run the
check -- accept_match and the search-metadata path in routes/metadata.py, and
_date_conflicted in core/bulk_metadata.py -- all have the number in scope
already. Callers that pass nothing get exactly the old behaviour.

The candidate is discarded before the "exactly one year" test rather than after,
which helps in both directions: "Topolino 1904.cbz" loses its only candidate so
the check abstains instead of rejecting, and "Topolino 1904 (1992).cbz" drops
from two candidates to one, so a file the check used to skip as ambiguous is now
checked against the year it actually states.

This can only turn a conflict into no opinion, or no opinion into a real
comparison. It cannot produce a new false rejection, because the only year it
removes is one the caller has already committed to reading as the issue number.

Measured over a 5,369-file library with the check enforcing. Of the files
carrying an issue number, 4,406 parse identically, 49 lose a year and 58 gain
one the check could not use before. Scored against the matched issue's real
date: 36 were being rejected and are now accepted, 10 lose a check whose verdict
changed nothing about what was written, and one flips from accepted to rejected
-- correctly, catching a file whose issue number is itself mis-parsed.

The cost is a publication numbered by its year: a yearbook such as "L'economia
di Zio Paperone 1992", issue 1992 published in 1992, loses the check rather than
passing it. Nothing is written differently, since a skipped check and a passed
check both write the metadata, and the filename cannot tell that case from the
broken one. There is a test pinning it so it stays a decision.

Closes allaboutduncan#540

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C2zch6Fz6z2WxRnVNCa2XZ
@github-actions github-actions Bot added bug Something isn't working and removed bug Something isn't working labels Sep 2, 2026
@allaboutduncan

Copy link
Copy Markdown
Owner

Reviewed. The diagnosis is right and the approach is the correct one — the caller knowing the issue number really is the only thing that can separate Topolino 1904 from Batman 001 (2016). Two things need changing before this lands.


1. Discarding before the count can create new false rejections

The docstring and the PR body both state the invariant:

The change can only turn a conflict into no opinion, or no opinion into a real comparison. It cannot produce a new false rejection.

That doesn't hold when the filename carries a second year that isn't the publication year — a scan or release year, which is endemic in exactly the Italian scene releases this targets. On this branch:

issue_year_from_filename("Topolino 1904 (c2c) (2008).cbz")          -> None   # main: abstains
issue_year_from_filename("Topolino 1904 (c2c) (2008).cbz", "1904")  -> 2008   # this branch

Issue #1904 is dated 1992-05, so under enforce with the default tolerance of 2 the branch now reports a 16-year conflict and throws away a match main accepted. This is the same shape as the one accepted→rejected flip already in the PR body; that one is attributed to the bracket-parsing bug in extract_issue_number, but the mechanism is general and doesn't depend on that bug.

The cause is the placement. Removing the issue-number candidate before len(plausible) != 1 doesn't only drop a bad candidate — it promotes an ambiguous filename to a confident-but-wrong one, and the filename cannot tell a publication year from a scan year.

Suggested change: apply the discard after the count, so it can only ever produce abstention:

plausible = {year for year in years if 1900 <= year <= upper_bound}

if len(plausible) != 1:
    return None

year = plausible.pop()
return None if _is_the_issue_number(year, issue_number) else year

That still fixes #540 in full — Topolino 1904.cbz has one candidate, it is the issue number, the check abstains. What it gives up is the Topolino 1904 (1992).cbz improvement (ambiguous → checked). That case was never part of the reported bug, and buying it costs a class of false rejection on files where the second year is a scan year, which is the failure mode this PR exists to prevent.

If you'd rather keep the promotion, then the docstring and PR body need the invariant corrected rather than restated — the next reader will otherwise rely on a guarantee that isn't there.


2. .isdigit() misses decimal/suffixed issue numbers, and its stated rationale is wrong

The _is_the_issue_number docstring says:

A number carrying any suffix -- "1904A", "1904.1" -- is not a bare four-digit run of digits and never produced a year in the first place, since _YEAR requires a non-alphanumeric on both sides.

True for 1904A, false for 1904.1. is non-alphanumeric, so _YEAR matches happily. On this branch:

issue_year_from_filename("Topolino 1904.1.cbz",  "1904.1")  -> 1904
issue_year_from_filename("Topolino 1904.MU.cbz", "1904.MU") -> 1904

Point issues keep the exact bug this PR is fixing: the issue number is still read as the year, and the correct match is still rejected under enforce. This repo supports that numbering style deliberately elsewhere (the decimal/suffix handling in the renamer and the issue↔file matcher), so it isn't a hypothetical.

Suggested change: compare the leading integer run instead of requiring the whole string to be digits:

_LEADING_DIGITS = re.compile(r"^(\d+)")

...
    text = str(issue_number).strip()
    match = _LEADING_DIGITS.match(text)
    if not match:
        return False
    return int(match.group(1)) == year

This makes the docstring true as written, and it closes a second small hole for free: str.isdigit() is True for characters outside Nd ('²', '①') where int() raises ValueError, so _is_the_issue_number(1904, "²") currently raises — against the module docstring's promise that these functions "are pure and never raise", now on the batch/single/bulk hot paths where that aborts a run rather than skipping a file. \d doesn't match those characters, so the regex form returns False instead.

Worth a test for 1904.1 / 1904.MU alongside the existing ones.


Two smaller things, not blockers:

  • routes/metadata.py:1777 and :4388 have no regression guard. tests/routes/test_metadata_routes.py::TestDateCheckWiring already pins this wiring with inspect.getsource assertions precisely because the route bodies can't be exercised directly, but nothing asserts either site passes issue_number. A later refactor dropping the third argument restores the bug with a fully green suite.
  • core/bulk_metadata.py:493 passes the provider's issue number, not the filename's. The safety argument rests on the number being one the caller already committed to reading from the filename; _date_conflicted passes issue.issue_number. Equivalent only because matches was selected via issues_by_norm, keyed on the normalized filename number — an invisible coupling. Passing the already-computed issue_text would make the call site say what it means.

Everything else checks out: grep for DATE_MODE_ENFORCE / _date_conflicted confirms all three enforcement sites were updated with no missed caller, and the branch's tests pass as claimed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Q1t9fNhnZEymgFbZ24MkzF

Two blocking issues from review on allaboutduncan#541.

1. Discarding the issue-number candidate before the "exactly one year"
   count could promote an ambiguous filename to a confident-but-wrong
   one, instead of only ever turning a conflict into abstention.
   Italian scene releases routinely carry a second year that is a scan
   or reprint date, not the publication date:

       issue_year_from_filename("Topolino 1904 (c2c) (2008).cbz", "1904")

   used to return 2008 -- issue #1904 is dated 1992-05, so under
   `enforce` that reported a 16-year conflict and threw away a match
   main accepted, the same false-rejection shape the PR exists to fix.
   The discard now happens after the candidate count, so it can only
   ever turn a single confident-but-wrong candidate into abstention;
   an ambiguous filename stays ambiguous whether or not one of its
   years happens to equal the issue number. This gives up the
   "Topolino 1904 (1992).cbz" -> 1992 improvement described in the PR
   body -- that file abstains again, same as before this branch -- but
   allaboutduncan#540 itself (`Topolino 1904.cbz` alone) is unaffected, since a
   single-candidate filename was never at risk.

2. `_is_the_issue_number` required `str.isdigit()` on the whole issue
   number, so a decimal or lettered point issue ("1904.1", "1904.MU")
   was never recognised as the issue number and kept the exact bug
   this PR fixes -- "." is non-alphanumeric, so `_YEAR` matches "1904"
   inside "Topolino 1904.1.cbz" same as it matches "Topolino 1904.cbz".
   It now compares against the issue number's leading integer run
   instead. That also closes a smaller hole the docstring's "pure and
   never raises" promised against: `str.isdigit()` is True for digit
   characters `int()` rejects (superscript '²', circled '①'); a regex
   match on `\d` doesn't match those characters, so it never reaches
   `int()` on them.

Also addresses both non-blocking notes:

- `routes/metadata.py`'s two enforcement sites already passed
  `issue_number` through, but had no regression guard -- app.py can't
  be imported in tests, so nothing pinned it structurally. Added
  `test_both_paths_pass_the_issue_number_to_the_date_check`, alongside
  the existing provider-argument guard.
- `core/bulk_metadata.py:_date_conflicted` took the provider's own
  `issue.issue_number` rather than the filename-derived text the
  caller already has in scope. Equivalent today only because both
  callers select `issue` via `issues_by_norm`, keyed on the filename's
  normalised number -- an invisible coupling. It now takes
  `issue_text` explicitly and both call sites pass it.

Tests: scan-year abstention
(`test_a_second_year_that_is_a_scan_credit_stays_a_conflict_free_abstention`),
the corrected two-candidate invariant
(`test_a_second_candidate_stays_ambiguous_even_once_the_issue_number_is_known`),
point-issue numbers (`test_a_decimal_or_lettered_point_issue_is_still_the_issue_number`),
non-ASCII digits never raising (`test_a_non_ascii_digit_never_raises`),
the routes/metadata.py wiring guard, and bulk_metadata now exercising
`issue_text` against a deliberately-different `issue.issue_number`
(`test_uses_the_filename_issue_number_not_the_providers`).

Full suite: 4,622 passed (excluding 11 pre-existing failures in
tests/unit/test_monitor.py unrelated to this change -- a macOS pytest
tmp-dir path containing a space that a permission check rejects,
reproduced identically on 317873a before this commit).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q8MJHCSmP9Sx7GARqxLShc
@github-actions github-actions Bot added bug Something isn't working and removed bug Something isn't working labels Sep 3, 2026
@andreasorbelli

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review — pushed 1af4801 addressing both blockers and both notes.

1. Discarding before the count

Fixed as suggested: the discard now happens strictly after the "exactly one candidate" count.

plausible = {year for year in years if 1900 <= year <= upper_bound}

if len(plausible) != 1:
    return None

year = plausible.pop()
return None if _is_the_issue_number(year, issue_number) else year

Confirmed against your repro:

issue_year_from_filename("Topolino 1904 (c2c) (2008).cbz")           # -> None
issue_year_from_filename("Topolino 1904 (c2c) (2008).cbz", "1904")   # -> None (was 2008)

I went with correcting the invariant rather than keeping the promotion, per your suggestion — it's the smaller, provable claim, and #540 itself only ever needed the single-candidate case. The "ambiguous → checked" improvement for Topolino 1904 (1992).cbz is gone (it abstains again, matching main); I struck that part of the PR description and added a note at the top explaining the correction rather than rewriting history quietly. Test added:
test_a_second_year_that_is_a_scan_credit_stays_a_conflict_free_abstention, plus
test_a_second_candidate_stays_ambiguous_even_once_the_issue_number_is_known pinning the corrected invariant directly.

2. .isdigit() vs. leading-integer-run

Also fixed as suggested — _is_the_issue_number now matches ^(\d+) and compares that against the year, so it works off the leading integer run instead of requiring the whole string to be digits:

issue_year_from_filename("Topolino 1904.1.cbz",  "1904.1")   # -> None (was 1904)
issue_year_from_filename("Topolino 1904.MU.cbz", "1904.MU")  # -> None (was 1904)

Docstring corrected to match. Added test_a_decimal_or_lettered_point_issue_is_still_the_issue_number for both cases, and test_a_non_ascii_digit_never_raises for the str.isdigit()-accepts-non-ASCII-digits point ('²', '①') — confirmed those never reach int() with the regex form.

Non-blockers

  • Added test_both_paths_pass_the_issue_number_to_the_date_check in TestDateCheckFallthrough (tests/routes/test_metadata_routes.py), asserting via inspect.getsource that both accept_match in batch_metadata and the automatic-match block in search_metadata pass issue_number into evaluate_issue_date, alongside the existing provider-argument guard.
  • core/bulk_metadata.py:_date_conflicted now takes issue_text explicitly instead of reading issue.issue_number, and both call sites pass the filename-derived issue_text they already have in scope. Added test_uses_the_filename_issue_number_not_the_providers, which deliberately sets issue.issue_number and issue_text to different values to prove the call site would misbehave if it regressed to the provider's copy.

Full suite: 4,622 passed. (11 pre-existing failures in tests/unit/test_monitor.py are environmental — a permission error on a macOS pytest tmp-dir path containing a space — reproduced identically on 317873a before this commit, unrelated to this change.)

@github-actions github-actions Bot added bug Something isn't working and removed bug Something isn't working labels Sep 3, 2026
main landed the same issue-number exemption as part of the INDUCKS
provider (allaboutduncan#539), so core/metadata_dates.py conflicted in five hunks with
two independent implementations of one idea. Both discard a sole
candidate year that equals the issue number, and both do it after the
"exactly one candidate" count -- the invariant this branch's review
turned on -- so the conflict is over form, not behaviour.

Resolved to this branch's version throughout, which is a superset:

- _is_the_issue_number compares the issue number's leading integer run
  rather than requiring str.isdigit() on the whole string, so a decimal
  or lettered point issue ("1904.1", "1904.MU") is recognised too.
  main's inline int(str(issue_number).strip()) raises on those and
  swallows it, leaving the exact bug this branch exists to fix -- "."
  is non-alphanumeric, so _YEAR matches "1904" inside "Topolino
  1904.1.cbz" the same as in "Topolino 1904.cbz".
- The docstrings record why the discard has to happen after the count
  and what the fix deliberately gives up (a yearbook numbered by its
  year loses the check).

main's own addition to the file, "inducks" joining
_ISSUE_YEAR_PROVIDERS, is untouched.

routes/metadata.py, core/bulk_metadata.py and the three test files
merged without conflict; main's INDUCKS call sites already pass
issue_number to issue_year_from_filename, and the wiring guard added
here covers the two enforcement sites unchanged.

Tests: 4,726 passed, 7 skipped. The 10 failures in
tests/routes/test_folder_access.py and test_library_access.py reproduce
identically on a clean origin/main worktree at a8fedb9 and are unrelated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q1t9fNhnZEymgFbZ24MkzF
@github-actions github-actions Bot added bug Something isn't working and removed bug Something isn't working labels Sep 4, 2026
@allaboutduncan
allaboutduncan merged commit 5187885 into allaboutduncan:main Sep 4, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Date check reads a four-digit issue number as a year, rejecting correct matches

2 participants