Skip to content

Fix CatchAll fields breaking on reuse across structural contexts - #253

Open
sammck wants to merge 1 commit into
rnag:mainfrom
mckelvie-org:fix/catchall-reuse-across-contexts
Open

Fix CatchAll fields breaking on reuse across structural contexts#253
sammck wants to merge 1 commit into
rnag:mainfrom
mckelvie-org:fix/catchall-reuse-across-contexts

Conversation

@sammck

@sammck sammck commented Jul 26, 2026

Copy link
Copy Markdown

Summary

A dataclass with a CatchAll field can currently only be successfully
loaded/dumped in one distinct structural context per process. "Context"
means: used as the direct top-level target of .from_dict()/.to_dict(),
vs. used as a field of enclosing class X, vs. used as a field of enclosing
class Y. Whichever context is hit first works; any second, differently-shaped
use of that same class then fails permanently for the rest of the process
with:

TypeError: issubclass() arg 1 must be a class

raised from _loaders.py::load_dispatcher_for_annotation (and the analogous
spot in _dumpers.py) at if issubclass(origin, t):.

This isn't recursion-specific — plain, non-self-referential classes hit it
too (e.g. calling .from_dict() directly on some class Leaf with a
CatchAll field, then later parsing a different class that has Leaf as a
nested field, breaks the second call). Self-referential classes
(children: list[Self]) are a special case that fails on their very first
use, since resolving them as a top-level target already requires generating
codegen for the same class twice in one pass (once as the entry point, once
as the nested self-reference).

Repro

from dataclasses import dataclass, field
from dataclass_wizard import JSONWizard
from dataclass_wizard.models import CatchAll

@dataclass
class Leaf(JSONWizard):
    name: str
    extra: CatchAll = field(default_factory=dict)

@dataclass
class Wrapper(JSONWizard):
    value: Leaf

Leaf.from_dict({"name": "a", "unrecognized": "x"})                # OK
Wrapper.from_dict({"value": {"name": "b", "unrecognized": "y"}})  # TypeError: issubclass() arg 1 must be a class

Root cause

load_func_for_dataclass() (and dump_func_for_dataclass()) read the
per-class alias mapping via resolve_dataclass_field_to_alias_for_load() /
_for_dump(), which returns a direct reference to a dict cached once per
class in a module-level WeakKeyDictionary — not a copy. The code then did:

catch_all_field: str | None = field_to_aliases.pop(CATCH_ALL, None)

.pop() destructively mutates that shared cached dict. The first time
codegen runs for a class, has_catch_all is correctly True and the
CatchAll field is stripped/special-cased as intended — but the marker is
now permanently gone from the shared cache. If codegen runs again for the
same class in a different structural context (which doesn't hit a
compiled-function cache, since nested-field codegen always calls
load_func_for_dataclass/dump_func_for_dataclass directly rather than
reusing cls.__dataclass_wizard_from_dict__), has_catch_all is now
incorrectly False. The field's declared type, CatchAll = NewType('CatchAll', Mapping),
is a typing.NewType — callable, but not a class — so it falls through to
the normal dispatch path, which calls issubclass(origin, t) on it and
raises.

Fix

Use .get(CATCH_ALL) instead of .pop(CATCH_ALL, None) in both
_loaders.py and _dumpers.py. CATCH_ALL is a sentinel string
('<-|CatchAll|->') that never collides with a real field name, so nothing
downstream relies on the entry being removed from the dict — only on
reading whether it's present. This makes the read side-effect-free, so the
shared per-class cache stays correct no matter how many different
structural contexts the class is later used in.

Test plan

  • Added test_catch_all_reused_across_structural_contexts in
    tests/unit/test_loaders.py, covering a CatchAll-bearing class used
    as a top-level target, then nested in one enclosing class, then nested
    in a second, different enclosing class — all in the same process.
    Confirmed it reproduces the original TypeError against the
    unpatched source and passes with the fix.
  • Full existing suite passes: 885 passed, 2 skipped, 6 xfailed, 4 xpassed,
    0 failed.

load_func_for_dataclass()/dump_func_for_dataclass() popped the CATCH_ALL
marker off of the per-class alias dict returned by
resolve_dataclass_field_to_alias_for_load()/_for_dump(). That dict is a
direct reference into a WeakKeyDictionary cached once per class, not a
copy, so the first codegen pass for a class permanently stripped the
marker from the shared cache. Any later codegen pass for the same class
in a different structural context (e.g. used as a nested field after
being used as a top-level from_dict()/to_dict() target, or nested in a
second, different enclosing class) then saw has_catch_all as False, let
the field's CatchAll (a typing.NewType, not a class) fall through to the
normal dispatch path, and hit issubclass(origin, t) with a non-class
origin, raising TypeError.

Use .get() instead of .pop() so the cached dict is only ever read, never
mutated, fixing reuse in any number of structural contexts including the
self-referential case (which hits two conflicting contexts in a single
top-level call).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant