Skip to content

fix: only decode fields used by the data class in Relation.load()#2636

Draft
joelnguemeta wants to merge 1 commit into
canonical:mainfrom
joelnguemeta:fix-relation-load-unit-databag
Draft

fix: only decode fields used by the data class in Relation.load()#2636
joelnguemeta wants to merge 1 commit into
canonical:mainfrom
joelnguemeta:fix-relation-load-unit-databag

Conversation

@joelnguemeta

Copy link
Copy Markdown

Description

Relation.load() decodes every value in the databag before filtering by the class fields. Unit databags always contain keys automatically set by Juju (egress-subnets, ingress-address, private-address) whose values are plain strings, not JSON, so loading unit data into a dataclass or Pydantic model raised json.decoder.JSONDecodeError: Extra data — even when the class doesn't declare those fields.

This change filters by the class fields first and only decodes matching values, so keys that are not part of the data class are ignored without ever reaching the decoder. Round-trip semantics are unchanged: declared fields are still decoded with json.loads (or the custom decoder), so types survive save()/load() round trips.

Out of scope: for classes that are neither dataclasses nor Pydantic models, _juju_fields() cannot determine the fields, so the whole databag is still decoded and passed to __init__ (documented behaviour — such classes were already incompatible with unit databags via the unexpected keyword arguments).

Testing

  • New regression test test_relation_load_unit_data_ignores_juju_keys, parametrized over dataclass, Pydantic dataclass and BaseModel: saves to and loads from the unit databag with the Juju-populated keys present (ops-scenario's defaults). It fails with JSONDecodeError: Extra data before the fix and passes after.
  • Also reproduced the exact charm from the issue (peer relation, save/load in _on_start) via ops-scenario: same traceback as reported before the fix, loads correctly after.
  • test_model_relation_data_class.py (66) and test_model.py (257) all pass; tox -e lint clean.

Fixes #2591

Juju automatically sets keys like 'ingress-address' on unit databags,
with values that are not JSON-encoded. Relation.load() decoded every
databag value before filtering by the class fields, so loading unit
data into a dataclass or Pydantic model raised JSONDecodeError.
Filter by the class fields first, and only decode matching values.

Fixes canonical#2591
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ops.Relation.load() on unit data results in JSONDecodeError: Extra data

1 participant