Skip to content

fix Pydantic union types disable Pyrefly assignment checks #4131 - #4337

Open
asukaminato0721 wants to merge 1 commit into
meta:mainfrom
asukaminato0721:4131
Open

asukaminato0721 wants to merge 1 commit into
meta:mainfrom
asukaminato0721:4131

Conversation

@asukaminato0721

Copy link
Copy Markdown
Collaborator

Summary

Fixes #4131

Custom class union members no longer degrade to Any.

Invalid Pydantic boolean literals are rejected according to Pydantic’s accepted values (https://docs.pydantic.dev/latest/api/standard_library_types/#booleans), while valid alternate union branches remain accepted.

Test Plan

add test, updated the lax-conversion documentation.

@meta-codesync

meta-codesync Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D114011818. (Because this pull request was imported automatically, there will not be any future comments.)

@codspeed

codspeed Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 35 untouched benchmarks
⏩ 9 skipped benchmarks1


Comparing asukaminato0721:4131 (2eb723a) with main (1af3067)

Open in CodSpeed

Footnotes

  1. 9 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown

Diff from mypy_primer, showing the effect of this PR on open source code:

============================================================
SUMMARY
============================================================
Total: +21 new errors, -0 fixed errors
By preset: +21/-0 (default), +21/-0 (strict)

Projects with changes (3):
  openlibrary: +1 -0
  freqtrade: +1 -0
  pandera: +19 -0
============================================================

FULL DIFF DETAILS
------------------------------------------------------------

openlibrary (https://github.com/internetarchive/openlibrary)
+ ERROR openlibrary/bookworm/tests/test_opds.py:156:15-80: Argument `list[dict[str, str]]` is not assignable to parameter `links` with type `Iterable[Link]` in function `openlibrary.bookworm.opds.Publication.__init__` [bad-argument-type]

freqtrade (https://github.com/freqtrade/freqtrade)
+ ERROR freqtrade/rpc/api_server/api_ws.py:112:49-56: Argument `dict[str, Any]` is not assignable to parameter `data` with type `WSAnalyzedDFMessage.AnalyzedDFData` in function `freqtrade.rpc.api_server.ws_schemas.WSAnalyzedDFMessage.__init__` [bad-argument-type]

pandera (https://github.com/pandera-dev/pandera)
+ ERROR tests/pandas/test_pydantic.py:263:42-54: Argument `type[SimpleSchema]` is not assignable to parameter `pa_schema` with type `SimpleSchema` in function `DataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:271:42-43: Argument `Literal[1]` is not assignable to parameter `pa_schema` with type `SimpleSchema` in function `DataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:274:42-66: Argument `DataFrameSchema` is not assignable to parameter `pa_schema` with type `SimpleSchema` in function `DataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:285:33-46: Argument `type[test_invalid_dataframemodel.InvalidSchema]` is not assignable to parameter `pa_schema` with type `test_invalid_dataframemodel.InvalidSchema` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:300:47-53: Argument `type[test_dataframemodel_inheritance.Parent]` is not assignable to parameter `pa_schema` with type `test_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:301:47-52: Argument `type[test_dataframemodel_inheritance.Child]` is not assignable to parameter `pa_schema` with type `test_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:307:51-59: Argument `type[test_dataframemodel_inheritance.NotChild]` is not assignable to parameter `pa_schema` with type `test_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:324:43-44: Argument `Literal[1]` is not assignable to parameter `pa_schema` with type `DataFrameSchema | None` in function `DataFrameSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:327:39-42: Argument `Literal['1']` is not assignable to parameter `pa_mi` with type `MultiIndex | None` in function `DataFrameSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:345:47-48: Argument `Literal[1]` is not assignable to parameter `pa_series_schema` with type `SeriesSchema | None` in function `SeriesSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:348:40-43: Argument `Literal['1']` is not assignable to parameter `pa_column` with type `Column | None` in function `SeriesSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:351:39-42: Argument `Literal['1']` is not assignable to parameter `pa_index` with type `Index | None` in function `SeriesSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:86:48-66: Argument `type[SimplePolarsSchema]` is not assignable to parameter `pa_schema` with type `SimplePolarsSchema` in function `PolarsDataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:94:48-49: Argument `Literal[1]` is not assignable to parameter `pa_schema` with type `SimplePolarsSchema` in function `PolarsDataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:97:48-78: Argument `DataFrameSchema` is not assignable to parameter `pa_schema` with type `SimplePolarsSchema` in function `PolarsDataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:112:47-53: Argument `type[test_polars_dataframemodel_inheritance.Parent]` is not assignable to parameter `pa_schema` with type `test_polars_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:113:47-52: Argument `type[test_polars_dataframemodel_inheritance.Child]` is not assignable to parameter `pa_schema` with type `test_polars_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:119:51-59: Argument `type[test_polars_dataframemodel_inheritance.NotChild]` is not assignable to parameter `pa_schema` with type `test_polars_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:174:49-50: Argument `Literal[1]` is not assignable to parameter `pa_schema` with type `DataFrameSchema | None` in function `PolarsDataFrameSchemaPydantic.__init__` [bad-argument-type]

@rchen152 rchen152 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Left a suggestion to avoid some false positives and another to make the LaxBool handling a little nicer-looking. The false positives definitely need to be addressed before this can go in; feel free to push back on the LaxBool refactor if it doesn't make sense.

Type::ClassType(_) => self
.get_atomic_lax_conversion(ty)
.unwrap_or_else(|| self.heap.mk_any_explicit()),
.unwrap_or_else(|| ty.clone()),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now too strict. The mypy primer delta shows some false positives that all seem to be caused by this change:

  • openlibrary test_opds.py:156 and freqtrade api_ws.py:112: each passes a dict, or a list of dicts, to a field typed as a nested BaseModel (Link, AnalyzedDFData). Pydantic validates the dict into the model. The same applies to stdlib dataclass fields.
  • pandera, 6 errors (pandas lines 263, 300, 301; polars lines 86, 112, 113): a field typed as pa.DataFrameModel accepts the model class itself, via pandera's __get_pydantic_core_schema__. These tests assert that the call succeeds.

Claude suggested this narrower rule:

  • BaseModel and dataclass types convert to T | Mapping[str, Any].
  • Classes that define __get_pydantic_core_schema__ stay Any, because they validate input their own way.
  • Other classes stay as themselves.

field_name: Name,
constraints: PydanticRangeConstraints,
range: Option<PydanticRangeConstraints>,
lax_bool_field: Option<Type>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a separate field for LaxBool looks sort of clunky. Here's a suggestion to avoid that (again from Claude, so feel free to push back if this doesn't make sense):

keep the separate literal check, but run it per union member and build it on the existing lax conversion.

For most members, "accepts" means is_subset_eq(literal, lax(member)), using the same conversion that builds the signature. The one refinement is bool: it accepts int literals 0/1 and the listed strings. Put that predicate next to the bool entry in the atomic lax conversion table in pydantic_lax.rs, so the definition of what bool accepts lives in one place.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pydantic union types disable Pyrefly assignment checks

2 participants