Repository navigation
fix Pydantic union types disable Pyrefly assignment checks #4131 - #4337
asukaminato0721 wants to merge 1 commit into
Conversation
|
This pull request has been imported. If you are a Meta employee, you can view this in D114011818. (Because this pull request was imported automatically, there will not be any future comments.) |
Merging this PR will not alter performance
Comparing Footnotes
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
4e6ecc2 to
315f203
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Diff from mypy_primer, showing the effect of this PR on open source code: ============================================================
SUMMARY
============================================================
Total: +21 new errors, -0 fixed errors
By preset: +21/-0 (default), +21/-0 (strict)
Projects with changes (3):
openlibrary: +1 -0
freqtrade: +1 -0
pandera: +19 -0
============================================================
FULL DIFF DETAILS
------------------------------------------------------------
openlibrary (https://github.com/internetarchive/openlibrary)
+ ERROR openlibrary/bookworm/tests/test_opds.py:156:15-80: Argument `list[dict[str, str]]` is not assignable to parameter `links` with type `Iterable[Link]` in function `openlibrary.bookworm.opds.Publication.__init__` [bad-argument-type]
freqtrade (https://github.com/freqtrade/freqtrade)
+ ERROR freqtrade/rpc/api_server/api_ws.py:112:49-56: Argument `dict[str, Any]` is not assignable to parameter `data` with type `WSAnalyzedDFMessage.AnalyzedDFData` in function `freqtrade.rpc.api_server.ws_schemas.WSAnalyzedDFMessage.__init__` [bad-argument-type]
pandera (https://github.com/pandera-dev/pandera)
+ ERROR tests/pandas/test_pydantic.py:263:42-54: Argument `type[SimpleSchema]` is not assignable to parameter `pa_schema` with type `SimpleSchema` in function `DataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:271:42-43: Argument `Literal[1]` is not assignable to parameter `pa_schema` with type `SimpleSchema` in function `DataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:274:42-66: Argument `DataFrameSchema` is not assignable to parameter `pa_schema` with type `SimpleSchema` in function `DataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:285:33-46: Argument `type[test_invalid_dataframemodel.InvalidSchema]` is not assignable to parameter `pa_schema` with type `test_invalid_dataframemodel.InvalidSchema` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:300:47-53: Argument `type[test_dataframemodel_inheritance.Parent]` is not assignable to parameter `pa_schema` with type `test_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:301:47-52: Argument `type[test_dataframemodel_inheritance.Child]` is not assignable to parameter `pa_schema` with type `test_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:307:51-59: Argument `type[test_dataframemodel_inheritance.NotChild]` is not assignable to parameter `pa_schema` with type `test_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:324:43-44: Argument `Literal[1]` is not assignable to parameter `pa_schema` with type `DataFrameSchema | None` in function `DataFrameSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:327:39-42: Argument `Literal['1']` is not assignable to parameter `pa_mi` with type `MultiIndex | None` in function `DataFrameSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:345:47-48: Argument `Literal[1]` is not assignable to parameter `pa_series_schema` with type `SeriesSchema | None` in function `SeriesSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:348:40-43: Argument `Literal['1']` is not assignable to parameter `pa_column` with type `Column | None` in function `SeriesSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/pandas/test_pydantic.py:351:39-42: Argument `Literal['1']` is not assignable to parameter `pa_index` with type `Index | None` in function `SeriesSchemaPydantic.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:86:48-66: Argument `type[SimplePolarsSchema]` is not assignable to parameter `pa_schema` with type `SimplePolarsSchema` in function `PolarsDataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:94:48-49: Argument `Literal[1]` is not assignable to parameter `pa_schema` with type `SimplePolarsSchema` in function `PolarsDataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:97:48-78: Argument `DataFrameSchema` is not assignable to parameter `pa_schema` with type `SimplePolarsSchema` in function `PolarsDataFrameModelPydantic.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:112:47-53: Argument `type[test_polars_dataframemodel_inheritance.Parent]` is not assignable to parameter `pa_schema` with type `test_polars_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:113:47-52: Argument `type[test_polars_dataframemodel_inheritance.Child]` is not assignable to parameter `pa_schema` with type `test_polars_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:119:51-59: Argument `type[test_polars_dataframemodel_inheritance.NotChild]` is not assignable to parameter `pa_schema` with type `test_polars_dataframemodel_inheritance.Parent` in function `PydanticModel.__init__` [bad-argument-type]
+ ERROR tests/polars/test_polars_pydantic.py:174:49-50: Argument `Literal[1]` is not assignable to parameter `pa_schema` with type `DataFrameSchema | None` in function `PolarsDataFrameSchemaPydantic.__init__` [bad-argument-type] |
rchen152
left a comment
There was a problem hiding this comment.
Thanks! Left a suggestion to avoid some false positives and another to make the LaxBool handling a little nicer-looking. The false positives definitely need to be addressed before this can go in; feel free to push back on the LaxBool refactor if it doesn't make sense.
| Type::ClassType(_) => self | ||
| .get_atomic_lax_conversion(ty) | ||
| .unwrap_or_else(|| self.heap.mk_any_explicit()), | ||
| .unwrap_or_else(|| ty.clone()), |
There was a problem hiding this comment.
This is now too strict. The mypy primer delta shows some false positives that all seem to be caused by this change:
- openlibrary
test_opds.py:156and freqtradeapi_ws.py:112: each passes a dict, or a list of dicts, to a field typed as a nestedBaseModel(Link,AnalyzedDFData). Pydantic validates the dict into the model. The same applies to stdlib dataclass fields. - pandera, 6 errors (pandas lines 263, 300, 301; polars lines 86, 112, 113): a field typed as
pa.DataFrameModelaccepts the model class itself, via pandera's__get_pydantic_core_schema__. These tests assert that the call succeeds.
Claude suggested this narrower rule:
BaseModeland dataclass types convert toT | Mapping[str, Any].- Classes that define
__get_pydantic_core_schema__stayAny, because they validate input their own way. - Other classes stay as themselves.
| field_name: Name, | ||
| constraints: PydanticRangeConstraints, | ||
| range: Option<PydanticRangeConstraints>, | ||
| lax_bool_field: Option<Type>, |
There was a problem hiding this comment.
Adding a separate field for LaxBool looks sort of clunky. Here's a suggestion to avoid that (again from Claude, so feel free to push back if this doesn't make sense):
keep the separate literal check, but run it per union member and build it on the existing lax conversion.
For most members, "accepts" means
is_subset_eq(literal, lax(member)), using the same conversion that builds the signature. The one refinement is bool: it accepts int literals 0/1 and the listed strings. Put that predicate next to the bool entry in the atomic lax conversion table in pydantic_lax.rs, so the definition of what bool accepts lives in one place.
Summary
Fixes #4131
Custom class union members no longer degrade to Any.
Invalid Pydantic boolean literals are rejected according to Pydantic’s accepted values (https://docs.pydantic.dev/latest/api/standard_library_types/#booleans), while valid alternate union branches remain accepted.
Test Plan
add test, updated the lax-conversion documentation.