Repository navigation
Fix SpannerIO PerformanceTests workflows - #40683
Conversation
There was a problem hiding this comment.
📋 Review Summary
This PR fixes type checking errors in the SpannerIO Python performance tests (spannerio_read_perf_test.py and spannerio_write_perf_test.py) by explicitly specifying .with_output_types(_Mutator) on the FlatMap(make_insert_mutations) transform.
🔍 General Feedback
- The changes are clean, minimal, and directly resolve the type inference mismatch between
FlatMapand_MakeMutationGroupsFn(which expectsUnion[MutationGroup, _Mutator]). - Both read and write performance test workflows have been verified with passing runs.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
| | 'Format' >> Map(format_record) | ||
| | 'Make mutations' >> FlatMap(make_insert_mutations) | ||
| | 'Make mutations' >> FlatMap( | ||
| make_insert_mutations).with_output_types(_Mutator) |
There was a problem hiding this comment.
Do we know what change caused this to start failing? Did we make a breaking change to something?
There was a problem hiding this comment.
I found a possible trigger: #40413, merged on October 8, switched Python load/performance tests from 3.10 to 3.11. This may have exposed the existing FlatMap type inference issue. I haven't confirmed it yet, but I didn't find any changes to the SpannerIO API or type checking itself.
There was a problem hiding this comment.
Ok - could we try to run before/after that change and see if it shows the issue? If yes, then I am fine with the change. If no, we should really understand why we need this first
There was a problem hiding this comment.
Confirmed - Python 3.10 works https://github.com/apache/beam/actions/runs/39619250068
|
Assigning reviewers: R: @tvalentyn for label python. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Fixes #40660, fixes #40661
Successful runs:
https://github.com/apache/beam/actions/runs/39405250368
https://github.com/apache/beam/actions/runs/39405225694