Skip to content

Fix Dataflow ValueProvider serialization - #39614

Open
bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/fix-runtime-value-provider-serialization
Open

Fix Dataflow ValueProvider serialization#39614
bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/fix-runtime-value-provider-serialization

Conversation

@bvolpato

@bvolpato bvolpato commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #39499.

Python Dataflow client migration changed ValueProvider serialization to str(v). Unset runtime providers were consequently sent in sdk_pipeline_options as their object representation, which Flex Template workers treated as the runtime value.

This restores previous JSON behavior:

  • accessible providers serialize their resolved value
  • inaccessible providers serialize as null
  • focused coverage checks both static and unresolved runtime providers
  • CHANGES.md documents the 2.76.0 bug fix

Reproduction

Before the fix, the new test fails with:

AssertionError: 'RuntimeValueProvider(option: at_vp_arg1, type: str, default_value: None)' is not None

The same test passes after restoring value-aware serialization.

Testing

  • uvx tox -e py311-cloud -- apache_beam/runners/dataflow/internal/apiclient_test.py -k test_value_provider_options_serialization
  • target/.tox/py311-cloud/bin/pytest -q apache_beam/runners/dataflow/internal/apiclient_test.py apache_beam/internal/gcp/json_value_test.py (78 passed, 1 skipped)
  • uvx pre-commit run --files CHANGES.md sdks/python/apache_beam/runners/dataflow/internal/apiclient.py sdks/python/apache_beam/runners/dataflow/internal/apiclient_test.py

  • Linked relevant issue.
  • Updated CHANGES.md.
  • Small contribution; ICLA not applicable.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @shunping for label python.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@bvolpato

bvolpato commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

retest this please

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python SDK 2.75 Flex Template: RuntimeValueProvider is being stringified

1 participant