[Bug]: Cross-language schema decoding fails for ARRAY<STRING> schema options

[Amar3tto]

What happened?

A Python pipeline fails to decode a schema produced by a Java cross-language transform when the schema contains an ARRAY<STRING> option.

This was observed while exposing DebeziumIO through DebeziumReadSchemaTransformProvider. The output schema includes the following metadata:

Schema.Options.builder()
    .setOption(
        "primaryKeyColumns",
        Schema.FieldType.array(Schema.FieldType.STRING),
        primaryKeyColumns)

When this schema crosses the Java/Python expansion boundary and is consumed by a schema-aware Python transform such as MapToFields, pipeline construction fails with:

ValueError: Failed to decode schema due to an issue with Field proto

The relevant Runner API schema contains:

options {
  name: "primaryKeyColumns"
  type {
    array_type {
      element_type {
        atomic_type: STRING
      }
    }
  }
  value {
    array_value {
      element {
        atomic_value {
          string: "id"
        }
      }
    }
  }
}

Current workaround:
Temporarily omit the primaryKeyColumns option from the Debezium output schema.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Prism Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner