Skip to content

FlightSQL client only displays last batch from multi-partition query results #346

Description

@matago

Describe the bug

When connecting to a FlightSQL server and executing queries that return multiple
batches (e.g., GROUP BY without ORDER BY), dft only displays the last batch
received instead of concatenating all batches together.

To Reproduce

Tested against two independent FlightSQL server implementations with identical results:

  1. Custom server using arrow-flight 53.x + DataFusion 44
  2. datafusion-flight-sql-server 0.4.7 + DataFusion 46

Both servers return correct results when queried with a Rust FlightSqlServiceClient
(all batches collected via stream.try_collect()).

Steps:

  1. Connect to a FlightSQL server:
    dft --flightsql-host "http://localhost:50051"
    
  2. Run an aggregation query:
    SELECT symbol, AVG(close) as avg_close FROM sample_stocks GROUP BY symbol;
  3. Observe only 1 row is returned
  4. Run the same query multiple times - the single row changes between executions
    (partition completion order is non-deterministic)

Expected behavior

All 3 rows should be displayed (one per group).

Workaround

Adding ORDER BY forces coalescing into a single batch:
SELECT symbol, AVG(close) as avg_close FROM sample_stocks GROUP BY symbol ORDER BY symbol;
This correctly returns all 3 rows.

Environment

  • dft version: 0.2.3
  • Tested against:
    • Custom FlightSQL server (arrow-flight 53, DataFusion 44)
    • datafusion-flight-sql-server 0.4.7 (DataFusion 46)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions