Skip to content

[BUG] analytics-engine LIST/VALUES aggregates reject DATE/TIMESTAMP/IP/BINARY despite allowing them #22581

Description

@alchemist51

Bug

The PPL LIST and VALUES aggregates reject field types that their own signature lists as allowed. Every call over a DATE/TIMESTAMP/IP/BINARY field returns a self-contradictory 400:

"reason": "Invalid Query",
"details": "Aggregation function LIST expects field type
   {[BYTE]|[SHORT]|[INTEGER]|[LONG]|[FLOAT]|[DOUBLE]|[STRING]|[BOOLEAN]|[DATE]|[TIME]|[TIMESTAMP]|[IP]|[BINARY]},
   but got [TIMESTAMP]",
"type": "ExpressionEvaluationException"

[TIMESTAMP] is in the allowed set, yet it is rejected. Same for [DATE], [IP], [BINARY].

Steps to reproduce

  1. Index a doc with a date field:
    PUT /t/_doc/1?refresh
    { "ts": "2004-07-09 10:17:35" }
    
  2. Run a list aggregate over it via the analytics-engine PPL route:
    POST /_plugins/_ppl
    { "query": "source=t | stats list(ts)" }
    
  3. Response is 400 Invalid Query with the details shown above. Expected: a list of the field's values.

Or via the sandbox IT:

./gradlew ':sandbox:qa:analytics-engine-rest:integTest' \
  --tests 'org.opensearch.analytics.qa.ListAggregateMultiTypeIT.testListBinary'

Failing tests (:sandbox:qa:analytics-engine-rest:integTest)

DatetimeCoverageIT.testListFunctionDateUsesSpace, testListFunctionTimestampUsesSpace; ListAggregateMultiTypeIT.testListBinary, testListDate, testListDateNanos, testListIp; ListAggregateMultiShardIT.testListBinaryTwoShards, testListDateTwoShards, testListIpTwoShards; TwoShardAggregationIT.testReduceCorrectnessAcrossTwoShards; TwoShardOversamplingAggregationIT.testReduceCorrectnessAcrossTwoShards.

Notes

Fails deterministically on main (reproduced on unrelated PRs, e.g. #22553). The VALUES variant lists both [BINARY] and [BINARY,INTEGER] as allowed but still rejects [BINARY], suggesting the operand check compares full type identity rather than the SqlTypeName. Likely area: sandbox/plugins/analytics-engine/src/main/java/org/opensearch/analytics/planner/rules/OpenSearchAggregateSplitRule.java / OpenSearchValuesRule.java.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions