Skip to content

Offload SDMX response parsing from the event loop #500

Description

@ypldan

StatGPT Backend version

0.12.0 (development)

What is the problem this feature will solve?

The SDMX client parses XML responses synchronously on the event loop. Large data messages take seconds of pure-Python parsing, during which every concurrent request — including SSE heartbeats — is stalled.

What is the proposed feature or solution?

Run the parse in a worker thread via asyncio.to_thread, mirroring the existing offload of the SDMX-to-pandas conversion. Serialize DSD-bearing parses with a lock if the sdmx reader mutates the shared data structure definition.

What alternatives have you considered?

A process pool — unnecessary; the goal is unblocking the loop, not parse parallelism.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions