Skip to content

Fix missing optimize/fp16/external-data handling in PIR exporter - #1640

Open
982945902 wants to merge 2 commits into
PaddlePaddle:developfrom
982945902:fix/pir-external-data-export
Open

Fix missing optimize/fp16/external-data handling in PIR exporter#1640
982945902 wants to merge 2 commits into
PaddlePaddle:developfrom
982945902:fix/pir-external-data-export

Conversation

@982945902

Copy link
Copy Markdown

Summary

ModelExporter::Run(const PaddlePirParser& ...) currently diverges from the legacy PaddleParser path in a critical way: it builds the ONNX graph and immediately calls SerializeToString(&out) without applying:

  • enable_optimize
  • export_fp16_model
  • big-model external-data handling via SaveExternalData(...)

For large PIR-exported models, that means Paddle2ONNX still tries to serialize a single in-memory ModelProto, which can exceed the protobuf 2GB limit and produce an unusable ONNX export.

This change brings the PIR exporter path back in line with the legacy exporter path by applying the same post-export steps before checker/serialization.

What this patch changes

Inside ModelExporter::Run(const PaddlePirParser& ...), this PR adds:

  1. ONNX optimization when enable_optimize is enabled
  2. FP16 conversion when export_fp16_model is enabled
  3. external-data emission when onnx_model_.ByteSizeLong() > INT_MAX

The logic mirrors the existing implementation already used in ModelExporter::Run(const PaddleParser& ...).

Why this matters

Without this parity, PIR-exported large models can fail even though the codebase already has a supported external-data flow for large ONNX graphs.

In local validation with a real PIR-exported uie-m-large model:

  • before this patch, export hit the >2GB protobuf limit and produced an unusable result
  • after this patch, export produced a small model.onnx plus model.onnx.data
  • the saved model passed path-based onnx.checker.check_model(model_path) validation

Scope

This PR intentionally keeps the change minimal:

  • no API changes
  • no behavior changes for the legacy parser path
  • no new flags or new codepaths beyond restoring parity for the PIR exporter

If helpful, I can follow up with a focused regression test once there is an agreed-upon large-model fixture strategy for CI.

@CLAassistant

CLAassistant commented Apr 15, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@982945902
982945902 force-pushed the fix/pir-external-data-export branch from 3c711be to 3f96025 Compare April 15, 2026 10:30
@982945902
982945902 force-pushed the fix/pir-external-data-export branch from ff540c7 to 4778b7d Compare June 30, 2026 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants