Skip to content

[Performance] Avoid redundant CTC logits copy during text recognition - #5182

Open
1443858742 wants to merge 1 commit into
PaddlePaddle:developfrom
1443858742:fix/ctc-label-decode-no-copy
Open

[Performance] Avoid redundant CTC logits copy during text recognition#5182
1443858742 wants to merge 1 commit into
PaddlePaddle:developfrom
1443858742:fix/ctc-label-decode-no-copy

Conversation

@1443858742

Copy link
Copy Markdown

Summary

  • replace the unconditional np.array conversion in CTCLabelDecode with
    np.asarray
  • avoid copying NumPy logits already returned by the inference runners
  • preserve conversion support for list and other array-like inputs
  • add focused tests for zero-copy NumPy handling and array-like compatibility

Closes #5181.

Why

The Paddle static and HPI runner contracts return List[np.ndarray]. Calling
np.array(pred[0]) therefore allocates and copies the complete recognition
logits before CTC decoding. On a complex real-world page in our profiling,
19 recognition batches produced 794.45 MiB of cumulative logits.

Across 75 paired observations from 75 real document images:

Placement Images Official mean np.asarray mean Paired mean reduction p95 before p95 after
Local NUMA 41 1.1557 s 0.8484 s 25.47% 1.8380 s 1.3849 s
Cross-NUMA 34 3.1997 s 1.5222 s 31.26% 10.8966 s 2.4293 s

Direct conversion timing for the 794.45 MiB logits was 0.2472 s locally and
1.2089 s across NUMA with np.array, versus about 0.00002 s with
np.asarray.

All 75 paired outputs were identical in line count, character count, title and
full-text hashes, parsed OCR lines JSON, and downstream semantic fields JSON.

Compatibility

np.asarray returns the same object for an existing NumPy array and still
converts list and other array-like inputs. The decoder only reads preds
through argmax and max; it does not mutate the runner output.

Validation

  • python -m pytest -q tests: 5 passed
  • Black 24.4.2: passed
  • Flake8 7.0.0 with the repository's selected rules: passed
  • license header check: passed
  • git diff --check: passed

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


lxy seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants