Skip to content

fix(py-hftbacktest): eliminate per-event Vec allocation in market depth fusion - #327

Open
caiyi0616 wants to merge 1 commit into
nkaz001:masterfrom
caiyi0616:fix/issue320-zero-alloc-fusion
Open

fix(py-hftbacktest): eliminate per-event Vec allocation in market depth fusion#327
caiyi0616 wants to merge 1 commit into
nkaz001:masterfrom
caiyi0616:fix/issue320-zero-alloc-fusion

Conversation

@caiyi0616

Copy link
Copy Markdown

Summary

Replace per-call Vec<Event> allocation in FusedHashMapMarketDepth update
methods with append-style *_into variants that write into caller-owned storage.

Motivation

Issue #320: The Python market-depth fusion path creates a temporary
Vec<Event> for each accepted depth or snapshot event, causing
approximately one short-lived allocation per accepted input event.

Performance improvement:

Platform Baseline ns/event Candidate ns/event Improvement
Windows 11 58.100 32.124 44.69%
Debian WSL2 41.911 30.116 28.37%

Allocation reduction (1M events):

  • Median allocation calls: 980,071.5 → 31
  • Cumulative requested bytes: -65.15%

Changes

hftbacktest/src/depth/fuse.rs

Add *_into variants that accept &mut Vec<Event> as output parameter:

  • update_bid_depth_into(&mut self, ev: Event, out: &mut Vec<Event>)
  • update_ask_depth_into(&mut self, ev: Event, out: &mut Vec<Event>)
  • update_best_bid_into(&mut self, ev: Event, out: &mut Vec<Event>)
  • update_best_ask_into(&mut self, ev: Event, out: &mut Vec<Event>)

Existing vector-returning methods are retained as thin wrappers that call *_into.

py-hftbacktest/src/fuse.rs

Replace slf.fused.append(&mut evs) with direct *_into calls.
When add=false, pass a ManuallyDrop<Vec> scratch buffer to still
update depth state without recording fused events—preserving the
existing semantics.

Verification

Closes #320

…th fusion (closes nkaz001#320)

Implements append-style *_into variants in FusedHashMapMarketDepth.
The Python binding now calls these directly, reusing slf.fused
as the output buffer instead of allocating a new Vec per call.

Performance improvement:
  Windows 11:  58.1 ns/event → 32.1 ns/event (-44.7%)
  Debian WSL2: 41.9 ns/event → 30.1 ns/event (-28.4%)

1M-event allocation reduction:
  - Median calls: 980,071 → 31
  - Bytes requested: -65.15%

Changes:
- hftbacktest/src/depth/fuse.rs: Add update_*_into() variants
- py-hftbacktest/src/fuse.rs: Use *_into methods, pass scratch buffer when add=false

Closes nkaz001#320
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid per-event allocation in Python market-depth fusion

1 participant