Skip to content

Add optional Rust accelerator for bidict - #375

Draft
jab wants to merge 18 commits into
mainfrom
native-extension
Draft

Add optional Rust accelerator for bidict#375
jab wants to merge 18 commits into
mainfrom
native-extension

Conversation

@jab

@jab jab commented May 27, 2026

Copy link
Copy Markdown
Owner

Summary

  • add an optional Rust/PyO3 native helper for bulk bidict build and update paths
  • extend the native path to cover mapping-specialized bulk operations, forceupdate routing, and a no-dup fast path in the Rust ingestion loop
  • add Rust tooling and hooks via nix, plus native-aware test and benchmark CI coverage
  • improve the benchmark PR comment so it reports notable per-benchmark deltas from CI

Benchmark notes

  • local and Linux A/B measurements show clear wins on large bulk initialization and update workloads
  • benchmark CI is now wired to post notable improvements and regressions back to the PR comment

Follow-up before ready for review

  • update CHANGELOG.rst with release-note coverage for the optional native helper and the related CI/tooling additions

jab and others added 13 commits May 26, 2026 19:52
Make the Rust toolchain hermetic via nix and wire an optional PyO3 helper package into the development workflow. Add an initial native fast path for building dict-backed bidict state, along with tests and related ignore/doc updates.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Stop ignoring benchmark_linux_arm64.sh and add it to the repository as a local helper for running the existing cachegrind microbenchmark workflow on this Mac via a Linux container.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tighten the Linux benchmark helper to use a pinned modern Rust image and extend the optional native helper to accelerate rollback-safe bulk updates into existing dict-backed bidicts. Add coverage for the new update path and inverse-preservation behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Allow the optional native helper import to be disabled via BIDICT_DISABLE_NATIVE so benchmarking and debugging can easily force the pure-Python path. Add coverage for the env-var gate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Benchmark report

⚠️ Informational benchmark regression detected.

  • Baseline asset: GHA-linux-cachegrind-x86_64-CPython-3.14-baseline.json
  • Threshold: min:5%
  • Result: one or more benchmarks slowed down by at least min:5%; workflow kept green because PR benchmarking is informational
  • Run: benchmark workflow run

This benchmark check is informational and does not block merging.

Deltas below compare benchmark min time against GHA-linux-cachegrind-x86_64-CPython-3.14-baseline.json for benchmarks with baseline >= 1 ms and absolute delta >= 0.1 ms.

Notable improvements

Benchmark Baseline PR Delta
test_bi_init_from_dict[10000-int] 661.963 ms 55.507 ms -91.61%
test_bi_init_from_dict[10000-str] 693.033 ms 61.633 ms -91.11%
test_bi_init_fail_late_dupval[10000] 663.970 ms 59.886 ms -90.98%
test_bi_init_from_dict[1000-int] 67.781 ms 6.549 ms -90.34%
test_bi_init_from_dict[1000-str] 69.929 ms 7.132 ms -89.80%
test_bi_init_fail_late_dupval[1000] 67.399 ms 7.151 ms -89.39%
test_bi_update_partial_overlap[10000] 467.907 ms 70.205 ms -85.00%
test_bi_update_partial_overlap[1000] 47.581 ms 7.999 ms -83.19%

Notable regressions

Benchmark Baseline PR Delta
test_bi_update_fail_early_dupval[10000] 1.016 ms 8.765 ms +762.35%
test_bi_setitem_replace_existing_key[10000] 2.019 ms 2.292 ms +13.53%
test_bi_forceput_existing_value[10000] 1.909 ms 2.096 ms +9.81%
test_bi_forceput_existing_value[100] 1.719 ms 1.849 ms +7.58%
test_bi_setitem_new_item[10000] 2.002 ms 2.110 ms +5.39%
test_orderedbi_items_equals_with_equal_dict_items[10000] 10.646 ms 10.956 ms +2.91%
test_bi_equals_with_equal_dict[10000] 10.648 ms 10.949 ms +2.83%

jab and others added 5 commits May 27, 2026 14:55
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the non-empty native-update duplicate-value prescan so mapping updates preserve the same duplication-error precedence as the Python path, and update native tests to pin the intended routing and precedence behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant