perf: use available cores for embedding ONNX session intra-op threads

[ryoma0421]

What

The embedding ONNX session is created with with_intra_threads(1), which bottlenecks CPU-mode execution: on Apple Silicon the full pipeline runs at only ~1-2x realtime (measured with a 5.7-minute meeting recording; embedding inference dominates wall time). This PR lets ORT use up to 6 cores (available_parallelism().min(6), same cap as the segmentation session).

Measurements (M-series, CPU mode, 5.7 min Japanese meeting audio, 4 speakers)

config	speed	diarization output
intra_threads(1), minimal models	1.03x RT	baseline
intra_threads(1), + batched models	2x RT	identical
this patch, + batched models	8-9x RT	identical

Speaker assignment agreement against a labeled ground-truth set is unchanged (96.1% in all three configs, same speaker count).

Happy to adjust the cap or gate it behind a config option if you prefer.

Summary by CodeRabbit

• Performance Improvements
  • Improved embedding processing performance by automatically using available CPU parallelism, capped at six threads.
  • Retained a conservative fallback for environments where parallelism information is unavailable.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: cdb397bb-994d-40b7-8422-1b9d144adb4a

📥 Commits

Reviewing files that changed from the base of the PR and between b0756b1 and e70e3ea.

📒 Files selected for processing (1)

• src/inference/embedding/session.rs

---

📝 Walkthrough

Walkthrough

build_session_with_graph now configures ONNX Runtime intra-op execution with available parallelism capped at six threads. It uses one thread when parallelism detection is unavailable. Inter-op execution remains single-threaded.

Changes

Embedding session threading

Layer / File(s)	Summary
Configure session thread counts src/inference/embedding/session.rs	build_session_with_graph derives the intra-op thread count from available parallelism, caps it at six, and uses one as the fallback. Inter-op threading remains set to one.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main performance change: using available CPU cores for ONNX session intra-operation threads.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Greptile Summary

This PR removes the single-thread bottleneck from embedding inference by sizing the ONNX Runtime intra-op thread pool from available CPU parallelism, capped at six threads with a one-thread fallback.

• Uses std::thread::available_parallelism() to select the embedding session’s intra-op thread count.
• Retains the existing independent thread pool, single inter-op thread, and memory-pattern configuration.
• Aligns embedding inference with the bounded threading strategy already used by segmentation inference.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete correctness, security, or resource-management failure identified.

The new thread count is bounded between one and six, preserves a safe fallback when parallelism cannot be determined, and does not alter inference outputs or session selection behavior.

Important Files Changed

Filename	Overview
src/inference/embedding/session.rs	Replaces the fixed one-thread embedding session configuration with a bounded available-core count; no actionable correctness issue was identified.

_{Reviews (1): Last reviewed commit: "perf: use available cores for embedding ..." | Re-trigger Greptile}