Skip to content

Fix multimask batching: load the batched tail at MULTI_MASK_BATCH_SIZE - #8

Open
attevon-admin wants to merge 1 commit into
avencera:masterfrom
attevon-llc:fix/multimask-batch-size
Open

Fix multimask batching: load the batched tail at MULTI_MASK_BATCH_SIZE#8
attevon-admin wants to merge 1 commit into
avencera:masterfrom
attevon-llc:fix/multimask-batch-size

Conversation

@attevon-admin

@attevon-admin attevon-admin commented Aug 25, 2026

Copy link
Copy Markdown

Closes reported perf issue from #7.

Before: trace shows flushes == chunks (batch size 1, per-chunk fbank).
After: flushes=33 for 1041 chunks on a 36-min AMI meeting; RTTM bit-identical to the batch-1 path. Crash repro: exporting a true batch-64 multimask graph under the expected -b64 name kills the embedding worker ("receiver disconnected") because runtime buffers are sized 32 — reproduced on RTX 3080 Ti.

Summary by CodeRabbit

  • Bug Fixes

    • Fixed multi-mask batch processing to use the correct batch size, improving compatibility with the corresponding runtime buffers and model configuration.
  • Chores

    • Updated project ignore rules for local upstream work files.

The loader requested wespeaker-multimask-tail-b64.onnx (PRIMARY_BATCH_SIZE)
but the exporter writes -b32 (MULTI_MASK_BATCH_SIZE) and every runtime buffer
is sized 32. Consequences: the batched session is silently None, so
multi_mask_batch_size() falls back to 1 and the CUDA pipeline flushes one
chunk at a time (per-chunk fbank + batch-1 predicts, trace: flushes == chunks);
and a genuine batch-64 graph placed under the expected name overruns the
32-sized buffers and kills the embedding worker (receiver disconnected).

Measured after the fix (RTX 3080 Ti, 36-min AMI meeting ES2004a): batching
engages (flushes 33 for 1041 chunks), RTTM bit-identical to the batch-1 path.
@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a6b62605-56d3-4a4d-9780-9ef40be1be18

📥 Commits

Reviewing files that changed from the base of the PR and between b0756b1 and f7d506c.

📒 Files selected for processing (2)
  • .gitignore
  • src/inference/embedding/load/sessions.rs

Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review.


📝 Walkthrough

Walkthrough

The multi-mask ONNX session lookup now uses MULTI_MASK_BATCH_SIZE. The repository also ignores the upstream-work/ directory.

Changes

Multi-mask session lookup

Layer / File(s) Summary
Align multi-mask batch lookup
src/inference/embedding/load/sessions.rs
The multi-mask batched model path uses MULTI_MASK_BATCH_SIZE instead of PRIMARY_BATCH_SIZE.

Repository ignore rule

Layer / File(s) Summary
Ignore upstream work directory
.gitignore
The repository ignores upstream-work/.

Estimated code review effort: 2 (Simple) | ~5 minutes

Merge Risk: ⚪ Minimal · up to f7d50

This is a localized batching change with no actionable merge-blocking risk remaining beyond normal checks and review.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (1 skipped: 1 … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: loading the multimask batched tail with MULTI_MASK_BATCH_SIZE.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown

Greptile Summary

This PR aligns multi-mask session discovery with the runtime’s batch-size-32 capacity, enabling the existing optimized model instead of silently falling back to per-chunk inference.

  • Selects wespeaker-multimask-tail-b32.onnx, matching the exporter, model manifests, and fixed runtime buffers.
  • Preserves batch-1 behavior when the optional batched asset is absent.
  • Ignores the local upstream-work/ directory.

Confidence Score: 5/5

The PR appears safe to merge, with model selection, runtime buffers, and batching behavior consistently aligned at batch size 32.

The changed loader now resolves the only exported and distributed batched multi-mask asset, while its absence still follows the established single-item fallback path.

Important Files Changed

Filename Overview
src/inference/embedding/load/sessions.rs Correctly changes optional multi-mask session discovery from batch 64 to batch 32, matching exported assets and runtime tensor capacities.
.gitignore Adds a repository-local workspace directory to ignored files without affecting runtime behavior.

Reviews (1): Last reviewed commit: "Fix multimask batching: load the batched..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants