Skip to content

resolve profile penalty during GPU inference (sensevoice) - #1868

Merged
kyakuno merged 1 commit into
masterfrom
mogi/remove_sensevoice_dnn_profile_penalty
Jun 2, 2026
Merged

resolve profile penalty during GPU inference (sensevoice)#1868
kyakuno merged 1 commit into
masterfrom
mogi/remove_sensevoice_dnn_profile_penalty

Conversation

@kzmogi

@kzmogi kzmogi commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

SenseVoice の CUDA 推論が CPU よりも遅い問題に対処する RP です。

SenseVoice の推論スクリプトに --profile オプションを指定しない場合も ailia のインスタンスを作成する際に set_profile_mode() を呼んで同期実行モードで動いている問題がありました。

i7-11700 + NVIDIA RTX PRO 4000 Blackwell の環境で --benchmark -i rd1457.mp3 -e 2 ( rd1457.mp3 ) 引数で推論した場合の消費時間が次のように変化し、評価環境では CPU よりも CUDA(FP32) の方が高速になります。

環境 s2t processing time
CUDA(fp32) + OLD 18473 ms
CUDA(fp32) + this PR 9625 ms
参考 (CPU / -e 0) 13596 ms

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a performance issue in SenseVoice GPU inference where profiling was always enabled, forcing synchronous execution on CUDA.

Changes:

  • Only call set_profile_mode() when profile=True.
  • Use ailia.PROFILE_AVERAGE instead of True for the profile mode argument.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@kzmogi
kzmogi requested a review from kyakuno June 2, 2026 10:31
@kyakuno

kyakuno commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

なるほどです!対応、ありがとうございます。

@kyakuno
kyakuno merged commit 39e6205 into master Jun 2, 2026
1 check passed
@kyakuno
kyakuno deleted the mogi/remove_sensevoice_dnn_profile_penalty branch June 2, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants