Skip to content

Add support for parallel scoring with --num_workers option - #85

Merged
ftshijt merged 4 commits into
wavlab-speech:mainfrom
RogerTzeng:main
Jul 20, 2026
Merged

Add support for parallel scoring with --num_workers option#85
ftshijt merged 4 commits into
wavlab-speech:mainfrom
RogerTzeng:main

Conversation

@RogerTzeng

Copy link
Copy Markdown
Contributor
  • Introduced --num_workers argument in scorer.py for CPU-based parallel processing.
  • Implemented parallel scoring logic in scorer_shared.py to enhance performance.
  • Added tests to validate functionality of parallel scoring and ensure input order preservation.

- Introduced --num_workers argument in scorer.py for CPU-based parallel processing.
- Implemented parallel scoring logic in scorer_shared.py to enhance performance.
- Added tests to validate functionality of parallel scoring and ensure input order preservation.
@ftshijt

ftshijt commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Thanks for your contribution! There are some issues I would like to point:

  • versa/scorer_shared.py:554: the parallel resume path rewrites the whole JSONL file with _write_jsonl_scores(..., "w").

    This changes --resume from append-safe to rewrite-at-end. If the process is interrupted during the final write, or the write fails, previously completed resume results can be lost or truncated. It also means newly completed parallel work is not checkpointed until all jobs finish.

    I’d suggest either appending safely like the serial path, or writing to a temp file and atomically replacing the output only after a successful full write.

  • --num_workers > 1 with --scoring_mode metric or corpus-only configs is accepted but ignored.

    A warning or parser error would make that clearer, since the README says those modes remain serial.

After the above issues are addressed, please also make sure you pass the CI.

… add support for resume checkpointing in JSONL output
@ftshijt

ftshijt commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Tahnks again for prompting fix. Two minor nits:

  • --num_workers > 1 preserves input order for newly computed results. With --resume, existing rows keep their original positions while new rows are appended in input order, so the complete JSONL file may not follow input-key order. Please narrow the README claim to mention this caveat.
  • The README says corpus/distributional metrics remain serial, but a corpus-only configuration with --num_workers > 1 exits with --num_workers > 1 requires at least one utterance-level metric. Please either ignore --num_workers for corpus-only runs or document this restriction explicitly.

Both are documentation/expectation issues rather than implementation blockers. ALso, please make sure to pass CI so that we can merge it smoothly

@ftshijt

ftshijt commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Thanks for your great contribution. PR merged.

@ftshijt
ftshijt merged commit fa6aaea into wavlab-speech:main Jul 20, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants