Summary
pysbd_segmentation.py exists primarily to isolate and work around PySBD-specific behavior. Since yasbd-lib was designed around immutable boundary detection instead of text mutation followed by span reconstruction, most of that compatibility layer becomes unnecessary.
Why consider replacing it?
The current wrapper handles several PySBD-specific issues:
- Suppressing import
SyntaxWarnings.
- Avoiding
char_span=True because of incorrect offsets for repeated sentences.
- Rebuilding character offsets manually.
- Verifying that segmented text reconstructs the original input.
- Guarding against pathological runtime with
_MAX_SEGMENTER_CHARS.
These workarounds exist because of how PySBD operates internally rather than because of SpanTrack's requirements.
yasbd-lib avoids those design limitations while also providing:
- Support for 39 languages (vs. PySBD's 23).
- Higher published accuracy.
- Significantly faster segmentation.
- Exact boundaries from immutable source text without reconstructing spans afterward.
- Generator-based
detect() and segment() APIs that can operate directly on strings or open file objects, making them suitable for large-document pipelines.
- A PySBD-compatible adapter if API compatibility is preferred during migration, although the native
BoundaryDetector API is recommended for new integrations.
Comparison: https://dev.to/speed_k_7e1b449706e59e433/yasbd-lib-vs-pysbd-two-philosophies-of-sentence-boundary-detection-i88
Proposed change
Rather than swapping the engine behind the existing wrapper, I'd suggest replacing the PySBD-based implementation with the native BoundaryDetector API.
The result would be a simpler sentence-segmentation pipeline with less defensive code dedicated to PySBD-specific edge cases while keeping SpanTrack's goal of exact text-to-audio synchronization intact.
I'd be happy to open a PR if you're interested.
Summary
pysbd_segmentation.pyexists primarily to isolate and work around PySBD-specific behavior. Sinceyasbd-libwas designed around immutable boundary detection instead of text mutation followed by span reconstruction, most of that compatibility layer becomes unnecessary.Why consider replacing it?
The current wrapper handles several PySBD-specific issues:
SyntaxWarnings.char_span=Truebecause of incorrect offsets for repeated sentences._MAX_SEGMENTER_CHARS.These workarounds exist because of how PySBD operates internally rather than because of SpanTrack's requirements.
yasbd-libavoids those design limitations while also providing:detect()andsegment()APIs that can operate directly on strings or open file objects, making them suitable for large-document pipelines.BoundaryDetectorAPI is recommended for new integrations.Comparison: https://dev.to/speed_k_7e1b449706e59e433/yasbd-lib-vs-pysbd-two-philosophies-of-sentence-boundary-detection-i88
Proposed change
Rather than swapping the engine behind the existing wrapper, I'd suggest replacing the PySBD-based implementation with the native
BoundaryDetectorAPI.The result would be a simpler sentence-segmentation pipeline with less defensive code dedicated to PySBD-specific edge cases while keeping SpanTrack's goal of exact text-to-audio synchronization intact.
I'd be happy to open a PR if you're interested.