fix(console): centralize UTF-8 stdio so emoji logs never fail translations - #186
Merged
Merged
Conversation
…tions Providers print status lines with emoji (💬, ✅, ❌,⚠️ ). On Windows cp1252 consoles those raise UnicodeEncodeError, and inside a provider's broad `except` the failing print() was mistaken for a request failure, so the unit's translation failed despite a valid model response. Every entrypoint already worked around this, except the main CLI (translate.py), and with two divergent implementations. Replace them with a single ensure_utf8_stdio() helper (src/utils/console.py): idempotent, prefers TextIOWrapper.reconfigure(), falls back to wrapping the raw buffer, and never raises. Wired into translate.py (the missing one), translation_api.py and benchmark/cli.py (deduplicating the two prior copies). Verified end-to-end on a real cp1252 console: a Poe CLI translation now prints its 💬 token line and completes, where it previously crashed. Tests: 5 unit tests for the helper (reconfigure, idempotency, buffer fallback, reconfigure-raises fallback, missing streams). Full suite green (1129 passed, 1 skipped). Fixes #184 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
losskot
pushed a commit
to losskot/TranslateBooksWithLLMs
that referenced
this pull request
Jul 10, 2026
…tions (hydropix#186) * fix(console): centralize UTF-8 stdio so emoji logs never fail translations Providers print status lines with emoji (💬, ✅, ❌,⚠️ ). On Windows cp1252 consoles those raise UnicodeEncodeError, and inside a provider's broad `except` the failing print() was mistaken for a request failure, so the unit's translation failed despite a valid model response. Every entrypoint already worked around this, except the main CLI (translate.py), and with two divergent implementations. Replace them with a single ensure_utf8_stdio() helper (src/utils/console.py): idempotent, prefers TextIOWrapper.reconfigure(), falls back to wrapping the raw buffer, and never raises. Wired into translate.py (the missing one), translation_api.py and benchmark/cli.py (deduplicating the two prior copies). Verified end-to-end on a real cp1252 console: a Poe CLI translation now prints its 💬 token line and completes, where it previously crashed. Tests: 5 unit tests for the helper (reconfigure, idempotency, buffer fallback, reconfigure-raises fallback, missing streams). Full suite green (1129 passed, 1 skipped). Fixes hydropix#184 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(release): bump version to 1.4.5 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: hydropix <5154373+hydropix@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #184: emoji status lines printed by providers (💬, ✅, ❌,⚠️ ) crash on
Windows cp1252 consoles with
UnicodeEncodeError. Worse, when the failingprint()sits inside a provider's broadexcept, it is mistaken for a requestfailure and the unit's translation fails despite a valid model response.
Every entrypoint already worked around this except the main CLI
(
translate.py), and with two divergent implementations.Changes
src/utils/console.py— newensure_utf8_stdio()helper: idempotent, prefersTextIOWrapper.reconfigure(), falls back to wrapping the raw buffer, neverraises.
translate.py— wire in the helper (the entrypoint that was missing it).translation_api.py,benchmark/cli.py— replace the two prior divergentcopies with the shared helper.
src/__version__.py— bump to 1.4.5 (patch) to cut a release.Tests
tests/unit/test_console_utf8.py— 5 unit tests: reconfigure path,idempotency on already-UTF-8 streams, buffer-wrapping fallback,
reconfigure-raises fallback, and missing streams.
its
💬token line and completes, where it previously crashed.Release / builds
Builds (Windows, macOS, Docker) trigger on a
v*tag. After merge, taggingv1.4.5onmainwill kick off the executable + Docker release.Fixes #184