Skip to content

Parity: escape undecodable subprocess output bytes with backslashreplace semantics #296

Description

@leongdl

Context

openjd-sessions-for-python now decodes subprocess stdout with errors="backslashreplace" so that bytes invalid in the configured encoding are escaped (e.g. b"\x97" becomes the text \x97), preserving the original byte values in session logs. This helps identify the codepage a DCC application is emitting (common on Windows, where applications such as Unreal Engine write cp1252 to stdout). The same convention is used by openjd-adaptor-runtime-for-python (see openjd-adaptor-runtime-for-python PRs #275 and #277).

Current openjd-rs behavior

crates/openjd-sessions/src/subprocess.rs reads raw bytes and decodes each line with String::from_utf8_lossy (line ~700). The reader never crashes on invalid bytes (good), but undecodable bytes are collapsed to U+FFFD, so the original byte values are lost and the two implementations log different text for the same child output.

Ask

Match the Python backslashreplace semantics when decoding subprocess output lines, so v0 (Python) and the Rust engine produce identical session logs for non-UTF-8 output. Test cases worth porting: single invalid byte, consecutive invalid bytes, cp1252 text runs, truncated UTF-8 multi-byte sequences, and valid multi-byte UTF-8 passing through unmodified.

The Python-side fix with the regression and mutation-tested suite is in the linked openjd-sessions-for-python PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions