sd cannot search & replace binary bytes in the range 0x80 to 0xFF

[dungsaga]

sd converts search string and replacement string into UTF-8.
Thus, sd cannot search & replace binary bytes in the range 0x80 to 0xFF.

For example:
sd '\xC4' 'xC7' binary-file would not replace byte 0xC4 with byte 0xC7

[oriongonza]

You need to escape the replacement string too

sd '\xC4' '\xC7' binary-file

[dungsaga]

look at my screenshot

[dungsaga]

it also doesn't work with LANG=C when I run LANG=C sd '\xcc' '\xff' aaa

[dungsaga]

I read the source code in src/unescape.rs and realize that sd convert each escaped byte (such as \xdd) into a unicode codepoint, and encoded it as a UTF-8 string. So \xdd becomes \xc3\x9d ("Ý").

sd consider input string as unicode string. Thus, it's not suitable for replacement of raw bytes.

[terefang]

maybe add a -R option so that sd knows to work with raw bytes ?

[dungsaga]

It would be great if sd use a regex library that supports both ASCII mode and Unicode mode. Then we just need a switch parameter to turn if on/off.

…

Message ID: ***@***.***>