feat: use bstr::BString for shell values to fix UTF-8 lossy conversion

[lu-zero]

Summary

• Convert ShellValue, ShellValueLiteral, and ArrayLiteral from String to bstr::BString so arbitrary bytes are preserved throughout the shell variable system
• Fix silent data corruption in echo -e, mapfile, ANSI-C quoting ($'...'), ${var@E}, PWD/OLDPWD, and From<OsString> for ShellValue

Closes #1115

Details

Bash treats all data (filenames, variable values, command output) as raw byte sequences. Rust's String requires valid UTF-8, so every from_utf8_lossy or to_string_lossy call was a potential silent data corruption point.

The core change replaces String with bstr::BString (backed by Vec<u8>) in ShellValue::String, AssociativeArray, and IndexedArray, plus all related types. Key fixes:

• echo -e '\x80': now writes raw bytes to stdout instead of lossy-converting
• mapfile: stores raw bytes as BString
• ANSI-C quoting ($'\x80'): preserves non-UTF-8 bytes
• ${var@E}: same as above
• PWD/OLDPWD: uses OsStrExt::as_bytes() on Unix to preserve raw path bytes
• From<OsString>: uses into_vec() on Unix instead of to_string_lossy()

24 files changed, all tests passing (49 unit + 224 parser + 25 builtins + 1696 compat).

Co-Authored-by: opencode opencode@anomaly.co
Assisted-by: glm-5.1

[github-actions]

Performance Benchmark Report

Benchmark name	Baseline (μs)	Test/PR (μs)	Delta (μs)	Delta %
clone_shell_object	17.46 μs	17.58 μs	0.12 μs	🟠 +0.70%
eval_arithmetic	0.15 μs	0.15 μs	-0.00 μs	⚪ Unchanged
expand_one_string	1.53 μs	1.62 μs	0.09 μs	🟠 +5.68%
for_loop	30.43 μs	32.55 μs	2.12 μs	🟠 +6.96%
full_peg_complex	56.19 μs	56.65 μs	0.46 μs	🟠 +0.82%
full_peg_for_loop	6.30 μs	6.25 μs	-0.05 μs	⚪ Unchanged
full_peg_nested_expansions	16.64 μs	16.78 μs	0.14 μs	⚪ Unchanged
full_peg_pipeline	4.29 μs	4.21 μs	-0.08 μs	🟢 -1.75%
full_peg_simple	1.83 μs	1.81 μs	-0.02 μs	⚪ Unchanged
function_call	3.31 μs	3.41 μs	0.09 μs	⚪ Unchanged
instantiate_shell	55.57 μs	55.28 μs	-0.28 μs	⚪ Unchanged
instantiate_shell_with_init_scripts	27282.06 μs	28134.31 μs	852.25 μs	🟠 +3.12%
parse_peg_bash_completion	2102.19 μs	2089.46 μs	-12.73 μs	⚪ Unchanged
parse_peg_complex	19.44 μs	18.97 μs	-0.47 μs	🟢 -2.42%
parse_peg_for_loop	1.90 μs	1.96 μs	0.06 μs	🟠 +3.11%
parse_peg_pipeline	1.98 μs	2.05 μs	0.07 μs	🟠 +3.65%
parse_peg_simple	1.08 μs	1.08 μs	0.00 μs	⚪ Unchanged
run_echo_builtin_command	16.98 μs	16.85 μs	-0.13 μs	⚪ Unchanged
tokenize_sample_script	3.56 μs	3.50 μs	-0.06 μs	🟢 -1.60%

Code Coverage Report: Only Changed Files listed

Package	Base Coverage	New Coverage	Difference
brush-builtins/src/declare.rs	🟢 87.4%	🟢 87.47%	🟢 0.07%
brush-builtins/src/export.rs	🟠 73.17%	🟠 74.12%	🟢 0.95%
brush-builtins/src/mapfile.rs	🟢 88.35%	🟢 88.24%	🔴 -0.11%
brush-builtins/src/read.rs	🟢 91.42%	🟢 91.55%	🟢 0.13%
brush-core/src/arithmetic.rs	🟢 98.31%	🟢 98.32%	🟢 0.01%
brush-core/src/completion.rs	🟠 54.35%	🟠 54.5%	🟢 0.15%
brush-core/src/env.rs	🟢 87.16%	🟢 87.23%	🟢 0.07%
brush-core/src/expansion.rs	🟢 95.51%	🟢 95.42%	🔴 -0.09%
brush-core/src/interp.rs	🟢 92.97%	🟢 93.02%	🟢 0.05%
brush-core/src/patterns.rs	🟢 97.62%	🟢 97.32%	🔴 -0.3%
brush-core/src/shell/fs.rs	🟢 97.39%	🟢 97.44%	🟢 0.05%
brush-core/src/shell/io.rs	🟢 83.67%	🟢 84.31%	🟢 0.64%
brush-core/src/shell/readline.rs	🟠 61.9%	🔴 44.83%	🔴 -17.07%
brush-core/src/variables.rs	🟢 91.41%	🟢 91.29%	🔴 -0.12%
brush-core/src/wellknownvars.rs	🟢 83.76%	🟢 84.06%	🟢 0.3%
brush-interactive/src/interactive_shell.rs	🟠 64.1%	🟠 62.86%	🔴 -1.24%
brush-test-harness/src/comparison.rs	🟢 80.77%	🟢 76.92%	🔴 -3.85%
brush-test-harness/src/reporting.rs	🔴 29.93%	🔴 11.62%	🔴 -18.31%
brush-test-harness/src/runner.rs	🟢 84.77%	🟢 84.17%	🔴 -0.6%
brush-test-harness/src/util.rs	🟠 71.64%	🔴 47.76%	🔴 -23.88%
Overall Coverage	🟢 74.72%	🟢 74.42%	🔴 -0.3%

Minimum allowed coverage is 70%, this run produced 74.42%

Test Summary: bash-completion test suite

Outcome	Count	Percentage
✅ Pass	1582	75.01
❗️ Error	18	0.85
❌ Fail	155	7.35
⏩ Skip	339	16.07
❎ Expected Fail	13	0.62
✔️ Unexpected Pass	2	0.09
📊 Total	2109	100.00

[Elsie19]

Suggested change

	s = Vec::new();
	s = Vec::with_capacity(self.args.len());

If you do this you can avoid a couple reallocations.