Soundness: Unsound `str::from_utf8_unchecked` on untrusted compiled binary input causes Undefined Behavior

[Manishearth]

Note

This finding was identified during an agentic unsafe Rust code review performed by Gemini AI, followed by human review and verification.

The Issue

In src/parser/compiled.rs, slices extracted from binary compiled terminfo files (source.names at line 38 and sub-slices of extended.table at line 76) are converted directly to &str using unsafe { str::from_utf8_unchecked(s) }

rust-terminfo/src/parser/compiled.rs

Line 38 in 7bbf236

.map(|s| unsafe { str::from_utf8_unchecked(s) })

rust-terminfo/src/parser/compiled.rs

Line 76 in 7bbf236

.map(|s| unsafe { str::from_utf8_unchecked(s) })

The binary parser (compiled::parse) extracts these byte slices from external binary input using nom combinators (take, take_until) without validating that the bytes conform to valid UTF-8 encoding. The public loading functions Database::from_buffer and Database::from_path are marked safe. If a caller passes a malformed or crafted binary buffer containing non-UTF-8 byte sequences (such as 0x80..=0xFF or truncated multi-byte sequences) in the terminal names or extended capability names table, str::from_utf8_unchecked constructs a &str violating Rust's fundamental type validity invariant. According to the Rust Reference, producing a &str pointing to invalid UTF-8 is immediate Undefined Behavior, even if the resulting string is never inspected or accessed.

Minimal Reproduction (Miri)

fn main() {
    // Malformed compiled terminfo binary data containing invalid UTF-8 (0xFF) in the terminal names table.
    // Structure:
    // Magic: 0x011A ([0x1A, 0x01])
    // name_size: 2 ([0x02, 0x00])
    // bool_count: 0 ([0x00, 0x00])
    // num_count: 0 ([0x00, 0x00])
    // string_count: 0 ([0x00, 0x00])
    // table_size: 0 ([0x00, 0x00])
    // names table (2 bytes): [0xFF, 0x00]
    let malformed_data = [
        0x1A, 0x01, // magic
        0x02, 0x00, // name_size
        0x00, 0x00, // bool_count
        0x00, 0x00, // num_count
        0x00, 0x00, // string_count
        0x00, 0x00, // table_size
        0xFF, 0x00, // names: invalid UTF-8 byte followed by null terminator
    ];

    // Calling the safe public API with untrusted binary data triggers Undefined Behavior
    // due to internal `str::from_utf8_unchecked`.
    let _ = terminfo::Database::from_buffer(malformed_data);
}

error: Undefined Behavior: entering unreachable code
  --> /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/validations.rs:48:23
   |
48 |     let y = unsafe { *bytes.next().unwrap_unchecked() };
   |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Undefined Behavior occurred here
   |
   = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
   = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
   = note: stack backtrace:
           0: core::str::validations::next_code_point::<'_, std::slice::Iter<'_, u8>>
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/validations.rs:48:23: 48:54
           1: <std::str::Chars<'_> as std::iter::Iterator>::next
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/iter.rs:42:18: 42:49
           2: <std::str::CharIndices<'_> as std::iter::Iterator>::next
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/iter.rs:184:15: 184:31
           3: <std::str::pattern::MultiCharEqSearcher<'_, fn(char) -> bool {std::char::methods::<impl char>::is_whitespace}> as std::str::pattern::Searcher<'_>>::next
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/pattern.rs:694:31: 694:39
           4: <std::str::pattern::MultiCharEqSearcher<'_, fn(char) -> bool {std::char::methods::<impl char>::is_whitespace}> as std::str::pattern::Searcher<'_>>::next_reject
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/pattern.rs:265:19: 265:30
           5: <std::str::pattern::CharPredicateSearcher<'_, fn(char) -> bool {std::char::methods::<impl char>::is_whitespace}> as std::str::pattern::Searcher<'_>>::next_reject
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/pattern.rs:789:13: 789:33
           6: core::str::<impl str>::trim_matches::<fn(char) -> bool {std::char::methods::<impl char>::is_whitespace}>
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/mod.rs:2369:31: 2369:52
           7: core::str::<impl str>::trim
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/str/mod.rs:2172:9: 2172:47
           8: terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#2}
               at /usr/local/google/home/manishearth/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/terminfo-0.9.0/src/parser/compiled.rs:39:13: 39:21
           9: std::ops::function::impls::<impl std::ops::FnOnce<(&str,)> for &mut {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#2}}>::call_once
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:310:13: 310:35
           10: std::option::Option::<&str>::map::<&str, &mut {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#2}}>
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:1165:29: 1165:33
           11: <std::iter::Map<std::iter::Map<std::slice::Split<'_, u8, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#0}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#1}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#2}}> as std::iter::Iterator>::next
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/adapters/map.rs:107:9: 107:42
           12: <std::vec::Vec<&str> as std::vec::spec_from_iter_nested::SpecFromIterNested<&str, std::iter::Map<std::iter::Map<std::slice::Split<'_, u8, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#0}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#1}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#2}}>>>::from_iter
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/vec/spec_from_iter_nested.rs:24:32: 24:47
           13: <std::vec::Vec<&str> as std::vec::spec_from_iter::SpecFromIter<&str, std::iter::Map<std::iter::Map<std::slice::Split<'_, u8, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#0}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#1}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#2}}>>>::from_iter
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/vec/spec_from_iter.rs:33:9: 33:48
           14: <std::vec::Vec<&str> as std::iter::FromIterator<&str>>::from_iter::<std::iter::Map<std::iter::Map<std::slice::Split<'_, u8, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#0}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#1}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#2}}>>
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/vec/mod.rs:3895:9: 3895:76
           15: <std::iter::Map<std::iter::Map<std::slice::Split<'_, u8, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#0}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#1}}>, {closure@terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from::{closure#2}}> as std::iter::Iterator>::collect::<std::vec::Vec<&str>>
               at /usr/local/google/home/manishearth/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/traits/iterator.rs:2092:9: 2092:38
           16: terminfo::parser::compiled::<impl std::convert::From<terminfo::parser::compiled::Database<'_>> for terminfo::Database>::from
               at /usr/local/google/home/manishearth/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/terminfo-0.9.0/src/parser/compiled.rs:35:19: 40:24
           17: terminfo::Database::from_buffer::<[u8; 14]>
               at /usr/local/google/home/manishearth/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/terminfo-0.9.0/src/database.rs:230:7: 230:22
           18: main
               at src/bin/repro1.rs:23:13: 23:60

Suggested Fix

Replace str::from_utf8_unchecked with safe standard library conversions (str::from_utf8) and propagate a parsing error (Err(Error::Parse)) if UTF-8 validation fails:

- .map(|s| unsafe { str::from_utf8_unchecked(s) })
+ .map(|s| str::from_utf8(s).expect("invalid utf-8")) // or propagate a nom parsing error

Note

The full audit report below also contains additional minor findings (such as missing safety comments or undocumented FFI assumptions) that are probably worth fixing as well but not the primary goal of this issue. The audit report has not been human-reviewed, it may contain misleading claims.

Full Gemini Codebase Audit Report Appendix

Unsafe Rust Review: terminfo (v0_9)

Overall Safety Assessment

terminfo (v0_9) is a terminal capability database library that parses textual terminfo source descriptions and compiled binary terminfo files, enabling terminal capability querying and formatting.

The crate contains 13 unsafe blocks across three parsing modules (src/parser/util.rs, src/parser/source.rs, and src/parser/compiled.rs). Architecturally, terminfo exposes several safe public loading APIs (Database::from_buffer, Database::from_path, Database::from_env) that accept arbitrary untrusted binary input buffers or disk paths and parse them into capability databases.

Unfortunately, the crate contains critical soundness vulnerabilities. The binary parser (src/parser/compiled.rs) extracts unvalidated byte slices from untrusted binary input buffers and converts them directly to &str using unsafe { str::from_utf8_unchecked(...) }. Passing malformed compiled terminfo data containing non-UTF-8 bytes to the safe public APIs triggers immediate Undefined Behavior. Furthermore, the textual parser relies on implicit bitmask invariants in static lookup tables (util::ASCII) to guarantee ASCII byte ranges before calling from_utf8_unchecked, and makes gratuitous use of get_unchecked indexing on fixed 256-byte arrays. None of the 13 unsafe blocks in the crate have safety comments.

Critical Findings

1. Unsound str::from_utf8_unchecked on untrusted binary parser input causes 🚨 Undefined Behavior (src/parser/compiled.rs:38 and src/parser/compiled.rs:76) 🔴 🚨

• Priority: 🔴 High
• Threat Vector: 🚨 Untrusted Input
• Bug Type: Missing UTF-8 Validation
• Description: Slices extracted from binary compiled terminfo files (source.names and sub-slices of extended.table) are converted to &str using unsafe { str::from_utf8_unchecked(s) }. The binary parser (compiled::parse) extracts these byte slices from external binary input using nom combinators (take, take_until) without validating that the bytes conform to valid UTF-8 encoding.
• Soundness Violation: The public functions Database::from_buffer<T: AsRef<[u8]>> and Database::from_path are marked safe. If a caller passes a malformed or crafted binary buffer containing non-UTF-8 byte sequences (e.g., 0x80..=0xFF or truncated multi-byte sequences) in the terminal names or extended capability names table, str::from_utf8_unchecked constructs a &str violating Rust's fundamental type validity invariant. Creating a &str reference pointing to invalid UTF-8 violates core language rules for UTF-8 slices, causing immediate Undefined Behavior, even if the resulting string is never inspected or accessed.
• Remediation: Replace str::from_utf8_unchecked with safe standard library conversions (str::from_utf8) and propagate a parse error (Err(Error::Parse)) if UTF-8 validation fails.

Fishy Findings

1. Gratuitous unsafe lookup table indexing (src/parser/util.rs:71, 76, 81, ⚠️ 86, 91) 🟡 ⚠️

• Priority: 🟡 Low

• Threat Vector: ⚠️ Accidental Misuse

• Bug Type: Gratuitous Unsafe

• Description: Character classification helper functions (is_ws, is_eol, is_printable_no_pipe, is_printable_no_comma, is_printable_no_control) use unsafe { ASCII.get_unchecked(ch as usize) } to index the static ASCII bitmask table.

• Suspicious Rationale: ASCII is defined as a fixed 256-byte array ([u8; 256]) and ch is a u8 (guaranteeing ch as usize <= 255 < 256). Safe indexing ASCII[ch as usize] is statically proven by the compiler to be within bounds, and LLVM trivially elides any runtime bounds checks. Using get_unchecked introduces unnecessary unsafe code and proof obligations for zero performance benefit.

2. Fragile invariant coupling in textual parsing (src/parser/source.rs:54, ⚠️ 59, 81, 98) 🟡 ⚠️

• Priority: 🟡 Low

• Threat Vector: ⚠️ Accidental Misuse

• Bug Type: Fragile Invariant

• Description: The textual terminfo parser uses take_while(is_printable_*) combinators to extract byte slices and converts them to &str using unsafe { str::from_utf8_unchecked(n) }.

• Suspicious Rationale: This operation is currently sound only because the upper 128 entries (indexes 128..255) of util::ASCII happen to be initialized to NONE (0), causing is_printable_* predicates to return false for all non-ASCII bytes. The correctness of from_utf8_unchecked in source.rs is tightly coupled to this undocumented bitmask property in a separate file (util.rs). If util::ASCII were modified or refactored in the future, source.rs would silently become unsound.

3. Unhandled parser bounds panics on corrupted binary input 🚨 (src/parser/compiled.rs:65, 89) 🟡 🚨

• Priority: 🟡 Low
• Threat Vector: 🚨 Untrusted Input
• Bug Type: Unhandled Panic
• Description: In From<Database<'a>> for crate::Database, the parser slices string tables using offsets (&table[offset as usize..]) and searches for null terminators using .unwrap() (string.iter().position(|&c| c == 0).unwrap()).
• Suspicious Rationale: While panicking on invalid input is safe in Rust (not UB), in a public parsing library designed to load arbitrary files from disk (/usr/share/terminfo, $TERMINFO), panicking on malformed or truncated files rather than returning Err(Error::Parse) creates a Denial of Service (DoS) vulnerability for host applications.

Missing Safety Comments

• src/parser/util.rs:71: Missing // SAFETY: comment before unsafe { ASCII.get_unchecked(ch as usize) & SPACE == SPACE }. 🔴

Proposed proof comment:

```rust

// SAFETY: `ASCII` is a static array of length 256 (`[u8; 256]`). Since `ch` is a `u8`, `ch as usize` is strictly in the range `0..=255 < 256`, satisfying the bounds precondition of `slice::get_unchecked`.

```

• src/parser/util.rs:76: Missing // SAFETY: comment before unsafe { ASCII.get_unchecked(ch as usize) & EOL == EOL }. 🔴

Proposed proof comment:

```rust

// SAFETY: `ASCII` has length 256 (`[u8; 256]`). `ch: u8` cast to `usize` is in `0..=255 < 256`, satisfying the bounds precondition of `get_unchecked`.

```

• src/parser/util.rs:81: Missing // SAFETY: comment before unsafe { ASCII.get_unchecked(ch as usize) & (PRINT | PIPE) == PRINT }. 🔴

Proposed proof comment:

```rust

// SAFETY: `ASCII` has length 256 (`[u8; 256]`). `ch: u8` cast to `usize` is in `0..=255 < 256`, satisfying the bounds precondition of `get_unchecked`.

```

• src/parser/util.rs:86: Missing // SAFETY: comment before unsafe { ASCII.get_unchecked(ch as usize) & (PRINT | COMMA) == PRINT }. 🔴

Proposed proof comment:

```rust

// SAFETY: `ASCII` has length 256 (`[u8; 256]`). `ch: u8` cast to `usize` is in `0..=255 < 256`, satisfying the bounds precondition of `get_unchecked`.

```

• src/parser/util.rs:91: Missing // SAFETY: comment before unsafe { ASCII.get_unchecked(ch as usize) & (PRINT | CONTROL) == PRINT }. 🔴

Proposed proof comment:

```rust

// SAFETY: `ASCII` has length 256 (`[u8; 256]`). `ch: u8` cast to `usize` is in `0..=255 < 256`, satisfying the bounds precondition of `get_unchecked`.

```

• src/parser/util.rs:151: Missing // SAFETY: comment before str::from_utf8_unchecked(&[a, b, c]). 🔴

Proposed proof comment:

```rust

				// SAFETY: `a`, `b`, and `c` have each been verified by `is_digit` (`nom::character::is_digit`) to be ASCII digits in the range `b'0'..=b'9'` (`0x30..=0x39`). Since all three bytes are valid ASCII (`< 128`), the slice `&[a, b, c]` is guaranteed to be valid UTF-8.

```

• src/parser/source.rs:54: Missing // SAFETY: comment before unsafe { str::from_utf8_unchecked(n) }. 🔴

Proposed proof comment:

```rust

// SAFETY: `take_while(is_printable_no_pipe)` returns a slice `n` where every byte `b` satisfies `is_printable_no_pipe(b)`. In `util::ASCII`, all entries for indexes `128..=255` are `NONE` (`0`), so `is_printable_no_pipe` returns `false` for any byte `>= 128`. Therefore, all bytes in `n` are `< 128` (ASCII), making `n` valid UTF-8.

```

• src/parser/source.rs:59: Missing // SAFETY: comment before unsafe { str::from_utf8_unchecked(n) }. 🔴

Proposed proof comment:

```rust

// SAFETY: `take_while(is_printable_no_comma)` returns a slice `n` where every byte `b` satisfies `is_printable_no_comma(b)`. Since `util::ASCII` entries for `128..=255` are `NONE` (`0`), all matched bytes are `< 128` (ASCII), ensuring `n` is valid UTF-8.

```

• src/parser/source.rs:81: Missing // SAFETY: comment before unsafe { str::from_utf8_unchecked(n) }. 🔴

Proposed proof comment:

```rust

// SAFETY: `take_while(is_printable_no_control)` returns a slice `n` where every byte `b` satisfies `is_printable_no_control(b)`. Since `util::ASCII` entries for `128..=255` are `NONE` (`0`), all matched bytes are `< 128` (ASCII), ensuring `n` is valid UTF-8.

```

• src/parser/source.rs:98: Missing // SAFETY: comment before unsafe { str::from_utf8_unchecked(n) }. 🔴

Proposed proof comment:

```rust

// SAFETY: `take_while(is_printable_no_control)` returns a slice `n` where every byte `b` satisfies `is_printable_no_control(b)`. Since `util::ASCII` entries for `128..=255` are `NONE` (`0`), all matched bytes are `< 128` (ASCII), ensuring `n` is valid UTF-8.

```

• src/parser/source.rs:108: Missing // SAFETY: comment before unsafe { str::from_utf8_unchecked(n) }. 🔴

Proposed proof comment:

```rust

		// SAFETY: `take_while(is_digit)` matches only ASCII digits `b'0'..=b'9'` (`< 128`). Since all bytes in `n` are valid ASCII, `n` is guaranteed to be valid UTF-8.

```

• src/parser/compiled.rs:38: Missing // SAFETY: comment before unsafe { str::from_utf8_unchecked(s) }. 🔴 Note: As detailed in Critical Findings, this call is unsound and causes Undefined Behavior when parsing untrusted binary buffers containing non-UTF-8 bytes. No valid safety proof exists. It should be replaced with safe code:

  	.map(|s| str::from_utf8(s).expect("invalid utf-8"))

• src/parser/compiled.rs:76: Missing // SAFETY: comment before unsafe { str::from_utf8_unchecked(s) }. 🔴 Note: As detailed in Critical Findings, this call is unsound and causes Undefined Behavior when parsing untrusted binary buffers containing non-UTF-8 bytes. No valid safety proof exists. It should be replaced with safe code:

  		.map(|s| str::from_utf8(s).expect("invalid utf-8"))