diff --git a/docs/PRD.md b/docs/PRD.md index 73dba2a..5c00fd3 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -3,7 +3,7 @@ *Reverse-written from the shipped code, README, and git history (2026-07-24). Every current-state claim is grounded in a same-session read of `Cargo.toml`, `src/`, and `docs/`. The load-bearing engineering decisions live as ADRs -[0001](decisions/0001-lean-core-full-app-split.md)–[0009](decisions/0009-embed-attack-table-drop-forensicnomicon-dep.md) +[0001](decisions/0001-lean-core-full-app-split.md)–[0011](decisions/0011-duckdb-msvc-fmt-floor.md) under [`docs/decisions/`](decisions/). Product tier: blazehash ships a binary an examiner runs (the `blazehash` CLI, plus a `tui` mode and an `mcp` server), so it carries a full PRD per the fleet PRD & ADR standard.* @@ -58,7 +58,8 @@ custody. Grounded in `src/` and `src/cli.rs`: - **Recursive, parallel hashing** with memory-mapped and direct (uncached) I/O - (`src/hash.rs`), across 25+ algorithms — BLAKE3 (default), the SHA-1/2/3, + (`src/hash.rs`), across 25+ algorithms — BLAKE3 (default, + [ADR 0010](decisions/0010-blake3-default-hashdeep-superset.md)), the SHA-1/2/3, MD5, Tiger, Whirlpool, BLAKE2, SM3, Streebog, RIPEMD, K12, checksums (CRC/CRC32C/Adler), fast hashes (xxh3), and fuzzy/similarity hashes (ssdeep, TLSH) — the last two in `blazehash-core::fuzzy`. @@ -75,8 +76,10 @@ Grounded in `src/` and `src/cli.rs`: Shannon entropy, and VirusTotal batch lookup (`src/vt.rs`). - **Forensic-image and device input:** EWF/E01 verification (`src/forensic_image/`, `forensic-image` feature) and raw block-device sizing (`src/device.rs`). -- **Output formats:** the hashdeep text formats plus SQLite, Parquet, DuckDB, - JSON/JSONL, STIX 2.1, and ECS NDJSON (`src/format/`, `src/output.rs`). +- **Output formats:** the hashdeep text formats plus SQLite, Parquet, DuckDB + (`duckdb-output`, bundled engine floored for the MSVC toolchain — + [ADR 0011](decisions/0011-duckdb-msvc-fmt-floor.md)), JSON/JSONL, STIX 2.1, and + ECS NDJSON (`src/format/`, `src/output.rs`). - **Remote storage:** S3/GCS/Azure/60+ backends via opendal, plus FTP/SFTP, and Google Drive hash-without-download (`src/remote/`, `remote` feature — [ADR 0002](decisions/0002-batteries-included-remote-opt-in.md)). @@ -152,3 +155,5 @@ Grounded in `src/` and `src/cli.rs`: | [0007](decisions/0007-dual-release-pipeline.md) | release-plz for the library, `v[0-9]*` tag for the binary | | [0008](decisions/0008-pure-rust-zip-forensic-core.md) | Pure-Rust `zip-forensic-core` reader; dev-only `zip` for fixtures | | [0009](decisions/0009-embed-attack-table-drop-forensicnomicon-dep.md) | Embed the ATT&CK table; no compile-time forensicnomicon dep | +| [0010](decisions/0010-blake3-default-hashdeep-superset.md) | BLAKE3 is the default algorithm; hashdeep flags still select the rest | +| [0011](decisions/0011-duckdb-msvc-fmt-floor.md) | Floor `duckdb-output` at DuckDB 1.5.5 (crate 1.10505.0) for the MSVC 14.51 fmt break | diff --git a/docs/decisions/0010-blake3-default-hashdeep-superset.md b/docs/decisions/0010-blake3-default-hashdeep-superset.md new file mode 100644 index 0000000..f269875 --- /dev/null +++ b/docs/decisions/0010-blake3-default-hashdeep-superset.md @@ -0,0 +1,50 @@ +# 10. BLAKE3 is the default algorithm, while remaining a strict hashdeep superset + +Date: 2026-07-27 +Status: Accepted + +## Context + +blazehash's whole positioning is a *drop-in superset of hashdeep*: every hashdeep +flag and output format works unchanged, so an examiner's existing scripts keep +running (README "What's New vs hashdeep"; PRD "Scope"). hashdeep, however, computes +MD5 + SHA-256 when no algorithm is named — the general-purpose cryptographic hashes +of its era. Defaulting to those makes blazehash's headline advantage (BLAKE3 at +~1,640 MB/s, README "Performance") invisible to the most common invocation, where a +user simply points the tool at a tree and takes the default. + +The tension: a superset is expected to behave like the original, but the original's +default no longer reflects the fastest, most modern primitive the tool ships. Two +things pull in opposite directions — compatibility (keep hashdeep's default) and the +product's reason to exist (lead with BLAKE3). + +## Decision + +1. **The zero-argument default algorithm is BLAKE3.** `-c/--compute` defaults to + `"blake3"` (`src/cli.rs:18`), and the fall-through when no algorithm resolves is + `vec![Algorithm::Blake3]` (`src/cli.rs:488`). A plain `blazehash ` produces + BLAKE3 digests. +2. **Every hashdeep algorithm remains selectable, unchanged.** `-c` takes a + comma-separated list parsed by `parse_algorithms` into the `Algorithm` enum + (`core/src/algorithm.rs`), which includes MD5, the SHA-1/2/3 families, Tiger, + Whirlpool, and the rest — so `blazehash -c md5,sha256` reproduces hashdeep's + default behaviour exactly. Compatibility is preserved by *keeping the flags*, not + by keeping the default. +3. **The algorithm set lives in the lean engine.** `Algorithm`, its parsing, and + dispatch are `blazehash-core` ([ADR 0001](0001-lean-core-full-app-split.md)); the + app selects among them. The default is an application/CLI decision, not baked into + the engine. +4. **Correctness rests on audited upstream crates.** BLAKE3 via the `blake3` crate, + the rest via RustCrypto; blazehash writes no hand-rolled hash math (PRD "Non-Goals"). + +## Consequences + +- The common path showcases the differentiator: the fastest invocation is also the + one a new user runs first, matching the README's "BLAKE3 by default" tagline. +- A user migrating a hashdeep workflow that relied on the *implicit* MD5+SHA-256 + default must name those algorithms explicitly (`-c md5,sha256`). This is the one + visible behavioural divergence from hashdeep, and it is deliberate — the flags + make it a one-token fix, so the superset promise holds for anything that named its + algorithms (the norm in forensic scripting). +- Because the algorithms are all present, changing the default is a low-risk, + reversible CLI choice, not an architectural commitment. diff --git a/docs/decisions/0011-duckdb-msvc-fmt-floor.md b/docs/decisions/0011-duckdb-msvc-fmt-floor.md new file mode 100644 index 0000000..41f2bf4 --- /dev/null +++ b/docs/decisions/0011-duckdb-msvc-fmt-floor.md @@ -0,0 +1,62 @@ +# 11. Floor the `duckdb-output` dependency at DuckDB 1.5.5 (crate 1.10505.0) for the MSVC 14.51 fmt break + +Date: 2026-07-27 +Status: Accepted + +## Context + +The `duckdb-output` feature emits a `.duckdb` manifest via the `duckdb` crate with +the `bundled` feature (`Cargo.toml`; `src/format/duckdb_fmt.rs`), which vendors and +compiles DuckDB's C++ engine at build time rather than linking a system library. +`duckdb-output` is in the `default` feature set, so it is exercised by CI's +`cargo test --all-features` on every platform. + +GitHub's `windows-latest` runner moved to Visual Studio 2026, whose MSVC 14.51 +toolset **removed `stdext::checked_array_iterator`**. The DuckDB engine bundled by +`duckdb` 1.10501.0 (engine 1.5.1) vendors an old `fmt` whose `format.h:326` still +declares `checked_ptr = stdext::checked_array_iterator`. The C++ compile of +`libduckdb-sys` therefore fails on the new MSVC with `C2653` ("'stdext' is not a +class or namespace name") and a cascade of syntax errors, reddening the +`test (windows-latest)` job. macOS/Linux (clang) are unaffected, so the failure is +Windows-MSVC-specific. + +The manifest requirement `^1.4.4` already *allowed* a fixed version, but the +committed `Cargo.lock` (batteries-included repos commit the lock — +[ADR 0002](0002-batteries-included-remote-opt-in.md) and the fleet lock rule) held +CI at the broken 1.10501.0. This is the classic "lock behind the requirement" +staleness layer: a green-looking requirement with a broken resolved version. + +## Decision + +1. **Bump the lock and raise the manifest floor to the first fixed release.** DuckDB + dropped the removed `stdext` usage in PR #23261 / #23239, shipped in engine 1.5.5 + = crate **1.10505.0**. `Cargo.toml` pins `duckdb = { version = "1.10505.0", + features = ["bundled"], optional = true }` with an inline comment recording the + MSVC 14.51 cause, and `Cargo.lock` is updated to match (commit `4233c08`, + branch `fix/windows-ci-duckdb-msvc-fmt`). +2. **Fix the cause, do not suppress the signal.** The build break is repaired by + moving to the fixed dependency — not by skipping the `duckdb-output` test on + Windows, dropping the feature from `--all-features`, or excluding the runner + (fleet "Root-Cause Over Suppression"). The floor is verified against the bundled + sources: 1.10501.0's `format.h` contains the removed symbol at line 326 (matching + the CI error line); 1.10505.0's `format.h` has zero references. +3. **Scope is the app only.** `duckdb-output` is a `blazehash` (app) feature; the + published `blazehash-core` library has no `duckdb` dependency, so this floor does + not touch the engine's MSRV or its downstream compatibility promise + ([ADR 0001](0001-lean-core-full-app-split.md), + [ADR 0006](0006-msrv-floor-vs-pinned-toolchain.md)). + +## Consequences + +- Windows-MSVC CI compiles `libduckdb-sys` again; `cargo test --all-features` is + green across all three platforms. +- The floor is a lower bound, not a ceiling — Renovate/`cargo update` may advance + `duckdb` further; the requirement only forbids regressing below the MSVC-safe + engine. +- The decision is a dependency floor tied to a real, documented upstream toolchain + discontinuity (a removed MSVC extension), so it is a domain fact, not an arbitrary + pin: it may be revisited only if DuckDB's minimum-supported MSVC changes again. +- *State note: the floor bump lands via commit `4233c08` on branch + `fix/windows-ci-duckdb-msvc-fmt`; on `main` at the time of writing `Cargo.toml` + still reads the pre-fix `duckdb = "1.4.4"`. This ADR records the decision; the two + changes converge when both branches merge.*