Scope remote OpenDAL backends to evidence-transfer targets (ADR-0012) - #3
Merged
Conversation
…rgets Record the decision to narrow the remote feature's OpenDAL service set from ~50 services to forensically-relevant evidence-transfer targets (object stores, gdrive, WebHDFS/HDFS, SQL, fs/http/webdav + SFTP/FTP), dropping the native DB/KV engines + caches, dev/ML, decentralized, alt-runtime, and regional-drive tail. remote stays opt-in (ADR-0002 unchanged); this narrows which services it compiles. Removes the librocksdb-sys C++ build and the etcd/tikv gRPC stack, which lets the protoc CI step go away. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds unit tests to src/remote/operator.rs asserting that a representative dropped scheme per category (mongodb, redis, rocksdb, ipfs, onedrive, github, compfs, etcd) returns an "unsupported URI scheme" error, per ADR-0010. These FAIL now because the match arms are still present. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s (ADR-0010) Narrow the `remote` feature's OpenDAL service set from ~50 services to the forensically-relevant evidence-transfer targets DFIR collectors read from and write to. `remote` stays opt-in and remote-free-by-default (ADR-0002 unchanged). KEEP (20 services + blocking): object storage (s3, gcs, azblob, azdls, azfile, b2, cos, obs, oss, swift), cloud drive (gdrive), Hadoop (webhdfs, hdfs-native), SQL (mysql, postgresql, sqlite), local/protocol (fs, http, webdav, memory). Non-opendal SFTP (ssh2) and FTP (suppaftp) are unchanged. DROP (37 services + the rocksdb-storage feature): native DB/KV + caches (rocksdb, etcd, tikv, mongodb, gridfs, redis[+native-tls], memcached, sled, redb, persy, moka, mini-moka, dashmap, cacache), dev/ML/CI-cache (github, huggingface, vercel-blob, vercel-artifacts, ghac), big data (alluxio, lakefs, dbfs), decentralized (ipfs, ipmfs), alt-runtime fs (compfs, monoiofs), Cloudflare (cloudflare-kv, d1), consumer drives (onedrive, dropbox, aliyun-drive, yandex-disk, pcloud, koofr, seafile, upyun). - Removes the etcd/tikv gRPC (tonic/prost) chains, so the protoc install step leaves ci.yml (all three OSes + coverage + msrv jobs). - Removes librocksdb-sys (bundled C++), easing Windows-linker memory pressure. - Dropped schemes fall through to the `other =>` arm -> "unsupported URI scheme". - mod.rs still detects the dropped schemes as remote URIs, so they yield a clear unsupported-scheme error rather than being mistaken for local paths; the two dead `rocksdb-storage` cfg refs were removed (the feature no longer exists). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rewrite docs/remote.md to the kept evidence-transfer backends (object stores, gdrive, HDFS/WebHDFS, SQL, fs/http/webdav + SFTP/FTP); drop the sections/tables for the removed native DB/KV, dev/ML, decentralized, alt-runtime, and regional-drive backends. Fix stale '50+/60+ protocols' claims and dropped-backend lists in README, PRD, index, and cli-reference. Replace remote.md's links to the site-excluded decisions/ ADRs with plain-text references (no dangling links). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ision with open PR #2's 0010/0011)
h4x0r
changed the title
…emote-opendal # Conflicts: # .github/workflows/ci.yml # Cargo.lock
…emote-opendal # Conflicts: # docs/PRD.md
This was referenced Aug 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Records ADR-0010 and enacts it: narrows the
remotefeature's OpenDAL service set from ~50 services to the forensically-relevant evidence-transfer targets, dropping the native DB/KV engines + caches, the dev/ML, decentralized, alt-runtime, and regional-drive tail.remotestays opt-in and remote-free-by-default (ADR-0002 unchanged); this narrows which services it compiles.KEEP (20 services +
blocking)Object storage:
s3 gcs azblob azdls azfile b2 cos obs oss swift· Drive:gdrive(only drive with dedicated wiring) · Hadoop:webhdfs hdfs-native· SQL:mysql postgresql sqlite· Local/protocol:fs http webdav memory+ SFTP (ssh2) + FTP (suppaftp).DROP (37 services + the
rocksdb-storagefeature)Native DB/KV + caches (
rocksdb etcd tikv mongodb gridfs redis memcached sled redb persy moka mini-moka dashmap cacache), dev/ML/CI-cache (github huggingface vercel-blob vercel-artifacts ghac), big-data (alluxio lakefs dbfs), decentralized (ipfs ipmfs), alt-runtime FS (compfs monoiofs), Cloudflare (cloudflare-kv d1), regional/consumer drives (onedrive dropbox aliyun-drive yandex-disk pcloud koofr seafile upyun).Why it matters (verified, structural)
librocksdb-sysC++ build (the Windowslink.exe LNK1102OOM culprit) and theetcd/tikvtonic/prostgRPC chains — so theprotocinstall step leaves CI on all three OSes.--features remotebuild: from-scratch 81.7 s / ~1.80 GiB peak RSS; the reduced graph builds without any bundled C++ from the remote stack.TDD
9f55917— 8 tests asserting dropped schemes return "unsupported URI scheme" (7 failed as expected before the change).81eaa49— remove the arms + features (+124 / −2665).0204d14— align README/PRD/index/cli-reference/remote.md to the kept set; no dangling links (mkdocs build --strictclean).Gates (all green locally)
cargo build(default) ·cargo build --features remote·cargo clippy --features remote --all-targets -D warnings·cargo test --features remote(948 passed) ·cargo deny check·cargo fmt --check.Note
refactor(remote)!— this removes supportedremoteURI schemes, so it's a breaking change to that feature's surface. On merge, release-plz will compute the version bump. Merge is the human checkpoint; not merging here.