Skip to content

Commit fdbf35c

Browse files
authored
Merge pull request #482 from ModernRelay/codex/blob-http-delivery
feat(blob): add bounded HTTP delivery
2 parents 3357913 + 1a61be4 commit fdbf35c

18 files changed

Lines changed: 3270 additions & 91 deletions

File tree

AGENTS.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -268,7 +268,7 @@ omnigraph policy explain --cluster ./company-brain --graph knowledge --actor act
268268
|---|---|---|
269269
| Columnar storage on object store | ✅ Arrow/Lance | URI normalization, S3 env-var plumbing |
270270
| Per-dataset versioning + time travel || `snapshot_at_version`, `entity_at`, snapshot-pinned reads across many tables |
271-
| Blob-v2 cell access | ✅ Blob-v2 descriptors and range readers | `read_blob_at` selects one logical node/edge Blob cell at an exact branch or snapshot without exposing Lance placement. Managed readers are bounded and snapshot-pinned; external values are descriptor-only. Identity ambiguity and malformed state fail closed. See [RFC-033](docs/rfcs/0033-blob-management.md) and [execution internals](docs/dev/execution.md). |
271+
| Blob-v2 cell access | ✅ Blob-v2 descriptors and range readers | `read_blob_at` selects one logical node/edge Blob cell at an exact branch or snapshot without exposing Lance placement. Managed readers are bounded and snapshot-pinned; HTTP GET/explicit HEAD adds ranges, conditionals, and a two-chunk backpressure envelope, while whole-object external values redirect without target-object I/O. Identity ambiguity, ranged external delivery, and malformed state fail closed. See [RFC-033](docs/rfcs/0033-blob-management.md) and [execution internals](docs/dev/execution.md). |
272272
| Stable schema + table identity || Persisted accepted SchemaIR v2 owns one graph identity domain and a shared monotonic no-reuse allocator for nonzero type, property, and table-incarnation IDs. Internal manifest schema v5 introduced identity-keyed registration, version, tombstone, OCC, recovery ownership, and identity-derived node/edge paths; v6 preserves that contract and adds exact non-null `id` fencing. `table_key` is a mutable alias: rename preserves identity/path/history, while drop/re-add mints a new lifetime. This binary serves exactly v6; released v4 graphs rebuild by export/init/load, and abandoned unreleased v7-v19 roots are refused as future formats. |
273273
| Per-dataset branches || **Graph-level** refs are logically atomic through authoritative `__manifest` `BranchContents`; native create/delete crash gaps are classified and reclaimed under a single-writer-process boundary; live names are path-prefix-disjoint; data-table forks are lazy; system branches are filtered |
274274
| Atomic single-dataset commits || **Multi-table publication has three layers:** each participant's Lance effect, one identity-aware `__manifest` CAS for graph visibility, and ordinary recovery-v9 for the gap. Mutation/Load, SchemaApply, BranchMerge, EnsureIndices, and Optimize arm identity-bearing sidecars before their first durable effect. Every pin, effect, registration, rename, tombstone, and output carries stable table/incarnation identity. Writers pre-mint exact Lance transactions where available, confirm complete outcomes before publication, and fail closed on ambiguous or foreign effects. Completed recovery is audited in `_graph_commit_recoveries.lance`. |

Cargo.lock

Lines changed: 25 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

crates/omnigraph-api-types/src/lib.rs

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -371,6 +371,35 @@ pub struct QueryRequest {
371371
pub snapshot: Option<String>,
372372
}
373373

374+
/// Logical graph entity selected by the Blob delivery surface.
375+
///
376+
/// This is intentionally graph vocabulary. The wire contract never exposes a
377+
/// Lance dataset, table key, stable row id, or per-table lane.
378+
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
379+
#[serde(rename_all = "snake_case")]
380+
pub enum BlobEntityKind {
381+
Node,
382+
Edge,
383+
}
384+
385+
/// Query parameters shared by `GET` and `HEAD /graphs/{graph_id}/blob`.
386+
#[derive(Debug, Clone, Serialize, Deserialize, IntoParams)]
387+
#[into_params(parameter_in = Query)]
388+
pub struct BlobReadQuery {
389+
/// Select a logical node or edge cell.
390+
pub entity: BlobEntityKind,
391+
/// Accepted-schema node or edge type name.
392+
pub r#type: String,
393+
/// Logical entity id within the selected type.
394+
pub id: String,
395+
/// Accepted-schema Blob property name.
396+
pub property: String,
397+
/// Branch to read. Mutually exclusive with `snapshot`; defaults to `main`.
398+
pub branch: Option<String>,
399+
/// Immutable graph snapshot id. Mutually exclusive with `branch`.
400+
pub snapshot: Option<String>,
401+
}
402+
374403
#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)]
375404
pub struct ChangeRequest {
376405
/// GQ mutation source containing `insert`, `update`, or `delete` statements.
@@ -700,6 +729,17 @@ pub struct ResourceLimitOutput {
700729
pub actual: u64,
701730
}
702731

732+
/// Normalized half-open range details for an unsatisfiable managed Blob read.
733+
///
734+
/// HTTP also returns `Content-Range: bytes */N`; these fields let SDKs inspect
735+
/// the failure without parsing either that header or the human-readable text.
736+
#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)]
737+
pub struct BlobRangeOutput {
738+
pub start: u64,
739+
pub end: u64,
740+
pub length: u64,
741+
}
742+
703743
/// Structured details for an allowed external Blob source that could not be
704744
/// probed or read. The top-level `code` remains optional so this additive
705745
/// detail can roll out without extending the closed [`ErrorCode`] enum.
@@ -753,6 +793,10 @@ pub struct ErrorOutput {
753793
/// rejected attempt has no durable sidecar and no table effect.
754794
#[serde(skip_serializing_if = "Option::is_none")]
755795
pub resource_limit: Option<ResourceLimitOutput>,
796+
/// Set with HTTP 416 for a valid but unsatisfiable managed Blob byte range.
797+
/// `start..end` is half-open and `length` is the selected Blob length.
798+
#[serde(skip_serializing_if = "Option::is_none")]
799+
pub blob_range: Option<BlobRangeOutput>,
756800
/// Set with HTTP 424 when an external Blob URI passed admission policy but
757801
/// its source could not be probed or read. This optional detail is the
758802
/// rolling-safe machine-readable discriminator; `code` is omitted because

crates/omnigraph-cli/src/helpers.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -455,6 +455,7 @@ pub(crate) fn precondition_failed_cli(
455455
read_set_conflict: None,
456456
key_conflict: None,
457457
resource_limit: None,
458+
blob_range: None,
458459
external_blob_source: None,
459460
recovery_required: None,
460461
precondition_failure: Some(omnigraph_api_types::PreconditionFailureOutput {

crates/omnigraph-server/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ tracing-subscriber = { workspace = true }
3636
tower-http = { workspace = true }
3737
utoipa = { workspace = true }
3838
futures = { workspace = true }
39+
headers = "0.4.1"
3940
sha2 = { workspace = true }
4041
subtle = { workspace = true }
4142
async-trait = { workspace = true }

0 commit comments

Comments
 (0)