feat(sidecar): version and publish the .rrdata format (v1) - #1423
Draft
magnusrodseth wants to merge 2 commits into
Draft
feat(sidecar): version and publish the .rrdata format (v1)#1423magnusrodseth wants to merge 2 commits into
magnusrodseth wants to merge 2 commits into
Conversation
`load_sidecar` returns `ImageMetadata::default()` on any parse failure, and
sidecar writes are read-modify-write on top of it:
let mut metadata = load_sidecar(&sidecar_path); // -> defaults
metadata.adjustments = final_adjustments;
fs::write(&sidecar_path, to_string_pretty(&metadata)?)?;
So a sidecar that fails to parse for any reason, a truncated write, a
half-synced file, a hand-edited one, does not merely fail to load. The next
save writes the defaults back over it, and the image's rating, tags and
cached EXIF are gone with no warning. `save_metadata_and_update_thumbnail`
and `set_color_label_for_paths` both take this path.
An unreadable sidecar is now copied to `<name>.rrdata.bak` before anything
can overwrite it, with a warning naming both paths. The app still continues
with defaults, so one bad file cannot make an image uneditable, and the
behaviour for every sidecar that parses is unchanged.
Quarantine never destroys data. A backup whose contents differ is never
overwritten; the next free slot is used instead. An identical backup is a
no-op, so the repeated loads performed by the thumbnail and metadata workers
cannot pile up copies.
Also preserves unknown top-level keys. `adjustments` is an opaque Value and
already round-tripped anything it did not recognise, but unknown envelope
keys were dropped on every save, so external tooling could not annotate a
sidecar without the GUI silently stripping it. ImageMetadata now carries
them through with #[serde(flatten)].
Adds the repo's first tests (9, covering both quarantine paths and the round
trip) and a `cargo test` job to run them.
Third-party tooling is already written against .rrdata, which makes it a public API whether or not it is documented as one. This publishes the contract and gives the version marker a meaning. Refs CyberTimon#1421. Stacked on the data-loss fix in the preceding commit, which introduces the quarantine mechanism this reuses for version mismatches. Versioning `version` has been written into every sidecar RapidRAW has ever saved, always as 1, and is read by nothing. Rather than adding a second `schemaVersion` field beside it, this makes the existing field normative. Every file already on disk is therefore stamped and negotiable, and no migration is needed. A sidecar declaring a higher version is quarantined rather than parsed as if it were v1. Schema schema/rrdata-v1.schema.json, hand-written in draft-07 for the widest third-party validator support, plus three worked examples that CI validates. Hand-written rather than generated because value ranges exist in neither type system: Adjustments is `[index: string]: any` in TypeScript and an opaque serde_json::Value in Rust, and the bounds live only in JSX slider props. A generator would emit the one thing integrators can already infer (the shape) and omit the one thing they cannot (the ranges). additionalProperties stays true. Rejecting unknown keys would contradict the preservation guarantee, so the schema would be describing a format that does not exist. It is strict on every key it does know, so a wrong type or an out-of-range value is caught while a forward-compatible addition is not. Docs docs/sidecar-format.md covers what a schema cannot: version negotiation, what survives a round trip, tag namespacing, and preset merge semantics. Applying a preset is a single one-level spread, so a preset carrying `hsl` replaces all eight bands times three properties rather than merging into them. That trap is documented with a worked example, along with the three differently-scaled meanings of `hue` and the asymmetric parametric whiteLevel/blackLevel ranges. `centré` is documented as normative in v1. The accented key is historical and load-bearing through TypeScript, Rust and every sidecar on disk; it is frozen here rather than quietly renamed, and flagged as a v2 candidate. Drift protection A hand-written schema can fall behind the code it describes. scripts/check-schema-drift.mjs fails CI when the schema and INITIAL_ADJUSTMENTS disagree about which keys exist, and asserts the schema still rejects malformed documents and still accepts unknown ones. Wired into lint.yml as a hard failure.
magnusrodseth
force-pushed
the
feat/sidecar-schema-v1
branch
from
July 25, 2026 16:18
8e65f99 to
45d0313
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 1 of #1421. Stacked on #1422 and contains its commit, so review that one first; this diff shrinks to just the format work once it merges.
Third-party tooling is already written against
.rrdata, which makes it a public API whether or not it is documented as one. This publishes the contract and gives the version marker a meaning.Your three open questions, answered with what I found
3.
schemaVersionin.rrdata? Better than I proposed: it is already there.versionhas been written into every sidecar RapidRAW has ever saved, always as1, and is read by nothing. Rather than adding a second field beside a vestigial one, this makes the existing field normative. Every file already on disk is stamped and negotiable, and there is no migration.2. Committed or generated? Committed, and this turned out to be a fact rather than a preference. Value ranges exist in neither type system:
Adjustmentsis[index: string]: anyin TypeScript and an opaqueserde_json::Valuein Rust, and the bounds live only in JSX slider props (min={-100} max={100}). A generator emits the one thing integrators can already infer (the shape) and omits the one thing they cannot (the ranges). I extracted all 89 sliders programmatically to build the schema.1. The local socket? Not touched here. That question only gates Phase 0, which I have deliberately not built pending your answer.
What is in it
schema/rrdata-v1.schema.json, draft-07 for the widest third-party validator support, plus three worked examples CI validates.docs/sidecar-format.mdfor what a schema cannot express: version negotiation, round-trip guarantees, tag namespacing, and preset merge semantics.scripts/check-schema-drift.mjs, wired into CI as a hard failure.additionalPropertiesstaystrue. Rejecting unknown keys would contradict the preservation guarantee, so the schema would be describing a format that does not exist. It is strict on every key it does know, so a wrong type or an out-of-range value is caught while a forward-compatible addition is not.Two things worth your call
centré. The detail-centring key genuinely ends in U+00E9, and it is load-bearing through TypeScript, Rust (pub centré: f32) and every sidecar on disk. Publishing v1 freezes it as public API. I documented it as normative rather than quietly renaming it, since a rename needs a migration and is your decision, not something to smuggle into a docs PR. Flagged as a v2 candidate.Drift is real, so it is guarded. A hand-written schema can fall behind the code. The drift check fails CI when the schema and
INITIAL_ADJUSTMENTSdisagree about which keys exist, and asserts the schema still rejects malformed documents and still accepts unknown ones. I verified it has teeth by mutating the schema three ways (drop a key, invent a key, loosen a constraint); each was caught.Things the docs pin down that surprised me
hslreplaces all 8 bands times 3 properties rather than merging into them. Documented with a worked example, since this silently breaks generated presets.huemeans three differently-scaled things:-180..180globally,-100..100per HSL band,0..360on a colour-grading wheel.whiteLevelis-100..0whileblackLevelis0..100.lumaNoiseReduction/colorNoiseReductionare0..100globally but-100..100inside a mask.tagsis a namespaced flat list (color:,user:, bare for AI tags).Checks
cargo fmt,cargo clippy -D warnings,cargo test(11),npm run schema:check, eslint and prettier all pass.Draft, as I would rather agree the shape than have you review a large diff you did not ask for. Happy to change any of it, or to land #1422 alone and drop this.