Skip to content

fix(loader): refuse wrong-typed JSON dates instead of coercing them to NULL - #669

Merged
azimafroozeh merged 2 commits into
ModernRelay:mainfrom
azimafroozeh:loader-date-wrong-type-null
Sep 6, 2026
Merged

fix(loader): refuse wrong-typed JSON dates instead of coercing them to NULL#669
azimafroozeh merged 2 commits into
ModernRelay:mainfrom
azimafroozeh:loader-date-wrong-type-null

Conversation

@azimafroozeh

Copy link
Copy Markdown
Contributor

What & why

Closes #628. A JSONL load through omnigraph load or POST /graphs/{id}/load stored NULL for a nullable Date or DateTime property whose JSON value was a float (19723.0), a boolean, or an object, and reported success; the same row through POST /graphs/{id}/load/ndjson was refused.

  • Cause: both date parsers try null, integer, then string, and fall through to Ok(None), the value a real null returns, so the column builder cannot tell absent from unrecognized; only the strict mode's input-versus-output null comparison recovered the difference.
  • Fix: the fall-through is a typed refusal, invalid Date value 19723.0 for property 'since': expected an integer day count or a date string (DateTime: integer millisecond count or datetime string). Both conversion modes, scalars and list items, nullable or not, inherit it from the parser.
  • The range errors on integer counts now name the property too, matching every neighbouring loader refusal.
  • Integer counts, date strings, and null load as before.

Backing issue / RFC

Checklist

  • Change is focused (the two date parsers' fall-through, plus the property name threaded into their messages)
  • Tests added/updated for behavior changes (a table-driven unit test over both modes, both types, scalar and list, nullable and not, with a stored-value oracle for the accepted shapes; an end-to-end load_jsonl test that pins the message and an unchanged store version after the refusal)
  • Public docs updated if user-facing surface changed (docs/user/mutations/index.md §Bulk loading names the accepted shapes and the refusal; skills/omnigraph/SKILL.md and references/queries.md gain integer epoch milliseconds for DateTime; docs/dev/ingestion.md states the compatibility coercion rule and the dates exception; release note)
  • Reviewed against docs/dev/invariants.md — no Hard Invariant weakened, no deny-list item hit (invariant 8 is strengthened: a lost error becomes a typed failure; the deny-list's swallowed-error shape loses one instance)

Local verification

  • cargo test -p omnigraph-engine --lib loader:: — 29 passed
  • cargo test -p omnigraph-gqt — 116 self-tests + 13 cases passed
  • cargo clippy --workspace --all-targets — clean
  • cargo fmt --all --check — clean
  • scripts/check-docs.py — OK (129 files)
  • typos — clean
  • cargo test -p omnigraph-server — all targets green (387 passed, 0 failed, 2 ignored)
  • cargo test -p omnigraph-cli — all targets green (337 passed, 0 failed, 15 ignored)
  • full engine integration suite, DST, s3_storage — not run: the change is confined to two parser functions whose only callers are the loader column arms covered above

Notes for reviewers

  • Only Date and DateTime refuse a wrong JSON type. String, integer, float, and Bool properties keep the compatibility loader's historical NULL coercion, now stated in docs/dev/ingestion.md. Widening the refusal to every scalar is a contract change for existing load users and is deliberately not part of this fix.
  • Strict-mode message changes for dates: the parser refuses before the post-build check, so strict property 'day' expects Date32, got 19723.0 becomes the invalid Date value message. No test asserted the old text for dates.
  • A whole-number float is a float to the loader; the release note says to send the integer rather than adding a float-to-integer coercion that the strict path does not have.
  • The two regression tests are Rust rather than .gqt because a seed load failure is a harness error, not an expectable outcome; the FIXME(#628) lines above them name the conversion to one .gqt case once GQ has a load statement.
  • Out of scope: a Date string carrying a UTC offset stores the UTC day. That is the string parser, pre-existing, and reported separately.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@azimafroozeh
azimafroozeh merged commit d520d2b into ModernRelay:main Sep 6, 2026
23 checks passed
@azimafroozeh
azimafroozeh deleted the loader-date-wrong-type-null branch September 6, 2026 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: JSONL load stores NULL for a Date or DateTime given a float, bool, or object

1 participant