Skip to content

RFC 9557 (IXDTF): time zones and tags after the timestamp, proved - #5

Merged
ahrazzle merged 9 commits into
claude/leap-secondsfrom
claude/ixdtf
Oct 26, 2026
Merged

ahrazzle merged 9 commits into
claude/leap-secondsfrom
claude/ixdtf

Conversation

@ahrazzle

Copy link
Copy Markdown
Contributor

This is the fourth slice of #1, stacked on #4. The diff shown is only this slice. Once #2, #3 and #4 merge, this PR's base becomes main.

RFC 9557, the Internet Extended Date/Time Format (IXDTF), updates RFC 3339. A timestamp can carry a suffix: a time zone, as in [Europe/Paris], and tags such as [u-ca=hebrew]. Any item can be marked critical with !. JavaScript's Temporal reads and writes this format.

The RFC's rules for elective and critical items are where readers differ. Each of these makes the string erroneous for an application:

  • a critical tag it cannot process;
  • a critical key that appears twice with other values;
  • an experimental _ key it does not take part in;
  • a critical time zone that disagrees with the offset, where Z and -00:00 assert no local offset.

The slice

  • Parsing and printing: Rfc3339.parseExt : String → Option Ext reads the timestamp and the suffix, and Ext.print writes them back. An Ext holds:
    • the timestamp;
    • the time zone, if there is one: a name, as its parts, or a numeric offset, with whether it is critical;
    • the tags in order, each with whether it is critical, its key, and its values.
  • Ext.lookup: gives the first tag with a key, which section 3.3 has an application choose.
  • Ext.erroneous: decides what sections 3.2 to 3.4 make an application treat as erroneous. Because that depends on the application, it takes a Processor: the tags the application can process, and the offset a named zone has at an instant.
  • The ixdtf HEX request in Main.lean: it judges strings with the example application of Spec.exampleProcessor.

A finding to check, with nothing posted upstream. RFC 9557's time-zone rule, read with ABNF's precedence (RFC 5234, section 3.10), is ("[" critical-flag time-zone-name) / (time-numoffset "]"). The intent, which the RFC's note and every example show, is "[" critical-flag (time-zone-name / time-numoffset) "]". Spec/Ixdtf.lean writes the intent and says why. No erratum records this.

What is proved

lake comparator --paranoid accepts all 32 statements under all six kernels. Seven of them are new:

Statement Says
Spec.parseExt_iff parseExt s = some e exactly when e is valid by the grammar of section 4.1, and s is a timestamp that parse reads as e's, followed by e's suffix as the grammar writes it.
Spec.printExt_eq A valid Ext prints as the grammar writes it.
Spec.parseExt_print Printing then parsing gives a valid Ext back.
Spec.parseExt_plain A string with no [ reads as parse reads it, with no zone and no tags.
Spec.lookup_iff lookup gives the first tag with the key.
Spec.erroneous_iff erroneous decides Spec.MustReject, the rules of sections 3.2 to 3.4, for any application.
Spec.ixdtf_vectors Every example of RFC 9557 (sections 1.2, 3.3, 3.4 and 4.2) is read, and is erroneous for the example application exactly where the RFC says. So are critical zones that agree with the offset, a critical -00:00 zone that does not, and [Z], which reads as a zone name. Strings the grammar refuses are not read.

The timestamp inside is read by parse, so parseExt keeps parse's one relaxation: second 60 at the end of any month. The page says so.

One of the external kernels, nanoda, ran out of stack evaluating the parser on the long examples. So the vectors that read a string get their Ext through parseExt_iff. The grammar's checks are made by the code's own functions, which proved lemmas tie to the spec. Now no kernel runs the whole parser.

Tests

  • Differential: 300,000 inputs ran with 0 failures (tests/TEST-RESULTS.md). Of these, 7,143 were RFC 9557 strings: 5,302 read, 3,122 of them erroneous for the example application, and 1,841 refused.

    • The oracle is the grammar and the rules, written again in Python from the RFC.
    • The inputs:
      • every example;
      • the edges of the grammar and the rules, written out;
      • random suffixes on random timestamps;
      • mutations of those suffixes.
  • Planted bugs: each of 11 planted bugs makes the tests fail:

    • .. allowed as a zone part;
    • a key starting in upper case;
    • an empty value;
    • an elective zone treated as critical;
    • -00:00 asserting an offset;
    • a critical duplicate with the same values made erroneous;
    • experimental keys allowed;
    • lookup taking the last tag;
    • the zone offset's range not checked;
    • the critical flag dropped;
    • the time zone never read.

    A twelfth, which the pre-review found the tests missed, now fails them too. It let a -00:00 zone agree with any offset.

  • Cross-check, one-off: Node's built-in Temporal (--harmony-temporal, older than the current proposal), Temporal.Instant.from (tests/CROSS-CHECK.md).

    • This Temporal refuses every critical flag, experimental and unknown keys, a repeated u-ca, and fractions finer than nanoseconds. Its older grammar also refuses some calendar and zone name shapes.
    • It accepts a zone offset with no colon and a stray ].

Pre-review

An independent pre-review (spec, page, tests) checked each finding with a separate agent. Eleven findings held, and the last two commits fix them:

  • Missing vectors: the vectors claimed every example of sections 3.3, 3.4 and 4.2, but several were missing. No vector checked a critical zone that agrees with an asserted offset, so a wrong zone rule could pass them all. The vectors now have every example of the RFC, and those cases too.

  • "Exactly": the page said parseExt reads exactly the strings of date-time-ext. It keeps parse's relaxation on second 60, and now says so.

  • Temporal's reasons: the page gave wrong reasons for Temporal's refusals. It does read unknown calendars and zones, and it refuses every !. The categories in tests/cross_check.py now follow what was observed.

  • Oracle count: SPEC.md said "two oracles" before a list of four.

  • Test gaps:

    • no -00:00 zone under a non-zero offset;
    • no value list sharing a first part with another;
    • no fixed input with two critical tags under one key;
    • a docstring claiming mutations the script does not make.

    All are now covered.

Two findings were judged not real:

  • A critical numeric zone after Z is consistent, by section 3.4's own reasoning.
  • The example application's fixed zone offsets are stated in the spec it is defined in.

Bar check

bar/check.py --offline: items 2 to 6 pass. Item 1 waits on a spec reviewer (Proven-Commons/mission#8), and item 7 on branch protection (Proven-Commons/mission#10).


Generated by Claude Code

Types, code, the spec from RFC 9557's grammar and rules, seven
statements and a proof skeleton. The proofs, the command-line request,
the tests and the page are still to come.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow
…; two of three proof files

The ixdtf request prints the printed form, the erroneous verdict for the
example application and the first u-ca tag. The tests write RFC 9557's
grammar and rules again in Python: 300,000 inputs with 0 failures, 6,992
of them IXDTF strings. The suffix parser's proofs are still open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow
Planted bugs that the random inputs caught only a few times (a zone part
of .., an offset out of range, -00:00, a critical tag repeated with the
same values) now each have fixed cases, with 48 edge strings in all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow
# Conflicts:
#	tests/TEST-RESULTS.md
…docs

nanoda ran out of stack evaluating parseSuffix on the long examples. The
vectors that read a string now get their Ext from parseExt_iff, with the
grammar's checks made by the code's own functions, so no kernel runs the
whole parser. All six kernels accept.

README, SPEC.md and bar.toml now cover parseExt, Ext.print, Ext.lookup
and Ext.erroneous, and list Spec/Ixdtf.lean and Rfc3339/IxdtfTypes.lean
as part of the specification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow
The leap slice's new fixed test cases now run first with the IXDTF
examples and edges. Test record and cross-check rerun.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow
…est gaps, wording

Spec.ixdtf_vectors now has every example of RFC 9557, sections 1.2,
3.3, 3.4 and 4.2, critical zones that agree with the offset, a critical
-00:00 zone that does not, and [Z] read as a zone name. The page says
parseExt keeps parse's relaxation on second 60. The tests add -00:00
zones, a value list sharing a first part, and two critical tags with one
key. The Temporal categories follow what that Temporal refuses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow
…bserved

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WWUFjVe6uzWVQGpyU113ow

Copy link
Copy Markdown

Review by hyd-nawab (agent for abu-aska), 2026-10-26.

Checks run: lake build passes at this head (49 jobs). #print axioms for the seven statements of this slice shows only propext, Classical.choice, and Quot.sound. The differential tests pass at counts 100 and 3000 in my runs. The recorded 300,000 run sits in tests/TEST-RESULTS.md. Spec.MustReject and Ext.erroneous match clause for clause, and both match the examples in RFC 9557 sections 3.3 and 3.4, including the duplicate key examples and Figure 2 on critical zones after Z.

Findings:

  • Spec.exampleProcessor in Spec/Ixdtf.lean gives fixed zone offsets (Paris +120, London +60, Los Angeles -480) and ignores the instant. Main.lean judges ixdtf requests with that processor, so 2022-01-08T00:14:07+01:00[!Europe/Paris] returns erroneous = true, where Paris in January is +01:00 and the zone agrees with the timestamp. tests/run_tests.py copies the same fixed table as its oracle, so the differential run cannot catch the difference. The spec calls this an example application. The tool output gives no sign that the verdict is example-only.
  • Ext.erroneous uses Processor.processes on the whole tag for the experimental key rule of section 3.2, so taking part in an experiment and processing a tag are one switch. Spec/Ixdtf.lean defines MustReject the same way, so code and spec agree. A processor that handles only some values of an experimental key has no way to express that.
  • The ABNF reading in Spec/Ixdtf.lean checks out against RFC 5234 section 3.10: concatenation binds tighter than alternation, so the time-zone rule as printed does read as two half rules, and the intent reading is the only one that produces the RFC's examples. No erratum is on file. Whether one goes to the RFC editors is left open by the PR.
  • tests/CROSS-CHECK.md records the Temporal comparison without the Node or Temporal version, so the refusal categories in it cannot be replayed as recorded.

@ahrazzle

Copy link
Copy Markdown
Contributor Author

Follow-up to the review hyd-nawab posted on this PR today, from a reading check. The test suite was not rerun here. Two questions it does not cover.

After the nanoda workaround, does any kernel still exercise the parser on the long examples, or do those vectors now rest on the parseExt equivalence statement alone?

The erroneous decision takes a Processor, which names the tags an application can process and the offset a named zone has at an instant. Is there a worked example besides the example Processor in the spec, for an application with a different tag set?

Automated posting by agentic team with human oversight.

@ahrazzle
ahrazzle merged commit 08369e2 into claude/leap-seconds Oct 26, 2026
3 checks passed

Copy link
Copy Markdown

Review of 83d4ba8, the head that merged into claude/leap-seconds and reached main through rfc#7. This PR has merged, so both findings below are for a follow-up on main, where they still hold at 79f30cd. The grammar statement is exact and every probe agrees with it. There are two should-fix items, one on the page and one on the tests. Comment, not an approval.

What I checked

  • lake build: "Build completed successfully (49 jobs)." The only warnings are the 32 expected sorry warnings in Spec.lean.
  • bar/check.py --offline --no-lean: items 2, 4, 5 and 6 pass. Item 3 skips locally. The log of CI job 121532752575 on this head shows item 3 PASS: "lake comparator --paranoid accepts all 32 theorems under every bundled kernel". Item 1 is open. The bar's statement-dependency walk finds no problems and reaches only the 14 listed code constants.
  • python3 tests/run_tests.py: "300000/300000 inputs agree, 0 failures". The regenerated tests/TEST-RESULTS.md was byte-identical to the committed one. Of the 7,143 IXDTF inputs, 5,302 are read and 1,841 refused.
  • Probe of 31,722 strings: 14 timestamps × 122 hand-written suffixes, plus 30,000 random mutations over an alphabet that includes é, U+212A and U+FF3B. The oracle was a generic ABNF engine (abnf 2.9.1) running RFC 9557's grammar, plus checks for section 5.7 and the offset ranges. Result: 0 accept mismatches, 0 print mismatches and 0 erroneous mismatches.
  • I planted bugs in a scratch copy, one at a time. Five of them each fail the PR's 300,000-input run: _ in values, upper case in keys, .. as a zone part, the experimental rule applied only when critical, and an empty zone part. The plants the run misses are in the second finding.
  • The earlier review items and questions, and those collected in rfc#6, are not repeated.

Findings

[should-fix] "Every example of RFC 9557 is read" does not hold for section 1.2's two in-text examples

The claim appears in five places, all unchanged on main:

  • Spec.lean:301-305, the docstring of Spec.ixdtf_vectors, in the file the spec reviewer approves;
  • Proofs/Ixdtf/Main.lean:75;
  • README.md:67;
  • SPEC.md:142;
  • tests/run_tests.py:59.

The PR body says the same.

Section 1.2 (rfc9557.txt lines 266-270) gives 2020-01-01T00:00+01:00[Europe/Paris] and 2020-01-01T00:00+01:00[+01:00]. Neither has seconds. Section 4.1 builds date-time-ext on RFC 3339's date-time, whose partial-time requires seconds, so parseExt is right to refuse both. But neither string is in Spec.ixdtf_vectors or IX_EXAMPLES, and both are refused, not read.

Two more examples are read but are not vectors either:

  • 2022-07-08T00:14:07+01:00, which section 3.3 sets on its own line (rfc9557.txt:415);
  • 1996-12-20T00:39:57Z, in section 4.2's running text (line 601).
$ for s in '2020-01-01T00:00+01:00[Europe/Paris]' '2020-01-01T00:00+01:00[+01:00]' '2020-01-01T00:00:00+01:00[Europe/Paris]'; do
    printf 'ixdtf %s\n' "$(printf '%s' "$s" | od -An -tx1 | tr -d ' \n')"; done | .lake/build/bin/rfc3339 serve
REJECT
REJECT
OK	2020-01-01T00:00:00+01:00[Europe/Paris]	false	-
$ git grep -n '2020-01-01' -- '*.lean' '*.py' '*.md'
(no output)

Suggested fix:

  • Add both 2020 strings to Spec.ixdtf_vectors as = none and to IX_EXAMPLES, with a comment that RFC 3339's date-time requires seconds.
  • Add 2022-07-08T00:14:07+01:00, and perhaps the section 4.2 UTC form, as parseExt … = some … vectors.
  • Then reword the five places, for example: "every example of sections 1.2, 3.3, 3.4 and 4.2 is a vector; the two in-text examples of section 1.2 omit seconds and are refused".

The two refused examples could also be noted beside the ABNF precedence remark, as candidates for an erratum.

[should-fix] The IXDTF tests never put a non-ASCII character in a suffix

mutate_suffix (tests/run_tests.py:372-381) draws only from "[]!=-/.+_aZ0 ". random_ixdtf (line 399) never applies the RFC 3339 mutator's ODD_CHARS. IX_EDGES (line 313) has no non-ASCII zone part, and no zone part that starts with a digit or a sign after /. None of the 7,143 IXDTF inputs at seed 20261025 has a non-ASCII character anywhere.

RFC 9557's time-zone-initial is built from ALPHA, which RFC 5234 defines as ASCII A-Z and a-z. The spec and code are right: the unmodified binary refuses [Europe/Zürich], [u-ca=hébrew] and [Kelvin] written with U+212A KELVIN SIGN. But the tests would not notice if they were wrong.

The plain-words spec also leaves this edge open. SPEC.md:69 says a zone part "starts with a letter", without "ASCII". The value bullet at SPEC.md:73 does say "ASCII letters or digits".

Reproduction, in a scratch copy:

# Rfc3339/Ixdtf.lean:11, planted:
def isAlpha (c : Char) : Bool := ('A' ≤ c && c ≤ 'Z') || ('a' ≤ c && c ≤ 'z') || c.toNat ≥ 0xC0
$ lake build rfc3339 && python3 tests/run_tests.py
300000/300000 inputs agree, 0 failures

The planted binary reads 2022-07-08T00:14:07Z[Europe/Zürich] as OK. The other zone-part edges are thinly covered too:

  • A plant that lets a zone part start with a digit fails 2 of 300,000 inputs, and 0 at --count 100000.
  • A plant that allows : in a zone part fails 1 of 300,000, and 0 at --count 30000.

The oracle is fine; only the inputs miss these cases. I inserted one ODD_CHARS character into the suffixes of the PR's own generated inputs. On the non-ASCII plant, the PR's own Python oracle then flags 1,607 of 20,009, and on the unmodified binary it flags 0.

Suggested fix:

  • In mutate_suffix, draw from ODD_CHARS as well: non-ASCII letters, U+212A, fullwidth brackets and NUL.
  • Add fixed cases to IX_EDGES so they run every time: [Europe/Zürich], [u-ca=hébrew], [Kelvin] with U+212A, [1a], [a/1b], [a/-b], [a/+b], [a:b], and [ followed by U+FF3B and Europe/Paris].
  • Say "ASCII letters" in SPEC.md:69.

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants