Skip to content

chore(tests): run the W3C Notation3 spec suites in CI - #640

Draft
jeswr wants to merge 4 commits into
rdfjs:mainfrom
jeswr:chore/n3-spec-tests
Draft

chore(tests): run the W3C Notation3 spec suites in CI#640
jeswr wants to merge 4 commits into
rdfjs:mainfrom
jeswr:chore/n3-spec-tests

Conversation

@jeswr

@jeswr jeswr commented Jul 2, 2026

Copy link
Copy Markdown
Member

Runs the W3C Notation3 parser and extended suites (manifest-parser.ttl and manifest-extended.ttl) against N3.js as spec-n3-parser and spec-n3-extended jobs in the CI spec matrix. No project-maintained skip list is applied: this PR remains draft and the jobs deliberately expose unsupported cases. Closes #331.

The suites moved to the Notation3 CG, so their canonical manifests are under https://w3c-cg.github.io/N3/tests/N3Tests/ (the old w3c.github.io/N3/tests/… URLs now return 404). manifest-extended.ttl contains parser syntax tests rather than reasoning or evaluation tests.

Why a local checkout + -m instead of -c: rdf-test-suite constructs all test cases concurrently, producing roughly 1,000 simultaneous GitHub Pages fetches and reliable UND_ERR_CONNECT_TIMEOUT failures. PR #612 makes warm .rdf-test-suite-cache runs reliable but cannot safely populate a cold N3 cache. spec-n3-fetch therefore creates a sparse, blob-filtered checkout of w3c-cg/N3 and the suites use rdf-test-suite mapping from URLs to files, preserving their canonical test IRIs without the request burst.

The fixture checkout is pinned to the N3 revision in spec/cache-key.txt. The weekly refresh workflow checks w3c-cg/N3 alongside the RDF 1.1/1.2 suites and proposes revision changes for review, so test updates do not arrive silently.

Current results at the pinned revision: parser 173/222 pass (16 suite-rejected tests skipped); extended 974/978 pass (107 suite-rejected tests skipped). Two parser-manifest TestN3Eval entries are not constructed because the released rdf-test-suite package cannot load their expected-result files.

Validated dependency stack: parser 223/223 pass (16 skipped); extended 978/978 pass (107 skipped). The regular N3.js suite also passes: 20/20 suites and 6,861/6,861 tests with 100% coverage, plus lint.

Merge dependencies

N3.js parser support:

Upstream test infrastructure and fixtures:

The undeclared-empty-prefix compatibility question is tracked upstream as w3c-cg/N3#235. The current-major opt-in and next-major default are tracked in #693 and #699 respectively.

cc @jeswr

Comment thread spec/n3-skips.js Outdated
@jeswr jeswr added the ai-generated Authored or prepared by an AI coding agent label Jul 9, 2026
jeswr and others added 2 commits August 23, 2026 23:40
@jeswr
jeswr force-pushed the chore/n3-spec-tests branch from 3c8e1ba to 7769cc8 Compare August 23, 2026 22:42
@jeswr jeswr added the needs-author-review Awaiting author review; agent has done its part (author removes if follow-ups remain) label Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-generated Authored or prepared by an AI coding agent needs-author-review Awaiting author review; agent has done its part (author removes if follow-ups remain)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spec testing for N3

1 participant