fix: enforce RFC 3987 IRI rules and close IDNA, WPT, and BCV gaps#71
Merged
Conversation
asciiLenientDomainToAscii split the raw domain on a literal U+002E before deciding, per label, whether to skip validation for an all-ASCII label. That missed the other three UTS-46 dot-separator variants (U+3002, U+FF0E, U+FF61), so a label boundary formed by one of them was invisible to the gate and the whole span got treated as one non-ASCII label, running the strict pipeline instead of getting the per-label leniency. Mirror domainToAscii's own step order instead: map the domain first, then split the mapped/NFC text into labels. Mapping ASCII input up front is safe since none of it changes under UseSTD3ASCIIRules=false beyond the case-folding asciiLowercase already does.
Iri.toUri previously percent-encoded any non-ASCII code point in an IRI's userinfo, path, query, or fragment without checking whether it was actually legal there. RFC 3987 restricts those components to the ucschar/iprivate repertoire (§2.2) and forbids bidirectional formatting characters like LRM and RLE anywhere in the IRI (§4.1). Add an IriRepertoire lookup for the fixed ucschar/iprivate ranges and the seven forbidden bidi characters, and run both checks in IriMapping before any host mapping or percent-encoding, so a violation is now a ParseResult.Err (UriParseError.IriInvalidCodePoint or IriBidiFormattingCharacter) instead of being silently encoded. The host stays out of this check since IDNA/UTS-46 already validates it independently. Threading component offsets through the IRI splitter so these errors can point at the exact offending index. The RFC's §4.2 per-component directionality rule is a SHOULD, not a MUST, and isn't enforced here; docs are updated to say so precisely.
tools/internal/idnaref re-derives kuri's own domainToAscii algorithm as a second implementation, so a bug shared between the Kotlin engine and its Go port would pass the known-failures ratchet undetected. Its value is catching porting slips between the two, not verifying the algorithm against an outside source. WPT is the actual independent oracle IdnaConformanceTest checks kuri against; make that explicit in the package doc, the codegen generator, IdnaConformanceTest's class doc, and the Unicode-update runbook instead of leaving it implied.
Add icu4j as a jvmTest-only dependency and run kuri's Idna.domainToAscii against ICU4J's UTS-46 implementation over every WPT IDNA case, as a genuinely third-party check alongside the existing WPT and idnaref comparisons. icu4j has no multiplatform artifact so it is scoped to the JVM test classpath only. kuri and ICU4J disagree in two structural, documented ways rather than everywhere: ICU4J's UTS46 has no switch to disable CheckHyphens or VerifyDnsLength, so it always rejects a leading/trailing hyphen, a "??--" label, an empty label, or a length overflow that kuri's Url profile deliberately permits; and ICU4J 77.1 bundles Unicode 16.0 data against kuri's 17.0, so a few inputs carrying newly assigned CJK ideographs disagree until ICU4J catches up. Both are asserted, not silently skipped: the first is filtered by ICU4J's own error codes with a one-line reason per code, and the second is pinned to an exact input list that ratchets if either side's Unicode version changes.
binary-compatibility-validator's built-in Kotlin Multiplatform support only recognizes an Android target compilation named "release" (the classic AGP com.android.library + androidTarget() build-variant naming). The AGP KMP library plugin used here (com.android.kotlin.multiplatform.library) is single-variant and names its production compilation "main" instead, so BCV never matched it and silently skipped the android surface: no androidApiBuild/androidApiCheck/androidApiDump tasks ever existed, which is how kuri/api/android/kuri.api went stale with nothing to regenerate or check it. Wire an equivalent android leg by hand with BCV's own public task classes (KotlinApiBuildTask/KotlinApiCompareTask) against the android target's compileAndroidMain output, using the same api/android/kuri.api layout BCV's other target legs already use, and hook it into the existing apiDump/apiCheck entry points so it runs as part of the same gate rather than needing a separate invocation.
Bring the android API snapshot current now that androidApiDump/apiCheck actually run it: it now matches api/jvm/kuri.api's public surface (minus the jvmMain-only java.net interop, which isn't compiled for android) and includes every API addition landed on this branch so far, including ValidationError.INVALID_CREDENTIALS.
x-www-form-urlencoded parsing had no conformance corpus wired up, unlike the URL/IDNA/percent-encoding paths. There is no standalone JSON resource for it upstream, so the array literal at the top of WPT's url/urlencoded-parser.any.js is hand-extracted and vendored as strict JSON under tools/urlencoded/urlencoded-parser.json, keeping only the plain input/output pairs that feed its "URLSearchParams constructed with" subtest -- the sibling formData() subtests exercise non-UTF-8 charsets kuri doesn't implement. Add a new Go generator (urlencoded.go) that reads that corpus and materializes a Kotlin fixture, following the same chunked-builder shape as the other WPT-backed generators, and wire it into the codegen dispatch table and the generateFixtures Gradle task.
Generate the UrlEncodedTestData fixture from the vendored WPT corpus and run every case through FormUrlEncoded.parse, mirroring the ratchet structure the percent-encoding and IDNA suites already use. All 35 in-scope cases pass against the existing implementation, so the tracked known-failures baseline is empty.
The parsing/serialization path is now checked against the WPT urlencoded-parser corpus with no known failures, so it earns the same Conformant label the other measured standards carry, and the new suite's result is listed in the Conformance table.
…label The per-label ASCII leniency in the Url domain-to-ASCII path mapped the whole domain before splitting into labels and checking each label's ASCII-ness. That let a non-ASCII sequence that renders down to a literal ASCII string under UTS-46 mapping (e.g. a fullwidth "xn--a") take the unconditional-keep branch and skip the Punycode decode/validate step entirely, even though the direct domainToAscii pipeline still rejects it. Split the raw domain on the literal dot separator and its three non-ASCII UTS-46 variants instead of mapping first, and make the ASCII/non-ASCII decision on each raw label before any mapping runs. This keeps the label boundary a mapped dot variant forms visible without letting mapped content leak into the leniency check.
…e catalog The hand-wired android apiBuild/apiCheck classpath pinned org.ow2.asm:asm and asm-tree with literal version strings, bypassing gradle/libs.versions.toml as the single source of truth for dependency versions. Add an asm entry to the catalog and reference it from build.gradle.kts so a future ASM bump has one place to update and shows up to Dependabot.
This was
linked to
issues
Jul 13, 2026
Closed
…a-jvm into the catalog androidApiBuild (the hand-wired android BCV task) never set inputDependencies, unlike every BCV-auto-configured target, which wires it from the compilation's compileDependencyFiles. Wire the same thing from the android target's main compilation for parity, and move the kotlin-metadata-jvm coordinate from an inline string into gradle/libs.versions.toml alongside asm-core/asm-tree, so every dependency in this classpath is declared the same way.
… checks Iri.toUri runs a bidi-formatting-character scan over the whole raw IRI before the repertoire check, and returns either of those precheck failures before ever attempting to map the host. Neither ordering had a dedicated test: add one proving a bidi violation wins even when a repertoire violation appears earlier in the string, and one proving a precheck failure wins over an IDNA-invalid host.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five independent fixes/features, each landed as its own commit(s):
IDNA per-label leniency now recognizes every UTS-46 dot-separator variant. The WHATWG web-compat leniency that keeps an invalid
xn--label as-is (rather than rejecting the whole domain) decides ASCII-ness per label by splitting on the literal.. It missed that UTS-46 also maps U+3002 (ideographic full stop), U+FF0E (fullwidth full stop), and U+FF61 (halfwidth ideographic full stop) to.— a label boundary formed by one of those was invisible to the split, so e.g.xn--a。büchergot treated as one non-ASCII "label" and failed outright instead of getting the intended per-label leniency. Fixed by recognizing all four separator code points when splitting the raw, pre-mapping domain (not by mapping first, which would let a non-ASCII sequence that renders down toxn--...under UTS-46 mapping bypass Punycode validation entirely — caught and fixed during review).Iri.toUrinow enforces RFC 3987's IRI repertoire and forbidden bidi-formatting characters. Previously any non-ASCII code point in a non-host component was percent-encoded unconditionally, with no check against RFC 3987 §2.2'sucschar/iprivaterepertoire, and no check for the seven bidirectional formatting characters (LRM, RLM, LRE, RLE, PDF, LRO, RLO) that §4.1 forbids anywhere in an IRI's stored form. Both are now enforced as fatalParseResult.Errcases. (RFC 3987 §4.2 additionally lists SHOULD-level structural recommendations for constructing bidi-friendly IRIs — those are producer-side advice, not receiver-side validity rules, so they're intentionally not enforced here; only the two actualMUST-level rules from §2.2 and §4.1 are.) This narrowsIri.toUri's accepted input: a caller feeding it a noncharacter or other out-of-repertoire code point (or a bidi formatting character) that previously got silently percent-encoded will now get aParseResult.Errinstead.Clarified that WPT, not
idnaref, is kuri's real independent IDNA oracle, sinceidnarefis a Go port of kuri's own algorithm and would reproduce a shared bug in both places undetected. Added a JVM-only differential test that cross-checks kuri's IDNA output against ICU4J over the vendored WPT corpus for additional, genuinely independent confidence.Wired up Android binary-compatibility checking, which had silently never run: binary-compatibility-validator's multiplatform auto-detection only recognizes an Android compilation literally named
release, but the AGP plugin this project uses (com.android.kotlin.multiplatform.library) names its single production compilationmain, so the android leg was skipped entirely andkuri/api/android/kuri.apihad gone stale (missing, e.g., theINVALID_CREDENTIALSentry from a previous fix). Hand-wired equivalentandroidApiBuild/androidApiCheck/androidApiDumptasks using BCV's own public task classes and hooked them into the existingapiCheck/apiDumpentry points, then regenerated the snapshot.Vendored the WPT
urlencoded-parsercorpus and added a Go generator (mirroring the existing URL/IDNA corpus pipeline) plus a conformance test forQueryParameters' form-urlencoded parsing, which previously had no vendored-corpus coverage. Updated the README's status for this feature from "Supported" to "Conformant" now that it's measured against the standard.Closes #64, #65, #66, #67, #69.
Test plan
:kuri:jvmTest :kuri:ktlintCheck :kuri:detekt :kuri:apiCheckgreen (including the newandroidApiCheckandklibApiChecklegs):kuri:jsNodeTestand:kuri:wasmJsNodeTestgreengenerateFixturesregenerates the new form-urlencoded fixture with no diff against what's committed