Skip to content

test: Add 20 function-library fixtures incl. repr_* round-trips - #159

Open
leongdl wants to merge 1 commit into
OpenJobDescription:mainlinefrom
leongdl:conformance-func-lib-gaps
Open

test: Add 20 function-library fixtures incl. repr_* round-trips#159
leongdl wants to merge 1 commit into
OpenJobDescription:mainlinefrom
leongdl:conformance-func-lib-gaps

Conversation

@leongdl

@leongdl leongdl commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Summary

20 function-library fixtures from a coverage audit of RFC 0006. Suite: EXPR 350 → 368 passing, 0 failing (2 new fixtures are windows-gated and skip on POSIX).

Areas covered

  • repr_* round-trips — the security-critical surface. The audit found all five pre-existing repr_* fixtures assert the quoted text as a substring; none executed it. Quoting is only correct if the target interpreter agrees, so these fixtures pipe the quoted output through the real target: repr_sh through bash (15 adversarial payloads: metacharacters, $(...), backtick, nested quotes, backslash, glob, .., leading -, unicode, empty, whitespace-only), repr_py through ast.literal_eval (16 payloads), repr_json through json.loads (15 payloads) — each asserted byte-identical after the round trip.
  • repr_cmd/repr_pwsh round-trips: written and windows-gated (runOn: [windows]); they skip on POSIX and need a Windows CI run to count. %VAR%/!VAR! payloads excluded from the cmd battery per the spec's own .bat-context scoping.
  • Unicode codepoint semantics (the audit verified zero non-ASCII data existed anywhere in the EXPR suite): len on CJK/emoji/combining-mark/astral strings; case transforms restricted to 1:1 mappings; isascii/isalpha classification; padding to codepoint widths.
  • int64 inside functions: abs(-2^63) rejected (negation overflow), sum overflow rejected, floor(1e300)/ceil(1e300) rejected, int() at the exact-double 9e18 boundary.
  • Predicate/conversion edges: isupper("123")/islower("123") false (the "no cased characters" clause), predicates on "", bool("TRUE")/bool("1") case-insensitive forms, range_expr(" ") rejected.
  • Regex-dialect pins (divergence probes): lookahead, backreferences, and group references in re_sub replacements are all rejected by openjd-rs exactly as §2.2.5 requires — but Python's re natively supports all three, so a naive Python-backed implementation would accept them. These three .invalid fixtures now pin the conservative dialect for every implementation.

Deliberately excluded

~160 arity/type error cases (indistinguishable under the current runner, which asserts only a non-zero exit) and ~78 null-argument cases (the spec defines null behaviour for only 6 of 84 functions — a spec decision is needed first). Writing these now would manufacture confidence.

Companion PR

Three parked fixtures with dual-implementation observations (both-implementation repr_py newline bug; center()/isdigit() spec gaps where both implementations agree) are in the expected-failures PR (branch conformance-func-lib-expected-failures).

20 fixtures from the RFC 0006 coverage audit: repr_sh/repr_py/repr_json adversarial round-trips executed through the real target interpreter, repr_cmd/repr_pwsh round-trips (windows-gated), unicode codepoint semantics for len/case/predicates/padding, int64 overflow inside abs/sum/floor/ceil, predicate and conversion edges, and three regex-dialect pins (lookahead, backreference, group-reference rejection per spec 2.2.5). Suite: 350 -> 368 passing. Spec-question fixtures that fail today are split to conformance-func-lib-expected-failures.

Signed-off-by: David Leong <116610336+leongdl@users.noreply.github.com>
@leongdl
leongdl requested a review from a team as a code owner August 2, 2026 02:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant