Skip to content

Run the WCAG gate in an engine that does layout, and fix what it found - #29

Merged
ChelseaKR merged 1 commit into
mainfrom
feat/a11y-in-a-real-browser
Aug 29, 2026
Merged

Run the WCAG gate in an engine that does layout, and fix what it found#29
ChelseaKR merged 1 commit into
mainfrom
feat/a11y-in-a-real-browser

Conversation

@ChelseaKR

Copy link
Copy Markdown
Owner

Closes #14. Adds ADR-0008. Independent of #26, #27 and #28; base is main.

What it found, on the first run

The scroll containers holding the wide tables were not keyboard reachable.

[serious] scrollable-region-focusable: Scrollable region must have keyboard access
    .tall.scroll:nth-child(7)
    .scroll:nth-child(19)

Every wide table on these pages sits in a container with overflow-x: auto. That is the conforming answer to WCAG 2.2 SC 1.4.10 Reflow, because the page itself does not scroll. On its own it is also a WCAG 2.1.1 Keyboard failure: a container that scrolls and cannot be focused scrolls for a mouse and a trackpad and not for a keyboard, so a sighted keyboard user cannot read the columns past the right edge at all.

Nothing in this repository could have found it. Whether a container scrolls is a fact about layout, and jsdom does none. tools/a11y.mjs had been green over it for three weeks, and so had every careful reading of the markup.

Each container is now a <section> with tabindex="0", named from the table's own caption, which every table here already carries. html-validate refuses role="region" in favour of the element, which is also the better answer: a named section is a region landmark natively, with no ARIA attribute to go stale.

The gate

Two engines, both kept.

jsdom (tools/a11y.mjs) Chromium (tools/a11y_browser)
Needs a browser binary no yes
Undecidable list four rules, each declared with a reason none: an undecided rule fails
SC 1.4.10 Reflow not possible 320 by 256
SC 2.5.8 Target size undecidable, covered nowhere decided
Colour contrast undecidable; measured arithmetically in test_pages_html.py decided

Both read the built pages off disk as file:// URLs. Nothing is served, no port is opened, and CI reaches the network only to fetch the browser. The reflow spec is ported from ledger's tools/a11y_browser/reflow.spec.ts, which runs against a served app; the adaptation is that these pages are static files.

Wired into make pages, so it is inside make verify and CI runs it as the same target, which is what CONTRIBUTING.md promises. make browser-audit runs npm audit --audit-level=high against the harness: a gate's own toolchain is not exempt from the check the gate exists to apply.

Proof the new gate can fail

Run unchanged against main's built pages (built in a detached worktree, PERIMETER_SITE_DIR pointed at it):

  2 failed
    [chromium] › axe.spec.ts:27:7 › axe in a browser: dins.html
    [chromium] › axe.spec.ts:27:7 › axe in a browser: perimeters.html
  6 passed (2.8s)

The six that pass there are positive controls that pass in both states: three reflow assertions, two directory guards, and index.html, which carries no scroll container.

tests/test_a11y_browser_gate.py, in the shape tests/test_a11y_gate.py already uses, is the ADR-0004 evidence. Eleven cases:

  • a conformant page passes both specs (positive control, parametrised over both)
  • a block 900px wide fails reflow at 320px
  • a 300-character unbreakable string inside a correctly sized paragraph fails reflow, which is the shape a layout inspector hides
  • an unfocusable scroll container fails axe, which is the defect above, kept as a regression
  • #aaa on #fff fails axe on contrast, which jsdom files as undecided
  • an 8px pointer target fails axe on target-size, which is SC 2.5.8
  • an empty directory fails both, rather than passing
  • a missing directory fails both, rather than passing

It refuses to skip when CI is set, because a skipped gate test reads as a passing one.

Gate output

$ make verify
...
Required test coverage of 90% reached. Total coverage: 100.00%
============================= 574 passed in 22.55s =============================
...
  8 passed (2.5s)
found 0 vulnerabilities
determinism: build/run-one and build/run-two are byte-identical (5 files)
MAKE_VERIFY_EXIT=0

Exit code read from $?. html-validate clean, both axe runs clean, reflow clean, both dependency trees clean.

About the waivers.yml the issue asks for

Nothing is suppressed anywhere after this, so there is none.

tools/a11y.mjs still declares four rules undecidable in jsdom, but every one of them is now decided by the browser run, and target-size, the one the issue correctly identified as covered by nothing, is decided rather than waived. An empty registry would be a file saying nothing. The absence is the claim, and the Accessibility row in README.md states it, with A11Y-06 and A11Y-09 recorded as met rather than waived.

README.md's "what still needs a person" list loses reflow and target size and keeps print, focus appearance against real backgrounds, a screen reader, and looking at the layout. It also now says that focus appearance matters more than it did, because every scroll container is a new tab stop.

Cost, stated

make verify is slower (roughly 22s of test time against 9s) and needs a browser binary, which is a one-time download. The local gate stays byte-for-byte identical to CI, so the cost is paid in both places or neither.

🤖 Generated with Claude Code

…t found

The scroll containers holding the wide tables were not keyboard reachable.

Every wide table on these pages sits in a container with overflow-x: auto,
which is the conforming answer to WCAG 2.2 SC 1.4.10 Reflow, because the page
itself does not scroll. On its own it is also a WCAG 2.1.1 Keyboard failure: a
container that scrolls and cannot be focused scrolls for a mouse and a trackpad
and not for a keyboard, so a sighted keyboard user cannot read the columns past
the right edge at all. axe rates it serious. Each container is now a <section>
with tabindex="0", named from the table's own caption, which every table here
already carries. html-validate refuses role="region" in favour of the element,
which is also the better answer: a named section is a region landmark natively
with no ARIA attribute to go stale.

Nothing in this repository could have found that. Whether a container scrolls is
a fact about layout, and jsdom does none. tools/a11y.mjs had been green over it
for three weeks and so had every careful reading of the markup.

So the gate now runs in two engines and both are kept:

  tools/a11y.mjs in jsdom is the floor. No browser binary, runs wherever node
  does, and its declared-undecidable list is the honest record of what a
  DOM-only engine cannot settle.

  tools/a11y_browser in Chromium is the ceiling and declares nothing. There is
  no undecidable list because there is no rule this engine cannot decide: a
  violation fails it and so does an undecided rule. It also carries SC 1.4.10
  Reflow at 320x256, which is 1280x1024 at 400% zoom and which no engine decides
  from a DOM, because it is a property of the viewport rather than the document.

Both read the built pages off disk as file:// URLs. Nothing is served, no port
is opened, and CI reaches the network only to fetch the browser. The reflow spec
is ported from ledger, which runs it against a served app; the adaptation is
that these pages are static files.

SC 2.5.8 Target size is now decided too, which closes the one rule
tools/a11y.mjs declared undecidable and covered nowhere.

Measured on this tree. Run unchanged against main's built pages, the new gate
fails:

    2 failed
      [chromium] > axe.spec.ts:27:7 > axe in a browser: dins.html
      [chromium] > axe.spec.ts:27:7 > axe in a browser: perimeters.html
    6 passed (2.8s)

      + "[serious] scrollable-region-focusable: Scrollable region must have
      +   keyboard access
      +     .tall.scroll:nth-child(7)
      +     .scroll:nth-child(19)",

The six that pass there are the positive controls: three reflow assertions, two
directory guards, and index.html, which carries no scroll container.

tests/test_a11y_browser_gate.py is the failure evidence ADR-0004 requires, in
the shape tests/test_a11y_gate.py already uses: eleven cases covering a page
that scrolls sideways at 320px, text spilling out of a correctly sized block, an
unfocusable scroll container, text below the contrast threshold, an 8px pointer
target, an empty directory, a missing directory, and a conformant page that must
pass both specs. It refuses to skip when CI is set, because a skipped gate test
reads as a passing one.

The browser harness is a second dependency tree and gets the same SCA the first
one gets: make browser-audit runs npm audit --audit-level=high against it.

Nothing is suppressed anywhere after this, so this repository has no waivers.yml.
Issue #14 asked for one covering whatever remained; nothing does, and the
standards row in README.md says so rather than shipping an empty registry.
README's "what still needs a person" list loses reflow and target size and keeps
print, focus appearance against real backgrounds, a screen reader, and looking at
the layout.

make verify exits 0: 574 tests, 100% branch coverage, html-validate clean, both
axe runs clean, reflow clean, both dependency trees clean, determinism
byte-identical.

Closes #14. See ADR-0008.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ChelseaKR
ChelseaKR merged commit ae406e3 into main Aug 29, 2026
6 checks passed
@ChelseaKR
ChelseaKR deleted the feat/a11y-in-a-real-browser branch August 29, 2026 00:15
ChelseaKR added a commit that referenced this pull request Aug 29, 2026
The rebase conflict in this function was resolved by keeping both sides, which
is right for a changelog and wrong for code. Main's #29 wrapped the table in a
focusable <section class="scroll tall" tabindex="0">; this branch still closed
it with the older </div>. Emitting both left <main> unclosed and produced four
stray end tags, which make htmlvalidate caught: the generated page was invalid
in a way no diff of the template would show.

Keeps main's </section> close, drops the orphaned </div> tail, and keeps this
branch's {access_note}. site/dins.html regenerated from data/raw/ rather than
hand-edited.
ChelseaKR added a commit that referenced this pull request Aug 29, 2026
The rebase conflict in this function was resolved by keeping both sides, which
is right for a changelog and wrong for code. Main's #29 wrapped the table in a
focusable <section class="scroll tall" tabindex="0">; this branch still closed
it with the older </div>. Emitting both left <main> unclosed and produced four
stray end tags, which make htmlvalidate caught: the generated page was invalid
in a way no diff of the template would show.

Keeps main's </section> close, drops the orphaned </div> tail, and keeps this
branch's {access_note}. site/dins.html regenerated from data/raw/ rather than
hand-edited.
ChelseaKR added a commit that referenced this pull request Aug 29, 2026
…file (#28)

* fix: an access table whose denominators could hold fewer records than the file

completeness_by_access reports each DINS field twice, over records with a damage
assessment and over records recorded as Inaccessible. Both populations are read
off DAMAGE, and access_field_coverages built its two denominators from the two
populations that answer, so a record whose DAMAGE is blank or holds a marker
answered neither question and left the table without appearing anywhere in it.

Measured against the pre-change code over four records, one assessed, one
inaccessible, one with a blank damage field and one holding a marker:

    records handed to the access table: 4
    assessed_total     : 1
    inaccessible_total : 1
    sum of denominators: 2
    records in no column: 2

Nothing published said which two were missing or that any were.

No record in the acquired file is in that state, which is why this held. access_split
has counted damage_not_recorded and damage_explicit_unknown since it was written,
precisely so a record like that would be reported rather than folded into the
assessed population, and both are 0 in the 2026-08-07 retrieval. Every denominator
happened to be complete. The table could not have said so and would not have said
otherwise, which is the same shape as the coverage exclusion in ADR-0004 and the
audit gate in ADR-0006: a measurement over a set that excluded part of what it
should have included, with the exclusion invisible and the check around it green
because the excluded part was empty on the day.

AccessFieldCoverage now carries a third population, derived by subtraction from
the records handed in rather than by a third predicate, so the three partition the
input by construction and a damage state nobody anticipated cannot fall between
them. counted_records is the sum, asserted equal to the record count in the code,
in the built artifact, and in the committed site/. The artifact publishes
undetermined_present, undetermined_total and undetermined_tenths_pct per field.

The page states what its two columns are counted over and what is in neither, in
both cases. On this retrieval it reads: "The two columns are counted over 131,931
assessed records and 591 recorded as Inaccessible, which between them is every
record in this file." Saying that in words is the part whose absence made this
invisible.

Measured on this tree. Against the pre-change source, all seven new tests fail,
including KeyError: 'undetermined_total' on the committed artifact, and the
existing access tests pass unchanged. After, make verify exits 0: 588 tests, 100%
branch coverage, determinism byte-identical.

See ADR-0007.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(site): regenerate after rebase onto main

The rebase conflict in site/dins.html was resolved by taking one side, which is
not a resolution for a generated file. This is make site's output from data/raw/,
so the committed HTML now matches its source rather than a merge of two
independently generated copies.

* fix(render): close the access table once, with main's focusable section

The rebase conflict in this function was resolved by keeping both sides, which
is right for a changelog and wrong for code. Main's #29 wrapped the table in a
focusable <section class="scroll tall" tabindex="0">; this branch still closed
it with the older </div>. Emitting both left <main> unclosed and produced four
stray end tags, which make htmlvalidate caught: the generated page was invalid
in a way no diff of the template would show.

Keeps main's </section> close, drops the orphaned </div> tail, and keeps this
branch's {access_note}. site/dins.html regenerated from data/raw/ rather than
hand-edited.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Two WCAG success criteria are suppressed in tools/a11y.mjs with no waiver record, and one of them has a working implementation in another repo

1 participant