Skip to content

Launch gate: hidden shapes and tolerance matrix #2

Description

@metaforismo

Goal

Create the hidden correctness suite and dtype-specific tolerance matrix for KernelForge.

Launch checklist

  • Add hidden shapes for RMSNorm, RoPE, causal attention prefill, and KV decode.
  • Include edge cases: near-zero RMSNorm rows, repeated keys, long-ish sequences, and uneven score distributions.
  • Define dtype policy for float32, float16/bfloat16 where supported, and Python smoke floats.
  • Define max absolute and relative tolerances per primitive/dtype.
  • Add invalid-pattern checks for hardcoded public cases.
  • Add hidden-case hash metadata without exposing values.
  • Add verifier output fields for primitive-level pass/fail.

Acceptance criteria

  • A candidate must pass hidden correctness before timing counts.
  • Tolerance decisions are documented before official entries open.
  • Hidden cases cannot be inferred from public smoke data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions