^bytes store fast path, direct (byte x), and a core name a namespace can own

CHANGELOG.md

@@ -7,8 +7,88 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Performance
+
+- **A hinted `(aset ^bytes a i v)` stores into the bytevector, and `(byte x)` is
+  a direct call.** `bench/byte-arrays`' `bfill` phase was the largest single
+  jolt/JVM ratio in the suite, and both halves of its one hot line were paying to
+  get to work that costs a few nanoseconds.
+
+  `^bytes` was absent from the `:v-aset` fast path `^longs`/`^ints`/`^objects`
+  take, because a byte array narrows its value to signed 8 bits at the store and
+  that narrowing lived on the generic path — so the hint bought a byte store
+  nothing at all, and it emitted the same `jolt-aset3` an unhinted one does. Per
+  element: the array test, the kind read and an `eq?` on it, `na-byte-of`'s
+  `truncate`/`exact`/`bitwise-and` over the generic (bignum-capable) tower,
+  `(exact (na-idx i))` to coerce an index that was already a fixnum, then
+  `ja-set!`'s *second* read of the backing, a bounds pre-check and a four-way
+  `cond` — **34.7ns** to reach a `bytevector-s8-set!`, against 5.9ns for the
+  matching `(aget ^bytes a i)`.
+
+  `jolt-baset` is the byte kind's own store target, split from `jolt-vaset`
+  rather than folded into it because `jolt-vaset` answers its argument and a byte
+  store has to answer what was stored. A fixnum already inside -128..127 — what a
+  byte-filling loop hands it — is by construction what `na-byte-of` would answer
+  for it, so it goes straight into the bytevector; a flonum, a bignum, an
+  out-of-range value, a non-fixnum index, a byte array whose backing an older
+  image left boxed, and a lying `^bytes` hint on an array of any other kind all
+  fall through to the generic seam. That fallback is not a slow path added for
+  the helper, it is the path every unhinted `aset` takes, which is why the
+  narrowing contract is unchanged — checked by differential test over 600+ value
+  shapes, every index shape and all five backing kinds. **34.7ns → 7.1ns.**
+
+  The other half was `(byte x)`. `double`, `long`, `int` and `float` lower to a
+  `:coerce` node carrying their checked runtime helper; `byte` and `short` were
+  the two casts missing from that table, so `(byte v)` stayed a var-deref plus a
+  `jolt-invoke1` around `jolt-byte-cast`'s two fixnum compares — 17.9ns an
+  element, *more than the store it fed*. They join it with kind `:long`, sound for
+  the reason `int` takes it: `jolt-checked-cast` answers a value inside
+  `[lo, hi]` or throws, and both ranges are fixnums on every tower jolt has. The
+  checked semantics are untouched — `(byte 200)` is still an
+  `IllegalArgumentException`, not a wrap to -56; `(byte 127.000001)` still throws
+  where `(byte 1.9)` is 1; `byte` in value position is still the var.
+
+  Together, on `bench/byte-arrays` at 400 passes: `bfill` **185.0ms → 41.0ms
+  (4.5×)**, or 66× the JVM down to 14.6×. Every other phase is flat to within 1%
+  — `copy-full` 24.6/24.6, `copy-region` 25.3/25.4, `drain` 221.1/226.5,
+  `round-trip` 38.7/38.7, `bsum` 38.1/37.5 — which is what says the change is
+  where it claims to be.
+
+  `na-byte-of` and `na-array-set!` took the same fixnum-first shape while they
+  were open, so the generic seam is faster too: an unhinted `aset`, and every
+  other door into a byte array (`into-array`, `Arrays/fill`, `na-list->backing`).
+
 ### Fixed
 
+- **A namespace-level `(defn double …)` owns the name, as it already did for
+  `(defn first …)`.** jolt has two layers that rewrite a `clojure.core` call into
+  something cheaper, and they disagreed about who owns a name. The op-registry
+  lowering resolves the head and checks the resolved var's namespace
+  (`backend_scheme/native-op`), so a user-defined `first` was always called. The
+  analyzer's numeric-cast and `*unchecked-math*` rewrites matched on the bare
+  source name behind a `shadowed` guard that only sees **locals** — so a
+  namespace-level definition, `:refer-clojure :exclude` or not, was silently
+  ignored in call position:
+
+  ```clojure
+  (ns shadow (:refer-clojure :exclude [double first]))
+  (defn double [x] :my-double)
+  (defn first  [x] :my-first)
+  (double 5)   ; jolt: 5.0        JVM Clojure: :my-double
+  (first [1 2]); jolt: :my-first  JVM Clojure: :my-first
+  ```
+
+  The same held for `long`/`int`/`float` (and `byte`/`short`, new above), and for
+  every name `*unchecked-math*` rewrites — a user's `+` became `unchecked-add`.
+  Both rewrites now ask the question the op-registry layer already asked, so the
+  two agree and all of it matches the reference. A `clojure.core/`-qualified head
+  names its namespace outright and still lowers, in the very namespace that
+  redefined the bare name.
+
+  Resolution happens only after a name AND arity have matched, so an ordinary
+  call — every other list head in the program — pays nothing for it, and the
+  numbers above are unchanged with the check in place.
+
 - **The tree-shake gate asserts how MUCH was shaken, and covers the
   spliced-callee bail class.** `make shakelocal` asked only for the string
   `tree-shake kept` in the build's report, so a regression that shook but kept

bench/README.md

@@ -265,24 +265,32 @@ figures the release claimed, restated at the size the suite runs:
 | `copy-region` | the same bytes at unequal offsets | 7.7 | 12.2 | **0.63×** |
 | `drain` | 1MB through an 8KB `InputStream/read` loop (128 reads) | 80.3 | 11.6 | 6.9× |
 | `round-trip` | 4 × `(String. (.getBytes s))` over a 5KB string | 21.0 | 9.9 | 2.1× |
-| `bfill` | 8192 `(aset ^bytes a i v)` | 95.2 | 0.6 | 159× |
+| `bfill` † | 8192 `(aset ^bytes a i v)` | 95.2 | 0.6 | 159× |
 | `bsum` | 8192 `(aget ^bytes a i)` | 19.4 | 2.1 | 9.2× |
 
+† `bfill` predates the `^bytes` store fast path and is the one row this sitting no
+longer describes; the bullet below has the current figures.
+
 The block copies **beat the JVM** — 19.5µs per megabyte against its 30.8µs — which
 is the backing change arriving: both sides of that seam are bytevectors, so the
 copy is one `bytevector-copy!` rather than an element loop with a sign fold per
-byte. Two gaps came out of writing the row, and it now watches both:
-
-- **`(aset ^bytes a i v)` costs 29ns an element**, where the matching
-  `(aget ^bytes a i)` costs 5.9ns. The read takes the direct backing read; the
-  hinted STORE is deliberately left on the generic path
-  (`jolt-core/jolt/passes/numeric.clj`) because a byte array narrows its value to
-  signed 8 bits at the store and that narrowing lives there, so `^bytes` is absent
-  from the `:v-aset` fast path `^longs`/`^ints`/`^objects` take. It is the largest
-  single ratio in the row.
-- **an 8KB `InputStream/read` costs 1.57µs**, against the JVM's 0.23µs. The
-  transfer itself is a block move now; what is left is per-call overhead on the way
-  to it.
+byte. Two gaps came out of writing the row. The store one is closed:
+
+- **`(aset ^bytes a i v)` stores into the bytevector directly.** The byte kind is
+  the one whose store narrows — to signed 8 bits, answering what it stored — so it
+  has its own target (`jolt-baset`) rather than joining the `:v-aset` fast path
+  `^longs`/`^ints`/`^objects` take: an in-range fixnum goes straight to
+  `bytevector-s8-set!`, anything else falls through to the generic seam that
+  narrows it. `(byte x)`, the other half of the `bfill` line, lowers to
+  `jolt-byte-cast` through the same checked-cast table `double`/`long`/`int`/
+  `float` use. On a dev box rather than the release machine above, same binary
+  shape and 400 passes either side: `bfill` **185.0ms → 41.0ms (4.5×)**, and 66×
+  the JVM → 14.6×. The store alone is 34.7ns → 7.1ns an element, against the
+  matching `(aget ^bytes a i)` at 5.9ns. Every other phase is flat to within 1%,
+  which is what says the change is where it claims to be.
+- **an 8KB `InputStream/read` costs 1.57µs**, against the JVM's 0.23µs — still
+  open. The transfer itself is a block move; what is left is per-call overhead on
+  the way to it.
 
 `gc-arrays`, per collection, 8-million-element arrays, same sitting:
 

bench/byte_arrays.clj

@@ -14,12 +14,12 @@
 ;; boxed vector of small integers, this whole bench runs 49x faster (13150.9ms ->
 ;; 270.6ms through ci/bench-gate.sh; the per-phase split is in README.md).
 ;;
-;; The element-access phase is here for the two halves of a hinted ^bytes access,
-;; which pull in opposite directions: (aget ^bytes a i) lowers to the direct
-;; backing read (jolt-vaget, skipping the generic nth dispatch walk), while
-;; (aset ^bytes a i v) deliberately stays on the generic path, because the store
-;; has to narrow its value to signed 8 bits. A codegen round that moves either one
-;; lands here.
+;; The element-access phase is here for the two halves of a hinted ^bytes access:
+;; (aget ^bytes a i) lowers to the direct backing read (jolt-vaget, skipping the
+;; generic nth dispatch walk) and (aset ^bytes a i v) to jolt-baset, which owns
+;; the narrowing to signed 8 bits that a byte store has to do. The fill line also
+;; carries (byte v), a checked cast that lowers to jolt-byte-cast rather than
+;; going through a var. A codegen round that moves any of the three lands here.
 ;;
 ;; No unhinted twin: the generic (aget a i) / (aset a i v) dispatch walk is already
 ;; covered by `arrays-unhinted`, and it is the same walk for every element kind.

host/chez/java/natives-array.ss

@@ -284,7 +284,16 @@
 ;; the low 8 bits, fold the sign bit. na-bytearray->bv masks back to 0..255 at the
 ;; raw-byte seam, so the two carriers round-trip byte-exactly.
 (define (na-u8->byte b) (if (fx<? b 128) b (fx- b 256)))
-(define (na-byte-of v) (na-u8->byte (bitwise-and (exact (truncate v)) #xff)))
+;; The fixnum arm is the same answer by a shorter route, not a second rule: a
+;; fixnum IS its own (exact (truncate v)), so all that is left of the general
+;; case is the mask and the fold. Worth splitting because the general case spends
+;; three GENERIC (bignum-capable) operations to discover that — 14.5ns — and every
+;; door into a byte array comes through here: into-array, Arrays/fill,
+;; na-list->backing, and the untyped aset.
+(define (na-byte-of v)
+  (if (fixnum? v)
+      (na-u8->byte (fxand v #xff))
+      (na-u8->byte (bitwise-and (exact (truncate v)) #xff))))
 ;; Narrow a value being STORED into an array to its element kind. Only 'byte has a
 ;; range jolt must maintain (the u8 <-> s8 bridge above depends on it); the other
 ;; kinds hold whatever integer/flonum they are given, as they always have.
@@ -457,9 +466,12 @@
 ;; THE array write seam: the aset overlay, jolt-aset3, and any jolt.host/ref-put!
 ;; on an array all land here. Narrows the value to the element kind (na-elem-of) and
 ;; answers what was actually stored, so aset's return agrees with a following aget.
+;; The index takes the fixnum-first route jolt-vaget/jolt-flaget take: (exact
+;; (na-idx k)) on a fixnum is two procedure calls to answer k itself, 9ns of an
+;; untyped store.
 (define (na-array-set! a k v)
   (let ((sv (na-elem-of (jolt-array-kind a) v)))
-    (ja-set! a (exact (na-idx k)) sv) sv))
+    (ja-set! a (if (fixnum? k) k (exact (na-idx k))) sv) sv))
 (define %na-ref-put! jolt-ref-put!)
 (set! jolt-ref-put!
   (lambda (t k v)
@@ -509,10 +521,9 @@
 ;; fixnum, but a promoted array's is whatever integer was stored, so an element
 ;; is still not provably one and the numeric pass must not type it :long.
 ;;
-;; 'byte is deliberately NOT a write target here. A byte array's elements are
-;; signed 8-bit and na-elem-of is the one place a value entering one is narrowed;
-;; routing a write around it would let a byte array hold 200 — and the bytevector
-;; backing would refuse it outright. Reads are fine — the narrowing already
+;; 'byte reads here but writes through jolt-baset below: its elements are signed
+;; 8-bit, and a store has to narrow to that range and answer what it stored,
+;; neither of which this pair does. Reads need neither — the narrowing already
 ;; happened at the store.
 ;;
 ;; The unboxed backings need no index pre-check: their own range check IS the
@@ -546,6 +557,38 @@
             (na-oob-throw j (vector-length bk))))
     v))
 
+;; (aset ^bytes a i v) — the byte kind's own store target, split from jolt-vaset
+;; rather than folded into it because a byte array is the one kind whose store
+;; NARROWS: na-elem-of folds the value to signed 8 bits and na-array-set! answers
+;; what was actually stored, so a helper that returned its argument the way
+;; jolt-vaset does would disagree with the following aget.
+;;
+;; What it skips is the generic seam's walk to the same bytevector-s8-set!:
+;; jolt-aset3's array test, the kind read and the eq? on it, na-byte-of's
+;; truncate/exact/bitwise-and over the numeric tower, the index coercion, then
+;; ja-set!'s SECOND read of the backing, ja-check, and a four-way cond — ~34ns of
+;; a 40ns store, against 7ns here.
+;;
+;; The guard is the whole contract. A fixnum already inside -128..127 is what
+;; na-byte-of answers for it (mask to 0..255, fold the high half back), so storing
+;; it directly is the same value by a shorter route. Everything else — a flonum, a
+;; bignum, a value out of range, a non-fixnum index, a byte array whose backing an
+;; older image left boxed, and a LYING ^bytes hint on an array of any other kind —
+;; falls to na-array-set!. That fallback is not a slow path bolted on for this
+;; helper; it is the generic seam every unhinted aset takes, which is why the
+;; narrowing contract here is exactly the one it has always had.
+;;
+;; No index pre-check: bytevector-s8-set!'s own range check IS the array bounds
+;; contract here, as on the ^longs and ^doubles paths, and host-faults.ss already
+;; classifies a bytevector-s8-set! condition as an ArrayIndexOutOfBoundsException.
+;; A non-array receiver raises out of jolt-array-vec, exactly as it does for
+;; jolt-vaget/jolt-vaset.
+(define (jolt-baset a i v)
+  (let ((bk (jolt-array-vec a)))
+    (if (and (bytevector? bk) (fixnum? i) (fixnum? v) (fx<=? -128 v 127))
+        (begin (bytevector-s8-set! bk i v) v)
+        (na-array-set! a i v))))
+
 ;; A range condition escaping jolt-flaget/jolt-flaset IS the array bounds error
 ;; on the proven ^doubles path (a typed pre-check there costs ~1ns/access, ~11%
 ;; on an array-walking loop; wrapping in guard costs ~30ns/call). The catch

host/chez/run-flarr.ss

@@ -6,6 +6,10 @@
 ;; mirrors it: a proven index AND a :double value inline (flvector-set! ... v),
 ;; returning the stored value (JVM contract); an int value keeps (jolt-flaset ...)
 ;; (it owns exact->inexact). Covers both ^doubles PARAMS and ^doubles LET bindings.
+;;
+;; The tail of the file gates the OTHER primitive kinds' accesses: the direct
+;; backing read (jolt-vaget) they share, and the byte store's own helper
+;; (jolt-baset), which exists because that kind narrows at the store.
 (import (chezscheme))
 (load "host/chez/run-gate-harness.ss")
 (define analyze (var-deref "jolt.analyzer" "analyze"))
@@ -66,6 +70,33 @@
 (let ((e (emit-num "(def _ (fn [^doubles a ^long i] (aset a i 4)))")))
   (gate-check "(5b) aset ^doubles,int val keeps jolt-flaset (exact->inexact)" (gate-sub? e "jolt-flaset") #t))
 
+;; --- the other primitive kinds: the direct backing read, and the byte STORE ----
+;; ^longs/^ints/^bytes/^objects have no flvector to unbox, so their win is
+;; skipping jolt-nth's dispatch walk (jolt-vaget) and jolt-aset3's generic seam.
+;; The byte kind is the one that stores through its OWN helper: it narrows to
+;; signed 8 bits and must answer what it stored, which jolt-vaset (answering its
+;; argument) cannot do. Before jolt-baset existed the hint bought a byte store
+;; nothing — it emitted the same jolt-aset3 an unhinted one does.
+(let ((e (emit-num "(def _ (fn [^bytes a ^long i] (aset a i 7)))")))
+  (gate-check "(7) aset ^bytes -> jolt-baset" (gate-sub? e "(jolt-baset a i") #t)
+  (gate-check "(7) ...NOT the generic jolt-aset3" (gate-sub? e "jolt-aset3") #f)
+  (gate-check "(7) ...NOT jolt-vaset, which would answer the un-narrowed value" (gate-sub? e "jolt-vaset") #f))
+(let ((e (emit-num "(def _ (fn [^bytes a ^long i] (aget a i)))")))
+  (gate-check "(7a) aget ^bytes stays the direct backing read" (gate-sub? e "jolt-vaget") #t))
+(let ((e (emit-num "(def _ (fn [^longs a ^long i] (aset a i 7)))")))
+  (gate-check "(7b) aset ^longs keeps jolt-vaset" (gate-sub? e "jolt-vaset") #t)
+  (gate-check "(7b) ...and does not take the byte helper" (gate-sub? e "jolt-baset") #f))
+(let ((e (emit-num "(def _ (fn [^objects a ^long i] (aset a i 7)))")))
+  (gate-check "(7c) aset ^objects keeps jolt-vaset" (gate-sub? e "jolt-vaset") #t))
+(let ((e (emit-num "(def _ (fn [a i] (aset a i 7)))")))
+  (gate-check "(7d) an UNHINTED aset is untouched — still jolt-aset3" (gate-sub? e "jolt-aset3") #t)
+  (gate-check "(7d) ...and never the byte helper" (gate-sub? e "jolt-baset") #f))
+;; the bfill loop as it is actually written: the cast and the store are both
+;; direct calls now, so nothing in the body goes through a var.
+(let ((e (emit-num "(def _ (fn [^bytes a ^long n] (loop [i 0] (when (< i n) (aset a i (byte (bit-and i 127))) (recur (inc i))))))")))
+  (gate-check "(7e) a byte-fill loop is jolt-baset over jolt-byte-cast" (gate-sub? e "(jolt-baset a i (jolt-byte-cast") #t)
+  (gate-check "(7e) ...with no var-deref invoke left in it" (gate-sub? e "jolt-invoke1") #f))
+
 ;; --- runtime value semantics of jolt-flaget/jolt-flaset ---------------------------
 ;; Pin both index paths: the fixnum fast path (the hot case — loop counters are
 ;; :long) and the coercing slow path (flonum index floors via na-idx). Guards the