Recorded queries and the name cache (#149, third part)

README.md

@@ -26,7 +26,7 @@ so a test written against the reference reads the same here.
 | module | holds | state |
 |---|---|---|
 | `aephysics.math` | vectors, quaternions, transforms, 3x3 matrices, bounding boxes, segment distances, inertia helpers, the deterministic atan2/cos/sin, and the same vectors in single precision (`Vec3f`, `Quatf`, `Matrix3f`) for the geometry the reference keeps in `float` ([#42](https://github.com/aether-lang-dev/aephysics/issues/42)) | done, `test_math.ae` (6M checks) |
-| `aephysics.basics` | bit set, id pool, hash set, a long-to-int map, arrays, the stack and arena allocators, the content hash (the reference's core.c, named so a host's own `core` module and this one can share a program) | done, `test_basics.ae` (100k checks, test_container.c and test_allocator.c among them); `test_hash.ae` (36 checks: test_hash.c's properties on our hash, every bit and the length reaching it, families of real geometry never colliding) |
+| `aephysics.basics` | bit set, id pool, hash set, a long-to-int map, arrays, the stack and arena allocators, the content hash, the name cache (each body and shape name kept once per world under its 32-bit hash, the reference's name_cache.c, a collision taking the next free id instead of sharing the first name) (the reference's core.c, named so a host's own `core` module and this one can share a program) | done, `test_basics.ae` (100k checks, test_container.c and test_allocator.c among them, test_name_cache.c's CacheUnit with the reference's own hash values); `test_hash.ae` (36 checks: test_hash.c's properties on our hash, every bit and the length reaching it, families of real geometry never colliding) |
 | `aephysics.native` | the Aether face of the one C file built with every program: the threads' helpers Aether has not (a thread-local worker index, yield, pause, the processor count, atomics on an int in place, a semaphore) and the per-worker scratch every module keeps a block of | done |
 | `aephysics.parallel` | the reference's scheduler and `parallel_for`: worker threads made once and waiting on a semaphore, tasks in slots claimed by compare-and-swap, the main thread helping while it waits; a range in blocks the tasks claim, the caller working as worker 0 | done, `test_parallel.ae` (18 checks); [the step by worker count against the reference's](bench/RESULTS.md#parallel) |
 | `aephysics.dynamic_tree` | the bounding volume hierarchy under the broad phase: SAH insertion, rotations, enlarge, sweep refit, partial rebuild in depth-first order, box / closest / ray / swept-box queries, the tree saved and loaded as bytes (b3DynamicTree_Save and _Load; every corrupt input refused); its boxes in single precision rounded outward from double, as the reference's large-world build keeps them (32-byte nodes; the costs and rays measured on the widened boxes) | done, `test_dynamic_tree.ae` (15k checks: TreeSaveLoadRoundtrip among them, the loaded tree's queries against a scan, a scale, ten corruptions refused, and centimetre boxes at 1e7 each held by its leaf and found by every query that touches it); [same tree as the reference; inserts 0.4x its time, queries 1.1x, rays 0.9x](bench/RESULTS.md#dynamic_tree) |
@@ -52,7 +52,7 @@ so a test written against the reference reads the same here.
 | `aephysics.solver` | the Soft Step over the workers, the reference's stages: the work in blocks (the bodies', each colour's joints, wide and scalar constraints) that workers claim by compare-and-swap, the stages published as sync bits by worker 0 -- the constraints prepared, per sub-step the velocities integrated (gravity, damping, the gyroscopic torque), warm start, solve, positions, relax, colour by colour with the overflow first on worker 0; restitution, the impulses stored; the bodies finalised in parallel (sleep velocities, move events, fast bodies swept for the time of impact, bounds and proxies), the joint and hit events, the trees refit, bullets, the sensors' hits, islands put to sleep | `test_solver.ae` (45 checks: free fall against the closed form, resting and sleeping, a stack, a bounce and its hit event, a joint event, a pendulum, a fast sphere stopped by a thin wall, a bullet, a sensor swept, two worlds bit for bit alike) | [1.2-1.3x the reference's step, stage by stage](bench/RESULTS.md#stage-by-stage) |
 | `aephysics.physics_world` | the world's face: the step (pairs, narrow phase, solve, sensors, events), the events read back, the settings and counters, queries over every shape (overlap of a box or a proxy, the mover's planes, ray, shape and mover casts, the closest ray hit) and against one body at a transform of the caller's, explosions | `test_physics_world.ae` (259 checks: test_world.c whole, among it the hull database (equal hulls one world-owned copy, a caller's hull freed at once, transformed hulls baked once, every holder's release), empty and recycled worlds, every setting, the worker count set between steps, contact, hit, move and sensor events, a compound child's hit events and materials, the overflow colour pile, the explosion near and far, the bullet and sleep flags, the compound count, an enlarged proxy destroyed; test_body_query.c whole: casts, overlaps of hull proxies and turned compounds, mover planes, and the mover's time of impact against 2,000 random sweeps; test_broad_phase.c's static rebuild and its stress test, a swarm through every proxy change with the pair set checked by brute force every step; test_id.c; the world queries; a wave pile stepping alike twice) | [the reference's own benchmark scenes with the same checksums: 1.2x its AVX2 build on the large pyramid, 1.15x on the many pyramids, 1.05x on the joint grid, ours at eight lanes in AVX2 with float records (#121)](bench/RESULTS.md#the-collide-pass-as-the-reference-keeps-it-2026-10-09-121); [the rest of its suite, nine scenes and the SAT runs, at 1.04-1.21x](bench/RESULTS.md#suite-the-references-benchmark-suite) |
 | `aephysics.snapshot` | a world as bytes and back (the reference's world_snapshot.c): `save_world` and `load_world` keep everything a step reads (the settings, the id pools, the solver sets, bodies, shapes and their geometry once each, contacts with their manifolds and mesh caches, joints, sensors, islands, the trees and pair set, the graph's colours), so a loaded world steps on as the one it was taken from, bit for bit; `restore_world` loads one into an existing world in place, its id, workers and callbacks kept (what a replay rewinds with); a snapshot of another layout or any damage to its bytes loads no world | done, `test_snapshot.ae` (63 checks: a world of every kind of thing loaded at four moments and stepped 150 steps beside its original, the same hash and events every step; a loaded world saves to the same bytes; one worker's world loaded with four steps alike; a restore in place stepping alike and saving to the same bytes, a refused restore leaving the world empty and whole; every truncation and corruption refused, nothing leaked) |
-| `aephysics.recording` | recording and replay (the reference's recording.c and recording_replay.c): a world records every call that changes it, 160 kinds, field by field, a create's record with the id it made, a state hash after every step and the geometry the calls name once each; a player replays it into a world of its own, restored in place so its id stays, checking every step's hash and every created id, with keyframes under a memory budget for seeking, bodies in the order they were made, a pause before a step that makes bodies, and the recording's info; records it does not know are skipped, as a newer minor version may add some | `test_recording.ae` (116 checks: the snapshot test's scene recorded from an empty world and from mid stream with the host changing it between steps, played on one worker and several, every frame's hash as recorded; every one of the 160 calls recorded and replayed (the reference's AllOps); geometry kept once from two copies; seeking back and forth with and without keyframes and under a tight budget; a file round trip; every truncation, a skipped call, a changed id, a short record and a changed hash each caught, nothing leaked). Not yet: the queries and the name cache (#149) |
+| `aephysics.recording` | recording and replay (the reference's recording.c and recording_replay.c): a world records every call that changes it, 160 kinds, field by field, a create's record with the id it made, a state hash after every step and the geometry the calls name once each, and every world query with each visit and the caller's answer (a caller named by the query filter's id and name gets a key a viewer follows it by); a player replays it into a world of its own, restored in place so its id stays, checking every step's hash and every created id, asking every recorded query again with the recorded answers and comparing each visit, with keyframes under a memory budget for seeking, bodies in the order they were made, the queries of the frame last played with their hits, a pause before a step that makes bodies, and the recording's info; records it does not know are skipped, as a newer minor version may add some | `test_recording.ae` (152 checks: the snapshot test's scene recorded from an empty world and from mid stream with the host changing it between steps, played on one worker and several, every frame's hash as recorded; every one of the 160 calls recorded and replayed (the reference's AllOps); geometry kept once from two copies; seeking back and forth with and without keyframes and under a tight budget; the reference's QueryReplay and TaggedQuery, every query kind with answers that stop and clip them, a changed hit caught at its frame; the reference's NameRoundTrip, RollbackNames, GeometryHashCollision and ReservedHeaderBytes; a file round trip; every truncation, a skipped call, a changed id, a short record and a changed hash each caught, nothing leaked); `test_threads.ae` (four threads querying a recording world, every record whole and replayed alike) |
 | `aephysics.human` | the ragdoll: twelve capsule bones on spherical joints with cone and twist limits and revolute joints with angle limits -- a person's ranges by default (shoulders that reach overhead on a cone centred off the rest, hips on an elliptical cone through 125 degrees ahead, 25 back, 45 out and 30 across, knees and elbows that never pass straight), the reference's tight ones on request -- soles that grip, and on request (`Human.ankles`, past the reference) the feet as bones of their own on ankles, an elliptical cone through 50 degrees down, 35 up and 15 aside with 15 of roll, so a crouched figure's soles lie flat, a spring on every joint toward the reference pose, a motor whose torque limit is joint friction, filter joints for the limbs that clash; the align spring, kinematic anchors through motor or parallel joints and their targets driven over a step (the pose drive of an active ragdoll), velocity, kicks, bullets | `test_human.ae` (115 checks: the figure's shape, a fall to rest in one piece, the setters, a kick, standing under the align spring, the pose held on anchors, the anchors driven two metres and the figure walked behind them, the same drop twice bit for bit, a fitted shape, each arm reaching flexion 170, abduction 160 and extension 45 degrees standing and prone, the hips' cone through a person's four ranges and short of a split, the hinges stopping at straight and folding to their range, the soles' grip, the ankles' four ranges measured through the joint and the soles flat under shins tipped 25 degrees over them) | [the reference's rain benchmark, 300 ragdolls over 400 steps, at 0.9x](bench/RESULTS.md#human) |
 | `aephysics` | the public API | |
 

aephysics/basics/module.ae

@@ -34,6 +34,9 @@ exports (
     set_count, set_bytes, set_prefetch, SetItem,
     LongMap, map_create, map_destroy, map_clear, map_set, map_get, map_has, map_remove, map_count, map_bytes,
     hash_bytes, section_fits,
+    NameCache, NULL_NAME, name_cache_create, name_cache_destroy, name_hash32, name_cache_add, name_cache_find,
+    name_cache_text, name_cache_load, name_cache_count, name_cache_id_at, name_cache_name_at, name_cache_length_at,
+    name_cache_bytes,
     ints_create, ints_destroy, ints_reserve, ints_push, ints_pop, ints_get, ints_set, ints_clear,
     ints_remove_swap, ints_count, ints_capacity, ints_bytes,
     buffer_create, buffer_destroy, buffer_reserve, buffer_push, buffer_pop, buffer_clear,
@@ -50,6 +53,9 @@ extern free(block: ptr)
 extern memset(block: ptr, value: int, size: int) -> ptr
 extern memcpy(dst: ptr, src: ptr, size: int) -> ptr
 extern memmove(dst: ptr, src: ptr, size: int) -> ptr
+extern memcmp(a: ptr, b: ptr, size: int) -> int
+extern strlen(s: ptr) -> int
+extern memchr(block: ptr, value: int, size: int) -> ptr
 
 // --- allocation, counted ---------------------------------------------------
 
@@ -392,6 +398,151 @@ map_remove(m: *LongMap, key: long) -> bool {
 map_count(m: *LongMap) -> int { return m.count }
 map_bytes(m: *LongMap) -> int { return m.capacity * sizeof(MapItem) }
 
+// --- the name cache ------------------------------------------------------------
+
+// The names a world's bodies and shapes go by, each kept once (the
+// reference's name_cache.c): a body or shape holds its name's id, never a
+// copy of its own, so a record of it copies whole and a snapshot carries
+// every name once. The id is the name's 32-bit hash (name_hash32), and
+// NULL_NAME for no name or an empty one. A different name on an id already
+// taken goes to the next free id, where the reference gives both names the
+// first one. Names stay until the cache goes: they are for debugging, and
+// few.
+const NULL_NAME = 0
+
+struct NameEntry {
+    id: int
+    length: int
+    name: ptr          // the cache's own copy, its length and a terminating zero (alloc's)
+}
+
+struct NameCache {
+    entries: Buffer    // NameEntry
+    ids: LongMap       // a name's id to its entry
+}
+
+name_cache_create() -> NameCache {
+    return NameCache { entries: buffer_create(sizeof(NameEntry), 0), ids: map_create(16) }
+}
+
+name_entry_at(c: *NameCache, index: int) -> *NameEntry { return (c.entries.data + index * sizeof(NameEntry)) as *NameEntry }
+
+name_cache_destroy(c: *NameCache) {
+    i = 0
+    while i < c.entries.count {
+        entry = name_entry_at(c, i)
+        free_bytes(entry.name, entry.length + 1)
+        i = i + 1
+    }
+    buffer_destroy(&c.entries)
+    map_destroy(&c.ids)
+}
+
+// The bytes' 32-bit hash, unsigned: FNV-1a, then Murmur3's finish so short
+// names spread (the reference's b3Hash32; a recording's names keep their
+// ids only while it stays the same).
+name_hash32(data: ptr, length: int) -> long {
+    mask = (1 as long << 32) - (1 as long)
+    h = 0x811c9dc5 as long
+    i = 0
+    while i < length {
+        h = ((h ^ (native.load_u8(data, i as long) as long)) * (0x01000193 as long)) & mask
+        i = i + 1
+    }
+    h = h ^ (h >> 16)
+    h = (h * (0x85ebca6b as long)) & mask
+    h = h ^ (h >> 13)
+    h = (h * (0xc2b2ae35 as long)) & mask
+    h = h ^ (h >> 16)
+    return h
+}
+
+// The entry an id names, or NULL_INDEX.
+name_entry_of(c: *NameCache, id: int) -> int {
+    if id == NULL_NAME { return 0 - 1 }
+    return map_get(&c.ids, id as long, 0 - 1)
+}
+
+// The id of `name`, interned when it is new; NULL_NAME for none.
+name_cache_add(c: *NameCache, name: string) -> int {
+    text = native.name_copy(name)
+    if text == null { return NULL_NAME }
+    length = strlen(text)
+    id = NULL_NAME
+    if length > 0 { id = name_cache_add_bytes(c, text, length) }
+    native.name_free(text)
+    return id
+}
+
+// The id of the `length` bytes at `text`, interned when they are new.
+name_cache_add_bytes(c: *NameCache, text: ptr, length: int) -> int {
+    id = (name_hash32(text, length) as int)
+    if id == NULL_NAME { id = 1 }
+    while true {
+        index = name_entry_of(c, id)
+        if index < 0 { break }
+        entry = name_entry_at(c, index)
+        if entry.length == length && memcmp(entry.name, text, length) == 0 { return id }
+        // Another name on this id: the next one.
+        id = id + 1
+        if id == NULL_NAME { id = 1 }
+    }
+    copy = alloc(length + 1)
+    memcpy(copy, text, length)
+    name_cache_put(c, id, copy, length)
+    return id
+}
+
+name_cache_put(c: *NameCache, id: int, name: ptr, length: int) {
+    index = buffer_push(&c.entries)
+    entry = name_entry_at(c, index)
+    entry.id = id
+    entry.length = length
+    entry.name = name
+    map_set(&c.ids, id as long, index)
+}
+
+// The name an id stands for, as a C string; null for NULL_NAME or an id
+// the cache does not hold.
+name_cache_find(c: *NameCache, id: int) -> ptr {
+    index = name_entry_of(c, id)
+    if index < 0 { return null }
+    return name_entry_at(c, index).name
+}
+
+// The name an id stands for; empty for none.
+name_cache_text(c: *NameCache, id: int) -> string { return native.name_text(name_cache_find(c, id)) }
+
+// A name read back with its id (a snapshot's): the `length` bytes at
+// `text` copied in. False for NULL_NAME, an empty name, a name with a zero
+// in it, or an id the cache already holds.
+name_cache_load(c: *NameCache, id: int, text: ptr, length: int) -> bool {
+    if id == NULL_NAME || length <= 0 || name_entry_of(c, id) >= 0 { return false }
+    if memchr(text, 0, length) != null { return false }
+    copy = alloc(length + 1)
+    memcpy(copy, text, length)
+    name_cache_put(c, id, copy, length)
+    return true
+}
+
+name_cache_count(c: *NameCache) -> int { return c.entries.count }
+
+// The entry at `index`, in the order they came: its id, length and text.
+name_cache_id_at(c: *NameCache, index: int) -> int { return name_entry_at(c, index).id }
+name_cache_name_at(c: *NameCache, index: int) -> ptr { return name_entry_at(c, index).name }
+name_cache_length_at(c: *NameCache, index: int) -> int { return name_entry_at(c, index).length }
+
+// What the cache holds, in bytes.
+name_cache_bytes(c: *NameCache) -> int {
+    total = buffer_bytes(&c.entries) + map_bytes(&c.ids)
+    i = 0
+    while i < c.entries.count {
+        total = total + name_entry_at(c, i).length + 1
+        i = i + 1
+    }
+    return total
+}
+
 // --- a section of a block ------------------------------------------------------
 
 // `count` items of `size` bytes from `offset` lie inside a block of