From cd75650045eb4adcec684098484636014032f9c4 Mon Sep 17 00:00:00 2001 From: Nicolas Maman Date: Fri, 9 Oct 2026 04:19:26 -0300 Subject: [PATCH 1/2] Recorded queries and the name cache (#149, third part) The seven world queries record each visit with the caller's answer and are asked again on replay with the same answers, every visit compared; a query filter's id and name tag its caller; the player keeps the frame's queries and hits. Records go in under the recording's lock, so queries from several threads stay whole. Names are interned once per world under the reference's 32-bit hash (a collision takes the next free id); bodies and shapes hold ids, and the snapshot (version 2) carries the cache once. --- README.md | 4 +- aephysics/basics/module.ae | 151 +++++++ aephysics/dynamics/module.ae | 253 ++++++++++-- aephysics/physics_world/module.ae | 269 ++++++++++++- aephysics/recording/module.ae | 639 +++++++++++++++++++++++++++++- aephysics/shape/module.ae | 8 +- aephysics/snapshot/module.ae | 73 ++-- design.md | 11 +- tests/test_basics.ae | 53 +++ tests/test_recording.ae | 358 +++++++++++++++++ tests/test_threads.ae | 45 ++- 11 files changed, 1752 insertions(+), 112 deletions(-) diff --git a/README.md b/README.md index 7879bba..c967b03 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ so a test written against the reference reads the same here. | module | holds | state | |---|---|---| | `aephysics.math` | vectors, quaternions, transforms, 3x3 matrices, bounding boxes, segment distances, inertia helpers, the deterministic atan2/cos/sin, and the same vectors in single precision (`Vec3f`, `Quatf`, `Matrix3f`) for the geometry the reference keeps in `float` ([#42](https://github.com/aether-lang-dev/aephysics/issues/42)) | done, `test_math.ae` (6M checks) | -| `aephysics.basics` | bit set, id pool, hash set, a long-to-int map, arrays, the stack and arena allocators, the content hash (the reference's core.c, named so a host's own `core` module and this one can share a program) | done, `test_basics.ae` (100k checks, test_container.c and test_allocator.c among them); `test_hash.ae` (36 checks: test_hash.c's properties on our hash, every bit and the length reaching it, families of real geometry never colliding) | +| `aephysics.basics` | bit set, id pool, hash set, a long-to-int map, arrays, the stack and arena allocators, the content hash, the name cache (each body and shape name kept once per world under its 32-bit hash, the reference's name_cache.c, a collision taking the next free id instead of sharing the first name) (the reference's core.c, named so a host's own `core` module and this one can share a program) | done, `test_basics.ae` (100k checks, test_container.c and test_allocator.c among them, test_name_cache.c's CacheUnit with the reference's own hash values); `test_hash.ae` (36 checks: test_hash.c's properties on our hash, every bit and the length reaching it, families of real geometry never colliding) | | `aephysics.native` | the Aether face of the one C file built with every program: the threads' helpers Aether has not (a thread-local worker index, yield, pause, the processor count, atomics on an int in place, a semaphore) and the per-worker scratch every module keeps a block of | done | | `aephysics.parallel` | the reference's scheduler and `parallel_for`: worker threads made once and waiting on a semaphore, tasks in slots claimed by compare-and-swap, the main thread helping while it waits; a range in blocks the tasks claim, the caller working as worker 0 | done, `test_parallel.ae` (18 checks); [the step by worker count against the reference's](bench/RESULTS.md#parallel) | | `aephysics.dynamic_tree` | the bounding volume hierarchy under the broad phase: SAH insertion, rotations, enlarge, sweep refit, partial rebuild in depth-first order, box / closest / ray / swept-box queries, the tree saved and loaded as bytes (b3DynamicTree_Save and _Load; every corrupt input refused); its boxes in single precision rounded outward from double, as the reference's large-world build keeps them (32-byte nodes; the costs and rays measured on the widened boxes) | done, `test_dynamic_tree.ae` (15k checks: TreeSaveLoadRoundtrip among them, the loaded tree's queries against a scan, a scale, ten corruptions refused, and centimetre boxes at 1e7 each held by its leaf and found by every query that touches it); [same tree as the reference; inserts 0.4x its time, queries 1.1x, rays 0.9x](bench/RESULTS.md#dynamic_tree) | @@ -52,7 +52,7 @@ so a test written against the reference reads the same here. | `aephysics.solver` | the Soft Step over the workers, the reference's stages: the work in blocks (the bodies', each colour's joints, wide and scalar constraints) that workers claim by compare-and-swap, the stages published as sync bits by worker 0 -- the constraints prepared, per sub-step the velocities integrated (gravity, damping, the gyroscopic torque), warm start, solve, positions, relax, colour by colour with the overflow first on worker 0; restitution, the impulses stored; the bodies finalised in parallel (sleep velocities, move events, fast bodies swept for the time of impact, bounds and proxies), the joint and hit events, the trees refit, bullets, the sensors' hits, islands put to sleep | `test_solver.ae` (45 checks: free fall against the closed form, resting and sleeping, a stack, a bounce and its hit event, a joint event, a pendulum, a fast sphere stopped by a thin wall, a bullet, a sensor swept, two worlds bit for bit alike) | [1.2-1.3x the reference's step, stage by stage](bench/RESULTS.md#stage-by-stage) | | `aephysics.physics_world` | the world's face: the step (pairs, narrow phase, solve, sensors, events), the events read back, the settings and counters, queries over every shape (overlap of a box or a proxy, the mover's planes, ray, shape and mover casts, the closest ray hit) and against one body at a transform of the caller's, explosions | `test_physics_world.ae` (259 checks: test_world.c whole, among it the hull database (equal hulls one world-owned copy, a caller's hull freed at once, transformed hulls baked once, every holder's release), empty and recycled worlds, every setting, the worker count set between steps, contact, hit, move and sensor events, a compound child's hit events and materials, the overflow colour pile, the explosion near and far, the bullet and sleep flags, the compound count, an enlarged proxy destroyed; test_body_query.c whole: casts, overlaps of hull proxies and turned compounds, mover planes, and the mover's time of impact against 2,000 random sweeps; test_broad_phase.c's static rebuild and its stress test, a swarm through every proxy change with the pair set checked by brute force every step; test_id.c; the world queries; a wave pile stepping alike twice) | [the reference's own benchmark scenes with the same checksums: 1.2x its AVX2 build on the large pyramid, 1.15x on the many pyramids, 1.05x on the joint grid, ours at eight lanes in AVX2 with float records (#121)](bench/RESULTS.md#the-collide-pass-as-the-reference-keeps-it-2026-10-09-121); [the rest of its suite, nine scenes and the SAT runs, at 1.04-1.21x](bench/RESULTS.md#suite-the-references-benchmark-suite) | | `aephysics.snapshot` | a world as bytes and back (the reference's world_snapshot.c): `save_world` and `load_world` keep everything a step reads (the settings, the id pools, the solver sets, bodies, shapes and their geometry once each, contacts with their manifolds and mesh caches, joints, sensors, islands, the trees and pair set, the graph's colours), so a loaded world steps on as the one it was taken from, bit for bit; `restore_world` loads one into an existing world in place, its id, workers and callbacks kept (what a replay rewinds with); a snapshot of another layout or any damage to its bytes loads no world | done, `test_snapshot.ae` (63 checks: a world of every kind of thing loaded at four moments and stepped 150 steps beside its original, the same hash and events every step; a loaded world saves to the same bytes; one worker's world loaded with four steps alike; a restore in place stepping alike and saving to the same bytes, a refused restore leaving the world empty and whole; every truncation and corruption refused, nothing leaked) | -| `aephysics.recording` | recording and replay (the reference's recording.c and recording_replay.c): a world records every call that changes it, 160 kinds, field by field, a create's record with the id it made, a state hash after every step and the geometry the calls name once each; a player replays it into a world of its own, restored in place so its id stays, checking every step's hash and every created id, with keyframes under a memory budget for seeking, bodies in the order they were made, a pause before a step that makes bodies, and the recording's info; records it does not know are skipped, as a newer minor version may add some | `test_recording.ae` (116 checks: the snapshot test's scene recorded from an empty world and from mid stream with the host changing it between steps, played on one worker and several, every frame's hash as recorded; every one of the 160 calls recorded and replayed (the reference's AllOps); geometry kept once from two copies; seeking back and forth with and without keyframes and under a tight budget; a file round trip; every truncation, a skipped call, a changed id, a short record and a changed hash each caught, nothing leaked). Not yet: the queries and the name cache (#149) | +| `aephysics.recording` | recording and replay (the reference's recording.c and recording_replay.c): a world records every call that changes it, 160 kinds, field by field, a create's record with the id it made, a state hash after every step and the geometry the calls name once each, and every world query with each visit and the caller's answer (a caller named by the query filter's id and name gets a key a viewer follows it by); a player replays it into a world of its own, restored in place so its id stays, checking every step's hash and every created id, asking every recorded query again with the recorded answers and comparing each visit, with keyframes under a memory budget for seeking, bodies in the order they were made, the queries of the frame last played with their hits, a pause before a step that makes bodies, and the recording's info; records it does not know are skipped, as a newer minor version may add some | `test_recording.ae` (152 checks: the snapshot test's scene recorded from an empty world and from mid stream with the host changing it between steps, played on one worker and several, every frame's hash as recorded; every one of the 160 calls recorded and replayed (the reference's AllOps); geometry kept once from two copies; seeking back and forth with and without keyframes and under a tight budget; the reference's QueryReplay and TaggedQuery, every query kind with answers that stop and clip them, a changed hit caught at its frame; the reference's NameRoundTrip, RollbackNames, GeometryHashCollision and ReservedHeaderBytes; a file round trip; every truncation, a skipped call, a changed id, a short record and a changed hash each caught, nothing leaked); `test_threads.ae` (four threads querying a recording world, every record whole and replayed alike) | | `aephysics.human` | the ragdoll: twelve capsule bones on spherical joints with cone and twist limits and revolute joints with angle limits -- a person's ranges by default (shoulders that reach overhead on a cone centred off the rest, hips on an elliptical cone through 125 degrees ahead, 25 back, 45 out and 30 across, knees and elbows that never pass straight), the reference's tight ones on request -- soles that grip, and on request (`Human.ankles`, past the reference) the feet as bones of their own on ankles, an elliptical cone through 50 degrees down, 35 up and 15 aside with 15 of roll, so a crouched figure's soles lie flat, a spring on every joint toward the reference pose, a motor whose torque limit is joint friction, filter joints for the limbs that clash; the align spring, kinematic anchors through motor or parallel joints and their targets driven over a step (the pose drive of an active ragdoll), velocity, kicks, bullets | `test_human.ae` (115 checks: the figure's shape, a fall to rest in one piece, the setters, a kick, standing under the align spring, the pose held on anchors, the anchors driven two metres and the figure walked behind them, the same drop twice bit for bit, a fitted shape, each arm reaching flexion 170, abduction 160 and extension 45 degrees standing and prone, the hips' cone through a person's four ranges and short of a split, the hinges stopping at straight and folding to their range, the soles' grip, the ankles' four ranges measured through the joint and the soles flat under shins tipped 25 degrees over them) | [the reference's rain benchmark, 300 ragdolls over 400 steps, at 0.9x](bench/RESULTS.md#human) | | `aephysics` | the public API | | diff --git a/aephysics/basics/module.ae b/aephysics/basics/module.ae index 375b190..56f30bb 100644 --- a/aephysics/basics/module.ae +++ b/aephysics/basics/module.ae @@ -34,6 +34,9 @@ exports ( set_count, set_bytes, set_prefetch, SetItem, LongMap, map_create, map_destroy, map_clear, map_set, map_get, map_has, map_remove, map_count, map_bytes, hash_bytes, section_fits, + NameCache, NULL_NAME, name_cache_create, name_cache_destroy, name_hash32, name_cache_add, name_cache_find, + name_cache_text, name_cache_load, name_cache_count, name_cache_id_at, name_cache_name_at, name_cache_length_at, + name_cache_bytes, ints_create, ints_destroy, ints_reserve, ints_push, ints_pop, ints_get, ints_set, ints_clear, ints_remove_swap, ints_count, ints_capacity, ints_bytes, buffer_create, buffer_destroy, buffer_reserve, buffer_push, buffer_pop, buffer_clear, @@ -50,6 +53,9 @@ extern free(block: ptr) extern memset(block: ptr, value: int, size: int) -> ptr extern memcpy(dst: ptr, src: ptr, size: int) -> ptr extern memmove(dst: ptr, src: ptr, size: int) -> ptr +extern memcmp(a: ptr, b: ptr, size: int) -> int +extern strlen(s: ptr) -> int +extern memchr(block: ptr, value: int, size: int) -> ptr // --- allocation, counted --------------------------------------------------- @@ -392,6 +398,151 @@ map_remove(m: *LongMap, key: long) -> bool { map_count(m: *LongMap) -> int { return m.count } map_bytes(m: *LongMap) -> int { return m.capacity * sizeof(MapItem) } +// --- the name cache ------------------------------------------------------------ + +// The names a world's bodies and shapes go by, each kept once (the +// reference's name_cache.c): a body or shape holds its name's id, never a +// copy of its own, so a record of it copies whole and a snapshot carries +// every name once. The id is the name's 32-bit hash (name_hash32), and +// NULL_NAME for no name or an empty one. A different name on an id already +// taken goes to the next free id, where the reference gives both names the +// first one. Names stay until the cache goes: they are for debugging, and +// few. +const NULL_NAME = 0 + +struct NameEntry { + id: int + length: int + name: ptr // the cache's own copy, its length and a terminating zero (alloc's) +} + +struct NameCache { + entries: Buffer // NameEntry + ids: LongMap // a name's id to its entry +} + +name_cache_create() -> NameCache { + return NameCache { entries: buffer_create(sizeof(NameEntry), 0), ids: map_create(16) } +} + +name_entry_at(c: *NameCache, index: int) -> *NameEntry { return (c.entries.data + index * sizeof(NameEntry)) as *NameEntry } + +name_cache_destroy(c: *NameCache) { + i = 0 + while i < c.entries.count { + entry = name_entry_at(c, i) + free_bytes(entry.name, entry.length + 1) + i = i + 1 + } + buffer_destroy(&c.entries) + map_destroy(&c.ids) +} + +// The bytes' 32-bit hash, unsigned: FNV-1a, then Murmur3's finish so short +// names spread (the reference's b3Hash32; a recording's names keep their +// ids only while it stays the same). +name_hash32(data: ptr, length: int) -> long { + mask = (1 as long << 32) - (1 as long) + h = 0x811c9dc5 as long + i = 0 + while i < length { + h = ((h ^ (native.load_u8(data, i as long) as long)) * (0x01000193 as long)) & mask + i = i + 1 + } + h = h ^ (h >> 16) + h = (h * (0x85ebca6b as long)) & mask + h = h ^ (h >> 13) + h = (h * (0xc2b2ae35 as long)) & mask + h = h ^ (h >> 16) + return h +} + +// The entry an id names, or NULL_INDEX. +name_entry_of(c: *NameCache, id: int) -> int { + if id == NULL_NAME { return 0 - 1 } + return map_get(&c.ids, id as long, 0 - 1) +} + +// The id of `name`, interned when it is new; NULL_NAME for none. +name_cache_add(c: *NameCache, name: string) -> int { + text = native.name_copy(name) + if text == null { return NULL_NAME } + length = strlen(text) + id = NULL_NAME + if length > 0 { id = name_cache_add_bytes(c, text, length) } + native.name_free(text) + return id +} + +// The id of the `length` bytes at `text`, interned when they are new. +name_cache_add_bytes(c: *NameCache, text: ptr, length: int) -> int { + id = (name_hash32(text, length) as int) + if id == NULL_NAME { id = 1 } + while true { + index = name_entry_of(c, id) + if index < 0 { break } + entry = name_entry_at(c, index) + if entry.length == length && memcmp(entry.name, text, length) == 0 { return id } + // Another name on this id: the next one. + id = id + 1 + if id == NULL_NAME { id = 1 } + } + copy = alloc(length + 1) + memcpy(copy, text, length) + name_cache_put(c, id, copy, length) + return id +} + +name_cache_put(c: *NameCache, id: int, name: ptr, length: int) { + index = buffer_push(&c.entries) + entry = name_entry_at(c, index) + entry.id = id + entry.length = length + entry.name = name + map_set(&c.ids, id as long, index) +} + +// The name an id stands for, as a C string; null for NULL_NAME or an id +// the cache does not hold. +name_cache_find(c: *NameCache, id: int) -> ptr { + index = name_entry_of(c, id) + if index < 0 { return null } + return name_entry_at(c, index).name +} + +// The name an id stands for; empty for none. +name_cache_text(c: *NameCache, id: int) -> string { return native.name_text(name_cache_find(c, id)) } + +// A name read back with its id (a snapshot's): the `length` bytes at +// `text` copied in. False for NULL_NAME, an empty name, a name with a zero +// in it, or an id the cache already holds. +name_cache_load(c: *NameCache, id: int, text: ptr, length: int) -> bool { + if id == NULL_NAME || length <= 0 || name_entry_of(c, id) >= 0 { return false } + if memchr(text, 0, length) != null { return false } + copy = alloc(length + 1) + memcpy(copy, text, length) + name_cache_put(c, id, copy, length) + return true +} + +name_cache_count(c: *NameCache) -> int { return c.entries.count } + +// The entry at `index`, in the order they came: its id, length and text. +name_cache_id_at(c: *NameCache, index: int) -> int { return name_entry_at(c, index).id } +name_cache_name_at(c: *NameCache, index: int) -> ptr { return name_entry_at(c, index).name } +name_cache_length_at(c: *NameCache, index: int) -> int { return name_entry_at(c, index).length } + +// What the cache holds, in bytes. +name_cache_bytes(c: *NameCache) -> int { + total = buffer_bytes(&c.entries) + map_bytes(&c.ids) + i = 0 + while i < c.entries.count { + total = total + name_entry_at(c, i).length + 1 + i = i + 1 + } + return total +} + // --- a section of a block ------------------------------------------------------ // `count` items of `size` bytes from `offset` lie inside a block of diff --git a/aephysics/dynamics/module.ae b/aephysics/dynamics/module.ae index b61a506..e41c085 100644 --- a/aephysics/dynamics/module.ae +++ b/aephysics/dynamics/module.ae @@ -90,8 +90,11 @@ exports ( create_sphere_shape, create_capsule_shape, create_hull_shape, create_transformed_hull_shape, create_mesh_shape, create_height_field_shape, HullEntry, world_hull_count, add_hull_to_database, allocate_manifolds, OwnedGeometry, adopt_geometry, world_at, destroy_solver_set, Recording, RecordedGeometry, RECORDING_MAGIC, RECORDING_MAJOR_VERSION, RECORDING_MINOR_VERSION, RECORDING_HEADER_BYTES, recording_on, rec_begin, rec_end, rec_int, rec_long, + RecordedTag, recorded_tag_at, rec_lock, rec_unlock, rec_payload_create, rec_payload_destroy, rec_patch_int, query_tag_key, + intern_tag, clear_tags, intern_geometry, rec_commit_query, rec_aabb, rec_query_filter, rec_shape_proxy, rec_plane_result, rec_capsule_value, rec_float, rec_bool, rec_vec3, rec_quat, rec_transform, rec_id, rec_string, rec_raw, rec_geometry, finish_recording, recorded_geometry_at, - OP_STATE_HASH, OP_BOUNDS, OP_END, OP_CREATE_BODY, OP_DESTROY_BODY, OP_BODY_SET_TRANSFORM, OP_BODY_SET_LINEAR_VELOCITY, OP_BODY_SET_ANGULAR_VELOCITY, OP_BODY_SET_TARGET_TRANSFORM, OP_BODY_APPLY_FORCE, OP_BODY_APPLY_FORCE_TO_CENTER, OP_BODY_APPLY_TORQUE, OP_BODY_APPLY_LINEAR_IMPULSE, OP_BODY_APPLY_LINEAR_IMPULSE_TO_CENTER, OP_BODY_APPLY_ANGULAR_IMPULSE, OP_BODY_SET_BULLET, OP_BODY_SET_MASS_DATA, OP_BODY_APPLY_MASS_FROM_SHAPES, OP_BODY_SET_LINEAR_DAMPING, OP_BODY_SET_ANGULAR_DAMPING, OP_BODY_SET_GRAVITY_SCALE, OP_BODY_SET_AWAKE, OP_BODY_ENABLE_SLEEP, OP_BODY_SET_SLEEP_THRESHOLD, OP_BODY_SET_MOTION_LOCKS, OP_CREATE_SPHERE_SHAPE, OP_CREATE_CAPSULE_SHAPE, OP_CREATE_HULL_SHAPE, OP_CREATE_TRANSFORMED_HULL_SHAPE, OP_CREATE_MESH_SHAPE, OP_CREATE_HEIGHT_FIELD_SHAPE, OP_CREATE_COMPOUND_SHAPE, OP_DESTROY_SHAPE, OP_SHAPE_SET_DENSITY, OP_SHAPE_SET_FILTER, OP_SHAPE_SET_SURFACE_MATERIAL, OP_SHAPE_SET_SPHERE, OP_SHAPE_SET_CAPSULE, OP_SHAPE_SET_HULL, OP_SHAPE_SET_MESH, OP_SHAPE_SET_FRICTION, OP_SHAPE_SET_RESTITUTION, OP_SHAPE_SET_MESH_MATERIAL, OP_SHAPE_ENABLE_PRE_SOLVE_EVENTS, OP_BODY_ENABLE_HIT_EVENTS, OP_BODY_ALLOW_FAST_ROTATION, OP_BODY_ENABLE_CONTACT_RECYCLING, OP_BODY_SET_SAFETY_FACTOR, OP_SHAPE_APPLY_WIND, OP_BODY_SET_NAME, OP_SHAPE_SET_NAME, OP_CREATE_DISTANCE_JOINT, OP_CREATE_MOTOR_JOINT, OP_CREATE_FILTER_JOINT, OP_CREATE_PARALLEL_JOINT, OP_CREATE_PRISMATIC_JOINT, OP_CREATE_REVOLUTE_JOINT, OP_CREATE_SPHERICAL_JOINT, OP_CREATE_WELD_JOINT, OP_CREATE_WHEEL_JOINT, OP_DESTROY_JOINT, OP_JOINT_SET_LOCAL_FRAME_A, OP_JOINT_SET_LOCAL_FRAME_B, OP_JOINT_SET_COLLIDE_CONNECTED, OP_JOINT_WAKE_BODIES, OP_JOINT_SET_CONSTRAINT_TUNING, OP_JOINT_SET_FORCE_THRESHOLD, OP_JOINT_SET_TORQUE_THRESHOLD, OP_BODY_SET_TYPE, OP_BODY_DISABLE, OP_BODY_ENABLE, OP_SHAPE_ENABLE_SENSOR_EVENTS, OP_SHAPE_ENABLE_CONTACT_EVENTS, OP_SHAPE_ENABLE_HIT_EVENTS, OP_STEP, OP_ENABLE_SLEEPING, OP_ENABLE_WARM_STARTING, OP_ENABLE_CONTINUOUS, OP_ENABLE_SPECULATIVE, OP_SET_RESTITUTION_THRESHOLD, OP_SET_RESTITUTION_ITERATIONS, OP_ENABLE_RESTITUTION_PROPAGATION, OP_ENABLE_KINEMATIC_STATIC_SOFTNESS, OP_SET_HIT_EVENT_THRESHOLD, OP_SET_CONTACT_TUNING, OP_SET_CONTACT_RECYCLE_DISTANCE, OP_SET_MAXIMUM_LINEAR_SPEED, OP_SET_GRAVITY, OP_REBUILD_STATIC_TREE, OP_EXPLODE, OP_DISTANCE_JOINT_SET_LENGTH, OP_DISTANCE_JOINT_ENABLE_LIMIT, OP_DISTANCE_JOINT_SET_LENGTH_RANGE, OP_DISTANCE_JOINT_ENABLE_SPRING, OP_DISTANCE_JOINT_SET_SPRING_FORCE_RANGE, OP_DISTANCE_JOINT_SET_SPRING_HERTZ, OP_DISTANCE_JOINT_SET_SPRING_DAMPING_RATIO, OP_DISTANCE_JOINT_ENABLE_MOTOR, OP_DISTANCE_JOINT_SET_MOTOR_SPEED, OP_DISTANCE_JOINT_SET_MAX_MOTOR_FORCE, OP_MOTOR_JOINT_SET_LINEAR_VELOCITY, OP_MOTOR_JOINT_SET_ANGULAR_VELOCITY, OP_MOTOR_JOINT_SET_MAX_VELOCITY_TORQUE, OP_MOTOR_JOINT_SET_MAX_VELOCITY_FORCE, OP_MOTOR_JOINT_SET_LINEAR_HERTZ, OP_MOTOR_JOINT_SET_LINEAR_DAMPING_RATIO, OP_MOTOR_JOINT_SET_ANGULAR_HERTZ, OP_MOTOR_JOINT_SET_ANGULAR_DAMPING_RATIO, OP_MOTOR_JOINT_SET_MAX_SPRING_FORCE, OP_MOTOR_JOINT_SET_MAX_SPRING_TORQUE, OP_PARALLEL_JOINT_SET_SPRING_HERTZ, OP_PARALLEL_JOINT_SET_SPRING_DAMPING_RATIO, OP_PARALLEL_JOINT_SET_MAX_TORQUE, OP_PRISMATIC_JOINT_ENABLE_SPRING, OP_PRISMATIC_JOINT_SET_SPRING_HERTZ, OP_PRISMATIC_JOINT_SET_SPRING_DAMPING_RATIO, OP_PRISMATIC_JOINT_SET_TARGET_TRANSLATION, OP_PRISMATIC_JOINT_ENABLE_LIMIT, OP_PRISMATIC_JOINT_SET_LIMITS, OP_PRISMATIC_JOINT_ENABLE_MOTOR, OP_PRISMATIC_JOINT_SET_MOTOR_SPEED, OP_PRISMATIC_JOINT_SET_MAX_MOTOR_FORCE, OP_REVOLUTE_JOINT_ENABLE_LIMIT, OP_REVOLUTE_JOINT_SET_LIMITS, OP_REVOLUTE_JOINT_ENABLE_SPRING, OP_REVOLUTE_JOINT_SET_TARGET_ANGLE, OP_REVOLUTE_JOINT_SET_SPRING_HERTZ, OP_REVOLUTE_JOINT_SET_SPRING_DAMPING_RATIO, OP_REVOLUTE_JOINT_ENABLE_MOTOR, OP_REVOLUTE_JOINT_SET_MOTOR_SPEED, OP_REVOLUTE_JOINT_SET_MAX_MOTOR_TORQUE, OP_SPHERICAL_JOINT_ENABLE_CONE_LIMIT, OP_SPHERICAL_JOINT_SET_CONE_LIMIT, OP_SPHERICAL_JOINT_SET_CONE_AXIS, OP_SPHERICAL_JOINT_SET_CONE_ELLIPSE, OP_SPHERICAL_JOINT_ENABLE_TWIST_LIMIT, OP_SPHERICAL_JOINT_SET_TWIST_LIMITS, OP_SPHERICAL_JOINT_ENABLE_SPRING, OP_SPHERICAL_JOINT_SET_TARGET_ROTATION, OP_SPHERICAL_JOINT_SET_SPRING_HERTZ, OP_SPHERICAL_JOINT_SET_SPRING_DAMPING_RATIO, OP_SPHERICAL_JOINT_ENABLE_MOTOR, OP_SPHERICAL_JOINT_SET_MOTOR_VELOCITY, OP_SPHERICAL_JOINT_SET_MAX_MOTOR_TORQUE, OP_WELD_JOINT_SET_LINEAR_HERTZ, OP_WELD_JOINT_SET_LINEAR_DAMPING_RATIO, OP_WELD_JOINT_SET_ANGULAR_HERTZ, OP_WELD_JOINT_SET_ANGULAR_DAMPING_RATIO, OP_WHEEL_JOINT_ENABLE_SUSPENSION, OP_WHEEL_JOINT_SET_SUSPENSION_HERTZ, OP_WHEEL_JOINT_SET_SUSPENSION_DAMPING_RATIO, OP_WHEEL_JOINT_ENABLE_SUSPENSION_LIMIT, OP_WHEEL_JOINT_SET_SUSPENSION_LIMITS, OP_WHEEL_JOINT_ENABLE_SPIN_MOTOR, OP_WHEEL_JOINT_SET_SPIN_MOTOR_SPEED, OP_WHEEL_JOINT_SET_MAX_SPIN_TORQUE, OP_WHEEL_JOINT_ENABLE_STEERING, OP_WHEEL_JOINT_SET_STEERING_HERTZ, OP_WHEEL_JOINT_SET_STEERING_DAMPING_RATIO, OP_WHEEL_JOINT_SET_MAX_STEERING_TORQUE, OP_WHEEL_JOINT_ENABLE_STEERING_LIMIT, OP_WHEEL_JOINT_SET_STEERING_LIMITS, OP_WHEEL_JOINT_SET_TARGET_STEERING_ANGLE, + OP_STATE_HASH, OP_BOUNDS, OP_END, OP_QUERY_TAG, OP_QUERY_OVERLAP_AABB, OP_QUERY_OVERLAP_SHAPE, OP_QUERY_CAST_RAY, + OP_QUERY_CAST_SHAPE, OP_QUERY_CAST_RAY_CLOSEST, OP_QUERY_CAST_MOVER, OP_QUERY_COLLIDE_MOVER, OP_CREATE_BODY, OP_DESTROY_BODY, OP_BODY_SET_TRANSFORM, OP_BODY_SET_LINEAR_VELOCITY, OP_BODY_SET_ANGULAR_VELOCITY, OP_BODY_SET_TARGET_TRANSFORM, OP_BODY_APPLY_FORCE, OP_BODY_APPLY_FORCE_TO_CENTER, OP_BODY_APPLY_TORQUE, OP_BODY_APPLY_LINEAR_IMPULSE, OP_BODY_APPLY_LINEAR_IMPULSE_TO_CENTER, OP_BODY_APPLY_ANGULAR_IMPULSE, OP_BODY_SET_BULLET, OP_BODY_SET_MASS_DATA, OP_BODY_APPLY_MASS_FROM_SHAPES, OP_BODY_SET_LINEAR_DAMPING, OP_BODY_SET_ANGULAR_DAMPING, OP_BODY_SET_GRAVITY_SCALE, OP_BODY_SET_AWAKE, OP_BODY_ENABLE_SLEEP, OP_BODY_SET_SLEEP_THRESHOLD, OP_BODY_SET_MOTION_LOCKS, OP_CREATE_SPHERE_SHAPE, OP_CREATE_CAPSULE_SHAPE, OP_CREATE_HULL_SHAPE, OP_CREATE_TRANSFORMED_HULL_SHAPE, OP_CREATE_MESH_SHAPE, OP_CREATE_HEIGHT_FIELD_SHAPE, OP_CREATE_COMPOUND_SHAPE, OP_DESTROY_SHAPE, OP_SHAPE_SET_DENSITY, OP_SHAPE_SET_FILTER, OP_SHAPE_SET_SURFACE_MATERIAL, OP_SHAPE_SET_SPHERE, OP_SHAPE_SET_CAPSULE, OP_SHAPE_SET_HULL, OP_SHAPE_SET_MESH, OP_SHAPE_SET_FRICTION, OP_SHAPE_SET_RESTITUTION, OP_SHAPE_SET_MESH_MATERIAL, OP_SHAPE_ENABLE_PRE_SOLVE_EVENTS, OP_BODY_ENABLE_HIT_EVENTS, OP_BODY_ALLOW_FAST_ROTATION, OP_BODY_ENABLE_CONTACT_RECYCLING, OP_BODY_SET_SAFETY_FACTOR, OP_SHAPE_APPLY_WIND, OP_BODY_SET_NAME, OP_SHAPE_SET_NAME, OP_CREATE_DISTANCE_JOINT, OP_CREATE_MOTOR_JOINT, OP_CREATE_FILTER_JOINT, OP_CREATE_PARALLEL_JOINT, OP_CREATE_PRISMATIC_JOINT, OP_CREATE_REVOLUTE_JOINT, OP_CREATE_SPHERICAL_JOINT, OP_CREATE_WELD_JOINT, OP_CREATE_WHEEL_JOINT, OP_DESTROY_JOINT, OP_JOINT_SET_LOCAL_FRAME_A, OP_JOINT_SET_LOCAL_FRAME_B, OP_JOINT_SET_COLLIDE_CONNECTED, OP_JOINT_WAKE_BODIES, OP_JOINT_SET_CONSTRAINT_TUNING, OP_JOINT_SET_FORCE_THRESHOLD, OP_JOINT_SET_TORQUE_THRESHOLD, OP_BODY_SET_TYPE, OP_BODY_DISABLE, OP_BODY_ENABLE, OP_SHAPE_ENABLE_SENSOR_EVENTS, OP_SHAPE_ENABLE_CONTACT_EVENTS, OP_SHAPE_ENABLE_HIT_EVENTS, OP_STEP, OP_ENABLE_SLEEPING, OP_ENABLE_WARM_STARTING, OP_ENABLE_CONTINUOUS, OP_ENABLE_SPECULATIVE, OP_SET_RESTITUTION_THRESHOLD, OP_SET_RESTITUTION_ITERATIONS, OP_ENABLE_RESTITUTION_PROPAGATION, OP_ENABLE_KINEMATIC_STATIC_SOFTNESS, OP_SET_HIT_EVENT_THRESHOLD, OP_SET_CONTACT_TUNING, OP_SET_CONTACT_RECYCLE_DISTANCE, OP_SET_MAXIMUM_LINEAR_SPEED, OP_SET_GRAVITY, OP_REBUILD_STATIC_TREE, OP_EXPLODE, OP_DISTANCE_JOINT_SET_LENGTH, OP_DISTANCE_JOINT_ENABLE_LIMIT, OP_DISTANCE_JOINT_SET_LENGTH_RANGE, OP_DISTANCE_JOINT_ENABLE_SPRING, OP_DISTANCE_JOINT_SET_SPRING_FORCE_RANGE, OP_DISTANCE_JOINT_SET_SPRING_HERTZ, OP_DISTANCE_JOINT_SET_SPRING_DAMPING_RATIO, OP_DISTANCE_JOINT_ENABLE_MOTOR, OP_DISTANCE_JOINT_SET_MOTOR_SPEED, OP_DISTANCE_JOINT_SET_MAX_MOTOR_FORCE, OP_MOTOR_JOINT_SET_LINEAR_VELOCITY, OP_MOTOR_JOINT_SET_ANGULAR_VELOCITY, OP_MOTOR_JOINT_SET_MAX_VELOCITY_TORQUE, OP_MOTOR_JOINT_SET_MAX_VELOCITY_FORCE, OP_MOTOR_JOINT_SET_LINEAR_HERTZ, OP_MOTOR_JOINT_SET_LINEAR_DAMPING_RATIO, OP_MOTOR_JOINT_SET_ANGULAR_HERTZ, OP_MOTOR_JOINT_SET_ANGULAR_DAMPING_RATIO, OP_MOTOR_JOINT_SET_MAX_SPRING_FORCE, OP_MOTOR_JOINT_SET_MAX_SPRING_TORQUE, OP_PARALLEL_JOINT_SET_SPRING_HERTZ, OP_PARALLEL_JOINT_SET_SPRING_DAMPING_RATIO, OP_PARALLEL_JOINT_SET_MAX_TORQUE, OP_PRISMATIC_JOINT_ENABLE_SPRING, OP_PRISMATIC_JOINT_SET_SPRING_HERTZ, OP_PRISMATIC_JOINT_SET_SPRING_DAMPING_RATIO, OP_PRISMATIC_JOINT_SET_TARGET_TRANSLATION, OP_PRISMATIC_JOINT_ENABLE_LIMIT, OP_PRISMATIC_JOINT_SET_LIMITS, OP_PRISMATIC_JOINT_ENABLE_MOTOR, OP_PRISMATIC_JOINT_SET_MOTOR_SPEED, OP_PRISMATIC_JOINT_SET_MAX_MOTOR_FORCE, OP_REVOLUTE_JOINT_ENABLE_LIMIT, OP_REVOLUTE_JOINT_SET_LIMITS, OP_REVOLUTE_JOINT_ENABLE_SPRING, OP_REVOLUTE_JOINT_SET_TARGET_ANGLE, OP_REVOLUTE_JOINT_SET_SPRING_HERTZ, OP_REVOLUTE_JOINT_SET_SPRING_DAMPING_RATIO, OP_REVOLUTE_JOINT_ENABLE_MOTOR, OP_REVOLUTE_JOINT_SET_MOTOR_SPEED, OP_REVOLUTE_JOINT_SET_MAX_MOTOR_TORQUE, OP_SPHERICAL_JOINT_ENABLE_CONE_LIMIT, OP_SPHERICAL_JOINT_SET_CONE_LIMIT, OP_SPHERICAL_JOINT_SET_CONE_AXIS, OP_SPHERICAL_JOINT_SET_CONE_ELLIPSE, OP_SPHERICAL_JOINT_ENABLE_TWIST_LIMIT, OP_SPHERICAL_JOINT_SET_TWIST_LIMITS, OP_SPHERICAL_JOINT_ENABLE_SPRING, OP_SPHERICAL_JOINT_SET_TARGET_ROTATION, OP_SPHERICAL_JOINT_SET_SPRING_HERTZ, OP_SPHERICAL_JOINT_SET_SPRING_DAMPING_RATIO, OP_SPHERICAL_JOINT_ENABLE_MOTOR, OP_SPHERICAL_JOINT_SET_MOTOR_VELOCITY, OP_SPHERICAL_JOINT_SET_MAX_MOTOR_TORQUE, OP_WELD_JOINT_SET_LINEAR_HERTZ, OP_WELD_JOINT_SET_LINEAR_DAMPING_RATIO, OP_WELD_JOINT_SET_ANGULAR_HERTZ, OP_WELD_JOINT_SET_ANGULAR_DAMPING_RATIO, OP_WHEEL_JOINT_ENABLE_SUSPENSION, OP_WHEEL_JOINT_SET_SUSPENSION_HERTZ, OP_WHEEL_JOINT_SET_SUSPENSION_DAMPING_RATIO, OP_WHEEL_JOINT_ENABLE_SUSPENSION_LIMIT, OP_WHEEL_JOINT_SET_SUSPENSION_LIMITS, OP_WHEEL_JOINT_ENABLE_SPIN_MOTOR, OP_WHEEL_JOINT_SET_SPIN_MOTOR_SPEED, OP_WHEEL_JOINT_SET_MAX_SPIN_TORQUE, OP_WHEEL_JOINT_ENABLE_STEERING, OP_WHEEL_JOINT_SET_STEERING_HERTZ, OP_WHEEL_JOINT_SET_STEERING_DAMPING_RATIO, OP_WHEEL_JOINT_SET_MAX_STEERING_TORQUE, OP_WHEEL_JOINT_ENABLE_STEERING_LIMIT, OP_WHEEL_JOINT_SET_STEERING_LIMITS, OP_WHEEL_JOINT_SET_TARGET_STEERING_ANGLE, create_compound_shape, destroy_shape, shape_is_valid, get_shape_full_id, get_shape, shape_get_body, shape_get_aabb, shape_get_type, shape_get_density, shape_set_density, shape_get_filter, shape_set_filter, shape_get_surface_material, shape_set_surface_material, shape_get_material_count, shape_is_sensor, @@ -425,7 +428,7 @@ struct Body { id: int generation: int user_data: ptr - name: ptr // a plain C copy the world owns (native.name_copy), or null + name_id: int // the name's id in the world's name cache, or NULL_NAME } // What the solver reads and writes for an awake body. @@ -490,7 +493,7 @@ struct WorldShape { user_data: ptr generation: int flags: int - name: ptr // a plain C copy the world owns, or null + name_id: int // the name's id in the world's name cache, or NULL_NAME } struct ContactEdge { @@ -1298,6 +1301,7 @@ struct World { in_use: bool user_data: ptr recording: ptr // *Recording while the world records (aephysics.recording), else null + names: NameCache // the bodies' and shapes' names, each once } var g_worlds: ptr = null // World[MAX_WORLDS] @@ -1459,6 +1463,7 @@ init_world_contents(w: *World, def: *WorldDef) { w.compound_shape_count = 0 w.inv_h = 0.0 w.inv_dt = 0.0 + w.names = basics.name_cache_create() w.contact_state = basics.bitset_create(64) w.contact_begin_events = basics.buffer_create(sizeof(ContactBeginTouchEvent), 4) w.contact_end_events0 = basics.buffer_create(sizeof(ContactEndTouchEvent), 4) @@ -1562,21 +1567,9 @@ free_world_contents(w: *World) { basics.free_bytes(s.materials, s.material_count * sizeof(SurfaceMaterial)) s.materials = null } - if s.id != NULL_INDEX { - native.name_free(s.name) - s.name = null - } - i = i + 1 - } - i = 0 - while i < w.bodies.count { - named = body_at(w, i) - if named.set_index != NULL_INDEX { - native.name_free(named.name) - named.name = null - } i = i + 1 } + basics.name_cache_destroy(&w.names) i = 0 while i < w.sensors.count { sensor = sensor_at(w, i) @@ -1994,7 +1987,7 @@ create_body_internal(world_id: WorldId, def: *BodyDef) -> BodyId { body.inertia = math.mat3_zero() body.kind = def.kind body.flags = flags - body.name = native.name_copy(def.name) + body.name_id = basics.name_cache_add(&w.names, def.name) if set_id >= AWAKE_SET { create_island_for_body(w, set_id, body) } w.locked = false return BodyId { index1: body_id + 1, world0: w.world_id, generation: body.generation } @@ -2043,15 +2036,13 @@ destroy_body(id: BodyId) { s.materials = null s.material_count = 0 } - native.name_free(s.name) - s.name = null + s.name_id = basics.NULL_NAME basics.idpool_free(&w.shape_id_pool, shape_id) set_shape_fat_aabb(w, shape_id, AABBf { lower: math.vec3f_zero(), upper: math.vec3f_zero() }) s.id = NULL_INDEX shape_id = s.next_shape_id } - native.name_free(body.name) - body.name = null + body.name_id = basics.NULL_NAME remove_body_from_island(w, body) set = solver_set_at(w, body.set_index) moved_index = basics.buffer_remove_swap(&set.body_sims, body.local_index) @@ -2171,7 +2162,7 @@ create_shape(body_id: BodyId, def: *ShapeDef, geometry: Shape) -> ShapeId { set_shape_fat_aabb(w, shape_id, math.aabb_round_out(s.aabb)) s.generation = s.generation + 1 s.sensor_index = NULL_INDEX - s.name = native.name_copy(def.name) + s.name_id = basics.name_cache_add(&w.names, def.name) if kind == KIND_COMPOUND { w.compound_shape_count = w.compound_shape_count + 1 // A copy of the compound's materials, so every shape frees its array the same way. @@ -2284,6 +2275,9 @@ struct Recording { bounds: AABB // every step's world bounds, joined have_bounds: bool owner: int // the recording world's index1 while it records, else 0 + lock: int // taken while a record goes in (rec_lock) + tags: Buffer // RecordedTag: the tagged queries' callers, each key once + tag_heads: LongMap // a tag's key to its entry } // One geometry block the calls named: the recording's own pointer-free @@ -2301,12 +2295,20 @@ struct RecordedGeometry { // which older players skip. const RECORDING_MAGIC = 0x43525041 // "APRC", little-endian const RECORDING_MAJOR_VERSION = 1 -const RECORDING_MINOR_VERSION = 0 +const RECORDING_MINOR_VERSION = 1 const RECORDING_HEADER_BYTES = 32 const OP_STATE_HASH = 2 const OP_BOUNDS = 3 const OP_END = 4 +const OP_QUERY_TAG = 5 +const OP_QUERY_OVERLAP_AABB = 6 +const OP_QUERY_OVERLAP_SHAPE = 7 +const OP_QUERY_CAST_RAY = 8 +const OP_QUERY_CAST_SHAPE = 9 +const OP_QUERY_CAST_RAY_CLOSEST = 10 +const OP_QUERY_CAST_MOVER = 11 +const OP_QUERY_COLLIDE_MOVER = 12 const OP_CREATE_BODY = 16 const OP_DESTROY_BODY = 17 const OP_BODY_SET_TRANSFORM = 18 @@ -2495,9 +2497,11 @@ rec_raw(r: *Recording, src: ptr, n: int) { r.size = r.size + n } -// A record begun: its opcode and a length filled in by rec_end. +// A record begun: its opcode and a length filled in by rec_end, the +// recording held between them. rec_begin(world0: int, op: int) -> *Recording { r = world_at(world0).recording as *Recording + rec_lock(r) rec_int(r, op) r.record_start = r.size rec_int(r, 0) @@ -2507,6 +2511,7 @@ rec_begin(world0: int, op: int) -> *Recording { rec_end(r: *Recording) { length = r.size - r.record_start - 4 memcpy(r.data + r.record_start, (&length) as ptr, 4) + rec_unlock(r) } rec_int(r: *Recording, v: int) { @@ -2600,14 +2605,21 @@ rec_geometry(r: *Recording, kind: int, block: ptr) { c.tree.stack = null c.tree.stack_dist = null } - hash = basics.hash_bytes(copy, byte_count) + rec_int(r, intern_geometry(r, kind, basics.hash_bytes(copy, byte_count), copy, byte_count)) +} + +// The id of a block among the recording's (the reference's +// b3InternGeometry): the one with the same kind and bytes, found by its +// content hash and then every byte, so two blocks on one hash stay two; +// else `copy` (alloc's, the recording's from now, freed when found) +// becomes a new one. +intern_geometry(r: *Recording, kind: int, hash: long, copy: ptr, byte_count: int) -> int { index = basics.map_get(&r.geometry_heads, hash, NULL_INDEX) while index != NULL_INDEX { entry = recorded_geometry_at(r, index) if entry.kind == kind && entry.byte_count == byte_count && memcmp(entry.block, copy, byte_count) == 0 { basics.free_bytes(copy, byte_count) - rec_int(r, index) - return + return index } index = entry.next } @@ -2619,7 +2631,7 @@ rec_geometry(r: *Recording, kind: int, block: ptr) { entry.block = copy entry.next = basics.map_get(&r.geometry_heads, hash, NULL_INDEX) basics.map_set(&r.geometry_heads, hash, index) - rec_int(r, index) + return index } recorded_geometry_at(r: *Recording, index: int) -> *RecordedGeometry { return (r.geometry.data + index * sizeof(RecordedGeometry)) as *RecordedGeometry } @@ -2632,13 +2644,15 @@ finish_recording(w: *World) { r = w.recording as *Recording w.recording = null r.owner = 0 + rec_lock(r) rec_int(r, OP_BOUNDS) r.record_start = r.size rec_int(r, 0) rec_bool(r, r.have_bounds) rec_vec3(r, r.bounds.lower) rec_vec3(r, r.bounds.upper) - rec_end(r) + bounds_length = r.size - r.record_start - 4 + memcpy(r.data + r.record_start, (&bounds_length) as ptr, 4) rec_int(r, OP_END) rec_int(r, 0) registry_offset = r.size @@ -2651,9 +2665,168 @@ finish_recording(w: *World) { rec_raw(r, entry.block, entry.byte_count) i = i + 1 } + // The tagged queries' callers: each key, its id and its name. + rec_int(r, r.tags.count) + i = 0 + while i < r.tags.count { + tag = recorded_tag_at(r, i) + rec_long(r, tag.key) + rec_long(r, tag.id) + if tag.name == null { rec_int(r, NULL_INDEX) } else { + rec_int(r, tag.length) + rec_raw(r, tag.name, tag.length) + } + i = i + 1 + } registry_bytes = r.size - registry_offset memcpy(r.data + 12, (®istry_offset) as ptr, 4) memcpy(r.data + 16, (®istry_bytes) as ptr, 4) + rec_unlock(r) +} + +// --- recorded queries --------------------------------------------------------------------------- + +// A query's caller, as the recording knows it: the key (query_tag_key) +// the viewer follows it by across frames, and the id and name it hashes, +// kept once per key in the recording's tag table. +struct RecordedTag { + key: long + id: long + name: ptr // its bytes and a terminating zero (alloc's), or null + length: int +} + +recorded_tag_at(r: *Recording, index: int) -> *RecordedTag { return (r.tags.data + index * sizeof(RecordedTag)) as *RecordedTag } + +// The recording's own, while records go in from several threads: queries +// may run at once on a world that is not stepping. +rec_lock(r: *Recording) { + while native.atomic_cas((&r.lock) as ptr, 0, 1) == false { native.pause() } +} + +rec_unlock(r: *Recording) { native.atomic_store((&r.lock) as ptr, 0) } + +// A query's bytes, written while it runs (its hits come one visit at a +// time) and committed whole after, so concurrent queries never interleave. +rec_payload_create() -> *Recording { return basics.alloc(sizeof(Recording)) as *Recording } + +rec_payload_destroy(payload: *Recording) { + if payload.data != null { basics.free_bytes(payload.data, payload.capacity) } + basics.free_bytes(payload as ptr, sizeof(Recording)) +} + +// The int written `at` bytes into the payload replaced (a count reserved +// before its items). +rec_patch_int(payload: *Recording, at: int, v: int) { + holder = v + memcpy(payload.data + at, (&holder) as ptr, 4) +} + +// A query's caller as one key: 64-bit FNV-1a over the id's eight bytes, +// then the name's (the reference's b3HashQueryTag); never 0, so the key +// says whether the query was tagged. +query_tag_key(id: long, name: string) -> long { + // 14695981039346656037 and 1099511628211, the FNV-1a basis and prime. + hash = 0 - 3750763034362895579 + prime = 1099511628211 as long + i = 0 + while i < 8 { + hash = (hash ^ (basics.lsr(id, 8 * i) & (255 as long))) * prime + i = i + 1 + } + if name != null { + bytes = string.aether_string_raw_ptr(name) + n = string.length(name) + i = 0 + while i < n { + hash = (hash ^ (native.load_u8(bytes, i as long) as long)) * prime + i = i + 1 + } + } + if hash == (0 as long) { return 1 as long } + return hash +} + +// The key's id and name kept, the first ones for a key. +intern_tag(r: *Recording, key: long, id: long, name: string) { + if basics.map_has(&r.tag_heads, key) { return } + index = basics.buffer_push(&r.tags) + tag = recorded_tag_at(r, index) + tag.key = key + tag.id = id + tag.name = null + tag.length = 0 + if name != null && string.length(name) > 0 { + tag.length = string.length(name) + tag.name = basics.alloc(tag.length + 1) + memcpy(tag.name, string.aether_string_raw_ptr(name), tag.length) + } + basics.map_set(&r.tag_heads, key, index) +} + +// The recording's tags forgotten, ready for a fresh session. +clear_tags(r: *Recording) { + i = 0 + while i < r.tags.count { + tag = recorded_tag_at(r, i) + if tag.name != null { basics.free_bytes(tag.name, tag.length + 1) } + i = i + 1 + } + r.tags.count = 0 + basics.map_clear(&r.tag_heads) +} + +// A query's record committed into its world's recording, its tag record +// first when the filter names its caller (both under the lock, so they +// stay together whatever else records at once). +rec_commit_query(world0: int, op: int, payload: *Recording, tag_id: long, tag_name: string) { + r = world_at(world0).recording as *Recording + if r == null { return } + rec_lock(r) + if tag_id != (0 as long) || (tag_name != null && string.length(tag_name) > 0) { + key = query_tag_key(tag_id, tag_name) + intern_tag(r, key, tag_id, tag_name) + rec_int(r, OP_QUERY_TAG) + rec_int(r, 8) + rec_long(r, key) + } + rec_int(r, op) + rec_int(r, payload.size) + rec_raw(r, payload.data, payload.size) + rec_unlock(r) +} + +rec_aabb(r: *Recording, box: AABB) { + rec_vec3(r, box.lower) + rec_vec3(r, box.upper) +} + +// A query's filter: its bits (its caller rides the tag record). +rec_query_filter(r: *Recording, filter: QueryFilter) { + rec_long(r, filter.category_bits) + rec_long(r, filter.mask_bits) +} + +// A convex proxy: its points, then its radius. +rec_shape_proxy(r: *Recording, proxy: ShapeProxy) { + count = math.clamp_int(proxy.count, 0, distance.MAX_SHAPE_CAST_POINTS) + rec_int(r, count) + points = proxy.points as Vec3[] + i = 0 + while i < count { + rec_vec3(r, points[i]) + i = i + 1 + } + rec_float(r, proxy.radius) +} + +rec_plane_result(r: *Recording, plane: PlaneResult) { + rec_vec3(r, plane.plane.normal) + rec_float(r, plane.plane.offset) + rec_vec3(r, plane.point) + rec_int(r, plane.triangle_index) + rec_int(r, plane.child_index) + rec_int(r, plane.material_index) } rec_body_def(r: *Recording, v: *BodyDef) { @@ -3167,8 +3340,7 @@ destroy_shape(id: ShapeId, update_body_mass: bool) { s.materials = null s.material_count = 0 } - native.name_free(s.name) - s.name = null + s.name_id = basics.NULL_NAME basics.idpool_free(&w.shape_id_pool, s.id) set_shape_fat_aabb(w, s.id, AABBf { lower: math.vec3f_zero(), upper: math.vec3f_zero() }) s.id = NULL_INDEX @@ -3466,22 +3638,21 @@ shape_apply_wind(id: ShapeId, wind: Vec3, drag: float, lift: float, max_speed: f sim.torque = math.narrow3(math.add(math.widen3(sim.torque), torque)) } -// A label for debugging (b3Body_SetName and its siblings): copied, so the -// caller's string may go; the old one freed; an empty name for none. +// A label for debugging (b3Body_SetName and its siblings), kept in the +// world's name cache, so the caller's string may go; an empty name for +// none. body_set_name(id: BodyId, name: string) { if recording_on(id.world0) { recorder = rec_begin(id.world0, OP_BODY_SET_NAME); rec_id(recorder, id.index1, id.generation); rec_string(recorder, name); rec_end(recorder) } - body = get_body_full_id(world_at(id.world0), id) - native.name_free(body.name) - body.name = native.name_copy(name) + w = world_at(id.world0) + get_body_full_id(w, id).name_id = basics.name_cache_add(&w.names, name) } -body_get_name(id: BodyId) -> string { return native.name_text(get_body_full_id(world_at(id.world0), id).name) } +body_get_name(id: BodyId) -> string { w = world_at(id.world0); return basics.name_cache_text(&w.names, get_body_full_id(w, id).name_id) } shape_set_name(id: ShapeId, name: string) { if recording_on(id.world0) { recorder = rec_begin(id.world0, OP_SHAPE_SET_NAME); rec_id(recorder, id.index1, id.generation); rec_string(recorder, name); rec_end(recorder) } - s = get_shape_full_id(world_at(id.world0), id) - native.name_free(s.name) - s.name = native.name_copy(name) + w = world_at(id.world0) + get_shape_full_id(w, id).name_id = basics.name_cache_add(&w.names, name) } -shape_get_name(id: ShapeId) -> string { return native.name_text(get_shape_full_id(world_at(id.world0), id).name) } +shape_get_name(id: ShapeId) -> string { w = world_at(id.world0); return basics.name_cache_text(&w.names, get_shape_full_id(w, id).name_id) } // The world a body or shape belongs to, as an id (b3Body_GetWorld, b3Shape_GetWorld). body_get_world(id: BodyId) -> WorldId { return WorldId { index1: id.world0 + 1, generation: world_at(id.world0).generation } } diff --git a/aephysics/physics_world/module.ae b/aephysics/physics_world/module.ae index 79855aa..ae6d4f6 100644 --- a/aephysics/physics_world/module.ae +++ b/aephysics/physics_world/module.ae @@ -511,9 +511,8 @@ aabb_query_visitor(proxy_id: int, user_data: long, context: ptr) -> bool { return visitor(dynamics.make_shape_id(w, s), qc.user_context) } -// Every shape whose fat bounds overlap the box, through the visitor -// until it returns false; the tree visits made. -overlap_aabb(id: WorldId, box: AABB, filter: QueryFilter, visitor: fn(ShapeId, ptr) -> bool, context: ptr) -> TreeStats { +// overlap_aabb's work, unrecorded. +overlap_aabb_internal(id: WorldId, box: AABB, filter: QueryFilter, visitor: fn(ShapeId, ptr) -> bool, context: ptr) -> TreeStats { stats = TreeStats { node_visits: 0, leaf_visits: 0 } w = world_of(id) if w == null { return stats } @@ -540,8 +539,8 @@ shape_overlap_visitor(proxy_id: int, user_data: long, context: ptr) -> bool { return visitor(dynamics.make_shape_id(w, s), qc.user_context) } -// Every shape a convex proxy (given relative to the origin) overlaps. -overlap_shape(id: WorldId, origin: Vec3, proxy: ShapeProxy, filter: QueryFilter, visitor: fn(ShapeId, ptr) -> bool, context: ptr) -> TreeStats { +// overlap_shape's work, unrecorded. +overlap_shape_internal(id: WorldId, origin: Vec3, proxy: ShapeProxy, filter: QueryFilter, visitor: fn(ShapeId, ptr) -> bool, context: ptr) -> TreeStats { stats = TreeStats { node_visits: 0, leaf_visits: 0 } w = world_of(id) if w == null { return stats } @@ -584,10 +583,8 @@ mover_collide_visitor(proxy_id: int, user_data: long, context: ptr) -> bool { return true } -// The planes of every shape the character mover's capsule (relative to -// the origin) touches, a shape's planes at a time (up to 64), through -// the visitor until it returns false. -collide_mover(id: WorldId, origin: Vec3, mover: Capsule, filter: QueryFilter, visitor: fn(ShapeId, PlaneResult[], int, ptr) -> bool, context: ptr) { +// collide_mover's work, unrecorded. +collide_mover_internal(id: WorldId, origin: Vec3, mover: Capsule, filter: QueryFilter, visitor: fn(ShapeId, PlaneResult[], int, ptr) -> bool, context: ptr) { w = world_of(id) if w == null { return } if math.is_valid_position(origin) == false || valid_capsule(mover) == false { return } @@ -636,10 +633,8 @@ ray_cast_visitor(input: *RayCastInput, proxy_id: int, user_data: long, context: return input.max_fraction } -// A ray from the origin along the translation: each hit through the -// visitor, whose return clips the ray (0 stops it, 1 keeps everything, -// -1 ignores that shape); the tree visits made. -cast_ray(id: WorldId, origin: Vec3, translation: Vec3, filter: QueryFilter, +// cast_ray's work, unrecorded. +cast_ray_internal(id: WorldId, origin: Vec3, translation: Vec3, filter: QueryFilter, visitor: fn(ShapeId, Vec3, Vec3, float, long, int, int, ptr) -> float, context: ptr) -> TreeStats { stats = TreeStats { node_visits: 0, leaf_visits: 0 } w = world_of(id) @@ -675,8 +670,8 @@ closest_ray_visitor(shape_id: ShapeId, point: Vec3, normal: Vec3, fraction: floa return fraction } -// The closest hit of a ray, the common query of games. -cast_ray_closest(id: WorldId, origin: Vec3, translation: Vec3, filter: QueryFilter) -> RayResult { +// cast_ray_closest's work, unrecorded. +cast_ray_closest_internal(id: WorldId, origin: Vec3, translation: Vec3, filter: QueryFilter) -> RayResult { result = RayResult { shape_id: dynamics.null_shape_id(), point: math.vec3_zero(), normal: math.vec3_zero(), user_material_id: 0 as long, fraction: 0.0, triangle_index: 0, child_index: 0, node_visits: 0, leaf_visits: 0, hit: false } w = world_of(id) @@ -714,9 +709,8 @@ shape_cast_visitor(input: *BoxCastInput, proxy_id: int, user_data: long, context return input.max_fraction } -// A convex proxy (relative to the origin) swept along the translation, -// each hit through the visitor as the ray cast's. -cast_shape(id: WorldId, origin: Vec3, proxy: ShapeProxy, translation: Vec3, filter: QueryFilter, +// cast_shape's work, unrecorded. +cast_shape_internal(id: WorldId, origin: Vec3, proxy: ShapeProxy, translation: Vec3, filter: QueryFilter, visitor: fn(ShapeId, Vec3, Vec3, float, long, int, int, ptr) -> float, context: ptr) -> TreeStats { stats = TreeStats { node_visits: 0, leaf_visits: 0 } w = world_of(id) @@ -773,9 +767,8 @@ mover_cast_visitor(input: *BoxCastInput, proxy_id: int, user_data: long, context var g_mover_points: ptr = null // Vec3[2] mover_points() -> ptr { return native.scratch(g_mover_points, 2 * sizeof(Vec3)) } -// The fraction of the translation the mover's capsule can travel before -// touching a shape (its filter asked per shape when given). -cast_mover(id: WorldId, origin: Vec3, mover: Capsule, translation: Vec3, filter: QueryFilter, has_filter: bool, filter_fn: fn(ShapeId, ptr) -> bool, context: ptr) -> float { +// cast_mover's work, unrecorded. +cast_mover_internal(id: WorldId, origin: Vec3, mover: Capsule, translation: Vec3, filter: QueryFilter, has_filter: bool, filter_fn: fn(ShapeId, ptr) -> bool, context: ptr) -> float { w = world_of(id) if w == null { return 1.0 } if math.is_valid_position(origin) == false || math.is_valid_vec3(translation) == false || valid_capsule(mover) == false { return 1.0 } @@ -797,6 +790,240 @@ cast_mover(id: WorldId, origin: Vec3, mover: Capsule, translation: Vec3, filter: return mc.fraction } +// --- the queries recorded (#149) ------------------------------------------------------------------ + +// A query while its world records (the reference's b3RecQueryWriter): its +// own bytes, and the caller's visitor and context, which a trampoline +// calls, writing each visit and the caller's answer so a replay can ask +// the same query and give the same answers. Queries do not change the +// world, so the record goes in after the query, whole. +struct QueryRecorder { + payload: *Recording + overlap: fn(ShapeId, ptr) -> bool + cast: fn(ShapeId, Vec3, Vec3, float, long, int, int, ptr) -> float + planes: fn(ShapeId, PlaneResult[], int, ptr) -> bool + has_overlap: bool // the caller gave an overlap visitor (a cast mover's filter is optional) + context: ptr + count: int // the visits written +} + +accept_shape(shape_id: ShapeId, context: ptr) -> bool { return true } +keep_cast(shape_id: ShapeId, point: Vec3, normal: Vec3, fraction: float, user_material_id: long, triangle_index: int, child_index: int, context: ptr) -> float { return 1.0 } +accept_planes(shape_id: ShapeId, planes: PlaneResult[], count: int, context: ptr) -> bool { return true } + +query_recorder(context: ptr) -> QueryRecorder { + return QueryRecorder { payload: dynamics.rec_payload_create(), overlap: accept_shape, cast: keep_cast, planes: accept_planes, + has_overlap: false, context: context, count: 0 } +} + +// Whether the query asked of `id` is one to record: the world records and +// is not stepping (a stepping world answers no query). +recording_query(id: WorldId) -> bool { return world_of(id) != null && dynamics.recording_on(id.index1 - 1) } + +// The query's record committed with its caller's tag, its bytes freed. +finish_query(id: WorldId, op: int, q: *QueryRecorder, filter: QueryFilter) { + dynamics.rec_commit_query(id.index1 - 1, op, q.payload, filter.id, filter.name) + dynamics.rec_payload_destroy(q.payload) +} + +rec_tree_stats(payload: *Recording, stats: TreeStats) { + dynamics.rec_int(payload, stats.node_visits) + dynamics.rec_int(payload, stats.leaf_visits) +} + +// A visit written: the shape and the caller's answer. +record_overlap_visit(shape_id: ShapeId, context: ptr) -> bool { + q = context as *QueryRecorder + answer = true + if q.has_overlap { + visitor = q.overlap + answer = visitor(shape_id, q.context) + } + dynamics.rec_id(q.payload, shape_id.index1, shape_id.generation) + dynamics.rec_bool(q.payload, answer) + q.count = q.count + 1 + return answer +} + +// A hit written: everything the visitor was given and its answer. +record_cast_visit(shape_id: ShapeId, point: Vec3, normal: Vec3, fraction: float, user_material_id: long, triangle_index: int, child_index: int, context: ptr) -> float { + q = context as *QueryRecorder + visitor = q.cast + answer = visitor(shape_id, point, normal, fraction, user_material_id, triangle_index, child_index, q.context) + dynamics.rec_id(q.payload, shape_id.index1, shape_id.generation) + dynamics.rec_vec3(q.payload, point) + dynamics.rec_vec3(q.payload, normal) + dynamics.rec_float(q.payload, fraction) + dynamics.rec_long(q.payload, user_material_id) + dynamics.rec_int(q.payload, triangle_index) + dynamics.rec_int(q.payload, child_index) + dynamics.rec_float(q.payload, answer) + q.count = q.count + 1 + return answer +} + +// A shape's planes written: the shape, its planes and the caller's answer. +record_plane_visit(shape_id: ShapeId, planes: PlaneResult[], count: int, context: ptr) -> bool { + q = context as *QueryRecorder + visitor = q.planes + answer = visitor(shape_id, planes, count, q.context) + dynamics.rec_id(q.payload, shape_id.index1, shape_id.generation) + dynamics.rec_int(q.payload, count) + i = 0 + while i < count { + dynamics.rec_plane_result(q.payload, planes[i]) + i = i + 1 + } + dynamics.rec_bool(q.payload, answer) + q.count = q.count + 1 + return answer +} + +// Every shape whose fat bounds overlap the box, through the visitor +// until it returns false; the tree visits made. +overlap_aabb(id: WorldId, box: AABB, filter: QueryFilter, visitor: fn(ShapeId, ptr) -> bool, context: ptr) -> TreeStats { + if recording_query(id) == false { return overlap_aabb_internal(id, box, filter, visitor, context) } + q = query_recorder(context) + q.overlap = visitor + q.has_overlap = true + dynamics.rec_aabb(q.payload, box) + dynamics.rec_query_filter(q.payload, filter) + count_at = q.payload.size + dynamics.rec_int(q.payload, 0) + stats = overlap_aabb_internal(id, box, filter, record_overlap_visit, (&q) as ptr) + dynamics.rec_patch_int(q.payload, count_at, q.count) + rec_tree_stats(q.payload, stats) + finish_query(id, dynamics.OP_QUERY_OVERLAP_AABB, &q, filter) + return stats +} + +// Every shape a convex proxy (given relative to the origin) overlaps. +overlap_shape(id: WorldId, origin: Vec3, proxy: ShapeProxy, filter: QueryFilter, visitor: fn(ShapeId, ptr) -> bool, context: ptr) -> TreeStats { + if recording_query(id) == false { return overlap_shape_internal(id, origin, proxy, filter, visitor, context) } + q = query_recorder(context) + q.overlap = visitor + q.has_overlap = true + dynamics.rec_vec3(q.payload, origin) + dynamics.rec_shape_proxy(q.payload, proxy) + dynamics.rec_query_filter(q.payload, filter) + count_at = q.payload.size + dynamics.rec_int(q.payload, 0) + stats = overlap_shape_internal(id, origin, proxy, filter, record_overlap_visit, (&q) as ptr) + dynamics.rec_patch_int(q.payload, count_at, q.count) + rec_tree_stats(q.payload, stats) + finish_query(id, dynamics.OP_QUERY_OVERLAP_SHAPE, &q, filter) + return stats +} + +// The planes of every shape the character mover's capsule (relative to +// the origin) touches, a shape's planes at a time (up to 64), through +// the visitor until it returns false. +collide_mover(id: WorldId, origin: Vec3, mover: Capsule, filter: QueryFilter, visitor: fn(ShapeId, PlaneResult[], int, ptr) -> bool, context: ptr) { + if recording_query(id) == false { + collide_mover_internal(id, origin, mover, filter, visitor, context) + return + } + q = query_recorder(context) + q.planes = visitor + dynamics.rec_vec3(q.payload, origin) + dynamics.rec_capsule_value(q.payload, mover) + dynamics.rec_query_filter(q.payload, filter) + count_at = q.payload.size + dynamics.rec_int(q.payload, 0) + collide_mover_internal(id, origin, mover, filter, record_plane_visit, (&q) as ptr) + dynamics.rec_patch_int(q.payload, count_at, q.count) + finish_query(id, dynamics.OP_QUERY_COLLIDE_MOVER, &q, filter) +} + +// A ray from the origin along the translation: each hit through the +// visitor, whose return clips the ray (0 stops it, 1 keeps everything, +// -1 ignores that shape); the tree visits made. +cast_ray(id: WorldId, origin: Vec3, translation: Vec3, filter: QueryFilter, + visitor: fn(ShapeId, Vec3, Vec3, float, long, int, int, ptr) -> float, context: ptr) -> TreeStats { + if recording_query(id) == false { return cast_ray_internal(id, origin, translation, filter, visitor, context) } + q = query_recorder(context) + q.cast = visitor + dynamics.rec_vec3(q.payload, origin) + dynamics.rec_vec3(q.payload, translation) + dynamics.rec_query_filter(q.payload, filter) + count_at = q.payload.size + dynamics.rec_int(q.payload, 0) + stats = cast_ray_internal(id, origin, translation, filter, record_cast_visit, (&q) as ptr) + dynamics.rec_patch_int(q.payload, count_at, q.count) + rec_tree_stats(q.payload, stats) + finish_query(id, dynamics.OP_QUERY_CAST_RAY, &q, filter) + return stats +} + +// The closest hit of a ray, the common query of games. +cast_ray_closest(id: WorldId, origin: Vec3, translation: Vec3, filter: QueryFilter) -> RayResult { + if recording_query(id) == false { return cast_ray_closest_internal(id, origin, translation, filter) } + payload = dynamics.rec_payload_create() + dynamics.rec_vec3(payload, origin) + dynamics.rec_vec3(payload, translation) + dynamics.rec_query_filter(payload, filter) + result = cast_ray_closest_internal(id, origin, translation, filter) + rec_ray_result(payload, result) + dynamics.rec_commit_query(id.index1 - 1, dynamics.OP_QUERY_CAST_RAY_CLOSEST, payload, filter.id, filter.name) + dynamics.rec_payload_destroy(payload) + return result +} + +rec_ray_result(payload: *Recording, result: RayResult) { + dynamics.rec_bool(payload, result.hit) + dynamics.rec_id(payload, result.shape_id.index1, result.shape_id.generation) + dynamics.rec_vec3(payload, result.point) + dynamics.rec_vec3(payload, result.normal) + dynamics.rec_float(payload, result.fraction) + dynamics.rec_long(payload, result.user_material_id) + dynamics.rec_int(payload, result.triangle_index) + dynamics.rec_int(payload, result.child_index) + dynamics.rec_int(payload, result.node_visits) + dynamics.rec_int(payload, result.leaf_visits) +} + +// A convex proxy (relative to the origin) swept along the translation, +// each hit through the visitor as the ray cast's. +cast_shape(id: WorldId, origin: Vec3, proxy: ShapeProxy, translation: Vec3, filter: QueryFilter, + visitor: fn(ShapeId, Vec3, Vec3, float, long, int, int, ptr) -> float, context: ptr) -> TreeStats { + if recording_query(id) == false { return cast_shape_internal(id, origin, proxy, translation, filter, visitor, context) } + q = query_recorder(context) + q.cast = visitor + dynamics.rec_vec3(q.payload, origin) + dynamics.rec_shape_proxy(q.payload, proxy) + dynamics.rec_vec3(q.payload, translation) + dynamics.rec_query_filter(q.payload, filter) + count_at = q.payload.size + dynamics.rec_int(q.payload, 0) + stats = cast_shape_internal(id, origin, proxy, translation, filter, record_cast_visit, (&q) as ptr) + dynamics.rec_patch_int(q.payload, count_at, q.count) + rec_tree_stats(q.payload, stats) + finish_query(id, dynamics.OP_QUERY_CAST_SHAPE, &q, filter) + return stats +} + +// The fraction of the translation the mover's capsule can travel before +// touching a shape (its filter asked per shape when given). Recorded, the +// filter is asked through the recorder for every candidate, given or not, +// so a replay sees the same decisions. +cast_mover(id: WorldId, origin: Vec3, mover: Capsule, translation: Vec3, filter: QueryFilter, has_filter: bool, filter_fn: fn(ShapeId, ptr) -> bool, context: ptr) -> float { + if recording_query(id) == false { return cast_mover_internal(id, origin, mover, translation, filter, has_filter, filter_fn, context) } + q = query_recorder(context) + q.overlap = filter_fn + q.has_overlap = has_filter + dynamics.rec_vec3(q.payload, origin) + dynamics.rec_capsule_value(q.payload, mover) + dynamics.rec_vec3(q.payload, translation) + dynamics.rec_query_filter(q.payload, filter) + count_at = q.payload.size + dynamics.rec_int(q.payload, 0) + fraction = cast_mover_internal(id, origin, mover, translation, filter, true, record_overlap_visit, (&q) as ptr) + dynamics.rec_patch_int(q.payload, count_at, q.count) + dynamics.rec_float(q.payload, fraction) + finish_query(id, dynamics.OP_QUERY_CAST_MOVER, &q, filter) + return fraction +} + // --- the body queries ------------------------------------------------------------------------------------------------ no_body_hit() -> BodyCastResult { diff --git a/aephysics/recording/module.ae b/aephysics/recording/module.ae index 3eef9b9..4dbabf0 100644 --- a/aephysics/recording/module.ae +++ b/aephysics/recording/module.ae @@ -25,14 +25,21 @@ // differs marks the frame it was found at (the player has diverged) and the // replay goes on. // +// The world queries are recorded too, each with every visit and the +// caller's answer to it, and asked again on replay with the same answers; +// a query whose filter names its caller (its id and name) carries a key a +// viewer can follow it by. The player keeps the queries of the frame it +// last played, with their hits. +// // Not recorded: what the host wires in (user data, callbacks, filters, the -// worker count), and the world queries (#149 leaves them for later). +// worker count). import std.string import std.fs import aephysics.math import aephysics.native import aephysics.basics +import aephysics.distance import aephysics.hull import aephysics.mesh import aephysics.height_field @@ -46,7 +53,9 @@ import aephysics.physics_world import aephysics.snapshot exports ( - Player, PlayerInfo, create_recording, destroy_recording, recording_data, recording_size, start_recording, + Player, PlayerInfo, QueryInfo, QueryHit, QUERY_OVERLAP_AABB, QUERY_OVERLAP_SHAPE, QUERY_CAST_RAY, QUERY_CAST_SHAPE, + QUERY_CAST_RAY_CLOSEST, QUERY_CAST_MOVER, QUERY_COLLIDE_MOVER, player_frame_query_count, player_frame_query, + player_frame_query_hit, create_recording, destroy_recording, recording_data, recording_size, start_recording, stop_recording, save_recording_to_file, load_recording_from_file, create_player, destroy_player, player_step_frame, player_sub_step_frame, player_restart, player_seek_frame, player_world, player_frame, player_frame_count, player_at_end, player_at_pre_step, player_has_diverged, player_diverge_frame, player_failed, player_info, @@ -57,6 +66,7 @@ exports ( extern memcpy(dst: ptr, src: ptr, size: int) -> ptr extern memset(block: ptr, value: int, size: int) -> ptr +extern memchr(block: ptr, value: int, size: int) -> ptr const NULL_INDEX = 0 - 1 const STRING_SLOTS = 4 @@ -77,11 +87,16 @@ create_recording() -> *Recording { r.geometry_heads = basics.map_create(16) r.have_bounds = false r.owner = 0 + r.lock = 0 + r.tags = basics.buffer_create(sizeof(RecordedTag), 0) + r.tag_heads = basics.map_create(16) return r } -// The recording's geometry copies forgotten, ready for a fresh session. -clear_geometry(r: *Recording) { +// The recording's geometry copies and tags forgotten, ready for a fresh +// session. +clear_session(r: *Recording) { + dynamics.clear_tags(r) i = 0 while i < r.geometry.count { entry = dynamics.recorded_geometry_at(r, i) @@ -97,9 +112,11 @@ clear_geometry(r: *Recording) { destroy_recording(r: *Recording) { if r == null { return } if r.owner != 0 { dynamics.finish_recording(dynamics.get_world(r.owner - 1)) } - clear_geometry(r) + clear_session(r) basics.buffer_destroy(&r.geometry) basics.map_destroy(&r.geometry_heads) + basics.buffer_destroy(&r.tags) + basics.map_destroy(&r.tag_heads) if r.data != null { basics.free_bytes(r.data, r.capacity) } basics.free_bytes(r as ptr, sizeof(Recording)) } @@ -121,7 +138,7 @@ start_recording(world_id: WorldId, r: *Recording) -> bool { if snap == null { return false } r.size = 0 r.have_bounds = false - clear_geometry(r) + clear_session(r) dynamics.rec_int(r, dynamics.RECORDING_MAGIC) dynamics.rec_int(r, dynamics.RECORDING_MAJOR_VERSION) dynamics.rec_int(r, snap_count) @@ -239,6 +256,14 @@ struct Player { keyframe_budget: long keyframe_bytes: long last_keyframe_frame: int + tags: Buffer // PlayerTag: the tagged queries' callers + tag_index: LongMap // a tag's key to its entry + proxy_points: ptr // Vec3[MAX_SHAPE_CAST_POINTS]: a recorded proxy's points + replay_hits: Buffer // RecordedHit: the visits of the query being asked again + replay_planes: Buffer // PlaneResult: a mover's recorded planes + frame_queries: Buffer // FrameQuery: the queries of the frame last played + frame_hits: Buffer // QueryHit + pending_key: long // the key of the tag record just read, for the query after it } player_geometry_at(p: *Player, index: int) -> *PlayerGeometry { return (p.geometry.data + index * sizeof(PlayerGeometry)) as *PlayerGeometry } @@ -288,6 +313,13 @@ create_player(data: ptr, size: int, worker_count: int) -> *Player { p.bodies = basics.buffer_create(sizeof(BodyId), 0) p.frame0_bodies = basics.buffer_create(sizeof(BodyId), 0) p.keyframes = basics.buffer_create(sizeof(Keyframe), 0) + p.tags = basics.buffer_create(sizeof(PlayerTag), 0) + p.tag_index = basics.map_create(16) + p.proxy_points = basics.alloc(distance.MAX_SHAPE_CAST_POINTS * sizeof(Vec3)) + p.replay_hits = basics.buffer_create(sizeof(RecordedHit), 0) + p.replay_planes = basics.buffer_create(sizeof(PlaneResult), 0) + p.frame_queries = basics.buffer_create(sizeof(FrameQuery), 0) + p.frame_hits = basics.buffer_create(sizeof(QueryHit), 0) p.keyframe_interval = DEFAULT_KEYFRAME_INTERVAL p.keyframe_min_interval = DEFAULT_KEYFRAME_INTERVAL p.keyframe_budget = DEFAULT_KEYFRAME_BUDGET as long @@ -333,6 +365,12 @@ destroy_player(p: *Player) { basics.buffer_destroy(&p.keyframes) basics.buffer_destroy(&p.bodies) basics.buffer_destroy(&p.frame0_bodies) + free_tags(p) + basics.free_bytes(p.proxy_points, distance.MAX_SHAPE_CAST_POINTS * sizeof(Vec3)) + basics.buffer_destroy(&p.replay_hits) + basics.buffer_destroy(&p.replay_planes) + basics.buffer_destroy(&p.frame_queries) + basics.buffer_destroy(&p.frame_hits) release_record(p) basics.free_bytes(p.strings, STRING_SLOTS * 8) basics.free_bytes(p.string_bytes, STRING_SLOTS * 4) @@ -371,7 +409,7 @@ load_geometry(p: *Player, offset: int, byte_count: int) -> bool { if valid == false { return false } i = i + 1 } - return at == end + return load_tags(p, at, end) } // The records walked once without running them: their framing checked up @@ -639,6 +677,7 @@ keyframe_boundary(p: *Player) { player_step_frame(p: *Player) -> bool { p.at_pre_step = false if p.at_end { return false } + clear_frame_queries(p) stepped = false while true { if p.failed || p.cursor >= p.ops_end { @@ -668,6 +707,7 @@ player_step_frame(p: *Player) -> bool { // can draw the new bodies where they were made; the next steps. player_sub_step_frame(p: *Player) { if p.at_end { return } + if p.at_pre_step == false { clear_frame_queries(p) } stepped = false made_a_body = false while true { @@ -715,6 +755,7 @@ player_restart(p: *Player) { p.frame = 0 p.at_end = false p.at_pre_step = false + clear_frame_queries(p) copy_bodies(&p.bodies, p.frame0_bodies.data, p.frame0_bodies.count) } @@ -754,6 +795,7 @@ restore_keyframe(p: *Player, k: *Keyframe) { p.frame = k.frame p.at_end = false p.at_pre_step = false + clear_frame_queries(p) copy_bodies(&p.bodies, k.bodies, k.body_count) } @@ -879,6 +921,580 @@ validate_replay(data: ptr, size: int, worker_count: int) -> bool { return result } +// --- the queries replayed --------------------------------------------------------------------------- + +// A recorded query asked again in the player's world, through a +// trampoline that compares each visit with the recorded one and gives the +// caller's recorded answer back, so the query runs as it did; any +// difference (a shape, a point, a count, the tree visits) marks the +// player diverged. Then it is kept with its hits for the frame, for a +// viewer to read (b3RecPlayer_GetFrameQuery). + +// The kinds of query a frame keeps (b3RecQueryType). +const QUERY_OVERLAP_AABB = 0 +const QUERY_OVERLAP_SHAPE = 1 +const QUERY_CAST_RAY = 2 +const QUERY_CAST_SHAPE = 3 +const QUERY_CAST_RAY_CLOSEST = 4 +const QUERY_CAST_MOVER = 5 +const QUERY_COLLIDE_MOVER = 6 + +// A query of the frame last played (b3RecQueryInfo). +struct QueryInfo { + kind: int // QUERY_* + filter: QueryFilter // its bits, and its caller's id and name + aabb: AABB // where it looked: the box, or the shape swept along the translation + origin: Vec3 + translation: Vec3 + fraction: float // a cast mover's answer, the closest ray's hit; 1 otherwise + hit_count: int + key: long // its caller's key, 0 when untagged +} + +// One result of a query of the frame last played (b3RecQueryHit): a +// shape, and for a cast its point, normal and fraction; a mover's plane +// gives its point and normal. +struct QueryHit { + shape: ShapeId + point: Vec3 + normal: Vec3 + fraction: float +} + +// A visit as recorded, to compare and answer with. +struct RecordedHit { + shape: ShapeId + point: Vec3 + normal: Vec3 + fraction: float + user_material_id: long + triangle_index: int + child_index: int + answer_float: float + answer_bool: bool + plane_start: int // a mover's planes in replay_planes + plane_count: int +} + +struct FrameQuery { + kind: int + filter: QueryFilter + aabb: AABB + origin: Vec3 + translation: Vec3 + fraction: float + hit_start: int + hit_count: int + key: long +} + +// A tagged query's caller, from the recording's tag table. +struct PlayerTag { + key: long + id: long + name: ptr // its bytes and a zero (alloc's), or null + length: int +} + +struct ReplayQuery { + p: *Player + cursor: int + count: int +} + +recorded_hit_at(p: *Player, index: int) -> *RecordedHit { return (p.replay_hits.data + index * sizeof(RecordedHit)) as *RecordedHit } +frame_query_at(p: *Player, index: int) -> *FrameQuery { return (p.frame_queries.data + index * sizeof(FrameQuery)) as *FrameQuery } +frame_hit_at(p: *Player, index: int) -> *QueryHit { return (p.frame_hits.data + index * sizeof(QueryHit)) as *QueryHit } +player_tag_at(p: *Player, index: int) -> *PlayerTag { return (p.tags.data + index * sizeof(PlayerTag)) as *PlayerTag } +replay_plane_at(p: *Player, index: int) -> *PlaneResult { return (p.replay_planes.data + index * sizeof(PlaneResult)) as *PlaneResult } + +// The recording's tag table, after the geometry: each key, its id and its +// name. A recording made before queries were recorded has none. Bytes +// after it (a newer minor version's) are left alone. +load_tags(p: *Player, at: int, end: int) -> bool { + if at == end { return true } + if at + 4 > end { return false } + count = read_header_int(p.data, at) + at = at + 4 + if count < 0 || basics.section_fits(at, count, 20, end) == false { return false } + i = 0 + while i < count { + if at + 20 > end { return false } + key = read_header_long(p.data, at) + id = read_header_long(p.data, at + 8) + n = read_header_int(p.data, at + 16) + at = at + 20 + if key == (0 as long) || (n != NULL_INDEX && (n <= 0 || n > end - at)) || basics.map_has(&p.tag_index, key) { return false } + index = basics.buffer_push(&p.tags) + tag = player_tag_at(p, index) + tag.key = key + tag.id = id + tag.name = null + tag.length = 0 + if n != NULL_INDEX { + if memchr(p.data + at, 0, n) != null { return false } + tag.name = basics.alloc(n + 1) + tag.length = n + memcpy(tag.name, p.data + at, n) + at = at + n + } + basics.map_set(&p.tag_index, key, index) + i = i + 1 + } + return true +} + +read_header_long(data: ptr, at: int) -> long { + holder = 0 as long + memcpy((&holder) as ptr, data + at, 8) + return holder +} + +free_tags(p: *Player) { + i = 0 + while i < p.tags.count { + tag = player_tag_at(p, i) + if tag.name != null { basics.free_bytes(tag.name, tag.length + 1) } + i = i + 1 + } + basics.buffer_destroy(&p.tags) + basics.map_destroy(&p.tag_index) +} + +// The frame's queries forgotten, before the next frame's come. +clear_frame_queries(p: *Player) { + p.frame_queries.count = 0 + p.frame_hits.count = 0 + p.pending_key = 0 as long +} + +read_aabb(p: *Player) -> AABB { + lower = read_vec3(p) + upper = read_vec3(p) + return AABB { lower: lower, upper: upper } +} + +read_query_filter(p: *Player) -> QueryFilter { + filter = shape.default_query_filter() + filter.category_bits = read_long(p) + filter.mask_bits = read_long(p) + return filter +} + +// A proxy's points into the player's own block. +read_shape_proxy(p: *Player) -> ShapeProxy { + count = read_int(p) + if count < 0 || count > distance.MAX_SHAPE_CAST_POINTS { + p.ok = false + count = 0 + } + points = p.proxy_points as Vec3[] + i = 0 + while i < count { + points[i] = read_vec3(p) + i = i + 1 + } + radius = read_float(p) + return distance.shape_proxy(p.proxy_points, count, radius) +} + +read_plane_result(p: *Player) -> PlaneResult { + normal = read_vec3(p) + offset = read_float(p) + point = read_vec3(p) + triangle_index = read_int(p) + child_index = read_int(p) + material_index = read_int(p) + return PlaneResult { plane: Plane { normal: normal, offset: offset }, point: point, triangle_index: triangle_index, + child_index: child_index, material_index: material_index } +} + +// A count of items of at least `least` bytes each, which the record must +// have room for: a count from the file never sizes an allocation it could +// not fill. +read_count(p: *Player, least: int) -> int { + n = read_int(p) + if p.ok == false { return 0 } + if n < 0 || n > (p.record_end - p.cursor) / least { + p.ok = false + return 0 + } + return n +} + +// The recorded visits made ready: `n` of them, each zeroed. +ready_hits(p: *Player, n: int) { + p.replay_hits.count = 0 + basics.buffer_reserve(&p.replay_hits, n) + if n > 0 { memset(p.replay_hits.data, 0, n * sizeof(RecordedHit)) } + p.replay_hits.count = n +} + +// An overlap's visits: each shape and the caller's answer. +read_overlap_hits(p: *Player) -> int { + n = read_count(p, 12) + ready_hits(p, n) + i = 0 + while i < n { + h = recorded_hit_at(p, i) + h.shape = read_shape_id(p) + h.answer_bool = read_bool(p) + i = i + 1 + } + return n +} + +// A cast's hits: everything the visitor was given and its answer. +read_cast_hits(p: *Player) -> int { + n = read_count(p, 88) + ready_hits(p, n) + i = 0 + while i < n { + h = recorded_hit_at(p, i) + h.shape = read_shape_id(p) + h.point = read_vec3(p) + h.normal = read_vec3(p) + h.fraction = read_float(p) + h.user_material_id = read_long(p) + h.triangle_index = read_int(p) + h.child_index = read_int(p) + h.answer_float = read_float(p) + i = i + 1 + } + return n +} + +same_vec3(a: Vec3, b: Vec3) -> bool { return a.x == b.x && a.y == b.y && a.z == b.z } + +same_plane(a: PlaneResult, b: PlaneResult) -> bool { + return same_vec3(a.plane.normal, b.plane.normal) && a.plane.offset == b.plane.offset && same_vec3(a.point, b.point) && + a.triangle_index == b.triangle_index && a.child_index == b.child_index && a.material_index == b.material_index +} + +// The next recorded visit, or null when the query visits more than it did. +next_hit(rq: *ReplayQuery) -> *RecordedHit { + if rq.cursor >= rq.count { + rq.p.diverged = true + return null + } + h = recorded_hit_at(rq.p, rq.cursor) + rq.cursor = rq.cursor + 1 + return h +} + +replay_overlap_visit(shape_id: ShapeId, context: ptr) -> bool { + rq = context as *ReplayQuery + h = next_hit(rq) + if h == null { return false } + if shape_id.index1 != h.shape.index1 || shape_id.generation != h.shape.generation { rq.p.diverged = true } + return h.answer_bool +} + +replay_cast_visit(shape_id: ShapeId, point: Vec3, normal: Vec3, fraction: float, user_material_id: long, triangle_index: int, child_index: int, context: ptr) -> float { + rq = context as *ReplayQuery + h = next_hit(rq) + if h == null { return 0.0 } + if shape_id.index1 != h.shape.index1 || shape_id.generation != h.shape.generation || same_vec3(point, h.point) == false || + same_vec3(normal, h.normal) == false || fraction != h.fraction || user_material_id != h.user_material_id || + triangle_index != h.triangle_index || child_index != h.child_index { + rq.p.diverged = true + } + return h.answer_float +} + +replay_plane_visit(shape_id: ShapeId, planes: PlaneResult[], count: int, context: ptr) -> bool { + rq = context as *ReplayQuery + h = next_hit(rq) + if h == null { return true } + if shape_id.index1 != h.shape.index1 || shape_id.generation != h.shape.generation || count != h.plane_count { rq.p.diverged = true } + recorded = rq.p.replay_planes.data as PlaneResult[] + i = 0 + while i < count && i < h.plane_count { + if same_plane(planes[i], recorded[h.plane_start + i]) == false { rq.p.diverged = true } + i = i + 1 + } + return h.answer_bool +} + +// The visits compared to the end and the tree's work with the recorded. +check_replayed(p: *Player, rq: *ReplayQuery, stats: TreeStats, nodes: int, leaves: int) { + if rq.cursor != rq.count || stats.node_visits != nodes || stats.leaf_visits != leaves { p.diverged = true } +} + +// Where a query looked: its points (relative to the origin) at the origin +// and at the end of the translation, fattened by the radius. +query_bounds(origin: Vec3, translation: Vec3, points: ptr, count: int, radius: float) -> AABB { + local = points as Vec3[] + lower = origin + upper = origin + first = true + end = math.add(origin, translation) + i = 0 + while i < math.max_int(count, 1) { + offset = math.vec3_zero() + if count > 0 { offset = local[i] } + a = math.add(origin, offset) + b = math.add(end, offset) + if first { + lower = math.min_vec3(a, b) + upper = math.max_vec3(a, b) + first = false + } else { + lower = math.min_vec3(lower, math.min_vec3(a, b)) + upper = math.max_vec3(upper, math.max_vec3(a, b)) + } + i = i + 1 + } + r = math.vec3(radius, radius, radius) + return AABB { lower: math.sub(lower, r), upper: math.add(upper, r) } +} + +// The query kept for the frame, with the caller's key from the tag +// record before it, and its hits from the recorded visits. +stash_query(p: *Player, kind: int, filter: QueryFilter, aabb: AABB, origin: Vec3, translation: Vec3, fraction: float, hit_count: int) -> *FrameQuery { + index = basics.buffer_push(&p.frame_queries) + q = frame_query_at(p, index) + q.kind = kind + q.filter = filter + q.aabb = aabb + q.origin = origin + q.translation = translation + q.fraction = fraction + q.key = p.pending_key + p.pending_key = 0 as long + q.hit_start = p.frame_hits.count + q.hit_count = hit_count + return q +} + +stash_hit(p: *Player, shape_id: ShapeId, point: Vec3, normal: Vec3, fraction: float) { + index = basics.buffer_push(&p.frame_hits) + h = frame_hit_at(p, index) + h.shape = shape_id + h.point = point + h.normal = normal + h.fraction = fraction +} + +stash_recorded_hits(p: *Player, n: int) { + i = 0 + while i < n { + h = recorded_hit_at(p, i) + stash_hit(p, h.shape, h.point, h.normal, h.fraction) + i = i + 1 + } +} + +replay_overlap_aabb(p: *Player) { + box = read_aabb(p) + filter = read_query_filter(p) + n = read_overlap_hits(p) + nodes = read_int(p) + leaves = read_int(p) + if p.ok == false { return } + rq = ReplayQuery { p: p, cursor: 0, count: n } + stats = physics_world.overlap_aabb(p.world_id, box, filter, replay_overlap_visit, (&rq) as ptr) + check_replayed(p, &rq, stats, nodes, leaves) + stash_query(p, QUERY_OVERLAP_AABB, filter, box, math.vec3_zero(), math.vec3_zero(), 1.0, n) + stash_recorded_hits(p, n) +} + +replay_overlap_shape(p: *Player) { + origin = read_vec3(p) + proxy = read_shape_proxy(p) + filter = read_query_filter(p) + n = read_overlap_hits(p) + nodes = read_int(p) + leaves = read_int(p) + if p.ok == false { return } + rq = ReplayQuery { p: p, cursor: 0, count: n } + stats = physics_world.overlap_shape(p.world_id, origin, proxy, filter, replay_overlap_visit, (&rq) as ptr) + check_replayed(p, &rq, stats, nodes, leaves) + box = query_bounds(origin, math.vec3_zero(), proxy.points, proxy.count, proxy.radius) + stash_query(p, QUERY_OVERLAP_SHAPE, filter, box, origin, math.vec3_zero(), 1.0, n) + stash_recorded_hits(p, n) +} + +replay_cast_ray(p: *Player) { + origin = read_vec3(p) + translation = read_vec3(p) + filter = read_query_filter(p) + n = read_cast_hits(p) + nodes = read_int(p) + leaves = read_int(p) + if p.ok == false { return } + rq = ReplayQuery { p: p, cursor: 0, count: n } + stats = physics_world.cast_ray(p.world_id, origin, translation, filter, replay_cast_visit, (&rq) as ptr) + check_replayed(p, &rq, stats, nodes, leaves) + stash_query(p, QUERY_CAST_RAY, filter, query_bounds(origin, translation, null, 0, 0.0), origin, translation, 1.0, n) + stash_recorded_hits(p, n) +} + +replay_cast_shape(p: *Player) { + origin = read_vec3(p) + proxy = read_shape_proxy(p) + translation = read_vec3(p) + filter = read_query_filter(p) + n = read_cast_hits(p) + nodes = read_int(p) + leaves = read_int(p) + if p.ok == false { return } + rq = ReplayQuery { p: p, cursor: 0, count: n } + stats = physics_world.cast_shape(p.world_id, origin, proxy, translation, filter, replay_cast_visit, (&rq) as ptr) + check_replayed(p, &rq, stats, nodes, leaves) + box = query_bounds(origin, translation, proxy.points, proxy.count, proxy.radius) + stash_query(p, QUERY_CAST_SHAPE, filter, box, origin, translation, 1.0, n) + stash_recorded_hits(p, n) +} + +replay_cast_ray_closest(p: *Player) { + origin = read_vec3(p) + translation = read_vec3(p) + filter = read_query_filter(p) + hit = read_bool(p) + shape_id = read_shape_id(p) + point = read_vec3(p) + normal = read_vec3(p) + fraction = read_float(p) + user_material_id = read_long(p) + triangle_index = read_int(p) + child_index = read_int(p) + nodes = read_int(p) + leaves = read_int(p) + if p.ok == false { return } + got = physics_world.cast_ray_closest(p.world_id, origin, translation, filter) + if got.hit != hit || got.node_visits != nodes || got.leaf_visits != leaves { p.diverged = true } + else if hit { + if got.shape_id.index1 != shape_id.index1 || got.shape_id.generation != shape_id.generation || same_vec3(got.point, point) == false || + same_vec3(got.normal, normal) == false || got.fraction != fraction || got.user_material_id != user_material_id || + got.triangle_index != triangle_index || got.child_index != child_index { + p.diverged = true + } + } + hits = 0 + if hit { hits = 1 } + result_fraction = 1.0 + if hit { result_fraction = fraction } + stash_query(p, QUERY_CAST_RAY_CLOSEST, filter, query_bounds(origin, translation, null, 0, 0.0), origin, translation, result_fraction, hits) + if hit { stash_hit(p, shape_id, point, normal, fraction) } +} + +replay_cast_mover(p: *Player) { + origin = read_vec3(p) + mover = read_capsule(p) + translation = read_vec3(p) + filter = read_query_filter(p) + n = read_overlap_hits(p) + fraction = read_float(p) + if p.ok == false { return } + rq = ReplayQuery { p: p, cursor: 0, count: n } + got = physics_world.cast_mover(p.world_id, origin, mover, translation, filter, true, replay_overlap_visit, (&rq) as ptr) + if rq.cursor != n || got != fraction { p.diverged = true } + centres = p.proxy_points as Vec3[] + centres[0] = mover.center1 + centres[1] = mover.center2 + box = query_bounds(origin, translation, p.proxy_points, 2, mover.radius) + stash_query(p, QUERY_CAST_MOVER, filter, box, origin, translation, fraction, 0) +} + +replay_collide_mover(p: *Player) { + origin = read_vec3(p) + mover = read_capsule(p) + filter = read_query_filter(p) + groups = read_count(p, 16) + ready_hits(p, groups) + p.replay_planes.count = 0 + i = 0 + while i < groups && p.ok { + h = recorded_hit_at(p, i) + h.shape = read_shape_id(p) + count = read_count(p, 68) + h.plane_start = p.replay_planes.count + h.plane_count = count + k = 0 + while k < count { + plane = read_plane_result(p) + index = basics.buffer_push(&p.replay_planes) + stored = p.replay_planes.data as PlaneResult[] + stored[index] = plane + k = k + 1 + } + h.answer_bool = read_bool(p) + i = i + 1 + } + if p.ok == false { return } + rq = ReplayQuery { p: p, cursor: 0, count: groups } + physics_world.collide_mover(p.world_id, origin, mover, filter, replay_plane_visit, (&rq) as ptr) + if rq.cursor != groups { p.diverged = true } + centres = p.proxy_points as Vec3[] + centres[0] = mover.center1 + centres[1] = mover.center2 + box = query_bounds(origin, math.vec3_zero(), p.proxy_points, 2, mover.radius) + stash_query(p, QUERY_COLLIDE_MOVER, filter, box, origin, math.vec3_zero(), 1.0, p.replay_planes.count) + i = 0 + while i < groups { + h = recorded_hit_at(p, i) + k = 0 + while k < h.plane_count { + kept_plane = replay_plane_at(p, h.plane_start + k) + stash_hit(p, h.shape, kept_plane.point, kept_plane.plane.normal, 0.0) + k = k + 1 + } + i = i + 1 + } +} + +// A query record run; false for an opcode that is not one. +dispatch_query(p: *Player, op: int) -> bool { + if op == dynamics.OP_QUERY_TAG { + p.pending_key = read_long(p) + return true + } + if op == dynamics.OP_QUERY_OVERLAP_AABB { replay_overlap_aabb(p) } + else if op == dynamics.OP_QUERY_OVERLAP_SHAPE { replay_overlap_shape(p) } + else if op == dynamics.OP_QUERY_CAST_RAY { replay_cast_ray(p) } + else if op == dynamics.OP_QUERY_CAST_SHAPE { replay_cast_shape(p) } + else if op == dynamics.OP_QUERY_CAST_RAY_CLOSEST { replay_cast_ray_closest(p) } + else if op == dynamics.OP_QUERY_CAST_MOVER { replay_cast_mover(p) } + else if op == dynamics.OP_QUERY_COLLIDE_MOVER { replay_collide_mover(p) } + else { return false } + return true +} + +// The queries of the frame last played. +player_frame_query_count(p: *Player) -> int { return p.frame_queries.count } + +// A query of the frame last played, its caller's id and name from the tag +// table; an empty one for an index out of range. +player_frame_query(p: *Player, index: int) -> QueryInfo { + if index < 0 || index >= p.frame_queries.count { + return QueryInfo { kind: NULL_INDEX, filter: shape.default_query_filter(), aabb: AABB { lower: math.vec3_zero(), upper: math.vec3_zero() }, + origin: math.vec3_zero(), translation: math.vec3_zero(), fraction: 1.0, hit_count: 0, key: 0 as long } + } + q = frame_query_at(p, index) + filter = q.filter + if q.key != (0 as long) { + tag_index = basics.map_get(&p.tag_index, q.key, NULL_INDEX) + if tag_index != NULL_INDEX { + tag = player_tag_at(p, tag_index) + filter.id = tag.id + if tag.name != null { filter.name = native.name_text(tag.name) } + } + } + return QueryInfo { kind: q.kind, filter: filter, aabb: q.aabb, origin: q.origin, translation: q.translation, fraction: q.fraction, + hit_count: q.hit_count, key: q.key } +} + +// One result of a query of the frame last played; an empty one out of range. +player_frame_query_hit(p: *Player, query: int, hit: int) -> QueryHit { + no_hit = QueryHit { shape: dynamics.null_shape_id(), point: math.vec3_zero(), normal: math.vec3_zero(), fraction: 0.0 } + if query < 0 || query >= p.frame_queries.count { return no_hit } + q = frame_query_at(p, query) + if hit < 0 || hit >= q.hit_count { return no_hit } + hits = p.frame_hits.data as QueryHit[] + return hits[q.hit_start + hit] +} + // --- the records ------------------------------------------------------------------------------------ // The record's call made in the player's world, or its check done; false @@ -886,10 +1502,10 @@ validate_replay(data: ptr, size: int, worker_count: int) -> bool { dispatch(p: *Player, op: int) -> bool { if op == dynamics.OP_STATE_HASH { recorded = read_long(p) - if p.ok && dynamics.hash_world_state(dynamics.get_world(p.world_id.index1 - 1)) != recorded { - if p.diverged == false { p.diverge_frame = p.frame } - p.diverged = true - } + if p.ok && dynamics.hash_world_state(dynamics.get_world(p.world_id.index1 - 1)) != recorded { p.diverged = true } + // The first frame found parted: a query that answered differently + // before this frame's step is found here too. + if p.diverged && p.diverge_frame < 0 { p.diverge_frame = p.frame } return true } if op == dynamics.OP_BOUNDS { @@ -899,6 +1515,7 @@ dispatch(p: *Player, op: int) -> bool { _upper = read_vec3(p) return true } + if dispatch_query(p, op) { return true } return dispatch_call(p, op) } diff --git a/aephysics/shape/module.ae b/aephysics/shape/module.ae index 8b8ea94..5956598 100644 --- a/aephysics/shape/module.ae +++ b/aephysics/shape/module.ae @@ -73,6 +73,12 @@ struct Filter { struct QueryFilter { category_bits: long mask_bits: long + // Who asked, for a recording (the reference's id and name): hashed + // together into the key a replay viewer follows the query by across + // frames. Neither need be unique alone; 0 and no name is untagged. + // Ignored when the world is not recording. + id: long + name: string } // --- construction --------------------------------------------------------------------- @@ -138,7 +144,7 @@ compound_shape(c: *CompoundData) -> Shape { // --- filters ---------------------------------------------------------------------------- default_filter() -> Filter { return Filter { category_bits: 1 as long, mask_bits: all_bits(), group_index: 0 } } -default_query_filter() -> QueryFilter { return QueryFilter { category_bits: 1 as long, mask_bits: all_bits() } } +default_query_filter() -> QueryFilter { return QueryFilter { category_bits: 1 as long, mask_bits: all_bits(), id: 0 as long, name: null } } all_bits() -> long { return (0 as long) - (1 as long) } diff --git a/aephysics/snapshot/module.ae b/aephysics/snapshot/module.ae index f51f80b..2e31f31 100644 --- a/aephysics/snapshot/module.ae +++ b/aephysics/snapshot/module.ae @@ -40,11 +40,9 @@ exports ( extern memcpy(dst: ptr, src: ptr, size: int) -> ptr extern memset(block: ptr, value: int, size: int) -> ptr -extern strlen(s: ptr) -> int -extern malloc(size: int) -> ptr const SNAPSHOT_MAGIC = 0x4E535041 // "APSN", little-endian -const SNAPSHOT_VERSION = 1 +const SNAPSHOT_VERSION = 2 const NULL_INDEX = 0 - 1 // --- the bytes written ------------------------------------------------------------------------------- @@ -121,15 +119,17 @@ w_idpool(wr: *SnapWriter, p: *IdPool) { w_ints(wr, &p.free_ids) } -// A name: its length and its bytes, or -1 for none. -w_name(wr: *SnapWriter, name: ptr) { - if name == null { - w_int(wr, NULL_INDEX) - return +// The world's names, each once: its id, its length and its bytes. +w_names(wr: *SnapWriter, c: *NameCache) { + count = basics.name_cache_count(c) + w_int(wr, count) + i = 0 + while i < count { + w_int(wr, basics.name_cache_id_at(c, i)) + w_int(wr, basics.name_cache_length_at(c, i)) + w_bytes(wr, basics.name_cache_name_at(c, i), basics.name_cache_length_at(c, i)) + i = i + 1 } - n = strlen(name) - w_int(wr, n) - w_bytes(wr, name, n) } // --- the bytes read ---------------------------------------------------------------------------------- @@ -235,18 +235,25 @@ r_idpool(rd: *SnapReader) -> IdPool { return IdPool { free_ids: free_ids, next_index: next_index } } -// A name as a C copy (name_free's), or null. -r_name(rd: *SnapReader) -> ptr { - n = r_int(rd) - if rd.ok == false || n == NULL_INDEX { return null } - if r_need(rd, n) == false { return null } - block = malloc(n + 1) - memcpy(block, rd.data + rd.at, n) - memset(block + n, 0, 1) - rd.at = rd.at + n - return block +// The world's names into its empty cache, each id once. +r_names(rd: *SnapReader, c: *NameCache) { + count = r_count(rd, 8) + i = 0 + while i < count && rd.ok { + id = r_int(rd) + n = r_int(rd) + if rd.ok == false || n <= 0 || r_need(rd, n) == false || basics.name_cache_load(c, id, rd.data + rd.at, n) == false { + rd.ok = false + return + } + rd.at = rd.at + n + i = i + 1 + } } +// A body's or shape's name id: none, or one the cache holds. +name_known(c: *NameCache, id: int) -> bool { return id == basics.NULL_NAME || basics.name_cache_find(c, id) != null } + // --- the layout -------------------------------------------------------------------------------------- // This build's layout: the size of every record copied whole and the @@ -391,7 +398,8 @@ save_world(world_id: WorldId) -> (ptr, int) { i = i + 1 } - // The bodies, their pointers cleared in the copy, each name after. + // The names, then the bodies, their pointers cleared in the copy. + w_names(&wr, &w.names) w_int(&wr, w.bodies.count) i = 0 while i < w.bodies.count { @@ -399,9 +407,7 @@ save_world(world_id: WorldId) -> (ptr, int) { start = wr.size w_bytes(&wr, b as ptr, sizeof(Body)) body_copy = (wr.data + start) as *Body - body_copy.name = null body_copy.user_data = null - w_name(&wr, b.name) i = i + 1 } @@ -436,7 +442,7 @@ save_world(world_id: WorldId) -> (ptr, int) { } // The shapes, their pointers cleared in the copy; then each one's - // geometry entry, its materials and its name. + // geometry entry and its materials. w_int(&wr, w.shapes.count) i = 0 while i < w.shapes.count { @@ -450,7 +456,6 @@ save_world(world_id: WorldId) -> (ptr, int) { shape_copy.geometry.compound = null shape_copy.materials = null shape_copy.user_data = null - shape_copy.name = null w_int(&wr, indices[i]) if s.id != NULL_INDEX && s.materials != null { w_int(&wr, s.material_count) @@ -458,7 +463,6 @@ save_world(world_id: WorldId) -> (ptr, int) { } else { w_int(&wr, 0) } - if s.id != NULL_INDEX { w_name(&wr, s.name) } else { w_name(&wr, null) } i = i + 1 } basics.free_bytes(indices_block, math.max_int(w.shapes.count, 1) * 4) @@ -688,7 +692,10 @@ fill_world(w: *World, rd: *SnapReader) -> bool { } if rd.ok == false || w.solver_sets.count < 3 { return false } - // The bodies: each record with its pointers cleared, then its name. + // The names, then the bodies, each record with its pointers cleared and + // its name one the names hold. + r_names(rd, &w.names) + if rd.ok == false { return false } body_count = r_count(rd, sizeof(Body)) basics.buffer_destroy(&w.bodies) w.bodies = basics.buffer_create(sizeof(Body), math.max_int(body_count, 16)) @@ -697,9 +704,8 @@ fill_world(w: *World, rd: *SnapReader) -> bool { index = basics.buffer_push(&w.bodies) b = dynamics.body_at(w, index) r_bytes(rd, b as ptr, sizeof(Body)) - b.name = null b.user_data = null - b.name = r_name(rd) + if b.set_index != NULL_INDEX && name_known(&w.names, b.name_id) == false { rd.ok = false } i = i + 1 } if rd.ok == false { return false } @@ -740,8 +746,8 @@ fill_world(w: *World, rd: *SnapReader) -> bool { loaded = loaded + 1 } - // The shapes: each record with its pointers cleared, then its geometry, - // materials and name. + // The shapes: each record with its pointers cleared, then its geometry + // and materials; its name one the names hold. shape_count = r_count(rd, sizeof(WorldShape)) if rd.ok { basics.buffer_destroy(&w.shapes) @@ -758,7 +764,7 @@ fill_world(w: *World, rd: *SnapReader) -> bool { s.geometry.compound = null s.materials = null s.user_data = null - s.name = null + if s.id != NULL_INDEX && name_known(&w.names, s.name_id) == false { rd.ok = false } geometry_index = r_int(rd) material_count = r_count(rd, sizeof(SurfaceMaterial)) if rd.ok == false { break } @@ -781,7 +787,6 @@ fill_world(w: *World, rd: *SnapReader) -> bool { r_bytes(rd, s.materials, material_count * sizeof(SurfaceMaterial)) } } - s.name = r_name(rd) i = i + 1 } // The hulls' copies were only for the database to take its own from. diff --git a/design.md b/design.md index 0109577..6eb8985 100644 --- a/design.md +++ b/design.md @@ -371,7 +371,16 @@ started until its tests pass. the call runs, so nothing the call does is recorded twice; the player restores its world in place (`dynamics.reset_world`, the reference's deserialize into a shell), so ids handed to a viewer stay valid through - a seek. Recording the world queries, and the name cache, are next. Two + a seek. The world queries and the name cache are in (#149's third + part): a query records each visit with the caller's answer, and the + replay asks it again through a trampoline that gives the recorded + answers back, so a visitor that stops or clips a query is replayed + exactly; its record goes in whole under the recording's lock, since + queries may run on several threads at once. Names are interned per + world under the reference's 32-bit hash, the bodies and shapes holding + ids, so their records copy whole and a snapshot carries each name once; + where the reference gives two names on one hash the first name, the + second here takes the next free id. Two functions no test called hid type errors (`body_apply_force`, `body_get_local_point_velocity`) until the replay dispatcher called every recorded one: `ae build` checks only what a program reaches diff --git a/tests/test_basics.ae b/tests/test_basics.ae index 68304bf..07639be 100644 --- a/tests/test_basics.ae +++ b/tests/test_basics.ae @@ -38,6 +38,58 @@ struct Thing { tag: long } +// The name cache (the reference's test_name_cache.c CacheUnit): an empty +// or no name is the null id, which stands for nothing; distinct names get +// distinct ids and read back; the same name again is the same id and no +// second entry; a long name reads back whole; a different name on an id +// already taken gets another id and keeps its own text; a snapshot's +// names load with their ids, and bad ones are refused; everything comes +// back when the cache goes. +name_cache_unit() { + before = basics.alloc_count() + before_bytes = basics.bytes_in_use() + c = basics.name_cache_create() + ensure("names: empty is none", basics.name_cache_add(&c, "") == basics.NULL_NAME) + ensure("names: null is none", basics.name_cache_add(&c, null) == basics.NULL_NAME) + ensure("names: none stands for nothing", basics.name_cache_find(&c, basics.NULL_NAME) == null && string.length(basics.name_cache_text(&c, basics.NULL_NAME)) == 0) + crate = basics.name_cache_add(&c, "crate") + barrel = basics.name_cache_add(&c, "barrel") + ensure("names: distinct ids", crate != basics.NULL_NAME && barrel != basics.NULL_NAME && crate != barrel) + ensure("names: read back", string.equals(basics.name_cache_text(&c, crate), "crate") == 1 && string.equals(basics.name_cache_text(&c, barrel), "barrel") == 1) + ensure("names: the same name, the same id, one entry", basics.name_cache_add(&c, "crate") == crate && basics.name_cache_count(&c) == 2) + // The id is the reference's b3Hash32 of the bytes (FNV-1a, Murmur3's + // finish), its values from the reference's own function compiled in C. + ensure("names: the reference's hash of crate", basics.name_hash32(string.aether_string_raw_ptr("crate"), 5) == (3832558763 as long) && crate == 0 - 462408533) + ensure("names: the reference's hash of barrel", basics.name_hash32(string.aether_string_raw_ptr("barrel"), 6) == (2907559280 as long) && barrel == 0 - 1387408016) + long_block = calloc(4096, 1) + long_bytes = long_block as byte[] + i = 0 + while i < 4095 { + long_bytes[i] = (97 + i % 26) as byte + i = i + 1 + } + long_name = string.from_cstr(long_block) + long_id = basics.name_cache_add(&c, long_name) + ensure("names: a long name whole", string.length(basics.name_cache_text(&c, long_id)) == 4095 && string.equals(basics.name_cache_text(&c, long_id), long_name) == 1 && basics.name_cache_count(&c) == 3) + string.free(long_name) + free(long_block) + // Another name on a taken id: loaded under crate's id first, "crate" + // itself then probes past it. + d = basics.name_cache_create() + other = string.aether_string_raw_ptr("other") + ensure("names: loaded with its id", basics.name_cache_load(&d, crate, other, 5)) + moved = basics.name_cache_add(&d, "crate") + ensure("names: a collision takes the next id", moved == crate + 1 && string.equals(basics.name_cache_text(&d, crate), "other") == 1 && string.equals(basics.name_cache_text(&d, moved), "crate") == 1) + ensure("names: and finds it again", basics.name_cache_add(&d, "crate") == moved && basics.name_cache_count(&d) == 2) + ensure("names: a taken id refused", basics.name_cache_load(&d, crate, other, 5) == false) + ensure("names: the null id refused", basics.name_cache_load(&d, basics.NULL_NAME, other, 5) == false) + ensure("names: an empty name refused", basics.name_cache_load(&d, 77, other, 0) == false) + ensure("names: a zero inside refused", basics.name_cache_load(&d, 78, string.aether_string_raw_ptr("crate"), 6) == false) + basics.name_cache_destroy(&d) + basics.name_cache_destroy(&c) + ensure("names: nothing left (${basics.alloc_count() - before} blocks, ${basics.bytes_in_use() - before_bytes} bytes)", basics.alloc_count() == before && basics.bytes_in_use() == before_bytes) +} + main() { // Bit math. ensure("clz of 9 is 28", basics.clz32(9) == 31 - 3) @@ -369,6 +421,7 @@ main() { ensure("every allocation the module counted was freed", basics.alloc_count() == 0 && basics.bytes_in_use() == 0) + name_cache_unit() println("basics: ${checks} checks") if failures == 0 { println("basics: all checks passed") diff --git a/tests/test_recording.ae b/tests/test_recording.ae index b59a06a..97dcd85 100644 --- a/tests/test_recording.ae +++ b/tests/test_recording.ae @@ -15,6 +15,7 @@ import std.string import aephysics.math import aephysics.basics +import aephysics.distance import aephysics.hull import aephysics.mesh import aephysics.height_field @@ -670,6 +671,7 @@ geometry_dedup() { steps(world_id, 5) recording.stop_recording(world_id) ensure("dedup: two blocks for eleven shapes (${registry_count(r)})", registry_count(r) == 2) + ensure("dedup: the recording holds two", r.geometry.count == 2) ensure("dedup: replays", recording.validate_replay(recording.recording_data(r), recording.recording_size(r), 1)) hull.destroy_hull(first) hull.destroy_hull(twin) @@ -678,6 +680,35 @@ geometry_dedup() { recording.destroy_recording(r) } +// Blocks on one content hash with other bytes stay two blocks; either one +// interned again is found through the hash's chain, the one behind the +// head too, and adds nothing (the reference's GeometryHashCollision). +geometry_hash_collision() { + r = recording.create_recording() + shared = 0xABCD1234 as long + a = filled_block(170) + b = filled_block(187) + id_a = dynamics.intern_geometry(r, shape.SHAPE_HULL, shared, a, 16) + id_b = dynamics.intern_geometry(r, shape.SHAPE_HULL, shared, b, 16) + ensure("collision: two blocks, not one", id_a != id_b && r.geometry.count == 2) + ensure("collision: the first found again", dynamics.intern_geometry(r, shape.SHAPE_HULL, shared, filled_block(170), 16) == id_a && r.geometry.count == 2) + ensure("collision: the second found again", dynamics.intern_geometry(r, shape.SHAPE_HULL, shared, filled_block(187), 16) == id_b && r.geometry.count == 2) + ensure("collision: another kind is another block", dynamics.intern_geometry(r, shape.SHAPE_MESH, shared, filled_block(170), 16) != id_a && r.geometry.count == 3) + recording.destroy_recording(r) +} + +// Sixteen bytes of one value, counted (the recording frees what it takes). +filled_block(value: int) -> ptr { + block = basics.alloc(16) + bytes = block as byte[] + i = 0 + while i < 16 { + bytes[i] = value as byte + i = i + 1 + } + return block +} + // Seeking: backward and forward through keyframes, the world at every // frame as it was the first time through. seeking() { @@ -840,6 +871,14 @@ corruptions() { memcpy(copy, recording.recording_data(r), size) poke_int(copy, 8, 7) ensure("bad snapshot size refused", recording.create_player(copy, size, 1) == null) + // The reserved bytes and the minor version are not the player's to + // refuse (the reference's ReservedHeaderBytes): a newer minor version + // only adds records, which are skipped. + memcpy(copy, recording.recording_data(r), size) + poke_int(copy, 20, 7) + poke_int(copy, 24, 0x12345678) + poke_int(copy, 28, 0 - 1) + ensure("reserved header bytes and another minor version play", recording.validate_replay(copy, size, 1)) // The first record after the snapshot and its state hash. first = 32 + peek_int(recording.recording_data(r), 8) @@ -1312,14 +1351,333 @@ all_ops() { } +// --- names (the reference's test_name_cache.c) ------------------------------------------------------- + +// A ground and four named boxes, each with a named shape: two bodies share +// a name, two shapes share one, a shape shares a body's, and one of each is +// long. Ground is ordinal 0, the boxes 1 to 4. +named_body(i: int) -> string { + if i == 0 { return "crate" } + if i == 1 { return "barrel" } + if i == 2 { return "crate" } + return "a_very_long_body_name_that_exceeds_any_inline_name_buffer" +} + +named_shape(i: int) -> string { + if i == 0 { return "box_hull" } + if i == 1 { return "box_hull" } + if i == 2 { return "crate" } + return "a_very_long_shape_name_that_never_fit_the_old_inline_buffer" +} + +build_named_scene(world_id: WorldId) { + fixtures_ready() + physics_world.set_gravity(world_id, math.vec3(0.0, 0.0 - 10.0, 0.0)) + sdef = dynamics.default_shape_def() + ground_def = dynamics.default_body_def() + ground = dynamics.create_body(world_id, &ground_def) + floor_hull = hull.make_box_hull(20.0, 1.0, 20.0) + dynamics.create_hull_shape(ground, &sdef, floor_hull) + hull.destroy_hull(floor_hull) + i = 0 + while i < 4 { + bdef = dynamics.default_body_def() + bdef.kind = dynamics.BODY_DYNAMIC + bdef.position = math.vec3(0.0, 1.0 + 1.1 * (i as float), 0.0) + bdef.name = named_body(i) + b = dynamics.create_body(world_id, &bdef) + box_def = dynamics.default_shape_def() + box_def.name = named_shape(i) + dynamics.create_hull_shape(b, &box_def, g_box as *HullData) + i = i + 1 + } +} + +// Every box and its shape named as built, in the player's world. +names_hold(p: *Player) -> bool { + i = 0 + while i < 4 { + id = recording.player_body_id(p, 1 + i) + if dynamics.body_is_valid(id) == false { return false } + if string.equals(dynamics.body_get_name(id), named_body(i)) != 1 { return false } + if string.equals(dynamics.shape_get_name(first_shape(id)), named_shape(i)) != 1 { return false } + i = i + 1 + } + return true +} + +// Names live on the world, so the snapshot a recording starts from +// carries them (NameRoundTrip); a seek backward restores them from a +// keyframe every time and leaks nothing (RollbackNames). +names() { + wdef = world_def() + world_id = dynamics.create_world(&wdef) + build_named_scene(world_id) + w = dynamics.get_world(world_id.index1 - 1) + ensure("names: shared names kept once (${basics.name_cache_count(&w.names)})", basics.name_cache_count(&w.names) == 5) + r = recording.create_recording() + recording.start_recording(world_id, r) + steps(world_id, 20) + recording.stop_recording(world_id) + dynamics.destroy_world(world_id) + ensure("names round trip: validates", recording.validate_replay(recording.recording_data(r), recording.recording_size(r), 1)) + p = recording.create_player(recording.recording_data(r), recording.recording_size(r), 1) + recording.player_seek_frame(p, 20) + ensure("names round trip: every name after 20 frames", recording.player_has_diverged(p) == false && names_hold(p)) + recording.destroy_player(p) + recording.destroy_recording(r) + + // Rollback: a recording long enough for several keyframes, scrubbed. + world_id = dynamics.create_world(&wdef) + build_named_scene(world_id) + steps(world_id, 10) + r = recording.create_recording() + recording.start_recording(world_id, r) + steps(world_id, 80) + recording.stop_recording(world_id) + dynamics.destroy_world(world_id) + warm = recording.create_player(recording.recording_data(r), recording.recording_size(r), 1) + recording.player_seek_frame(warm, 80) + recording.destroy_player(warm) + before = basics.alloc_count() + p = recording.create_player(recording.recording_data(r), recording.recording_size(r), 1) + recording.player_seek_frame(p, 80) + ensure("rollback: played to the end", recording.player_frame(p) == 80 && recording.player_has_diverged(p) == false && recording.player_keyframe_count(p) == 5) + targets = [10, 60, 5, 70, 1, 40] + good = 0 + i = 0 + while i < 6 { + recording.player_seek_frame(p, targets[i]) + if recording.player_frame(p) == targets[i] && recording.player_has_diverged(p) == false && names_hold(p) { good = good + 1 } + i = i + 1 + } + ensure("rollback: every name at every frame (${good} of 6)", good == 6) + recording.destroy_player(p) + ensure("rollback: nothing leaked (${basics.alloc_count() - before})", basics.alloc_count() == before) + recording.destroy_recording(r) +} + +// --- queries (the reference's QueryReplay and TaggedQuery) ------------------------------------------- + +// What the host's visitors saw in the frame being watched, to compare +// with what the player keeps. +var g_watching = false +var g_seen: ptr = null // ShapeId[64]: the cast ray's hits, in order +var g_seen_count = 0 +var g_overlaps = 0 + +// An overlap visitor that stops at the second shape: the replay must give +// the same answers to visit the same shapes. +stop_at_second(shape_id: ShapeId, context: ptr) -> bool { + g_overlaps = g_overlaps + 1 + return g_overlaps % 2 != 0 +} + +accept_all(shape_id: ShapeId, context: ptr) -> bool { return true } + +// A cast visitor that clips the ray at each hit (the closest hits come +// after), watched in the frame of interest. +clip_at_hit(shape_id: ShapeId, point: Vec3, normal: Vec3, fraction: float, user_material_id: long, triangle_index: int, child_index: int, context: ptr) -> float { + if g_watching && g_seen_count < 64 { + seen = g_seen as ShapeId[] + seen[g_seen_count] = shape_id + g_seen_count = g_seen_count + 1 + } + return fraction +} + +keep_all(shape_id: ShapeId, point: Vec3, normal: Vec3, fraction: float, user_material_id: long, triangle_index: int, child_index: int, context: ptr) -> float { return 1.0 } + +accept_planes_visitor(shape_id: ShapeId, planes: PlaneResult[], count: int, context: ptr) -> bool { return true } + +// A ground and four spheres for the queries to find. +build_query_scene(world_id: WorldId) { + physics_world.set_gravity(world_id, math.vec3(0.0, 0.0 - 10.0, 0.0)) + sdef = dynamics.default_shape_def() + ground_def = dynamics.default_body_def() + ground = dynamics.create_body(world_id, &ground_def) + floor_hull = hull.make_box_hull(20.0, 1.0, 20.0) + dynamics.create_hull_shape(ground, &sdef, floor_hull) + hull.destroy_hull(floor_hull) + i = 0 + while i < 4 { + b = dynamic_body(world_id, math.vec3((i as float) - 1.5, 3.0, 0.0)) + dynamics.create_sphere_shape(b, &sdef, manifold.sphere(math.vec3_zero(), 0.5)) + i = i + 1 + } +} + +// The seven queries of a frame, one tagged by `tag` when it is not 0. +ask_queries(world_id: WorldId, frame: int) { + filter = shape.default_query_filter() + origin = math.vec3(0.0, 6.0, 0.0) + translation = math.vec3(0.0, 0.0 - 8.0, 0.0) + box = AABB { lower: math.vec3(0.0 - 5.0, 0.0 - 1.0, 0.0 - 5.0), upper: math.vec3(5.0, 6.0, 5.0) } + point_block = calloc(1, sizeof(Vec3)) + proxy = distance.shape_proxy(point_block, 1, 0.5) + mover = Capsule { center1: math.vec3_zero(), center2: math.vec3(0.0, 1.0, 0.0), radius: 0.3 } + g_overlaps = 0 + physics_world.overlap_aabb(world_id, box, filter, stop_at_second, null) + physics_world.overlap_shape(world_id, origin, proxy, filter, accept_all, null) + g_watching = frame == 15 + physics_world.cast_ray(world_id, origin, translation, filter, clip_at_hit, null) + g_watching = false + physics_world.cast_ray_closest(world_id, origin, translation, filter) + physics_world.cast_shape(world_id, origin, proxy, translation, filter, keep_all, null) + physics_world.cast_mover(world_id, origin, mover, translation, filter, false, accept_all, null) + // A mover sunk into the ground's top, so it has planes to give. + physics_world.collide_mover(world_id, math.vec3(0.0, 1.2, 0.0), mover, filter, accept_planes_visitor, null) + free(point_block) +} + +queries() { + fixtures_ready() + g_seen = calloc(64, sizeof(ShapeId)) + wdef = world_def() + world_id = dynamics.create_world(&wdef) + build_query_scene(world_id) + r = recording.create_recording() + recording.start_recording(world_id, r) + frame = 0 + while frame < 30 { + ask_queries(world_id, frame) + physics_world.step(world_id, 1.0 / 60.0, 4) + frame = frame + 1 + } + recording.stop_recording(world_id) + dynamics.destroy_world(world_id) + ensure("queries: the replay asks them again alike", recording.validate_replay(recording.recording_data(r), recording.recording_size(r), 1)) + ensure("queries: and on four workers", recording.validate_replay(recording.recording_data(r), recording.recording_size(r), 4)) + + p = recording.create_player(recording.recording_data(r), recording.recording_size(r), 1) + recording.player_seek_frame(p, 15) + ensure("queries: frame 15 not diverged", recording.player_has_diverged(p) == false && recording.player_frame(p) == 15) + ensure("queries: seven kept for the frame (${recording.player_frame_query_count(p)})", recording.player_frame_query_count(p) == 7) + kinds_ok = true + expected = [recording.QUERY_OVERLAP_AABB, recording.QUERY_OVERLAP_SHAPE, recording.QUERY_CAST_RAY, recording.QUERY_CAST_RAY_CLOSEST, + recording.QUERY_CAST_SHAPE, recording.QUERY_CAST_MOVER, recording.QUERY_COLLIDE_MOVER] + i = 0 + while i < 7 { + if recording.player_frame_query(p, i).kind != expected[i] { kinds_ok = false } + i = i + 1 + } + ensure("queries: in the order asked", kinds_ok) + aabb_query = recording.player_frame_query(p, 0) + ensure("queries: the overlap stopped at its second shape", aabb_query.hit_count == 2 && aabb_query.aabb.lower.x == 0.0 - 5.0 && aabb_query.aabb.upper.y == 6.0) + ray = recording.player_frame_query(p, 2) + seen = g_seen as ShapeId[] + same = ray.hit_count == g_seen_count && g_seen_count > 0 + i = 0 + while same && i < ray.hit_count { + hit = recording.player_frame_query_hit(p, 2, i) + if hit.shape.index1 != seen[i].index1 || hit.shape.generation != seen[i].generation || hit.fraction <= 0.0 || hit.fraction > 1.0 { same = false } + i = i + 1 + } + ensure("queries: the ray's hits as the host saw them (${ray.hit_count} of ${g_seen_count})", same) + ensure("queries: the ray's bounds along its translation", ray.aabb.lower.y == 0.0 - 2.0 && ray.aabb.upper.y == 6.0 && ray.translation.y == 0.0 - 8.0) + closest = recording.player_frame_query(p, 3) + ensure("queries: the closest ray kept its hit", closest.hit_count == 1 && closest.fraction > 0.0 && closest.fraction < 1.0) + mover_cast = recording.player_frame_query(p, 5) + ensure("queries: the mover cast kept its fraction", mover_cast.fraction > 0.0 && mover_cast.fraction < 1.0 && mover_cast.hit_count == 0) + mover_planes = recording.player_frame_query(p, 6) + plane = recording.player_frame_query_hit(p, 6, 0) + ensure("queries: the mover's planes (${mover_planes.hit_count})", mover_planes.hit_count > 0 && plane.normal.y > 0.9 && plane.shape.index1 != 0) + ensure("queries: untagged", aabb_query.key == (0 as long) && aabb_query.filter.id == (0 as long) && aabb_query.filter.name == null) + ensure("queries: none past the ends", recording.player_frame_query(p, 7).kind == 0 - 1 && recording.player_frame_query_hit(p, 2, ray.hit_count).shape.index1 == 0) + recording.player_restart(p) + ensure("queries: a restart forgets them", recording.player_frame_query_count(p) == 0) + recording.player_step_frame(p) + ensure("queries: the next frame's kept alone", recording.player_frame_query_count(p) == 7) + recording.destroy_player(p) + + // A recorded hit changed: the query answers differently, found at the + // frame it was asked in (its step's state check), and the replay goes on. + size = recording.recording_size(r) + copy = calloc(size, 1) + memcpy(copy, recording.recording_data(r), size) + at = 32 + peek_int(copy, 8) + queries_seen = 0 + while at < peek_int(copy, 12) && queries_seen < 4 { + if peek_int(copy, at) == dynamics.OP_QUERY_OVERLAP_AABB { + queries_seen = queries_seen + 1 + // Its payload: the box (48), the filter (16), the count (4), then the first shape's index. + if queries_seen == 4 { poke_int(copy, at + 8 + 68, peek_int(copy, at + 8 + 68) + 1) } + } + at = at + 8 + peek_int(copy, at + 4) + } + p = recording.create_player(copy, size, 1) + while recording.player_step_frame(p) {} + ensure("queries: a changed hit diverges at its frame (${recording.player_diverge_frame(p)})", recording.player_has_diverged(p) && recording.player_diverge_frame(p) == 4 && recording.player_failed(p) == false && recording.player_frame(p) == 30) + recording.destroy_player(p) + free(copy) + recording.destroy_recording(r) + free(g_seen) +} + +// Tagged queries: the caller's id and name hashed into a key that rides a +// tag record, the id and name kept once in the tag table; the same name +// under two ids is two keys; all survive a file round trip; an untagged +// query has no key. +tagged_queries() { + wdef = world_def() + world_id = dynamics.create_world(&wdef) + build_query_scene(world_id) + r = recording.create_recording() + recording.start_recording(world_id, r) + origin = math.vec3(0.0, 6.0, 0.0) + translation = math.vec3(0.0, 0.0 - 8.0, 0.0) + frame = 0 + while frame < 5 { + bullet = shape.default_query_filter() + bullet.id = 42 as long + bullet.name = "bullet" + physics_world.cast_ray_closest(world_id, origin, translation, bullet) + other = shape.default_query_filter() + other.id = 7 as long + other.name = "bullet" + physics_world.cast_ray_closest(world_id, origin, translation, other) + id_only = shape.default_query_filter() + id_only.id = 9 as long + physics_world.cast_ray_closest(world_id, origin, translation, id_only) + physics_world.cast_ray_closest(world_id, origin, translation, shape.default_query_filter()) + physics_world.step(world_id, 1.0 / 60.0, 4) + frame = frame + 1 + } + recording.stop_recording(world_id) + dynamics.destroy_world(world_id) + ensure("tags: saved", recording.save_recording_to_file(r, "target/tagged.aprc")) + loaded = recording.load_recording_from_file("target/tagged.aprc") + p = recording.create_player(recording.recording_data(loaded), recording.recording_size(loaded), 1) + recording.player_seek_frame(p, 3) + a = recording.player_frame_query(p, 0) + b = recording.player_frame_query(p, 1) + c = recording.player_frame_query(p, 2) + d = recording.player_frame_query(p, 3) + ensure("tags: four queries", recording.player_frame_query_count(p) == 4 && recording.player_has_diverged(p) == false) + ensure("tags: the caller's id and name", a.key != (0 as long) && a.filter.id == (42 as long) && a.filter.name != null && string.equals(a.filter.name, "bullet") == 1) + ensure("tags: the same name under another id is another key", b.key != (0 as long) && b.key != a.key && b.filter.id == (7 as long) && string.equals(b.filter.name, "bullet") == 1) + ensure("tags: an id alone has no name", c.key != (0 as long) && c.filter.id == (9 as long) && c.filter.name == null) + ensure("tags: untagged has no key", d.key == (0 as long) && d.filter.id == (0 as long) && d.filter.name == null) + ensure("tags: the key is the reference's hash of the id and name", a.key == dynamics.query_tag_key(42 as long, "bullet") && c.key == dynamics.query_tag_key(9 as long, null)) + recording.player_seek_frame(p, 1) + ensure("tags: the same keys every frame", recording.player_frame_query(p, 0).key == a.key && recording.player_frame_query(p, 1).key == b.key) + recording.destroy_player(p) + recording.destroy_recording(loaded) + recording.destroy_recording(r) +} + main() { round_trips() sessions() geometry_dedup() + geometry_hash_collision() seeking() files() corruptions() all_ops() + names() + queries() + tagged_queries() fixtures_free() if failures > 0 { println("recording: ${failures} of ${checks} checks failed") diff --git a/tests/test_threads.ae b/tests/test_threads.ae index 40d68e3..1c39f05 100644 --- a/tests/test_threads.ae +++ b/tests/test_threads.ae @@ -24,6 +24,7 @@ import aephysics.compound import aephysics.shape import aephysics.dynamics import aephysics.physics_world +import aephysics.recording extern exit(code: int) extern calloc(count: int, size: int) -> ptr @@ -51,6 +52,7 @@ world_id() -> WorldId { return WorldId { index1: g_world_index1, generation: g_w var g_answers: ptr = null // float[3 * QUERIES]: the ray's, the cast's and the overlap's answer var g_done = 0 var g_mismatches = 0 +var g_rounds = ROUNDS query_point(i: int) -> Vec3 { t = (i as float) / (QUERIES as float) @@ -102,7 +104,7 @@ thread_queries() { answers = g_answers as float[] bad = 0 k = 0 - while k < ROUNDS { + while k < g_rounds { i = 0 while i < QUERIES { if ray_answer(i) != answers[3 * i] { bad = bad + 1 } @@ -117,6 +119,26 @@ thread_queries() { native.atomic_add((&g_done) as ptr, 1) } +// The query records of a recording, its framing walked to the end marker. +recorded_queries(r: *Recording) -> int { + data = recording.recording_data(r) + at = 32 + header_int(data, 8) + end = header_int(data, 12) + count = 0 + while at < end { + op = header_int(data, at) + if op >= dynamics.OP_QUERY_OVERLAP_AABB && op <= dynamics.OP_QUERY_COLLIDE_MOVER { count = count + 1 } + if op == dynamics.OP_END { return count } + at = at + 8 + header_int(data, at + 4) + } + return 0 - 1 +} + +header_int(data: ptr, at: int) -> int { + bytes = data as byte[] + return (bytes[at] as int) | ((bytes[at + 1] as int) << 8) | ((bytes[at + 2] as int) << 16) | ((bytes[at + 3] as int) << 24) +} + // One query on a fresh thread, which then exits. var g_short_done = 0 short_thread() { @@ -281,6 +303,27 @@ main() { ensure("${THREADS} threads at once get the serial answers (${native.atomic_load((&g_mismatches) as ptr)} of ${THREADS * ROUNDS * QUERIES * 3} differ)", native.atomic_load((&g_mismatches) as ptr) == 0) + // The same queries from four threads while the world records: each + // query's record goes in whole under the recording's lock, none torn + // by another thread's, and the replay asks every one again alike. + r = recording.create_recording() + recording.start_recording(created, r) + g_rounds = 1 + native.atomic_store((&g_done) as ptr, 0) + native.atomic_store((&g_mismatches) as ptr, 0) + t = 0 + while t < THREADS { + worker.run_detached(| _c: ptr | { thread_queries() }) + t = t + 1 + } + while native.atomic_load((&g_done) as ptr) < THREADS { native.yield_thread() } + recording.stop_recording(created) + ensure("recorded: ${THREADS} threads still get the serial answers", native.atomic_load((&g_mismatches) as ptr) == 0) + ensure("recorded: every query's record whole (${recorded_queries(r)})", recorded_queries(r) == THREADS * QUERIES * 3) + ensure("recorded: the replay asks them all again alike", recording.validate_replay(recording.recording_data(r), recording.recording_size(r), 1)) + recording.destroy_recording(r) + g_rounds = ROUNDS + // Threads that come and go give their slots back: a hundred, one after // another, each holding a slot while it runs. n = 0 From cd41835ed31b3ebe406acffa99e66cc441615653 Mon Sep 17 00:00:00 2001 From: Nicolas Maman Date: Fri, 9 Oct 2026 04:29:48 -0300 Subject: [PATCH 2/2] test_parallel: the pool-joins check waits for the pool instead of racing it A macOS runner's calling thread finished all 10,000 items alone in eight runs in a row: the check asked whether a pool thread wakes before the calling thread could finish, which is the system's scheduling. The calling thread's first block now gives the pool up to two seconds to take a block, so the check holds what it means: the pool joins in. --- tests/test_parallel.ae | 35 ++++++++++++++++++++++++----------- 1 file changed, 24 insertions(+), 11 deletions(-) diff --git a/tests/test_parallel.ae b/tests/test_parallel.ae index 0318a9e..a33a724 100644 --- a/tests/test_parallel.ae +++ b/tests/test_parallel.ae @@ -7,6 +7,7 @@ // thousand rounds of tasks leave the pool as they found it. import std.string import std.sync +import std.os import aephysics.basics import aephysics.native import aephysics.parallel @@ -54,10 +55,22 @@ struct Range { } var g_spin = 0 // the work an item costs, so a block takes long enough for the pool's threads to join +var g_wait_for_pool = false // the calling thread's first block waits for a pool thread to take one +var g_pool_joined = 0 // a pool thread took a block (set atomically) +const POOL_WAIT_NS = 2000000000 range_callback(start: int, end: int, worker: int, context: ptr) { r = context as *Range marks = r.marks as int[] + if worker != 0 { native.atomic_store((&g_pool_joined) as ptr, 1) } + else if g_wait_for_pool { + // The calling thread gives the pool up to two seconds to wake and + // take a block, so the check is whether the pool joins in, not + // whether it wakes before the calling thread could finish alone. + deadline = os.now_monotonic_ns() + (POOL_WAIT_NS as long) + while native.atomic_load((&g_pool_joined) as ptr) == 0 && os.now_monotonic_ns() < deadline { native.yield_thread() } + g_wait_for_pool = false + } i = start while i < end { marks[i] = marks[i] + 1 @@ -144,17 +157,17 @@ main() { r = run_range(s, 10000, 16) ensure("10,000 items in blocks of 16: every item once", every_item_once(r)) ensure("10,000 items: the callback's worker is the thread's", r.worker_index_agreed) - // Whether a pool thread wakes before the main thread has taken every - // block is the system's scheduling: on a loaded machine the main thread - // can finish alone (it failed one run in four under load). The pool - // joining in is what is held, so a run where it did not is tried again. - attempts = 1 - while workers_used(r) < 2 && attempts < 8 { - free_range(r) - r = run_range(s, 10000, 16) - attempts = attempts + 1 - } - ensure("10,000 items: more than one worker took blocks (${workers_used(r)}, in ${attempts} runs)", workers_used(r) >= 2) + // Whether a pool thread wakes before the calling thread has taken every + // block is the system's scheduling: on a loaded machine the calling + // thread can finish alone (a macOS runner did, eight runs in a row). The + // pool joining in is what is held, so the calling thread's first block + // waits for a pool thread to take one. + free_range(r) + native.atomic_store((&g_pool_joined) as ptr, 0) + g_wait_for_pool = true + r = run_range(s, 10000, 16) + ensure("10,000 items, the pool given time: every item once", every_item_once(r)) + ensure("10,000 items: more than one worker took blocks (${workers_used(r)})", workers_used(r) >= 2) ensure("10,000 items: blocks of 16 until the count bounds them (${r.smallest_block}, ${r.largest_block})", r.largest_block >= 16 && r.largest_block <= 10000) ensure("10,000 items: the block count parallel_for reports is the range over the largest block (${r.block_count})", r.block_count == (10000 + r.largest_block - 1) / r.largest_block) free_range(r)