Recording and replay (#149, second part)

[nicolas-maman]

The second part of #149: recording and replay, as the reference's recording.c and recording_replay.c have them. The world queries and the name cache are the third part, so #149 stays open.

Recording

• aephysics.recording: create_recording, start_recording, stop_recording, save_recording_to_file, load_recording_from_file, destroy_recording.
• A recording is a 32-byte header (magic, major and minor version, snapshot size, the registry's offset and size), then:
  • a snapshot of the world it started from, and that world's state hash;
  • one record per call that changes the world (opcode, length, arguments field by field, never a struct's bytes);
  • a state hash after every step, the joined bounds and an end marker;
  • the geometry the calls named, each block once, found again by content hash and then every byte.
• Every one of the 160 recorded calls in dynamics, physics_world and joint_solver records itself in its first line. The hooks, the struct writers and readers, and the replay's dispatch cases are generated from the functions' signatures, so they cannot drift apart.
• A create's record ends with the id it made, which the replay must make too. The 17 creates are now thin recorded wrappers over an _internal body.
• Hosts can't misuse a recording mid-session: a recording another world is writing can't be started into, and freeing one stops its world first.

Replay

The player matches the reference's b3RecPlayer function for function: step, sub-step, restart, seek, info, worker count, keyframe policy, body ordinals and validate.

• In-place restore. The world is restored where it stands, so its id stays the same through a restart or a backward seek. This uses the new dynamics.reset_world, the reference's deserialize-into-shell, and snapshot.restore_world on top of it. Workers, callbacks, filters and user data are kept.
• Keyframes. One every 16 frames, under a 512 MiB budget. Over the budget, the spacing doubles and the keyframes off the wider grid go.
• Bodies. Kept in the order they were made, with a hole for each destroyed one, so a viewer's selection survives seeking.
• Failure and divergence. A record the player does not know is skipped, as a newer minor version may add some. A record that does not read whole, or a create that makes another id, stops the replay: the player has failed. A state hash that differs marks its frame as diverged, and the replay goes on.

Bugs found on the way

ae build type-checks only what the program reaches. Calling every recorded function from the dispatcher exposed one latent error, and running ae check on every module found two more:

• body_apply_force added a Vec3 to the Vec3f force accumulator, so it never compiled. It had no caller until now.
• body_get_local_point_velocity read BodyState fields that do not exist.
• mesh called hull.hull_proxy without importing hull; it only worked because manifold imports it.

All three are fixed and now tested. scripts/test.sh runs ae check on every module first, and a warning fails it too. The Aether side is filed as aether#2613 (uncalled functions in imported modules are never checked) and aether#2614 (ae build and ae check disagree about calls into a module that isn't imported).

Tests

• test_recording.ae, 116 checks:
  • the snapshot test's scene recorded from an empty world and from mid stream, with the host changing it between steps: velocities, impulses, forces, names, materials, filters, types, locks, mass, a hinge made, driven and destroyed, bodies destroyed and their slots reused, geometry the host frees at once;
  • played on one worker and on several, every frame's hash as recorded, and the bodies ending where the recorded ones did;
  • AllOps: every one of the 160 opcodes present in a recording, and its replay ending in the same world;
  • geometry dedup across two copies of one hull;
  • seeking with keyframes, with none in reach, and under a budget that widens the spacing;
  • sub-step pause, info, worker count changed mid replay, restart keeping the world id;
  • file round trip;
  • every truncation refused; a skipped call, a changed created id, a short record and a changed hash each caught; nothing leaked.
• test_snapshot.ae, 63 checks: restore in place keeps the id, workers and user data, steps on alike and saves to the same bytes; a refused restore leaves the world empty and whole, and leaks nothing.
• test_dynamics.ae: a force at a point equals a force at the centre plus its arm's torque, bit for bit; local and world point velocities agree on a turned body.

Pins

Aether 0.796.0 (latest release) and aeb v0.325 (latest). The whole gate, golden trace included, passed locally on 0.796.0.