route-oracle: measure a plan's stored route against the navmesh

README.md

@@ -95,6 +95,7 @@ turtle-agent map [--capture FILE] [--events FILE] [--report FILE] [--botwatch FI
 turtle-agent tune-diff <trace.jsonl>
 turtle-agent plan-export [--dir dungeons] [--allow-draft] [--drop-companions-loot] [--mmaps DIR] [--rev SHA] [--stamp YYYYMMDDHHMMSS] [--force] --out FILE|DIR (<plan>... | --all-live)
 turtle-agent plan-check [--dir dungeons] --mmaps DIR [--from-leg N [--from-pull-group N]] [--json] <plan>...
+turtle-agent route-oracle --plans DIR --mmaps DIR --out DIR [--mode walk|bosses] [--objectives plan|bosses] [--plan NAME]...
 turtle-agent dashboard [--listen 127.0.0.1:8790] [--allow-host HOST[,HOST]] [--agents auto|<rpc,...>] [--tests-dir DIR] [--traces-dir DIR] [--plans-dir DIR[,DIR]] [--minimap-dir DIR] [--embed-from ORIGIN[,ORIGIN]] [--config-dir DIR]
 turtle-agent minimap-extract --client DIR --out DIR [--maps 70,209|all-instances] [--mmaps DIR[,DIR]] [--prefer worldmap,minimap,navmesh] [--list]
 turtle-agent trace-score [--json] [--summary] [--set check=threshold]... <trace.jsonl>...
@@ -855,6 +856,27 @@ that only the server's own pathing gives (a tank with no firing position), so a
 has navmesh files here (`$TURTLE_AGENT_MMAPS`, `~/tb-ci/mmaps-turtle<map>` or `~/tortoise-wow-server/data/mmaps`); the plans that fail today are listed in `planCheckKnownFailing` (issue #187; most are drafts, which only warn);
 a listed plan that passes fails the test until it is taken off. The gate itself is tested with a stub navmesh, so it runs in CI without map files.
 
+### Route oracle
+
+`turtle-agent route-oracle --plans dungeons --mmaps DIR --out DIR` compares a plan's **stored route** with what the navmesh **plans live** for the same dungeon, and writes `<out>/<plan>.txt` (a human table) and `<out>/<plan>.json` (the machine report, Stage 1's fixture). It reads plan files and mmaps and nothing else; a plan's status, live or draft, changes nothing (the report is a measurement, not a gate). Without `--plan` it reports the five mapped plans: `wailing_caverns ragefire_chasm uldaman zulfarrak deadmines`.
+
+The walk it plans is the driver's own. Per leg the leg's stored points in the order planexport writes them — the `via` points, `to`, each pull group's `approach` points and its pull point (the positions' centroid, or a hold's spot), the steps' stand points, the boss's approach points and position, the after-boss stands — and every hop between two of them is planned twice: on the plain planner, and on a scratch planner with `SetAvoid` of the plan's avoid boxes. The avoid-aware paths are the live route the metrics use, because it is the one a run plans with; each stored hop is scored against its own navmesh hop (per-segment rows in the JSON, per-leg sums in the table).
+
+A hop that does not plan complete **fails its leg**: the leg takes the failing hop's status (`partial`, `no_path`), the leg's remaining hops are planned on from the point the failed path reached and marked `non_comparable` (that walk is not the plan's, so it scores nothing), and the legs after a failed leg are `indeterminate`. Every aggregate — the mean ratio, the packs on path, the break and redundancy counts — is over the complete legs only; the failed and indeterminate legs stay in the report for reading.
+
+Per leg the report holds: the status, end gap and live length of both runs; the stored hops' length and their ratio to the live length; the most and the mean distance of the leg's stored points to the live route; the route's Z jumps (consecutive samples of the corners and their ground sampling more than 8 yd apart in Z: ground the mesh joins without a climb); the leg's **breaks** — `via` points with `exact`, `direct`, `gm_cross` or `check_skip` — flagged when the live route passes within 6 yd of the foot, or when the failing hop is partial and its path's end stands within 10 yd of the foot; each pull group's corridor membership (one of its positions within the group's radius + 18 yd of the route, no more than 8 yd above or below it); and, for a complete leg, how many of the approach and via points are **redundant**: the navmesh path from the previous to the next stored point passes within 5 yd of the point, so the route would lose nothing without it. The plan's line sums it up, and the report names the SHA-256 of every `.mmap`/`.mmtile` file the planning used.
+
+The report is written under the **validated CLI plan name** (never the name inside the plan JSON — a plan file that says `name: "../escape"` is refused, and the destination must resolve under `--out`). `internal/routeoracle` holds the report; its tests plan over a synthetic mesh (`internal/navmesh/navfixture`) and need no real mmaps.
+
+**Modes.** `--mode walk` (the default) is everything above: the plan's own stored sequence, hop by hop. `--mode bosses` asks a different question, how much of a route the live navmesh finds with the plan's hand data left out: each objective is ONE avoid-aware navmesh plan from where the party stands, using no hand point and skipping no break hop (a `direct`, `gm_cross` or `check_skip` via). It writes `<plan>-bosses.txt/.json`. `--objectives` picks the objectives:
+
+- `--objectives plan` (the default): each stored leg's own objective, its boss, else its `to`, else its last step's point. A leg without a boss is planned to a stored `to` point (a hand-chosen waypoint), so this is not a bare boss list.
+- `--objectives bosses` (needs `--mode bosses`): ONLY the chain entrance -> boss 1 -> boss 2 -> ...; a leg without a boss is folded into the next boss's row (its pull groups, stands and `to` are measured against that row's live path; no stored `to` is an objective), and the legs after the last boss are not planned. It writes `<plan>-bosses-only.txt/.json`.
+
+An objective the mesh cannot reach is a failed leg (`partial` / `no_path`): it scores nothing, names the stored break it ran into (a break hop whose foot the mesh reaches and whose landing reaches the objective, or one within 15 yd of the live end or of the line to the objective), and the objective after it is planned from the last point reached and labelled non-comparable (excluded from every aggregate); it is also re-planned from the previous objective (`bridged`), beside the primary numbers. Both modes judge a pack the same way: one of its positions within its radius plus 18 yd of the route, 8 yd of height. A pull group with neither a position nor a hold is skipped and counted (`empty_groups`).
+
+**Output safety.** `--out` is refused when it is, or has a component that is, a symlink (a host where `/tmp` or `/home` is itself a symlink must name the real folder). A report file that already exists as anything but a regular file (a symlink, even a dangling one, a folder) is refused, and each file is written to a temp file in the folder and renamed over the target, so no existing link is followed. Documented limit: the folder is validated by path, so a concurrent rename of a writable ancestor between the check and the open could redirect the writes; this is a local, owner-run tool and the limit is accepted.
+
 ### Exporting plans for the TortoiseBots run leader
 
 `turtle-agent plan-export` writes the plan file as rows for the TortoiseBots run leader (`.bot run`; its tables `ai_dungeon_plan`, `ai_dungeon_leg`,

cmd/turtle-agent/main.go

@@ -90,6 +90,8 @@ func run(ctx context.Context, args []string, stdout, stderr io.Writer) int {
 		return planExportCmd(args[1:], stdout, stderr)
 	case "plan-check":
 		return planCheckCmd(args[1:], stdout, stderr)
+	case "route-oracle":
+		return routeOracleCmd(args[1:], stdout, stderr)
 	case "route-edits":
 		return routeEditsCmd(ctx, args[1:], stdout, stderr)
 	case "version":
@@ -117,6 +119,7 @@ func usage(w io.Writer) {
   turtle-agent minimap-extract --client DIR --out DIR [--maps 70,209|all-instances] [--mmaps DIR] [--list]
   turtle-agent plan-export [--dir dungeons] [--allow-draft] [--drop-companions-loot] [--mmaps DIR] [--rev SHA] [--stamp YYYYMMDDHHMMSS] [--force] --out FILE|DIR (<plan>... | --all-live)
   turtle-agent plan-check [--dir dungeons] --mmaps DIR [--from-leg N [--from-pull-group N]] [--json] <plan>...
+  turtle-agent route-oracle --plans DIR --mmaps DIR --out DIR [--plan NAME]...
   turtle-agent route-edits list [--edit-dir DIR] [--json] [<plan>] | export [--edit-dir DIR] [--config-dir DIR] --to DIR <plan> [vN|latest] | promote [--edit-dir DIR] --worktree PATH <plan> vN
   turtle-agent version`)
 }

cmd/turtle-agent/route_oracle_bosses_text.go

@@ -0,0 +1,139 @@
+package main
+
+import (
+	"bytes"
+	"fmt"
+	"text/tabwriter"
+
+	"github.com/pjerra/turtle-agent/internal/routeoracle"
+)
+
+// failedLine says why a leg's live plan did not reach its objective and which stored break it
+// ran into.
+func failedLine(l *routeoracle.BossLeg) string {
+	s := fmt.Sprintf("leg %d (%s): %s, the live path ends %.1f yd short of the %s at %.1f,%.1f,%.1f (from %.1f,%.1f,%.1f)",
+		l.Leg, l.Name, l.Status, l.Live.EndGapYd, l.Objective, l.To.X, l.To.Y, l.To.Z, l.From.X, l.From.Y, l.From.Z)
+	if l.Status == "partial" {
+		s += fmt.Sprintf("; it stops at %.1f,%.1f,%.1f", l.Live.End.X, l.Live.End.Y, l.Live.End.Z)
+	}
+	if len(l.BreakHits) == 0 {
+		return s + "; no stored break bridges it or lies within 15 yd of the end or of the line to the objective"
+	}
+	for _, b := range l.BreakHits {
+		s += fmt.Sprintf("; stored break leg %d via %d (%s) at %.1f,%.1f,%.1f (%s)", b.Leg, b.Via, b.Kind, b.At.X, b.At.Y, b.At.Z, b.Why)
+	}
+	return s
+}
+
+// failedBossLegs is the report's failed legs, one line each: the fair failures only (a
+// non-comparable continuation is not a finding about the plan).
+func failedBossLegs(rep routeoracle.BossesReport) []string {
+	var out []string
+	for i := range rep.Legs {
+		if rep.Legs[i].Failed() {
+			out = append(out, failedLine(&rep.Legs[i]))
+		}
+	}
+	return out
+}
+
+// bossesText is the plan's human table: the summary, one line per leg, the failed legs with the
+// breaks they hit, the non-comparable continuations, then what the complete legs say about the
+// stored hand data (packs off the live path, hand points farther than 10 yd, floor jumps).
+func bossesText(rep routeoracle.BossesReport) ([]byte, error) {
+	var buf bytes.Buffer
+	w := tabwriter.NewWriter(&buf, 0, 4, 2, ' ', 0)
+	fmt.Fprintf(w, "%s (map %d, status %q, %d avoid boxes), bosses mode, objectives: %s\n", rep.Plan, rep.Map, rep.Status, rep.Avoid, rep.Objectives)
+	if rep.Objectives == routeoracle.ObjectivesBosses {
+		fmt.Fprintf(w, "boss-only chain: entrance -> boss 1 -> boss 2 ...; a leg without a boss is folded into the next boss (name a+b), no stored to point is an objective; legs after the last boss are not planned: %v\n", rep.Unplanned)
+	}
+	fmt.Fprintln(w, rep.Summary())
+	fmt.Fprintln(w, "each leg is one avoid-aware navmesh plan from the previous objective to this leg's objective: no hand points, no break hops;")
+	fmt.Fprintln(w, "a leg that does not reach it FAILED and scores nothing; a leg planned from a failed leg's end is non-comparable (NC) and scores nothing")
+	fmt.Fprintf(w, "plan time %.0f ms\n", rep.PlanMs)
+	fmt.Fprintln(w, "packs: a pack position within its radius plus 18 yd of the live path (8 yd of height), the same definition as the walk mode; hand: via/to/approach/stand points within 10 yd of it (redundant); NC: non-comparable; bridged: the NC leg planned from the previous objective")
+	fmt.Fprintln(w, "leg\tname\tobjective\tfrom\tto\tstatus\tgap\tlive\tpacks\thand\tzj\tbreaks hit\tNC\tbridged")
+	for i := range rep.Legs {
+		l := &rep.Legs[i]
+		nc := ""
+		if l.NonComparable {
+			nc = "NC"
+		}
+		if l.Status == "complete" {
+			fmt.Fprintf(w, "%d\t%s\t%s\t%.0f,%.0f\t%.0f,%.0f\t%s\t%.1f\t%.1f\t%d/%d\t%d/%d\t%d\t-\t%s\t%s\n",
+				l.Leg, l.Name, l.Objective, l.From.X, l.From.Y, l.To.X, l.To.Y, l.Status, l.Live.EndGapYd, l.Live.LiveLenYd,
+				l.PacksOnPath, len(l.Packs), l.HandRedundant, len(l.Hand), len(l.ZJumps), nc, bridgedOf(l))
+			continue
+		}
+		fmt.Fprintf(w, "%d\t%s\t%s\t%.0f,%.0f\t%.0f,%.0f\t%s\t%.1f\t%.1f\t-\t-\t-\t%d\t%s\t%s\n",
+			l.Leg, l.Name, l.Objective, l.From.X, l.From.Y, l.To.X, l.To.Y, l.Status, l.Live.EndGapYd, l.Live.LiveLenYd, len(l.BreakHits), nc, bridgedOf(l))
+	}
+	for i := range rep.Legs {
+		l := &rep.Legs[i]
+		switch {
+		case l.Failed():
+			fmt.Fprintf(w, "FAILED %s\n", failedLine(l))
+			if l.Reason != "" {
+				fmt.Fprintf(w, "  planner: %s\n", l.Reason)
+			}
+		case l.NonComparable:
+			fmt.Fprintf(w, "NON-COMPARABLE leg %d (%s): %s, %s: %s\n", l.Leg, l.Name, l.Status, l.NonComparableWhy, l.Reason)
+			if b := l.Bridged; b != nil && b.Status != "complete" {
+				fmt.Fprintf(w, "  BRIDGED leg %d is a second failure: %s\n", l.Leg, failedLine(b))
+			}
+		}
+	}
+	for i := range rep.Legs {
+		l := &rep.Legs[i]
+		if !l.Counted() {
+			continue
+		}
+		for _, g := range l.Packs {
+			if !g.OnPath {
+				fmt.Fprintf(w, "leg %d (%s): pack group %d of stored leg %d (entries %v) nearest position %.1f yd (dz %.1f) from the live path, radius %.0f yd\n",
+					l.Leg, l.Name, g.Group, legOf(g.Leg), g.Entries, g.NearYd, g.DZYd, g.RadiusYd)
+			}
+		}
+		for _, h := range l.Hand {
+			if !h.Redundant {
+				fmt.Fprintf(w, "leg %d (%s): hand %s (stored leg %d, seq %d) at %.1f,%.1f,%.1f is %.1f yd from the live path\n",
+					l.Leg, l.Name, h.Kind, legOf(h.Leg), h.Seq, h.At.X, h.At.Y, h.At.Z, h.NearYd)
+			}
+		}
+		for _, b := range l.Breaks {
+			fmt.Fprintf(w, "leg %d (%s): stored break (stored leg %d) via %d (%s) at %.1f,%.1f,%.1f: live path passes it %v\n",
+				l.Leg, l.Name, legOf(b.Leg), b.Via, b.Kind, b.At.X, b.At.Y, b.At.Z, b.Touched)
+		}
+		for _, j := range l.ZJumps {
+			fmt.Fprintf(w, "leg %d (%s): z jump %.1f yd at %.1f,%.1f,%.1f\n", l.Leg, l.Name, j.DropYd, j.At[0], j.At[1], j.At[2])
+		}
+	}
+	fmt.Fprintln(w, "tiles used (sha-256):")
+	for _, t := range rep.Tiles {
+		fmt.Fprintf(w, "  %s\t%s\t%d bytes\n", t.Name, t.Sha256, t.Bytes)
+	}
+	if err := w.Flush(); err != nil {
+		return nil, err
+	}
+	return buf.Bytes(), nil
+}
+
+// bridgedOf is the table cell of a non-comparable leg's bridged re-plan.
+func bridgedOf(l *routeoracle.BossLeg) string {
+	if l.Bridged == nil {
+		return "-"
+	}
+	if l.Bridged.Status == "complete" {
+		return fmt.Sprintf("complete %.0f yd, packs %d/%d, hand %d/%d", l.Bridged.Live.LiveLenYd, l.Bridged.PacksOnPath, len(l.Bridged.Packs),
+			l.Bridged.HandRedundant, len(l.Bridged.Hand))
+	}
+	return fmt.Sprintf("%s, %.0f yd short", l.Bridged.Status, l.Bridged.Live.EndGapYd)
+}
+
+// legOf is a folded row's stored leg index (0 when the field is absent).
+func legOf(p *int) int {
+	if p == nil {
+		return 0
+	}
+	return *p
+}