Load the map at the speed the cache made possible (#136) (#139)

* Load the map at the speed the cache made possible (#136)

Three causes, each undoing part of what #134 bought.

**The parse brake was throttling cache hits.** The 250 ms gap exists so
back-to-back region PARSES do not hold the main thread — and it was applied to
every job in the queue, including the ones that read a cached region in about a
millisecond. After the disk cache landed, most reads are cache hits, so the
brake had ended up throttling almost exclusively the fast path. That is why
loading felt no quicker with the cache than without it. The queue now asks
whether a job will actually parse before waiting for anything.

**The client only asked again on the two-second position poll.** `mapFetchTiles`
runs from `mapDraw`, so a viewport needing six requests filled in over twelve
seconds, in visible bands — which is the "it loads chunk by chunk" in the
report. While anything is still pending it asks again in 180 ms.

**Empty chunks were requested forever.** A chunk was marked "known empty" only
when the whole response reported nothing pending, and on a busy viewport
something always is — so a chunk the server had read and found empty was never
marked and came back on every draw. The response now names them: `empty` is "I
read that region and there is nothing there", which is a different fact from
"not read yet", and the client can tell them apart. This is the "sometimes it
never loads".

**And an empty area now says so.** Ungenerated chunks were black, which is
indistinguishable from still-loading and from broken — an operator was waiting
for a load that was never coming. They are drawn as a faint hatch in the site's
own accent colour, with a line saying nobody has been there, once enough of the
view is empty to be worth explaining.

* Self-review: measure the parse instead of predicting it, and stop an infinite poll

I gated the brake behind a `willParse()` that re-stat'd the region file and
checked for a cache file — to guess at work the very next call was about to do
anyway. Two stats per queued job, thousands of jobs. The queue runs the job and
then looks at whether `lastParseAt` moved: no prediction, no extra filesystem
work, and the brake applies exactly when a parse happened rather than when one
was expected.

Worse, and mine from this PR: asking again every 180 ms while anything is
pending turns an unreadable region into a permanent polling loop. `loadRegion`
returned null on a read failure, `peekChunkTile` therefore answered "not read
yet" forever, and the client would have asked for that chunk every 180 ms for as
long as the page stayed open. A region that cannot be read is remembered as
EMPTY — which is honest, there is nothing to draw — and terminates.

Also dropped a `pending` counter I computed in the ungenerated-area note and
never used.

Author: CaYatur

src/main/core/worldTiles.ts

@@ -536,13 +536,28 @@ function loadRegion(
   }
 
   lastParseAt = Date.now()
+  /**
+   * A region that cannot be read is remembered as EMPTY rather than as "not
+   * read yet".
+   *
+   * Returning null leaves `peekChunkTile` answering `undefined` forever, so the
+   * client keeps asking, the queue keeps re-running it, and the chunk never
+   * resolves — one unreadable region file is a permanent polling loop. An empty
+   * entry is honest (there is nothing to draw) and terminates.
+   */
+  const giveUp = (): RegionEntry => {
+    const empty: RegionEntry = { at: Date.now(), mtimeMs, tiles: new Map() }
+    regions.set(path, empty)
+    return empty
+  }
+
   let file: Buffer
   try {
     file = readFileSync(path)
   } catch {
-    return null
+    return giveUp()
   }
-  if (file.length < SECTOR_HEADER) return null
+  if (file.length < SECTOR_HEADER) return giveUp()
 
   const table = parseLocationTable(file.subarray(0, 4096))
   const tiles = new Map<number, ChunkTile | null>()
@@ -629,12 +644,19 @@ export function requestTiles(
   dim: string,
   want: { cx: number; cz: number }[],
   opts: { marks?: boolean } = {}
-): { tiles: Record<string, { c: number[]; h: number[]; m?: StructureMark[] }>; pending: number } {
+): {
+  tiles: Record<string, { c: number[]; h: number[]; m?: StructureMark[] }>
+  /** Chunks read and found to hold nothing — as opposed to not read yet. */
+  empty: string[]
+  pending: number
+} {
   const tiles: Record<string, { c: number[]; h: number[]; m?: StructureMark[] }> = {}
+  const empty: string[] = []
   const missing: { cx: number; cz: number }[] = []
   for (const w of want) {
     const t = peekChunkTile(serverId, dim, w.cx, w.cz)
     if (t === undefined) missing.push(w)
+    else if (!t) empty.push(w.cx + ',' + w.cz)
     // Structures are omitted unless asked for. They are a spoiler, and a
     // payload that carries them "in case" is one the public feed could leak.
     else if (t) {
@@ -652,27 +674,35 @@ export function requestTiles(
     }
   }
   if (missing.length && !working) void drain()
-  return { tiles, pending: missing.length }
+  return { tiles, empty, pending: missing.length }
 }
 
 async function drain(): Promise<void> {
   working = true
   try {
     while (queue.length) {
-      // Yielding between chunks is not enough on its own: the first chunk of an
-      // unseen region parses the whole file. Wait for the parse budget.
-      if (!parseBudgetReady(queue[0]?.serverId)) {
-        await new Promise((r) => setTimeout(r, 60))
-        continue
-      }
       const job = queue.shift()
       if (!job) break
+      // The budget is a brake on PARSING, and applying it to every job throttled
+      // the cache hits too — which after #134 is almost every read, and is why
+      // loading felt no quicker with the cache than without it (#136).
+      //
+      // Measured rather than predicted: run the job, then look at whether it
+      // parsed. Predicting meant re-stat'ing the region file to guess at work
+      // the very next call was about to do anyway.
+      const before = lastParseAt
       try {
         chunkTile(job.serverId, job.dim, job.cx, job.cz)
       } catch {
         /* one bad region must not stop the queue */
       }
-      await new Promise((r) => setImmediate(r))
+      if (lastParseAt !== before) {
+        // It really parsed. Stand back for the configured gap so the console
+        // reader and everything else on this thread get a turn.
+        await new Promise((r) => setTimeout(r, perfFor(job.serverId).parseGapMs))
+      } else {
+        await new Promise((r) => setImmediate(r))
+      }
     }
   } finally {
     working = false

src/renderer/src/components/LiveMap.tsx

@@ -277,15 +277,22 @@ export function LiveMap({ serverId }: { serverId: string }): JSX.Element {
       .mapTiles(serverId, dim, want, marks)
       .then((r) => {
         tilesPending.current = false
+        // The empty list is "read, and nothing there" — as opposed to "not read
+        // yet". Marking null only when the whole response had nothing pending
+        // meant a genuinely empty chunk was re-requested on every draw (#136).
+        const known = new Set(r.empty ?? [])
         for (const w of want) {
           const k = w.cx + ',' + w.cz
           const t = r.tiles[k]
           if (t) {
             tiles.current.set(k, bakeTile(t))
             if (t.m) markStore.current.set(k, t.m)
-          } else if (!r.pending) tiles.current.set(k, null)
+          } else if (known.has(k) || !r.pending) tiles.current.set(k, null)
         }
         setTick2((n) => n + 1)
+        // Ask again while anything is still coming, rather than waiting for the
+        // 2-second position poll — that wait is why a view filled in bands.
+        if (r.pending > 0) window.setTimeout(() => setTick2((n) => n + 1), 180)
       })
       .catch(() => {
         tilesPending.current = false

src/shared/ipc.ts

@@ -351,6 +351,8 @@ export interface MsmsApi {
     marks?: boolean
   ): Promise<{
     tiles: Record<string, { c: number[]; h: number[]; m?: StructureMark[] }>
+    /** Chunks read and found to hold nothing, as opposed to not read yet. */
+    empty: string[]
     pending: number
   }>
   /** Drop every cached region. Returns how many files went. */

src/shared/mapUi.ts

@@ -41,6 +41,13 @@ export const MAP_CSS = `
   font-size:12px;font-variant-numeric:tabular-nums;pointer-events:none;
   background:rgba(0,0,0,.55);color:#fff}
 .mp-cursor:empty{display:none}
+/* Said in words as well as drawn, so "nobody has been here" is not left to a
+   faint hatch nobody reads as deliberate (#136). */
+.mp-ungen{position:absolute;right:10px;top:10px;max-width:230px;padding:7px 11px;border-radius:10px;
+  font-size:12px;line-height:1.35;pointer-events:none;
+  border:1px dashed color-mix(in srgb,var(--accent,#dc2727) 45%,transparent);
+  background:color-mix(in srgb,var(--accent,#dc2727) 10%,rgba(0,0,0,.55));color:#fff}
+.mp-ungen.hidden{display:none}
 .mp-legend{display:flex;flex-wrap:wrap;gap:10px;font-size:12px;opacity:.75}
 .mp-legend .mp-hint{margin-left:auto;opacity:.6}
 .mp-legend b{font-weight:800;opacity:1}
@@ -89,6 +96,7 @@ export const MAP_HTML = `
   <div class="mp-canvas-wrap">
     <canvas id="mpCanvas"></canvas>
     <div id="mpEmpty" class="mp-empty"></div>
+    <div id="mpUngen" class="mp-ungen hidden"></div>
     <div id="mpCursor" class="mp-cursor"></div>
   </div>
   <div class="mp-legend">
@@ -327,15 +335,24 @@ function mapFetchTiles(){
  mapGet(mapTilesUrl(MAP.dim,want.map(function(c){return c.cx+','+c.cz}).join(';'),MAP.marksOn)).then(function(d){
   MAP_TILE_PENDING=false;
   if(!d||!d.tiles)return;
+  /* The empty list is the server saying "I have read that region and there is
+     nothing in that chunk". Marking a chunk null only when the WHOLE response had
+     nothing pending was the bug: on a busy viewport something is always
+     pending, so genuinely empty chunks were never marked and were re-requested
+     on every single draw, forever (#136). */
+  var known={};
+  for(var e=0;e<(d.empty||[]).length;e++)known[d.empty[e]]=1;
   for(var i=0;i<want.length;i++){
    var k=mapTileKey(want[i].cx,want[i].cz);
    var t=d.tiles[k];
-   /* null, not undefined, for a chunk the server has parsed and found empty —
-      otherwise it is re-requested forever. A pending one is left unset so the
-      next poll asks again. */
    if(t){MAP_TILES[k]=mapBakeTile(t);if(t.m)MAP_MARKS[k]=t.m}
-   else if(!d.pending)MAP_TILES[k]=null}
-  mapDraw()})}
+   else if(known[k]||!d.pending)MAP_TILES[k]=null}
+  mapDraw();
+  /* Ask again straight away while anything is still coming. Waiting for the
+     2-second position poll is why a viewport filled in visible bands over ten
+     seconds instead of arriving at once. */
+  if(d.pending>0){clearTimeout(MAP_TILE_SOON);MAP_TILE_SOON=setTimeout(mapFetchTiles,180)}})}
+var MAP_TILE_SOON=null;
 /* Structure markers. Off by default: they are a spoiler for the players and
    clutter for everyone else. The server decides whether they arrive at all —
    the public feed sends none unless an operator published them. */
@@ -401,6 +418,42 @@ function mapBakeTile(t){
   img.data[o+2]=Math.max(0,Math.min(255,Math.round((c&255)*f)));
   img.data[o+3]=255}
  g.putImageData(img,0,0);return cv}
+/* Area nobody has ever been to.
+   Drawn as a deliberate, themed hatch rather than left black, because black is
+   indistinguishable from "still loading" and from "broken" — an operator was
+   waiting for a load that was never coming (#136). */
+function mapDrawUngenerated(g,w,h,dpr){
+ if(!MAP.world||!MAP.view)return;
+ var b=mapChunkBox();if(!b)return;
+ if((b.x1-b.x0+1)*(b.z1-b.z0+1)>4096)return;
+ var sx=w/MAP.vp.width,sy=h/MAP.vp.height;
+ var size=16*MAP.view.scale;
+ /* Too small to read as anything but noise; leave it plain. */
+ if(size*sx<3)return;
+ var accent=mapAccent();var ungen=0;
+ for(var cz=b.z0;cz<=b.z1;cz++)for(var cx=b.x0;cx<=b.x1;cx++){
+  if(MAP_TILES[mapTileKey(cx,cz)]!==null)continue;
+  var p=mapW2S({x:cx*16,z:cz*16});
+  var x=p.x*sx,y=p.y*sy,d=size*sx,dh2=size*sy;
+  g.fillStyle='rgba('+accent+',0.055)';
+  g.fillRect(x,y,d+1,dh2+1);ungen++}
+ mapUngenNote(ungen,(b.x1-b.x0+1)*(b.z1-b.z0+1))}
+/* How much of the view is area nobody has generated, said in words. A hatch on
+   its own is just a colour; the sentence is what stops someone waiting. */
+function mapUngenNote(ungen,total){
+ var el=document.getElementById('mpUngen');if(!el)return;
+ var show=total>0&&ungen/total>0.15;
+ el.classList.toggle('hidden',!show);
+ if(show)el.textContent='Shaded area has never been generated — no player has been there, so there is nothing to draw.'}
+/* The site accent as an "r,g,b" triple, so the hatch belongs to the theme
+   rather than being a hardcoded red. */
+function mapAccent(){
+ try{
+  var v=getComputedStyle(document.documentElement).getPropertyValue('--accent').trim();
+  var m=/^#([0-9a-f]{6})$/i.exec(v);
+  if(m){var n=parseInt(m[1],16);return ((n>>16)&255)+','+((n>>8)&255)+','+(n&255)}
+ }catch(e){}
+ return '220,39,39'}
 function mapDrawTiles(g,w,h){
  if(!MAP.world)return;
  var chunks=mapDrawableChunks();
@@ -471,6 +524,7 @@ function mapDraw(){
  var g=cv.getContext('2d');g.clearRect(0,0,w,h);
  var sx=w/MAP.vp.width,sy=h/MAP.vp.height;
  /* The world first: everything else is drawn on top of it. */
+ mapDrawUngenerated(g,w,h,dpr);
  mapDrawTiles(g,w,h);
  mapFetchTiles();
  var px=function(x){return mapW2S({x:x,z:0}).x*sx};