Skip to content

Commit 417535a

Browse files
Migrate onto dash-improve-my-llms 2.3.3 and the network directory
Works through handoff/existing_subdomains.md for this repo. PHASE A * requirements.txt: dash-improve-my-llms>=2.0.0 -> [flask]>=2.3.3. * No OAI-SearchBot workaround existed here to delete (grepped run.py and lib/ per the handoff — clean). * warn_missing_llms_doc False -> True, and mark_hidden("/admin/control-board") before add_llms_routes. That page is the only registered route with no llms.txt prose, so it was the sole reason the warning could never reach zero; hiding it also keeps it out of the sitemap, matching the robots disallowed_paths already set. * block_ai_training stays False, now with a comment that says why. 2.3.3 makes True safe, but for MIT component docs being in the training corpus is how a model recommends the library to someone who never visits. Recorded as a deliberate divergence from the fleet fingerprint, not drift. Verified by running this app under the boilerplate venv's real 2.3.3 (the compat runner grew DL2_PIN_MODULES so a module other than dash can be pinned from the target while the donor supplies the rest). On 2.3.3: 72/72, no warnings, no dv-banner leak, no directive leak in a page llms.txt, sitemap 27 urls with the control board correctly absent. Left on the installed 2.0.0 it still passes 72/72 with exactly two warnings — the network directory degrading on <2.1, and the missing-prose warning firing for "/" via the assign-vs-merge bug 2.2.0 fixes. Both silent on 2.3.3, which is the evidence that flipping the warning on was right. PHASE B * lib/network_directory.py copied from the boilerplate; HUB_URL is already https://2plot.dev and peers_for() self-excludes by URL, so no self-entry edit was needed. * network_directory.apply(BASE_URL) in run.py before add_llms_routes. Result: "## Network" in /llms.txt with 6 peers and 6 <link rel="related"> in head, none self-referential. * satellite_reporter.py deliberately NOT adopted, contrary to the handoff. The boilerplate splits tracking/rollup/transport across three modules and that stack has no /api/pageview and no /api/satellite/auth. Ours carries both — the SPA page-view beacon (without which every session reports as single-page) and the sign-in attribution beacon verified end-to-end against the hub's own receiver. Swapping would be a regression, so it stays. UPSTREAM FINDING pannellum.2plot.dev and emojimart.2plot.dev are NXDOMAIN, confirmed by DNS rather than a failed request — email/flows also failed the first curl but that was a Render free-tier cold start and they serve on retry. They are commented out in our copy with a note, because publishing them breaks the module's own rule that a dead directory entry makes an agent distrust the whole list. The fix belongs in dash-documentation-boilerplate, which is the single definition every satellite copies. PHASE C Canonical already defaults to https://leaflet.2plot.dev (the pannellum failure mode does not apply here), custom domain and Clerk subdomain are already live. The hub-side SATELLITES flip lives in pip-docs+. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent e089cd6 commit 417535a

4 files changed

Lines changed: 294 additions & 4 deletions

File tree

lib/network_directory.py

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
"""Cross-host directory for the 2plot network — one definition, every satellite.
2+
3+
Why this file exists
4+
--------------------
5+
Search engines follow links between hosts weakly; agents don't follow them at
6+
all. A model answering "what does this ecosystem provide?" fetches one or two
7+
URLs and reasons from what came back. Landing on ``leaflet.2plot.dev`` it sees
8+
one library, with nothing in the markup saying the other eleven hosts exist.
9+
``sitemap.xml`` cannot fix that — a sitemap is scoped to its own origin by
10+
design — so ``dash-improve-my-llms`` 2.1 emits an explicit machine-readable
11+
directory instead: ``<link rel="related">`` tags in ``<head>``, a ``## Network``
12+
section in ``/llms.txt``, and followed links in the prerendered body.
13+
14+
Keep the definition **here**, in the template, and import it. Twelve
15+
hand-maintained copies of the same peer list will drift, and a directory that
16+
disagrees with itself across hosts is worse than no directory at all.
17+
18+
Three tiers, and the distinction is load-bearing:
19+
20+
``PEERS``
21+
Same network, same operator. These build the cross-host graph you own.
22+
``AFFILIATED``
23+
Yours, on unrelated domains. Findable when asked "what else did you
24+
build?" without being swept into "what is the 2plot network?".
25+
``EXTERNAL``
26+
Third-party docs you reference but don't own. Emitted ``rel="nofollow"``
27+
— references, not endorsements.
28+
29+
Usage in a satellite's ``run.py``, before ``add_llms_routes(app)``::
30+
31+
from lib.constants import BASE_URL
32+
from lib import network_directory
33+
34+
app._base_url = BASE_URL
35+
network_directory.apply(BASE_URL)
36+
"""
37+
38+
from __future__ import annotations
39+
40+
from typing import Any, Dict, List
41+
42+
# Only list hosts that are actually live. A directory entry pointing at a
43+
# subdomain with no site is a dead link an agent will follow once and then
44+
# distrust the rest of the list for. muicharts.2plot.dev and
45+
# flexlayout.2plot.dev have no docs site yet — add them in the same change
46+
# that ships them, not before.
47+
# UPSTREAM DIVERGENCE — the only edit this file carries, and it is temporary.
48+
#
49+
# pannellum.2plot.dev and emojimart.2plot.dev are commented out below because
50+
# they are NXDOMAIN as of 2026-07-31 (verified via DNS, not just a failed
51+
# request — email/flows returned 000 on a first curl too, but that was a Render
52+
# free-tier cold start and they serve fine on retry).
53+
#
54+
# The rule this enforces is the module's own, three paragraphs up: "a directory
55+
# entry pointing at a subdomain with no site is a dead link an agent will follow
56+
# once and then distrust the rest of the list for." Shipping them would publish
57+
# two dead links from a live docs site.
58+
#
59+
# The real fix belongs in dash-documentation-boilerplate, which is the single
60+
# definition every satellite copies — otherwise each repo rediscovers this
61+
# independently. Once those hosts resolve, delete this note and re-copy the file
62+
# from the boilerplate rather than un-commenting by hand.
63+
PEERS: List[Dict[str, str]] = [
64+
{
65+
"name": "2plot.ai",
66+
"url": "https://2plot.ai",
67+
"description": "Network hub and account origin.",
68+
},
69+
{
70+
"name": "2plot.dev",
71+
"url": "https://2plot.dev",
72+
"description": "Package index for every open-source component in the network.",
73+
},
74+
{
75+
"name": "Documentation boilerplate",
76+
"url": "https://boilerplate.2plot.dev",
77+
"description": "The markdown-driven documentation template every satellite site is built from.",
78+
},
79+
{
80+
"name": "dash-leaflet2",
81+
"url": "https://leaflet.2plot.dev",
82+
"description": "Leaflet 2 maps as Dash components.",
83+
},
84+
{
85+
"name": "dash-mui-scheduler",
86+
"url": "https://muischeduler.2plot.dev",
87+
"description": "MUI X Scheduler — calendars and event scheduling for Dash.",
88+
},
89+
{
90+
"name": "dash-flows",
91+
"url": "https://flows.2plot.dev",
92+
"description": "Node-graph editors built on React Flow.",
93+
},
94+
# {
95+
# "name": "dash-pannellum",
96+
# "url": "https://pannellum.2plot.dev",
97+
# "description": "360° panorama and virtual-tour viewer.",
98+
# },
99+
# {
100+
# "name": "dash-emoji-mart",
101+
# "url": "https://emojimart.2plot.dev",
102+
# "description": "Emoji picker component.",
103+
# },
104+
{
105+
"name": "dash-email",
106+
"url": "https://email.2plot.dev",
107+
"description": "Email composition and delivery components.",
108+
},
109+
]
110+
111+
AFFILIATED: List[Dict[str, str]] = [
112+
{
113+
"name": "Pip Install Python",
114+
"url": "https://pip-install-python.com",
115+
"description": "The original component documentation site.",
116+
},
117+
{
118+
"name": "Pirate's Bargain",
119+
"url": "https://piratesbargain.com",
120+
"description": "Deal aggregator built on the same Dash stack.",
121+
},
122+
{
123+
"name": "ai-agent.buzz",
124+
"url": "https://ai-agent.buzz",
125+
"description": "Agent tooling directory.",
126+
},
127+
]
128+
129+
EXTERNAL: List[Dict[str, Any]] = [
130+
{
131+
"name": "Dash Mantine Components",
132+
"url": "https://www.dash-mantine-components.com",
133+
"description": "The UI component layer these docs are built with.",
134+
"llms_txt": "https://www.dash-mantine-components.com/llms.txt",
135+
},
136+
{
137+
"name": "Plotly Dash documentation",
138+
"url": "https://dash.plotly.com",
139+
"description": "Upstream framework documentation.",
140+
},
141+
]
142+
143+
NETWORK_NAME = "The 2plot network"
144+
NETWORK_DESCRIPTION = (
145+
"Open-source Dash component libraries by Pip Install Python. Each component "
146+
"has its own documentation site and its own llms.txt; 2plot.dev indexes all "
147+
"of them, and 2plot.ai is the hub."
148+
)
149+
HUB_URL = "https://2plot.dev"
150+
151+
# The mark drawn in the header of the rendered llms.txt view: "2" + morse
152+
# encoding of "plot" + "ai", as columns of dots and dashes.
153+
#
154+
# No period glyph between the halves — the morse block already separates them,
155+
# and a literal "." next to it reads as punctuation dropped into a graphic.
156+
# The renderer turns a suffix ending in "i" into an upward flourish, so "ai"
157+
# draws as "a" plus that mark; `label` carries the real domain for screen
158+
# readers and the SVG <title>, which is the only place the dot belongs.
159+
#
160+
# Defined here rather than per-app because this module is copied verbatim into
161+
# every satellite — that is what keeps one mark across the network instead of
162+
# twelve slightly different ones.
163+
WORDMARK = {
164+
"morse": "plot",
165+
"prefix": "2",
166+
"suffix": "ai",
167+
"label": "2plot.ai",
168+
}
169+
170+
171+
def peers_for(app_url: str) -> List[Dict[str, str]]:
172+
"""`PEERS` with this app removed.
173+
174+
A site listing itself as its own peer reads as generated rather than
175+
curated, and it wastes a slot in a list an agent may only skim.
176+
"""
177+
own = app_url.rstrip("/")
178+
return [p for p in PEERS if p["url"].rstrip("/") != own]
179+
180+
181+
def apply(app_url: str) -> None:
182+
"""Publish the directory for the app served at ``app_url``.
183+
184+
Degrades rather than fails on older releases of the package. A satellite
185+
pinned behind this file should still boot: losing the directory, or losing
186+
the wordmark, is a degradation — refusing to start is not.
187+
188+
That matters during a staged rollout, when this module reaches satellites
189+
before the new package does. ``register_network`` arrived in 2.1 and its
190+
``wordmark`` argument in 2.2, and Python raises ``TypeError`` on an unknown
191+
keyword, so the argument is only passed when the installed signature
192+
actually accepts it.
193+
"""
194+
try:
195+
from dash_improve_my_llms import register_network
196+
except ImportError: # pragma: no cover - only on <2.1
197+
import warnings
198+
199+
warnings.warn(
200+
"dash-improve-my-llms is older than 2.1, so the cross-host network "
201+
"directory will not be published. Upgrade to publish it.",
202+
RuntimeWarning,
203+
stacklevel=2,
204+
)
205+
return
206+
207+
import inspect
208+
209+
extra: Dict[str, Any] = {}
210+
if "wordmark" in inspect.signature(register_network).parameters:
211+
extra["wordmark"] = WORDMARK
212+
213+
register_network(
214+
name=NETWORK_NAME,
215+
description=NETWORK_DESCRIPTION,
216+
hub_url=HUB_URL,
217+
peers=peers_for(app_url),
218+
affiliated=AFFILIATED,
219+
external=EXTERNAL,
220+
**extra,
221+
)

requirements.txt

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,11 @@ dash_mantine_components==2.7.0
3030
dash-iconify==0.1.2
3131
markdown2dash
3232
python-frontmatter
33-
dash-improve-my-llms>=2.0.0
33+
# 2.3.3, not 2.0: the upgrade alone fixes the assign-vs-merge home-page stub
34+
# (2.2.0 merge semantics), directive leakage into agent markdown (2.3.3), and
35+
# the OAI/Anthropic robots taxonomy (2.3.2/2.3.3). The [flask] extra is just
36+
# flask>=2.0, already present via Dash.
37+
dash-improve-my-llms[flask]>=2.3.3
3438
python-dotenv>=1.0
3539
pydantic>=2.0
3640
requests>=2.31

run.py

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,10 +45,11 @@
4545
LLMSConfig,
4646
RobotsConfig,
4747
add_llms_routes,
48+
mark_hidden,
4849
register_page_metadata,
4950
)
5051

51-
from lib import auth, satellite_analytics
52+
from lib import auth, network_directory, satellite_analytics
5253
from lib.backend import get_backend_info, resolve_backend
5354
from lib.constants import APP_VERSION, BASE_URL, LEAFLET_VERSION, SITE_TITLE
5455

@@ -140,6 +141,19 @@
140141
# ----------------------------------------------------------------------------
141142
app._base_url = BASE_URL
142143
app._robots_config = RobotsConfig(
144+
# DELIBERATE open posture, reviewed against dash-improve-my-llms 2.3.3.
145+
#
146+
# 2.3.3 makes `True` safe — it blocks the real training crawlers (GPTBot,
147+
# ClaudeBot, CCBot) while still allowing Claude-User, Claude-SearchBot and
148+
# ChatGPT-User, so the old reason to run `False` (blocking broke claude.ai
149+
# fetches through the legacy aliases) is gone. We stay `False` anyway,
150+
# because for MIT-licensed component documentation being in the training
151+
# corpus is the point: it is how a model recommends this library to someone
152+
# who never visits the site. Flip to `True` if that calculus changes.
153+
#
154+
# Note this diverges from the fleet fingerprint in
155+
# handoff/existing_subdomains.md, whose verification expects
156+
# `ClaudeBot -> Disallow`. The divergence is intentional, not drift.
143157
block_ai_training=False,
144158
allow_ai_search=True,
145159
allow_traditional=True,
@@ -165,13 +179,37 @@
165179
# are not supported" errors when the search dropdown re-rendered).
166180
# ----------------------------------------------------------------------------
167181

182+
# Cross-host network directory. A sitemap is scoped to its own origin by
183+
# design, so nothing in this site's markup would otherwise say the other 2plot
184+
# hosts exist — an agent landing here sees one library and no ecosystem. This
185+
# emits <link rel="related"> tags, a "## Network" section in /llms.txt, and
186+
# followed links in the prerendered body. Must run BEFORE add_llms_routes.
187+
#
188+
# The peer list is kept in one place (copied verbatim from the boilerplate) and
189+
# self-excludes by URL, so there is nothing app-specific to edit here — editing
190+
# it per repo is how twelve copies drift apart.
191+
network_directory.apply(BASE_URL)
192+
193+
# /admin/control-board is an admin surface, not documentation. It is the one
194+
# registered page with no llms.txt prose, so it must be marked hidden BEFORE
195+
# add_llms_routes — otherwise it is the sole reason the missing-prose warning
196+
# below can never reach zero. mark_hidden also keeps it out of the sitemap and
197+
# out of /llms.txt, which matches the robots `disallowed_paths` above.
198+
mark_hidden("/admin/control-board")
199+
168200
# Wire up /llms.txt, /<page>/llms.txt, /robots.txt, /sitemap.xml + bot
169-
# middleware. dash-improve-my-llms 2.0 auto-detects the active backend
201+
# middleware. dash-improve-my-llms auto-detects the active backend
170202
# (flask / fastapi / quart) and dispatches to the matching adapter, so we
171203
# no longer have to gate this on the backend. Per-page prose is registered
172204
# through lib.page_visibility.register_llms_doc (see pages/markdown.py) so the
173205
# control board's llms.txt switch can swap a page's body for a stub.
174-
add_llms_routes(app, LLMSConfig(warn_missing_llms_doc=False))
206+
#
207+
# warn_missing_llms_doc is deliberately TRUE. It was silenced while the home
208+
# page still lost its prose to the assign-semantics bug in 2.0; 2.2.0 made
209+
# register_page_metadata MERGE, so every page now keeps the llms_doc the
210+
# markdown loader gave it and the warning should stay at zero. If it starts
211+
# firing, a page has genuinely lost its prose — which is worth hearing about.
212+
add_llms_routes(app, LLMSConfig(warn_missing_llms_doc=True))
175213

176214
# ----------------------------------------------------------------------------
177215
# 2plot.ai satellite analytics: /healthz for the hub's hourly health sweep,

scripts/_compat_runner.py

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,10 @@
1515
1616
Env:
1717
DL2_EXTRA_SITE donor site-packages directory
18+
DL2_PIN_MODULES comma-separated modules to import BEFORE the donor path is
19+
appended, so the target interpreter's copy wins. `dash` is
20+
always pinned; add others when the target holds the version
21+
under test (e.g. dash_improve_my_llms).
1822
DL2_SMOKE_ARGS arguments to forward to smoke_test.py, newline-separated
1923
"""
2024
import os
@@ -30,13 +34,36 @@
3034
_resolved = dash.__version__
3135
_origin = Path(dash.__file__).resolve()
3236

37+
# Any other module whose TARGET version is the thing under test has to be
38+
# imported here too, for the same reason: once it is in sys.modules the donor
39+
# path appended below cannot shadow it.
40+
_pinned = []
41+
for _name in (os.environ.get("DL2_PIN_MODULES") or "").split(","):
42+
_name = _name.strip()
43+
if not _name:
44+
continue
45+
try:
46+
_mod = __import__(_name)
47+
_ver = getattr(_mod, "__version__", None)
48+
if _ver is None:
49+
import importlib.metadata as _md
50+
try:
51+
_ver = _md.version(_name.replace("_", "-"))
52+
except Exception:
53+
_ver = "?"
54+
_pinned.append(f"{_name}=={_ver}")
55+
except Exception as _exc:
56+
_pinned.append(f"{_name} FAILED ({_exc})")
57+
3358
# 2. Now lend the target the docs-site libraries it is missing.
3459
extra = os.environ.get("DL2_EXTRA_SITE")
3560
if extra and extra not in sys.path:
3661
sys.path.append(extra)
3762

3863
# Report what actually got used, so a silently-wrong measurement is visible.
3964
print(f"[runner] dash {_resolved} from {_origin.parent.parent.parent.parent}")
65+
if _pinned:
66+
print(f"[runner] pinned from target: {', '.join(_pinned)}")
4067
print(f"[runner] donor site-packages: {extra or '(none)'}")
4168

4269
sys.argv = ["smoke_test.py"] + [

0 commit comments

Comments
 (0)