Brand visibility: derive it from stored answers, and chart it - #1
Merged
Conversation
Add three tables and the code that fills them from the raw responses:
- `brands` holds the brands to look for. The user supplies the list. This
repo supplies no names and makes no decision about who competes.
- `result_sources` holds one row for each link an engine returned. The
`kind` column records where the link came from. An advertisement and a
citation stay apart, because they are not the same evidence.
- `result_brand_mentions` holds one row for each completed result and each
enabled brand. The rows for the brands that the answer did not name are
necessary. Share of voice needs a denominator. A table that holds only
the hits cannot show the difference between "never named" and "never
asked".
`results.extraction_revision` is the queue. A NULL value means that the
row is not yet derived.
The refresh runs in the scheduler tick, after the submissions and the
sweep. Neon's free tier has no pg_cron. Thus the database has no
scheduler, and the one Vercel cron job is the only clock this app has.
The refresh is the last step because it is the only step that you can stop
early without a loss. A batch limit and a time budget keep the tick inside
the function timeout. The remaining work stays in a column, and the next
tick continues it. The budget always lets one result through, or a slow
tick would derive nothing and the queue would stay the same size.
For each result, the refresh deletes the old rows and writes new rows in
one transaction. The output for a result is a full set. A source that goes
away between two revisions must also go away from the table.
The tests found two defects, which this change corrects:
- `new URL("javascript:void")` gives an empty hostname. Such links became
rows with an empty domain, and each GROUP BY showed them as a real site.
- The brand `acme` matched the text `acme-killer`. A hyphen is now a word
character. A dot stays a boundary, which lets the brand `acme` match the
`acme.io` that an engine wrote.
Claude-Session: https://claude.ai/code/session_01WPdhHDqVmqKDAf88dVu6b5
Give the brand list a write path. Before this change, the extractor read a table that nothing could fill. REST: GET, POST on /api/brands, and GET, PATCH, DELETE on /api/brands/:id. MCP: list_brands, track_brand, untrack_brand and get_brand_visibility. Each write path that changes what the extractor reads calls markAllForReextraction(). Thus a brand that you add today has a full history, and does not start at zero on the day of the change. `isOwn` is an exception, because it is a label that the extractor does not read. DELETE does not re-derive. The foreign key removes the mention rows of the brand, and no other brand uses them. The domain field takes a bare host, such as "acme.io". It does not take a URL. A URL is easier for one week, until a person enters "acme.io/blog" and the citations do not agree. The extractor compares a hostname. Correct `isUniqueViolation` to look at the `cause` chain. Drizzle puts the driver error in a DrizzleQueryError, thus code 23505 is not on the error that it throws. A test showed a 500 in the place of a 409. AGENTS.md: record the three rules that hold the derived tables together, and make the scope limit more exact. Brand extraction is near the "no analysis layer" limit. It stays on the safe side because it is mechanical. The user declares the brands. The code finds a literal name. It does not decide if an answer is good. Claude-Session: https://claude.ai/code/session_01WPdhHDqVmqKDAf88dVu6b5
Ten panels over `result_sources` and `result_brand_mentions`: where we stand, brand ranking, visibility with time, visibility per engine, results per prompt, and the pages that the engines cite. `scripts/seed.mjs` fills a local database with 30 days of synthetic answers, so that a person can build and examine the panels immediately. It writes prompts, brands and raw results only. The scheduler tick then derives the tables, through the code that operates in production. A seed script that wrote the derived tables would show only that the seed script is correct. The seed data has cases that make a defective panel visible: - One brand that the answers never name, but whose pages get citations. A panel that reads one boolean shows this brand incorrectly. - Named and cited that do not agree. If a named brand always had a link, two columns that read the same data would look correct. - Google answers with no AI Overview, which have no prose at all. Each panel was examined in a browser with the data, which found five defects that a query test cannot find: - The brand table lost its last column off the edge of the panel. - The "Ours" column showed the words "false" and "true". The name now carries the mark, as "Acme (us)". - The header "Answers citing it" became "Answers citin" at width 110. - One percent column had no gauge, but the column beside it had one. - The default thresholds made a gauge red at 80. Each percent here is better when it is larger, so a perfect result looked like an alarm. The gauges now use one flat step: the length shows the value, the colour shows nothing. Claude-Session: https://claude.ai/code/session_01WPdhHDqVmqKDAf88dVu6b5
Add seven panels and the two extraction inputs that they need. New panels: how to read this, search queries that the engines issued, named but not tracked, top YouTube videos, top Reddit posts, prompt redundancy, and data quality. The panel for each prompt also shows the position now: `first_position` was in the table from the start, and no panel read it. Two new derived tables: - `result_search_queries` holds what a model typed before it got any page. ChatGPT, Copilot, Grok and Perplexity report this. The others do not. The queries that a model issues stay separate from the follow-ups that it suggests, because they are different acts. One is the reasoning of the engine, the other is navigation furniture. - `result_candidate_mentions` holds names from `lib/brand-candidates.json` that an answer mentioned. The candidates file is a new kind of input: it is a FILE, thus no code can call `markAllForReextraction()` when it changes. `EXTRACTION_STAMP` puts the sorted list of names into the value that goes in `results.extraction_revision`. An edited file stops agreeing with the stored value, and the next tick derives the history again. That column now holds a fingerprint of the inputs, not a version number. The file ships empty. This repo does not decide who competes with you. It also cannot find a name that nobody wrote down: to do that you must find entities in prose, and this repo does not interpret an answer. It records if a name that you chose is present. Two defects that only the browser showed: - A text panel with a datasource and an empty query stops Grafana from making the layout. Each panel on the page then shows nothing. A text panel must have no targets. - At height 7 the text panel cut off its last paragraph, with no scroll bar to show that more text was there. Two seed defects made two panels prove nothing. Each pair of prompts scored 100% commonality, because all prompts took sources from one small pool. And there was one YouTube URL and one Reddit URL, thus those two panels always had one row. Each prompt now has its own window on a larger list of publishers. Claude-Session: https://claude.ai/code/session_01WPdhHDqVmqKDAf88dVu6b5
Put the panels in the order and the geometry of the GEO dashboard in
cloro-dev/infra, so that a person who knows one can read the other. Two
rows, the first one collapsed with the guidance in it. The `Search URL`
text box and the 30 minute refresh come across too.
Three panels of that dashboard have no equal here. Two use "motion", a
prompt-set idea that this repo does not have, and the third needs a list
of pages that you are already on, which there is no place to keep. Their
slots close up, thus the ORDER is the same and only the offsets differ.
Remove the four headline tiles. That dashboard has no such panel, and this
one must look the same.
Follow infra c3b63ae: the by-engine chart is now "Brand visibility by
engine". The old title said which filter the chart ignores, which the
description says already, and it did not fit. Also follow infra 8edc722:
the YouTube and Reddit panels now show the title of the page, linked to
its URL, with the retrievals, the engines and how often the same answer
named you.
The candidates file takes alternative names now:
{ "name": "Acme", "aliases": ["Acme, Inc", "Acme Corp"] }
A plain string still works when a name needs no alternatives. Each
alternative counts to the same row, because one company must not become
two lines. The file ships fictional names, and says to replace them.
This found a defect that is older than the candidates:
A name and its own alternative can hit the same words. "Vandelay
Industries" holds "Vandelay", thus one phrase counted twice. This applied
to the brands table from the first commit, and each mention_count from
before now can be too high. Matching is not able to overlap any more: a
longer term takes its words first, thus the exact spelling wins and the
short form counts only where it is alone. EXTRACTION_REVISION goes to 3 to
make every stored row again.
Claude-Session: https://claude.ai/code/session_01WPdhHDqVmqKDAf88dVu6b5
The tick asked for results whose stamp is not the current one. No index can hold a value that the query gives at run time, thus Postgres read every completed row on every tick, and it found nothing after the backlog was done. EXPLAIN showed a sequential scan. Add `extraction_state`, one row that holds the stamp the stored rows were built at. The tick compares it to the stamp of the code and, if they differ, clears `results.extraction_revision` in one statement. The queue test is then `IS NULL`, which an index can answer, and the partial index holds only the rows that wait. A deployment that has caught up has an EMPTY index. Measured on 856 results: sequential scan 4.0 ms, bitmap index scan 0.4 ms, and the difference grows with the table. `markAllForReextraction` no longer asks for every row id to count them, which is the one place that could hold a whole table in memory. It writes only the rows that still carry a stamp, because NULL over NULL is a write that makes a dead tuple for no gain. It reports the depth of the queue, not the number of rows written: a caller asks how many answers will be derived again, and a result that already waited counts the same as one just reopened. `RefreshSummary` reports `reopened` too, thus a person can see the one tick after a deploy that changes the rules. README: two facts that the reader needs before deploy. - The catch-up. The tick does 250 results and the Hobby plan runs one job each day, thus a year of answers needs months. Point another scheduler at /api/cron, or call it in a loop. The tick is idempotent. Nothing is absent while it runs: the old rows stay until each result is built again, thus the numbers are old, never blank. - The size. The derived tables add about 5 KB for each answer, thus the budget is ~25 KB for each engine for each run, not 20 KB. The table of workloads now counts them. Measured: 4472 kB of derived rows against 3184 kB of raw rows, on 856 answers. Claude-Session: https://claude.ai/code/session_01WPdhHDqVmqKDAf88dVu6b5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Make the raw answers answer the GEO question: which brands the engines
name, which pages they cite, and where you are not present.
What this adds
Four derived tables, filled by the scheduler tick from
results.response:result_sources— one row for each link, with akindthat says whereit came from. An advertisement and a citation stay apart.
result_brand_mentions— one row for each completed answer and eachenabled brand, and for the brands that the answer did not name.
Share of voice needs a denominator. A table of only the hits cannot show
the difference between "never named" and "never asked".
result_search_queries— what a model typed before it got any page.result_candidate_mentions— names fromlib/brand-candidates.jsonthat an answer mentioned but that you do not track.
A REST and MCP surface for
brands, andgrafana/geo-visibility.jsonwith 13 panels.
Why the app derives this, not the database
Neon's free tier has no
pg_cron, thus a materialised view has nothing torefresh it. The one Vercel cron job is the only clock this app has. The
refresh is the last step of the tick, because it is the only step that you
can stop early with no loss. A batch limit and a time budget keep the tick
inside the function timeout, and the work that is left stays in a column
for the next tick.
The dashboard
Panel order and geometry match the internal GEO dashboard, so that a
person who knows one can read the other. Three panels of that dashboard
have no equal here: two use a prompt-set idea that this repo does not
have, and the third needs a list of pages that you are already on.
Verification
Each of the 12 panel queries ran through the query API of Grafana, and
each panel was looked at in a browser against 30 days of seeded data. That
found seven defects that a query test cannot find, among them a column
that fell off the edge of a panel, a gauge that made a perfect score red,
and a text panel that stopped Grafana from making the layout at all.
scripts/seed.mjsmakes that data. It writes prompts, brands and rawresults only, then the tick derives the tables through the code that runs
in production.
Defect found on the way
A name and its own alternative can hit the same words: "Vandelay
Industries" holds "Vandelay", thus one phrase counted twice. This applied
to the brands table from the first commit. Matching cannot overlap any
more, and
EXTRACTION_REVISIONgoes to 3 to make each stored row again.Before you merge
The first tick after deploy derives every completed result again, 250 at a
time. A deployment with a large history needs several days of ticks to
finish. The old rows stay in place until each result is done, thus the
dashboard shows values that are old, never values that are absent.
https://claude.ai/code/session_01WPdhHDqVmqKDAf88dVu6b5