How an ad carries attribution, and how it is read back - #247
Merged
Conversation
The write side asks one question of a destination -- does its ref carry the stratum inline or an opaque token -- and the read side asks one question of an extraction conf: is the value read the answer, or a token identifying the ad. The two are independent, and configured separately. Specifies both sides, the dashboard controls for each, the ad attributions surface, and the removals the tree needs to match it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Location says where to read; mapping says what the value read means. The two were tangled together: isAdTableLookup required the metadata location, getRetrieveFunc errored on a lookup anywhere else, and tokenLookupKey read one conf's key as the whole source's declaration of where the token is. Composing them instead — locationReader for where, resolveThroughAdTable wrapping whichever reader that names — is what lets a study attribute respondents who arrive by different routes. One recruited by a fly destination brings the token back in event metadata; one recruited by a web or app destination lands on the researcher's own page and brings it back as a Typeform or Qualtrics field. Each conf declares where its own token is, and two lookup confs under one source need not agree. adAttributionOutcome now reports one thing: a token that resolves to no row. vlab minted an ad and lost what it meant, so every respondent it recruits is dropped from stratum counts. It walks the source's lookup confs, asks each through its own reader, and returns on the first miss, so one event yields at most one outcome. An event carrying no token produces nothing — that is an expected arrival, not a failure — which retires the organic counter. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A ref either carries the stratum inline or carries a token that resolves to it.
`ref_mode` is that field and the only one: `include_metadata_in_ref` said the
same thing in a form that could not express "encoded", so the two needed a
validator to stop them contradicting each other, and every consumer had to know
which one to read. The mode "shortcode" goes with it — it named a ref that
routes and nothing else, whose join key came from Meta's referral webhook, which
reaches ~31% of Messenger ad entrants.
Optional is what keeps the migration free: a conf that states no mode resolves
to exactly the behaviour it has today, and no stored JSON is rewritten.
Every destination type is a RefModeDestination now, web and app included. What a
ref carries is a property of the ref rather than of the channel carrying it, so
`ad_ref_token` and the half-migration guard ask `resolved_ref_mode` of any
destination with no type check, and `messenger_ref` becomes `dotted_ref` —
serving Messenger's two carriers, multi's Messenger arm, and the `{ref}` a web
or app destination interpolates into its template. A destination with a
shortcode routes through fly, whose decoder recovers both from one string, so
its encoded ref is `r.<payload>`; one without has nothing to decode its ref and
swoosh compares the extracted value to `ref_token` verbatim, so its encoded ref
is the bare token.
The read side loses its two constraints on where a token may be read from:
`ExtractionConf` validates only that the mapping is known, and
`disagreeing_token_keys` — which existed because swoosh read one conf's key as
the whole source's declaration — has nothing left to warn about.
`csv_export` renders the mapping two ways from one definition, so the table the
dashboard shows and the file downloaded seconds later cannot disagree about the
columns, which are a union across rows in first-seen order.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two questions about a recruitment ad's ref, and they are independent. What it carries is `ref_mode`, on a destination; what the value read means is `mapping`, on an extraction conf. The forms now say so. `RefModeField` is one control rendered by all five destination forms rather than a copy in each, so a multi-channel study attributes exactly one way, and it is labelled by consequence: what a researcher decides is where their stratum data ends up and what the key is, so the words `ref_mode` and `encoded` never reach the screen. Changing a saved destination's mode warns, because the ref is part of the creative and reconciliation compares creatives — a change rewrites every ad in the study on the next run. An absent `ref_mode` is a real state: the conf predates the field. Three things hold it — `displayedRefMode` reports and never writes back, the default lives only in the two empty-state constructors, and the forms spread `...data` — and Destination.test.tsx pins the scenario they exist for: open a destination with no mode, edit its welcome message, save, and the field is still absent. The fly and Qualtrics extraction forms become one. Their locations were already identical, and what stayed per-source was the mapping — on the reasoning that only fly carries a token. It does not hold: a respondent recruited by a web or app destination lands on the researcher's own page and brings the token back as a Typeform or Qualtrics field. All that a source still decides is which response values its payload offers. Changing the location leaves the mapping alone, since location says nothing about what was read. A fly source with nothing saved starts with one lookup conf per variable declared in Variables. The researcher already named those, and the name is what the ad's frozen row is keyed by — asking again in a different vocabulary is what produces a silent half-config. A default, not a merge. The Ad Attributions step renders the mapping and downloads it as CSV from the rows it is showing, so the page and a file saved from it cannot disagree. `applyChange` derives `aggregate` from the conf's own location rather than pinning it to "first", which a `variable` lookup made visible: pinned, editing any other field demoted a survey answer back to the first value it ever saw. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`documentation/ad-attributions.md` describes the mechanism across components; `adopt/README.md`, `dashboard/README.md` and `inference/README.md` describe their own halves. Each now describes the system as it stands: one field saying what a ref carries, a mapping that composes with either location, and the one outcome swoosh reports. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
✅ Deploy Preview for vlab-dashboard ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
nandanrao
added a commit
that referenced
this pull request
Aug 27, 2026
Preserved during worktree cleanup. This is the specification PR #247 was built from; the implementation on this branch is its sibling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds
planning/ad-attribution-ref-mode-plan.md. Two questions about arecruitment ad's ref, and they are independent:
ref_mode, on a destinationmapping, on an extraction confNeither side validates, gates or reads the other. The one place they meet is a
warning:
thins_its_ref_without_reading_the_mapping.swoosh — a lookup composes with either location
locationReadersays where to read;resolveThroughAdTablewraps whicheverreader that names. So a lookup works on either location, which is what the whole
design buys: a respondent recruited by a fly destination brings the token back
in event metadata, one recruited by a web or app destination lands on the
researcher's own page and brings it back as a Typeform or Qualtrics field. Each
conf declares where its own token is, and two lookup confs under one source need
not agree.
adAttributionOutcomenow reports one thing at severityerror: a token thatresolves to no row. An event carrying no token produces nothing — that is an
expected arrival, not a failure — which retires the organic counter and
tokenLookupKeywith it.adopt — one field says what a ref carries
RefMode = Literal["metadata", "encoded"], andRefModeDestinationcarriesexactly one field.
include_metadata_in_refsaid the same thing in a form thatcould not express
"encoded", so the two needed a validator to stop themcontradicting each other; the
"shortcode"mode goes too, since its join keycame from Meta's referral webhook, which reaches ~31% of Messenger ad entrants.
Optionalkeeps the migration free: a conf that states no mode resolves to thebehaviour it has today, and no stored JSON is rewritten.
Every destination type is a
RefModeDestinationnow, web and app included, soad_ref_tokenand the half-migration guard askresolved_ref_modewith no typecheck.
messenger_refbecomesdotted_ref, serving Messenger's two carriers,multi's Messenger arm and the
{ref}a web or app destination interpolates.A destination with a shortcode encodes to
r.<payload>; one without has nothingto decode its ref and swoosh compares the extracted value to
ref_tokenverbatim, so it encodes to the bare token.
csv_exportrenders the mapping two ways from one definition, and the table isserved as JSON alongside the CSV.
dashboard
RefModeFieldis one control rendered by all five destination forms, labelledby consequence — the words
ref_modeandencodednever reach the screen.Changing a saved destination's mode warns, because it rewrites every ad in the
study on the next run.
An absent
ref_modeis a real state and stays absent:displayedRefModereports and never writes back, the default lives only in the two empty-state
constructors, and the forms spread
...data.Destination.test.tsxpins thescenario — open a destination with no mode, edit its welcome message, save, and
the field is still absent.
The fly and Qualtrics extraction forms become one; all a source still decides is
which response values its payload offers. A fly source with nothing saved starts
with one lookup conf per variable declared in Variables. A new Ad
Attributions step renders the mapping and downloads it as CSV.
Two things found along the way
applyChangepinnedaggregateto"first", so editing any field on avariableconf demoted it back to the first value it ever saw. Latent before;a
variablelookup makes it visible. Fixed, with a test.changes.
include_metadata_in_refdefaultedFalseon those two types, so anexisting conf resolved to a thin
form.<shortcode>ref; under the plan'ssingle field it resolves to
"metadata"and carries the stratum inline. Thatis what §2 specifies, and
check_whatsapp_refs_are_deliverablefails closedon values fly's entry pattern cannot parse — but it means an existing CTWA
study's ads get rewritten, and one with unsafe stratum values now fails config
validation instead of publishing a thin ref. Messenger, Web and App are
unaffected. Say the word and I will make those two types resolve to
"encoded"absent a stated mode instead.Verification
Documentation is its own commit:
documentation/ad-attributions.mdfor themechanism, and each app's README for its own half.
🤖 Generated with Claude Code