docs: add transcript-knowledge-capture skill and per-topic notes#782
docs: add transcript-knowledge-capture skill and per-topic notes#782Shivapriya92 wants to merge 3 commits into
Conversation
… notes Captures per-topic knowledge (AI Workforce, CRM, Growth Engine, Vibe, etc.) distilled from SME/partner call transcripts, feeding future learning-path authoring per ET-689. Raw transcripts are gitignored and stay local. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…tch) Extracted SME/partner-call knowledge from the second batch of 36 transcripts (the June/July VMKA activity-feed dump, triaged down from 44 by removing 7 internal-only meetings and 1 duplicate). One of the 36 (onboarding-call-reschedule-external-client.txt) had no extractable substance and was skipped, matching the skill's precedent for the first pass's fragment call. 42 new dated entries appended across 8 topic files: ai-workforce, crm, vendasta-services, advertising, reputation-management, social-marketing, local-seo-listings, websites. _index.md counts and rollups updated to reflect both passes. Nothing overwritten -- append-only per skill guardrails.
CalCooper
left a comment
There was a problem hiding this comment.
This is awesome work, Shiva. I love the idea of a two-part skill here — capture faithfully in one skill, validate at authoring time in the other. That separation is exactly the right architecture, and the output backs it up: 83 transcripts across two passes, clean [CALL] provenance on every entry, and PII kept out of git via the .gitignore change. This is already a great content machine. A few minor things to consider before we merge, none of it rework, all additive on top of what you have built.
They cluster around one theme: the notes currently treat every claim at the same level of trust. Provenance tells us who spoke, but not what kind of claim it is or how far to trust it. A few examples from ai-workforce.md:
- Product truth from an engineer ("the only type of AI employee that there is, is Conversations now") sits at the same level as a CS rep's roadmap guess ("a year or two at best" on platform unification) and an explicit "working theory / escalation" on the chatbot legal-name bug.
- One partner's negotiated terms ("$35/month per brand... net-45... ACH") read like general pricing when they are a single deal.
- A demo value ("$40 basic grooming") is a price from a fake rabbit-grooming test account — it must never surface anywhere as a real number.
crm.mdstill says "Find Accounts," which is now "Find Leads." That is correct capture behavior (faithful to the transcript), but it proves the notes carry field terminology that has to be corrected downstream — the notes are raw ore, not ratified metal.
The downstream skill corrects [PRODUCT] claims against the docs, but only for claims the docs cover. Out-of-docs field claims have no ratification path except a human checking by hand — which is exactly the fragile spot we just hit on #728. Two small, additive changes close most of that gap. Concrete drop-in wording below.
1. Tag every claim by type and source authority — add to the end of Step 3
#### Tag every claim by type and by source authority
Provenance says who spoke; these tags say what kind of claim it is and how far to trust it. Append an inline tag to any claim that is not a plain, corroborated product fact:
- `[PRODUCT]` — how the platform actually behaves or what it can do. These are the claims `learning-path-writing` must check against `docusaurus/docs/` before use; a `[PRODUCT]` claim not found in the docs needs live verification before it reaches a lesson.
- `[COMMERCIAL]` — a price, fee, tier, or billing term stated as general. Use `[COMMERCIAL — partner-specific]` for anything that applies to one partner's negotiated deal (net terms, per-brand rate, one-off onboarding fee).
- `[FIELD]` — a sales or usage technique, a recommendation, or an opinion. True as "what this person does or believes," not as product truth.
- `[SPECULATION]` — a roadmap guess, an ETA, a "working theory," or anything the speaker themselves framed as unconfirmed. Carry the hedge verbatim.
- `[DEMO]` — a value read off a demo, test, or fake account (a sample price, a placeholder business). Never a real figure.
Judge a claim by who is making it (authority ladder, highest first):
1. **Vendasta SME / engineer / product specialist** — authoritative on `[PRODUCT]`.
2. **Vendasta CS / CSM / account manager / sales engineer** — reliable on workflows; treat their `[COMMERCIAL]` and `[SPECULATION]` as approximate.
3. **Partner / agency** — authoritative on their own `[FIELD]` experience and their own `[COMMERCIAL — partner-specific]` terms; not authoritative on `[PRODUCT]`.
4. **Prospect** — lowest; reactions and opinions, not facts.
When a claim's reliability turns on who said it (a `[PRODUCT]` assertion from a partner rather than an SME, say), name the speaker's role right on the claim. Default: a corroborated product fact from a Vendasta SME can stay untagged; tag the exceptions. When in doubt, tag.2. Check against what is already filed (stop and flag) — new step inserted before "File it" (renumber the rest +1)
### Step 4: Check against what is already filed (stop and flag)
Before appending, read the existing topic file and compare the new claims to what is there. Two cases are a hard stop — surface them to the user, do not resolve them silently:
- **Contradiction.** A new claim conflicts with a filed one on the same topic (a price, capability, or step disagrees). Do not overwrite, do not quietly keep both. Stop and report: "`<file>` entry from <date/source> says X; this call says Y — needs a human to reconcile." Leave the prior entry intact and hold the new one until told how to file it.
- **Substantial duplication.** A new claim substantially repeats one already filed from a different call. Stop and ask whether to (a) file it as **corroboration** — a short line under the existing claim noting a second source confirms it, with its `[CALL]` tag, which is how a field claim earns trust across independent calls — or (b) skip it as redundant. Do not paste the full duplicate as a new standalone block.
Everything genuinely new gets filed normally in the next step. Only conflicts and duplicates stop the flow.(Report back — now Step 7 — gains one line: "Name any contradiction or duplicate you flagged for the user to resolve.")
3. Updated entry template — replaces the code block in "File it"
## <Date> — <source name/role> (<call type>)
**Source:** [CALL] <one-line description, e.g. "Alistair George (Vendasta SME), technical setup walkthrough, screen recording">
1. <Claim in your own words.> [PRODUCT]
2. <A price that applies only to this partner's deal.> [COMMERCIAL — partner-specific]
3. <A rep's recommended technique.> [FIELD]
4. <A roadmap guess the speaker hedged.> [SPECULATION]
- Corroborated by [CALL] <other source/date>. ← added when a later call confirms an existing claim (Step 4)4. Add to Guardrails
- **Tag claims, do not flatten them.** Every claim is weighed by type and by who said it (Step 3's tags and authority ladder). A partner asserting product behavior is not an SME stating it, and a demo price is not a real price — mark the difference so the downstream author is not guessing.
- **Never let a partner-specific or demo value read as general truth.** Negotiated terms carry `[COMMERCIAL — partner-specific]`; sample/test figures carry `[DEMO]`. Neither may reach a lesson as standard pricing or real data.
- **Contradictions and duplicates are a hard stop for a human (Step 4).** When a new claim conflicts with or substantially repeats a filed one, surface it and wait — never overwrite, never silently keep both, never paste a full duplicate.5. Synthesis as a flagged fast-follow (not built now) — add under the manifest step
**Maintenance note — when a topic file gets large.** Once a topic passes roughly 15–20 entries, the raw timeline gets expensive to re-read and hard to read for consensus. At that point flag the file to the user as a candidate for a synthesis pass: distill the corroborated, stable workflow into one consensus section, keep contested and single-source claims called out, and archive the raw per-call entries below (or into a dated archive) with provenance intact. This is a separate maintenance operation, not part of routine capture — flag it, do not do it inline.One judgment call for you
In the corroboration line (Step 4 / the template) I put the "second source confirms this" note under the existing claim rather than as a brand-new entry. That is deliberate — it keeps a hot topic file from ballooning with near-duplicate retellings, which is what makes every future read cheaper. The trade-off is the file stops being a pure append-only timeline. If you would rather keep strict append-only for audit cleanliness, corroboration becomes a full new entry and we lose the size win. I lean toward the drafted version, but it is your call.
On sample size and cost, since I know it will come up
There is no magic number of transcripts. For field technique, trust a claim once it is saturated — the same thing said consistently across ~3+ independent reps (which the corroboration line above makes visible). For product behavior, count is irrelevant: one SME statement still gets checked against the docs. Token-wise the skill is fine at current scale — the transcript reads dominate, the file re-reads are second-order until a topic gets large, which is what the synthesis note is there to catch.
None of this blocks the concept, which is excellent. Sending back just so the tagging and the stop-and-flag land before the notes start feeding real lessons. Great foundation.
|
One more small addition to the tagging convention, Shiva. The authority ladder above tells the downstream author how much to trust a claim by who said it. But there's a specific case it doesn't yet cover: a Concrete example from the AI Workforce lessons (PR #796): the "Level 3 Autonomous is the default" claim isn't in Was this you manually checking? In these instances I think a human should be confirming. What do you think? Proposed addition to the tag set (Step 3): - `[VERIFIED IN-PRODUCT — <verifier>, <date>, <evidence>]` — a claim the docs do not cover, checked live in the product by a named person. Use this instead of `[PRODUCT]` when there is no doc to check against. Two rules make it trustworthy:
- **The verifier signs it first-person, where a human is on record** — a line in the commit message or a PR comment in their own words ("I checked Level 3 is the default in-product on July 21 — screenshot attached"). An AI may draft the surrounding notes, but it must never author the attestation on the human's behalf. A verification the model claims on someone's behalf is not a verification.
- **Name the evidence** — screenshot, screen recording, or at minimum a dated "checked live." The stronger the claim (anything safety- or money-relevant, like an autonomy default), the more it earns a captured artifact rather than a memory.
- Any `[VERIFIED IN-PRODUCT]` claim also carries a standing flag to get it into the docs, so it can graduate to a real `[DOC]` citation and stop depending on one person's check.Why it's worth the extra line: the whole point of the tagging work is that a downstream author (or Cal, or an SME) can trust a claim without re-doing the legwork. A tag that says "a human verified this" is only as good as whether a human is actually on record — otherwise it's the same single-source risk we've been designing against, just wearing a confident label. This closes that last gap: the human attests, the model doesn't attest for them. Genuinely, the decision-log and |
…IN-PRODUCT tag Per CalCooper's PR #782 review: - Step 3 gains claim tagging ([PRODUCT], [COMMERCIAL], [COMMERCIAL - partner-specific], [FIELD], [SPECULATION], [DEMO]) plus a source-authority ladder (SME > CS > partner > prospect). - New Step 4: stop and flag contradictions and substantial duplicates against already-filed notes, rather than silently overwriting or duplicating. - Step 4 also folds in a rule Shivapriya asked for directly: any [PRODUCT] claim that disagrees with (or is missing from) the published Partner Center docs under docusaurus/docs/ is a hard stop too -- never silently trust the transcript or the docs, always ask the human doing the extraction to verify. - New [VERIFIED IN-PRODUCT -- <verifier>, <date>, <evidence>] tag per Cal's follow-up comment: a claim the docs do not cover, checked live by a named human who signs the attestation first-person -- the model may never author the attestation on the human's behalf. - Updated entry template, guardrail additions, and a synthesis fast-follow note for topic files once they pass ~15-20 entries. - Renumbered steps 4-6 to 5-7 accordingly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Addressed in the latest commit — all four items from your review:
Entry template, guardrails, and the synthesis fast-follow note are all updated per your drop-in text. Steps renumbered 4→7 accordingly. Separately (not in this PR): the Level 3 Autonomous claim from #796 now has a real signed attestation with screenshots — see the Verification log in |
Summary
transcript-knowledge-captureskill for distilling SME/partner call transcripts into per-topic knowledge noteslearn-revamp/transcript-notes/(AI Workforce, CRM, Growth Engine, Vibe, Websites, etc.) plus an_index.mdlearning-path-writingskill for future learning-path authoring (ET-689).gitignoreso raw call transcripts inlearn-revamp/transcripts/stay local; onlyREADME.mdthere is trackedTest plan
.claude/skills/transcript-knowledge-capture/SKILL.mdrenders/loads correctly as a skillai-workforce.md,crm.md) for accuracy against source transcripts.txttranscripts are not present in the PR diff (gitignored)🤖 Generated with Claude Code