Skip to content

Intelligence Layer

Jump to bottom
Ramiro Cantu edited this page May 28, 2026 · 1 revision

Intelligence Layer

Design doc. Describes Gradient's "intelligence layer" — the chain that turns raw study material (PDF pages, captured questions) into calibrated, outline-grounded tags and grounded atomic facts. This is the load-bearing half of the PKM loop (SPEC §G).

SPEC.md governs; this page is the longer-form narrative the spec invariants (V-L*, V-E*, V-KB*, V69) compress. Where a claim has an invariant, it's cited.

Lineage: LLM4Tag

The tagging engine is a single-user reimplementation of LLM4Tag (Tang et al., LLM4Tag: Automatic Tagging System for Information Retrieval via Large Language Models, KDD '25, arXiv:2502.13481). The paper's thesis drives the whole design:

An embedder alone is a weak tagger; an embedder as recall plus an LLM as judge is state-of-the-art.

In the paper's Table 1, pure embedding models (BGE/GTE/CONAN, picking tags by cosine) are the bottom rows; LLM4Tag beats them by 25–67%. So the embedder is demoted to retrieval and the LLM makes the actual decision. Gradient inherits this division of labour and the paper's three-module shape: recall → knowledge-enhanced generation → confidence calibration.

The pipeline

Two upstream jobs manufacture content; three downstream stages tag it. All stages run as scheduled, async jobs (app/scheduler.py) — never on a request path — which is why they default to the Flex processing tier (V-L5).

PDF inbox  ──poll_inbox──►  ingest_pdf                         (content creation)
                              │  render page → image  (PyMuPDF)
                              │  vision transcribe    → gpt-5.4-mini   (V-KB3)
                              │  extract atomic facts → gpt-5.4-nano   (V-KB4, structured)
                              ▼
                           atomic_facts (node_id = NULL)

run_embed_job  ────────────► embed_pending                     (the ENCODER step)
                              │  outline_node → embed >>-path   (B, V-L6)
                              │  atomic_fact / question → embed text
                              ▼
                           content_embeddings (text-embedding-3-small, dim 1536, V-E1)

run_grounded_tag_job  ─────► tag_pending → _tag_one                (LLM4Tag proper)
                              │  1. recall   → retrieve_candidates   (V-L3 / V-L6)
                              │  2. generate → gpt-5.4-nano picks from candidates (V44/V45)
                              │  3. calibrate→ gpt-5.4-nano (reasoning off) Yes/No logprobs (V69)
                              ▼
                           atomic_fact_tags / question_tags + denormalized node_id (V-T2/V-T3)

(downstream) Notion write-out: one page per node, facts as blocks (V-N1/V-N2)

Key seam: PDF ingest is not part of LLM4Tag — it produces the content vertices the paper assumes already exist. Content quality therefore propagates down the entire chain: a less-lossy fact → a better fact embedding → better recall → a better tag.

Component roles

Component File Role
PDF ingest app/services/kb/pdf_ingest.py render → vision-transcribe → extract atomic facts. Creates content. (V-KB3/V-KB4)
Inbox poller app/services/kb/inbox.py <slug>/*.pdf → course → ingest_pdf, V41 per-file isolation
Embedder app/services/kb/embeddings.py + embed_pending the LLM4Tag encoder: vectorize nodes + content (V-E1)
Similarity edges app/services/kb/similarity.py derive concept_edges.kind='similarity' (node↔node cosine, V-E2)
Recall app/services/kb/recall.py build the constrained candidate set — C2T + C2C2T + T2T (V-L3/V-L6)
Generation app/services/llm/grounded.py LLM picks from candidates, strict structured output (V44/V45)
Calibration app/services/llm/calibrator.py Yes/No logprob grade per tag → confidence (V69)
Persist app/services/kb/persist_tags.py write canonical tags, denormalize primary node_id (V-T2/V-T3)

LLM4Tag → Gradient mapping

LLM4Tag (paper) Gradient
Content vertex c atomic_fact / question
Tag vertex t / tag repository outline_node / outline_nodes tree
Deterministic edge c–t tag rows source ∈ {manual, schema_map} → node_id (V-T2)
Similarity edge (cosine, BGE) concept_edges.kind='similarity' (V-E2)
Encoder (BGE), Eq 2 text-embedding-3-small → content_embeddings
C2T meta-path _embedding_candidates (fact → outline-node cosine)
C2C2T meta-path _content_candidates (fact → similar tagged fact → its tag) — V-L6
Candidate tag set Φ(c) RecallResult.candidates, capped fan-out
Knowledge-enhanced generation generate_grounded_tags (nano picks from Φ)
SRKI (retrieved ICL) few-shot exemplars from prior calibrated tags + C2C2T neighbours
LSKI (SFT fine-tune) not implemented — see Divergences
Confidence calibration (Eq 10) calibrate_tag — V69, the same formula
Drop tags < 0.5 V-T3: < 0.5 ⇒ manual_review (surfaced, not dropped)

Recall layer (the part that bounds quality)

The candidate set is the recall ceiling: a node never recalled can never be tagged (the paper's HR#k metric, Table 2). Everything downstream — model tier, calibration — can only reject a bad candidate, never add a missing one. So recall completeness is the highest-leverage knob. retrieve_candidates merges three meta-paths, deduped by node (V-L3, fixes in V-L6):

  1. C2T — cosine of the target's vector against outline-node vectors in the course. Floored at min_score (δ, provisional 0.25) so a "least-bad" non-match isn't handed up; capped at top_k.
  2. C2C2T (_content_candidates) — the paper's hard-case rescuer. Cosine the target against other already-tagged atomic_facts (content↔content, first hop), then borrow each neighbour's tag (second hop). This reaches nodes whose label doesn't lexically match the content but whose meaning does — the case C2T fails. The second hop is trust-weighted:
    • gold (source ∈ {manual, schema_map}) → weight 1.0, via='content-gold'.
    • silver (source='llm' ∧ ¬manual_review ∧ confidence ≥ δ_silver) → silver_factor (0.6), via='content-silver'. The discount + the manual_review exclusion damp the echo / feedback loop (yesterday's model guess propagating to similar new content). The calibrator (V69) remains a hard backstop — C2C2T only proposes.
  3. T2T — concept_edges.kind='similarity' neighbours of the C2T hits (tag → similar tag). Capped at edge_top_n (5).

Both fan-out caps (C2C2T ≤ 5, T2T ≤ 5) mirror the paper's meta-path caps so the candidate list can't balloon and dilute the LLM's pick.

Why node embeddings use the full path (B): a node is embedded as its ancestor-qualified >> path (Metabolism >> Glycolysis >> Regulation), not the bare leaf token. The cosine matches a fact's prose against the tag's meaning, and the path carries far more of it than one word. Changing this embed input bumped embedding_version to -v2 (V-E1: full re-embed).

What the embedder does — and does not

The embedder turns each fact and each node into a 1536-dim vector. Cosine over those vectors does exactly two jobs: (a) C2T fact→node candidate recall, and (b) the C2C2T content↔content first hop. It does not assign tags — that is the LLM's job, with the calibrator scoring. The embedding choice matters not for its per-token price but because it sets the recall ceiling everything else is bounded by. Improving the embed input (B) typically beats upgrading the model tier.

Calibration (V69)

After generation, each picked (content, tag) pair is re-graded by a discrete Yes/No discriminator on a plain completion (not structured — the single token must be readable), and confidence is read from token logprobs:

Conf = exp(L_yes) / (exp(L_yes) + exp(L_no))

< 0.5 ⇒ manual_review = true (V-T3) — surfaced for the single user to curate, not silently dropped. The denormalized atomic_facts.node_id is set only from the highest-confidence non-review pick; a low-confidence fact stays NULL and is re-attempted next run.

The calibrator model must expose logprobs and run non-reasoning (V69). 5.4 models run with reasoning disabled satisfy this, so the calibrator is gpt-5.4-nano (reasoning off) — same model as generation, different flag (V-L5).

Model selection (V-L5)

Per-task, pinned by capability and cost; all jobs async ⇒ Flex tier (~50% off Standard).

Task Model Why
Vision PDF transcription gpt-5.4-mini nano vision is weaker than even the retired gpt-5-mini; a bad transcription poisons every downstream fact/tag/embedding
Classification / extraction / grounded-tag / ranking gpt-5.4-nano beats gpt-5-mini on text reasoning at ~4× lower cost; OpenAI's recommended tier for exactly these tasks
Calibration (logprobs Yes/No) gpt-5.4-nano, reasoning OFF exposes logprobs when non-reasoning (V69)
Embeddings text-embedding-3-small (1536) no 5.4-class embedding model; recall-ceiling input, not a price decision

Prices (Standard /1M tok): mini $0.75 / $4.50, nano $0.20 / $1.25, both 400k context. Flex ≈ half. Any rotation re-triggers the V-L2 measurement harness.

Divergences from the paper (and why they're sound)

  • No LSKI / fine-tuning. The paper SFTs a 7B model biweekly on 10k labels, and its ablation says dropping LSKI hurts more than dropping SRKI. Gradient gives up the bigger lever — justified because (a) a single user has no 10k-label corpus and no fine-tune ops, and (b) the tag repository is one outline tree (hundreds–thousands of nodes), not the paper's millions, so recall is tractable without learned recall. The non-parametric half (SRKI) is kept.
  • manual_review instead of drop. A single user can curate; the paper's hundreds-of-millions-of-users scale cannot, so it drops.
  • Calibrator not fine-tuned. Raw logprobs from the chosen model; fine at single-user scale.

Known limits / roadmap

  • pgvector ANN. All cosine is Python-side over JSONB vectors today. Fine at one-outline scale; C2C2T's content↔content scan is the first thing that will force the swap to pgvector vector(N) + <=> operators (similarity.py note, V-E1).
  • C2C2T is corpus-dependent. It contributes ~nothing on a cold KB and strengthens as content tags up — expect no day-one lift.
  • Edges aren't versioned. concept_edges similarity rows carry no embedding_version; after a re-embed (e.g. the -v2 bump) stale edges persist. Re-derive after a bump; consider stamping edges.
  • Vision/extraction fusion (planned). A single mini vision call emitting {transcription, facts} (image-grounded, less lossy) with prior-page context for cross-page facts — raises content quality and thus the recall ceiling. Amends V-KB3/V-KB4 when landed.
  • Question tagging is single-course only. Atomic facts carry course_id; questions are tagged only when exactly one course exists (recall needs a scope).

gradient-server

Reference

Features

Design Notes

Clone this wiki locally