Context window is hardcoded to 2048 for every family — non-Latin scripts get 2–4× less context

[kfaracik]

Part of the multilingual audit. Umbrella: #221. Closely related to #259 — see Relationship to #259 below.

Problem

CONTEXT_WINDOW_TOKENS_BY_FAMILY (constants/context-window.ts:21-28) assigns 2048 tokens to every family — Qwen 3, Qwen 2.5, LLaMA 3.2, LFM 2.5, Bielik, Gemma 4 — although several of them support far more.

Meanwhile getPromptCharBudget correctly scales the character budget by measured token density (constants/context-window.ts:99-106), and that density varies enormously by script:

Script	tokens/char
CJK	1.0
Thai, Burmese, Khmer	0.6
Devanagari, Arabic, Hebrew	0.5
Greek, Cyrillic	0.5
accented Latin	0.5
plain Latin	0.25

The scaling is right. The consequence is not: under a fixed 2048-token ceiling, a Hindi user gets roughly half and a Chinese user roughly a quarter of the retrieved content an English user gets, for the same question on the same device.

This affects ~26% of installs (hi, ur, ar, fa, ru, zh, ja, th) and it applies to document RAG and web search alike.

Why this is probably the largest single lever for non-Latin scripts

It requires no change to retrieval, ranking, chunking or prompting. It is a constant, plus whatever the runtime actually permits, plus memory measurement.

What to do

1. Establish the real context limit per family from the model cards and from what executorch actually accepts, rather than assuming 2048 everywhere.
2. Raise the per-family values where supported, measuring peak RAM per step — this interacts with the embedding model's residency, which already topples ≥1B models on a loaded 6 GB device.
3. Consider a per-device ceiling on top of the per-family one, since the constraint is memory, not the model.

Relationship to #259

#259 proposes slimming the --- Source N: <name> --- block delimiter to recover prompt tokens, estimated at ~20-48 tokens across a typical 5-8 block prompt.

These are complementary, not duplicates: #259 reduces per-block overhead, this issue raises the ceiling. But the ordering matters — raising 2048 to 4096 recovers 2048 tokens, which dwarfs 48. Do this issue first, then re-evaluate whether #259 still earns its risk, given that #259 touches a delimiter grammar that citation attribution and truncation both parse.

Acceptance

• Per-family context windows reflect measured, not assumed, limits.
• Peak memory measured per family on a 4 GB and a 6 GB device with the embedder resident.
• A Hindi and a Chinese prompt carry a comparable amount of content to an English one, not merely a comparable token count.