[IMPROVEMENT] Single source of truth for LLM context window sizes

[Vidit-Ostwal]

Feature Area

Core functionality

Is your feature request related to an existing bug? Please link it here.

Related: #7303 (o1 / o1-pro / o3 fall back to the 8k default because they are missing from some of the duplicated tables). That class of bug is what this issue is meant to stop.

Describe the solution you'd like

Context window sizes are hardcoded in six places. Adding or updating a model means editing llm.py and the native provider (and sometimes Azure or Bedrock too). The copies have already drifted.

Current maps

Location	Used by
lib/crewai/src/crewai/llm.py — LLM_CONTEXT_WINDOW_SIZES	LiteLLM fallback
lib/crewai/src/crewai/llms/providers/openai/completion.py	Native OpenAI
lib/crewai/src/crewai/llms/providers/azure/completion.py	Native Azure
lib/crewai/src/crewai/llms/providers/gemini/completion.py	Native Gemini
lib/crewai/src/crewai/llms/providers/anthropic/completion.py	Native Anthropic
lib/crewai/src/crewai/llms/providers/bedrock/completion.py	Native Bedrock

OpenAI / Azure / Gemini import LLM_CONTEXT_WINDOW_SIZES only to validate bounds, then ignore it and use a local dict.

Known drift (fix these as part of the merge)

• Native OpenAI has gpt-5 / gpt-5-mini / gpt-5-nano (1,047,576). LiteLLM gpt-5 falls through to 8192.
• Native Anthropic has 1M models (claude-sonnet-4-6, claude-opus-5, …) that llm.py does not list as bare Claude ids.
• Native Bedrock prefix anthropic.claude-sonnet-4 is 200k, so anthropic.claude-sonnet-4-6 (1M) gets 200k on the native path.
• OpenAI-compatible (deepseek/…, openrouter/…) inherits OpenAI’s local map, so LiteLLM-only keys like deepseek-chat become 8192.

Lookup rules also disagree: native providers first-match (keys inserted longest-first); llm.py last-match-wins.

Suggested implementation (good first issue)

One module owns lookup + constants. Provider files only declare their prefixes.

1. Add lib/crewai/src/crewai/llms/context_window.py

   • CONTEXT_WINDOW_USAGE_RATIO (0.85), DEFAULT_CONTEXT_WINDOW_SIZE (8192), MIN_CONTEXT, MAX_CONTEXT
   • resolve_context_window_size(model, sizes, *, default, extra_names=()) — longest prefix wins, then apply the 0.85 ratio; validate bounds once
   • Family maps (source of truth for sizes):
     • OPENAI_CONTEXT_WINDOWS — GPT / o-series (OpenAI + Azure)
     • AZURE_CONTEXT_WINDOWS — Azure-only extras (gpt-35-turbo, text-embedding)
     • ANTHROPIC_CONTEXT_WINDOWS — bare claude-* prefixes
     • GEMINI_CONTEXT_WINDOWS
     • BEDROCK_CONTEXT_WINDOWS — Titan, Nova, Llama, etc.
     • LITELLM_CONTEXT_WINDOWS — Groq / Mistral / Together / other LiteLLM-only ids currently in llm.py
   • Expand Bedrock Claude ids from ANTHROPIC_CONTEXT_WINDOWS (anthropic., us.anthropic., eu.anthropic., apac.anthropic., global.anthropic.) so a new Claude window is added once
   • LLM_CONTEXT_WINDOW_SIZES = merge of the above (LiteLLM + OpenAI-compatible)

2. Rewire providers — delete the local context_windows dicts. get_context_window_size becomes a one-liner against that provider’s map. Keep today’s defaults: OpenAI/Azure/Bedrock 8192, Anthropic 200k, Gemini 1M.

3. Re-export from lib/crewai/src/crewai/llm.py

   • LLM_CONTEXT_WINDOW_SIZES, CONTEXT_WINDOW_USAGE_RATIO, DEFAULT_CONTEXT_WINDOW_SIZE so existing tests (patch.dict("crewai.llm.LLM_CONTEXT_WINDOW_SIZES", …)) keep working.
   • LiteLLM get_context_window_size calls the shared resolver with the merged map and _context_window_model_name().

4. Do not change base_llm.DEFAULT_CONTEXT_WINDOW_SIZE = 4096 (custom LLM fallback). That is a different default.

After this, adding gpt-5.7 is one entry in OPENAI_CONTEXT_WINDOWS. Native OpenAI, Azure, LiteLLM, and OpenAI-compatible all pick it up. Adding claude-sonnet-4-7 is one Anthropic entry; Bedrock regional ids are generated.

Acceptance criteria

• One module owns raw window sizes and longest-prefix lookup. Provider completion files do not copy the full list.
• Adding a GPT / Claude / Gemini model is a single map entry (plus Bedrock prefix expansion for Claude).
• crewai.llm still exports LLM_CONTEXT_WINDOW_SIZES, CONTEXT_WINDOW_USAGE_RATIO, and DEFAULT_CONTEXT_WINDOW_SIZE.
• Existing tests stay green: lib/crewai/tests/test_llm.py, llms/openai/test_openai.py, llms/azure/test_azure.py, llms/anthropic/test_anthropic.py.
• New unit tests for resolve_context_window_size (longest prefix, bounds, default).
• Parity: same family id via native vs is_litellm=True returns the same window (gpt-5, claude-sonnet-4-6, gemini-2.0-flash).
• LLM(model="bedrock/anthropic.claude-sonnet-4-6").get_context_window_size() is int(1_000_000 * CONTEXT_WINDOW_USAGE_RATIO) (not 200k).
• LiteLLM gpt-5 uses the native 1,047,576 window, not 8192. gpt-5.4-mini stays 200k.

Describe alternatives you've considered

• Keep one giant dict in llm.py and have every provider call it. That is still one source of truth, but provider files would not own only the models they need, and Azure/Bedrock extras get mixed into the OpenAI/Anthropic lists.
• Fetch windows at runtime from LiteLLM’s model_prices_and_context_window.json. The CLI already references that URL; it is not used at runtime today and would add a network dependency. Out of scope.

Additional context

• Do not change frozen docs under docs/v*/.
• Follow AGENTS.md: keep the diff small; write tests for behavior (lookup + parity), not implementation details.

[Yao-Y-B]

Hi @Vidit-Ostwal , I’d like to work on this issue.

I’ve looked through the current context-window handling and understand the main problem as the duplicated model-to-context-window mappings across llm.py and the native providers, which can lead to inconsistent values and prefix-matching behavior.

I plan to:

• introduce a shared context-window module and resolver with longest-prefix matching;
• consolidate the provider/model mappings into a single source of truth;
• rewire the native providers and LiteLLM path while preserving the existing public exports and defaults;
• add unit tests for lookup, bounds/default handling, and native/LiteLLM parity.

I’ll keep the changes focused and follow the existing test structure. If there are any implementation preferences or constraints I should be aware of, please let me know.

[Vidit-Ostwal]

Sounds good, assigning it you @Yao-Y-B

[Vidit-Ostwal]

Hey @Yao-Y-B, it would be great if we can resolve this issue as well with this PR?
#7303
lmk if this makes sense

[Yao-Y-B]

Hey @Yao-Y-B, it would be great if we can resolve this issue as well with this PR? #7303 lmk if this makes sense

Hi @Vidit-Ostwal , following up here since this also relates to #7303:

I’ve completed an implementation for #7304 and verified it locally with the test suite. I also noticed that another contributor has opened #7329 for the same issue. From my review, there are still a couple of gaps compared with the requirements, including re-exporting the context-window mapping from crewai.llm for compatibility and preserving the Azure-specific mappings.

I’m the assignee for #7304 and have my implementation ready. I’m also prepared to include the #7303 changes in the same PR if that’s preferred.

Would you like me to open a PR with my implementation (including #7303)？Happy to follow whichever approach is easier for the maintainers.

[Vidit-Ostwal]

Yup, please open a single PR with all the changes.
Thanks

[Vidit-Ostwal]

If the changes are too big, make stacked PRs, leaving this to your discretion

[Yao-Y-B]

If the changes are too big, make stacked PRs, leaving this to your discretion

Hi, @Vidit-Ostwal
I noticed that in issue#7303，in the expected behaviors,context_window_usage_ratio is 0.75, should I keep it as same as 0.85 as this issue describes?

[Vidit-Ostwal]

Hey make a PR, will take all these changes in the review
You should keep it same

[Yao-Y-B]

Hey make a PR, will take all these changes in the review You should keep it same

@Vidit-Ostwal PR is up (#7340) — please review when you have time. It also covers #7303 as you asked. Thanks for your patience and time!