Context dump above 100k tokens

[bkutasi]

I've noticed the Perplexica lacks awareness of the endpoints capabilities when it comes to context size. This often leads to 100k+ prompts which obviously wastes compute.
Reproduction is easy, just try to summarise something like an article by just starting a chat like this:
Summary: https://www.wired.com/story/the-technologies-changing-how-youll-watch-the-2026-winter-olympic-games/
Response on openrouter from my docker logs:

unhandledRejection:  Error: 400 This endpoint's maximum context length is 256000 tokens. However, you requested about 355286 tokens (354870 of text input, 416 of tool input). Please reduce the length of either one, or use the "middle-out" transform to compress your prompt automatically.

For llama.cpp:

Error: 400 request (517154 tokens) exceeds the available context size (184832 tokens), try increasing it

Same on server side:

send_error: task id = 2514, error: request (539488 tokens) exceeds the available context size (65536 tokens), try increasing it

Locally you could use context-shift but thats generally against the notion that clients should manage the contexts of their requests.
Are there any plans for the future to implement context compaction and reduction?

[LeBohdan]

It seems Perplexica is demanding a ridiculous amount of context. In my tests, it's requesting over 600k tokens. But after I restarted the Perplexica Docker container, the exact same query worked fine without errors, with all the same settings.

[aniruddhaadak80]

This feels like a classic capability-boundary bug where the system is constructing prompts as if every endpoint can tolerate the same context size. Once that assumption breaks, the failure is not really about the article itself, but about the absence of a size-aware preparation step before the request is made.

A practical fix would be to push compaction or summarization earlier in the pipeline and make endpoint limits explicit in the request builder. That would turn an expensive hard failure into a predictable adaptation path.