We need to decide how / where sidebar context #59 will live in the agent
Right now it is sent as prepended to the user query. Pro is it it's simple + the LLM can already interpret it reasonably well, con is we can't do any validation, inflates token usage, might mess up context unless we disable changing context, can't reapply context across tools
The other big option is to send it as structured data, which breaks into two methods:
- Store in the system prompt. => like the current approach only automatically repeated across every message via natural lang
- or context_schema => tools can access programmatically, lower token count, but the LLM won't inherently reason on this context
Where they are the same is that the pros are basically the opposite of the current cons, the cons are this is probably more complex than is maybe needed right now.
I can see us sticking w/ the current approach, blending the 2 structured ones, or all 3
We need to decide how / where sidebar context #59 will live in the agent
Right now it is sent as prepended to the user query. Pro is it it's simple + the LLM can already interpret it reasonably well, con is we can't do any validation, inflates token usage, might mess up context unless we disable changing context, can't reapply context across tools
The other big option is to send it as structured data, which breaks into two methods:
Where they are the same is that the pros are basically the opposite of the current cons, the cons are this is probably more complex than is maybe needed right now.
I can see us sticking w/ the current approach, blending the 2 structured ones, or all 3