Currently, when the chat context reaches its size limit, the oldest messages are discarded, resulting in the loss of contextual information. This issue proposes implementing a "memory" mechanism to mitigate this.
Specifically, when the context limit is reached, instead of simply removing the oldest messages, the bot should generate a concise abstract summarizing the key information from those messages. This abstract will then be stored and included in subsequent interactions, allowing the bot to maintain a high-level understanding of the conversation even as older details are removed from the immediate context.
The abstract should be designed to be brief and focused on the essential points, ensuring it doesn't significantly increase the context size while still preserving crucial context. This will improve the bot's ability to maintain coherent and contextually relevant conversations over longer interactions.
Currently, when the chat context reaches its size limit, the oldest messages are discarded, resulting in the loss of contextual information. This issue proposes implementing a "memory" mechanism to mitigate this.
Specifically, when the context limit is reached, instead of simply removing the oldest messages, the bot should generate a concise abstract summarizing the key information from those messages. This abstract will then be stored and included in subsequent interactions, allowing the bot to maintain a high-level understanding of the conversation even as older details are removed from the immediate context.
The abstract should be designed to be brief and focused on the essential points, ensuring it doesn't significantly increase the context size while still preserving crucial context. This will improve the bot's ability to maintain coherent and contextually relevant conversations over longer interactions.