memory 50mb per deployed bot

[seshubonam]

Hi, i tried to use this bot for langchain through flowise.

each bot deployed uses approx 50mb RAM.

The way we deployed is to have seperate bots for each user. like there are multiple agents you can create in flowise.. so each user is able to create a new bot for a new agent they add in flowise.

however, this is not at all scalable as 100bots would consume 5gb ram.

is there a way to lower the memory if we are just using !lc command for flowise.

or would a serverless deploy option be ideal in this use case

alternative:
we could.i.plement one bot for flowise and the user selects the agent with !agent command.. but that would have to be done for everyroom everytime the user is accessing flowise related bots i guess.

what do you suggest. thanks

[hibobmaster]

I may add a sqlite db to manage users/rooms in matrix, so one instance is enough.

scope	privileges
user/room	can use command

And langchain command can bind a specific api endpoint or inherit from global configuration.

Besides, add an !agent command to allow user choose langchain models.

[seshubonam]

yay looking forward. thanks

[hibobmaster]

I am refactoring the code and need some time.

[hibobmaster]

I will add some commands for langchain admin.
https://github.com/hibobmaster/matrix_chatgpt_bot/wiki/Langchain-(flowise)

"""
username: user_id or room_id
- user_id: @xxxxx:xxxxx.xxxxx
- room_id: !xxxxx:xxxxx.xxxxx

agent_name: the name of the agent
api_url: api_endpoint
api_key: api_key (Optional)
permission: integer (can: 1, cannot: 0)

{1} update api_url
{2} update api_key
{3} update permission
{4} update agent name

# add langchain endpoint
!lcadmin add {username} {agent_name} {api_url} {api_key *Optional} {permission}

# update api_url
!lcadmin update {1} {username} {agent} {api_url}
# update api_key
!lcadmin update {2} {username} {agent} {api_key}
# update permission
!lcadmin update {3} {username} {agent} {permission}
# update agent name
!lcadmin update {4} {username} {agent} {api_url}

# delete agent
!lcadmin delete {username} {agent}

# delete all agent
!lcadmin delete {username}

# list agent
!lcadmin list {username}

# list all agents
!lcadmin list
"""

[hibobmaster]

https://github.com/hibobmaster/matrix_chatgpt_bot/releases/tag/v1.3.0

[seshubonam]

yup thanks. commands are useful and thanks for refactor
. exploring it.