Models should be exposed to admins as TaskProcessing providers to make fine-granular model selection possible

[marcelklehr]

I think the way we currently expose models in integration_openai is wrong.

Status quo

We have one TaskProcessing provider per task type for integration open ai. For these providers we add an optional input shape of "model" that allows users to select the model the task should be run with. Additionally, we allow configuring different openAI-compatible service providers for categories of tasks (e.g. text processing, I believe).

My opinion on the status quo

1. Users should not be able to change the model in the assistant UI, because it's error-prone (shows models that may not even work for the modality of the task type) and because the admin may not even want to expose all models that the service provider offers.
2. The UI for configuring additional service providers is confusing (from a UX perspective), but still not fine-grained enough for some use cases and would get more confusing if we implemented model and/or service provider choice for the admin, per task type.

My preferred solution

I think we should take a look at how llm2 and stt_whisper2 expose models. There should be a one-to-one correspondence between task processing providers and the models that will execute the task. This will allow the admin to use the TaskProcessing provider selection UI to select the model they want per task type. The integration_openai settings would shrink to just entering the credentials of one or multiple service providers, whose models would be automatically exposed to the admin. The optional "model" input shape would be removed.

[marcelklehr]

[julien-nc]

I think we still need the "categories" (text/chat, stt, tts, image) otherwise we will expose useless providers with irrelevant models like "OpenAI-Whisper" for the "Summarize" task type.

So concretely, in the UI:

• Keep the categories (text/chat, stt, tts, image)
• For each category, allow to add multiple service providers
• The "create" form asks for a URL, an API key (or basic auth user/pwd), a name, gets the model list and allows to select a subset of the model list

On the backend side, we might wanna create our own table for the service providers. Columns would be name, url, apiKey/passwd, user, model_ids.
The model_ids would be a coma-separated list of model IDs.

So each service provider would trigger the generation/registration of one taskprocessing provider per task type (of its category).
For example, a "text" service provider with the name "service1" and with modelA and modelB would produce those taskprocessing providers:

• "service1-modelA" for the "core:text2text" task type
• "service1-modelB" for the "core:text2text" task type
• "service1-modelA" for the "core:text2text:summary" task type
• "service1-modelB" for the "core:text2text:summary" task type
• etc...for all the task types that produce text (or are related with llms)

The meta providers can stay the same, like the text2sticker one from the Assistant (that schedules a text2image sub task):
The model selection would be done at the text2image level (the selected provider for this task type).

Sounds good to me.
Can you point to an example where anonymous taskprocessing provider classes are declared (I guess in AppAPI)?