Many Auto/* have the same candidate pool. #9202
Replies: 7 comments 2 replies
|
The 40 candidates: Model Provider Reach Breaker Cooldown Lockedstepfun/step-1v stepfun OK CLOSED - - |
|
If we could adjust the 12 factors of each auto/*, it might be helpful to resove it. |
|
Thanks for the data, @manchairwang -- I traced both observations against the auto-combo factory. Two separate effects, both confirmed. 1. Same pool across 2. The 4x duplicate entry of On the 12 factors per channel: the Auto-Combo weights are global today ( The duplicate-rows symptom is tracked in #9133 (open). If your goal is "one row per logical candidate" in the listing, that issue is the place to follow. The per-channel weight config is a separate ask -- I am opening a tracker for it right after this reply. Which would you like to follow? |
|
My understanding is auto/best-coding, auto/best-chat, auto/best-reasoning shoud be different, therefore I could use them in different condition. But now they all point to one model, and it's not best in my pool. I have to create my own combo myself to specify model. Actually I created 200+ combo today, which each for a different model. |
|
Hey @manchairwang -- quick follow-up on the two threads you opened: Pool duplication (4x rows for one logical entry): tracked in #9133 ("Only part of auto/* pool candidates were returned"). It already lists the per- Per- The |
|
The “12-factor per channel" is more important for me, because I could use it to adjust and affect each auto/* 's candidate pool. If it works well and each auto/* could have good candidates, there would be no need for "different candidate pool per channel" in my case. |
|
@manchairwang — makes sense. The 12-factor per-channel weight config (#9238) is the right lever for your use case. I will bump its priority; the candidate-pool dedupe (#9133) fixes the display artifact but does not change the actual routing weights. Follow #9238 for the per-channel knobs. |
|
Follow-up -- related to #9133 (only part of the auto/* pool candidates were returned), which is open and tracks pool-construction correctness. If you're still seeing heavy overlap between different auto/* channels on a current version, feel free to add your findings there. |
|
@manchairwang A correction: my last reply called #9133 open and said it tracked pool construction. It's closed, and the fix that closed it (#11994) only changed how the candidates inspector displays the pool. It didn't change how pools are built. What you reported is still true: many |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I pulled all auto/* candidate pools from omniroute 3.8.49 by the API exposed:
a few providers with many conenctions (4~6) occupied most positions in the pool, for most popular name. There auto/* all map into the same one single model in real world. Lots of other powerful models did not have a change to be picked.
The same issue happened on claude-opus and claude-sonnet. I thought that they were collection of opus or sonnet models, but they are not. They have the same candidate pools as most other popular auto/*.
Channel Count Reachable Unreachable
auto/best-coding 40 40 0
auto/best-reasoning 40 40 0
auto/best-fast 40 40 0
auto/best-vision 40 40 0
auto/best-chat 40 40 0
auto/best-coding-fast 40 40 0
auto/pro-coding 40 40 0
auto/pro-reasoning 40 40 0
auto/pro-vision 40 40 0
auto/pro-chat 40 40 0
auto/pro-fast 40 40 0
auto/coding 40 40 0
auto/fast 40 40 0
auto/chat 40 40 0
auto/cheap 40 40 0
auto/offline 40 40 0
auto/smart 40 40 0
auto/claude-opus 40 40 0
auto/claude-sonnet 40 40 0
auto/best-free 7 7 0
auto/best-chaos 9 9 0
auto/chaos 9 9 0
auto/coding:fast 40 40 0
auto/coding:cheap 40 40 0
auto/coding:free 7 7 0
auto/coding:pro 30 30 0
auto/coding:reliable 40 40 0
auto/reasoning 40 40 0
auto/reasoning:pro 30 30 0
auto/vision 0 0 0
auto/multimodal 0 0 0
auto/glm 4 4 0
auto/minimax 0 0 0
auto/mimo 0 0 0
auto/zai 3 3 0
auto/gemma 1 1 0
auto/llama 1 1 0
auto/gemini 1 1 0
All reactions