Feature: add auto_resubmit toggle to sambawiz bundle builder [DRAFT / on hold]#19
Draft
sethkneeland-sn wants to merge 1 commit into
Draft
Conversation
Contributor
Author
|
cc @snova-varunkrishna @ojha1008 — sambawiz maintainers, tagging for review. This adds the per-model |
Long generations that exceed the sequence length of the lowest-SS expert serving them stop at that boundary unless a larger-SS expert is wired via resubmit_to. fast-coe's BundleTemplate already supports a per-model `auto_resubmit` flag that auto-wires each expert's resubmit_to to the next-larger-SS expert; this surfaces it in sambawiz so authored bundles get it by default. - Per-model toggle in BundleForm (default ON), with a tooltip describing the latency/queue-load trade-off. - Serialize to BundleTemplate spec.models.<model>.auto_resubmit. Emitted for generative models when enabled; omitted for embedding models. Not gated on visible expert count, since DYT-style PEFs contain experts that do not surface in the PEF output, so a seemingly single-expert model may still have a ladder to climb (harmless no-op in fast-coe otherwise). - Parse auto_resubmit back out of loaded bundles (absent => off) so load/edit/save round-trips preserve intent; persist it in builder state. - Unit tests for the generator. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sethkneeland-sn
force-pushed
the
feature/auto-resubmit-toggle
branch
from
5396d06 to
6b292aa
Compare
sethkneeland-sn
changed the title
sethkneeland-sn
changed the title
Collaborator
|
@kaichan1201 , do we need to do anything for resubmit explicitly or is it handled automatically in the backend? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status: Draft / on hold
This PR adds a per-model
auto_resubmittoggle to the bundle builder.End-to-end testing showed that enabling
auto_resubmitalone does not resolve the output-length truncation it was meant to address for models whose experts are hosted by a single multi-expert PEF (the common case for current CoE models):Because of that, this change is on hold until the serving-side fixes land. When we revisit it, the toggle will likely be reworked — for example defaulting to off / opt-in with guardrails, and re-scoped as an authoring convenience rather than a fix for the truncation itself.
Keeping the branch open as a draft so the work isn't lost.
Original change (for reference)
auto_resubmittoggle in the bundle builder (currently default on — to be reconsidered).🤖 Generated with Claude Code