[staging CI] unslothai/unsloth#8943 - #328
Open
danielhanchen wants to merge 38 commits into
Open
Conversation
Picking the model that is already resident raised "Stop N running chat?" and reloaded it. selectModel decided whether a pick needed a load by comparing the picker row id against status.model_identifier, and only when the current checkpoint was an external provider. A cached row pinned to a snapshot dir loads by path while its row keeps the repo id, so the two sides hold different strings for one model and the resident model read as a different one. /load answers already_loaded ahead of its cancel hook, so nothing was ever going to be stopped. residentModelMatchesPick matches a pick against the resident model on either name each side holds (id / load path against active_model / model_identifier), normalized through normalizeModelIdentity so Windows paths compare by case and separator, and gated on the GGUF variant so a quant switch stays a real reload. The residency check now runs for any pick that does not force a reload, not only when the checkpoint is external. Verified end to end against a Studio instance with an MLX model loaded by snapshot path: re-picking it mid-generation raised the dialog before the change and adopts silently after, with the chat running to completion. Fixes #8893
residentModelMatchesPick hand-rolled the comparison that model-identity.ts already exports. It now uses residentModelIdMatches, modelIdsMatch and ggufVariantsMatch, the same pair hub-page.tsx compares a resident model with, which also collapses a snapshot path onto its repo id through publicModelId. A standalone .gguf is exempt from the variant comparison. The backend derives hf_variant from the filename for a direct path load and /status echoes it, while the picker row deliberately carries none (settingsGgufVariantForRow), so comparing the two left #8893 unfixed for On Device files: the variant alone answered "not resident" for the file that was loaded. The source-scan test now asserts the residency check is not gated on an external checkpoint again, which is the shape the bug had. Verified it fails against that shape and passes here.
Matching the pick against status.active_model treated any snapshot of a cached repo as the resident one. The public id is the same for every snapshot, while _repo_gguf_load_id repoints load_id at the newest by mtime, so a repo that advanced under a resident older snapshot read as resident and selectModel skipped a load that would have replaced stale weights. LlamaCppBackend.matches_load_source settles it on the raw identifiers, so residentModelMatchesPick now does the same: when status reports a model_identifier it has to equal what this pick would send as model_path, and the public-id alias is left for a native-lease load, which withholds the raw path and reports only its display label.
The adopt path cleared activeLoadId behind a condition that could never be false: both writers store the pin only when it differs from the id it was recorded against, so activeLoadId is either null or never equal to modelId. A pinned cached row therefore lost its snapshot pin the moment the pick was recognised as resident. Write the pin a completed load writes instead. The stale-pin case the old comment describes is still covered, since a pin taken for an earlier resident is overwritten either way.
… model Adopting on identity alone dropped a remembered config. The chat picker and Hub's Run button both pass one WITHOUT forceReload (chat-page.tsx stageOrLoad, hub-page.tsx handleRun), selection.config is only read inside performLoad, and the short-circuit returns before it. llama-server would have reloaded for any of those settings, so a remembered context length, KV dtype, drafter, slots, batch size, placement or extra arg silently did nothing, and the rollback to previousConfig made the panel agree with the server either way, so there was nothing on screen to say so. residentRuntimeMatchesConfig compares a pick's config against the invocation /status echoes, over the fields LlamaCppBackend._runtime_matches_intent reloads for plus the MLX pair _mlx_runtime_settings_match compares. A field the config leaves unset expresses no opinion, so a model the user never configured still adopts the resident copy and 8893 stays fixed; one the user did configure adopts it whenever the two already agree. Anything the status cannot report counts as a difference: answering "differs" costs one reload, which is what happened before any of this existed, while a wrong "matches" loses a setting invisibly. Three smaller ones on the same path: - a native pick no longer takes the short-circuit. It names its file by a bare label two different files can share, and the lease is written only by a completed load (activeNativePathToken, beside the pin), so adopting here would have kept a stale token. Every native call site forces a reload already; this pins the invariant where it can be seen. - the rollback uses restorePreviousConfig() rather than an inline apply, so it carries the loadedIsDiffusion flag the helper passes. Without it a resident image model restored its config through the wrong branch of applyPerModelConfigToRuntime. - the pin comment claimed applyActiveModelStatusToStore rewrites params.checkpoint. It writes residentCheckpoint; the caller sets the checkpoint one line earlier. The pin change itself stands. Tests: resident-config-match covers every compared field in matching, differing and not-reported form, plus the three states of llamaExtraArgs and the ordering rules (GPU ids are a set, argv is not). resident-model-match-matrix runs the identity comparator over Linux, macOS, Windows drive, UNC, WSL, relative, tilde, rooted, Unicode and spaced path shapes, against current, older and native-lease status shapes, and pins the /mnt/<letter> case fold as a known WSL-shaped limit.
residentModelMatchesPick settles residency on status.model_identifier, which names one revision. When the status carries no raw identifier it falls through to residentModelIdMatches, and that collapses a snapshot dir onto its repo id. Every snapshot of one repo publishes the same public id, so a pick pinned to a different revision read as already resident and the load was skipped, leaving the user on the weights that were there. This file's own docstring names that hazard for the arm above it; the arm below still had it. Two shapes reach it. A backend old enough to predate model_identifier, and a native-lease load, which withholds the raw path on every version by design. A literal match on the name the pick would load by still settles it, so an older backend that put the raw path in active_model keeps adopting. What is refused is only the collapse, and only for a pick naming a cache snapshot, which is where the id cannot distinguish revisions. A native lease, a standalone .gguf and an unpinned repo id are unaffected. Found by fuzzing the function against LlamaCppBackend.matches_load_source over the cache layouts of each supported host: across 355 pairs these rows were the only ones the frontend adopted and the server would have reloaded.
… fields residentRuntimeMatchesConfig reads the GPU fields, so the host it runs on is a real axis even though it compares no paths: CUDA and ROCm report a placement pool and an offload mode, a CPU-only host reports manual with zero layers, and an MLX server reports none of them and a KV width instead. Each accelerator is crossed with every setting a remembered config can pin, in all three states that matter: the resident load already runs it, it runs something else, and the status cannot report it at all. The last one is the direction that must never read as agreement. Also covers config blobs written by older Studio versions, which simply lack the keys a newer field uses, and asserts placement compares as a set rather than an order. The structural test is the one that lasts. It parses PerModelConfig and fails when a field is added without being either compared or listed as deliberately excluded, so a new setting cannot be silently dropped by an adopted pick.
residentRuntimeMatchesConfig treated speculativeType, gpuMemoryMode, gpuLayers and nCpuMoe as unpinned when the remembered config left them unset. applyPerModelConfigToRuntime resolves all four before the load runs (readPersistedSpeculativeType, readPersistedGpuMemoryMode, GPU_LAYERS_AUTO, 0), and selectModel sends the resolved value, so an unset field is not silence: a real /load would have carried the standing preference and could differ from the resident server. Compare the resolved value on both sides instead. The caller supplies the four defaults plus normalizeSpeculativeType, so the module stays a leaf the node suite can drive.
Same code, fewer lines of prose: the four 150 to 200 char single-line comments in selectModel are wrapped, and the docblocks that restated the same point twice say it once. 62 fewer comment lines across the seven files this PR touches. Verified by AST comparison rather than by eye: each file is parsed with the TypeScript compiler and reprinted with removeComments, and every one of the seven is identical to its previous revision. Suite still 3067 pass / 0 fail, tsc clean, and no new biome findings (the two in use-chat-model-runtime.ts are pre-existing at head).
residentRuntimeMatchesConfig treated an unset customContextLength, kvCacheDtype, mlxKvBits, specDraftNMax, nParallel, nBatch, nUbatch, chatTemplateOverride or selectedGpuIds as no opinion. It is one: applyModelLoadConfigToRuntime writes the config over the runtime store before selectModel runs (chat-page.tsx:3242, hub-page.tsx:1329) and resolves each of these with ?? null, so the snapshot performLoad takes reads null rather than inheriting the resident model's value. A real /load would send null and the backend would size the model from scratch, which is a different invocation from a server pinned to an explicit context, KV dtype, slot count, batch size, template or GPU. llamaExtraArgs stays conditionally pinned: it is the one field with no ?? null fallback, so undefined really does mean this copy never read it. The accelerator fixtures move with it. A default load requests no particular GPU and no MLX KV width, so those bases echo null; a pinned pool and a pinned width are swept as fields like everything else.
Two things the resident shortcut let slip. maxSeqLength is a client-side generation cap that no status field echoes, so applyActiveModelStatusToStore cannot correct it and the rollback that makes the panel agree with the resident server has the last word. That rollback carries the OUTGOING model's cap, so re-picking a resident model after an external provider silently changed truncation. Re-apply the pick's own cap after the rollback. _runtime_matches_intent deliberately answers False for a retryable drafter failure so the next identical load repairs it. Skipping the load suppressed that and left speculation off with nothing to retry it. residentSpeculativeNeedsRepair declines the shortcut for the reasons the backend actually retries, drafter_not_found and the two binary stand-downs, and only when the pick asks for a drafter at all. The Auto-mode policy downgrades are excluded on purpose: they never repair, so treating them as retryable would prompt to stop running chats on every re-pick, which is the bug this PR fixes.
…s for PR #8943 Two ways the resident shortcut could skip a load the backend was going to perform. performLoad sends reconcilePersistedGpuIds(selectedGpuIds, selectedGpuIndexKind), not the saved pick: a selection saved in another index namespace, or naming GPUs that are gone, becomes Automatic before /load. The residency comparison read the raw ids, so a saved physical [1] adopted a server pinned to Vulkan device 1, and an unreconcilable pick reloaded where the load would have agreed. The reconciler is passed in with the device cache warmed first, since load-on-selection can run before any GPU hook mounted and a cold cache passes the pick through unvalidated. adopt_load_intent_if_matched forces a reload when memory_state_satisfies_settings fails or the launched VRAM fraction differs from the active one. Both are server-wide, so a Model Memory or budget save between two picks of one model leaves the pick, its config and the status identical, and the setting never reached the child. The gate now reads both reload_required flags, and only an explicit true declines, so a failed read or a backend without the endpoints keeps the shortcut exactly as it was.
…y send for PR #8943 Five more ways the shortcut and the load path disagreed. With no saved config the gate returned true unconditionally. It is not a wildcard: applyModelLoadConfigToRuntime(null) resets the store to DEFAULT_PER_MODEL_CONFIG before selectModel runs (chat-page.tsx:3242, hub-page.tsx:1329) and performLoad reads the store for every field the config does not carry, so /load would request those defaults while this adopted whatever another tab or API client had left running. The gate now compares currentRuntimePerModelConfig(), which is what the load reads on both doors: defaults after a reset, and the resident values where nothing reset them. maxSeqLength was re-applied only when the pick named one. applyPerModelConfigToRuntime resolves an absent cap to defaultInferenceParams.maxSeqLength, so leaving it unset kept the OUTGOING model's truncation limit. customContextLength was compared raw against requested_context_length. resolveLoadMaxSeqLength answers 0 for a cross-model GGUF pick and the resident context for a re-pick, so null against either was a reload the backend would have deduplicated. The resolver is passed in, bound to the same store fields performLoad gives it. tensor_split was not compared at all. applyPerModelConfigToRuntime clears splitRatio and no config can carry it, so applying a remembered config asks for the default distribution while a resident manual load keeps its custom one, and status hydration then restored that split in the UI. A preserved Vulkan CPU fallback read as a placement disagreement. adopt_load_intent_if_matched runs _preserve_cpu_fallback_intent first, rewriting an eligible Auto request into the resident manual/zero-layer intent, so re-picking raised the prompt this PR exists to remove. The exemption mirrors _cpu_fallback_request_eligible and covers placement only: a request pinning its own GPUs, split, MoE layers or pass-through args is excluded, and any non-placement difference still reloads.
…cy for PR #8943 _resolve_parallel_slots fills an omitted n_parallel from the server-wide default and stores that resolved value as requested_parallel_slots, so the status never echoes null for it. Comparing the config's null against it rejected the shortcut for every pick that carries no slot count, which is the ordinary default Hub or Chat selection, and fell through to the stop-chats prompt for a load the backend would have answered already_loaded. The default comes from the llama-flags catalogue, which is session cached, and 0 there is its own unknown. Unknown compares as a reload, and so does a resident load pinned above the default, since defaultParallelSlots is the EFFECTIVE count and a build without --kv-unified serves one slot however many are configured. Both are the safe direction.
…#8943 Both are the shortcut declining a load that would answer already_loaded, which is the prompt this PR exists to remove. matches_gpu_ids accepts the request or the effective pool: a GGUF load may narrow the requested placement to the smallest fitting subset, so a later pick naming that subset round-trips without a reload. The comparison read only requested_gpu_ids, so [0] against a resident [0, 1] narrowed to [0] prompted for a load that dedupes. The effective pool is only consulted when the status actually echoes one, since an absent echo is no placement rather than Automatic, and reading it as Automatic would let an unpinned pick adopt every pinned server. spec_binary_fallback_can_retry needs a different llama-server installed before an identical /load repairs a binary stand-down. The status published the reason but nothing about that, so binary_no_mtp and binary_outdated were treated as unconditionally retryable and every re-pick of a stood-down model raised the stop-chats confirmation for a repair that could not happen. The status now reports spec_fallback_binary_changed, the cheap half of the predicate: answered only for those two reasons, since it is polled from first paint, and without the capability probe, which is a subprocess. Only an explicit false suppresses the reload, so a backend too old to report it keeps the coarser answer rather than swallow a repair an update was meant to enable, and an unreadable binary reads as unknown rather than as no.
for more information, see https://pre-commit.ci
… reason for PR #8943 _runtime_matches_intent rejects an identical load for a retryable DFlash sidecar fetch and for a capability probe that has started answering since a launch it degraded. Neither sets spec_fallback_reason, so a client reading only that reason adopted the degraded runtime, and since nothing else re-probes or re-fetches it stayed degraded for as long as the server ran. The status now reports both. The probe field is gated on _capability_probe_inconclusive before it probes at all, so a healthy runtime pays nothing on a route polled from first paint, and probe_server_capabilities caches on the binary's revision, which this route already relies on elsewhere. Neither is guessed at: an unreadable answer is None, and only an explicit true declines the shortcut. The mode gating follows the backend arm by arm. The DFlash arm applies to Auto and DFlash, since under Auto a failed fetch records nothing at all; the probe arm has no mode gate, so nor does this.
for more information, see https://pre-commit.ci
requested_context_length is set only by the llama.cpp path, so a safetensors or MLX status never carries it, while resolveLoadMaxSeqLength answers the generation length for a non-GGUF pick. Reading that absence as 0 rejected every re-pick of a non-GGUF resident, which is exactly the case where an external provider was selected and the local model stayed loaded. The check is now skipped when the status says the resident is not GGUF; every other field still answers for itself. _spec_fallback_binary_changed reported only the revision half of spec_binary_fallback_can_retry. For binary_no_mtp the predicate also asks whether the replacement advertises what the drafter kind needs, and a replacement that still lacks it never repairs: the live process keeps its launch revision, so the half answer prompted to stop running chats on every later re-pick and never stopped. It now reports the whole predicate. The probe caches on the binary's revision and only runs behind the reason gate, so a healthy runtime still pays nothing.
…isons for PR #8943 Two more places the shortcut declined a load the backend deduplicates. _runtime_matches_intent compares the offload knobs only under Manual, and the MoE count only with a non-negative layer pin beside it. The comparison here compared both unconditionally, so a config that kept its hidden nCpuMoe after the layer slider went back to Auto never matched a runtime llama.cpp had recorded as n_cpu_moe 0. Same guard now, arm for arm. The drafter_not_found arm reloads so the next Apply retries the fetch, but excludes the kinds whose absence is not transient: DFlash asks through _dflash_retry_needed instead, and an absent DSpark sidecar is the permanent state of every repo but one, so retrying it would relaunch an identical server forever. The kind was already published; spec_dspark_sidecar_absent is new, and only an explicit true excludes, so a backend too old to report it keeps the coarser answer. The accelerator sweep grows a live condition per field rather than asserting a comparison the backend does not make: the two offload knobs are swept under Manual, and the MoE count with a layer pin. Auto is covered as its own case, where neither is compared.
…ady accepts for PR #8943 _runtime_matches_intent rejects a draft-count difference only when intent.spec_draft_n_max is not None, so an unset limit asks for no change and /load answers already_loaded. Comparing null against the count the resident load was launched with prompted to stop running chats for a reload that could not happen, and would not have delivered the default if it had: that same already_loaded leaves the count alone. This one field is now conditionally pinned, unlike its neighbours, and the test that asserted the opposite records why. The architecture gate rewrites a tensor-parallel request to layer mode and records _arch_gate_dropped_tensor_parallel, and the backend accepts the same true request back against the runtime that rewriting produced. Status reported only the launched mode, so the comparison saw a disagreement on every re-pick of a model whose split was gated off. The drop is now published as tensor_parallel_dropped_by_arch_gate. It excuses a true request only: a pick asking for no split against a runtime that has one is still a reload, and an older backend that cannot report the drop keeps the coarser answer.
…#8943 paravirtual_normalized_request rewrites every GGUF request to manual, zero layers, no split and no MoE, and adopt_load_intent_if_matched applies it before deciding an identical load can be reused. So the ordinary Auto pick and the resident manual status are the same request there, and comparing them raw reloaded on every re-pick, which on such a host is every re-pick there is. The detector is published as gpu_placement_paravirtual. It is lru_cached, so it costs one probe per process and nothing after, and on a non-darwin host it answers without probing at all. When set, the placement fields and the tensor-parallel flag stop deciding; everything else still does. Only an explicit true exempts, and a detector that raises reads as unknown rather than as no, since claiming placement is comparable where it is not is the direction that adopts a runtime the user did not ask for. One narrower case is left as it is: the same rewrite strips offload flags out of extra_args, and llamaExtraArgs is still compared as written. That costs a reload for a pick carrying its own -ngl on a virtualised Mac, which is the safe direction.
for more information, see https://pre-commit.ci
…int for PR #8943 The shortcut grew awaits between the status read and the decision: the GPU device cache, the llama-flags catalogue and the two server-wide settings reads. Another tab swapping the resident model inside that window is invisible here, since subscribeModelLifecycle dispatches on its own window, so the opening status could be adopted after it stopped describing the server. That set the picker to this model while prompts went to the one now loaded. The verdict is now a named predicate applied twice: once to open the window and once to a status read immediately before adopting. A read that fails, or a verdict that no longer holds, falls out of the block to /load, where a real disagreement belongs. The status hydrated into the store is the second read, not the first. loadModelMemorySettings shared one request between concurrent callers, so a read already in flight from before a policy save answered this one too, and a reloadRequired false from that would suppress the load the save was made for. It takes force now, as the VRAM budget's reader already did, and the preflight uses it.
… PR #8943 The force option I added replaces an in-flight read, and that older request keeps running. It published its answer to every subscriber, which is the state its replacement was issued because of, in whichever order the two landed. Its finally also cleared inFlightModelMemory unconditionally, dropping the newer promise's sharing handle while that one was still in flight, so the next caller opened a third request rather than joining the second. A generation counter now settles both, as the VRAM budget's reader does. A displaced read still resolves for its own caller; it just stops publishing and stops owning the slot.
… fields for PR #8943 The diffusion runner receives no --parallel, no batch sizes and no pass-through args. _runtime_matches_intent guards all four on not self._is_diffusion, and _llama_runtime_fields nulls the ones the status publishes at all, so a config pinning any of them rejected a load the backend would have deduplicated and raised the prompt this PR removes. All four are marked chat-only and skipped when the status says the resident is diffusion. Everything else still decides, so a KV dtype or context difference on a diffusion runtime is a reload as before.
…r PR #8943 The diffusion runner drives one device, so matches_gpu_ids takes the lowest id of a requested pool and the status reports only that one. Comparing the configured set rejected a runtime the backend would have called identical, and raised the prompt this PR removes. The reconciled pick is reduced to its lowest id when the status says the resident is diffusion, and compared whole otherwise. Automatic is untouched, since there is nothing to reduce and it must still not adopt a pinned runtime.
…for PR #8943 Two branches of the backend that the comparison was ignoring. The non-GGUF branch of /load checks identity and _mlx_runtime_settings_match, which compares mlx_kv_bits_requested and chat_template_override and nothing else, then answers already_loaded. Every other field here is a llama.cpp flag it never reads, so a persisted Manual mode, tensor split, slot count or batch size raised the stop-chats prompt for a load that could not have changed anything. Only those two decide against a safetensors or MLX resident now. This generalises the context-length guard added earlier, which was the same bug seen through one field, so the inline check goes. The diffusion branch of _runtime_matches_intent replaces the placement comparison with one _diffusion_manual_ngl check against diffusion_requested_ngl. A config on Manual with Auto layers resolves to no explicit NGL, and an older shim that dropped a manual NGL leaves the status reporting Auto while the request still says Manual, so comparing the mode raw rejected a load that deduplicates. The four placement fields are replaced by that comparison on diffusion and unchanged off it. One test moves with this: a non-GGUF resident running a different cache_type_kv now adopts. That field is a llama.cpp flag, the non-GGUF branch never reads it, and asserting otherwise was stricter than the backend rather than safer.
… PR #8943 With no saved config the gate compared currentRuntimePerModelConfig(), which is the OUTGOING model's live settings. performLoad treats a different checkpoint or variant as a model switch and clears the per-model fields before sending, so on that door the request is the defaults. Both directions were wrong there: a resident already running the defaults raised the prompt, and a resident matching the outgoing settings was adopted for a load that would not have asked for them. The gate now mirrors resetsPerModelSettings and picks between the two. gpuMemoryMode is standing and kvCacheDtype and tensorParallel are not in the reset, so those three still come from the store, and splitRatio is null on that door because the reset clears it before the load reads it. Where nothing switches, the live runtime is still right, which is why neither constant works on its own.
…aring for PR #8943 Two arguments change what a request means rather than adding to it, and the route settles both before its already-loaded comparator runs, so comparing the raw config judged a request the server never received. resolve_tensor_parallel lets an explicit --split-mode last-win over the Tensor Parallelism toggle, so a config using the supported pass-through form never matched the runtime it asked for. Under manual, the route copies the last -ngl into request.gpu_layers and strips the offload family from the list, saying as much in its own comment about the comparator. The status therefore reports the layer count and a stripped list while the config still carries the raw form, which failed on both fields at once. Auto is untouched, since an inherited -ngl is respected there and reaches the child. The parsing mirrors llama_server_args.py rather than approximating it, including _flag_name's rule that shorts always start with a letter, without which -ngl -1, the commonest pass-through of all, reads as malformed. A malformed list answers null instead of throwing: this runs while deciding whether to skip a load, and a bad argument belongs to the load. One narrower normalization is left alone and noted here: the same block strips --tensor-split when _should_strip_tensor_split allows, and that is still compared as written. It costs a reload, which is the safe direction.
_reuse_loaded_gguf refuses its own already-loaded answer while _audio_probed is false, so load_model reaches its fast path and re-probes there. Nothing else re-probes, so a shortcut that skips /load leaves the model's audio capabilities undetected for as long as the server runs. That is a silent loss rather than one extra reload, which is why it is worth a status field. audio_probe_pending is reported with the same True default the route reads it with, so a backend that never tracked the probe is not mistaken for one with an outstanding probe, and only an explicit true declines the shortcut. The check sits inside the residency verdict, so the re-read before adopting judges it again.
…r PR #8943 The two failures are not symmetric, which is what my original wording missed. One extra reload costs the prompt this PR removes; adopting on a read that failed right after the user changed the Model Memory policy or the VRAM budget leaves the child on the old one with nothing on screen to say so. An absent route is still not a failed read, so a backend too old to serve either keeps the shortcut. Both fetchers now raise SettingsRouteAbsentError on a 404, and the budget reader gained a rethrow option for the one caller that has to tell them apart: it answers null for both, which every other caller is right to treat alike, since they paint rather than decide. The predicate takes three states now instead of a nullable one. Two source-shape tests move with it. The VRAM one pinned the catch as having no binding; the residency one anchored its slice on the first catch-with-binding in the file, which is above that branch now.
… #8943
The drafter_not_found arm of _runtime_matches_intent is guarded on
intent.gguf_path being None, and the route derives that field from the
identifier alone, so a directly loaded file dedupes rather than retrying
the fetch. Declining the shortcut for it reproduced the prompt this PR
removes.
The caller now tells the repair check whether the pick sends a path,
using the same endswith(".gguf") test the route uses rather than the
stricter isStandaloneGgufPath, which also demands a path-like shape. Only
this arm is excused: a binary stand-down, a DFlash retry and the
capability probe all repair whatever the pick names.
I recorded this one as remaining looseness when the rest of the arm was
mirrored, so this closes it.
… dropped for PR #8943 Two consequences of adopting where the backend would, without the side effects the backend performs while adopting. adopt_load_intent_if_matched records the incoming pool when it accepts a fitted subset (_record_matching_gpu_request), so the user's narrower pick becomes the request. Skipping /load skips that, and the hydration then put the status's wider pool back, restoring a GPU the user had removed. The pick's own reconciled selection is re-applied after the hydration, in the same place and for the same reason the sequence cap already is. diffusion_requested_ngl retains the request even when an older shim ignored the split, so once the installed shim gains --ngl support the same request has to go through and finally apply it. _runtime_matches_intent rejects it for exactly that window, and the comparison saw only the retained value. diffusion_split_supported is published for it, answered only for a diffusion runner, since off one there is no split to apply and False would claim the question was asked.
for more information, see https://pre-commit.ci
The tensor-parallel arch-gate excuse read the raw tensorParallel toggle while the backend's comparator reads resolve_tensor_parallel, so a config asking for a split through --split-mode tensor missed the excuse it exists for. parse_gpu_layers_override now reports absent, a value, and malformed as three states. The backend raises on a malformed override, so the load fails and says so; folding it into "no override" stripped the token, found the rest agreeable and adopted, losing the saved setting in silence.
The rollback before hydration restores the outgoing model's config, so the slot and batch controls in the store belong to the model the tab just left. The adopt path suppressed the model-change reseed, so a resident model running 4 slots could show the outgoing count, and the next Apply saved that over it. Every other load parameter at that call site already treats a changed checkpoint or variant as a model change, and the slot pair now does too.
danielhanchen
force-pushed
the
pr-8943-xplat-ci-ci
branch
from
August 16, 2026 15:16
5765b42 to
af124ec
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disposable CI run for unslothai/unsloth#8943. Do not merge; closed after CI.