[staging CI] unslothai/unsloth#8721 - #311
Conversation
Muse Glimmer needs a transformers release no installed version ships. Chat already handles this: /validate reports requires_transformers_upgrade, and the composer offers to install the latest sidecar before loading. Training never asked. Starting a Muse Glimmer run went straight to the loader and died there with an architecture the installed transformers does not know. Training now runs the same consent step chat does, before the custom code gate, because installing a newer transformers changes what the load would run. Two things the Configure preview got wrong and now discloses: - A model no installed transformers ships reads as ready to train. It is not; the run stops on a consent dialog first. - The latest sidecar loads 16-bit only, so a run that installs it cannot honour a 4-bit request. A "QLoRA, 4-bit" preview understated VRAM by roughly threefold. The preview now says 4-bit is unavailable for that model. /validate grows forces_16bit so the frontend does not have to re-derive the sidecar's rule, and resume runs go through the same gate as fresh ones.
for more information, see https://pre-commit.ci
Three things the gate got wrong. A resume can be attested against a 4-bit model load that the latest sidecar permanently refuses: effective_training_load_in_4bit raises ExactResumeResourcesUnavailable for it the moment latest_tier_active_for turns true, and that sidecar is a persistent overlay. Accepting Install from the new resume gate therefore stranded a checkpoint that would otherwise have resumed. The stored config cannot say so on its own (_sanitize_db_config strips require_exact_resume_resources and require_exact_model_resource before the row is written), so the resume names its run on /transformers-upgrade-check and the backend recomputes the requirement from the provenance marker. With a custom-code fallback the resume loads on the current runtime in the 4-bit mode it needs, so no install is offered; without one the install is the only way the run starts at all, and the dialog is raised as before. The check was handed the Hub identifier for a cached model while the remote-code gate and the worker both open the pinned snapshot, so a repo whose current config.json names a different architecture than the snapshot on disk was inspected instead of the model being loaded. The check now takes the same four cache-pin fields as /models/remote-code-scan and resolves them by that route's precedence, and each start path resolves the pin once for both gates. A merely offered upgrade no longer claims 16-bit when the model can load through its own repo code: the dialog offers that way out, taking it installs nothing, and the run loads bnb 4-bit. /validate already exempts this case.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d6932a5af3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| model_name, | ||
| get_base_model_from_lora_identifier, | ||
| model_name, |
There was a problem hiding this comment.
When a cached LoRA snapshot is selected and its adapter_config.json differs from the repository's current version, load_target correctly points at the pinned snapshot but the base-model lookup still uses model_name. The training worker resolves the base from load_target (core/training/worker.py:717), so this preflight can inspect the wrong base or miss it entirely, fail to offer a required Transformers upgrade, and then start a worker that dies during model loading. Pass load_target to get_base_model_from_lora_identifier and its offline guard.
Useful? React with 👍 / 👎.
| useEffect(() => { | ||
| if (!(key && selectedModel) || cache.has(key)) { | ||
| return; |
There was a problem hiding this comment.
After any successful latest-Transformers installation, the server's routing state changes without changing this cache key, and cache.has(key) prevents the model from ever being checked again during the session. The preview can therefore continue claiming an installation is pending; for a model with a custom-code fallback that was cached as forces16Bit: false, it can also keep advertising 4-bit even though choosing the install now routes the run to the 16-bit sidecar. Clear or update these cached checks whenever an installation succeeds, including installations initiated from another surface.
Useful? React with 👍 / 👎.
1 participant
Disposable CI run for unslothai/unsloth#8721. Do not merge; closed after CI.