Skip to content

fix: Do not autofill default_model_filename for Python-runtime PyTorch models - #518

Open
mattwittwer wants to merge 3 commits into
mainfrom
mwittwer/fix_python_runtime_autofill
Open

fix: Do not autofill default_model_filename for Python-runtime PyTorch models#518
mattwittwer wants to merge 3 commits into
mainfrom
mwittwer/fix_python_runtime_autofill

Conversation

@mattwittwer

@mattwittwer mattwittwer commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

What does the PR do?

Since #484, AutoCompleteBackendFields autofills default_model_filename: "model.pt" for every backend: "pytorch" model with an empty platform — including models using the Python-based runtime (runtime: "model.py"). For those models a sibling model.pt is the weights companion to the model.py class, and the runtime gives default_model_filename top priority, so it calls torch.jit.load() on the weights file and fails at load with PytorchStreamReader failed locating file constants.pkl.

Restores the pre-#484 guard — skip filename autofill when runtime is "model.py" — and extends it to the AOTI platform branch (model.pt2), which the Python runtime cannot load either. Output is unchanged for every other config. #484's stated goal (correct .pt2 selection for torch_aoti) is preserved and now pinned by tests.

Adds AutoCompleteBackendFieldsTest to repo_agent_test (which already compiles model_config_utils.cc): the two Python-runtime regression cases plus six cases pinning existing autofill and version-dir sniffing behavior. The regression cases fail without the fix and pass with it.

Checklist

  • PR title reflects the change and is of format <commit_type>: <Title>
  • Changes are described in the pull request.
  • Related issues are referenced.
  • Populated github labels field
  • Added test plan and verified test passes.
  • Verified that the PR passes existing CI.
  • Verified copyright is correct on all changed files.
  • Added succinct git squash message before merging ref.
  • All template sections are filled out.
  • Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

  • build
  • ci
  • docs
  • feat
  • fix
  • perf
  • refactor
  • revert
  • style
  • test

Related PRs:

triton-inference-server/server#8913

Where should the reviewer start?

Test plan:

Test case added here

  • CI Pipeline ID:

60326198

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown

Greptile Summary

The PR prevents Python-runtime PyTorch models from receiving an incompatible default model filename.

  • Skips model.pt and model.pt2 filename autofill when the configured runtime is model.py.
  • Preserves existing LibTorch and AOTI autofill behavior for default-runtime models.
  • Adds regression coverage for Python runtimes, explicit filenames, and model-layout sniffing.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
src/model_config_utils.cc Adds a focused runtime guard to PyTorch default-model-filename autofill while preserving platform selection.
src/test/repo_agent_test.cc Adds comprehensive unit coverage for the corrected Python-runtime behavior and existing autofill paths.

Reviews (3): Last reviewed commit: "update use_autofill_filename name" | Re-trigger Greptile

@mattwittwer mattwittwer changed the title draft: autofill model name fix fix: Do not autofill default_model_filename for Python-runtime PyTorch models Aug 3, 2026
@mattwittwer
mattwittwer requested review from Vinya567 and yinggeh August 3, 2026 23:05
Comment thread src/model_config_utils.cc Outdated
@mattwittwer
mattwittwer requested a review from yinggeh August 5, 2026 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR: fix A bug fix

Development

Successfully merging this pull request may close these issues.

2 participants