What happened?
Summary
entrypoint.py:34 declares one modality set for every model:
SUPPORTED_MODALITIES = frozenset({"text", "image", "audio", "video", "action", "scalar", "tensor"})
used at entrypoint.py:273 to validate input_modalities on intake. It has
no per-model dimension, so audio validates for BAGEL even though BAGEL has
no audio encoder — the request is accepted and only fails (or misbehaves)
downstream, instead of being rejected at intake with a clear message.
Current state
- One frozenset, checked against every model's
input_modalities regardless
of what that model actually implements.
- Each model already knows its own supported modalities implicitly (e.g. via
which load_* methods and prefill walks it defines), but nothing exposes
that to the entrypoint.
Proposed fix
Add a per-model modality declaration (e.g. a SUPPORTED_MODALITIES class
attribute or a supports_modality(name) method on Model), and have
entrypoint.py:273 consult the loaded model instead of the global frozenset.
Acceptance criteria
How to reproduce
No response
Environment
No response
What happened?
Summary
entrypoint.py:34declares one modality set for every model:used at
entrypoint.py:273to validateinput_modalitieson intake. It hasno per-model dimension, so
audiovalidates for BAGEL even though BAGEL hasno audio encoder — the request is accepted and only fails (or misbehaves)
downstream, instead of being rejected at intake with a clear message.
Current state
input_modalitiesregardlessof what that model actually implements.
which
load_*methods and prefill walks it defines), but nothing exposesthat to the entrypoint.
Proposed fix
Add a per-model modality declaration (e.g. a
SUPPORTED_MODALITIESclassattribute or a
supports_modality(name)method onModel), and haveentrypoint.py:273consult the loaded model instead of the global frozenset.Acceptance criteria
rejected at intake with a clear 400, not accepted and left to fail later
audio/videoinput modalitiesHow to reproduce
No response
Environment
No response