Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,18 @@ score = core.check_certainty(context, backend)

For lower-level control (custom adapters, model options), use `mfuncs.act()` with `Intrinsic` directly — see examples in `docs/examples/intrinsics/`.

### Weights binding shapes

`Adapter.weights` normalizes each deployment's activation mechanism behind one of
two shapes — a `WeightsBinding` lifecycle for weights you stage yourself, or
`EmbeddedBinding.apply_activation` for weights already in the served model. The
post-activation shape each produces:

| Binding | Reality | Lifecycle verbs | Caller invokes | Normalized post-activation state |
|---------|---------|------------------|-----------------|-----------------------------------|
| `LocalFileBinding` | LocalFile/PEFT | `prepare` / `activate` / `deactivate` / `release` | `activate()` / `deactivate()`, via `adapter_scope` | Backend-internal PEFT adapter state toggled; the outgoing request is untouched |
| `EmbeddedBinding` | Embedded/Granite Switch | none — weights are already in the served model | `apply_activation(request, identity)` | `request.extra_body["chat_template_kwargs"]["adapter_name"]` set; `request.api_params["model"]` removed if present |

### Project Resources

- **Canonical catalog**: `mellea/backends/adapters/catalog.py` — source of truth for adapter function names, HF repo IDs, and adapter types
Expand Down
75 changes: 75 additions & 0 deletions docs/docs/advanced/intrinsics.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,81 @@ For OpenAI backends with Granite Switch, adapters are loaded from the model's
Hugging Face repository configuration instead of the adapter function catalog.
Output format is task-specific — `requirement-check` returns `{"requirement_check": {"score": <float>}}`.

## Composable adapter construction (advanced)

> **Advanced:** `Adapter` composes an `Identity`, an `IOContract`, and a
> weights binding into a single, inspectable object. It's scaffolding for a
> future backend-integration surface (Epic #929) — today, neither backend
> accepts a composed `Adapter` directly: `LocalHFBackend.add_adapter` takes a
> `LocalFileBinding` or the `LocalHFAdapter` shim, while
> `OpenAIBackend.add_adapter` takes only the deprecated
> `EmbeddedIntrinsicAdapter` shim, which builds an `EmbeddedBinding`
> internally. The construction below is illustrative of the binding shapes;
> write a new backend integration against the bindings themselves.

Each weights binding models how its deployment turns an adapter on.
`LocalFileBinding` downloads and loads LoRA/aLoRA weights, so it exposes a
`prepare`/`activate`/`deactivate`/`release` lifecycle:

```python
# Requires: mellea[hf]
from mellea.backends.adapters import Adapter, EmbeddedBinding, Identity, IOContract, LocalFileBinding
from mellea.backends.huggingface import LocalHFBackend
from mellea.backends.openai import OpenAIBackend
from mellea.core import Component


class AnswerabilityContract(IOContract):
def build_prompt(self, **kwargs: object) -> Component:
raise NotImplementedError # request formatting lands with #1516

def parse(self, raw: str) -> dict[str, object]:
import json

return json.loads(raw)


# LocalFile/PEFT reality — LocalHFBackend downloads and loads the weights.
hf_backend = LocalHFBackend(model_id="ibm-granite/granite-4.1-3b")
hf_binding = LocalFileBinding.from_catalog("answerability")
hf_binding.bind_backend(hf_backend)
# hf_binding.prepare() downloads the weights and loads them into hf_backend.
# adapter_type must match the binding — from_catalog loads the first
# catalog-listed adapter type, which is LoRA for answerability.
hf_adapter = Adapter(
identity=Identity(name="answerability", adapter_type="lora"),
io_contract=AnswerabilityContract(),
weights=hf_binding,
)
```

`EmbeddedBinding` has no weights to manage — the adapter is already part of
the served base model — so it exposes a single method, `apply_activation`,
that edits the outgoing request instead of a lifecycle:

```python
switch_backend = OpenAIBackend(
model_id="granite-switch",
api_key="EMPTY",
base_url="http://localhost:8000/v1",
)
switch_adapter = Adapter(
identity=Identity(name="answerability", adapter_type="alora"),
io_contract=AnswerabilityContract(),
weights=EmbeddedBinding.from_base_model(switch_backend),
)
```

Weights-binding support by backend today — this tracks the binding
implementations, not whether a composed `Adapter` can be registered directly:

| Backend | `LocalFileBinding` (LocalFile/PEFT) | `EmbeddedBinding` (Embedded/Granite Switch) | `ServerMediatedBinding` |
| --- | --- | --- | --- |
| `LocalHFBackend` | ✅ shipping — `add_adapter` accepts a `LocalFileBinding` directly | 🔜 planned (#1018) | — |
| `OpenAIBackend` | — | ✅ shipping, via the deprecated `EmbeddedIntrinsicAdapter` shim above, which builds an `EmbeddedBinding` internally | — |

`ServerMediatedBinding` has no backend implementation yet — see discussion #1486.

---

## Guardian adapter functions
Expand Down
2 changes: 2 additions & 0 deletions mellea/backends/adapters/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from ._core import (
Adapter,
AdapterSchemaMismatchError,
EmbeddedActivationRequest,
EmbeddedBinding,
Identity,
IOContract,
Expand Down Expand Up @@ -34,6 +35,7 @@
"AdapterMixin",
"AdapterSchemaMismatchError",
"AdapterType",
"EmbeddedActivationRequest",
"EmbeddedBinding",
"EmbeddedIntrinsicAdapter",
"IOContract",
Expand Down
181 changes: 158 additions & 23 deletions mellea/backends/adapters/_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@
Also provides:

- :class:`LocalFileBinding`
- :class:`EmbeddedBinding` — stub :class:`WeightsBinding` subclass
- :class:`EmbeddedBinding` — Embedded/Granite Switch binding; `apply_activation`,
no weights lifecycle (issue #1142)
- :class:`ServerMediatedBinding` — stub :class:`WeightsBinding` subclass
- :class:`AdapterSchemaMismatchError`
- :class:`_DictContract`, :class:`_ListContract` — generic, capability-agnostic
Expand All @@ -37,7 +38,7 @@
import time
import warnings
from dataclasses import dataclass
from typing import TYPE_CHECKING, ClassVar, Literal
from typing import TYPE_CHECKING, Any, ClassVar, Literal

from ...core import Component, MelleaLogger
from ...helpers.event_loop_helper import _run_async_in_thread
Expand Down Expand Up @@ -680,31 +681,161 @@ def _fire_phase_complete(self, phase: str, duration_s: float) -> None:
)


class EmbeddedBinding(WeightsBinding):
"""Stub binding for weights embedded in a model artifact."""
@dataclass
class EmbeddedActivationRequest:
"""Mutable outgoing-request state that `EmbeddedBinding.apply_activation` edits.

Bundles the two dicts an OpenAI-compatible call site builds separately —
`extra_body` and the top-level API call kwargs — so a binding can edit
both in one call. Both are mutated in place. The caller must merge its
other edits (tool wiring, thinking mode, user options) before calling
`apply_activation`, because the binding performs the final framework edit
and the selected adapter is authoritative.

Attributes:
extra_body (dict[str, Any]): The provider's `extra_body` payload.
`apply_activation` writes the activation field here (e.g.
`chat_template_kwargs.adapter_name` for Granite Switch).
api_params (dict[str, Any]): Top-level request kwargs (e.g. `model`).
`apply_activation` removes entries an embedded adapter's
activation would make incorrect.
"""

extra_body: dict[str, Any]
api_params: dict[str, Any]


class EmbeddedBinding:
"""Weights binding for the Embedded/Granite Switch reality (Epic #929 Phase 2).

Adapter weights for this reality are already baked into the base model —
there is nothing to download, load, toggle, or unload. The only thing
that varies per call is one field on the outgoing request, so this
binding has a single method, `apply_activation`, rather than the four
`WeightsBinding` lifecycle verbs (see discussion #1486, which rescoped
issue #1142: declaring the four verbs here would mean four raises with
nothing behind them, as the previous stub did).

Stateless across calls: `apply_activation` reads only its arguments, so
activating one adapter for a call never leaks into the request built for
the next.

Attributes:
binding_type (ClassVar[str]): `"embedded"`.
source (str): Base model identifier this binding activates adapters
against — the backend's `base_model_name` (e.g. `granite-4.1-3b`
for a backend built against `ibm-granite/granite-4.1-3b`).
Stamped by `OpenAIBackend.add_adapter` at registration; not
otherwise used by `apply_activation`.
"""

binding_type: ClassVar[str] = "embedded"

def prepare(self) -> None:
raise NotImplementedError(
_PHASE_2_NOT_IMPLEMENTED.format(cls="EmbeddedBinding")
)
def __init__(self, source: str = "") -> None:
"""Constructs an EmbeddedBinding.

def activate(self) -> None:
raise NotImplementedError(
_PHASE_2_NOT_IMPLEMENTED.format(cls="EmbeddedBinding")
)
Args:
source: Base model identifier this binding activates adapters
against. Prefer `from_base_model` when a backend is on hand.
"""
self.source = source

def deactivate(self) -> None:
raise NotImplementedError(
_PHASE_2_NOT_IMPLEMENTED.format(cls="EmbeddedBinding")
)
@classmethod
def from_base_model(cls, backend: "AdapterMixin") -> "EmbeddedBinding":
"""Builds an EmbeddedBinding recording `backend`'s base model as the source.

def release(self) -> None:
raise NotImplementedError(
_PHASE_2_NOT_IMPLEMENTED.format(cls="EmbeddedBinding")
Args:
backend: The backend whose base model has the adapter embedded
(e.g. an `OpenAIBackend` pointed at a Granite Switch deployment).

Returns:
An `EmbeddedBinding` with `source` set to `backend.base_model_name`.
"""
return cls(source=backend.base_model_name)

async def apply_activation(
self, request: EmbeddedActivationRequest, identity: "Identity"
) -> None:
"""Edits `request` so the served model activates `identity`'s adapter.

Granite Switch (the only Embedded deployment today) reads the
adapter to activate from `chat_template_kwargs["adapter_name"]` in
the chat template. The rewriter config can also set the top-level
`model` parameter to the adapter's name; for an embedded adapter the
real model is the base model already being served, so that value is
dropped here rather than sent to the API.

Fires `adapter_function_phase_complete` (phase `"activate"`), so
Embedded calls contribute to the `mellea.adapter_function.phase_duration`
metric like every other binding. Does **not** fire
`adapter_function_invocation_complete`: unlike `LocalFileBinding`'s
verbs (driven by `adapter_scope`, which wraps the whole call and
knows the real outcome), this method only edits a request — the
actual generation and parsing happen later, asynchronously, once the
caller awaits the resulting `ModelOutputThunk`. Firing an
invocation-complete event here would have to guess an `outcome` that
this method cannot know, which is worse than not firing it: it would
report `outcome="success"` for calls that go on to fail. Wiring a
real invocation-complete signal in requires the caller (currently
`OpenAIBackend._generate_from_intrinsic`) to fire it once generation
and parsing resolve — tracked as a follow-up, not part of this method.

This method is `async` (unlike the rest of `EmbeddedBinding`'s
surface) purely because hook dispatch (`invoke_hook`) is async; its
own work is synchronous. Its one caller,
`OpenAIBackend._generate_from_intrinsic`, is already a coroutine, so
`await`ing here — rather than bridging through
`_run_async_in_thread`, which is for calling async code from sync
code — avoids spawning a throwaway event loop and thread per call.

Args:
request: The outgoing request state to edit; both of its dicts
are mutated in place.
identity: Identifies the adapter to activate.
"""
started_at = time.monotonic()
chat_template_kwargs = request.extra_body.pop("chat_template_kwargs", {}) or {}
chat_template_kwargs["adapter_name"] = identity.name
request.extra_body["chat_template_kwargs"] = chat_template_kwargs
request.api_params.pop("model", None)
duration_s = time.monotonic() - started_at

await self._fire_activate_phase_complete(identity.name, duration_s)

async def _fire_activate_phase_complete(self, name: str, duration_s: float) -> None:
"""Fires `adapter_function_phase_complete` for the activate phase.

`duration_s` is the cost of editing a dict, not of an adapter
activation in the sense `LocalFileBinding`'s real PEFT activation is —
the resulting `phase_duration` samples for `binding_type="embedded"`
are not comparable to `binding_type="local_file"` samples for the
same adapter name; the histogram carries no `binding_type` attribute
to separate them.

Args:
name: Adapter function name, used as the metric's `name` field.
duration_s: Wall-clock duration of `apply_activation`'s request edit.
"""
if not has_plugins(HookType.ADAPTER_FUNCTION_PHASE_COMPLETE):
return

from ...plugins.hooks.adapter_function import (
AdapterFunctionPhaseCompletePayload,
)

try:
payload = AdapterFunctionPhaseCompletePayload(
name=name, phase="activate", duration_ms=duration_s * 1000.0
)
await invoke_hook(HookType.ADAPTER_FUNCTION_PHASE_COMPLETE, payload)
except Exception:
MelleaLogger.get_logger().warning(
f"adapter_function_phase_complete hook dispatch failed for {name!r} "
"during 'activate'; ignoring so it does not turn a completed "
"request edit into an operation failure.",
exc_info=True,
)


class ServerMediatedBinding(WeightsBinding):
"""Stub binding for server-managed adapter weights."""
Expand Down Expand Up @@ -736,18 +867,22 @@ def release(self) -> None:
class Adapter:
"""Composable adapter dataclass (Epic #929 Phase 0).

Composes an :class:`Identity`, an :class:`IOContract`, and a
:class:`WeightsBinding` into a single, inspectable object.
Composes an :class:`Identity`, an :class:`IOContract`, and a weights
binding (a :class:`WeightsBinding` or :class:`EmbeddedBinding`) into a
single, inspectable object.

Attributes:
identity (Identity): Name, type, and capability for this adapter.
io_contract (IOContract): Prompt builder and output parser.
weights (WeightsBinding): Pluggable weights lifecycle handler.
weights (WeightsBinding | EmbeddedBinding): Pluggable weights handler —
either a `WeightsBinding` (a lifecycle to stage and switch on) or
an `EmbeddedBinding` (nothing to stage; activation edits the
outgoing request instead).
"""

identity: Identity
io_contract: IOContract
weights: WeightsBinding
weights: WeightsBinding | EmbeddedBinding

# NOTE(#1516): a construction-time cross-check that `weights.adapter_type`
# agrees with `identity.adapter_type` was tried here and backed out. It is the
Expand Down
Loading
Loading