Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ GET /api/events paged by row id, not timestamp
GET /api/summary enough for a status line
GET /api/control is the fleet claiming work?
POST /api/control pause, drain or resume
GET /api/roles where each role's calls go
GET /api/roles where each role's calls go, and which are called
PUT /api/roles re-route a role, live
GET /healthz open, cheap, needs no credential
```
Expand Down
13 changes: 10 additions & 3 deletions docs/DEPLOYMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,9 +184,16 @@ misconfigured one it tells you less than the line above.
| `checkout` | The project's `work_dir` is missing or is not a git repository *inside this process's filesystem* — a container needs it mounted. |
| `disk space` | The volume holding `work_dir` is below the project's configured `min_free_disk_gb` floor. Free and total GiB are included in the detail. |
| `github write` | `gh` is missing, unauthenticated, or the account lacks push on that repo. |
| `reviewer` | No reviewer route. `PUT /api/roles`, or restart with `--reviewer`/`--endpoint`. |
| `reviewer` | No reviewer route, globally or on the project. `PUT /api/roles`, restart with `--reviewer`/`--endpoint`, or give the project its own `roles.reviewer`. |
| `role reachability` | The route exists and the model does not answer it. The detail names the model and the status — an endpoint can advertise a model in `/models` and serve nothing behind it. Route the role somewhere that replies. |
| `base checks` | A configured command failed on an unmodified base-branch worktree — fix the command or its prerequisites before starting. Also reported when no run has happened yet (`not_run`) or one is still going. |

A project's `roles` override the global map **per role**: a project that names
only a reviewer still inherits every other role, and the reviewer it names is
the one its workers call.

Warnings do not block a start and are still worth reading: `checks` means
nothing verifies a diff before the reviewer sees it, and `reviewer
independence` means some share of reviews is a model grading its own work.
nothing verifies a diff before the reviewer sees it, `reviewer independence`
means some share of reviews is a model grading its own work, and `model
latency` means a model answered a one-token prompt slowly — usable, and every
call pays that first.
6 changes: 6 additions & 0 deletions docs/INTERNALS.md
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,12 @@ This was previously documented in three places and enforced in none. It is
deliberate choice, and blocking it would be the harness overruling an operator
about their own budget. What it must not be is a surprise.

It compares the reviewer to the implementer that **actually runs**. In session
mode that is the agent process, not a routed model, so the verdict says so
rather than comparing two routes that never meet: the configured `implementer`
is never called there, and a warning about it would be about a pairing that
does not exist.

---

## 7. Merge, and the only honest quality metric
Expand Down
9 changes: 8 additions & 1 deletion docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -400,6 +400,13 @@ Takes effect on the next call, no restart. This is possible only because a
call site names a **role**, never a model — so re-routing one is a data change
rather than a code change.

The response says which of those roles this deployment actually calls. In
session mode the agent process plans and implements with its own credentials
and endpoint, so `planner` and `implementer` come back `"used": false` with the
command that does that work instead: they are stored, the non-session executor
uses them, and nothing here will. A project can override any role for itself
with `roles` on its registration; unnamed roles still come from this map.

Worth doing deliberately: a reviewer on the same vendor as the implementer
means some share of reviews is a model grading its own work. Nothing enforces
that; it is your call.
Expand Down Expand Up @@ -471,7 +478,7 @@ GET /api/events paged by row id, not timestamp
GET /api/summary enough for a status line
GET /api/control is the fleet claiming work?
POST /api/control pause, drain or resume — never interrupts work
GET /api/roles where each role's calls go
GET /api/roles where each role's calls go, and which are called
PUT /api/roles re-route a role, live
GET /api/readiness can anything actually run, and why not
GET /healthz open, cheap, needs no credential
Expand Down
94 changes: 58 additions & 36 deletions src/agent_harness/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,15 +138,26 @@ def _http_transport(api_key: str) -> Any:
client = httpx.Client(timeout=httpx.Timeout(600.0, connect=30.0))

def transport(route: Any, messages: Any, options: Any) -> Any:
asked = float(options.get("timeout") or 0.0)
payload = {"model": route.model, "messages": list(messages)}
payload.update({k: v for k, v in options.items() if k != "role"})
# `timeout` instructs the transport; it is not a completion parameter,
# and sending it as one would have the provider reject the request.
# Preflight's reachability probe sets it, because a probe that
# inherited the work timeout would take ten minutes to establish that
# a model is not answering.
payload.update({k: v for k, v in options.items() if k not in ("role", "timeout")})
response = client.post(
f"{route.endpoint.rstrip('/')}/chat/completions",
headers={
"Authorization": f"Bearer {route.api_key or api_key}",
"content-type": "application/json",
},
json=payload,
timeout=(
httpx.Timeout(asked, connect=min(30.0, asked))
if asked
else httpx.USE_CLIENT_DEFAULT
),
)
return Response(response.status_code, dict(response.headers), response.text)

Expand All @@ -159,10 +170,9 @@ def _run(args: argparse.Namespace) -> int:
import json as _json
import shlex

from . import providers
from .executor import Checks, Executor
from .github import GitHub
from .model_client import ModelClient, Route
from .model_client import ModelClient, Route, routes_from_map
from .work import RUNNING, WorkQueue, WorkRecord

# With a session host the CLI agent does the implementing, so only the
Expand Down Expand Up @@ -282,16 +292,7 @@ def emit(event: dict[str, Any]) -> None:
print(f"note: the stored role map had no route for {', '.join(filled)}; used the flags.")

def live_routes() -> dict[str, Route]:
stored = queue.get_setting(ROLE_MAP_KEY) or {}
return {
name: Route(
route["model"],
route["endpoint"],
providers.PROVIDERS.get(route.get("provider", ""), providers.CLAW_BAY),
api_key=api_key,
)
for name, route in stored.items()
}
return routes_from_map(queue.get_setting(ROLE_MAP_KEY) or {}, api_key=api_key)

# Nothing claims work until every role this run needs can be routed. The
# alternative is finding out on the first model call -- after the project
Expand Down Expand Up @@ -322,7 +323,13 @@ def live_routes() -> dict[str, Route]:
# enforced in none, so a reviewer could be the same model as the
# implementer and nothing would mention it -- every review a model
# grading its own work, invisibly.
independent, why = client.reviewer_independence()
#
# Against the implementer that actually runs: in session mode the agent
# process writes the code, so comparing the reviewer to the configured
# implementer route would be a verdict about a pairing that never happens.
independent, why = client.reviewer_independence(
implemented_by=args.agent if session_mode else ""
)
print(("reviewer: " if independent else "WARNING: ") + why)

executor: Any
Expand Down Expand Up @@ -683,7 +690,9 @@ def main(argv: list[str] | None = None) -> int:
)
maintenance.start()

fleet, reviewer_client, host = _fleet_for_serve(args, queue_for_serve, audit=audit)
fleet, reviewer_client, host, executor_roles = _fleet_for_serve(
args, queue_for_serve, audit=audit
)
if fleet is None:
print(
"monitoring only: no --session-host, so no worker pool is attached and "
Expand All @@ -704,6 +713,7 @@ def main(argv: list[str] | None = None) -> int:
# Readiness probes it with a read. Passing the client rather
# than the URL keeps the token out of the API layer.
session_host=host,
executor_roles=executor_roles,
),
host=args.host,
port=args.port,
Expand All @@ -720,10 +730,10 @@ def main(argv: list[str] | None = None) -> int:

def _fleet_for_serve(
args: argparse.Namespace, queue: Any, *, audit: Any | None = None
) -> tuple[Any | None, Any | None, Any | None]:
) -> tuple[Any | None, Any | None, Any | None, Any | None]:
"""The supervised half of `serve`: a fleet the API's start action can use.

Returns (None, None) for a monitoring-only deployment. That mode is
Returns all-None for a monitoring-only deployment. That mode is
supported on purpose — a dashboard over someone else's harness should not
need a session host, a model key or a checkout — and the API already
refuses to start a project when nothing can claim.
Expand All @@ -732,18 +742,17 @@ def _fleet_for_serve(
the API's start action does, and only after preflight passes.
"""
if not args.session_host:
return (None, None, None)
return (None, None, None, None)

import json as _json
import shlex

from . import providers
from .api import ROLE_MAP_KEY
from .events import KINDS, MODEL_CALL, Event
from .fleet import Fleet
from .github import GitHub
from .model_client import ModelClient, Route
from .runtime import session_executor_factory
from .model_client import ModelClient, Route, effective_routes, routes_from_map
from .runtime import ExecutorRoles, session_executor_factory
from .session_executor import AgentSpec
from .session_host import HttpSessionHost

Expand All @@ -766,25 +775,29 @@ def _fleet_for_serve(
queue.set_setting(ROLE_MAP_KEY, stored)

def live_routes() -> dict[str, Route]:
current = queue.get_setting(ROLE_MAP_KEY) or {}
return {
name: Route(
route["model"],
route["endpoint"],
providers.PROVIDERS.get(route.get("provider", ""), providers.CLAW_BAY),
api_key=api_key,
)
for name, route in current.items()
if route.get("model") and route.get("endpoint")
}
return routes_from_map(queue.get_setting(ROLE_MAP_KEY) or {}, api_key=api_key)

def routes_for(project_id: str) -> dict[str, Route]:
"""One project's effective map, read live on every call.

The project row is read here rather than closed over so that a role
override written through the API reaches a worker that is already
running — the same reason the global map is read per call.
"""
project = queue.get_project(project_id)
return effective_routes(
live_routes(),
routes_from_map(getattr(project, "roles", None) or {}, api_key=api_key),
)

routes = live_routes()
if "reviewer" not in routes:
# Not fatal, and not silent: preflight blocks the start with exactly
# this reason, so the fleet may as well exist and say why now.
print(
"warning: no reviewer is routed. Preflight will refuse to start any "
"project — set one with --reviewer/--endpoint or PUT /api/roles.",
"warning: no reviewer is routed globally. Preflight will refuse to start "
"any project that does not override one — set a global reviewer with "
"--reviewer/--endpoint or PUT /api/roles.",
file=sys.stderr,
)
events_path = args.events or Path(args.db).with_name("events.jsonl")
Expand Down Expand Up @@ -849,11 +862,13 @@ def emit(event: dict[str, Any]) -> None:
)

host = HttpSessionHost(args.session_host, token=host_token)
agent = AgentSpec(command=tuple(shlex.split(args.agent)))
factory = session_executor_factory(
queue,
host=host,
agent=AgentSpec(command=tuple(shlex.split(args.agent))),
agent=agent,
reviewer=reviewer_client,
routes_for=routes_for,
github_for=GitHub,
ui_base_url=args.session_host,
on_event=emit,
Expand All @@ -863,7 +878,14 @@ def emit(event: dict[str, Any]) -> None:
print(f"events: {events_path}")
# The fleet emits into the same stream as the executors: a worker that
# dies is recorded next to the work it was doing, not in a separate log.
return (Fleet(queue, factory, poll_seconds=args.poll, on_event=emit), reviewer_client, host)
return (
Fleet(queue, factory, poll_seconds=args.poll, on_event=emit),
reviewer_client,
host,
# What this deployment will actually call, so the API can stop
# advertising the two roles the agent process does instead.
ExecutorRoles.for_session(agent),
)


if __name__ == "__main__":
Expand Down
Loading
Loading