Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,18 +56,20 @@ engine is one module and a few small changes:
- the domains its CLI needs (`egress`);
- where the CLI keeps its login (`home`), the variable that points it there (`home_env`), and its own login
command. An engine that runs on API keys or on a local model server has no login: it implements `Keyed`
instead, its `providers` (each with where to create a key and where its models are listed) and `with_key`,
which returns the engine on an account's key. The key reaches the step only as a file in `/ctx`
(`Invocation.files`), never on the command line or in the environment;
instead, its `providers` (each with where to create a key, where its models are listed and, when models.dev
has them, its id there, so that the Accounts page suggests its models) and `with_key`, which returns the
engine on an account's key. The key reaches the step only as a file in `/ctx` (`Invocation.files`), never on
the command line or in the environment;
- how a step becomes a command line (`invocation`), and how the output becomes events and a result (`parse` and
`result`).
2. **If the CLI can read the account's quota without calling the model,** add `quota_invocation` and
`quota_reading` (`QuotaProbe`). Without them, lastcall cannot plan bursts for that engine's accounts, unless the
engine is `Keyed`: then lastcall's ledger reads what its steps cost, so `result` must report `cost_usd`. A
window that limits only some models names the model in `Window.model`.
3. **Register it** in `src/lastcall/engines/__init__.py`.
4. **Pick its models** for each tier (fast, balanced, deep) in `src/lastcall/data/tiers.json`. An engine on API
keys may have none there: each of its accounts names its models.
4. **Pick its models** for each tier (fast, balanced, deep) in `src/lastcall/data/tiers.json`, and the names that
its CLI takes under `models`, which the Accounts page suggests. An engine on API keys may have none there:
each of its accounts names its models.
5. **Install its CLI**, at a pinned version, in `runner/Dockerfile`, and add its login to `runner/login.sh`, or, for
an engine with no login, a branch that says so. A test checks that every engine has one there.
6. **Regenerate `egress/filter`.** The command is in `tests/test_egress.py`.
Expand Down
81 changes: 70 additions & 11 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,12 +272,12 @@ accounts:
- id: claude-max
engine: claude-code
login: cli # official login inside the account's credential volume
tiers: {fast: haiku/medium, balanced: sonnet/high, deep: opus/high}
tiers: {fast: haiku/medium, balanced: sonnet/high, deep: opus/xhigh}
spend: {policy: before-reset, reserve_pct: 10}
- id: chatgpt-plus
engine: codex
login: cli
tiers: {fast: gpt-6-luna/high, balanced: gpt-6.1-sol/high, deep: gpt-6.1-sol/high}
tiers: {fast: gpt-6-luna/medium, balanced: gpt-6.1-sol/high, deep: gpt-6.1-sol/xhigh}
spend: {policy: before-reset, reserve_pct: 10}
- id: openrouter # v0.5
engine: opencode
Expand All @@ -301,6 +301,33 @@ goes back to the default. The data file also lists the effort levels that each e
`--effort` (low to max), Codex's `model_reasoning_effort` (minimal to max) and OpenCode's `--variant`, whose levels
depend on the model. More effort finds more, and spends more of the quota or the budget.

The default efforts differ by tier because the work does: `fast` is only asked for by a repository's own reviewers,
so it takes `medium`; `balanced` runs the reviewers, at `high`; and `deep` runs the judge, the implementers and the
improvement plans, where quality matters most and lastcall spends quota that would expire anyway, at `xhigh`. Both
engines' effort lists include `xhigh`.

**Suggested models.** The model fields of that page suggest names with a `<datalist>`: the owner picks one or types
any other, since the catalogs change every month. Where the names come from depends on the account:

- An account on a CLI login gets the `models` of its engine in the data file, the names that its CLI takes: Claude
Code's aliases `haiku`, `sonnet` and `opus`, which mean the newest model of their family on the Anthropic API,
and the models that Codex's own catalog lists for choosing and has not replaced with a newer one
(`codex debug models --bundled`). `fable` is left out on purpose: `claude -p` can bill a Fable request to usage
credits without asking, and lastcall never spends paid usage.
- An account on an API key (OpenCode on OpenRouter, Anthropic or OpenAI) gets the models of its provider from
models.dev's public catalog, `https://models.dev/api.json`, where OpenCode reads its prices too. The id of the
provider there is `Provider.catalog`. The web UI asks through the same `fetch` as its other lookups, with a
`User-Agent` of its own (models.dev answers 403 to the default one) and gzip (the catalog weighs 6 MB, or 0.6 MB
packed), and waits 5 seconds at most. One request answers for every provider, and the answer stays in memory for
a day, since the Accounts page asks on every load. A failure is kept for 10 minutes: meanwhile the fields suggest
nothing and the page works as before. Of each provider it keeps at most 1000 models, the ones that a step can
run on: they call tools, have a price (the budget needs one) and are not deprecated, and their ids are ones that
an account may name.
- The form that adds an API key cannot follow the provider that it has chosen, since the page runs no script: its
model fields suggest the models of every provider together (OpenRouter writes the maker first, as in
`qwen/qwen3-coder`).
- A local model server (Ollama) gets no suggestions: the web UI cannot reach it.

**Several accounts on one project.** Different providers find different problems (section 16). A review step can
therefore fan out across accounts, and one judge merges the results.

Expand Down Expand Up @@ -461,11 +488,11 @@ multiplies the score.
read.
- **Container logs** (`lastcall.logs`), for apps that run in Docker on lastcall's own host, with no change to their
code. None is read until the owner gives it a project:
- The settings page lists the host's containers (`lastcall logs list` in the terminal), and the owner chooses the
project of each one to read. lastcall suggests one from the container's image (`ghcr.io/<owner>/<name>`, without
its tag), else its Compose project, else its name (`<name>-web-1`), when only one project fits. A suggestion is
read only once the owner saves it, and a no to it is remembered, so that it is not suggested again. The choices
live in the service's database.
- The settings page lists the host's containers (`lastcall logs list` in the terminal; section 13 describes the
page), and the owner chooses the project of each one to read. lastcall suggests one from the container's image
(`ghcr.io/<owner>/<name>`, without its tag), else its Compose project, else its name (`<name>-web-1`), when only one
project fits. A suggestion is read only once the owner chooses it, and a no to it is remembered, so that it is not
suggested again. The choices live in the service's database.
- A container labelled `lastcall.project=<owner/name>` in its compose file is read for that project, matched
without regard to case, with no choice on the settings page, which shows it with its project: the choice can
live with the app. The two ways work together, and the label wins over a choice.
Expand Down Expand Up @@ -789,10 +816,10 @@ the notice sticks to the top from 48 rem up (when the window is also at least 36
clear when it scrolls to the card or to a focused control; on a phone it stays in the flow above the cards, so it never
hides what it brought you to. The settings page holds the connection to an error tracker (Sentry or GlitchTip: the
address, the organization and a read-only token, saved only when one request with them gets through), the container logs
(the host's containers, each with its project or the one lastcall suggests, and the label that works as well), the logs
in Loki (the connection, the label that names the sources, and the project of each, with suggestions), and the
notifications: the destinations, chosen by service with only the fields that service needs (Telegram finds the owner's
chat from the bot's recent messages) or as any Apprise URL, saved only when their test notification gets through, kept
(the host's containers, and the label that works as well), the logs in Loki (the connection, the label that names the
sources, and the sources), both lists in three parts, which the paragraphs below describe, and the notifications: the
destinations, chosen by service with only the fields that service needs (Telegram finds the owner's chat from the bot's
recent messages) or as any Apprise URL, saved only when their test notification gets through, kept
in the secret store and shown with their secrets hidden; the events (a pull request opens, decisions or ideas wait,
lastcall merges an approved pull request or proposes a revert, a run fails or stops early, an account has paid usage on
or its quota cannot be read, and, if the owner wants, a burst starts) and their language. A problem is told once when it
Expand All @@ -816,6 +843,38 @@ brings the owner back after the installation starts a reading of the suggestions
Adding by an address, which asks GitHub whether the App is installed there, stays for a repository that is not in the
list.

The containers of the settings page and the sources in Loki come in three parts, so that dozens of them stay short
(`_lists.html` draws both). `Read now` comes first: what lastcall reads, in compact rows. A container with the label
`lastcall.project` says that its label decides and has nothing to stop; one whose label names no project says so, and is
read once the project is added. One that the owner chose shows its project, as the Projects page writes it, with a Stop
reading button, which records the owner's no. The suggested ones, which have no label, no choice and one project that
fits, use the tiles of the Projects page's add-many form: a checkbox with the container and the project that it would be
read for, a button that reads all of them, and a bar that reads or skips the ticked ones (a skip records the no, as
`lastcall logs set <container> -` does, so that the suggestion does not come back). Every other container or source goes
in a fold that says how many they are, closed unless a filter is set or a typed project is refused. From the seventh on,
the fold has a filter by name (`logs_q` and `loki_q` in the address, ignoring case, each keeping the other's, and the
fold opens while one is set), and each of them gets one text field whose `<datalist>` holds the projects, so that the
page lists the projects once, not once in every row. A choice for a project that was removed shows apart in that fold,
as before, and no save erases it: an empty field changes nothing. lastcall's own containers, which are in its Compose
project or are named `lastcall-...`, never show, in either list.

The forms of those parts post to `POST /settings/logs` and `POST /settings/loki/sources` (which also says the label that
names the sources: the choices belong to it). Each row is `name-N` and, for a project, `project-N`, and `pick` says what
the button asks: `selected` (the rows ticked in `tick`), `suggested` (all of them), `skip` or `stop` (a no), or, without
it, the projects typed in the fold of the others. The server asks the host and Loki nothing: it checks every project
against the projects, ignoring case, and writes it as the Projects page writes it. A project typed that is none of them
saves nothing: the section says so and gives back what was typed, with the fold open. A form that names a container or a
source that is not acceptable (one of lastcall's steps), or a suggestion for a project that does not exist, is refused
as a forged form, and a button for the ticked ones with none ticked says so.

With more than 6 projects, or while one is set, the Projects page offers a search by name (`search`, in the whole
repository name, ignoring case) and the filter links All, Active, Inactive and Waiting for you (`show`: pull requests to
review, decisions in an issue or ideas to choose, as the dashboard lists them), each with its count for the current
search, and the current one with `aria-current`. The form and the links bring the page back to the bar (`#find`) and
keep the suggestions' filter (`q`), which in turn keeps these; a search that leaves no project says so, with a link to
show them all. Also with more than 6 projects, each card's Run now starts folded, like its Settings, so that the cards
are short: its title is the summary that opens it, and its "?" moves inside, after a line that says what it does.

The state at the top of a project's card, its chip ("active · priority ×2" or "inactive"), is a button in a small form:
one click turns the project's bursts on or off (`POST /projects/<owner>/<name>/active`). The form says the state that it
wants and does not flip the one that it finds, so that a page that is out of date never turns the project the wrong way.
Expand Down
33 changes: 24 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,9 +113,13 @@ spending limit at the provider too.

**Models and effort.** Every step asks for a tier: the reviewers for *balanced*; the judge, the implementers and the
improvement plans for *deep*; your repository's own reviewers for the one they name. Each engine has a default
model and effort per tier, such as opus at `high` effort for Claude Code's *deep*. On the Accounts page, under
model and effort per tier, such as opus at `xhigh` effort for Claude Code's *deep*. The efforts differ because the
work does: *deep* runs at `xhigh` because quality matters most there and the quota would expire anyway, *balanced* at
`high`, and *fast*, which only your repository's own reviewers ask for, at `medium`. On the Accounts page, under
*Models and effort*, or with `lastcall account tier`, you can change both for one account, subscriptions included.
More effort finds more, and spends more of the quota.
The Model field suggests names and takes any other you type: the ones that the engine's CLI takes or, for an API key,
the models that [models.dev](https://models.dev) lists for its provider. More effort finds more, and spends more of
the quota.

The rest of lastcall only knows the engine contract. Adding another tool is one module and a few small changes:
[CONTRIBUTING.md](CONTRIBUTING.md#adding-an-engine) lists them.
Expand Down Expand Up @@ -169,6 +173,8 @@ Projects, and Settings holds a fourth that is optional:
- 📁 **Projects.** lastcall suggests your repositories and says why. Tick the ones you want, or add the suggested ones
or all of them at once, with a filter when you have many. Any other repository goes in by its address. A click on
a project's state turns its bursts on or off, and its trash button removes it from lastcall but keeps its history.
With many projects, a search by name and the links All, Active, Inactive and Waiting for you (each with its count)
keep the list short, and each card's Run now starts folded.
- 🔔 **Settings.** Where lastcall should notify you. Choose the service and fill in what it asks for: Telegram (paste
your bot's token and lastcall finds your chat), ntfy, Discord, Slack or email, or any of the
[services that Apprise reaches](https://github.com/caronc/apprise#supported-notifications). lastcall saves a
Expand Down Expand Up @@ -341,10 +347,14 @@ code where they happen, and the reviewers look for their causes first. Projects
first. The messages are masked before they reach a model.

**Container logs.** If your apps run in Docker on the same machine as lastcall, it can read their logs too, with or
without a tracker, and with no change to their code. On the Settings page, under *Container logs*, choose the
project of each container to read, and save. lastcall lists the machine's containers and suggests a project for
each from its image (`ghcr.io/owner/name`), its Compose project or its name. It reads none until you save, and it
remembers when you say no to a suggestion. In the terminal: `lastcall logs list`, then
without a tracker, and with no change to their code. On the Settings page, under *Container logs*, lastcall lists the
machine's containers in three parts, so that dozens of them stay short. *Read now* comes first: the containers that
lastcall reads, each with its project and how it was chosen, and a button to stop reading one. Then come the suggested
ones, which lastcall matched to a project from the container's image (`ghcr.io/owner/name`), its Compose project or
its name: tick the ones to read, or read all of them at once. The rest wait in a fold that says how many they are,
with a filter by name: type a project next to a container to read it, and the list of your projects narrows as you
type. lastcall reads none until you choose it, and it remembers when you skip a suggestion or stop reading a
container. lastcall's own containers never show there. In the terminal: `lastcall logs list`, then
`lastcall logs set <container> <owner/name>`.

You can also keep the choice in the app's compose file: label its containers with the project's name, as the
Expand All @@ -371,7 +381,9 @@ collects their logs, lastcall reads them there, with nothing to install on those
1. On the Settings page, under *Logs in Loki*, give Loki's address and, if it asks for one, a login that can only
read. In Grafana Cloud, that is the user number of your Loki and a token with the `logs:read` scope.
2. lastcall lists the sources that Loki saw in the last 7 days, named by their `container` label when they have one,
and suggests the project of each, as for the containers on its own machine. Choose them, and save.
in the same three parts as the containers on its own machine: *Read now*, the suggested ones, which you tick (or
read all at once), and the rest in a fold with a filter by name, where you type a project next to a source. Each
shows the machines where it ran and its Compose project.

It asks Loki only for the lines of the last day that may be part of an error, the newest 5,000 of each source, and
reads them as it reads a container's. In the terminal: `lastcall loki connect --url http://<host>:3100`,
Expand All @@ -387,7 +399,9 @@ in lastcall only, never in `.lastcall.yml`, so that nobody can turn them on from
later run, on the current code.
- **Watch after the merge.** For a day after a merge, lastcall looks at the checks of the merged commit and, if you
give one, at a health address every 10 minutes. When the checks fail where they passed before, or the address
fails 3 times in a row, it opens a pull request that reverts the change, pauses the project and tells you.
fails 3 times in a row, it opens a pull request that reverts the change, pauses the project and tells you. Nothing
else takes part in the watch: container logs, Loki and your error tracker feed the audits, which use them to find
the causes of errors.

Both need the GitHub App to read checks and commit statuses, which Apps created by lastcall 0.7.2 or later have; the
Projects page says what to do for an older App.
Expand Down Expand Up @@ -457,7 +471,8 @@ With a local model on Ollama, it goes to no model provider.
The names and versions of your dependencies go to [osv.dev](https://osv.dev) in every audit, and to
[deps.dev](https://deps.dev) for upgrades. The agents' steps reach nothing beyond the allowlist in
[egress/filter](egress/filter), and errors from production are masked before they reach a model. lastcall itself
sends no telemetry.
sends no telemetry. Opening the Accounts page makes it ask [models.dev](https://models.dev), about once a day, for
the names of the models that an API key's provider offers, in a request that carries nothing of yours.

### What if a file in my repository tricks the agent?

Expand Down
Loading
Loading