Feat/api passthrough - #3
Open
mikezupper wants to merge 14 commits into
Open
Conversation
… ollama Serve GET /.well-known/livepeer-runner (runner-contract.md 1.1.0): one entry per served model — SERVED_MODEL_NAME collapses to one, otherwise an array over the discovered, allowlist-filtered models. Declares openai-usage over USAGE_FIELD, http-openai-model-ready against a new GET /v1/models, identity.openai.model + provider from UPSTREAM_KIND, and x-* keys for the operator metadata. Default CAPABILITY_NAME becomes openai:chat-completions. The X-Livepeer-Work-Units constant is renamed to workUnitsHeader: it is a trailer on streams and a header on weighted unary responses, and it is informational either way — the broker bills from the body. UPSTREAM_KIND=ollama is accepted again; the vendor transport change had dropped it without a decision. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls1b4hHRYHR8CvNiF3pVvk
GET /.well-known/livepeer-runner declares openai:embeddings over unary, openai-usage on USAGE_FIELD, readiness via a new GET /v1/models, and identity.openai.model per served model. The usage header stays as the runner's own claim. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls1b4hHRYHR8CvNiF3pVvk
…tecture Each runner gets a pure contract.py (no torch, no FastAPI) with unit tests, wired to GET /.well-known/livepeer-runner; /options is gone. - audio: two entries in one array (openai:audio-transcriptions, openai:audio-translations), multipart, audio_seconds via response-header; CAPABILITY_NAME unset serves both, one id serves one, anything else is a startup error. MODEL_ALIAS is the identity. - tts: openai:audio-speech, input_chars via request-formula over the code points of $.input. - image-generation: openai:images-generations, images via request-formula over $.n (default 1); MODEL_ID is the identity. - rerank: text:rerank with the plain identity.model key, documents via response-header — the runner now emits X-Livepeer-Work-Units with the number of documents scored; meta.model is the alias. gpu_probe.py now compares the device's compute capability with the torch build's arch list and exits non-zero naming the -pascal flavor when there are no kernels for it (cu128 wheels ship sm_75+ only). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls1b4hHRYHR8CvNiF3pVvk
…cal flavor Default TAG v2.0.0 in the build script, Dockerfiles, compose overlays and setup-models. CAPABILITY_NAME defaults move to the colon form; the audio image leaves it unset so both entries are advertised. infra/env/*.env.example were gitignored by the env/ pattern; add the exception and track them. ./build-images.sh test runs go vet/test for both Go modules and python unittest for the four Python runners in Docker; build.yml gates on it. release.yml also builds and pushes the four CUDA runners at <tag>-pascal from the cu126 wheel index, which carries sm_60 kernels, for GTX 1080-class cards the default cu128 wheels cannot run on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls1b4hHRYHR8CvNiF3pVvk
BROKER-CONTRACT.md now describes the contract from the runner's side — endpoint, body, how the agent relays it, how work is counted per runner, readiness, framing, and a full example document per image — and points at runner-contract.md / runner-attach.md as the source of truth. RUNNER-INVARIANTS, CANONICAL-CAPABILITIES, RUNNERS and the rest follow: colon ids, no /options, MODEL_ALIAS, the Pascal flavor, tests in CI. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls1b4hHRYHR8CvNiF3pVvk
…modules infra/scripts/build-images.sh follows the network-modules script: one image table, substring filters, PUSH=1 that refuses a dirty tree and prints + records pushed digests (infra/build/<TAG>-digests.txt), VERSION derived from git via infra/build/git-version.sh, default tag and toolchain pins in infra/build/image-versions.env. Added on top of that pattern, because this repo needs them: base-image dependencies resolved automatically and passed as BASE_IMAGE, local-only images under a separate registry prefix, buildx for the multi-arch Go runners (--push when publishing, --load otherwise), and the PYTORCH_INDEX_URL knob the -pascal flavor rides on. validate-compose.sh and test.sh are their own scripts; the root build-images.sh is a shim for the old subcommands so docs and habits keep working. Every Dockerfile takes ARG VERSION and sets the OCI source/licenses/ version labels on its final stage; the Go runners link the version in and log it at startup. The release workflow builds and pushes in one pass, then attaches the digest records to the GitHub release and includes them in the notes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls1b4hHRYHR8CvNiF3pVvk
Per the network-modules team (2026-09-05): FLUX.1-dev cannot fit an 8 GB card whatever the CUDA base, and the catalog will not admit Pascal for image generation. The -pascal flavor covers audio, TTS and rerank; the catalog selects it through the nvidia/gtx-1080 image-map key (ac786ca). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls1b4hHRYHR8CvNiF3pVvk
A push run reported a parse error at the digest block on one machine that does not reproduce under bash 4.0 through 5.2 here. The only unusual constructs on those lines were an em dash inside a double-quoted string and a command substitution beside it; both are gone, at no loss. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls1b4hHRYHR8CvNiF3pVvk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.