Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 29 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ supervisor) came from **audiosilo-sidecars** with its history (`git log --follow
and `git blame` reach the reliability fixes). sidecars stays usable until Phase 4,
then is archived with the manager. Phase 0 imported and tested the engine; Phase 1a
runs it (`internal/app/engine.go`) behind the `sidecars.generate` job
(`internal/generate`), which writes the results into the book's `.audiosilo/`.
(`internal/generate`), which writes the results into the book's `.audiosilo/`;
Phase 1c adds the description step and the `sidecars.describe` job beside it.

## Build / test / gate

Expand Down Expand Up @@ -98,7 +99,8 @@ internal/
0004 workers)
auth/ the page's single password + hashed sessions + login limiter
-- Phase 1a --
generate/ sidecars.generate: server job -> engine book -> .audiosilo/ + rescan
generate/ sidecars.generate: server job -> engine book -> .audiosilo/ + rescan;
sidecars.describe (Phase 1c): a description from a finished run's notes
agentproto/ the staging protocol shared with the agent container (a leaf)
agentbox/ the Bridge's side: an agent.Runner over the staging dir
agentrun/ the agent container's side: `agent-runner` (claim, run, harvestable out/)
Expand Down Expand Up @@ -143,8 +145,9 @@ in PACKAGES.md.
use, checked again before each write), temp file then rename, umask 022, a
targeted rescan after, deletes to `.audiosilo-trash/`. A library that isn't mapped
and writable refuses a new write job (a resumed one parks and waits). Today the
only writer is `generate`: `characters.json`, `recaps.json` and `bridge.json` in the
book's `.audiosilo/` (a single-file book's prefixed `<stem>.`), nothing else.
only writer is `generate` (its two jobs): `characters.json`, `recaps.json`,
`description.json` (Phase 1c) and `bridge.json` in the book's `.audiosilo/` (a
single-file book's prefixed `<stem>.`), nothing else.
- **The agent container's files are untrusted** (and the Bridge's are to it): every
read goes through `agentproto.OpenRegular`/`ReadRegular` inside an `os.Root`, the
harvest is capped, and a credential never leaves the agent container (only its
Expand Down Expand Up @@ -191,7 +194,28 @@ install`/`uninstall` (launchd agent, systemd user unit), worker tokens on the pa
the protocol-version refusal, and the engine's ASR as `workers.Router`: remote when a
worker is connected, else `asr.worker_fallback` (`local`, the default, which yields
to a worker that connects mid-decode; or `wait`, which parks `asr_worker_waiting`
until one connects). See BRIDGE-PLAN.md's Status table for what is next.
until one connects).

**Phase 1c (own-words descriptions):** the description step
(`pipeline.Executor.Describe`, not an engine stage: its own `describing` stage run
on the book, which the reconcile and the supervisor skip, run under the
scheduler's `Hold` on the book, or inside the contributing stage) writes a spoiler-free
description from the fact notes of the book's opening chapters ONLY (the staged dir
holds `opening.md` and nothing else: no transcript, no `knowledge-final.md`, no later
chapter), validated (shape, length, voice, meta's description schema, the n-gram
check against whatever source text is left: the retention removes an audio book's
transcripts, and an ebook's text goes as it reaches done, so the contributing stage
of a book a `sidecars.generate` job follows runs it first, best effort and once,
while that text is there) and composed with the run's work
and sources. `sidecars.generate` runs it at its finish, after the characters and
recaps are saved (reusing the contributing stage's for free, its check kept; once
more at done when it first ran at a contributing park), and
writes `description.json` with them, but never fails, parks or holds for it (the
result says `description: false` and why; a regenerate that gets none keeps the
earlier file vouched for in `bridge.json`); `sidecars.describe` (advertised beside
`sidecars.generate`) writes one for a book already generated here, parking for the
agent's parks. Local only: a contribution never carries it. See BRIDGE-PLAN.md's
Status table for what is next.

Operative caveats carried from sidecars (PACKAGES.md has the detail):
- **audiosilo-meta is pinned** and the schema drift guard
Expand Down
215 changes: 210 additions & 5 deletions PACKAGES.md

Large diffs are not rendered by default.

55 changes: 55 additions & 0 deletions internal/agent/prompts/description.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
You write the own-words description of "{{.Title}}" by {{.Authors}}{{if .Series}} ({{.Series}}{{if .SeriesPos}} book {{.SeriesPos}}{{end}}){{end}}:
the one paragraph a stranger reads before deciding to listen. It appears where a
publisher's blurb would, and everyone who opens the book sees it, whatever they
have heard so far.

## What you have

You work in the current directory. It contains ONLY:

- `opening.md` - private fact notes for the opening of the book: its first
{{.Chapters}} of {{.ChapterCount}} chapters, plus any front matter before chapter 1.
This is your whole source.{{if .IsEbook}} The `## Chapter N` headings are the
ebook's chapter numbers.{{else}} The `## Chapter N` headings number the audio
files, not the story's chapters (an opening announcement or preface can take a
file), so read them as labels only.{{end}}
- `out/` - the ONLY place you write.

You deliberately have nothing else: no book text, no transcript, no notes on
later chapters, no publisher copy. Do not use any tool other than reading and
writing files in this directory. No web access, and do not draw on your own
memory of this book or of anything written about it.

## The rules

- **Setup only.** The premise, the situation, who the book is about and what it
is asking. Nothing past the opening act: no reveals, no turns, no deaths, no
roles a character only takes on later, and never the ending. A note in
`opening.md` that points ahead ("named later", "revealed to be") is not yours
to use. When a detail is arguably a reveal, leave it out.
- **Grounded in the notes.** Every claim must come from `opening.md`. Leave out
what the notes don't support: skip over invent, always. Don't name a place,
time or person the notes don't name.
- **Your own words.** Never a blurb, and never the book's own sentences: an
8-word run copied from the book fails a mechanical check. Quote nothing.
- **Neutral reference-guide voice.** Say what the book is and let the reader
decide. No sales adjectives (gripping, unputdownable, page-turner, must-read,
masterpiece, breathtaking, captivating, riveting, thrilling, unforgettable, and
their like), no second person (no "you" or "your"), no rhetorical questions
(no sentence ends in a question mark), no star-rating by adjective.
- **Length:** 200 to 1500 characters. Aim for 80 to 150 words in one paragraph;
two paragraphs at most.
- **Punctuation:** use a hyphen, never an em dash.
- Present tense, third person. A sequel may say which series it continues, but
must not summarise the earlier books' endings.

## Output

Write `out/description.json` containing exactly one JSON object with exactly one
key, and nothing else:

```json
{"text": "The description paragraph."}
```

No other keys, no Markdown, no comments, nothing before or after the object.
24 changes: 24 additions & 0 deletions internal/agent/prompts/prompts_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -213,3 +213,27 @@ func TestFactPassBranchesShareTheSpoilerContract(t *testing.T) {
t.Error("ebook branch names the wrong text directory")
}
}

// The description prompt renders both source branches, states the exact output
// shape its strict reader enforces, and keeps to hyphens.
func TestDescriptionPromptRenders(t *testing.T) {
type data struct {
Title, Authors, Series, SeriesPos string
Chapters, ChapterCount int
IsEbook bool
}
for _, ebook := range []bool{false, true} {
out, err := Render("description.md", data{Title: "T", Authors: "A", Series: "S", SeriesPos: "2", Chapters: 3, ChapterCount: 12, IsEbook: ebook})
if err != nil {
t.Fatal(err)
}
for _, want := range []string{"opening.md", `{"text": "The description paragraph."}`, "exactly one", "3 of 12 chapters", "S book 2", "skip over invent"} {
if !strings.Contains(out, want) {
t.Errorf("ebook=%v: prompt lacks %q", ebook, want)
}
}
if strings.Contains(out, "\u2014") {
t.Error("the prompt has an em dash")
}
}
}
5 changes: 3 additions & 2 deletions internal/app/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,8 @@ func New(ctx context.Context, opts Options) (*Bridge, string, error) {
}
return nil
})
b.gen = generate.New(generate.Options{DB: db, Engine: b.engine.generateEngine, Libraries: b.libs, Mount: b.mount,
b.gen = generate.New(generate.Options{DB: db, Engine: b.engine.generateEngine, Describer: b.engine.describer,
Libraries: b.libs, Mount: b.mount,
Server: func() generate.Server {
if cl := b.conn.Client(); cl != nil {
return cl
Expand All @@ -160,7 +161,7 @@ func New(ctx context.Context, opts Options) (*Bridge, string, error) {
WorkRoot: filepath.Join(opts.DataDir, "work"), Version: opts.Version, RetentionDays: cfg.Transcripts.RetentionDays,
CanContribute: b.canContribute, Log: opts.Logger})
b.runner = jobs.New(db, b.server, b.tokenRefused, jobs.Options{Log: opts.Logger,
ServerID: func() string { return b.conn.Pairing().ServerID }}, verifier, b.gen)
ServerID: func() string { return b.conn.Pairing().ServerID }}, verifier, b.gen, b.gen.DescribeJob())
conn.onChange = b.linkChanged
return b, oneTime, nil
}
Expand Down
4 changes: 2 additions & 2 deletions internal/app/app_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ func TestPairBeatMapAndRevoke(t *testing.T) {
return ok && len(st.Libraries) == 1 && st.Libraries[0].Mapped && st.Libraries[0].Writable
})
st, _ := fs.lastBeat()
if st.Version != "0.1.0" || strings.Join(st.Kinds, ",") != "bridge.verify,sidecars.generate" || st.UIURL != "http://bridge.lan:8091" {
if st.Version != "0.1.0" || strings.Join(st.Kinds, ",") != "bridge.verify,sidecars.describe,sidecars.generate" || st.UIURL != "http://bridge.lan:8091" {
t.Fatalf("heartbeat = %+v", st)
}
// Once the engine runs, every heartbeat carries the estimates.
Expand Down Expand Up @@ -277,7 +277,7 @@ func TestEngineViewAndJobRoutes(t *testing.T) {
if _, err := b.Job(context.Background(), 5); !errors.Is(err, api.ErrNotFound) {
t.Fatalf("an unknown job = %v", err)
}
if k := strings.Join(b.runner.Kinds(), ","); k != "bridge.verify,sidecars.generate" {
if k := strings.Join(b.runner.Kinds(), ","); k != "bridge.verify,sidecars.describe,sidecars.generate" {
t.Fatalf("kinds = %s", k)
}
}
Expand Down
5 changes: 3 additions & 2 deletions internal/app/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -226,8 +226,9 @@ func (b *Bridge) Job(ctx context.Context, id int64) (*store.ServerJob, error) {
return job, err
}

// RetryJob retries a parked sidecars.generate job: the engine re-admits its book,
// or a job waiting for its book's folder to be writable checks again now.
// RetryJob retries a parked sidecars job: the engine re-admits a generate job's
// book, or a job waiting in a park of its own (its book's folder not writable, a
// description waiting out an agent park) tries again now.
func (b *Bridge) RetryJob(ctx context.Context, id int64) error {
err := b.gen.Retry(ctx, id)
switch {
Expand Down
9 changes: 9 additions & 0 deletions internal/app/engine.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,15 @@ func (s *engineState) generateEngine() generate.Engine {
return nil
}

// describer is the engine's description step for the generate and describe
// jobs (nil until it started: never a typed nil inside the interface).
func (s *engineState) describer() generate.Describer {
if e := s.get(); e != nil {
return e.exec
}
return nil
}

// agentBackend is the CLI the agent container runs (claude unless codex is set).
func agentBackend(cfg config.Config) string {
if cfg.Agent.Backend == config.AgentBackendCodex {
Expand Down
30 changes: 17 additions & 13 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -408,18 +408,21 @@ func Default() Config {

// defaultClaudeModels seeds the claude model routing map: which model runs each
// agent stage under the claude backend. Cheap stages get sonnet; the load-bearing
// synthesis/audit/fix stages get opus. Model aliases ("sonnet"/"opus"/"haiku") are
// what the claude CLI accepts - never a pinned vendor model id. The openai map is
// intentionally empty (an unset codex model = the codex CLI's default model).
// synthesis/audit/fix stages get opus. The description step (state.Describing, one
// small notes file in, one paragraph out) is a cheap one too. Model aliases
// ("sonnet"/"opus"/"haiku") are what the claude CLI accepts - never a pinned vendor
// model id. The openai map is intentionally empty (an unset codex model = the codex
// CLI's default model).
func defaultClaudeModels() map[string]string {
return map[string]string{
"markers_normalizing": "sonnet",
"qa_adjudicating": "sonnet",
"spelling_research": "sonnet",
"fact_pass": "sonnet",
"synthesizing": "opus",
"auditing": "opus",
"fixing": "opus",
"markers_normalizing": "sonnet",
"qa_adjudicating": "sonnet",
"spelling_research": "sonnet",
"fact_pass": "sonnet",
"synthesizing": "opus",
"auditing": "opus",
"fixing": "opus",
string(state.Describing): "sonnet",
}
}

Expand Down Expand Up @@ -837,14 +840,15 @@ func (c Config) Validate() error {
return fmt.Errorf("pricing.rates[%q] values must be >= 0", key)
}
}
// Model-map keys must name agent-lane stages; a typo would silently never route.
// Model-map keys must name agent-lane stages (or the description step); a typo
// would silently never route.
for key := range c.Agent.Claude {
if !state.IsAgent(state.State(key)) {
if !state.RoutesModel(state.State(key)) {
return fmt.Errorf("agent.claude has key %q, which is not an agent stage", key)
}
}
for key := range c.Agent.OpenAI {
if !state.IsAgent(state.State(key)) {
if !state.RoutesModel(state.State(key)) {
return fmt.Errorf("agent.openai has key %q, which is not an agent stage", key)
}
}
Expand Down
5 changes: 3 additions & 2 deletions internal/config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -239,8 +239,8 @@ func TestValidateAgentFields(t *testing.T) {
good := Default()
good.Agent.Backend = AgentBackendClaude
good.Agent.TimeoutMinutes = 1
good.Agent.Claude = map[string]string{"fact_pass": "sonnet", "auditing": "opus"}
good.Agent.OpenAI = map[string]string{"synthesizing": "gpt-x"}
good.Agent.Claude = map[string]string{"fact_pass": "sonnet", "auditing": "opus", "describing": "haiku"}
good.Agent.OpenAI = map[string]string{"synthesizing": "gpt-x", "describing": "gpt-y"}
if err := good.Validate(); err != nil {
t.Errorf("Validate() = %v, want nil for valid agent config", err)
}
Expand All @@ -259,6 +259,7 @@ func TestDefaultSeedsClaudeModels(t *testing.T) {
"synthesizing": "opus",
"auditing": "opus",
"fixing": "opus",
"describing": "sonnet",
}
for k, v := range want {
if cfg.Agent.Claude[k] != v {
Expand Down
Loading
Loading