Skip to content

feat(tasks): answer "what's next" by priority, ten at a time, and say when there is more - #28

Merged
bkearns merged 4 commits into
mainfrom
feat/task-list-priority-paging
Aug 25, 2026
Merged

feat(tasks): answer "what's next" by priority, ten at a time, and say when there is more#28
bkearns merged 4 commits into
mainfrom
feat/task-list-priority-paging

Conversation

@bkearns

@bkearns bkearns commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Four commits: one on the task board's ordering, three on skill/corpus ingest.


1. feat(tasks) — answer "what's next" by priority

The problem

The board could only answer "what changed recently". sort_newest_first was
the only ordering, so an agent asking for the most relevant next work got the
most recent work — a P90 blocker filed yesterday lost to a P30 note filed
this morning.

Three changes

1. Priority ordering. sort_by_priority — priority desc, then newest, then
task_id.

The task_id tiebreak is not cosmetic. Paging splits one ordering across
several reads, so any pair left in an unspecified order can swap between calls,
and a task that swaps across a page boundary is shown twice or never.
sort_newest_first already documents that reasoning; priority ordering needed
the same guarantee.

2. Default page of 10, replacing the paged read's unwrap_or(100) and the
tool's advertised 50. The common question is "what should I do next", and the
answer is a shortlist — not fifty rows of tool-result tokens the caller will
never read.

3. A next-page hint the model can act on. page_hint(total, offset, returned)
adds a next_page field to the response:

Showing 1-10 of 47. 37 more — call again with offset=10 for the next page.

It returns None for a complete answer, so a finished result set does not
invite a pointless follow-up call.

Why the hint is the load-bearing part

This board already learned this once: a LIMIT that silently dropped recent
tasks left an agent unable to see its own write. The existing truncated flag
covers the MAX_FETCH_ROWS bound — but nothing covered the caller's own
page
, which is far more likely to be hit.

A truncated answer that does not announce itself reads as a complete one.

Note on the opposite call elsewhere

This deliberately goes the other way from ferrosa-memory#229, where the limit
was removed entirely. That is an internal work queue whose worker must
process everything, so a bound there only manufactured an ordering requirement
and a buffer to satisfy it.

A limit is right here precisely because this is a relevance-ranked,
agent-facing query and the caller is told there is more.

Closes t_8a5e3c05.


2. feat(ingest) — tell fmem where each entity came from

fmem derives an entity's TIER from where it came from, and forge was sending
nothing. Every corpus document and every skill therefore arrived with no origin
and was filed as raw capture — the same tier as session exhaust — however
curated the file behind it was.

Corpus and code entities now carry source_path, sent as attrs.source_path
by the graph loader. Skills carry it as a top-level field on ingest_skill,
which fmem validates against a known-key list, so both sides had to learn it
together.

Absolute and canonicalised, not relative. The tier rules match a root
through an alias table, and a relative path like corpus/x.md only resolves if
someone happened to register that exact spelling. The absolute path is what the
file actually is, and it matches the alias for whichever checkout it lives in.
Canonicalising means the same file tiers identically whatever directory the run
started from.

parse_from is split out of parse rather than changing it: the walker is the
only caller that knows the path, and every other caller has bytes alone. A
skill with no known path sends None rather than an empty string — fmem records
a source only when one is stated, and "" would file the skill under a root of
"".

Two tests hold the wire, because a regression here is silent: ingest still
succeeds, the skill is still searchable, and it simply sits in the wrong tier
for anyone deciding what to share.


3. fix(skill-ingest) — wait for fmem, and prefer the one already running

Two failures that both looked like a broken cluster and were neither.

Wait for ready. A cold fmem answers the MCP handshake immediately and
connects to its cluster in the background, so "the process started" is not "the
tools work". The run fired all 17 taxonomy edges into a server that was still
connecting and reported 17 failures against a cluster that was healthy. It now
probes with count_entities_by_type — a read that changes nothing — until the
server can serve. Only a "connection not yet established" answer is waited out;
any other error means the probe cannot tell us anything, so the run proceeds and
the real calls report their own failures. A readiness check that can veto on
errors it does not understand is a second failure mode, not a fix.

Prefer the running server. Spawning was unconditional, so on a machine with a
live fmem this started a second one against the same cluster. Both ran
consolidation, contended for the same partitions, and phase A timed out after
143 seconds. The corpus path has always used the configured HTTP endpoint;
skills now do the same, and spawning is the fallback. An explicit --server is
still honoured, because naming one is a deliberate choice.

The mock scripts learned the probe rather than the probe learning to skip mocks:
the wire genuinely changed, and a script that omits it is asserting the run
writes into a server it never checked.

Against the live store this took the skill catalog from 0 ingested to 95 of 98,
with the taxonomy complete.


4. fix(skill-ingest) — a reference outside the skill is kept, not fatal

Three of 98 skills failed the whole catalog over legitimate cross-references:
roadmap cites a sibling skill's conventions, and the c and rust skills
cite the corpus distillations they were built on.

Those are now recorded as references rather than inlined. The guard's
purpose is unchanged and the tests now assert the property that actually
matters rather than the error type: not one byte from outside the skill
directory is read, through ../ or through a symlink that looks local. What
changes is that keeping the pointer no longer costs the skill.

Not inlining is also the right answer for the corpus cases on its own terms:
pulling a corpus document into a skill would duplicate Information-tier text
into Wisdom, where it does not belong.

Whether a supplementary was inlined is part of the content hash. A file that
moves out of the skill directory stops contributing its content, and without
that marker the skill would read as unchanged while what it carries had shrunk.

A declared file that cannot be read stays fatal — that is a typo or a
deleted file, and passing it silently lets a skill lose half its content with
nobody noticing.

Live catalog: 98 of 98 ingested, 98 verified, 0 failed.


Tests

Board ordering: priority beats recency; ties fall back to newest; identical
priority+timestamp still yields a stable total order independent of input order;
the hint states the true total and exact next offset; the hint advances with the
offset; a complete, final, or empty result set has no hint; the default page is
ten.

Ingest: two tests hold the source_path wire; the run harness mocks carry
the readiness probe; the supplementary tests assert no bytes cross the skill
boundary via ../ or symlink while the reference itself survives.

cargo test --workspace green (86 test binaries + doc-tests, 0 failures);
cargo fmt --all -- --check and cargo clippy --workspace --all-targets -D warnings
clean.

Rebase note

This branch was rebased onto main after #27 was squash-merged — its commit is
tree-identical to the copy this branch carried, so the duplicate was dropped
rather than merged. A pair of duplicated #[test] attributes in
supplementary.rs (a clippy -D warnings failure) was folded into commit 4.

@bkearns
bkearns added this pull request to the merge queue Aug 24, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Aug 24, 2026
… when there's more

The board could only answer "what changed recently". `sort_newest_first` was
the single ordering, so an agent asking for the most relevant next work got
the most RECENT work: a P90 blocker filed yesterday lost to a P30 note filed
this morning.

Three changes, all on the read path an agent actually calls:

1. `sort_by_priority` — priority desc, then newest, then task_id. The
   task_id tiebreak is not cosmetic: paging splits one ordering across
   several reads, so any pair left in an unspecified order can swap between
   calls, and a task that swaps across a page boundary is shown twice or
   never. `sort_newest_first` already documents that reasoning; this needed
   the same guarantee.

2. `default_page_limit()` is 10, replacing the paged read's `unwrap_or(100)`
   and the tool's advertised 50. The common question is "what should I do
   next" and the answer to that is a shortlist, not fifty rows of
   tool-result tokens the caller will not read.

3. `page_hint(total, offset, returned)` and a `next_page` field on the
   response. When more matched than were returned the caller is told, in
   terms it can act on without guessing a parameter name:

       Showing 1-10 of 47. 37 more — call again with offset=10 for the next page.

   It returns None for a complete answer, so a finished result set does not
   invite a pointless follow-up call.

Point 3 is the load-bearing one, and it is the same principle this board
already learned once: a LIMIT that silently dropped recent tasks made an
agent unable to see its own write. The `truncated` flag covers the
MAX_FETCH_ROWS bound; nothing covered the caller's own page, which is far
more likely to be hit. A truncated answer that does not announce itself
reads as a complete one.

Note this is the opposite call from the consolidation queue, where the limit
was removed entirely (ferrosa-memory #229). That is an internal work queue
whose worker must process everything, so a bound only manufactured an
ordering requirement and a buffer. A limit is right HERE precisely because
this is a relevance-ranked, agent-facing query and the caller is told there
is more.

Tests: priority beats recency, ties fall back to newest, identical
priority+timestamp still yields a stable total order independent of input
order, the hint states the true total and exact next offset, the hint
advances with the offset, a complete or empty result set has no hint, and
the default page is ten.
fmem derives an entity's TIER from where it came from, and forge was sending
nothing. Every corpus document and every skill therefore arrived with no
origin and was filed as raw capture -- the same tier as session exhaust --
however curated the file behind it was.

Corpus and code entities now carry `source_path`, sent as `attrs.source_path`
by the graph loader. Skills carry it as a top-level field on ingest_skill,
which fmem validates against a known-key list, so both sides had to learn it
together.

ABSOLUTE and canonicalised, not relative. The tier rules match a root through
an alias table, and a relative path like `corpus/x.md` only resolves if
someone happened to register that exact spelling. The absolute path is what
the file actually is, and it matches the alias for whichever checkout it lives
in. Canonicalising means the same file tiers identically whatever directory
the run started from.

`parse_from` is split out of `parse` rather than changing it: the walker is
the only caller that knows the path, and every other caller has bytes alone. A
skill with no known path sends None rather than an empty string -- fmem
records a source only when one is stated, and "" would file the skill under a
root of "".

Two tests hold the wire, because a regression here is silent: ingest still
succeeds, the skill is still searchable, and it simply sits in the wrong tier
for anyone deciding what to share.
Two failures that both looked like a broken cluster and were neither.

WAIT FOR READY. A cold fmem answers the MCP handshake immediately and connects
to its cluster in the background, so "the process started" is not "the tools
work". The run fired all 17 taxonomy edges into a server that was still
connecting and reported 17 failures against a cluster that was healthy. It now
probes with count_entities_by_type -- a read that changes nothing -- until the
server can serve. Only a "connection not yet established" answer is waited
out; any other error means the probe cannot tell us anything, so the run
proceeds and the real calls report their own failures. A readiness check that
can veto on errors it does not understand is a second failure mode, not a fix.

PREFER THE RUNNING SERVER. Spawning was unconditional, so on a machine with a
live fmem this started a second one against the same cluster. Both ran
consolidation, contended for the same partitions, and phase A timed out after
143 seconds. The corpus path has always used the configured HTTP endpoint;
skills now do the same, and spawning is the fallback. An explicit --server is
still honoured, because naming one is a deliberate choice.

The mock scripts learned the probe rather than the probe learning to skip
mocks: the wire genuinely changed, and a script that omits it is asserting the
run writes into a server it never checked.

Against the live store this took the skill catalog from 0 ingested to 95 of
98, with the taxonomy complete.
Three of 98 skills failed the whole catalog over legitimate
cross-references: roadmap cites a sibling skill's conventions, and the c
and rust skills cite the corpus distillations they were built on.

Those are now RECORDED as references rather than inlined. The guard's
purpose is unchanged and the tests now assert the property that actually
matters rather than the error type: not one byte from outside the skill
directory is read, through `../` or through a symlink that looks local. What
changes is that keeping the pointer no longer costs the skill.

Not inlining is also the right answer for the corpus cases on its own terms:
pulling a corpus document into a skill would duplicate Information-tier text
into Wisdom, where it does not belong.

Whether a supplementary was inlined is part of the content hash. A file that
moves out of the skill directory stops contributing its content, and without
that marker the skill would read as unchanged while what it carries had
shrunk.

A declared file that cannot be READ stays fatal -- that is a typo or a
deleted file, and passing it silently lets a skill lose half its content
with nobody noticing.

Live catalog: 98 of 98 ingested, 98 verified, 0 failed.
@bkearns
bkearns force-pushed the feat/task-list-priority-paging branch from c3708d8 to 8c6bbb8 Compare August 25, 2026 05:24
@bkearns
bkearns added this pull request to the merge queue Aug 25, 2026
Merged via the queue into main with commit 8938fd0 Aug 25, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant