[BUG](fn-consumer): Skip stale work fetch - #7523
Conversation
Reviewer ChecklistPlease leverage this checklist to ensure your code review is thorough before approving Testing, Bugs, Errors, Logs, Documentation
System Compatibility
Quality
|
e6c268b to
773e009
Compare
773e009 to
d02a95c
Compare
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d02a95ce40
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ) -> Result<i64, CompactionError> { | ||
| let mut sysdb = compaction_context.sysdb.clone(); | ||
| let attached_function = sysdb | ||
| .get_attached_functions(None, Some(collection_id), vec![attached_function_id], true) |
There was a problem hiding this comment.
Avoid the single-ID SysDB request that always errors
Every fn-consumer batch now reaches this lookup before fetching logs, but GrpcSysDb::get_attached_functions encodes a one-element ids vector into both the deprecated id field and the ids field (rust/sysdb/src/sysdb.rs:2451-2464). The coordinator explicitly rejects requests containing both fields (go/pkg/sysdb/coordinator/task.go:928-930), so in the production gRPC SysDB configuration this returns InvalidArgument for every work item and prevents all async attached-function execution. Use a lookup that populates only one of those fields, or correct the helper before calling it here.
Useful? React with 👍 / 👎.
Summary
Avoid materializing logs for stale async attached-function queue records, and explicitly acknowledge records that are already complete at the current collection head.
Root cause
fn-consumerfetched and materialized every input collection before comparing the attached-function completion frontier with the queued compaction frontier. A stale input could therefore load millions of records and then be discarded.Change
FinishWorkdirectly. This advances/records the invocation frontier and removes the queue item without boundary resolution or function execution.Validation
cargo test -p worker function_execution --lib