[GSoC] Add failure-investigation accessors to the core client (stacked on #3348) - #3349
Draft
Aryan95614 wants to merge 6 commits into
Draft
[GSoC] Add failure-investigation accessors to the core client (stacked on #3348)#3349Aryan95614 wants to merge 6 commits into
Aryan95614 wants to merge 6 commits into
Conversation
…explicit kwargs
- Gate the cursor-paginated path on metadata-service version (>= 2.5.1) with an
X-Limit capability-header check; fall back to legacy listing for older
services and reject server-side filters when the service cannot honor them.
- Generalize pagination to all collection types, not just flow/run listings.
- Sort iter_objects results newest-first by ts_epoch so max_runs returns the
newest runs regardless of the provider's ordering.
- Replace Flow.runs(**kwargs) with explicit keyword-only filters/page_size/
max_runs, and apply positional tags locally so flow.runs("prod") works with
local metadata as well as the service.
- Extract collection listing into private helpers for readability.
- Use mocks instead of fake providers in the run-listing tests; add coverage
for legacy fallback, capability checks, and newest-first ordering.
Follow-up on the paginated listing path: - Explain why _get_object_internal materializes pages into a list: get_object's object-or-list contract must stay stable, the goal here is to relieve server-side pressure, and callers needing to stream large collections go through iter_objects()/_iter_paginated_records (e.g. Flow.runs()). - Document that result ordering is implicitly newest-first (descending ts_epoch) because there is no _order query param yet, in both the service iterator and the base MetadataProvider.iter_objects sort, so the two paths stay consistent.
…tion - Bump _MIN_SERVICE_VERSION_WITH_CURSOR_PAGINATION to 2.6.0 -- the release that ships pagination + filtering (per maintainer). - The paginated get_object path returned [] where the legacy path returned None for a missing (404) collection. Add a raise_on_missing flag threaded through _iter_paginated_records and _legacy_get_collection so get_object keeps legacy's atomic contract: a 404 at any point in the listing (first page, mid-pagination, or the no-X-Limit legacy fallback) resolves to None, never an empty or silently truncated list. Streaming iter_objects is unchanged: a 404 just ends the stream. - Lift get_object's obj_type/sub_type validation guards into a shared _validate_object_query helper and call it from the paginated listing path, so streamed access rejects the same nonsensical combinations as materialized access. - Cover all of the above with tests.
Replaces the separate metaflow/agent module with core-client accessors, per
the review that record iteration and normal client operations belong in core:
- Flow.failed_runs(*, since=None, max_runs=None): iterate failed runs newest
first via runs(filters={"status:eq": "failed"}); raises on local/older
services like any other server-filtered listing.
- Run.failed_task: the first unsuccessful task in the run (newest step first).
- Task.failure_summary -> FailureSummary: normalized exception type/message/
stacktrace/attempt, or None when the task has no exception. Read errors
propagate rather than being swallowed into diagnostic strings.
Drops the injectable resolver/fetcher and the TaskRef indirection (uses real
Task objects). Adds test/unit/test_client_failure_investigation.py.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3349 +/- ##
=========================================
Coverage ? 31.14%
=========================================
Files ? 381
Lines ? 52869
Branches ? 9348
=========================================
Hits ? 16468
Misses ? 35170
Partials ? 1231 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GSoC 2026 project: metadata service request improvements (mentor: @saikonen). Upstream twin of the fork review PR (Aryan95614#2), opened per mentor request for a static PR number.
Stacked on #3348 — until that merges, this diff includes its commits; only the top two commits (
Fold failure investigation into the core client+ the docstring fix) are this PR's own.Summary
Folds bounded failure investigation into the regular Metaflow client as three
accessors on the existing objects, instead of a separate
metaflow/agentmodule. Built on the core pagination/filtering from PR #1.
Context / Motivation
Review on the earlier revision steered the run-listing and pagination pieces
into the core client (that's PR #1). Following the same direction, the failure
investigation layer now lives on the client objects rather than a parallel
metaflow.agentpackage, so there's a single, discoverable API.Changes Made
Flow.failed_runs(*, since=None, max_runs=None) -> Iterator[Run]— failedruns newest-first, via
runs(filters={"status:eq": "failed"});sinceaddsan inclusive
ts_epoch:gebound.Run.failed_task -> Optional[Task]— the first unsuccessful task in the run(newest step first), or None.
Task.failure_summary -> Optional[FailureSummary]— normalized{exception_type, message, stacktrace, attempt}, or None when the task hasno exception. Read errors propagate rather than being swallowed.
metaflow/agentmodule (failure_finder,pagination,run_lister,TaskRef/FailureInfo, injectable resolver/fetcher).Behavior note:
failed_runsrelies on server-side filtering, so it raisesagainst the local metadata provider or a service without pagination/filtering
support — the same gate as
runs(filters=...).Testing
test/unit/test_client_failure_investigation.py— 14 tests coveringfailed_runsfilter/bounds forwarding,failed_taskiteration order,failure_summaryconstruction + error propagation, and_normalize_exceptionacross mapping/object/string inputs.
black(repo-pinned 25.12.0) clean.Trade-offs / Design Decisions
task_resolver/artifact_fetcherand theTaskRefindirection — unnecessary once this is core; real
Taskobjects are used andthe methods are testable with mocks.
FailureSummaryas a small frozen dataclass incore.pyrather thanreturning a bare dict, so the normalized fields are typed and discoverable.