Skip to content

Add AGENTS.md with observability guidance - #253

Merged
Peeja merged 9 commits into
mainfrom
claude/happy-bell-cfh5qb-agents-observability
Oct 16, 2026
Merged

Peeja merged 9 commits into
mainfrom
claude/happy-bell-cfh5qb-agents-observability

Conversation

@Peeja

@Peeja Peeja commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Written by Claude.

Depends on #248. The new section points to the severity definitions that PR adds, so merge it first.

This adds a short Observability section to AGENTS.md. When a change adds or alters something operational, the section tells an agent to decide how on-call would see it fail and to add what's useful in the same PR: an alert, a dashboard panel, or a line in docs/observability.md. It also says to skip this for changes nothing operational depends on.

The section is kept short because AGENTS.md loads into every session. That follows bajtos's review of fil-forge/infra-nodes#174. The how-to lives in a new "Adding an alert or panel" section of docs/observability.md. It lists the labels each rule needs (team_name, component, a severity chosen by the definitions #248 adds, stage, notification_settings), notes that check-alert-routing.sh enforces routing and that new rules go last because uids follow position, describes dashboard JSON with normalise-dashboard.sh, and says application metrics don't ship yet.

This PR no longer adds a CLAUDE.md. Claude Code reads AGENTS.md itself from v2.1.277 (docs).

Companion PRs: fil-forge/piri#174, fil-forge/infra-nodes#174 and the Forge service repos.

Checks

  • make check, git diff --check.

✴️

🤖 Generated with Claude Code

https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ


Generated by Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Petra Jaros <petra@fil.org>
Claude-Session: https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ
Copilot AI balanced review requested due to automatic review settings October 15, 2026 16:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Petra Jaros <petra@fil.org>
Claude-Session: https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ

Peeja commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Claude reviewer says… (1st round, 0a1e717)

Approve with nits; nothing blocking or should-fix.

Verified

  • Required labels: all 25 rules carry team_name = "forge", a severity and policy = local.forge_policy.
  • Routing check: check-alert-routing.sh runs on the Grafana plan (check-and-deploy.yml:529).
  • Dashboard normalisation: make check runs normalise-dashboard.sh --check.
  • Panel links: dashboard_url, __dashboardUid__ and __panelId__ are used on 12-13 rules.
  • Application metrics: "Not shipped. Application metrics" is at observability.md:21.
  • Severity definitions: they come from Define alert severities in the observability docs #248, and the body says so.
  • Checks: make check passes and git diff --check is clean.

Findings

  1. nit, required labels.
    • Every rule also has a component label.
    • "The rule group for the component" is ambiguous, because the appliance has three groups.
    • stage comes from the query on Central rules but from local.appliance_stage_label on appliance rules.
  2. nit, uid-by-position. The in-repo comment backing this is only on open Make alert severities match their definitions #249; the body names only Define alert severities in the observability docs #248 as a prerequisite.
  3. nit, dashboards. "Edit in the UI, export" is a human step, and observability.md says UI editing in that folder is untested.
  4. nit, references before Define alert severities in the observability docs #248. Until Define alert severities in the observability docs #248 merges, the severity line points at definitions that don't exist yet.

✴️


Generated by Claude Code

Peeja commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Claude author says…

Fixed in 6659a79.

  1. Labels and groups.
    • The rule list now names component (central, appliance or usage).
    • It says to pick the group "whose evaluation interval and no-data handling fit; the appliance has three".
    • It says where stage comes from. On a Central rule it's the query's labels, a label template over them, or a literal for a one-stage rule. I checked: the Lambda rule uses a template and the VPN rules use "prod". Appliance rules use local.appliance_stage_label.
  2. Prerequisites. The body now names Make alert severities match their definitions #249 alongside Define alert severities in the observability docs #248.
  3. Dashboards. The step now reads "Edit the JSON, or export it from the UI, then run scripts/normalise-dashboard.sh".
  4. Merge order. Left to merge order, which the body states. I haven't changed the PR's draft state.

I also added piri's "Skip it for changes nothing operational depends on" line.

make check passes and git diff --check is clean. This round found only nits, so there won't be another.

✴️


Generated by Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Petra Jaros <petra@fil.org>
Claude-Session: https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ
claude and others added 2 commits October 16, 2026 16:31
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Petra Jaros <petra@fil.org>
Claude-Session: https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Petra Jaros <petra@fil.org>
Claude-Session: https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ

Peeja commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Claude says… plan-prod-apps failed on c3f5f79, the merge of main into this branch. This PR adds only AGENTS.md and CLAUDE.md, so nothing in it reaches that plan. The same job passed on #228, #248 and #249 against the same main, and #248's plan-dev-apps failed the same way earlier today, then passed. I can't read the job log from this session, so the cause is unknown. Re-running once.

✴️


Generated by Claude Code

Following bajtos's review of fil-forge/infra-nodes#174: the guidance
file loads into every session, so its Observability section is now a
short reminder of what to add and when to skip it, with a pointer to
docs/observability.md, which holds the detail. A CLAUDE.md that only
imported AGENTS.md is gone where this PR added one: Claude Code reads
AGENTS.md itself from v2.1.277.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Petra Jaros <petra@fil.org>
Claude-Session: https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ

Peeja commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Claude reviewer says… (round on the restructure, 0bc8bb7)

This round reviewed all eight guidance PRs together; findings below are this PR's.

Approve. Only nits here, and none of the eight PRs has anything blocking or should-fix.

Verified, across all eight PRs

  • git diff --check origin/main...HEAD is clean in each. In infra-central, make check passes at 0bc8bb7.
  • Nothing was lost in the move. Each repo's previous section matches the new doc line by line, and every accurate sentence survives.
  • Each short section still says when to act, what to add and when to skip. Each pointer names a heading that exists: "Adding a signal" in piri, ingot, delegator, swarf and piri-signing-service, and "Adding an alert or panel" on Add AGENTS.md with observability guidance #253's branch.
  • Only infra-central and swarf drop a CLAUDE.md, and both had been added by the PR itself. Every other repo's existing CLAUDE.md is untouched.
  • The cited paths, headings and links resolve. Each description matches its diff.

Findings for this PR

  1. nit (PR body): the doc's "severity definitions above" (docs/observability.md:128) exist only once Define alert severities in the observability docs #248 merges. A trial merge of b400c5b puts them at L87–91, above the new section. The dependency was stated, but only as the last bullet under Checks.
  2. nit (PR body): the "Companion PRs" line lists this PR.

✴️


Generated by Claude Code

Peeja commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Claude author says…

Both fixes are in the description; there's no code change, so the head stays 0bc8bb7.

  1. A "Depends on Define alert severities in the observability docs #248" line now opens the description, and the Checks bullet no longer repeats it.
  2. The self-reference is gone.

This round found only nits, so there won't be another.

✴️


Generated by Claude Code

claude and others added 2 commits October 16, 2026 20:54
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Petra Jaros <petra@fil.org>
Claude-Session: https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Petra Jaros <petra@fil.org>
Claude-Session: https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ
Peeja added a commit to fil-forge/piri that referenced this pull request Oct 16, 2026
…done (#174)

*Written by Claude.*

This adds a short Observability section to `AGENTS.md`. When a change
adds or alters something operational, the section tells an agent to
decide how on-call would see it fail and to add what's useful in the
same PR: OpenTelemetry metrics or traces, entries in Piri's telemetry
and monitoring docs, and an alert or dashboard panel in
fil-forge/infra-central. It also says to skip this for changes nothing
operational depends on.

The section is kept short because `AGENTS.md` loads into every session.
That follows bajtos's review of fil-forge/infra-nodes#174. The how-to
lives in a new `docs/observability.md`, under "Adding a signal". It sits
outside `docs/content/`, so it isn't published on the operator site. It
covers where to instrument (`telemetry.go`, `*_metrics.go`), keeping
attributes low-cardinality, which docs pages to update
(`concepts/telemetry.md`, and `operator-guide/monitoring.md`'s Key
Metrics, Alerts to Configure and PromQL block), and the infra-central
companion PR.

The repo's existing `CLAUDE.md` already imports `AGENTS.md`, so it's
unchanged.

Companion PRs: fil-forge/infra-central#253, fil-forge/infra-nodes#174
and the Forge service repos.

## Checks

- `git diff --check`. `mkdocs build --strict` wasn't run here; the new
file is outside `docs_dir`.


[✴️](https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ


---
_Generated by [Claude Code](https://claude.ai/code)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
@Peeja
Peeja enabled auto-merge (squash) October 16, 2026 21:35
Peeja added a commit to fil-forge/hilt that referenced this pull request Oct 16, 2026
…done (#98)

*Written by Claude.*

**Merge after fil-forge/infra-central#253.** This PR's
`docs/observability.md` points to the "Adding an alert or panel" section
that PR adds.

This adds a short Observability section to `AGENTS.md`. When a change
adds or alters something operational, the section tells an agent to
decide how on-call would see it fail and to add what's useful in the
same PR: logs, complete trace spans, and an alert or dashboard panel in
fil-forge/infra-central. It also says that when a feature's health is
best read as a number, the PR description should name the metric and
[FIL-1143](https://linear.app/filecoin-foundation/issue/FIL-1143). It
also says to skip this for changes nothing operational depends on.

The section is kept short because `AGENTS.md` loads into every session.
That follows bajtos's review of fil-forge/infra-nodes#174. The how-to
lives in a new `docs/observability.md`, with Logs (only CloudWatch logs
and AWS metrics reach Grafana from Central today), Metrics, Traces
(`tracing.NewHTTPClient()` in `pkg/tracing`) and Alerts and dashboards,
which points to infra-central's "Adding an alert or panel".

The repo's existing `CLAUDE.md` already imports `AGENTS.md`, so it's
unchanged.

Companion PRs: fil-forge/piri#174, fil-forge/infra-central#253,
fil-forge/infra-nodes#174 and the Forge service repos.

## Checks

- `git diff --check`.


[✴️](https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ


---
_Generated by [Claude Code](https://claude.ai/code)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Peeja added a commit to fil-forge/sprue that referenced this pull request Oct 16, 2026
…done (#140)

*Written by Claude.*

**Merge after fil-forge/infra-central#253.** This PR's
`docs/observability.md` points to the "Adding an alert or panel" section
that PR adds.

This adds a short Observability section to `AGENTS.md`. When a change
adds or alters something operational, the section tells an agent to
decide how on-call would see it fail and to add what's useful in the
same PR: logs, complete trace spans, and an alert or dashboard panel in
fil-forge/infra-central. It also says that when a feature's health is
best read as a number, the PR description should name the metric and
[FIL-1144](https://linear.app/filecoin-foundation/issue/FIL-1144). It
also says to skip this for changes nothing operational depends on.

The section is kept short because `AGENTS.md` loads into every session.
That follows bajtos's review of fil-forge/infra-nodes#174. The how-to
lives in a new `docs/observability.md`, with Logs, Metrics, Traces
(`tracing.NewHTTPClient()` in `internal/tracing`) and Alerts and
dashboards, which points to infra-central's "Adding an alert or panel".

The repo's existing `CLAUDE.md` already imports `AGENTS.md`, so it's
unchanged.

Companion PRs: fil-forge/piri#174, fil-forge/infra-central#253,
fil-forge/infra-nodes#174 and the Forge service repos.

## Checks

- `git diff --check`.


[✴️](https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ


---
_Generated by [Claude Code](https://claude.ai/code)_
Peeja added a commit to fil-forge/ingot that referenced this pull request Oct 16, 2026
…done (#266)

*Written by Claude.*

**Merge after fil-forge/infra-central#253.** This PR's
`docs/observability.md` points to the "Adding an alert or panel" section
that PR adds.

This adds a short Observability section to `CLAUDE.md`. When a change
adds or alters something operational, the section tells an agent to
decide how on-call would see it fail and to add what's useful in the
same PR: metrics, trace spans, and an alert or dashboard panel in
fil-forge/infra-central. It keeps the instruction to list each new
metric in a README metrics table. It also says to skip this for changes
nothing operational depends on.

The section is kept short because `CLAUDE.md` loads into every session.
That follows bajtos's review of fil-forge/infra-nodes#174. The how-to
lives in a new `docs/observability.md`. Where signals go: OTLP to Alloy,
with container logs shipping too. Adding a signal: an injectable
`MeterProvider` as in `s3frontend.Deps.MeterProvider` and
`Layered.CountBlobReads`, low-cardinality attributes, trace spans, and
alerts in infra-central. It links to the README's OTLP variables and
Local disk metrics table rather than repeating them.

`CLAUDE.md` is Ingot's full guidance file and stays; only its
Observability section changed.

Companion PRs: fil-forge/piri#174, fil-forge/infra-central#253,
fil-forge/infra-nodes#174 and the Forge service repos.

## Checks

- `git diff --check`.


[✴️](https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ


---
_Generated by [Claude Code](https://claude.ai/code)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Peeja added a commit to fil-forge/delegator that referenced this pull request Oct 16, 2026
…done (#61)

*Written by Claude.*

**Merge after fil-forge/infra-central#253.** This PR's
`docs/observability.md` points to the "Adding an alert or panel" section
that PR adds.

This adds a short Observability section to `AGENTS.md`. When a change
adds or alters something operational, the section tells an agent to
decide how on-call would see it fail and to add what's useful in the
same PR: logs, and an alert or dashboard panel in
fil-forge/infra-central. It also says that when a feature's health is
best read as a number, the PR description should name the metric and
[FIL-1152](https://linear.app/filecoin-foundation/issue/FIL-1152). It
also says to skip this for changes nothing operational depends on.

The section is kept short because `AGENTS.md` loads into every session.
That follows bajtos's review of fil-forge/infra-nodes#174. The how-to
lives in a new `docs/observability.md`: what reaches Grafana from
Central (logs, and AWS metrics for the load balancer, ECS and its
DynamoDB tables), how to log a failure on-call can act on, the metrics
note, and a pointer to infra-central's "Adding an alert or panel".

The repo's existing `CLAUDE.md` already imports `AGENTS.md`, so it's
unchanged.

Companion PRs: fil-forge/piri#174, fil-forge/infra-central#253,
fil-forge/infra-nodes#174 and the Forge service repos.

## Checks

- `git diff --check`.


[✴️](https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ


---
_Generated by [Claude Code](https://claude.ai/code)_
Peeja added a commit to fil-forge/swarf that referenced this pull request Oct 16, 2026
*Written by Claude.*

**Merge after fil-forge/infra-central#253.** This PR's
`docs/observability.md` points to the "Adding an alert or panel" section
that PR adds.

This adds a short Observability section to `AGENTS.md`. When a change
adds or alters something operational, the section tells an agent to
decide how on-call would see it fail and to add what's useful in the
same PR: logs, and an alert or dashboard panel in
fil-forge/infra-central. It also says that when a feature's health is
best read as a number, the PR description should name the metric and
[FIL-1152](https://linear.app/filecoin-foundation/issue/FIL-1152). It
also says to skip this for changes nothing operational depends on.

The section is kept short because `AGENTS.md` loads into every session.
That follows bajtos's review of fil-forge/infra-nodes#174. The how-to
lives in a new `docs/observability.md`: what reaches Grafana from
Central (logs, and AWS metrics for the load balancer, ECS and RDS), how
to log a failure on-call can act on, the metrics note, and a pointer to
infra-central's "Adding an alert or panel".

This PR no longer adds a `CLAUDE.md`. Claude Code reads `AGENTS.md`
itself from v2.1.277
([docs](https://code.claude.com/docs/en/memory#agents-md)).

Companion PRs: fil-forge/piri#174, fil-forge/infra-central#253,
fil-forge/infra-nodes#174 and the Forge service repos.

## Checks

- `git diff --check`.


[✴️](https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ


---
_Generated by [Claude Code](https://claude.ai/code)_
Peeja added a commit to fil-forge/piri-signing-service that referenced this pull request Oct 16, 2026
…done (#22)

*Written by Claude.*

**Merge after fil-forge/infra-central#253.** This PR's
`docs/observability.md` points to the "Adding an alert or panel" section
that PR adds.

This adds a short Observability section to `AGENTS.md`. When a change
adds or alters something operational, the section tells an agent to
decide how on-call would see it fail and to add what's useful in the
same PR: logs, and an alert or dashboard panel in
fil-forge/infra-central. It also says that when a feature's health is
best read as a number, the PR description should name the metric and
[FIL-1152](https://linear.app/filecoin-foundation/issue/FIL-1152). It
also says to skip this for changes nothing operational depends on.

The section is kept short because `AGENTS.md` loads into every session.
That follows bajtos's review of fil-forge/infra-nodes#174. The how-to
lives in a new `docs/observability.md`: what reaches Grafana from
Central (logs, and AWS metrics for the load balancer and ECS), how to
log a failure on-call can act on, the metrics note, and a pointer to
infra-central's "Adding an alert or panel".

The repo's existing `CLAUDE.md` already imports `AGENTS.md`, so it's
unchanged.

Companion PRs: fil-forge/piri#174, fil-forge/infra-central#253,
fil-forge/infra-nodes#174 and the Forge service repos.

## Checks

- `git diff --check`.


[✴️](https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01JCF8gyb8jNRmHmWqQmVhRJ


---
_Generated by [Claude Code](https://claude.ai/code)_
@Peeja
Peeja merged commit 1a343cd into main Oct 16, 2026
25 checks passed
@Peeja
Peeja deleted the claude/happy-bell-cfh5qb-agents-observability branch October 16, 2026 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants