Add .agents/ instruction files for Chai Bot personas

[dustymabe]

Add instruction files for two Chai Bot (https://github.com/redhat-chai-bot) personas for the CoreOS team:

• coreos_pipeline: monitors the RHCOS Jenkins pipeline, triages build failures, and tracks issues in Jira (COS project). Serves #jenkins-rhcos-art.

• coreos_internal: general-purpose CoreOS engineering assistant covering RHCOS architecture, builds, CVEs, and team processes. Serves #dev-coreos and #forum-rhel-coreos.

Files are organized as:

• Persona domain instructions (always loaded by the persona)
• Scheduled task prompts (cron-driven pipeline monitoring)

These files will be consumed by ship-help-bot via %include() directives, following the same pattern used by HyperShift and ROSA teams.

Content adapted from existing triage workflows and domain knowledge in:

• https://github.com/cverna/coreos-agent-tools (main branch)
• https://github.com/suppathak/coreos-agent-tools (feature/pipeline-triage-workflow branch)

Assisted-By: <anthropic/claude-opus-4.6>

[dustymabe]

Most of the content here has been pulled from

• https://github.com/cverna/coreos-agent-tools (main branch)
• https://github.com/suppathak/coreos-agent-tools (feature/pipeline-triage-workflow branch)

Thanks @cverna and @suppathak for those great resources!

The content was adapted (using AI) to fit into a chai-bot persona for coreos-internal and coreos-pipeline.

Once this lands I'll open a chai-bot persona add request internally so we can get onboarded.

I'm sure we'll tweak these as we go and find things that need improving.

[cverna]

Suggested change

	- `x86_64` (AMD64/Intel) -- built by `build` job
	- `aarch64` (ARM 64-bit) -- built by `build-arch`
	- `ppc64le` (IBM POWER) -- built by `build-arch`
	- `s390x` (IBM Z) -- built by `build-arch`
	- `x86_64` -- built by `build` job
	- `aarch64`, `ppc64le`, `s390x` -- built by `build-arch`

[cverna]

I'll continue the review tomorrow but it might be interesting to pass the .md files through https://github.com/stbenjam/skillsaw. I think it might make the prompt more agent efficient and token efficient.

[dustymabe]

I'll continue the review tomorrow but it might be interesting to pass the .md files through https://github.com/stbenjam/skillsaw. I think it might make the prompt more agent efficient and token efficient.

I ran this against the repo and pushed the results here. Here's a summary of what if came up with:

.agents: fix skillsaw lint warnings in agent instruction files

    Run skillsaw (https://skillsaw.org/) v0.17.0 against the .agents/
    instruction files added in f6be12e. Skillsaw does not natively recognize
    the .agents/ directory layout (it expects .claude/, CLAUDE.md, AGENTS.md,
    or agentskills.io structures), so as a workaround each .agents/*.md file
    was copied into a temporary directory as CLAUDE.md and linted individually
    with --type dot-claude.

    This resolved 5 warnings across 5 files (all 12 files now grade A+):

    coreos_internal_role.md (content-tautological):
      "be thorough" is too vague for an LLM instruction. Replaced with
      concrete actions: cover background context, link to docs, define
      acronyms.

    coreos_pipeline_architecture.md (content-negative-only):
      "Does NOT automatically detect RHEL repo package updates" stated a
      prohibition without a positive alternative. Restructured to lead with
      the actionable workaround: trigger a build with FORCE=true.

    coreos_pipeline_failure_patterns.md (content-critical-position):
      A MUST directive at line 20 (flaky tests are not root cause) sat in
      the LLM attention dead zone (20-80% of file). Promoted the rule to
      the top of the file as a critical-rule callout and back-referenced it
      from the original location.

    scheduled_coreos_failure_monitor.md (content-weak-language):
      "to correctly identify" uses a vague adverb. Replaced with a concrete
      explanation of the ordering rationale: collect parent build numbers
      first so they are skipped during the build pass.

    scheduled_coreos_pipeline_health.md (content-negative-only):
      "Do not use <!channel> or broad mentions" stated only what to avoid.
      Rephrased to lead with the positive instruction: post without any
      @-mentions.

[cverna]

I had some specific instructions to download the kola logs, not sure how that would work with the Jenkins MCP, but it might useful to let the agent know it can get these logs too.
https://github.com/cverna/coreos-agent-tools/blob/9fb240812e5d5df9b2692dab90e7ebee6cd408cc/go/skills/pipeline-failures/SKILL.md?plain=1#L117

[dustymabe]

I've added a commit here to include this, but this exposed a large limitation of the jenkins MCP server, which is that it currently doesn't have a function to download the artifacts from a build (i.e. this is where we stash our kola tarballs).

Some WIP on closing the gap over in jenkinsci/mcp-server-plugin#42

I'm not sure if this means we'd need to change our approach. We'd probably be able to pull that PR forward if we needed it bad enough, which we probably do.

[cverna]

I'll continue the review tomorrow but it might be interesting to pass the .md files through https://github.com/stbenjam/skillsaw. I think it might make the prompt more agent efficient and token efficient.

I ran this against the repo and pushed the results here. Here's a summary of what if came up with:

.agents: fix skillsaw lint warnings in agent instruction files

    Run skillsaw (https://skillsaw.org/) v0.17.0 against the .agents/
    instruction files added in f6be12e. Skillsaw does not natively recognize
    the .agents/ directory layout (it expects .claude/, CLAUDE.md, AGENTS.md,
    or agentskills.io structures), so as a workaround each .agents/*.md file
    was copied into a temporary directory as CLAUDE.md and linted individually
    with --type dot-claude.

    This resolved 5 warnings across 5 files (all 12 files now grade A+):

    coreos_internal_role.md (content-tautological):
      "be thorough" is too vague for an LLM instruction. Replaced with
      concrete actions: cover background context, link to docs, define
      acronyms.

    coreos_pipeline_architecture.md (content-negative-only):
      "Does NOT automatically detect RHEL repo package updates" stated a
      prohibition without a positive alternative. Restructured to lead with
      the actionable workaround: trigger a build with FORCE=true.

    coreos_pipeline_failure_patterns.md (content-critical-position):
      A MUST directive at line 20 (flaky tests are not root cause) sat in
      the LLM attention dead zone (20-80% of file). Promoted the rule to
      the top of the file as a critical-rule callout and back-referenced it
      from the original location.

    scheduled_coreos_failure_monitor.md (content-weak-language):
      "to correctly identify" uses a vague adverb. Replaced with a concrete
      explanation of the ordering rationale: collect parent build numbers
      first so they are skipped during the build pass.

    scheduled_coreos_pipeline_health.md (content-negative-only):
      "Do not use <!channel> or broad mentions" stated only what to avoid.
      Rephrased to lead with the positive instruction: post without any
      @-mentions.

These looks like good improvements, WDYT?

[yasminvalim]

Same as before, maybe add that it needs to be clear about knowledge gaps and not create random answers if does not certain about the response.

[dustymabe]

I think this might be covered below with:

Be concise and evidence-based -- cite build numbers, log lines, and package versions

To me "evidence-based" means you need to have proof and the rest of the line gives examples of stating proof.

So I think maybe we're covered?

Do you have a specific suggestion (wording/edit) you'd like to make?

[yasminvalim]

I suggested a update in "Be concise and evidence-based -- cite build numbers, log lines, and package versions" just to clarify that it can't create data or extrapolate.

[yasminvalim]

Hey Dusty, thanks for working on this. I just added a comment about safeguards that we could use. I saw @cverna saying that the models are already smart to understand some stuff, but I am not sure how explicit we need to be with them.

And another idea would be having a small agents/README.md and add something like the PR description just to explain what it it and what it does in a simple way so people don't need to read all the the files to understand what the agents are doing.

Overall, LGTM. :)

[dustymabe]

@cverna
These looks like good improvements, WDYT?

yep. I actually uploaded the changes that implementing the recommendations at the same time I made that comment. I wanted to put the recommendations here in the comments for the record.

[dustymabe]

And another idea would be having a small agents/README.md and add something like the PR description

Done! .agents/README.md exists now

[yasminvalim]

LGTM!

[cverna]

[jbtrystram]

That's awesome ! I am looking forward to see that in action