feat(eval): add eval evaluator CLI commands (CRUD + LLaJ/code-based)

[jariy17]

What

Adds the imperative agentcore eval evaluator command surface (CLI only). Covers LLM-as-a-Judge and code-based evaluator create/update plus type-agnostic get/list/delete, following the DevX Evaluations/Optimization refactor doc and the existing identity handler conventions.

Command structure

agentcore eval                              # evaluate and optimize AgentCore agents
└── evaluator                               # manage AgentCore evaluators
    ├── llm-as-a-judge                       # LLM-as-a-Judge evaluators
    │   ├── create
    │   └── update
    ├── code-based                           # code-based (Lambda-backed) evaluators
    │   ├── create
    │   └── update
    ├── get                                  # get an evaluator by id (type-agnostic)
    ├── list                                 # list evaluators (client-side --type filter)
    └── delete                               # delete an evaluator by id (requires --yes)

Flags

eval evaluator llm-as-a-judge create

Flag	Required	Notes
--name	✓	evaluator name
--level	✓	SESSION | TRACE | TOOL_CALL
--model	✓	Bedrock model id used to judge
--instructions	✓	source-aware: inline, file://<path>, or - (stdin)
--rating-scale	✓*	preset: 1-5-quality | 1-3-simple | pass-fail | good-neutral-bad
--rating-scale-json	✓*	raw RatingScale JSON (source-aware). Mutually exclusive with --rating-scale; exactly one required
--kms-key-arn		customer-managed KMS key ARN
--tags		JSON object (source-aware)
--client-token		idempotency token

eval evaluator llm-as-a-judge update

Flag	Required	Notes
--id	✓	evaluator id
--instructions		source-aware
--model
--rating-scale / --rating-scale-json		mutually exclusive
--kms-key-arn
--client-token

Fields left unset are preserved: UpdateEvaluator replaces the whole evaluatorConfig union, so the client does get-then-merge over the current definition.

eval evaluator code-based create

Flag	Required	Notes
--name	✓	evaluator name
--level	✓	SESSION | TRACE | TOOL_CALL
--lambda-arn	✓	Lambda that scores a session
--timeout		seconds (1–300). No CLI default; the service applies its own (60s)
--kms-key-arn
--tags		JSON object (source-aware)
--client-token

eval evaluator code-based update

Flag	Required	Notes
--id	✓	evaluator id
--lambda-arn
--timeout
--kms-key-arn
--client-token

eval evaluator get

Flag	Required	Notes
--id	✓	evaluator id

eval evaluator list

Flag	Required	Notes
--next-token		pagination token (server-side)
--max-results		max items (server-side)
--type		filters the returned page client-side: Builtin | code-based | llm-as-a-judge. The ListEvaluators API paginates only, so the filter is page-local

eval evaluator delete

Flag	Required	Notes
--id	✓	evaluator id
--yes	✓	confirm deletion (required in this headless/JSON-only path)

Notes / decisions

• Source-aware field values (--instructions, --rating-scale-json, --tags): inline, file://<path>, or - for stdin, following the AWS CLI file:// convention. One flag reads stdin per command.
• Rating scale offers presets (the common defaults) and a raw-JSON escape hatch (what the API supports directly).
• --timeout has no CLI default; the service default applies when omitted.
• Dependency inversion: CoreEvalClient is declared next to the handlers (src/handlers/eval/types.tsx) and implemented in src/core/eval.tsx.

Testing

• bun test — full suite green (356 tests).
• bun run typecheck, bun run lint:check, bun run format:check — clean.
• bun run build + bundle smoke (node dist/index.js eval evaluator --help) — works.
• New tests: handler behavior + validation/source/list-filter via TestCoreClient, EvalClient get-then-merge unit tests, and source-resolver tests.

---

TUI flows for these commands will come in a later PR.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 88.48614% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.45%. Comparing base (272e50e) to head (02e4def).

Files with missing lines	Patch %	Lines
src/core/eval.tsx	72.27%	28 Missing ⚠️
...c/handlers/eval/evaluator/llm-as-a-judge/index.tsx	87.15%	14 Missing ⚠️
src/handlers/eval/evaluator/code-based/index.tsx	87.34%	10 Missing ⚠️
src/handlers/eval/evaluator/get/index.tsx	86.66%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           refactor    #1822      +/-   ##
============================================
- Coverage     94.84%   94.45%   -0.40%     
============================================
  Files           143      153      +10     
  Lines          7120     7589     +469     
============================================
+ Hits           6753     7168     +415     
- Misses          367      421      +54     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.