You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Issue #9 — Live BTP integration via BTP_SERVICE_KEY
Task
Add support for running the benchmark against a real BTP ABAP Cloud
system by accepting a BTP_SERVICE_KEY env var or --service-key
CLI flag. The runner converts the service key into the ABAP_URL / ABAP_USER / ABAP_PASSWORD env vars that the
configured MCP server expects. Add a pre-flight reachability check
and an explicit --allow-live-system safety flag.
agents/mastracode/profiles/live-arc-1.mcp.json — existing live profile
Background: the v1 design mandates that v1 is deterministic and
CI-friendly. v1.1 must add a real-system path that is opt-in and never runs in CI unless explicitly enabled.
Out of scope
Persisting credentials. The service key is read once per run and
discarded.
BTP-specific OAuth flows beyond the service-key format. The oauth2 and x509 profiles are future work.
Files to add or modify
packages/bench-cli/src/cli.ts — add --service-key flag handling.
docs/reproducing.md — new section "Live BTP runs".
Steps
Implement parseServiceKey(json: string) that:
Parses the JSON.
Extracts uaa.url, uaa.clientid, uaa.clientsecret, the abap endpoint, and the identityzone / tenantid.
Returns { url, clientid, clientsecret, abap_url, abap_user, abap_password }
where abap_user and abap_password are derived from a
service-key OAuth flow (simplest: client credentials grant
against uaa.url/oauth/token with grant_type=client_credentials).
Implement checkReachable(url) that does a GET /sap/bc/adt/discovery
with the credentials and expects 200.
Add --service-key <path|-> to bench:run (read from file or stdin).
Add --allow-live-system flag. Without it, the runner refuses to
proceed if a live profile is selected.
Add an env-var fallback: if BTP_SERVICE_KEY is set and --allow-live-system is true, use it.
pnpm bench:run --profile live-arc-1 --service-key ./key.json --allow-live-system --scenario create-class-hello runs end-to-end
against a real system (when one is configured).
docs/reproducing.md documents the live-system path.
Test plan
Unit: btp.spec.ts > parseServiceKey with a sample BTP service key.
Integration: manual run against a real system (out of scope for CI).
Acceptance gate
pnpm verify exits 0.
Live runs require both --allow-live-system and a service key.
The CI workflow never sees BTP_SERVICE_KEY.
Definition of done
parseServiceKey and checkReachable implemented and unit-tested.
--service-key and --allow-live-system flags work.
docs/reproducing.md updated.
pnpm verify exits 0.
Conventional commit: feat(cli): live BTP integration via BTP_SERVICE_KEY.
PR opened; this issue closed.
Dependencies
Blocked by #1 (real runner needed to actually exercise the live
system).
Issue #9 — Live BTP integration via BTP_SERVICE_KEY
Task
Add support for running the benchmark against a real BTP ABAP Cloud
system by accepting a
BTP_SERVICE_KEYenv var or--service-keyCLI flag. The runner converts the service key into the
ABAP_URL/ABAP_USER/ABAP_PASSWORDenv vars that theconfigured MCP server expects. Add a pre-flight reachability check
and an explicit
--allow-live-systemsafety flag.Context
docs/design.md— safety defaultspackages/bench-cli/src/run.ts— current CLIpackages/bench-cli/src/cli.ts—executeRunagents/mastracode/profiles/live-arc-1.mcp.json— existing live profileCI-friendly. v1.1 must add a real-system path that is opt-in and
never runs in CI unless explicitly enabled.
Out of scope
discarded.
oauth2andx509profiles are future work.Files to add or modify
packages/bench-cli/src/cli.ts— add--service-keyflag handling.packages/bench-cli/src/run.ts— same.packages/bench-cli/src/btp.ts(new) —parseServiceKey(json),checkReachable(url, user, password).packages/bench-cli/src/btp.spec.ts(new)packages/bench-cli/specs/SPEC.md— update.docs/reproducing.md— new section "Live BTP runs".Steps
parseServiceKey(json: string)that:uaa.url,uaa.clientid,uaa.clientsecret, theabapendpoint, and theidentityzone/tenantid.{ url, clientid, clientsecret, abap_url, abap_user, abap_password }where
abap_userandabap_passwordare derived from aservice-key OAuth flow (simplest: client credentials grant
against
uaa.url/oauth/tokenwithgrant_type=client_credentials).checkReachable(url)that does aGET /sap/bc/adt/discoverywith the credentials and expects 200.
--service-key <path|->tobench:run(read from file or stdin).--allow-live-systemflag. Without it, the runner refuses toproceed if a live profile is selected.
BTP_SERVICE_KEYis set and--allow-live-systemis true, use it.BTP_SERVICE_KEY. The workflow file from infra: add GitHub Actions CI workflow #6does not pass it.
Deliverables
parseServiceKeyandcheckReachableunit-tested.pnpm bench:run --profile live-arc-1 --service-key ./key.json --allow-live-system --scenario create-class-helloruns end-to-endagainst a real system (when one is configured).
docs/reproducing.mddocuments the live-system path.Test plan
btp.spec.ts > parseServiceKeywith a sample BTP service key.Acceptance gate
pnpm verifyexits 0.--allow-live-systemand a service key.BTP_SERVICE_KEY.Definition of done
parseServiceKeyandcheckReachableimplemented and unit-tested.--service-keyand--allow-live-systemflags work.docs/reproducing.mdupdated.pnpm verifyexits 0.feat(cli): live BTP integration via BTP_SERVICE_KEY.Dependencies
Blocked by #1 (real runner needed to actually exercise the live
system).