Skip to content

Add it for AI-Workspace CLI#2672

Open
ShakyaPr wants to merge 6 commits into
wso2:mainfrom
ShakyaPr:tests/it_ai_ws_cli
Open

Add it for AI-Workspace CLI#2672
ShakyaPr wants to merge 6 commits into
wso2:mainfrom
ShakyaPr:tests/it_ai_ws_cli

Conversation

@ShakyaPr

Copy link
Copy Markdown
Contributor

Purpose

@github-actions

Copy link
Copy Markdown
Contributor

Dependency Validation Results

Dependency name: github.com/cucumber/godog
Version: v0.15.0
Allowed range: >=v0.15.0
Approved: ✅ Yes

⚠️ Please verify the scope of the dependencies usage is necessary

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: fe99b3f4-16f0-4ee6-a7f5-faa27faba9df

📥 Commits

Reviewing files that changed from the base of the PR and between 90cf0d5 and 2e68a72.

📒 Files selected for processing (5)
  • tests/ai-workspace-cli-e2e/resources/llm-provider/definition.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-provider/edit/runtime.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-proxy/create/runtime.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-proxy/definition.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-proxy/edit/runtime.yaml
🚧 Files skipped from review as they are similar to previous changes (3)
  • tests/ai-workspace-cli-e2e/resources/llm-provider/edit/runtime.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-proxy/definition.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-provider/definition.yaml

📝 Walkthrough

Walkthrough

Adds a daily/manual Docker-backed Godog E2E suite for AI Workspace CLI artifact creation and updates across LLM providers, LLM proxies, and MCP proxies. It also adds resource fixtures and updates a subscription command test payload expectation.

Changes

AI Workspace CLI end-to-end suite

Layer / File(s) Summary
E2E environment and suite orchestration
.github/workflows/ai-workspace-cli-e2e.yml, tests/ai-workspace-cli-e2e/{go.mod,docker-compose.yaml,platform-api-config.toml,README.md}, tests/ai-workspace-cli-e2e/suite_test.go
Builds the backend and CLI, starts the authenticated Docker Compose environment, prepares projects and gateways, registers an isolated CLI workspace, and runs cleanup.
Artifact lifecycle scenarios
tests/ai-workspace-cli-e2e/features/*, tests/ai-workspace-cli-e2e/steps_test.go
Defines LLM provider, LLM proxy, and MCP scenarios that initialize, build, apply, retrieve, list, edit, and verify gateway associations.
Artifact resource fixtures
tests/ai-workspace-cli-e2e/resources/{llm-provider,llm-proxy,mcp}/...
Adds create/edit metadata, runtime manifests, OpenAPI definitions, and MCP prompts, resources, server metadata, and tool contracts.

Subscription command test expectation

Layer / File(s) Summary
Create-subscription payload assertion
cli/src/cmd/devportal/subscription/commands_test.go
Updates the expected create-subscription JSON field from subscriptionPlanName to subscriptionPlanId with the gold value.

Estimated code review effort: 4 (Complex) | ~45 minutes

Suggested reviewers: krishanx92, renuka-fernando, rakhitharr, malinthaprasan, pubudu538

Sequence Diagram(s)

sequenceDiagram
  participant Godog
  participant APCli
  participant PlatformAPI
  Godog->>APCli: initialize and stage artifact resources
  Godog->>APCli: build and apply artifact
  APCli->>PlatformAPI: create or update artifact
  Godog->>APCli: get and list artifact
  APCli->>PlatformAPI: retrieve persisted artifact
  PlatformAPI-->>APCli: return artifact and gateway association
Loading
🚥 Pre-merge checks | ✅ 2 | ❌ 3

❌ Failed checks (2 warnings, 1 inconclusive)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is missing most required template sections and only includes a minimal Purpose note with a related issue link. Fill in the remaining template sections: Goals, Approach, User stories, Documentation, tests, security checks, Samples, Related PRs, and Test environment.
Docstring Coverage ⚠️ Warning Docstring coverage is 57.89% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title is too vague to describe the main change and does not clearly summarize the added AI-Workspace CLI integration tests. Use a specific title such as 'Add AI-Workspace CLI integration tests'.
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (2)
.github/workflows/ai-workspace-cli-e2e.yml (1)

25-27: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Add a job-level timeout-minutes.

Without it, a hung Docker build or CLI build could occupy the runner for GitHub's default 360-minute cap, well beyond the intended ~20m test budget.

⏱️ Proposed fix
 jobs:
   ai-workspace-cli-e2e:
     runs-on: ubuntu-24.04
+    timeout-minutes: 45
     steps:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ai-workspace-cli-e2e.yml around lines 25 - 27, Add a
job-level timeout-minutes setting to the ai-workspace-cli-e2e job, using a value
aligned with the intended approximately 20-minute test budget. Keep the existing
runs-on and steps configuration unchanged.
tests/ai-workspace-cli-e2e/suite_test.go (1)

74-79: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Consider pinning MinVersion on the test HTTP client.

InsecureSkipVerify is justified here (local self-signed cert, #nosec-annotated), but the tls.Config has no MinVersion. Purely defense-in-depth for this localhost-only test client.

🔒 Optional hardening
 var httpClient = &http.Client{
 	Timeout: 30 * time.Second,
 	Transport: &http.Transport{
-		TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, // `#nosec` G402 — local self-signed cert
+		TLSClientConfig: &tls.Config{InsecureSkipVerify: true, MinVersion: tls.VersionTLS13}, // `#nosec` G402 — local self-signed cert
 	},
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/ai-workspace-cli-e2e/suite_test.go` around lines 74 - 79, Harden the
test HTTP client’s TLS configuration by setting an explicit modern minimum TLS
version in the tls.Config used by httpClient. Preserve the existing
InsecureSkipVerify setting and its justification, and limit the change to this
local test client.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@portals/developer-portal/configs/config.yaml`:
- Line 70: Quote the value assigned to validateResponses so YAML parses it as
the literal string "off" rather than a boolean, while leaving the existing
configuration key and intended value unchanged.

In `@tests/ai-workspace-cli-e2e/resources/llm-provider/definition.yaml`:
- Around line 1383-1401: Update the minimum constraints for
ephemeral_1h_input_tokens, ephemeral_5m_input_tokens, and
ServerToolUseUsage.web_search_requests in the schema so their default value of 0
is valid, preferably by changing each minimum from 1 to 0 while preserving the
existing defaults.
- Around line 4191-4201: Update all five generate, improve, and templatize
request/response schemas in the YAML definition so their required lists use
messages instead of message, matching the defined messages property. Ensure
messages is required consistently in each affected schema and remove the
incorrect message requirement.

In `@tests/ai-workspace-cli-e2e/resources/llm-provider/edit/runtime.yaml`:
- Line 1: Update the apiVersion in the llm-provider edit fixture to
gateway.api-platform.wso2.com/v1, matching the create fixture and other e2e
resources; do not change any other manifest content.

In `@tests/ai-workspace-cli-e2e/resources/llm-proxy/create/runtime.yaml`:
- Around line 51-72: Update both policy path entries in
tests/ai-workspace-cli-e2e/resources/llm-proxy/create/runtime.yaml lines 51-72
and tests/ai-workspace-cli-e2e/resources/llm-proxy/edit/runtime.yaml lines 64-85
from /v1/experimental/generate_prompt to the declared OpenAPI path /v1/messages;
leave the policy parameters unchanged.

In `@tests/ai-workspace-cli-e2e/resources/llm-proxy/definition.yaml`:
- Around line 856-868: Update the message schema unions containing the shown
refs and the corresponding union around the later occurrence to include both
RequestMCPToolResultBlock and ResponseMCPToolUseBlock, preserving all existing
entries so valid MCP request and response exchanges pass schema validation and
remain available to generated clients.

---

Nitpick comments:
In @.github/workflows/ai-workspace-cli-e2e.yml:
- Around line 25-27: Add a job-level timeout-minutes setting to the
ai-workspace-cli-e2e job, using a value aligned with the intended approximately
20-minute test budget. Keep the existing runs-on and steps configuration
unchanged.

In `@tests/ai-workspace-cli-e2e/suite_test.go`:
- Around line 74-79: Harden the test HTTP client’s TLS configuration by setting
an explicit modern minimum TLS version in the tls.Config used by httpClient.
Preserve the existing InsecureSkipVerify setting and its justification, and
limit the change to this local test client.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e2063544-7391-4c93-9c5b-9fad1e9eb1f4

📥 Commits

Reviewing files that changed from the base of the PR and between 8fec15c and 90cf0d5.

⛔ Files ignored due to path filters (2)
  • go.work is excluded by !**/*.work
  • tests/ai-workspace-cli-e2e/go.sum is excluded by !**/*.sum
📒 Files selected for processing (25)
  • .github/workflows/ai-workspace-cli-e2e.yml
  • cli/src/cmd/devportal/subscription/commands_test.go
  • portals/developer-portal/configs/config.yaml
  • tests/ai-workspace-cli-e2e/README.md
  • tests/ai-workspace-cli-e2e/docker-compose.yaml
  • tests/ai-workspace-cli-e2e/features/ai-workspace-cli.feature
  • tests/ai-workspace-cli-e2e/go.mod
  • tests/ai-workspace-cli-e2e/platform-api-config.toml
  • tests/ai-workspace-cli-e2e/resources/llm-provider/create/metadata.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-provider/create/runtime.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-provider/definition.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-provider/edit/metadata.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-provider/edit/runtime.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-proxy/create/metadata.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-proxy/create/runtime.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-proxy/definition.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-proxy/edit/metadata.yaml
  • tests/ai-workspace-cli-e2e/resources/llm-proxy/edit/runtime.yaml
  • tests/ai-workspace-cli-e2e/resources/mcp/create/metadata.yaml
  • tests/ai-workspace-cli-e2e/resources/mcp/create/runtime.yaml
  • tests/ai-workspace-cli-e2e/resources/mcp/definition.yaml
  • tests/ai-workspace-cli-e2e/resources/mcp/edit/metadata.yaml
  • tests/ai-workspace-cli-e2e/resources/mcp/edit/runtime.yaml
  • tests/ai-workspace-cli-e2e/steps_test.go
  • tests/ai-workspace-cli-e2e/suite_test.go

Comment thread portals/developer-portal/configs/config.yaml Outdated
Comment thread tests/ai-workspace-cli-e2e/resources/llm-provider/definition.yaml Outdated
Comment thread tests/ai-workspace-cli-e2e/resources/llm-provider/definition.yaml
Comment thread tests/ai-workspace-cli-e2e/resources/llm-provider/edit/runtime.yaml Outdated
Comment thread tests/ai-workspace-cli-e2e/resources/llm-proxy/create/runtime.yaml Outdated
Comment thread tests/ai-workspace-cli-e2e/resources/llm-proxy/definition.yaml
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Validation Results

Dependency name: github.com/cucumber/godog
Version: v0.15.0
Allowed range: >=v0.15.0
Approved: ✅ Yes

⚠️ Please verify the scope of the dependencies usage is necessary

@github-actions

Copy link
Copy Markdown
Contributor

Dependency Validation Results

Dependency name: github.com/cucumber/godog
Version: v0.15.0
Allowed range: >=v0.15.0
Approved: ✅ Yes

⚠️ Please verify the scope of the dependencies usage is necessary

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Validation Results

Dependency name: github.com/cucumber/godog
Version: v0.15.0
Allowed range: >=v0.15.0
Approved: ✅ Yes

⚠️ Please verify the scope of the dependencies usage is necessary

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Validation Results

Dependency name: github.com/cucumber/godog
Version: v0.15.0
Allowed range: >=v0.15.0
Approved: ✅ Yes

⚠️ Please verify the scope of the dependencies usage is necessary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant