Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .specify/feature.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{
"feature_directory": "specs/009-json-output-refactor"
"feature_directory": "specs/010-remove-entra-id"
}
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!-- SPECKIT START -->
For additional context about technologies to be used, project structure,
shell commands, and other important information, read the current plan
at specs/009-json-output-refactor/plan.md
at specs/010-remove-entra-id/plan.md
<!-- SPECKIT END -->
20 changes: 18 additions & 2 deletions GOAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Feature 7 - Local AI support
- Use the api-grade's JSON format output so the AI is able to process and reformat the information in a way that suits its requirements
- Leverage the AI support to not just grade the API, but also resolve the "non-breaking change" issues highlighted by the grading that are bringing down the result
- Any local AI tooling support must explicitly include Claude Code and GitHub Copilot
- Add support for rulesets hosted on GitHub private repos (via PAT) and Entra ID protected environments (e.g. SharePoint, OneDrive)
- Add support for rulesets hosted on GitHub private repos (via PAT)
- Support both direct installed and containerised execution of the functionality.

Feature 8 - CLI GitHub PAT
Expand All @@ -83,7 +83,23 @@ Feature 10 - Remove Entra ID
- Remove Entra ID related functionality from the project. This includes all functionality and all end-user/developer documentation in all of the packages.
- Entra ID related functionality was adding extra dependencies, administration requirements, and was proving impractical to implement. It was an incidental feature that can be more practically delivered outside of this software. Removing this unproven feature is the most appropriate solution.

Feature 11 - Remote AI support
Feature 11 - Rename quick fixes only

- Rename all user visible "quick fixes only" command line arguments or MCP tools to "remediation safety",
with an argument (level). The estimated remediation safety level of safe equates to quick fixes only. e.g. CLI's `--quick-fixes-only` argument would be replaced by `--remediation-safety safe`.
- Propose how the MCP server's equivalent tools should be named and work. Should the tools accept the safety level as an argument, or do we need to supply one tool per safety level (e.g. tool `grade-api-remediation-safety-safe`)
- Make this change at a superficial level only. It should change the visible user experience and user focussed documentation, but the underlying implementation can still refer to "quick fixes only" in function names, etc. The full implementation of this feature will be implemented in an upcoming feature.

Feature 12 - Remediation safety

- Build a ruleset analyser that determines the level of risk associated with remediating violations identified by each of its rules. The analyser should also identify its level of confidence in each risk level.
- Extend the remediation safety feature to support additional safety levels: humanreview, unsafe.
- Remediation safety is calculated using the output of the ruleset analyser.
- Implement the ruleset analyser and remediation safety calculation in alignment with specification automated_remediation_safety_algorithm_spec.md .
- Remediation safety should be made visible in the JSON and human output formats of the various tools and packages.
- Complete refactoring of the code base (including documentation) to move away from the older "quick fixes only" concept to "remediation safety". "Quick fixes" should no longer be mentioned in the code base or user facing documentation.

Feature 13 - Remote AI support

- Allow API grading to be performed directly (remotely) from LLMs and agentic AI tooling
- Update AI support to include remote access via streamable/HTTP transport
Expand Down
4 changes: 0 additions & 4 deletions docs/cli/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,10 +221,6 @@ per the order above). Both `config set-ruleset`/`get-ruleset` and the
the exact same file — a workspace configured via one is immediately usable by the
other.

> **Note:** Microsoft Entra ID authentication (used by the MCP server) is not
> supported by the CLI. If a shared config file specifies `auth.type: "entra-id"`,
> the CLI exits with a clear error rather than attempting it.

---

## Configuration File (`.apigrade.json`)
Expand Down
1 change: 0 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
| [MCP Server Overview](package/api-grade-mcp.md) | All six MCP tools and their inputs/outputs |
| [MCP Quick Start](mcp/quick-start.md) | Install and configure the MCP server in minutes |
| [MCP Configuration Reference](mcp/configuration.md) | Default rulesets, auth, and scope precedence |
| [MCP Entra ID Setup](mcp/entra-id-setup.md) | One-time Azure-side app registration for Entra ID auth |
| [MCP GitHub Token Setup](mcp/github-pat-setup.md) | One-time GitHub PAT creation for `github-pat` ruleset auth |
| [MCP Troubleshooting](mcp/troubleshooting.md) | Auth failures, missing tools, and common errors |
| [Backstage Plugins](backstage-plugins/README.md) | Display grades on Backstage API entity pages |
Expand Down
2 changes: 0 additions & 2 deletions docs/mcp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,6 @@ npx -y @dawmatt/api-grade-mcp
|-------|---------|
| [Quick Start](./quick-start.md) | Install and configure in minutes — covers Claude Code, Copilot, Claude Desktop |
| [Configuration Reference](./configuration.md) | Default rulesets, auth, scope precedence, and config file format |
| [Entra ID Setup](./entra-id-setup.md) | One-time Azure-side app registration required for Entra ID auth |
| [GitHub Token Setup](./github-pat-setup.md) | One-time GitHub PAT creation required for `github-pat` ruleset auth |
| [Troubleshooting](./troubleshooting.md) | Auth failures, missing tools, and common errors |

Expand All @@ -71,7 +70,6 @@ npx -y @dawmatt/api-grade-mcp

- [→ Quick Start](./quick-start.md) — get the MCP server running in your AI tool
- [→ Configuration Reference](./configuration.md) — configure a default ruleset and auth
- [→ Entra ID Setup](./entra-id-setup.md) — Azure-side app registration for Entra ID auth
- [→ GitHub Token Setup](./github-pat-setup.md) — create and configure a GitHub PAT for `github-pat` auth
- [→ Troubleshooting](./troubleshooting.md) — fix common issues
- [→ Package Documentation](../package/api-grade-mcp.md) — full tool reference with all inputs and outputs
Expand Down
32 changes: 1 addition & 31 deletions docs/mcp/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,17 +70,13 @@ Or for a local file:
|-------|------|-------------|
| `rulesetPath` | `string \| null` | Absolute path or HTTPS URL to a Spectral-compatible ruleset. `null` clears the scope. |
| `auth` | `object \| null` | Authentication config for remote URLs. `null` means no auth (public URL). |
| `auth.type` | `"github-pat" \| "entra-id"` | Auth mechanism to use. |
| `auth.type` | `"github-pat"` | Auth mechanism to use. |
| `auth.githubToken` | `string` (optional) | Inline PAT for GitHub. Not recommended — use the `GITHUB_TOKEN` env var instead. |
| `auth.tenantId` | `string` | Entra ID tenant ID (required for `entra-id`). |
| `auth.clientId` | `string` | Entra ID client/application ID (required for `entra-id`). |

---

## Authentication

GitHub PAT and Entra ID use different storage mechanisms because the credentials behave differently, not out of inconsistency: a GitHub PAT is a static secret you provide once, while an Entra ID token is dynamically issued and refreshed by Microsoft and must survive server restarts without you re-entering anything.

### No Authentication (Public URLs)

For publicly accessible ruleset URLs, no auth configuration is needed:
Expand Down Expand Up @@ -115,31 +111,6 @@ Setting `GITHUB_TOKEN` in the environment:
export GITHUB_TOKEN=ghp_xxxx # then start your AI tool
```

### Microsoft Entra ID (Device Code Flow) — the server handles tokens for you, no env var needed

For rulesets hosted on SharePoint or other Entra ID-protected sites. This requires an Entra ID app registration to already exist in your tenant — see [Entra ID Setup](entra-id-setup.md) for the one-time administrator steps to create it before continuing here.

```json
{
"rulesetPath": "https://mycompany.sharepoint.com/sites/api-standards/ruleset.yaml",
"auth": {
"type": "entra-id",
"tenantId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"clientId": "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy"
}
}
```

On the first grading request after configuration, the server initiates a **device code flow**:

1. The grading tool returns an `ENTRA_AUTH_REQUIRED` response containing a `userCode` and `verificationUri`.
2. Visit the URI and enter the code to authenticate (typically 15-minute window).
3. Retry the grading request — the token is now cached.

**Token cache location**: `~/.api-grade/entra-token-cache.json` — the same pattern Azure CLI uses at `~/.azure`. Written to the user home directory only, never the workspace.

Cached tokens are reused on subsequent requests, including after restarting the MCP server, with no further action from you. If the token expires, the device code flow restarts automatically on the next grading request.

---

## Ruleset Fetch Failure Recovery
Expand Down Expand Up @@ -192,7 +163,6 @@ Passing `rulesetPath: null` clears the configuration at that scope without affec
## Further Reading

- [Quick Start](quick-start.md) — install and configure in minutes
- [Entra ID Setup](entra-id-setup.md) — one-time Azure-side app registration required for Entra ID auth
- [GitHub Token Setup](github-pat-setup.md) — one-time GitHub PAT creation required for `github-pat` auth
- [Troubleshooting](troubleshooting.md) — auth failures, missing tools, and common errors
- [Package Documentation](../package/api-grade-mcp.md) — full tool reference with all parameters
Expand Down
111 changes: 0 additions & 111 deletions docs/mcp/entra-id-setup.md

This file was deleted.

21 changes: 2 additions & 19 deletions docs/mcp/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ openapi.yaml

## `RULESET_AUTH_FAILED` on Every Request

**Cause**: The configured default ruleset's host was reached, but the request was rejected on credentials — wrong, missing, or expired token (401/403), or Entra ID re-authentication is needed.
**Cause**: The configured default ruleset's host was reached, but the request was rejected on credentials — wrong, missing, or expired token (401/403).

**Diagnosis**:
1. Run `get-ruleset-config` to see what ruleset URL is configured.
Expand All @@ -78,7 +78,7 @@ openapi.yaml

**Cause**: The stored auth configuration for the configured default ruleset (in `.api-grade/config.json` or `~/.api-grade/config.json`) is malformed or missing required fields — this is distinct from `RULESET_AUTH_FAILED`, since no credentials were even sent to the host; the configuration itself failed validation before a request could be made.

**Fix**: Run `get-ruleset-config` to inspect the stored `auth` block for the affected scope, then use `set-ruleset-config` to supply a complete, valid `auth` configuration (e.g. `tenantId` and `clientId` for `entra-id`, or `type: "github-pat"` with `GITHUB_TOKEN` set in the environment for GitHub).
**Fix**: Run `get-ruleset-config` to inspect the stored `auth` block for the affected scope, then use `set-ruleset-config` to supply a complete, valid `auth` configuration (`type: "github-pat"` with `GITHUB_TOKEN` set in the environment for GitHub).

---

Expand All @@ -97,23 +97,6 @@ When the default ruleset fetch fails, the grading tool returns four recovery opt

---

## Entra ID Device Code Not Completing

**Cause**: The device code has a short expiry window (typically 15 minutes).

**Fix**: If you miss the window, retry the grading request — the server initiates a new device-code flow and returns a fresh code.

Check:
- `tenantId` and `clientId` are correct in the config
- `~/.api-grade/entra-token-cache.json` is writable

To force a fresh authentication (clearing the cached token):
```sh
rm ~/.api-grade/entra-token-cache.json
```

---

## GitHub Token Issues

**Symptom**: `RULESET_AUTH_FAILED` with `failureReason: "auth-failed"` when fetching from GitHub Enterprise. If you haven't created a token yet, see [GitHub Token Setup](github-pat-setup.md) for the one-time steps.
Expand Down
2 changes: 1 addition & 1 deletion docs/package/api-grade-mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ All grading tools support an optional `rulesetPath` parameter for one-off custom
**Global default** (`~/.api-grade/config.json`):
> Set my global default ruleset to `/Users/jane/rulesets/personal-standards.yaml`

For full configuration options including GitHub PAT and Entra ID authentication, see the [Configuration Reference](../mcp/configuration.md).
For full configuration options including GitHub PAT authentication, see the [Configuration Reference](../mcp/configuration.md).

---

Expand Down
Loading
Loading