Repository navigation
chore(AIC-3500): fail CI on an undeclared breaking change to the public API - #196
Merged
Merged
Conversation
jeffdupont
marked this pull request as ready for review
October 20, 2026 22:24
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 06050f1. Configure here.
Each public module (every package root, and each module of the core package's experimental subpackage) now has a committed report at packages/<package>/api-report/<module>.txt, generated by scripts/api_report.py with griffe. A report lists each name in __all__ with its full signature, and a class's public members indented below it. tests/test_api_report.py fails while any report is out of date, so `make test` and CI catch a public name or signature change whose report wasn't regenerated. It also checks the generator against a fixture package. `make api-report` writes the reports, `make api-report-check` checks them, and .github/CODEOWNERS gives them to team-ai-agents. griffelib is now an explicit dev dependency. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to the public API scripts/api_breaking.py compares the names in each public module's __all__ with another git ref, using griffe: - `pr`: compares with the PR's base. A breaking change passes only if the PR title declares it with `!` (so release-please cuts a major version, or a minor one before 1.0.0), or if an owner applied the api-compatible label. - `release`: on the release-please PR, compares each package whose version changes with its last release tag, and fails on a breaking change unless the new version is a major one (a minor one before 1.0.0). Experimental modules are skipped. griffe reports removed names, parameter changes, changed defaults and changed attribute values; the script adds names dropped from __all__ and changed return types, which griffe does not flag. The API compatibility workflow runs it on every PR, and re-runs it when the title or labels change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… too An owner can now judge a flagged change compatible on the release PR, as on any other PR. Without it, a compatible change flagged on the release PR (the capture_content default on #166) could only pass by bumping the version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jeffdupont
force-pushed
the
jdupont/AIC-3500/api-report
branch
from
October 20, 2026 22:33
13e38fb to
741f773
Compare
jeffdupont
force-pushed
the
jdupont/AIC-3500/api-breaking
branch
from
October 20, 2026 22:33
06050f1 to
b7acfb0
Compare
… reports Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aking check Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # pyproject.toml
apucacao
approved these changes
Oct 22, 2026
# Conflicts: # scripts/api_report.py
jeffdupont
added a commit
that referenced
this pull request
Oct 23, 2026
Main's **Tests** job has failed since #202 merged (run 39959179499): `tests/test_api_report.py::test_committed_reports_are_current` reports that `launchdarkly_ai_server.experimental.evaluations.txt` is out of date. #202 removed `EvaluationsModule.api`. Its last commit was pushed before the API report check (#195, #196) merged, so its own CI never ran the check. This PR is the output of `make api-report`: one line removed (`api: LDApiClient [property]`). Checked locally: `make api-report-check` reports "15 API reports are current", and `tests/test_api_report.py` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Regenerates the committed **evaluations** API report so it matches the current experimental public surface after `EvaluationsModule.api` was removed in #202. > > The only report change is dropping the `api: LDApiClient` property line from `EvaluationsModule`; this unblocks `test_committed_reports_are_current` / `make api-report-check`. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 728bc50. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Stacked on #195 (the API reports).
Why
The API reports make every public change visible, but they don't stop a breaking change from shipping in a minor or patch release. After 1.0 a breaking change has to be deliberate: declared, and released as a major version.
What
scripts/api_breaking.pyuses griffe to compare the names in each public module's__all__with another git ref. The new API compatibility workflow runs it on every PR, and re-runs it when the title or labels change.pr): compares with the PR's base. A breaking change fails unless one of these is true:!(feat!: ...,fix(client)!: ...), which makes release-please cut a major version (a minor one before 1.0.0);api-compatiblelabel because they judge the flagged change compatible.release): for each package whose version changes in.release-please-manifest.json, compares the code with that package's last release tag (<component>-<version>). It fails on a breaking change unless the new version is a major one (a minor one before 1.0.0), or an owner appliedapi-compatibleto the release PR. A package with no release tag yet is reported and skipped.__all__, and changed return types, which griffe doesn't flag.Tested
tests/test_api_breaking.py(23 tests) uses a throwaway git repo with a tagged release. It covers:api-compatiblepassing an undeclared breaking release, and another label not passing it.The fixture isolates git from the developer's global config.
Full suite:
uv run pytestgives 2579 passed, 14 skipped.ruff checkandruff format --checkare clean, andactionlintpasses on the workflow.Against real history:
pragainstmain: no breaking changes, so it passes.pragainstlaunchdarkly-ai-server-0.3.0: 23 breaking changes. That's 14 names that moved toexperimental, pluscapture_content's default changing fromFalsetoNonein every provider factory,create_handlerandProviderHandler. It exits 1 with a plain title, and 0 with!or the label.The open release PR #166
Running
releaseagainst #166 flags 6 provider packages:Each changed
capture_content's default fromFalsetoNone(#158) since its last release, but is getting only a patch bump.client(0.3.0 → 0.4.0) passes.If
Nonemeans "use the default fromconfigure()", callers see no difference. So theapi-compatiblelabel now also passes the release check (06050f1): an owner can judge a release PR compatible, the same way as any other PR. Verified on #166: it exits 1 without the label and 0 with it. The label now exists in this repo.🤖 Generated with Claude Code
Note
Overview
Adds CI enforcement so undeclared public API breaks cannot ship in patch/minor releases.
A new API compatibility GitHub Actions workflow runs on every PR (including title/label edits). For normal PRs it runs
scripts.api_breaking pr, comparing stable__all__exports against the merge base via griffe (plus extra checks for__all__drops and return-type changes). CI fails unless the title uses conventional-commit!or an owner appliesapi-compatible. On release-please branches it runsrelease, comparing each bumped package to its last release tag and requiring a major (or pre-1.0 minor) bump when breaking changes slipped through.api_report.resolveis refactored to take aModulesCollectionso the breaking checker can reuse the same resolution logic.tests/test_api_breaking.pycovers detection rules, title/label gates, semver alignment, experimental skips, and removed packages.Reviewed by Cursor Bugbot for commit a61cf9e. Bugbot is set up for automated code reviews on this repo. Configure here.