Skip to content

Release v2.0.0: Registry Service - #9

Draft
roncodes wants to merge 71 commits into
mainfrom
release/v2.0.0
Draft

roncodes wants to merge 71 commits into
mainfrom
release/v2.0.0

Conversation

@roncodes

Copy link
Copy Markdown
Member

Roll-up for the Fleetbase Extensions Registry re-architecture. Every PR for this repo merges into this release branch; this PR merges into main only when the whole project (registry, registry-bridge, fleetbase-cli, registry-infra) is complete, and deployments happen after that. Do not merge piecemeal.

Contains: the Laravel Registry Service (accounts, catalog, npm and Composer endpoints, licence gate, instance API, importer, releases pipeline), with Verdaccio kept under legacy/ until cutover. Supersedes #4.

Open PRs into this branch: #6 (website backend), #7 (licence gate reconcile), #8 (Inertia website, stacked on #6).

The v2 Registry Service takes over the repository root. The Verdaccio
build, compose file, config and plugin submodules keep working from
legacy/; the deploy workflow now runs the bake file from that directory.
Laravel 12 skeleton at the repository root (Laravel 11 is blocked by
Composer's security-advisory policy; fleetbase/core-api cannot be a
dependency because it pins illuminate ^9|^10 and sanctum 3.2.4 and
auto-loads conflicting migrations). The core-api conventions the plan
relies on are ported into the app with the same names and behaviour:
Model (UUID keys, findById), HasUuid, HasPublicId, Money cast, Filter,
FleetbaseResource and FleetbaseResourceCollection with the flat meta
pagination block.
Tables from the plan's data model: accounts, organizations,
memberships, tokens, categories, extensions, extension_packages,
releases, release_checks, reviews, prices, files, instances, installs,
licences, licence_activations, purchases and events, plus
verification_codes for email verification codes. Public id prefixes:
ext_, rel_, lic_, inst_, tok_ (and acct_, org_, cat_, file_, pur_).
Categories are seeded from registry-bridge's ExtensionsCategorySeeder.
- POST /v1/auth/register creates the account (pending_verification) and
  a personal organization with an owner membership, then emails a
  6-digit code; /v1/auth/verify activates it; /v1/auth/verify/resend
  never reveals whether an account exists.
- POST /v1/auth/login exchanges email and password (Hash::check, the
  same bcrypt scheme as core-api's Auth::checkPassword) for a personal
  token returned once; /v1/auth/logout revokes the current token.
- Personal tokens: flbp_ prefix, SHA-256 hashed at rest, scoped (read,
  publish, install, instance:register, *), expiring (max 365 days),
  revocable, last-use stamped; prefix + last four shown in lists.
  TokenGuard backs the auth:token guard; scope:<name> middleware.
- GET /v1/account returns the caller with organizations and roles.
- Fleetbase error envelope {errors: [...]} for API exceptions, rate
  limits on auth routes, Pest feature tests for every flow.
GET /v1/catalog/extensions (q, category, pricing, hosting, publisher,
tag, sort, page, limit), /v1/catalog/extensions/{slug},
/{slug}/releases, /v1/catalog/publishers/{slug} and
/v1/catalog/categories. Only published listings are visible; install
counts come from installs rows; fulltext search on MySQL with a LIKE
fallback elsewhere. Responses are cached for 60 seconds per URL with a
matching Cache-Control header and use Fleetbase-style resources with
the flat meta pagination block.
- Dockerfile: composer vendor stage, serversideup php 8.3 fpm-nginx
  runtime, app target for local use and aws-app target wrapped with
  ssm-parent like the v1 deploy; docker-bake.hcl builds aws-app.
- docker-compose.yml: app, queue worker, MySQL 8, MinIO (+ bucket
  init), Mailpit for verification emails.
- .env.example for a zero-dependency SQLite run; .env.docker for the
  compose stack.
- GitHub Actions CI: Pint and Pest on PHP 8.3 and 8.4, then a Docker
  build of the app target. The deploy/* workflow still builds the
  Verdaccio image from legacy/.
…ha1 and SRI sha512

Objects live at {prefix}/{file uuid}/{filename} under tarballs/, dist/, icons/ and
screenshots/ on the registry disk. Package tarballs keep their decoded package.json or
composer.json in files.meta so metadata endpoints never open archives. A small streaming
tar reader replaces PharData for that. Downloads answer If-None-Match and re-hash the
bytes as they stream. Fixture packages for the seed helper live in database/fixtures.
GET /npm/{name} builds the packument from published, non-yanked releases that carry an
engine tarball: versions from the stored package.json with authoritative name, version
and dist (tarball, shasum, integrity), dist-tags by SemVer precedence (latest = highest
stable, beta = highest beta, latest falls back to beta when nothing is stable), time,
and a content ETag with 304 support. Scoped names resolve whether npm encodes the slash
or not. GET /npm/{name}/-/{file}.tgz streams the tarball; yanked versions stay
downloadable for lockfiles. npm writes (publish, unpublish, dist-tag) get 405 pointing
at flb publish. Package routes return npm's error string beside the Fleetbase envelope.
…alls

Instance tokens (flbi_) go through TokenGuard with a kind check and the new
auth:instance guard. Installs claim licence seats, creating an unlimited free
licence on first install of a free extension. Updates compare installed
releases with the newest published release whose compat ranges accept the
instance's console and API versions (composer/semver).

Also adds source_uuid columns for the importer.
GET /composer/packages.json returns metadata-url and available-packages (every Composer
package with a published release). GET /composer/p2/{vendor}/{package}.json lists every
published release newest first in the composer/2.0 minified format, built from the
stored composer.json (root-only keys dropped) with dist {type: tar, url, shasum,
reference}; ~dev.json is always empty. GET /composer/dist/{vendor}/{package}/{version}.tar
streams the API archive. Versions Composer cannot parse are left out of p2.
licence:npm and licence:composer guard the packument, tarball, p2 and dist routes.
Free packages pass through. Paid packages need a bearer token: a personal token with the
install scope (its pinned organization, or every organization of the account) or an
instance token (instances.token_hash). Access is granted when one of those organizations
owns the extension or holds an active, unexpired licence; 401 with WWW-Authenticate
without a usable token, 403 otherwise. Paid metadata and archives are Cache-Control
private and vary on Authorization.

registry:seed seeds categories; with --seed-release it fabricates a published free and
a paid extension from the fixture tarballs, a buyer organization holding a licence and
an outsider without one, and prints install-scoped tokens for both (--json for scripts).
tests/Conformance/packages.sh drives real npm and Composer clients against a running
service with that seed.
Organizations gain an is_staff flag (not mass assignable; set with registry:staff).
extension_packages names in a reserved namespace (registry.reserved_scopes, matched on
the npm scope or Composer vendor, case-insensitively) can only be saved for an extension
whose organization is staff; creation, renames and moves to another listing are checked
and fail with a 422 on the name field. Package names are stored lowercase. The unused
REGISTRY_STAFF_ORGANIZATION setting is removed in favour of the flag.
…ints

The package-conformance job starts the service on SQLite with the local disk, seeds the
fixture releases and runs tests/Conformance/packages.sh: npm view, pack and install,
composer require with secure-http off, 401 and 403 for the paid package, and a licensed
install with each client.
Reads a MySQL DSN, a sqlite:// file or a .sql/.sql.gz dump (loaded into a
temporary SQLite file, no MySQL server needed) and maps registry extensions,
bundles, purchases, installs, developer accounts, cloud registry users,
companies and categories per the plan's mapping rules. Rows are keyed by
source_uuid so re-runs update in place. --dry-run runs the same mapping in a
rolled-back transaction and prints counts plus an anomaly report. Bundle
archives, icons and screenshots are copied with SHA-256 checksums; releases
without a retrievable tarball import as yanked.

Documents the instance API and the importer in the README.
# Conflicts:
#	README.md
#	app/Models/Instance.php
#	routes/api.php
#	tests/Pest.php
Semver is now the one version comparator: normalize() pads loose versions
(v1.2 -> 1.2.0), compare() normalizes before applying SemVer precedence, and
isNewer() moves over from Compat. Compat keeps range evaluation and gains
validRange() for the release checks.
Publisher API (publish scope, owner/admin/publisher role): create listings
idempotently on package names with the namespace rules, edit listing fields,
upload releases (manifest, engine, api, optional prebuilt dist), submit,
withdraw, yank.

RunReleaseChecks inspects each archive once and records release_checks for
the manifest (extension.json JSON Schema shared with registry-bridge), compat
ranges, package names against extension_packages, version order per channel,
size limits, archive safety (traversal, links, devices, forbidden types,
native binaries), secrets (built-in gitleaks-style rules, plus gitleaks when
installed) and the build-engine-release dist manifest.

Release state machine with a reviews row per transition; review policy D6
(human review for first releases and package name, pricing or permissions
changes, auto-approve and publish otherwise); staff admin API for the queue,
claim, approve, reject with reason and items, publish and suspend. Publishing
sets published_at, lists draft extensions, flushes the catalog cache through
a generation key, and emails the uploader.
…a (Phase 2 Workstream F)

The service side of the publisher website and the staff review console.

- Web sessions: a `web` guard backed by server-side `flbs_` session tokens, with
  GET/POST/DELETE /v1/auth/session. Requests that carry the session or XSRF cookie run
  through Laravel's session and CSRF middleware; Bearer clients, the catalog and the
  instance API stay cookieless. Account, publisher and admin routes take a Bearer token
  or the session (auth:token,web). Sessions table migration.
- Account: PATCH /v1/account, POST /v1/account/password; AccountResource adds is_staff.
- Organizations and members: list, create, show, update; add by email, change role,
  remove or leave, with owner-only owner changes and a last-owner guard.
- Listing media and pricing: icon upload and removal, up to eight ordered screenshots,
  PUT price (paid needs the organization's Stripe Connect account). The publisher list
  now includes each listing's releases. promotional_text accepts 20,000 characters.
- Staff: archive downloads, unsuspend, GET /v1/admin/review-items (R1 to R6), and the
  listing preview on release responses.
- SPA fallback: non-API GET paths serve public/index.html; the service info moves to
  GET /v1.
- registry:seed-website seeds staff and publisher accounts, published demo extensions,
  a queued release and a draft for the website's end-to-end tests.

Pest: 279 tests pass.
LicenceGate hashed and looked up flbi_ tokens itself; it now calls
Instance::findByToken() so hashing and the revoked status check live on
the model. Package downloads keep checking the organization's licence
only, not a seat on the calling instance; seats are claimed when the
instance reports the install. README records the decision and the
tests pin it (no activation needed, unlicensed instance gets 403).
…design

The publisher website and staff review console, served by the Laravel app with Inertia v3,
React 19, Tailwind v4 and Vite, replacing the Ember prototype.

- Page controllers (app/Http/Controllers/Web) read through the /v1 API controllers in
  process, so props have the API's JSON shapes; pages write to /v1 with the session cookie
  and XSRF header. Website routes use auth:web; unknown paths render the website 404.
- Public: Discover (search, intents, filters, console compatibility), extension pages with
  install steps and a version check, publisher profiles.
- Account: sign in, create account, email code, profile and password, organizations and
  members, personal tokens shown once.
- Studio: inbox, new listing, the listing editor with a live readiness rail, uploads, live
  release checks, submit / withdraw / rejected states, history, insights placeholder.
- Review console: queue with tabs, filters and J/K, the R1 to R6 decision rail, publish,
  suspend and lift, the To publish page.
- Light and dark themes, phone layouts, error pages through Inertia.
- Vitest component and page-logic tests, Pest page tests, a Playwright smoke test with its
  own seeded server (scripts/e2e-server.sh), CI jobs, and a Docker stage for the assets.
…nd Studio features

Catalog
- GET /v1/catalog/extensions takes console=<version> (each item gets compatibility:
  {console, compatible, newest_compatible}) and compatible=1 (only listings with a published
  release for that console). Discover filters on the server, so counts and pages cover the
  whole catalog; ?compat=any shows every version.

Server-side rendering
- The SSR bundle is self-contained (ssr.noExternal), listens on 127.0.0.1 and takes
  INERTIA_SSR_PORT. The Docker image installs Node and runs it as an s6 service beside nginx
  and php-fpm (docker/s6/inertia-ssr); INERTIA_SSR_ENABLED=false turns it off.
- The Playwright server runs the renderer with SSR errors failing the request; tests wait for
  hydration (html[data-ready]). Write forms are method=post and the search and version check
  forms are real GET forms, so nothing typed before hydration lands in a URL.

Account
- Forgot password: POST /v1/auth/password/forgot (no enumeration) and /reset with a six-digit
  code; resetting revokes every token and session. Codes now die after five wrong guesses.
- Sign in with GitHub (laravel/socialite): match by github_id, then verified email, else a
  new account; connect and disconnect on the profile. accounts.github_id becomes unique.

Studio
- GET /v1/publisher/packages/availability, sharing one rule set (PackageNames) with create;
  live Available chips on the new-listing page.
- submitted_at and decided_at on release summaries: "Submitted N days ago" and "day N".
- PATCH /v1/publisher/releases/{id} edits release notes (not when yanked).
- Drag to reorder screenshots, keeping the buttons for keyboard and screen readers.

Review console
- previous_version (and the previous release) on admin release responses, without N+1:
  "Update from vX" in the queue and what changed since it on the release page.
- Request changes on an approved release, from the release page and To publish.

Also: Vitest timeout 20 s for slow CI runners.
The plugin refuses to start its dev server when CI is set, which failed the
Vitest step. Tests need neither the manifest nor the hot file.
…d sale pricing; document the instance checkout contract
…m-side Stripe failures, highlight the selected organization on Billing
…voke the licence, staff licence management, return URLs on the instance host, members see licences
…otals, monthly CSV, refunds through Stripe, and the Finance page (Phase 4 M)
…st/{version} behind the licence gate, and dist size, sha256 and builtAgainst on catalog and update releases
Re-registering an instance (Admin > Extensions Registry, then
`flb instance register`) created a second active instances row: the old
one kept its installs and licence seats, the new one reported nothing
installed (D17 from the 23-24 Oct e2e walk).

POST /v1/instances/register now takes an optional previous_token. When
it resolves to an active instance in the code's organization, that row
is updated and issued a new token (the old one stops working) and the
response is 200 with the same instance id. When it belongs to another
organization, a new instance is created, the old one is revoked with its
seats released, and its open installs move to the new instance. Unknown
or revoked tokens register a new instance as before.
Comment thread .github/workflows/ci.yml
Comment on lines +14 to +54
name: Pest (PHP ${{ matrix.php }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
php: ["8.3", "8.4"]

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
extensions: mbstring, intl, bcmath, gd, zip, pdo_sqlite, sqlite3
coverage: none
tools: composer:v2

- name: Cache Composer packages
uses: actions/cache@v4
with:
path: ~/.composer/cache
key: composer-${{ runner.os }}-${{ matrix.php }}-${{ hashFiles('composer.lock') }}
restore-keys: composer-${{ runner.os }}-${{ matrix.php }}-

- name: Install dependencies
run: composer install --no-interaction --prefer-dist --no-progress

- name: Prepare environment
run: |
cp .env.example .env
php artisan key:generate --ansi

- name: Lint (Pint)
run: vendor/bin/pint --test

- name: Run tests
run: vendor/bin/pest --ci --colors=always

package-conformance:
Comment thread .github/workflows/ci.yml
Comment on lines +55 to +112
name: npm and Composer conformance
runs-on: ubuntu-latest
needs: tests
# .env.example already runs on SQLite (database/database.sqlite) with the local disk;
# `artisan serve` reads .env rather than the job environment.
env:
PHP_CLI_SERVER_WORKERS: 4

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: "8.3"
extensions: mbstring, intl, bcmath, zip, pdo_sqlite, sqlite3, phar, zlib
coverage: none
tools: composer:v2

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "22"

- name: Cache Composer packages
uses: actions/cache@v4
with:
path: ~/.composer/cache
key: composer-${{ runner.os }}-8.3-${{ hashFiles('composer.lock') }}
restore-keys: composer-${{ runner.os }}-8.3-

- name: Install dependencies
run: composer install --no-interaction --prefer-dist --no-progress

- name: Prepare the service
run: |
cp .env.example .env
php artisan key:generate --ansi
touch database/database.sqlite
php artisan migrate --force

- name: Start the service
run: |
php artisan serve --host=127.0.0.1 --port=8000 > storage/logs/serve.log 2>&1 &
for i in $(seq 1 30); do curl -sf http://127.0.0.1:8000/up > /dev/null && exit 0; sleep 1; done
cat storage/logs/serve.log; exit 1

- name: Run npm and Composer against seeded releases
run: tests/Conformance/packages.sh http://127.0.0.1:8000

- name: Service logs
if: failure()
run: |
cat storage/logs/serve.log || true
cat storage/logs/laravel.log || true

docker:
Comment thread .github/workflows/ci.yml
Comment on lines +113 to +130
name: Docker image builds
runs-on: ubuntu-latest
needs: tests
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build app target
uses: docker/build-push-action@v6
with:
context: .
target: app
push: false
cache-from: type=gha
cache-to: type=gha,mode=max
roncodes and others added 16 commits October 24, 2026 02:52
# Conflicts:
#	app/Http/Controllers/Catalog/CatalogController.php
D17 follow-up. An instance that registers again without a usable
previous_token (the console lost its settings, or registered through a path
that does not send it) still created a second instance. Registration now also
matches the active instance of the code's organization with the same host
(scheme, case and a trailing slash ignored; the most recently seen first) and
rotates it: same id, installs and seats, new token, old token refused.

A new or empty host, or the same host in another organization, still creates
a new instance. The response carries `rotated` (200 when true, 201 when
false), and the event records `matched_by` (token or host). The README
documents the re-registration rules.
# Conflicts:
#	app/Models/Instance.php
Resolve instance tokens in LicenceGate through Instance::findByToken
Website backend: cookie sessions, organizations, listing media (Phase 2 Workstream F)
Registry website: Inertia (React) rebuild from the Waypoints design
Comment thread .github/workflows/ci.yml
Comment on lines +113 to +146
name: Website (lint, Vitest, build)
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "22"
cache: npm

- name: Install dependencies
run: npm ci --no-audit --no-fund

- name: Lint (ESLint)
run: npm run lint

- name: Vitest
run: npm test

- name: Build (client and SSR)
run: npm run build

- name: Upload build
uses: actions/upload-artifact@v4
with:
name: website-build
path: |
public/build
bootstrap/ssr
retention-days: 3

website-e2e:
Comment thread .github/workflows/ci.yml
Comment on lines +147 to +196
name: Website smoke test (Playwright)
runs-on: ubuntu-latest
needs: [tests, website]
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: "8.3"
extensions: mbstring, intl, bcmath, gd, zip, pdo_sqlite, sqlite3, phar, zlib
coverage: none
tools: composer:v2

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "22"
cache: npm

- name: Install dependencies
run: |
composer install --no-interaction --prefer-dist --no-progress
npm ci --no-audit --no-fund
npx playwright install --with-deps chromium

# The artifact keeps paths from the repository root: public/build and bootstrap/ssr. With
# the SSR bundle present, scripts/e2e-server.sh also runs the renderer.
- name: Download the website build
uses: actions/download-artifact@v4
with:
name: website-build
path: .

- name: Run the smoke test
run: CI=true npx playwright test

- name: Playwright report
if: failure()
uses: actions/upload-artifact@v4
with:
name: playwright-report
path: |
playwright-report
test-results
storage/logs
retention-days: 7

docker:
roncodes and others added 4 commits October 24, 2026 17:23
Commerce: Stripe Connect, checkout, webhooks, payouts and billing (Phase 4 K, L, M)
Staff finance and refunds (Phase 4 M)
Publisher analytics: stats APIs and Studio Insights
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants