Skip to content

Registry Service v2 (ongoing): Laravel service replacing the Verdaccio-based registry - #4

Closed
roncodes wants to merge 17 commits into
mainfrom
v2
Closed

roncodes wants to merge 17 commits into
mainfrom
v2

Conversation

@roncodes

@roncodes roncodes commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Registry Service v2 — ongoing development (draft)

This branch is the rewrite of the Fleetbase Extensions Registry as a single Registry Service: the system of record for accounts and publishers, the public catalog, releases and review, package delivery for npm and Composer, licences and purchases. Every Fleetbase instance (cloud or self-hosted), the flb CLI and the publisher website become clients of it.

Why: today the catalog lives in the cloud API's registry-bridge tables, Verdaccio delegates auth to that API on every request, self-hosted instances query their own empty tables, CLI publishes bypass review, and console publishes never reach Verdaccio.

What is on this branch

Area State
Verdaccio v1 registry moved to legacy/ (still deployable from there until the publishing cutover) done
Laravel 12 service at the repo root; core-api conventions (UUID models, public ids, Money cast, Filter, FleetbaseResource) ported into app/ done
Data model: 18 tables + verification codes, category seeder (41 categories) done
Account auth: register, email verification code, login/logout, personal tokens (flbp_, SHA-256 at rest, scoped, expiring, shown once) done
Public catalog API /v1/catalog/* (extensions, detail, releases, publishers, categories) with 60 s cache done
Docker image, compose stack (app, queue, MySQL, MinIO, Mailpit), CI (Pest), docker-bake.hcl aws-app target done
Package store (ReleaseStorage: sha256, sha1, SRI sha512), npm packument and tarball endpoints, Composer v2 repository, real-client conformance job in CI done
Licence gate on package routes (personal install token or flbi_ instance token; 401 without, 403 without licence) and @fleetbase/* / fleetbase/* namespace protection done
Instance API: registration codes, register, me, updates (compat ranges), installs with licence seats done
registry:import from the cloud registry-bridge tables: dry run, anomaly report, idempotent by source_uuid, tarball copy done

Next on this branch (Phase 1 Workstream B/C)

  • npm packument and tarball endpoints generated from releases
  • Composer v2 repository (/composer/packages.json, p2/{vendor}/{package}.json, dist streaming)
  • Shared licence gate for both package endpoints (401 without token, 403 without licence)
  • Instance API: register, me, updates, installs
  • Importer from the cloud registry-bridge tables (dry run implemented; weekly rehearsal against a cloud snapshot still to schedule)
  • Staging deploy (ECS task, SSM env, S3 bucket)
  • Storage, Sentry, health endpoint, Horizon
  • Fold Support\Compat::compare into Support\Semver (two semver comparators exist after merging the package and instance workstreams)

Later phases (separate PRs): releases pipeline and review, publisher website, CLI v2, installer v2, Stripe Connect and licensing, cloud cutover and decommission of legacy/.

Related work in flight

  • fleetbase/ember-core #99 and fleetbase/fleetbase #730: runtime engines, so a prebuilt engine release mounts without a console rebuild (Phase 3, decision D3).

  • fleetbase/registry-bridge phase-0/server-fixes and phase-0/central-catalog: stabilisation of the current bridge and an interim read of the central catalog for self-hosted Explore.

  • fleetbase/fleetbase-cli phase-0/cli-fixes: config file, --host, token handling, scaffold fixes.

  • This repo phase-0/harden (main-based, pending): secrets out of the tree, warn-level logs, no npmjs proxy for @fleetbase/*, JWT tokens, gitleaks workflow. Credentials found in history must be rotated before that branch is pushed.

Try it

composer install && cp .env.example .env && php artisan key:generate
php artisan migrate --seed && php artisan serve
# or: docker compose up --build
composer test   # 158 tests on v2 as of 22 Oct 2026

Decisions this branch assumes (D1–D6 in the plan): standalone Laravel service; Verdaccio retired; prebuilt engine dist as the install model (spike result: works with an exact ember-ui/ember-core pin and a runtime loader in the console); licences per organization with per-instance activations; publisher experience on registry.fleetbase.io; automated checks on every release, human review on first release and sensitive changes.

The v2 Registry Service takes over the repository root. The Verdaccio
build, compose file, config and plugin submodules keep working from
legacy/; the deploy workflow now runs the bake file from that directory.
Laravel 12 skeleton at the repository root (Laravel 11 is blocked by
Composer's security-advisory policy; fleetbase/core-api cannot be a
dependency because it pins illuminate ^9|^10 and sanctum 3.2.4 and
auto-loads conflicting migrations). The core-api conventions the plan
relies on are ported into the app with the same names and behaviour:
Model (UUID keys, findById), HasUuid, HasPublicId, Money cast, Filter,
FleetbaseResource and FleetbaseResourceCollection with the flat meta
pagination block.
Tables from the plan's data model: accounts, organizations,
memberships, tokens, categories, extensions, extension_packages,
releases, release_checks, reviews, prices, files, instances, installs,
licences, licence_activations, purchases and events, plus
verification_codes for email verification codes. Public id prefixes:
ext_, rel_, lic_, inst_, tok_ (and acct_, org_, cat_, file_, pur_).
Categories are seeded from registry-bridge's ExtensionsCategorySeeder.
- POST /v1/auth/register creates the account (pending_verification) and
  a personal organization with an owner membership, then emails a
  6-digit code; /v1/auth/verify activates it; /v1/auth/verify/resend
  never reveals whether an account exists.
- POST /v1/auth/login exchanges email and password (Hash::check, the
  same bcrypt scheme as core-api's Auth::checkPassword) for a personal
  token returned once; /v1/auth/logout revokes the current token.
- Personal tokens: flbp_ prefix, SHA-256 hashed at rest, scoped (read,
  publish, install, instance:register, *), expiring (max 365 days),
  revocable, last-use stamped; prefix + last four shown in lists.
  TokenGuard backs the auth:token guard; scope:<name> middleware.
- GET /v1/account returns the caller with organizations and roles.
- Fleetbase error envelope {errors: [...]} for API exceptions, rate
  limits on auth routes, Pest feature tests for every flow.
GET /v1/catalog/extensions (q, category, pricing, hosting, publisher,
tag, sort, page, limit), /v1/catalog/extensions/{slug},
/{slug}/releases, /v1/catalog/publishers/{slug} and
/v1/catalog/categories. Only published listings are visible; install
counts come from installs rows; fulltext search on MySQL with a LIKE
fallback elsewhere. Responses are cached for 60 seconds per URL with a
matching Cache-Control header and use Fleetbase-style resources with
the flat meta pagination block.
- Dockerfile: composer vendor stage, serversideup php 8.3 fpm-nginx
  runtime, app target for local use and aws-app target wrapped with
  ssm-parent like the v1 deploy; docker-bake.hcl builds aws-app.
- docker-compose.yml: app, queue worker, MySQL 8, MinIO (+ bucket
  init), Mailpit for verification emails.
- .env.example for a zero-dependency SQLite run; .env.docker for the
  compose stack.
- GitHub Actions CI: Pint and Pest on PHP 8.3 and 8.4, then a Docker
  build of the app target. The deploy/* workflow still builds the
  Verdaccio image from legacy/.
Comment thread .github/workflows/ci.yml
Comment on lines +14 to +54
name: Pest (PHP ${{ matrix.php }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
php: ["8.3", "8.4"]

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
extensions: mbstring, intl, bcmath, gd, zip, pdo_sqlite, sqlite3
coverage: none
tools: composer:v2

- name: Cache Composer packages
uses: actions/cache@v4
with:
path: ~/.composer/cache
key: composer-${{ runner.os }}-${{ matrix.php }}-${{ hashFiles('composer.lock') }}
restore-keys: composer-${{ runner.os }}-${{ matrix.php }}-

- name: Install dependencies
run: composer install --no-interaction --prefer-dist --no-progress

- name: Prepare environment
run: |
cp .env.example .env
php artisan key:generate --ansi

- name: Lint (Pint)
run: vendor/bin/pint --test

- name: Run tests
run: vendor/bin/pest --ci --colors=always

docker:
Comment thread .github/workflows/ci.yml
Comment on lines +55 to +72
name: Docker image builds
runs-on: ubuntu-latest
needs: tests
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build app target
uses: docker/build-push-action@v6
with:
context: .
target: app
push: false
cache-from: type=gha
cache-to: type=gha,mode=max
…ha1 and SRI sha512

Objects live at {prefix}/{file uuid}/{filename} under tarballs/, dist/, icons/ and
screenshots/ on the registry disk. Package tarballs keep their decoded package.json or
composer.json in files.meta so metadata endpoints never open archives. A small streaming
tar reader replaces PharData for that. Downloads answer If-None-Match and re-hash the
bytes as they stream. Fixture packages for the seed helper live in database/fixtures.
GET /npm/{name} builds the packument from published, non-yanked releases that carry an
engine tarball: versions from the stored package.json with authoritative name, version
and dist (tarball, shasum, integrity), dist-tags by SemVer precedence (latest = highest
stable, beta = highest beta, latest falls back to beta when nothing is stable), time,
and a content ETag with 304 support. Scoped names resolve whether npm encodes the slash
or not. GET /npm/{name}/-/{file}.tgz streams the tarball; yanked versions stay
downloadable for lockfiles. npm writes (publish, unpublish, dist-tag) get 405 pointing
at flb publish. Package routes return npm's error string beside the Fleetbase envelope.
…alls

Instance tokens (flbi_) go through TokenGuard with a kind check and the new
auth:instance guard. Installs claim licence seats, creating an unlimited free
licence on first install of a free extension. Updates compare installed
releases with the newest published release whose compat ranges accept the
instance's console and API versions (composer/semver).

Also adds source_uuid columns for the importer.
GET /composer/packages.json returns metadata-url and available-packages (every Composer
package with a published release). GET /composer/p2/{vendor}/{package}.json lists every
published release newest first in the composer/2.0 minified format, built from the
stored composer.json (root-only keys dropped) with dist {type: tar, url, shasum,
reference}; ~dev.json is always empty. GET /composer/dist/{vendor}/{package}/{version}.tar
streams the API archive. Versions Composer cannot parse are left out of p2.
licence:npm and licence:composer guard the packument, tarball, p2 and dist routes.
Free packages pass through. Paid packages need a bearer token: a personal token with the
install scope (its pinned organization, or every organization of the account) or an
instance token (instances.token_hash). Access is granted when one of those organizations
owns the extension or holds an active, unexpired licence; 401 with WWW-Authenticate
without a usable token, 403 otherwise. Paid metadata and archives are Cache-Control
private and vary on Authorization.

registry:seed seeds categories; with --seed-release it fabricates a published free and
a paid extension from the fixture tarballs, a buyer organization holding a licence and
an outsider without one, and prints install-scoped tokens for both (--json for scripts).
tests/Conformance/packages.sh drives real npm and Composer clients against a running
service with that seed.
Organizations gain an is_staff flag (not mass assignable; set with registry:staff).
extension_packages names in a reserved namespace (registry.reserved_scopes, matched on
the npm scope or Composer vendor, case-insensitively) can only be saved for an extension
whose organization is staff; creation, renames and moves to another listing are checked
and fail with a 422 on the name field. Package names are stored lowercase. The unused
REGISTRY_STAFF_ORGANIZATION setting is removed in favour of the flag.
…ints

The package-conformance job starts the service on SQLite with the local disk, seeds the
fixture releases and runs tests/Conformance/packages.sh: npm view, pack and install,
composer require with secure-http off, 401 and 403 for the paid package, and a licensed
install with each client.
Reads a MySQL DSN, a sqlite:// file or a .sql/.sql.gz dump (loaded into a
temporary SQLite file, no MySQL server needed) and maps registry extensions,
bundles, purchases, installs, developer accounts, cloud registry users,
companies and categories per the plan's mapping rules. Rows are keyed by
source_uuid so re-runs update in place. --dry-run runs the same mapping in a
rolled-back transaction and prints counts plus an anomaly report. Bundle
archives, icons and screenshots are copied with SHA-256 checksums; releases
without a retrievable tarball import as yanked.

Documents the instance API and the importer in the README.
# Conflicts:
#	README.md
#	app/Models/Instance.php
#	routes/api.php
#	tests/Pest.php
Comment thread .github/workflows/ci.yml
Comment on lines +55 to +112
name: npm and Composer conformance
runs-on: ubuntu-latest
needs: tests
# .env.example already runs on SQLite (database/database.sqlite) with the local disk;
# `artisan serve` reads .env rather than the job environment.
env:
PHP_CLI_SERVER_WORKERS: 4

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: "8.3"
extensions: mbstring, intl, bcmath, zip, pdo_sqlite, sqlite3, phar, zlib
coverage: none
tools: composer:v2

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "22"

- name: Cache Composer packages
uses: actions/cache@v4
with:
path: ~/.composer/cache
key: composer-${{ runner.os }}-8.3-${{ hashFiles('composer.lock') }}
restore-keys: composer-${{ runner.os }}-8.3-

- name: Install dependencies
run: composer install --no-interaction --prefer-dist --no-progress

- name: Prepare the service
run: |
cp .env.example .env
php artisan key:generate --ansi
touch database/database.sqlite
php artisan migrate --force

- name: Start the service
run: |
php artisan serve --host=127.0.0.1 --port=8000 > storage/logs/serve.log 2>&1 &
for i in $(seq 1 30); do curl -sf http://127.0.0.1:8000/up > /dev/null && exit 0; sleep 1; done
cat storage/logs/serve.log; exit 1

- name: Run npm and Composer against seeded releases
run: tests/Conformance/packages.sh http://127.0.0.1:8000

- name: Service logs
if: failure()
run: |
cat storage/logs/serve.log || true
cat storage/logs/laravel.log || true

docker:
Semver is now the one version comparator: normalize() pads loose versions
(v1.2 -> 1.2.0), compare() normalizes before applying SemVer precedence, and
isNewer() moves over from Compat. Compat keeps range evaluation and gains
validRange() for the release checks.
Publisher API (publish scope, owner/admin/publisher role): create listings
idempotently on package names with the namespace rules, edit listing fields,
upload releases (manifest, engine, api, optional prebuilt dist), submit,
withdraw, yank.

RunReleaseChecks inspects each archive once and records release_checks for
the manifest (extension.json JSON Schema shared with registry-bridge), compat
ranges, package names against extension_packages, version order per channel,
size limits, archive safety (traversal, links, devices, forbidden types,
native binaries), secrets (built-in gitleaks-style rules, plus gitleaks when
installed) and the build-engine-release dist manifest.

Release state machine with a reviews row per transition; review policy D6
(human review for first releases and package name, pricing or permissions
changes, auto-approve and publish otherwise); staff admin API for the queue,
claim, approve, reject with reason and items, publish and suspend. Publishing
sets published_at, lists draft extensions, flushes the catalog cache through
a generation key, and emails the uploader.
@roncodes

Copy link
Copy Markdown
Member Author

Superseded by #9 (release/v2.0.0 -> main). Project policy: every PR targets release/v2.0.0; nothing merges to main until the registry, registry-bridge, fleetbase-cli and registry-infra releases are all complete. The v2 branch stays as history; base new work on release/v2.0.0.

@roncodes roncodes closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants