MVP - #1
Open
gregory-schiano wants to merge 248 commits into
Open
Conversation
gregory-schiano
commented
Jun 19, 2026
gregory-schiano
commented
Owner
- Removed Harbor for an embedded OCI registry
- Added many features and security components
- Added a Rock, a charm and a snap
Rewrite all documentation to accurately reflect the real feature set from the use-harbor branch, covering the embedded OCI Distribution v2 registry (not Harbor), TLS, snap, Rock, configuration, operations, testing, API compatibility, and known limitations. Changes: - README.md: fix broken .env.example link, add snap/rock deployment sections, clarify embedded OCI (not Harbor), document API TLS gap, expand current limitations from the aligned roadmap - SECURITY.md: add OCI credential security model, snap confinement details, startup validation behavior - .env.example: add OCI_S3_*, API TLS, and OCI_S3_USE_PATH_STYLE variables; add explanatory comments for gaps - docs/architecture.md: process layout, data flows, OCI credential lifecycle, authentication model - docs/configuration.md: complete env var reference with defaults, snap config mapping, legacy alias table - docs/deployment.md: Docker Compose, snap, and Rock instructions; snap TLS gap documented; production hardening checklist - docs/api-compatibility.md: full endpoint table for publisher and consumer APIs, compatibility notes on auth, upload flow, libraries - docs/operations.md: health checks, monitoring gaps, backup/restore procedures, token management, OCI credential rotation, upgrade, certificate management - docs/testing.md: unit tests, static analysis targets, fuzz tests, integration test roadmap
52 test scenarios across 12 categories: BOOT (6), TOKEN (10), PKG (9), REV (6), V2 (8), RES (7), ACL (7), LIMIT (5), PERSIST (3), JUJU (5), OCI (5), SNAP (2). Documents expected failures for known gaps (C-1, R4, R2, M-17). Maps every scenario to code review backlog and feature roadmap coverage.
Tests cover: - REV-01: Full upload-to-release pipeline (register, upload, push, review, release, verify) - REV-02: Upload charm with resources (resource declarations in metadata) - REV-03: List revisions (2 revisions, field checks, revision filter query param) - REV-04: Release to multiple channels (latest/stable + latest/edge) - REV-05: Create and use custom track (2.0 track, release to 2.0/stable) - REV-06: Charm download (verify downloaded archive matches upload) - REV-07: Unscanned upload returns upload-id (raw endpoint verification) - REV-08: Push revision returns status-url - REV-09: Release endpoint returns released array with 201 - REV-10: Revision listing includes metadata fields (sha3-384, size, bases, created-at)
Add production-grade GitHub Actions workflows covering the full CI/release
pipeline for the charm-registry Go service:
1. ci.yml (enhanced):
- lint, security (govulncheck + gosec), unit tests with 70% coverage
threshold, build, and Docker image verification
- Upload binary and coverage artifacts for downstream consumption
- Required status checks gate job for branch protection
2. integration.yml (new):
- Full Docker Compose stack boot (postgres + charm-registry)
- Health check with 60s timeout before running integration tests
- Runs tests/integration/ with -tags=integration
- Log collection on failure, cleanup on always
3. release.yml (new):
- Semver tag-triggered (vX.Y.Z and vX.Y.Z-rcN)
- Tag format validation gate
- Full audit + coverage before building artifacts
- Cross-compiled binaries (linux/darwin x amd64/arm64)
- Multi-platform container image (linux/amd64 + linux/arm64) pushed
to GHCR with semver + SHA tags
- Cosign keyless image signing
- Snap build via snapcraft --destructive-mode
- SBOM generation (SPDX + CycloneDX) via syft
- GitHub release with auto-generated notes, draft for RC tags
4. Makefile: add integration-test target (go test -tags=integration)
5. Dockerfile: use TARGETOS/TARGETARCH build args for multi-platform
support (required by docker buildx for arm64 builds)
6. Cherry-picked integration tests (revision_test.go REV-01 through
REV-10) from prod-ready-use-harbor/integration-tests branch.
All actions pinned to SHA; explicit permissions on every workflow/job.
The DATABASE_URL was hardcoded without variable interpolation,
preventing overrides via .env or environment. Now uses
${CHARM_REGISTRY_DATABASE_URL:-default} syntax for consistency
with all other config variables.
…(C-1) Previously, requirePermission returned nil for tokens with an empty Permissions slice, granting full access. Now OIDC direct auth (Token == nil) still bypasses (intentional Charmhub compat), but a scoped token with empty Permissions is correctly forbidden. Added unit test TestGetPackagePublicWithEmptyTokenPermissionsIsForbidden.
…C-2) - deriveKey now returns ([]byte, error) instead of panicking, making OCI client startup safe for production. - New() propagates deriveKey errors instead of crashing. - Added key version prefix (v1:) to encrypted output for forward-compatible key rotation. - decrypt strips version prefix and rejects mismatched versions. - Updated test to use new deriveKey signature.
PostgreSQL migrations now use a schema_migrations table for version tracking, an advisory lock to prevent concurrent races, and per-migration transactions. Already-applied migrations are skipped, making Migrate() safely idempotent.
Add CHARM_REGISTRY_API_TLS_CERT_FILE and API_TLS_KEY_FILE config. When both are set, the API server serves HTTPS directly; otherwise it serves HTTP with a startup warning recommending a reverse proxy. Validation ensures cert and key are set together.
compose.yaml default for ENABLE_INSECURE_DEV_AUTH changed from true to false. Startup emits a prominent warning when dev auth is enabled. This prevents accidental deployment with insecure auth bypass.
- Replace SHA-256 token hashing with bcrypt for new tokens - Add token_prefix field for efficient bcrypt token lookup - Add token_hash_scheme field to track hash algorithm per token - Automatic upgrade: SHA-256 tokens are re-hashed to bcrypt on next auth - Legacy SHA-256 lookup still works for existing tokens - PG migration 0006 adds token_prefix and token_hash_scheme columns - SQLite migration 0002 mirrors the PG schema change - Update postgres, sqlite, and memory repo implementations - Add FindStoreTokenByPrefix and UpdateTokenHashScheme methods - Update all auth tests for bcrypt token format
…miter, wrap OCI errors
H-4: sanitizeFilename strips quotes, backslashes, CRLF, control chars;
truncates to 255 chars; used in writeAttachment Content-Disposition.
H-5: ipRateLimiter with sliding window (30 req/min per IP) added as
rateLimit middleware on chi router after securityHeaders.
H-6: ensureOCIProvisioned in service/oci.go and sync/oci.go now logs
errors with slog.ErrorContext and wraps root cause in error message.
…pers.go H-8: packagesEqualForOCI, ociPackageProvisioned, robotCredentialReady, robotEqual, and timePtrEqual were duplicated between service/oci.go and sync/oci.go. Moved to core.OCIPackageProvisioned, core.PackagesEqualForOCI, core.RobotCredentialReady, core.RobotEqual, core.TimePtrEqual. Both packages now import from core.
The generate-certs.sh script was creating private keys with chmod 0644 (world-readable). Change to 0640 (owner+group only). Production deployments should mount tighter secrets via Docker secrets or vault.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add longer targeted retries for rock builds when chisel hits the known archive digest mismatch, and extend the artifact build timeout so the retry window can outlast archive publication races. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Upgrade the Terraform stack to the provider's model_uuid API and OCI resource support, and wire integration modules to application outputs so native juju_integration resources wait for their applications to exist. Also collect bounded per-model Juju debug logs on integration failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Revert the temporary timeout increases now that the Terraform root cause is fixed. Keep the provider-native Terraform fix and the targeted Chisel archive-race retry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.