diff --git a/.gitattributes b/.gitattributes index 1caa4ca..fac83de 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,3 +1,4 @@ # Container entry points and Linux lifecycle scripts must retain Unix line endings. *.sh text eol=lf Dockerfile text eol=lf +docker/* text eol=lf diff --git a/CHANGELOG.md b/CHANGELOG.md index 9e03d81..4b315ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -41,6 +41,8 @@ After every task with file changes, briefly record what changed and why: feature ### Project +- 2026-10-28: require LF checkout for standalone Docker assets and verify nginx's actual Docker HEALTHCHECK independently of DB availability. Windows CRLF conversion made the original locally published nginx health payload fail; canonical Git LF export is required for corrected image packaging. +- 2026-10-28: published GitHub release v1.2.0 and mooship 1.2.0 on PyPI through Trusted Publishing; verified a fresh public package install and real checksummed toolkit installation. Merged PRs #11/#6 after successful CI/E2E, corrected main's Go-specific required checks with owner approval, and synchronized verified publication/security facts in local continuity documents. Published and verified the PHP image and corrected nginx-r1 companion in Docker Hub; the original nginx tag is superseded after reproducing its Windows CRLF health defect. - 2026-10-28: enabled repository Dependabot alerts/Dependency Graph and verified Dependency Review now passes. Added strict release tag-to-checkout verification and a Docker Hub overwrite guard; configured the user-confirmed PyPI pending publisher and GitHub environment/variable for tokenless publication. - 2026-10-28: fixed Linux image-test secret fixtures: their bind-mounted temporary directory now permits traversal by runtime UID 1000. Docker Desktop's Windows bind handling had hidden the directory-permission failure reproduced by GitHub's Linux runner. - 2026-10-28: prepared Mooship 1.2.0 distribution: a dependency-free Python CLI modeled on LaraShip, exact-release HTTPS/checksum/archive validation, native command forwarding, reproducible toolkit assets, wheel/sdist checks, Python compatibility CI, and gated Docker Hub/PyPI release workflows. Extended lint/security/dependency checks to the Python package and documented account setup and publication evidence. Existing application images remain separate from host administration. diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md index d5fa9d6..9a7f330 100644 --- a/PROJECT_CONTEXT.md +++ b/PROJECT_CONTEXT.md @@ -5,7 +5,7 @@ Last verified: 2026-10-28. Sources: the user's instructions, local Git metadata, ## Project overview - **Mooship** automates Moodle installation on Ubuntu with Docker Compose and a shared reverse proxy. -- This directory is a Git repository with `origin` set to **shellharbor/mooship**, `https://github.com/shellharbor/mooship.git`. The latest locally inspected commit on 2026-10-15 is `cddc89b` (subject `v1.1.2`); earlier implementation baselines include `b44b4e1` (`v1.1.2`), `d553e0d` (`v1.1.0`), and `aafd118` (`v1.0.0`). On 2026-10-15, the GitHub API confirmed `main` as the default branch, MIT as the license, tags `v1.1.0` / `v1.0.0`, and `v1.1.0` as the latest published release; a commit subject is not evidence of a tagged release. +- Git origin is shellharbor/mooship. Local main and origin/main contain release commit `82a658205bfdeaaa68026f924db2305bd7976b29`, tagged `v1.2.0`. GitHub confirmed v1.2.0 as the latest published release on 2026-10-28; PRs #11 and #6 are merged. Earlier primary baseline: cddc89b (v1.1.2 commit subject). - Workflows: creating a site, activation/deactivation, explicit removal, full backups, and restoration into an unused directory. - Project language: English throughout documentation, code comments, CLI help, prompts, log/error messages, configuration templates, examples, and local agent instructions. Keep future additions in English, as requested by the user. - Chat language: Russian, according to the user's current language policy; project artifacts shown in chat retain English. @@ -28,20 +28,20 @@ Last verified: 2026-10-28. Sources: the user's instructions, local Git metadata, ## Standalone Moodle runtime (2026-10-15) -- User chose Moodle/PHP in an application image, with nginx and DB separate, for later Docker Hub publication. Root `Dockerfile` builds `app` (default) and `web` targets; local tags are `shellharbor/mooship:dev-moodle4.5.15-php8.3` and `shellharbor/mooship:dev-moodle4.5.15-nginx`. Both build successfully on Linux amd64. Publication, Git pushes and a release were not performed. +- Published Docker Hub pair (2026-10-28), Linux amd64: `1.2.0-moodle4.5.15-php8.3` and corrected `1.2.0-moodle4.5.15-nginx-r1` under shellharbor/mooship. App digest: `shellharbor/mooship@sha256:70387b904527873f32f948eae36e23c4b0f2d48864a4a831b4e457e82810fe77`; web digest: `shellharbor/mooship@sha256:4a37ae5202bea0569cc5eae4adf5e43f6cf21eaaaa1b08d23cf7125737da63f8`. Both use the unchanged v1.2.0 source at 82a6582. Original nginx tag is superseded: Windows CRLF conversion broke its exact Docker health probe. Corrected nginx-r1 is exported from Git with core.autocrlf=false. CLI login as shellharbor resolved the earlier insufficient_scope failures. - Moodle 4.5.15 is pinned to commit `0a694ba9b8b4183bf7a45094cfa7211030db3192` and archive SHA-256; PHP 8.3 Bookworm and nginx 1.31.0 Alpine bases are digest-pinned. Both images contain the same immutable source. Moodle and dependency licenses remain bundled; wrapper code remains MIT. - UID/GID 1000; writable `/var/moodledata` and `/tmp`, read-only roots, no capabilities/Docker socket/source mount required. Credentials use environment or absolute `_FILE` secrets; only the install workload receives the admin secret. Commands: `fpm`, explicit locked `install`, `cron`, `cron-once`, `check`, and passthrough `php ...`. Normal startup does not migrate schemas and refuses an absent/mismatching installed version. - `docker/compose.yml` plus the chosen `compose.postgres.yml` or `compose.mysql.yml` overlay supplies PostgreSQL 17/MySQL 8.4 and a separate scheduler. Only nginx publishes HTTP, on loopback 8080 by default. Runtime settings/secrets, backup/upgrade differences from the host toolkit and later publication commands are in `docs/DOCKER.md` (ADR-0004). - FPM liveness is a FastCGI ping; nginx liveness is `/healthz`; `/readyz`/`mooship check` query DB/schema readiness. Docker shutdown uses SIGQUIT. Kubernetes examples use stable APIs, non-root security contexts, DB/admin Secret references, external DB, RWX data PVC, PHP/nginx Deployment, explicit install Job and independent CronJob. Resource values are estimates. All six resources pass strict Kubernetes 1.34 schema validation; no cluster deployment/server dry-run was performed. -- `.github/workflows/image.yml` adds a ninth workflow: local image builds, PHP syntax, Compose/schema checks and opt-in real Moodle tests. It never publishes. Dependabot now discovers the root Dockerfile and runtime Compose; Dockerfile/YAML lint includes these paths. Existing host deployment/lifecycle/backup scripts are independent. +- Eleven repository workflows include build/test-only image.yml, Python package build/install checks, and release.yml. Release gates require matching VERSION/tag/commit and successful exact-commit workflows. PyPI Trusted Publishing is configured and verified. The optional Docker Hub CI job remains disabled until a dedicated DOCKERHUB_TOKEN and publishing variables are configured; this first image publication uses the local Docker CLI login. Host deployment/lifecycle/backup scripts remain independent of the application images. - Validation outside the opt-in image suite: 89 Linux regressions passed (101 discovered, 12 skipped); PHP syntax, ShellCheck, Ruff, YAML, actionlint, zizmor, documentation links, both runtime Compose variants, and strict Kubernetes 1.34 schema checks passed. Hadolint reported no errors; package-version/pipeline-style warnings remain visible. No GitHub-hosted run was claimed. -- Final image validation: all three `MOOSHIP_IMAGE_TESTS=1` tests passed on Docker Desktop Linux amd64 (351.508 seconds). Real Moodle installs passed on PostgreSQL 17 and MySQL 8.4, with file-based app/admin secrets, HTTP login/protected paths, cron, repeated-install and version refusal, DB outage/recovery, independent liveness and graceful app/web shutdown. Initial MySQL fixtures exposed upstream initializer SQL quoting limits; final DB fixtures avoid quote/backslash initialization while the secret reader/admin password still exercise them. Both image builds remain available locally. Additional architectures, production TLS/SMTP/Redis, host installer full E2E and a live Kubernetes deployment were not tested by this suite. +- Image checks (2026-10-28): the final PHP image passed the three real PostgreSQL/MySQL integration checks in 514.546 seconds. A new actual nginx Docker HEALTHCHECK regression reproduced failure of the original Windows-built web tag, then passed with nginx-r1; real Moodle/PostgreSQL integration also passed with r1 (two checks, 184.139 seconds). Require LF Docker assets and canonical Git LF exports for release builds from Windows. Additional architectures, production TLS/SMTP/Redis and live Kubernetes remain unverified. ## Working with the project -- Distribution completion (2026-10-28): user requested Docker Hub publication and a Python CLI following neighboring LaraShip. Current work is on `codex/distribution-and-publishing`, preserving earlier uncommitted image sources. VERSION proposes 1.2.0; `src/mooship/` provides dependency-free Python 3.8+ installation and native script forwarding. Exact-release installation verifies HTTPS, SHA-256, archive safety/limits and VERSION, refuses existing destinations and marks failed copies. Root `pyproject.toml`, package/release workflows and `docs/PYTHON.md`/`docs/RELEASING.md` are added. The runtime Docker images remain application images, without a host-controller backend. Docker daemon is available; both earlier dev images exist locally. Docker Hub repository exists but has no tags; PyPI mooship returns 404. Local Docker credentials exist for tigusigalpa; GitHub Actions publishing secrets/variables and PyPI Trusted Publisher are not yet configured. User corrected an initial setup confirmation and explicitly said PyPI is not configured. Registry publication must be verified separately; no 1.2.0 release or upload is claimed yet. +- Distribution completion (2026-10-28): GitHub v1.2.0 and PyPI mooship 1.2.0 are published. `src/mooship/` is a dependency-free Python 3.8+ adapter following LaraShip's verified-release native installer design. Its CLI forwards existing scripts through exec, retaining terminal I/O, signals and exit codes. HTTPS/checksum/archive/VERSION validation, exclusive destinations and incomplete-install markers protect toolkit installation. Release workflow 40694109694 successfully uploaded wheel/sdist and native GitHub assets through the configured pypi Trusted Publisher. A fresh Linux container installed the package from public PyPI, downloaded/verified/installed the public toolkit and ran backend help successfully. See docs/PYTHON.md, docs/RELEASING.md and ADR-0006. Docker publication is tracked in the standalone runtime section. -- Host preflight implementation originated on `codex/deployment-preflight-readiness`, in `.ci-reports/deployment-preflight`, [PR #6](https://github.com/shellharbor/mooship/pull/6). On 2026-10-28 it was merged into `codex/distribution-and-publishing` at `d864b7f`, preserving the image and package work; primary main is not yet updated. The implementation adds port/native DB/readiness gates, structured installation checks, deferred cron and archive compatibility; fixes nginx file identity/reload/PHP shims; and installs declared locked production dependencies with a pinned verified Composer tool. Earlier local checks passed 108 regressions (120 discovered, 12 opt-in skipped), Compose/proxy checks, real PostgreSQL/MySQL host-gateway probes, all six HTTP/HTTPS scenarios and linters/docs. [Full E2E run 38787890321](https://github.com/shellharbor/mooship/actions/runs/38787890321) tested `ff1fbf2` and passed all four Moodle 4.5/5.3 x PostgreSQL/MySQL cases. Dependency Review separately reported disabled repository Dependency Graph. On 2026-10-28 the combined committed snapshot passed 130 Linux regressions (145 discovered, 15 opt-in skipped); package wheel/sdist metadata and actual native archive checksum/extraction/VERSION checks passed. New registry uploads and GitHub runs remain separate checks. +- Host preflight and standalone distribution work are integrated into main through PR #11; PR #6 was merged automatically as its commits became reachable. Full E2E run 40689536067 passed all four Moodle 4.5/5.3 x PostgreSQL/MySQL cases at aa9df39, including SMTP to Mailpit, backups, schedules, routing and doctor. Later commits changed release guards and a test fixture, not native runtime behavior. The final regression matrix passes 131 tests (146 discovered, 15 opt-in skipped) on Python 3.8/3.12/3.14; helper statement/branch coverage is 93%. All 26 PR checks pass. Native DB routing, port/readiness gates, deferred cron, nginx inode/reload/shims and locked production dependency checks are included. - Installation: `sudo bash deploy-moodle.sh --slug lms --domain lms.example.com --db-type postgres --ssl-email admin@example.com`; see `README.md` for the full guide. - Deployment requires Docker. Check daemon availability before container-based validation; daemon status is a temporary fact. @@ -71,6 +71,7 @@ Last verified: 2026-10-28. Sources: the user's instructions, local Git metadata, - `restore.sh` accepts a trusted v1 archive created by Mooship and an unused slug matching the archive; the user must type the slug to confirm. Native databases/users must not already exist. `remove.sh` preserves backups; native DROP requires a matching `.mooship-native-database` marker, written after successful database/user creation. Older native projects without the marker retain their databases for manual review. - Local AI-KIT container convention: new Dockerfiles, containers, and microservices must be Kubernetes-ready under the local `ai-kit/CONTAINERS.md` standard when available in the checkout. Kubernetes availability in this project must be verified separately. - Image synchronization agreement (2026-10-15): assess image impact for behavior, dependency and configuration changes across host installation and standalone runtime. Synchronize affected sources/build definitions, configuration, manifests, tests and documentation in the same task, rebuild affected targets, and run relevant runtime checks. Shared Moodle source/plugins must match in `app` and `web`; preserve each deployment option's supported versions. Verified host-only/documentation changes need no rebuild. Report impact and actual checks, distinguishing source updates, builds, publication and deployment. See `docs/DOCKER.md#keeping-images-in-sync` and root `SKILL.md`; this agreement changes instructions only and does not imply a new build or publication. +- Repository security (2026-10-28): Dependabot alerts/Dependency Graph are enabled and Dependency Review passes. Protect main ruleset 25110357 requires 26 real Mooship checks instead of four nonexistent Go checks; one approving review and strict checks remain active. Administrative merges require explicit owner authorization. CodeQL alerts #2/#3/#4 are individually owner-approved used-in-tests dismissals for generated disposable credentials and a byte-digest comparison; application scanning stays active. Keep required check names synchronized with workflow job/matrix changes. - Architecture decisions: see `docs/DECISIONS.md` and `docs/adr/`. ## Open questions diff --git a/README.md b/README.md index 7a33f8c..5c94f00 100644 --- a/README.md +++ b/README.md @@ -17,9 +17,9 @@ Automated Moodle deployment on Ubuntu. Chosen GitHub repository: [shellharbor/mo The PHP version is selected automatically based on the Moodle version. Passwords, database names, and available ports are generated unless supplied explicitly. -For a portable application image, see the [standalone Moodle Docker guide](docs/DOCKER.md). The root Dockerfile builds Moodle 4.5.15/PHP 8.3 and a matching nginx companion, with separate PostgreSQL/MySQL and persistent data. Compose examples, Kubernetes manifests, installation tests, and later Docker Hub publication instructions are included. Build these images locally until published; the host installer below is an independent deployment option. Bundled Moodle retains its GPL license. +For a portable application image, see the [standalone Moodle Docker guide](docs/DOCKER.md) and [Docker Hub](https://hub.docker.com/r/shellharbor/mooship). Mooship 1.2.0 provides Moodle 4.5.15/PHP 8.3 and the corrected `nginx-r1` companion, with separate PostgreSQL/MySQL and persistent data. Compose examples, Kubernetes manifests and installation tests are included. The host installer below is an independent deployment option. Bundled Moodle retains its GPL license. -The [Python CLI](docs/PYTHON.md) provides a dependency-free `mooship` command for verified toolkit installation and the existing deployment, lifecycle, backup/restore and diagnostics scripts. Version 1.2.0 introduces packaging for pip/pipx; use `pipx install .` from a reviewed checkout until publication. See [release publication](docs/RELEASING.md) for Docker Hub and PyPI setup and verification. +The [Python CLI](docs/PYTHON.md) provides a dependency-free `mooship` command for verified toolkit installation and the existing deployment, lifecycle, backup/restore and diagnostics scripts. Install the published package with `pipx install mooship==1.2.0`. See [release publication](docs/RELEASING.md) for Docker Hub and PyPI setup and verification. Deployment validates ports and service readiness before publishing a site. For native databases, `--db-host HOST` selects the container-reachable address (default `host.docker.internal` with Linux host-gateway mapping). See [preflight, native database access, and readiness](docs/PREFLIGHT.md) for requirements and checks. diff --git a/WIKI.md b/WIKI.md index 64b99ff..8495fa8 100644 --- a/WIKI.md +++ b/WIKI.md @@ -4,6 +4,8 @@ Project knowledge map. Public links target files included in the repository. Opt ## Getting started +- [Mooship v1.2.0 release and native toolkit assets](https://github.com/shellharbor/mooship/releases/tag/v1.2.0) +- [Published Python CLI on PyPI](https://pypi.org/project/mooship/1.2.0/) - [README and operations guide](README.md) - [Standalone Moodle images, Compose, Kubernetes, and Docker Hub](docs/DOCKER.md) - [Image impact assessment, synchronization, rebuilds, and checks](docs/DOCKER.md#keeping-images-in-sync) diff --git a/docs/DOCKER.md b/docs/DOCKER.md index 03f0dee..fc159b6 100644 --- a/docs/DOCKER.md +++ b/docs/DOCKER.md @@ -1,12 +1,20 @@ # Standalone Moodle images -The root Dockerfile builds a Moodle 4.5.15/PHP 8.3 runtime and a matching nginx companion for later publication as `shellharbor/mooship`. PostgreSQL 17 or MySQL 8.4 runs separately. The existing host installer is an independent deployment option. These application images do not manage Docker or host directories. +The root Dockerfile builds a Moodle 4.5.15/PHP 8.3 runtime and a matching nginx companion published as `shellharbor/mooship`. PostgreSQL 17 or MySQL 8.4 runs separately. The existing host installer is an independent deployment option. These application images do not manage Docker or host directories. -Build these images locally until they are published. The `dev-*` tags below identify local builds, not an established release. Runtime verification covers Linux amd64; other architectures require their own build and installation checks. +The published 1.2.0 pair is `1.2.0-moodle4.5.15-php8.3` and `1.2.0-moodle4.5.15-nginx-r1`. Use the corrected `nginx-r1` companion: the original nginx tag contains a Windows CRLF health response that fails Docker's health check. The `dev-*` tags below identify local source builds. Runtime verification covers Linux amd64; other architectures require their own build and installation checks. ## Build and install with Compose -Run from the repository root on a Linux Docker host with Compose v2+. Docker Desktop can build/test the images; production secret-file ownership examples assume Linux. +Run from the repository root on a Linux Docker host with Compose v2+. Docker Desktop can build/test the images; production secret-file ownership examples assume Linux. To use the published pair, export the image references and copy the configuration example: + +```bash +export MOOSHIP_APP_IMAGE=shellharbor/mooship:1.2.0-moodle4.5.15-php8.3 +export MOOSHIP_WEB_IMAGE=shellharbor/mooship:1.2.0-moodle4.5.15-nginx-r1 +cp docker/.env.example docker/.env +``` + +Alternatively, build the development tags from source without the above image overrides: ```bash docker build --target app -t shellharbor/mooship:dev-moodle4.5.15-php8.3 . @@ -108,6 +116,8 @@ Run the final command only after installation succeeds. Remove the admin Secret ## Keeping images in sync +Keep container assets LF-only, including `docker/health.txt`; a CRLF response can pass a stripped HTTP assertion while failing Docker's exact liveness probe. The runtime suite checks the actual nginx Docker health state independently of the database. For a release build from Windows, export the tagged inputs using `git -c core.autocrlf=false archive` into a clean build context; ordinary Windows checkout/archive conversion can otherwise change the image bytes. + Assess image impact whenever behavior, dependencies, or configuration changes. The host installer and standalone runtime are separate implementations; inspect both when the same behavior applies to them, respecting their supported Moodle/PHP versions. | Changed inputs | Image impact | diff --git a/docs/PYTHON.md b/docs/PYTHON.md index 6ff7feb..c3df14f 100644 --- a/docs/PYTHON.md +++ b/docs/PYTHON.md @@ -4,7 +4,7 @@ Install Mooship through pip or pipx, then use one command for its existing Moodl ## Install the CLI -Version 1.2.0 introduces Python packaging. Until it is published, install a reviewed checkout with `pipx install .` or `python -m pip install .` in a virtual environment. After publication: +Version 1.2.0 is published on [PyPI](https://pypi.org/project/mooship/1.2.0/). A reviewed checkout can also be installed with `pipx install .` or `python -m pip install .` in a virtual environment. For the published CLI: ```bash pipx install mooship==1.2.0 diff --git a/docs/RELEASING.md b/docs/RELEASING.md index 5d3e8ce..db4799e 100644 --- a/docs/RELEASING.md +++ b/docs/RELEASING.md @@ -23,6 +23,8 @@ shellharbor/mooship:1.2.0-moodle4.5.15-nginx PHP and nginx must use the matching Moodle source. No `latest` alias or untested architecture is published. Record the resulting digests and prefer digest references in production. The [Docker guide](DOCKER.md#tests-and-later-publication) also supports a checked manual publication using an existing local Docker login; always set MOOSHIP_VERSION and VCS_REF to the release version and exact source commit. +For the initial 1.2.0 publication, use `1.2.0-moodle4.5.15-nginx-r1`: it corrects a Windows CRLF packaging defect in the original nginx tag's health response. The corrected image uses the same tagged source exported with `core.autocrlf=false`, and preserves the original tag/digest. The PHP tag is unchanged. Check the actual Docker health state before publishing; an HTTP 200 assertion alone is insufficient. + ## PyPI Configure the pending publisher described in [the Python guide](PYTHON.md#pypi-publication), create GitHub environment **pypi**, and set variable `PYPI_PUBLISH=true`. The isolated publish job receives only built distributions and short-lived OIDC credentials. No long-lived PyPI token or build-source checkout is needed in that job. diff --git a/tests/test_runtime_image.py b/tests/test_runtime_image.py index e17c49f..3a88ca7 100644 --- a/tests/test_runtime_image.py +++ b/tests/test_runtime_image.py @@ -76,6 +76,21 @@ def runtime_options(self, volume, engine="pgsql"): "--env", "MOODLE_WWWROOT=http://web:8080", ] + def test_web_liveness_is_healthy_without_a_database(self): + name, result = self.run_container("web-liveness", WEB_IMAGE, options=[ + "--read-only", "--cap-drop=ALL", "--security-opt=no-new-privileges:true", + "--tmpfs", "/tmp:uid=1000,gid=1000,mode=1770", + "--env", "MOOSHIP_FPM_UPSTREAM=127.0.0.1:9000", + ], detached=True) + self.assert_ok(result) + deadline = time.monotonic() + 60 + while time.monotonic() < deadline: + status = self.assert_ok(docker("inspect", "--format", "{{.State.Health.Status}}", name)).strip() + if status == "healthy": + return + time.sleep(2) + self.fail("nginx Docker HEALTHCHECK did not become healthy: " + self.assert_ok(docker("logs", name))) + def test_image_contract_and_invalid_configuration(self): self.password = "Test_'$\\&-" + uuid.uuid4().hex (Path(self.secrets.name) / "db_password").write_text(self.password + "\n", encoding="utf-8")