feat: add omnibus installer for ubuntu/debian CADT setup - #1630
Open
TheLastCicada wants to merge 38 commits into
Open
feat: add omnibus installer for ubuntu/debian CADT setup#1630TheLastCicada wants to merge 38 commits into
TheLastCicada wants to merge 38 commits into
Conversation
Interactive bash installer for soup-to-nuts Chia, chia-tools, CADT, and nginx deployment. Includes shellcheck/bats lint workflow and e2e CI on ubuntu-22.04 and ubuntu-24.04 against testneta.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
apply_config_defaults did not generate a CADT_API_KEY when invoked from the fully-flagged (non-interactive) code path. A --yes install without --api-key left CADT without authentication. Generate one with openssl rand when the key is empty and not in read-only mode.
The ASCII art said "CADT Installed" (past tense) which made no sense at the start of an installation. Replace with clean "CADT" figlet art and "Automated Installer" subtitle.
Replace plain info() calls for phase transitions with a dedicated phase() helper that renders bold blue horizontal rules around the phase name, making section breaks stand out from regular output.
Route apt-get stdout/stderr directly to the install log file instead of the terminal. Users see a spinner and checkmark while the full output remains available in the log for debugging.
Piping the mnemonic to stdin does not work because chia's click-based CLI reads interactively from /dev/tty. Write the mnemonic to a chmod-600 temp file and pass it via --filename (-f) instead, with a label of "CADT" for identification.
Ensures systemd picks up any newly installed unit files before attempting to enable and start the chia services.
Nginx is already enabled and started by apt install. The explicit enable --now produced noisy SysV compat output and was unnecessary; we only need the reload to pick up the new site config.
The datalayer file server always runs on port 80 — it has no relation to the DataLayer protocol port. Remove the --datalayer-port flag and interactive prompt since it was a misleading configuration knob.
Set server_name to the configured DATALAYER_HOST instead of the wildcard '_' so nginx properly matches requests for the expected hostname.
Redesign the omnibus installer's nginx configuration: - Replace --datalayer-host with --public-address (strips http/https prefix automatically) - Add --local-only flag (skip CADT API proxy, still serve datalayer) - Add --https flag for Let's Encrypt certbot certificate issuance - Add --certbot-dry-run flag for CI testing - Nginx proxies CADT API on / and serves DataLayer files on /data/ - CADT binds to 127.0.0.1:31310 (not publicly exposed) - DATALAYER_FILE_SERVER_URL set to http(s)://<address>/data - Two-phase HTTPS setup: HTTP-only config first for certbot webroot validation, then full HTTPS config with 301 redirect - Interactive flow auto-detects public IP via ip.chia.net with validation loop and static IP warning - Fix duplicate API key line in final summary - Add E2E tests for proxy, /data/ path, and --local-only mode
- normalize_apt_version now converts -rc to ~rc for Debian versioning (previously was a no-op identity function) - Use systemctl restart instead of reload for initial nginx setup (nginx is not running yet on fresh installs, reload fails) - Remove duplicate validate_supported_apt_versions call
The CADT apt repository uses git tag names directly (e.g. 1.7.26-rc28) without Debian's ~rc convention. Revert normalize_apt_version to identity and remove denormalize_tag_from_apt which served no purpose. Simplify validate_supported_apt_versions to check only -rc patterns.
- Quote inner cat <<'PROXY' heredocs to prevent double-expansion of nginx variables ($host, $remote_addr, etc.) which crashed under set -u (nounset) in non-local-only mode - Fix numeric version picker for stable-only list: use inline jq filter matching the displayed stable-only set instead of pick_release_from_json which indexes all non-draft releases
main() already resolves VERSION_CHOICE → APT_VER before calling run_prompts(), so the inner branches checking VERSION_CHOICE were dead code. Simplify to direct prompt_version_choice calls.
- Add _require_arg guard to all space-separated flag variants to prevent unbound variable crash under set -u when value is missing - Use trap RETURN to ensure mnemonic temp files are always deleted even if chia keys add fails (prevents key material leak on disk) - Reset ENABLE_HTTPS after certbot dry-run so CADT config gets correct http:// URLs (no real cert exists in dry-run mode) - Default get_free_disk_gib to 0 when df output is empty
Address blind review, senior engineer, and QA findings: --yes validation, mnemonic extraction, temp file cleanup, expanded bats, and CI workflow hardening.
GitHub Releases publish before the apt build pipeline finishes, so a brand-new tag (e.g. cadt 1.7.26-rc29) makes verify_apt_package_version abort the install. Add verify_or_fallback_apt_version that downshifts to the highest published version on the same track (stable vs -rc*) that is also <= the requested pin, with a clear warning. Never upgrades past what the user requested. Also fix bugbot finding: integer-divided "7 GiB RAM" display contradicted the "7.5 GiB" requirements message. Add format_gib_from_kib so both lines render with one decimal place from the same helper.
Under set -Eeuo pipefail, a non-zero exit from `chia keys generate_and_print` propagates through the command substitution and trips the ERR trap before reaching the descriptive die for empty output. Add `|| true` to the pipeline so the empty-string check below produces "Failed to generate mnemonic" instead of a generic "Install failed at line N". Apply the same fix to grep-based selected_network parsing in get_chia_network_dir. Add a bats test that reproduces the failure with a fake chia binary returning non-zero, and asserts the descriptive die fires.
Avoid nesting the DataLayer server files directory when a prior installer attempt left the nginx web root behind.
Check the filesystem that will hold Chia and DataLayer state so systems with small root disks and SSD-backed Chia roots can pass preflight without later serving files from the wrong volume.
Drop the superseded exact-match verifier so installer version checks have a single fallback-aware implementation and matching focused test coverage.
Allow testneta installs to keep nginx and public serving configured while leaving DATALAYER_FILE_SERVER_URL blank so peers do not discover an unintended testing mirror.
Let testing-only installs proceed after the user confirms the warning while keeping the DataLayer file server URL blank for unadvertised mirrors.
Surface transient GitHub release API failures with actionable guidance and let exact version pins bypass release fetching entirely.
Handle long prerelease streams when resolving stable versions and make the wallet mnemonic display easier to identify during interactive installs.
Allow Chia CLI beta/prerelease selections by enabling the documented apt prerelease repository when needed, while keeping chia-tools pinned to stable packages and preserving a copy-friendly mnemonic display.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f4fb8e3. Configure here.
Use the shared prerelease classifier for CADT apt repo selection so beta and other prerelease pins resolve against the test package source.
These run in the existing bats job on every PR (no new CI infra) and fill the two cheap-to-verify gaps in option->outcome coverage: - the interactive version menus (prompt_version_choice) were only ever driven via preset flags, so the numbered "choose option N" dispatch had no coverage; queue-stubbing prompt_default exercises stable, pre-release, pick-from-list, and the invalid-selection die paths - nginx config generation was only checked end-to-end; asserting the rendered text guards the --local-only contract (drop the reverse proxy, keep the /data/ alias) and the HTTPS 443/redirect block
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tools/install-omnibus.sh, an interactive bash installer that sets up Chia CLI, chia-tools, CADT, and nginx for DataLayer file serving on Ubuntu/DebianTest plan
bash -n tools/install-omnibus.shbats tools/test/test_install_omnibus.bats./tools/install-omnibus.shon a fresh Ubuntu VM (optional)Note
Medium Risk
The installer mutates apt, systemd, nginx, and Chia/CADT config on target hosts and handles wallet mnemonics/API keys; risk is mitigated by guards, tests, and CI e2e, but production misconfiguration or apt pin fallbacks could still affect deployments.
Overview
Adds
tools/install-omnibus.sh, a new Ubuntu/Debian installer that wires up Chia CLI, chia-tools, CADT, nginx (DataLayer/data/plus optional CADT reverse proxy), systemd services, wallet key generate/import, and CADTconfig.yamlpatching. It supports interactive prompts or--yeswith explicit flags (network, public address, mnemonic file, version pins,--local-only, HTTPS/certbot, read-only mode, etc.), refuses root and repeat installs, and hardens mnemonic handling (no/dev/null, atomic writes, secrets off the tee’d log).CI & tests: New Install Omnibus Lint workflow (
bash -n, shellcheck, shfmt, bats) and Install Omnibus E2E (full testneta install on Ubuntu 24.04 for PRs; 22.04+24.04 weekly) covering service health, nginx/CADT health, config assertions, idempotent re-run failure,--local-only, and root rejection. Large bats suite with JSON fixtures and a localtools/test/lint.sh.Docs: README Tools section documents the script and Multipass-based local testing.
Reviewed by Cursor Bugbot for commit 0f3d558. Bugbot is set up for automated code reviews on this repo. Configure here.