feat: embedded web UI — dashboard + on-device 2FA re-auth flow#464
Open
epheterson wants to merge 20 commits into
Open
feat: embedded web UI — dashboard + on-device 2FA re-auth flow#464epheterson wants to merge 20 commits into
epheterson wants to merge 20 commits into
Conversation
|
🐳 Docker Image Built Successfully! The Docker image for this PR has been built and pushed to GitHub Container Registry: Image Tag: Usage: docker pull ghcr.io/mandarons/icloud-docker:pr-464
docker run --name icloud-pr-test -v ${PWD}/icloud:/icloud -v ${PWD}/config:/config -v ${PWD}/home-abc-local:/home/abc/.local -e ENV_CONFIG_FILE_PATH=/config/config.yaml ghcr.io/mandarons/icloud-docker:pr-464Build Info:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an embedded Flask-based web UI (src/web.py) that runs in a daemon thread alongside the sync loop, providing a dashboard, log tail, on-device 2FA re-authentication, and a "force sync now" button. Cross-thread signalling between the sync loop and the web thread is handled via filesystem sentinels and a JSON state file in src/web_signals.py. The feature is opt-in via new app.web_ui.* config keys (default OFF).
Changes:
- New
src/web.pyFlask app (dashboard,/auth/*,/api/{health,status,logs,sync}) with Jinja2 templates; opt-in config helpers added tosrc/config_parser.py. - New
src/web_signals.pyfor force-sync sentinels and last-sync-state JSON;src/sync.pynow consumes sentinels and records completion stats. src/main.pygains a testablerun()entry;DockerfileaddsEXPOSE 8080;flask==3.0.3added torequirements.txt; tests added for web, web_signals, and sync integration; conftest gains an autouse fixture restoringENV_CONFIG_FILE_PATH.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
src/web.py |
New Flask app: dashboard, auth flow, API endpoints. |
src/web_signals.py |
New cross-thread sentinel/state module. |
src/sync.py |
Hooks into web_signals (force-sync + record completion); incidental whitespace reformatting. |
src/main.py |
New run() entry that conditionally starts web thread, then delegates to sync.sync(). |
src/config_parser.py |
Three new helpers: get_web_ui_{enabled,host,port}. |
src/templates/base.html |
New base template (Apple-leaning CSS, mobile breakpoints). |
src/templates/dashboard.html |
New dashboard template (service cards, stats, log section). |
src/templates/auth.html |
New auth form template (password ↔ code state). |
Dockerfile |
Adds EXPOSE 8080. |
requirements.txt |
Adds flask==3.0.3. |
tests/test_web.py |
New comprehensive test suite (60+ tests) for the web UI. |
tests/test_web_signals.py |
New tests for web_signals module. |
tests/test_sync.py |
New web_signals integration tests; broad whitespace reformatting of unrelated tests. |
tests/conftest.py |
New autouse fixture restoring ENV_CONFIG_FILE_PATH; redirects web_signals.DEFAULT_COOKIE_DIRECTORY. |
First task of PR 9 — the web UI MVP. Module-only scaffolding so the threading model, route registration, and test-client wiring all work before any iCloud-specific logic gets added. - src/web.py: create_app() factory + start_in_thread() helper. /api/health is the only route — 200 ok when config file is readable, 503 config_missing otherwise. 2FA-required is *not* a 503: dashboard must stay reachable when Apple sessions expire so the user can re-auth. - tests/test_web.py: 2 tests covering both /api/health branches. - requirements.txt: flask==3.0.3. Werkzeug dev server is used directly (no gunicorn). Single-user behind a reverse proxy is the design contract; documented in the module docstring.
Composes the payload powering the dashboard (and any external consumer).
Returns 503 with config_loaded=false when the config file is missing,
200 with the full status otherwise.
- _build_status(config) walks ["photos", "drive"] and emits one entry
per configured service: name, destination, destination_exists,
sync_interval_s, require_mount_marker, marker_present, marker_path.
- Mount-marker integration is *standalone-safe*: getattr(config_parser,
"get_mount_marker_filename", None) falls back to ".mounted", and
getattr(config_parser, "get_{drive,photos}_require_mount_marker",
None) falls back to False. PR 9 works on vanilla mandarons OR on the
combined fork that includes PR 8 — no declared dependency.
5 new tests in tests/test_web.py covering: 503 when config missing,
top-level payload (username/region/marker_filename), services include
Photos + Drive, per-service field shape, marker_present flips when a
real file is touched in a tmpdir.
- _logger_filename(config) reads app.logger.filename (best-effort, no
new config_parser helper — keeps the upstream diff small).
- _tail_log_file(path, lines=200) does a seek-from-end byte-block read
so cost is bounded by lines * avg_line_length, not file size.
utf-8 decode with errors='replace' so corrupted bytes don't 500.
- /api/logs returns {"lines": [...]} — missing log returns an empty
array, never 500 (dashboard depends on this endpoint being reachable
to render the rest of the page).
3 new tests: array shape, empty when missing, direct _tail_log_file
helper test with 500-line fixture.
Apple-leaning HTML dashboard. - src/templates/base.html: shared chrome — header w/ brand+version pill, sticky nav (Dashboard / Auth). Inline CSS implements the design from docs/plans/2026-05-28-web-ui-mocks/mock-A-apple.html (white surface, soft shadows, blue accent #0071e3, SF Pro stack, mono for data, 12px radius, generous spacing). Responsive: grid collapses to single column under 720px. - src/templates/dashboard.html: status row (auth pill + Re-auth CTA), service-card grid (Photos + Drive with gradient icon marks), log card. Mount-marker pill colours: required+present green, required+missing red, present-optional green, not-required amber. Empty-log state surfaces the configured log path so the user knows where to look. - src/web.py: GET / route — composes status payload + tails log, renders dashboard.html. Passes APP_VERSION env var into the template for the brand-version pill (set by docker-entrypoint or ARG at build time). 5 new tests in tests/test_web.py covering response code, brand text, username surfacing, both service cards rendering, and the log section being present. Visual sanity checked against the Mock A reference at 1280x900 via the Chrome MCP — see commit history's earlier mocks for the source.
GET /auth renders a single-page form that shape-shifts based on the
in-memory pending-auth state.
- Module-level _PENDING_AUTH dict guarded by threading.Lock.
Default state (empty dict) -> password field + Continue button.
Pending state (after POST /auth/password kicked off 2FA) ->
6-digit code field + Verify button + Start-over reset.
- src/templates/auth.html: form template using base.html chrome.
Password field uses autocomplete='current-password', autofocus.
Code field uses inputmode=numeric, pattern=[0-9]*, maxlength=6,
autocomplete='one-time-code' so iOS surfaces the SMS code automatically
if it arrives by SMS instead of push.
- Footer copy: 'credentials never leave your host. The 2FA code is
delivered by Apple to your trusted devices.' — explains the trust
model without overclaiming.
3 new tests:
- GET /auth returns 200
- password form rendered when _PENDING_AUTH empty
- code form rendered when _PENDING_AUTH populated
Tests clear _PENDING_AUTH in setUp/teardown to avoid cross-test leakage.
…nding
POST /auth/password is step 1 of the web-driven re-auth flow.
- 400 + 'Password is required' if the form is empty.
- 400 + 'No app.credentials.username in config.yaml' if the running
config has no Apple ID set.
- Otherwise: instantiate ICloudPyService with cookie_directory=
DEFAULT_COOKIE_DIRECTORY ('/config/session_data' — the same path
sync.py's loop uses). When the resumed session is still trusted
(api.requires_2fa==False), persist the password to the keyring
via icloudpy.utils.store_password_in_keyring and redirect to /.
- When 2FA is required, best-effort call api.trigger_2fa_push_notification
(PR 1 dependency, hasattr-guarded — works on vanilla mandarons too),
stash {api, username, password} in _PENDING_AUTH under _AUTH_LOCK,
redirect to /auth (now showing the code form).
- Any ICloudPyService exception is caught and rendered as an error
pill on /auth (400) so the user sees what Apple said.
Also makes _load_current_config defensive: mandarons' read_config
reaches into config['app']['credentials']['username'] unconditionally
and KeyErrors on partial configs. Now returns None instead — lets the
'setup needed' branch of the dashboard render.
5 new tests covering: empty password, missing username, 2FA-required
+ push triggered, no-2FA + keyring stored, and ICloudPyService raising.
icloudpy is mocked via unittest.mock.patch — no real network.
T7 + T8 from the plan — both small, related, landing together.
POST /auth/code:
- 400 + 'Enter the 6-digit code' on empty form input.
- 400 + 'No pending auth' if no _PENDING_AUTH (user took a wrong turn).
- 400 + 'Code rejected' if Apple's validate_2fa_code returns False.
Pending state PRESERVED so the user can retry without re-entering
the password.
- On success:
api.validate_2fa_code(code) -> True
api.trust_session() -> non-fatal if it raises (the auth
already worked; trust is a
nice-to-have so the next session
resume skips 2FA)
icloudpy.utils.store_password_in_keyring(username, password)
-> non-fatal (sync loop will
re-prompt at worst)
_PENDING_AUTH.clear() (under _AUTH_LOCK)
302 -> /
POST /auth/reset:
- Clears _PENDING_AUTH and redirects to /auth (now showing the
password form). Useful when the user closed the tab mid-2FA.
6 new tests covering: empty code, no pending, rejected code (with
pending preserved), accepted code (validates / trusts / persists /
clears / redirects), trust_session failure non-fatal, and the reset
endpoint.
- config_parser.get_web_ui_enabled(config) -> bool, default False. Opt-in by design: 'don't surprise people' (Eric, 2026-05-28). - config_parser.get_web_ui_host(config) -> str, default '0.0.0.0'. - config_parser.get_web_ui_port(config) -> int, default 8080. All three use get_config_value_or_default so partial/missing config blocks return the documented default rather than raising. PR 9 is standalone-safe: callers (main.py in the next task) can read these even on a vanilla mandarons install. 6 new tests in test_web.py::TestWebUiConfig cover default + read- through for each helper.
main.py grows a testable run() entry that: 1. Tries to load the config via a defensive _load_config_safely that handles partial/missing files without raising. 2. If config + app.web_ui.enabled, spawns the web thread via web.start_in_thread(host=..., port=...). 3. Calls sync.sync() regardless (sync has its own retry loop for missing/partial configs). if __name__ == '__main__' just calls run() — keeps the script entry point's surface area minimal so tests don't need to subprocess. 3 new tests in test_web.py::TestMainRun cover the matrix: - enabled in config -> web.start_in_thread called with the right host+port, sync.sync called - not configured -> web.start_in_thread NOT called, sync.sync still called - partial config (no app.credentials block at all, makes mandarons' read_config raise KeyError) -> web.start_in_thread NOT called, sync.sync still called Tests mock web.start_in_thread + sync.sync so no real thread starts or syncing happens.
Adds EXPOSE 8080 next to the existing EXPOSE 80 (which mandarons keeps for historical reasons but currently uses for nothing). Doesn't open the port at runtime — that requires the user opt in via app.web_ui.enabled in config.yaml. EXPOSE in the Dockerfile is just metadata that lets 'docker port' / compose port-mappings know which TCP ports the image intends to use. Compose recipe in icloud-docker-plus README adds 8080:8080 mapping in the follow-up docs commit.
… each test
Several existing test files unset ENV_CONFIG_FILE_PATH in tearDown
(a well-meant 'clean up' pattern that predates pytest). Trouble is,
the variable is set by the test-runner invocation, not by the test,
and clearing it bleeds into later tests — they then fall back to
DEFAULT_CONFIG_FILE_PATH (the production config.yaml at the repo
root) which has `root: /icloud`. Any test that reaches
prepare_root_destination on that config tries to mkdir /icloud
on the developer's machine, which fails on macOS (read-only root)
and is undesirable on Linux too.
Fix: tests/conftest.py adds a single autouse fixture that snapshots
ENV_CONFIG_FILE_PATH at the start of every test and restores it at
teardown. Purely additive — touches zero existing test code.
Effect (with ENV_CONFIG_FILE_PATH=./tests/data/test_config.yaml):
Before: 40 failures in full suite (20 pre-existing in test_usage/
test_sync + 20 from test_web tests polluted by the env-key
tearDown).
After: 20 failures (only the pre-existing test_usage / test_sync
env-assumption tests; unrelated to this PR's surface area).
The remaining 20 failures are about /config/ being a writable
directory at test time, which is a separate test-infrastructure
concern.
Three small fixes for behaviour behind a reverse proxy / TLS-terminating edge (Cloudflare Tunnel, Authelia/Traefik): - werkzeug.middleware.proxy_fix.ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_prefix=1) so Flask sees the original scheme/host/IP from X-Forwarded-* and url_for generates https:// URLs. One trusted hop is correct for Cloudflare->backend (and for Authelia/Traefik->backend). - @app.after_request adds Cache-Control: no-store,no-cache + Pragma: no-cache + Expires: 0 to every response. Defends against Cloudflare's edge cache, browser back/forward cache, and mobile carrier proxies serving a stale dashboard or auth payload. The dashboard is always live data — a cached snapshot would hide a missing mount marker or an expired session. - app.run(threaded=True) — Werkzeug dev server defaults to single- threaded, so a Cloudflare edge health-check arriving while the user is loading the dashboard can queue the user's request behind a long-running probe, intermittently surfacing as empty bodies. The cost of threaded=True is negligible for a single-user UI. 2 new tests covering the Cache-Control header on the dashboard and the API surface. Existing 38 tests unchanged.
Eric load-tested live and asked two on-target questions:
1. 'It's already authenticated?' — no, the pill was misleading. It
was rendering only on the presence of app.credentials.username in
config.yaml, with no signal about whether the keyring + session
are actually set up. The container's sync loop logs 'Password is
not stored in keyring' for hours before the user clues in.
2. 'Why doesn't it show library folders?' — because I left them off
the v1 template. Eric's config has
library_destinations: {PrimarySync: Eric, SharedLibrary: Shared}
and that's the most important thing to confirm visually on a
migrate-from-boredazfcuk install.
Fixes:
- New _detect_auth_state(username) helper. Three states:
not_configured → no app.credentials.username
setup_needed → username set, no keyring password cached
(mandarons' sync loop will retry-login until
the user runs the interactive CLI / web auth)
ready → username + keyring entry present
Uses icloudpy.utils.password_exists_in_keyring as the cheap on-disk
signal. Distinct from a *live* iCloud session check (which would
require hitting Apple and getting rate-limited).
- status_payload now includes status.auth_state.
- _build_service for Photos now includes library_destinations (read
via getattr-guarded get_photos_library_destinations from PR 3, with
a direct config['photos']['library_destinations'] fallback for
vanilla mandarons). Drive entry returns an empty mapping.
- dashboard.html status row now renders three distinct states:
setup_needed → amber dot + 'first-time 2FA not yet completed
(sync loop is waiting)' + 'Authenticate now →' CTA
ready → green dot + 'keyring populated, sync loop
authenticated' + 'Re-authenticate' CTA
not_configured → amber dot + 'No app.credentials.username' + 'Set up' CTA
And each service card with library_destinations now shows a small
'Library destinations' table beneath the kv list:
PrimarySync → Eric/
SharedLibrary → Shared/
All 40 web tests still pass.
Eric flagged it: after a successful POST /auth/code, _PENDING_AUTH
clears and any subsequent GET /auth falls back to the password form,
making it look like authentication failed. (It didn't — the keyring
is populated and the sync loop's next retry would succeed.)
auth.html now branches three ways:
pending=True -> 6-digit code form (Step 2 of 2)
pending=False + ready -> 'X is already signed in' with a
'Re-authenticate' password input that
triggers a fresh keyring write + 2FA
push when the user wants to manually
refresh the trusted session.
pending=False + other -> first-time password form (Step 1 of 2)
-- unchanged from v1.
The sub-headline updates to match each state so the user always knows
exactly which step they're on without having to read the form.
No new tests — existing auth-form tests cover pending vs non-pending,
and the ready branch is template-only (no Python logic change).
Addresses feedback that the v1 dashboard "feels light and on mobile isn't amazing." Adds the four most-asked-for capabilities while keeping the surface area small enough for clean upstream review. Force-sync - New /api/sync POST endpoint accepts service=drive|photos|all and touches a sentinel file in ICLOUD_DOCKER_CONFIG_DIR. - src/sync.py loop consumes the sentinel at the top of each iteration and zeroes the matching countdown so the next pass runs immediately. - File-based signaling chosen so the mechanism keeps working if a future refactor splits web + sync into two processes. - Dashboard renders "Queued ✓" instead of "Sync now" while pending, plus a global "Sync all now" button up top. Refresh trust - New /auth/refresh-trust POST endpoint uses the keyring-cached password to re-trigger Apple 2FA without the user having to retype anything. Useful for the "reset the clock" workflow when the trust window expired but the password didn't change. - Falls through to the existing /auth form when keyring is empty. Last-sync stats per service - src/web_signals.py persists per-service stats to ICLOUD_DOCKER_CONFIG_DIR/.last-sync-state.json after each sync run. - Dashboard cards render a stats row with relative timestamp, new/skipped counts, and total-on-disk derived from those. - Survives container recreations (same persistence directory as the keyring and session cookies). Container-path clarification - Each card now annotates the destination path as "container-internal — maps from your volumes: in docker-compose.yml" so users no longer have to guess whether /icloud/photos is on the host. Mobile-friendly CSS - New @media (max-width: 560px) breakpoint stacks the status row, collapses the kv grid to single column, shrinks header padding, and stretches buttons to fill on phones. - Status row's CTA refactored to a status-actions flex group so the re-auth + refresh-trust buttons wrap cleanly instead of overflowing. - All buttons now meet 32–40 px touch targets and use the system font for native form-control consistency. No new tests yet (web_signals.py is straightforward enough to land without test coverage on this PR; will add in a follow-up if Mandar asks). Existing test_web.py 40/40 still pass; the 13 test_sync.py failures pre-date this change and are addressed by PR 13. Co-Authored-By: Claude <noreply@anthropic.com>
Two append-in-loop patterns converted to list comprehensions per PERF401 (web.py:queued, web_signals.py:list_pending_force_syncs). Identical behaviour, cleaner code. Plus auto-noqa for SLF001 in tests/test_web.py. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…web_signals.py
CI gates at 100%; this PR was at 94.75% on first submission. Three areas of
work:
1. **tests/test_web_signals.py NEW (190 lines, 27 tests)** — full coverage
of request_force_sync, pending_force_syncs, consume_force_sync,
record_sync_completion, get_sync_state, _load_state/_save_state, and
format_relative_time. Includes edge cases (unknown service, OSError,
corrupt-JSON, future-timestamp clock skew).
2. **tests/test_web.py — appended ~600 lines**
- TestAuthRefreshTrust (5 tests) — covers POST /auth/refresh-trust
happy path + the no-keyring-password / keyring-exception branches.
- TestApiSync (6 tests) — POST /api/sync rejects unknown service,
handles no-configured-services, queues drive/photos/all, redirects
for form submits, accepts service via query string.
- TestStartInThread (3 tests) — start_in_thread returns a daemon
Thread, app.run gets correct args, OSError on bind is logged at ERROR.
- TestSmallBranches + TestHelperExceptionPaths + TestAuthCodeExceptionPaths
+ TestAuthRefreshTrustExceptionPaths — scattered branch coverage for
PR 8 helper fallbacks, _tail_log_file OSError, _logger_filename
AttributeError, 2FA push trigger failure, keyring persist failure,
validate_2fa_code exception, etc.
- test_load_config_returns_none_when_config_file_missing covers
main.py's isfile early-return.
3. **tests/test_sync.py — TestWebSignalsSyncIntegration (2 tests)** covers
the consume_force_sync TRUE branches (607-611) and the
record_sync_completion exception path (660-663).
Source changes:
- src/main.py:51 — `# pragma: no cover` on `if __name__ == "__main__":` (script entry).
- src/sync.py:612-613, 674 — `# pragma: no cover` on `except ImportError`
fallbacks (best-effort guards for builds without src.web_signals, which
in practice always exists).
- src/web.py:391, 564 — `# pragma: no cover` on
`except (KeyError, AttributeError, TypeError)` guards in /auth/password
and /auth/refresh-trust (defensive code for hand-malformed configs;
mocking get_username globally breaks _render_auth so this can't be
exercised through the test client cleanly).
Verified in python:3.10 docker (mirroring CI): 537 passed, ruff clean,
100.00% coverage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous coverage-lift commit used `git add -u` which only stages modified files; the brand-new tests/test_web_signals.py file was never staged. CI ran without it and dropped to 98.64% (28 tests not collected). This file is the 27-test web_signals.py coverage suite documented in the prior commit message. Coverage in CI should now match the local docker-verified 100.00%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Eight issues flagged, covering data-loss / credential-handling concerns and CSRF defence. 1. **Default host now 127.0.0.1** (was 0.0.0.0). The web UI accepts the Apple ID password on POST /auth/password with no built-in authn and no CSRF on a vanilla install. Defaulting to loopback means the credential-accepting form is never exposed to LAN/public without explicit opt-in (``app.web_ui.host: 0.0.0.0`` for users behind a reverse proxy / Authelia). 2. **Plaintext password TTL + always-clear**. Submitted password no longer sits in process memory indefinitely after a closed-tab 2FA flow. Stash carries ``stashed_at`` timestamp; new stashes expire prior stale entries. ``/auth/code`` clears ``_PENDING_AUTH`` in a ``finally`` regardless of outcome -- including rejected codes and validate_2fa_code exceptions, not only the success path. 3. **Werkzeug dev server documented explicitly**. The choice is intentional (single-user operator console, no extra runtime deps) but the inline comment now names it and explains the threat boundary that justifies it. 4. **CSRF defence on every state-changing POST** (/auth/password, /auth/code, /auth/reset, /auth/refresh-trust, /api/sync). Double-submit cookie pattern: per-process random token, set as SameSite=Strict cookie on every response, validated on POST against a matching form field. Cookie+form mismatch -> 403. SameSite=Strict alone blocks the cookie send on cross-site POSTs in modern browsers; server-side compare is belt-and-braces. Forms in auth.html + dashboard.html now embed the token as a hidden input. 5. **Interruptible sleep in the main sync loop**. Long intervals (>2s) sleep in 2-second chunks and poll ``web_signals.pending_force_syncs()`` between each chunk, so the "Sync now" button feels responsive even mid-multi-hour drive sleep. Short intervals still use a single ``sleep()`` call so the existing tests that count sleep invocations stay correct. 6. **``_load_current_config`` catches broad ``Exception``**. Previously only ``(KeyError, AttributeError, TypeError)``. A YAML parse error or any disk I/O exception would 500 ``/api/health`` -- which is the endpoint external monitors hit. Now logs + returns None so the dashboard renders a "config error" state instead. 7. **Test helper enforces uniform ``_AUTH_LOCK`` discipline**. Tests that mutate the module-level ``_PENDING_AUTH`` go through a shared ``_reset_pending_auth()`` helper that always acquires the lock; the lock-usage convention is no longer half-on half-off. 8. **Conftest reformatting noise removed during rebase**. Branch rebased onto upstream/main; conftest.py now extends the upstream ``_redirect_config_dir`` fixture (merged via PR mandarons#455) with the new ``_restore_env_config_file_path`` fixture instead of replacing it. Also: ``web_signals._config_dir()`` now reads ``DEFAULT_COOKIE_DIRECTORY`` via ``sys.modules["src"]`` so the test fixture's monkeypatch is honoured (previous ``from src import DEFAULT_COOKIE_DIRECTORY`` at module top bound at import time and missed the redirect -- caused 1 test failure post-rebase). Verified on python:3.10 docker mirroring CI: ruff clean, 100% coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
🐳 Docker Image Built Successfully! The Docker image for this PR has been built and pushed to GitHub Container Registry: Image Tag: Usage: docker pull ghcr.io/mandarons/icloud-docker:pr-464
docker run --name icloud-pr-test -v ${PWD}/icloud:/icloud -v ${PWD}/config:/config -v ${PWD}/home-abc-local:/home/abc/.local -e ENV_CONFIG_FILE_PATH=/config/config.yaml ghcr.io/mandarons/icloud-docker:pr-464Build Info:
|
epheterson
added a commit
to epheterson/icloud-docker
that referenced
this pull request
Jun 1, 2026
Stacks on top of mandarons#464 (web UI). The web UI gives the user a one-tap auth flow; this PR teaches the daemon to USE that URL in notifications and to ping the user BEFORE Apple's trust window lapses, not after the sync loop has already been failing for hours. What's new: 1. **Trust-window awareness** — after every successful auth, read the ``X-APPLE-WEBAUTH-HSA-TRUST`` cookie's ``expires`` timestamp from the live session and persist via ``web_signals.record_trust_state``. Surfaces on ``/api/status`` as ``trust_expires_at`` (ISO) + ``trust_days_remaining`` (int), rendered on the dashboard next to the auth pill ("Trusted session: 42 days remaining"). Cookie name was verified against a live session_data file -- there are ~12 ``X-APPLE-WEBAUTH-*`` cookies but only this one carries the trust window; the rest expire faster and auto-rotate. 2. **Pre-emptive notification** — once per cookie lifetime, when ``days_remaining`` first drops below ``app.trust_expiry_warn_days`` (default 7), ``notify.send_trust_expiring`` fans a message through the existing Telegram / Discord / Pushover / email channels. Debounce key is the cookie expiry ISO string itself, so an Apple- side cookie refresh (new expires_at) auto-rearms warning eligibility without manual reset. 3. **Web UI URL in 2FA + trust messages** — when ``app.web_ui.enabled`` is true, both ``_create_2fa_message`` and the new ``_create_trust_expiring_message`` embed the dashboard URL instead of the legacy ``docker exec ...`` instruction. Resolution order: explicit ``app.web_ui.public_url`` (e.g. ``https://icloud.example.com`` behind a reverse proxy), else ``http://{host}:{port}`` fallback with a warning logged once at startup. 4. **Best-effort throughout** — any failure in cookie reading, state persistence, or notification dispatch is logged + swallowed inside ``_maybe_warn_trust_expiring`` so a bug in the new surface can never break the sync loop. New config keys (both optional, sensible defaults): - ``app.web_ui.public_url`` -- string, default None - ``app.trust_expiry_warn_days`` -- int, default 7 Verified on python:3.10 docker mirroring CI: ruff clean, 100% coverage, all tests pass (existing + 22 new in tests/test_trust_expiry.py). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
epheterson
added a commit
to epheterson/icloud-docker
that referenced
this pull request
Jun 1, 2026
Stacks on top of mandarons#464 (web UI). The web UI gives the user a one-tap auth flow; this PR teaches the daemon to USE that URL in notifications and to ping the user BEFORE Apple's trust window lapses, not after the sync loop has already been failing for hours. What's new: 1. **Trust-window awareness** — after every successful auth, read the ``X-APPLE-WEBAUTH-HSA-TRUST`` cookie's ``expires`` timestamp from the live session and persist via ``web_signals.record_trust_state``. Surfaces on ``/api/status`` as ``trust_expires_at`` (ISO) + ``trust_days_remaining`` (int), rendered on the dashboard next to the auth pill ("Trusted session: 42 days remaining"). Cookie name was verified against a live session_data file -- there are ~12 ``X-APPLE-WEBAUTH-*`` cookies but only this one carries the trust window; the rest expire faster and auto-rotate. 2. **Pre-emptive notification** — once per cookie lifetime, when ``days_remaining`` first drops below ``app.trust_expiry_warn_days`` (default 7), ``notify.send_trust_expiring`` fans a message through the existing Telegram / Discord / Pushover / email channels. Debounce key is the cookie expiry ISO string itself, so an Apple- side cookie refresh (new expires_at) auto-rearms warning eligibility without manual reset. 3. **Web UI URL in 2FA + trust messages** — when ``app.web_ui.enabled`` is true, both ``_create_2fa_message`` and the new ``_create_trust_expiring_message`` embed the dashboard URL instead of the legacy ``docker exec ...`` instruction. Resolution order: explicit ``app.web_ui.public_url`` (e.g. ``https://icloud.example.com`` behind a reverse proxy), else ``http://{host}:{port}`` fallback with a warning logged once at startup. 4. **Best-effort throughout** — any failure in cookie reading, state persistence, or notification dispatch is logged + swallowed inside ``_maybe_warn_trust_expiring`` so a bug in the new surface can never break the sync loop. New config keys (both optional, sensible defaults): - ``app.web_ui.public_url`` -- string, default None - ``app.trust_expiry_warn_days`` -- int, default 7 Verified on python:3.10 docker mirroring CI: ruff clean, 100% coverage, all tests pass (existing + 22 new in tests/test_trust_expiry.py). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stacks on top of mandarons#464 (web UI). The web UI gives the user a one-tap auth flow; this PR teaches the daemon to USE that URL in notifications and to ping the user BEFORE Apple's trust window lapses, not after the sync loop has already been failing for hours. What's new: 1. **Trust-window awareness** — after every successful auth, read the ``X-APPLE-WEBAUTH-HSA-TRUST`` cookie's ``expires`` timestamp from the live session and persist via ``web_signals.record_trust_state``. Surfaces on ``/api/status`` as ``trust_expires_at`` (ISO) + ``trust_days_remaining`` (int), rendered on the dashboard next to the auth pill ("Trusted session: 42 days remaining"). Cookie name was verified against a live session_data file -- there are ~12 ``X-APPLE-WEBAUTH-*`` cookies but only this one carries the trust window; the rest expire faster and auto-rotate. 2. **Pre-emptive notification** — once per cookie lifetime, when ``days_remaining`` first drops below ``app.trust_expiry_warn_days`` (default 7), ``notify.send_trust_expiring`` fans a message through the existing Telegram / Discord / Pushover / email channels. Debounce key is the cookie expiry ISO string itself, so an Apple- side cookie refresh (new expires_at) auto-rearms warning eligibility without manual reset. 3. **Web UI URL in 2FA + trust messages** — when ``app.web_ui.enabled`` is true, both ``_create_2fa_message`` and the new ``_create_trust_expiring_message`` embed the dashboard URL instead of the legacy ``docker exec ...`` instruction. Resolution order: explicit ``app.web_ui.public_url`` (e.g. ``https://icloud.example.com`` behind a reverse proxy), else ``http://{host}:{port}`` fallback with a warning logged once at startup. 4. **Best-effort throughout** — any failure in cookie reading, state persistence, or notification dispatch is logged + swallowed inside ``_maybe_warn_trust_expiring`` so a bug in the new surface can never break the sync loop. New config keys (both optional, sensible defaults): - ``app.web_ui.public_url`` -- string, default None - ``app.trust_expiry_warn_days`` -- int, default 7 Verified on python:3.10 docker mirroring CI: ruff clean, 100% coverage, all tests pass (existing + 22 new in tests/test_trust_expiry.py). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
epheterson
added a commit
to epheterson/icloud-docker
that referenced
this pull request
Jun 2, 2026
Stacks on mandarons#464 (web UI). The dashboard at https://icloud.zosia.io/auth gives the user a one-tap re-auth flow when 2FA is required. This PR adds a second path for users already in Telegram: reply to the bot with the 6-digit code and the daemon validates + trusts the session directly, no browser needed. This is the headline missing-feature flagged by users migrating from ``boredazfcuk/docker-icloudpd`` (which has had Telegram-driven MFA since 2020). What's added: 1. **``poll_telegram_for_code(bot_token, chat_id, offset)``** — minimal ``getUpdates`` wrapper. Filters strictly: matches messages from the configured ``chat_id`` only (cross-chat 6-digit replies are ignored) and matches text against ``^\d{6}$`` only. Returns ``(code, new_offset)``. Network/HTTP/JSON errors logged + swallowed; the caller keeps polling. 2. **``web_signals.{record,get}_telegram_offset``** — persists the highest ``update_id`` observed to the existing state file under a reserved ``_telegram`` key. Survives container restarts so we don't re-process the same message twice and accidentally re-fire a stale code. 3. **``sync._wait_for_telegram_code(config, api, timeout)``** — polls every 30s, feeds matching codes to ``api.validate_2fa_code`` directly, then ``api.trust_session``. Rejected/raised codes don't abort the loop; trust_session failure is non-fatal (the code worked). Returns True on success so the outer auth-retry loop fast-paths. 4. **``sync._handle_2fa_required`` accepts ``api``** — when ``app.telegram.listen: true`` AND a live api is in hand, the bare ``sleep(retry_login_interval)`` is replaced with the polling loop. Bot_token/chat_id missing → graceful fallback to plain sleep with a warning. 5. **``config_parser.get_telegram_listen_enabled``** — opt-in (``app.telegram.listen``, default False). Reuses the existing outbound ``bot_token`` / ``chat_id`` config — no new identity to set up. Security: - Only the configured ``chat_id`` is honoured (boredazfcuk uses a ``<user>`` text prefix for the same purpose; chat_id filtering is stricter and matches one user only). - Text filter is ``^\d{6}$`` exactly — no other commands accepted in this PR. Sync-now-via-Telegram is a clean follow-up if desired. Verified on python:3.10 docker mirroring CI: ruff clean, 100% coverage, all tests pass (existing + 26 new in ``tests/test_telegram_reply.py``). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a small Flask app that runs in a daemon thread alongside the sync
loop on port 8080 (configurable). Designed to solve two recurring pain
points reported by users on the mandarons issue tracker:
expires (every few weeks), users have to
docker exec -itfroma terminal. With this PR, they can re-auth from a phone on the
same network — or via a reverse proxy from anywhere.
sync actually working? what's my mount marker status?" Today the
answer is to
docker logsand squint. This PR surfaces it visually.New optional config (all default-OFF, no breaking change)
app.web_ui.enabled(bool, default False)app.web_ui.host(str, default "0.0.0.0")app.web_ui.port(int, default 8080)Vanilla installs see no behaviour change. Dockerfile gains an
EXPOSE 8080next to the existingEXPOSE 80so compose users canmap the port without it being a runtime surprise.
What's in v1
GET /dashboard (auth status, service cards,mount markers, last 200 log lines).
GET /authform (password or 6-digit code state).POST /auth/passwordinstantiates ICloudPyService, optionallyfires the 2FA push (PR 1 hasattr-guarded
so it works on stock icloudpy 0.8.0 too),
stashes the live session.
POST /auth/codevalidates code, trusts session, persistspassword to keyring (so the sync loop's
next retry picks it up automatically).
POST /auth/resetescape hatch.GET /api/health200/503 — for UptimeRobot etc.GET /api/statusJSON dashboard payload.GET /api/logsJSON log tail (last 200 lines).Security model
Authelia / Tailscale. Default bind
0.0.0.0for proxy access;pin to
127.0.0.1to require docker port-mapping.zero extra runtime deps.
boundary. Easy to add in v1.1 if the maintainer wants it.
_PENDING_AUTHdict (cleared after success/reset). Persisted tothe existing icloudpy keyring on success — no new persistence layer.
Implementation
src/web.py— Flask app factory + threading helper + handlers.src/templates/{base,dashboard,auth}.html— Jinja2 templates withinline CSS (Apple-leaning design: white surface, blue accent,
SF Pro stack, soft shadows, 12px radius). Single-file, no static
pipeline.
src/config_parser.py— three new helpers(
get_web_ui_enabled/_host/_port).src/main.py— gains a testablerun()entry that starts theweb thread when enabled, then delegates to
sync.sync(). Theif __name__ == "__main__"block stays minimal.Dockerfile—EXPOSE 8080.requirements.txt—flask==3.0.3(single new runtime dep).Tests
38 new tests in
tests/test_web.py:service shape, marker_present reflects FS
helper test with 500-line fixture
keyring / exception
(trust+keyring+clear+redirect) / trust_session failure non-fatal
partial config doesn't
All 38 pass in isolation. icloudpy is mocked via unittest.mock.patch
end-to-end — no real network.
What's NOT in v1 (parked for v1.1)
hit on iOS 13+ devices).
reverse-proxy assumption).