Add default-on CLI telemetry and lifecycle events - #97
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #97 +/- ##
==========================================
+ Coverage 94.41% 95.80% +1.38%
==========================================
Files 46 59 +13
Lines 3167 3905 +738
==========================================
+ Hits 2990 3741 +751
+ Misses 177 164 -13 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Persist default-off pseudonymous telemetry state outside generated projects, expose explicit user controls, and document the privacy contract. Checks: make check (656 tests) Refs ENG-197 Refs ENG-198
Map provider-neutral telemetry envelopes through typed PostHog DTOs, read the public project token only from the process environment, and contain all delivery failures. Checks: make check (676 tests) Refs ENG-202
Attempt one allowlisted terminal event for create outcomes, normalize aliases and unknown values without names or paths, suppress telemetry in tests and evals, and smoke-test the packaged controls. Checks: make check (700 tests); run_evals.py --tier smoke; binary packaging smoke Refs ENG-200
Suppress synthetic Cloudflare smoke events and skip descriptor chmod where the platform does not provide it. Findings: 1, 2 Checks: make check (701 tests); bash -n scripts/ci/smoke-cloudflare-worker.sh Refs ENG-197 Refs ENG-200
Make configured telemetry default-on with lazy, stable identity creation while preserving persistent and process-level opt-outs. Add closed install and upgrade outcome events, typed upgrade results, fail-closed symlink handling, and move the binary runbook under docs/operations.\n\nRefs ENG-197\nRefs ENG-198\nRefs ENG-200\nRefs ENG-201\nRefs ENG-202\nRefs ENG-203\n\nTests: make check (781 passed)\nCoverage: make coverage (95.8%)
The default-enabled status tests deliberately remove pytest suppression, so also neutralize GitHub Actions' inherited CI=true value.\n\nRefs ENG-198\nRefs ENG-201\n\nTests: CI=true make check (781 passed)
Classify PATH-only changes as success, unresolved requested PATH integration as partial success, and path-phase failures as partial whenever a usable binary is already present. Document and test each boundary.\n\nRefs ENG-203\n\nTests: CI=true make check (784 passed)\nCoverage: CI=true make coverage (95.8%)
woud420
force-pushed
the
codex/eng-200-create-telemetry
branch
from
July 19, 2026 23:08
533d91e to
24d1076
Compare
Correct the inherited ENG-10 wording without changing updater behavior. The existing nesting defect is tracked separately as ENG-204. Tests: CI=true make check (784 passed)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Primary changes
telemetry status|enable|disable|reset-idcontrols plusDO_NOT_TRACK,KICKSTART_TELEMETRY_DISABLED, CI, pytest, evaluation, and source-checkout development suppressions.src/model/dto/; isolate PostHog behind a replaceable sink.scaffold_create_completed,cli_install_completed, andcli_upgrade_completed.phc_token into trusted wheel, source-distribution, and binary builds fromPOSTHOG_PUBLIC_CUSTOMER_API_TOKEN.docs/operations/install-binaries.md, preserve ENG-10's substantive maintainer-entrypoint guidance without promising an unstable internal payload path, and document first-run opt-out commands and event-counting limits.Reviewer walkthrough
docs/contracts/telemetry.md,README.md, anddocs/operations/install-binaries.md.src/telemetry/state.py,src/telemetry/policy.py, andsrc/cli/telemetry.py.src/model/dto/telemetry.py,src/model/dto/upgrade.py, andsrc/telemetry/events.py.src/cli/main.pyandsrc/utils/updater.py.src/model/dto/posthog.py,src/telemetry/posthog.py, andsrc/telemetry/reporter.py.scripts/embed_posthog_token.py,scripts/verify_embedded_telemetry_artifact.py, and the GitHub workflows.Correctness and invariants
telemetry statusdoes not create state or send an event.--check, uninstall, help, version, completion, and telemetry-control commands emit no usage event.cli_install_completedmeasures onlykickstart install;cli_upgrade_completedmeasures onlykickstart upgrade. They are not universal installation/upgrade totals.success; an unresolved requested PATH update reports non-fatalpartial_success/path_update; a true binary/PATH no-op reportsno_change.phx_personal keys and other read-capable credentials are rejected.Local Development Entrypointguidance while this PR relocates the complete runbook underdocs/operations/. The physical payload path remains an implementation detail; the pre-existing updater nesting defect is tracked separately in ENG-204.Testing and QA
dd0c284,CI=true make checkpasses Ruff, mypy, type hygiene, import hygiene, template audit, and all 784 tests.cd website && bun run checkpasses TypeScript validation and all 18 website tests.git diff --check origin/master...HEADpasses, andgit range-diffreports all ten telemetry commits patch-equivalent to their pre-rebase versions.make coverage: 95.8% overall. Telemetry DTOs, events, policy, and reporter are at 100%; state is 93.9%; updater is 98.7%.scaffold_create_completedfor synthetic distinct IDcc3e0145-c8a5-41de-ab97-f7a3ce105967.CI=truein two default-status tests; commit04c639aisolates that test environment, andCI=true make checkpasses locally after the rebase.CI=true make checkstill passes all 784 tests, and the focused consistency re-audit reports zero remaining findings.5235f75and24d1076fixed symlinked-state handling and PATH outcome classification.Risk / Rollout
POSTHOG_PUBLIC_CUSTOMER_API_TOKENis exported at runtime.Resolves ENG-197
Resolves ENG-198
Refs ENG-199
Resolves ENG-200
Refs ENG-201
Resolves ENG-202
Resolves ENG-203