telemetry: close privacy-review gaps in exegraph + hooks; backfill audit docs#8600
Draft
hemarina wants to merge 7 commits into
Draft
telemetry: close privacy-review gaps in exegraph + hooks; backfill audit docs#8600hemarina wants to merge 7 commits into
hemarina wants to merge 7 commits into
Conversation
Collaborator
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix https://github.com/Azure/azure-dev-pr/issues/1772
Summary
Closes telemetry privacy-review gaps surfaced by an audit of the metrics-audit docs in
docs/specs/metrics-audit/against the actual telemetry-emitting code undercli/azd/.The audit found:
exegraph.step.nameandexegraph.step.depswere emitted RAW even though step names embed user-definedazure.yamlservice/layer names (e.g.,deploy-<svc.Name>,provision-<layer.Name>).hooks.namewas emitted RAW unconditionally frompkg/ext/hooks_runner.go; only the (lower-traffic)cmd/hooks.gopath used an allowlist.internal/tracing/fields/fields.go,internal/tracing/events/events.go) were undocumented in the privacy-review materials. Several existing rows also had inaccurate field names and enum descriptions.Privacy fixes (code changes)
e2d6080pkg/exegraph/scheduler.go: switchExeGraphStepNameKey/ExeGraphStepDepsKeytofields.StringHashed/fields.StringSliceHashed. Step names embedazure.yamlservice/layer names (verified atinternal/cmd/service_graph.go:356-358,up_graph.go:770,provision_graph.go:520-522).Tagsleft raw — verified all&exegraph.Step{Tags: ...}constructors use hardcoded string literals only.74cf84epkg/ext/hooks_runner.gonow hasheshooks.nameby default and only emits raw when the name is in a shared allowlist of built-in lifecycle hooks. Extracted the allowlist fromcmd/hooks.gointo a newpkg/ext/known_hooks.go(KnownHookNames) so both emit sites stay in sync.Documentation backfill
380d822telemetry-schema.md— documented all 136 attribute keys infields.goand all 25 event constants inevents.go. Added Hashing section and PII-Risk Assessment for newly-documented fields.cf8f417feature-telemetry-matrix.md— added Tool Management, Copilot (Agent), and Cross-Cutting Subsystems sections; corrected ✅/❌ flags on commands now known to emit additional telemetry (e.g.,provision,deploy,package,extension).5d59b84privacy-review-checklist.md— split the hashed-fields list into "Fields That Must Be Hashed" (always-hashed) and "Fields With Conditional Hashing" (subscription.id,pack.builder.image/tag) to remove a previously-misleading "always hashed" claim.1ff3773hooks.kindenum description (it's the script runtimesh/pwsh/js/ts/python/dotnet, not pre/post), replaced fictionalextension.source.id/.type/.dependency.*field names with the real keys, removed falseextension sourcesubcommand attribution, and closed thetool.install.strategyenum gap.Verification
Every documented field/event/enum value was verified at a concrete code location, not against comments or documentation. Spot examples:
events.ArmDeploySubscriptionEvent→pkg/azapi/standard_deployments.go:228;events.CopilotInitializeEvent→internal/agent/copilot_agent.go:81;events.AksPostprovisionSkipEvent→pkg/project/service_target_aks.go:990.validation.preflight.outcomevalues verified against constants atpkg/infra/provisioning/bicep/bicep_provider.go:2549-2554;provision.cancellationvalues verified atpkg/infra/provisioning/bicep/interrupt.go;hooks.kindvalues verified atpkg/tools/language/executor.go:14-31.subscription.idverified atpkg/environment/local_file_data_store.go:198-202andstorage_blob_data_store.go:186-190(UUID-valid → raw, otherwise hashed);pack.builder.image/tagverified atpkg/project/container_helper.go:1126-1137(userDefinedImage == true→ hashed).hooks.namepost-fix — both emit sites (cmd/hooks.go:149-152andpkg/ext/hooks_runner.go:180-183) now use the sharedext.KnownHookNamesallowlist; grep confirms no third raw emit site exists.Files changed
Backward compatibility
fields.StringHashed/fields.StringSliceHashed) — the existing helpers used throughout the codebase for the other 4 always-hashed fields (project.template.id,project.template.version,project.name,env.name).Testing
pkg/ext/,pkg/exegraph/,cmd/middleware/continue to pass.Test_Lazy_AzdContext_Resolution/Test_Lazy_Project_Config_Resolutionfailures on Windows are unrelated (caused by a strayC:\Users\<user>\azure.yamlfrom prior test runs walking up to the user-home root; reproduces on baselinemain).