Record how users configure tests in the anonymous usage report

[inancgumus]

Problem

The anonymous usage report records the environment (k6 version, OS, architecture, CI) and the run volume (duration, max VUs, iterations, executor types, module and output names). It records nothing about how users configure a test: thresholds, options, scenario settings, checks, groups, custom metrics, lifecycle functions, and secret sources are all invisible. Maintainers cannot see which configuration features people use.

Significance

Configuration features carry permanent maintenance cost: threshold parsing and validation, more than 60 options, six executors with their own settings, three summary modes. Decisions about them come up regularly. Can the legacy summary mode retire? Does anyone use abortOnFail? How many runs set insecureSkipTLSVerify? Today the report cannot answer these questions. The closest proxies are GitHub issues and support tickets, and they over-represent power users.

Cost of inaction

Deprecation and investment decisions continue to rest on anecdote. A rarely used option keeps its maintenance cost forever, and a change to a widely used one breaks users nobody counted. Gaps also stay invisible: the web dashboard never appears in the report, because internal/cmd/outputs.go records only built-in output names and web-dashboard is not one of them.

Desired state

The report tells how users configure tests with counts, booleans, and names from k6's fixed vocabulary. It never contains a string the user types. For example, "3 thresholds: 2 on built-in metrics, 1 on a custom metric, with p(95) and rate" is acceptable. The threshold expression itself is not, because expressions and option values can contain URLs, hostnames, and custom names.

Signal	Record	Never record
Thresholds	Threshold count. Per threshold: the target metric (a built-in name, or custom), the aggregation method (avg, min, max, med, p(N), count, rate, or value), and whether it uses a tag filter, abortOnFail, or delayAbortEval.	The expression string, custom metric names, tag names, tag values.
Options	The names of the options the user sets. Values only when k6 defines them as a fixed list, for example summaryMode and compatibilityMode.	Values that carry user data: hosts, blockHostnames, tags, userAgent, environment variables, paths, addresses.
Scenarios	Per scenario: the stage count, and whether it sets exec, startTime, env, or tags. Executor types are already recorded.	Scenario names, env content, tags content.
Script structure: lifecycle, checks, groups, custom metrics	Whether the script defines setup, teardown, and handleSummary. Counts of distinct checks, groups, and custom metrics per type (Counter, Gauge, Rate, Trend).	Check names, group names, metric names.
Secret sources	The built-in source type names: file, mock, url, cloud.	The source configuration: file paths, URLs, keys.
Web dashboard	On or off.	Dashboard configuration.

Counts that grow with script size (checks, groups, custom metrics, stages) go into cumulative buckets. Grafana OSS uses this technique for concurrent users. The existing internal/usage API (Strings, Uint64, Values) supports all of these signals, and assembly stays in internal/cmd/report.go.

The report stays anonymous and opt-out (--no-usage-report, K6_NO_USAGE_REPORT). The same review bar as extension telemetry (#6126) applies: nothing the user authors leaves the machine.

Out of scope

• Numeric option values such as durations, rates, and stage targets. vus_max and iterations already record run magnitude.
• The place where an option is set: CLI flag, environment variable, script, or config file.
• Imports from jslib.k6.io, such as the k6-testing assertions library. Remote imports need an allowlist like the extension catalog in #6126. That is a separate brief.
• Protocol usage. Imports of k6/http, k6/ws, k6/net/grpc, and k6/browser are already recorded under modules.