diff --git a/CHANGELOG.md b/CHANGELOG.md index f8321cd94d..8be9cbf0be 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Changes to be including in future/planned release notes will be added here. ## [0.6.9] - `allowedQueryParams` rule checks are now case-insensitive (e.g., `$top` and `$TOP` are treated as equivalent). - `gcp`: **beta** first-class optional external Application Load Balancer (ALB) via `external_api_alb` on `gcp-host` (replaces the prior root `external-api-alb.tf` composition). Cloud Armor IP rules apply only when `allowed_data_access_ip_blocks` is non-null. BYO ALB remains available via `api_connector_external_lb_host`. If you applied the old root composition, destroy those root ALB resources (or `terraform state mv` into the new module addresses) before upgrading. Enabling the ALB may add `hashicorp/tls` to your root provider lockfile (self-signed PoC path). +- `codex-enterprise-analytics` new connector in **beta**; imports per-user daily Codex usage (threads, turns, credits, token counts, per-client/per-model breakdowns, code attribution) from OpenAI's Codex Enterprise Analytics API; see [docs/sources/codex-enterprise-analytics/README.md](docs/sources/codex-enterprise-analytics/README.md) - `aws`/`gcp`: fix Terraform plan failure when `enable_remote_resources = true` but no artifacts bucket exists (e.g. with a prebuilt `deployment_bundle`). When remote resources are enabled, an artifacts bucket is now provisioned if one is not already created or provided via `artifacts_bucket_name` / `custom_artifacts_bucket_name`. - `msft-onedrive`: adding support for a new connector for fetching Microsoft OneDrive data from users and groups via Microsoft Graph API. See [docs](docs/sources/microsoft-365/msft-onedrive/README.md) for details. @@ -49,7 +50,7 @@ Changes to be including in future/planned release notes will be added here. **BREAKING / UPGRADE NOTES:** - `aws`: Minimum `hashicorp/aws` provider version is now `~> 6.0`. When upgrading, users must update the version constraint under `terraform { required_providers { ... } }` for `hashicorp/aws` to this version or later, and run `terraform init -upgrade` to apply. -- `gcp`: +- `gcp`: - Minimum `hashicorp/google` provider version is now `~> 7.0`. When upgrading, users must update the version constraint under `terraform { required_providers { ... } }` for `hashicorp/google` to this version or later, and run `terraform init -upgrade` to apply. Our GCP example sets `version = "~> 7.0"` in root `main.tf`. - `default_labels` is no longer passed into Psoxy modules. Set it on the root `provider "google"` block instead (see `infra/examples-dev/gcp/main.tf`). Provider 7.x applies `default_labels` to supported resources automatically; remove any `default_labels` arguments from module calls when upgrading from 0.5.x. - Terraform provisions a Docker repository in Artifact Registry for Cloud Functions Gen 2. Your deployer needs **Artifact Registry Editor** (`roles/artifactregistry.editor`) — the least-privileged predefined role that includes `artifactregistry.repositories.create`. (`roles/artifactregistry.repoAdmin` manages artifacts in existing repositories but cannot create them; `roles/artifactregistry.admin` also works but grants additional permissions such as `repositories.setIamPolicy`.) Custom roles built from `required_gcp_perms_to_provision_host` need `create`, `update`, `get`, and `list`; `repositories.delete` is only required for `terraform destroy` and is included in `required_gcp_permissions_to_host` instead. See [docs/gcp/getting-started.md](docs/gcp/getting-started.md). diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index f2f281b796..f70dce95ce 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -70,6 +70,7 @@ * [Atlassian Organization (Rovo)](sources/atlassian/organization/README.md) * [Badge](sources/badge/README.md) * [ChatGPT Enterprise](sources/chatgpt-enterprise/README.md) + * [Codex Enterprise Analytics](sources/codex-enterprise-analytics/README.md) * [Cursor](sources/cursor/README.md) * [Dropbox Business](sources/dropbox-business/README.md) * [GitHub](sources/github/README.md) diff --git a/docs/sources/README.md b/docs/sources/README.md index eadd951fd9..1e004835b5 100644 --- a/docs/sources/README.md +++ b/docs/sources/README.md @@ -16,6 +16,7 @@ To add a source, add its Connector ID to the `enabled_connectors` list in your ` | `claude` | [Claude](anthropic/claude/README.md) | API | BETA | | `claude-enterprise-analytics` | [Claude Enterprise Analytics](anthropic/claude-enterprise-analytics/README.md) | API | BETA | | `claude-code` | [Claude Code](anthropic/claude-code/README.md) | API | BETA | +| `codex-enterprise-analytics` | [Codex Enterprise Analytics](codex-enterprise-analytics/README.md) | API | BETA | | `confluence-cloud` | [Confluence Cloud](atlassian/confluence/README.md) | API | BETA | | `cursor` | [Cursor](cursor/README.md) | API | BETA | | `dropbox-business` | [Dropbox Business](dropbox-business/README.md) | API | DEPRECATED | diff --git a/docs/sources/codex-enterprise-analytics/README.md b/docs/sources/codex-enterprise-analytics/README.md new file mode 100644 index 0000000000..fe1e69a2f1 --- /dev/null +++ b/docs/sources/codex-enterprise-analytics/README.md @@ -0,0 +1,58 @@ +# Codex Enterprise Analytics + +**Connector ID:** `codex-enterprise-analytics` + +**Availability:** Beta + +This API allows ChatGPT Enterprise and Codex administrators to pull daily Codex adoption and code +review metrics for a workspace. It is intended for dashboarding, reporting, and trend analysis, +with metrics aggregated by day and, where available, broken down per-user. Endpoints return +time-ordered pages using cursor-based pagination. + +## Data Collected + +| Endpoint | Description | Per-user? | +|---|---|---| +| `GET /v1/analytics/codex/workspaces/{workspace_id}/usage` | Per-user daily Codex usage: threads, turns, credits, token counts, per-client and per-model breakdowns, and code attribution (lines of code / commits with a Codex contribution). | Yes | + +OpenAI's API also exposes `code_reviews` and `code_review_responses` endpoints, but both only +report workspace-level daily totals with no per-user breakdown (no `user_id`/`actor` field, and no +`group` parameter to request one). Since they can't be attributed to a person, they are not +implemented by this connector. + +### Privacy + +`user_id` and `actor.user_id`/`actor.email` on the `usage` endpoint are pseudonymized by the +Worklytics DLP Proxy before leaving your network. No message/prompt content is collected by this +connector — only aggregate usage counters. + +## Instructions to Connect + +1. Add `codex-enterprise-analytics` to `enabled_connectors` in `terraform.tfvars`, then run + `terraform apply`. +2. Complete source authorization per the `TODO 1 - codex-enterprise-analytics` file: + 1. Create a Platform API key at https://platform.openai.com/settings/organization/api-keys. + 2. Email support@openai.com to add the `codex.enterprise.analytics.read` scope to that key and + remove all other scopes. + 3. Copy the key into the proxy as the `PSOXY_CODEX_ENTERPRISE_ANALYTICS_ACCESS_TOKEN` parameter + value in your proxy's host platform. +3. Run the proxy health checks in the `TODO 2 - test psoxy-codex-enterprise-analytics` file + generated by `terraform apply`. +4. Create the **Codex Enterprise Analytics via Psoxy** connection in Worklytics; see the + `TODO 3 - connect codex-enterprise-analytics in Worklytics.md` file generated by + `terraform apply`. That file lists the values to enter in Worklytics, including: + - **Psoxy Base URL** — your deployed proxy/function endpoint URL + - **Workspace Id** — your Codex workspace id (found in the ChatGPT Admin console at + https://chatgpt.com/admin, on the workspace settings page) + +## Rate Limits + +Default: **30 requests/minute**, adjustable via the `DEFAULT` system setting on the data source +connection. OpenAI does not publish a documented rate limit for this endpoint, so this is a +conservative default. + +## Examples + +- [Example Rules](codex-enterprise-analytics.yaml) +- Example Data: [original/usage.json](example-api-responses/original/usage.json) | + [sanitized/usage.json](example-api-responses/sanitized/usage.json) diff --git a/docs/sources/codex-enterprise-analytics/codex-enterprise-analytics.yaml b/docs/sources/codex-enterprise-analytics/codex-enterprise-analytics.yaml new file mode 100644 index 0000000000..20901704f8 --- /dev/null +++ b/docs/sources/codex-enterprise-analytics/codex-enterprise-analytics.yaml @@ -0,0 +1,140 @@ +--- +endpoints: +- pathTemplate: "/v1/analytics/codex/workspaces/{workspaceId}/usage" + allowedQueryParams: + - "start_time" + - "end_time" + - "limit" + - "page" + transforms: + - ! + jsonPaths: + - "$.data[*].user_id" + - "$.data[*].actor['user_id','email']" + encoding: "URL_SAFE_TOKEN" + responseSchema: + type: "object" + properties: + object: + type: "string" + data: + type: "array" + items: + $ref: "#/definitions/UsageRow" + has_more: + type: "boolean" + next_page: + type: "string" +definitions: + AccountUserActor: + type: "object" + properties: + email: + type: "string" + type: + type: "string" + user_id: + type: "string" + ClientUsage: + type: "object" + properties: + cached_text_input_tokens: + type: "integer" + client_id: + type: "string" + credits: + type: "number" + text_output_tokens: + type: "integer" + text_total_tokens: + type: "integer" + threads: + type: "integer" + turns: + type: "integer" + uncached_text_input_tokens: + type: "integer" + CodeAttributionMetrics: + type: "object" + properties: + commits: + $ref: "#/definitions/CommitAttribution" + lines_of_code: + $ref: "#/definitions/LinesOfCodeAttribution" + CommitAttribution: + type: "object" + properties: + with_codex_contribution: + type: "integer" + with_codex_contribution_percentage: + type: "number" + LinesOfCodeAttribution: + type: "object" + properties: + added: + type: "integer" + committed: + type: "integer" + committed_percentage: + type: "number" + removed: + type: "integer" + ModelUsage: + type: "object" + properties: + cached_text_input_tokens: + type: "integer" + credits: + type: "number" + model: + type: "string" + speed: + type: "string" + text_output_tokens: + type: "integer" + text_total_tokens: + type: "integer" + uncached_text_input_tokens: + type: "integer" + Usage: + type: "object" + properties: + cached_text_input_tokens: + type: "integer" + credits: + type: "number" + text_output_tokens: + type: "integer" + text_total_tokens: + type: "integer" + threads: + type: "integer" + turns: + type: "integer" + uncached_text_input_tokens: + type: "integer" + UsageRow: + type: "object" + properties: + actor: + $ref: "#/definitions/AccountUserActor" + clients: + type: "array" + items: + $ref: "#/definitions/ClientUsage" + code_attribution: + $ref: "#/definitions/CodeAttributionMetrics" + end_time: + type: "integer" + models: + type: "array" + items: + $ref: "#/definitions/ModelUsage" + object: + type: "string" + start_time: + type: "integer" + totals: + $ref: "#/definitions/Usage" + user_id: + type: "string" diff --git a/docs/sources/codex-enterprise-analytics/example-api-responses/original/usage.json b/docs/sources/codex-enterprise-analytics/example-api-responses/original/usage.json new file mode 100644 index 0000000000..46429a6386 --- /dev/null +++ b/docs/sources/codex-enterprise-analytics/example-api-responses/original/usage.json @@ -0,0 +1,102 @@ +{ + "object": "page", + "data": [ + { + "object": "workspace.codex.usage.result", + "start_time": 1780272000, + "end_time": 1780358400, + "user_id": "user_abc123", + "actor": { + "type": "ACCOUNT_USER", + "user_id": "user_abc123", + "email": "alice@example.com" + }, + "totals": { + "threads": 123, + "turns": 456, + "credits": 12.5, + "uncached_text_input_tokens": 1000, + "cached_text_input_tokens": 250, + "text_output_tokens": 400, + "text_total_tokens": 1650 + }, + "clients": [ + { + "client_id": "CODEX_WEB", + "threads": 80, + "turns": 300, + "credits": 4.5, + "uncached_text_input_tokens": 700, + "cached_text_input_tokens": 150, + "text_output_tokens": 250, + "text_total_tokens": 1100 + }, + { + "client_id": "CODEX_CLI", + "threads": 43, + "turns": 156, + "credits": 8.0, + "uncached_text_input_tokens": 300, + "cached_text_input_tokens": 100, + "text_output_tokens": 150, + "text_total_tokens": 550 + } + ], + "models": [ + { + "model": "gpt-5.4", + "speed": "standard", + "credits": 12.5, + "uncached_text_input_tokens": 1000, + "cached_text_input_tokens": 250, + "text_output_tokens": 400, + "text_total_tokens": 1650 + } + ], + "code_attribution": { + "lines_of_code": { + "added": 123, + "removed": 45, + "committed": 123, + "committed_percentage": 27 + }, + "commits": { + "with_codex_contribution": 12, + "with_codex_contribution_percentage": 57.1 + } + } + }, + { + "object": "workspace.codex.usage.result", + "start_time": 1780272000, + "end_time": 1780358400, + "user_id": "user_def456", + "actor": { + "type": "ACCOUNT_USER", + "user_id": "user_def456", + "email": null + }, + "totals": { + "threads": 10, + "turns": 30, + "credits": 1.2, + "uncached_text_input_tokens": null, + "cached_text_input_tokens": null, + "text_output_tokens": null, + "text_total_tokens": null + }, + "clients": [ + { + "client_id": "CODEX_CLI", + "threads": 10, + "turns": 30, + "credits": 1.2 + } + ], + "models": null, + "code_attribution": null + } + ], + "has_more": true, + "next_page": "cursor_token_abc" +} diff --git a/docs/sources/codex-enterprise-analytics/example-api-responses/sanitized/usage.json b/docs/sources/codex-enterprise-analytics/example-api-responses/sanitized/usage.json new file mode 100644 index 0000000000..b092d14901 --- /dev/null +++ b/docs/sources/codex-enterprise-analytics/example-api-responses/sanitized/usage.json @@ -0,0 +1,102 @@ +{ + "object":"page", + "data":[ + { + "object":"workspace.codex.usage.result", + "start_time":1780272000, + "end_time":1780358400, + "user_id":"t~Wd2U4lmivWESn4fNgZWtjnW3j4TXciQNif50hO_QMVM", + "actor":{ + "type":"ACCOUNT_USER", + "user_id":"t~Wd2U4lmivWESn4fNgZWtjnW3j4TXciQNif50hO_QMVM", + "email":"t~knxWE5tapsutcz6pLWSC3wfb5t34lCrXNZsKnhwyVXk@example.com" + }, + "totals":{ + "threads":123, + "turns":456, + "credits":12.5, + "uncached_text_input_tokens":1000, + "cached_text_input_tokens":250, + "text_output_tokens":400, + "text_total_tokens":1650 + }, + "clients":[ + { + "client_id":"CODEX_WEB", + "threads":80, + "turns":300, + "credits":4.5, + "uncached_text_input_tokens":700, + "cached_text_input_tokens":150, + "text_output_tokens":250, + "text_total_tokens":1100 + }, + { + "client_id":"CODEX_CLI", + "threads":43, + "turns":156, + "credits":8.0, + "uncached_text_input_tokens":300, + "cached_text_input_tokens":100, + "text_output_tokens":150, + "text_total_tokens":550 + } + ], + "models":[ + { + "model":"gpt-5.4", + "speed":"standard", + "credits":12.5, + "uncached_text_input_tokens":1000, + "cached_text_input_tokens":250, + "text_output_tokens":400, + "text_total_tokens":1650 + } + ], + "code_attribution":{ + "lines_of_code":{ + "added":123, + "removed":45, + "committed":123, + "committed_percentage":27 + }, + "commits":{ + "with_codex_contribution":12, + "with_codex_contribution_percentage":57.1 + } + } + }, + { + "object":"workspace.codex.usage.result", + "start_time":1780272000, + "end_time":1780358400, + "user_id":"t~rUNP43joGY2IUQzpGJcOH7nY9pzIOXRLPOKi_PdHieA", + "actor":{ + "type":"ACCOUNT_USER", + "user_id":"t~rUNP43joGY2IUQzpGJcOH7nY9pzIOXRLPOKi_PdHieA", + "email":null + }, + "totals":{ + "threads":10, + "turns":30, + "credits":1.2, + "uncached_text_input_tokens":null, + "cached_text_input_tokens":null, + "text_output_tokens":null, + "text_total_tokens":null + }, + "clients":[ + { + "client_id":"CODEX_CLI", + "threads":10, + "turns":30, + "credits":1.2 + } + ], + "models":null, + "code_attribution":null + } + ], + "has_more":true, + "next_page":"cursor_token_abc" +} diff --git a/infra/modules/worklytics-connector-specs/auth_config.tftest.hcl b/infra/modules/worklytics-connector-specs/auth_config.tftest.hcl index 5c1361f0c0..039f2d31f5 100644 --- a/infra/modules/worklytics-connector-specs/auth_config.tftest.hcl +++ b/infra/modules/worklytics-connector-specs/auth_config.tftest.hcl @@ -37,7 +37,7 @@ run "oauth_refresh_token_access_tokens" { assert { error_message = "all 19 oauth connectors use ACCESS_TOKEN (all except dropbox??)" - condition = 19 == length([for k, v in output.available_oauth_data_source_connectors : + condition = 20 == length([for k, v in output.available_oauth_data_source_connectors : v if anytrue([for var in v.secured_variables : var.name == "ACCESS_TOKEN"]) ]) } diff --git a/infra/modules/worklytics-connector-specs/docs/chatgpt/codex-enterprise-analytics/instructions.tftpl b/infra/modules/worklytics-connector-specs/docs/chatgpt/codex-enterprise-analytics/instructions.tftpl new file mode 100644 index 0000000000..e342d5c845 --- /dev/null +++ b/infra/modules/worklytics-connector-specs/docs/chatgpt/codex-enterprise-analytics/instructions.tftpl @@ -0,0 +1,23 @@ +See OpenAI's Codex Enterprise Analytics documentation for the latest, but as of 2026 you must +create a Platform API key and have OpenAI grant it a dedicated read-only analytics scope. + +1 - Create a new [Platform API key](https://platform.openai.com/settings/organization/api-keys). + - Note that this must be a new key. Once the `codex.enterprise.analytics.read` scope is granted, + all other scopes are revoked. + - Reminder: this key can only be viewed/copied once. +2 - Copy the key immediately - you won't see it again; then paste it as the + `${path_to_instance_parameters}ACCESS_TOKEN` parameter value in the proxy's Secret Manager. +3 - Send an email to [support@openai.com](mailto:support@openai.com) requesting that the + `codex.enterprise.analytics.read` scope be added to the key (and all other scopes removed). + Include: + - (1) the last 4 digits of the API key + - (2) the Key Name + - (3) the Created By Name +OpenAI's team will verify the key and grant the requested scope. + +4. After your administrator deploys the Psoxy instance (`terraform apply`), follow the + `TODO 3 - connect codex-enterprise-analytics in Worklytics.md` file generated in your Terraform + working directory. That file provides the **Psoxy Base URL** (your proxy/function endpoint), + which you must enter in the Worklytics connector settings, along with your Codex **Workspace + Id** (example: `${workspace_id}`), found in the ChatGPT Admin console + (https://chatgpt.com/admin) on the workspace settings page. diff --git a/infra/modules/worklytics-connector-specs/main.tf b/infra/modules/worklytics-connector-specs/main.tf index 673b9d145e..3f4a12c03e 100644 --- a/infra/modules/worklytics-connector-specs/main.tf +++ b/infra/modules/worklytics-connector-specs/main.tf @@ -155,6 +155,35 @@ EOT path_to_instance_parameters = "PSOXY_CHATGPT_ENTERPRISE_" }) } + codex-enterprise-analytics = { + source_kind : "codex-enterprise-analytics", + availability : "beta", + enable_by_default : false, + worklytics_connector_id : "codex-enterprise-analytics-psoxy" + display_name : "Codex Enterprise Analytics" + worklytics_connector_name : "Codex Enterprise Analytics via Psoxy" + target_host : "api.chatgpt.com" + source_auth_strategy : "oauth2_access_token" + secured_variables : [ + { + name : "ACCESS_TOKEN" # OpenAI calls this a 'Platform API key', but it's actually an access token + writable : false + sensitive : true + value_managed_by_tf : false + } + ], + settings_to_provide = { + "Workspace Id" = local.chat_gpt_enterprise_example_workspace_id + } + example_api_calls_user_to_impersonate : null + example_api_calls : [ + "/v1/analytics/codex/workspaces/${local.chat_gpt_enterprise_example_workspace_id}/usage?start_time=${time_static.deployment.unix - 86400 * 30}&end_time=${time_static.deployment.unix}", + ] + external_token_todo : templatefile("${path.module}/docs/chatgpt/codex-enterprise-analytics/instructions.tftpl", { + workspace_id = local.chat_gpt_enterprise_example_workspace_id, + path_to_instance_parameters = "PSOXY_CODEX_ENTERPRISE_ANALYTICS_" + }) + } claude = { source_kind : "claude" availability : "beta", diff --git a/java/core/src/main/java/co/worklytics/psoxy/rules/chatgpt/PrebuiltSanitizerRules.java b/java/core/src/main/java/co/worklytics/psoxy/rules/chatgpt/PrebuiltSanitizerRules.java index 358e00cf1b..40f3b8a9d5 100644 --- a/java/core/src/main/java/co/worklytics/psoxy/rules/chatgpt/PrebuiltSanitizerRules.java +++ b/java/core/src/main/java/co/worklytics/psoxy/rules/chatgpt/PrebuiltSanitizerRules.java @@ -12,9 +12,11 @@ public class PrebuiltSanitizerRules { static final RESTRules ENTERPRISE = Rules2.load("sources/chatgpt-enterprise/chatgpt-enterprise.yaml"); + static final RESTRules CODEX_ENTERPRISE_ANALYTICS = Rules2.load("sources/codex-enterprise-analytics/codex-enterprise-analytics.yaml"); static public final Map DEFAULT_RULES_MAP = ImmutableMap.builder() .put("chatgpt-enterprise", ENTERPRISE) + .put("codex-enterprise-analytics", CODEX_ENTERPRISE_ANALYTICS) .build(); } diff --git a/java/core/src/test/java/co/worklytics/psoxy/rules/chatgpt/CodexEnterpriseAnalyticsTest.java b/java/core/src/test/java/co/worklytics/psoxy/rules/chatgpt/CodexEnterpriseAnalyticsTest.java new file mode 100644 index 0000000000..0c3d1ac89d --- /dev/null +++ b/java/core/src/test/java/co/worklytics/psoxy/rules/chatgpt/CodexEnterpriseAnalyticsTest.java @@ -0,0 +1,33 @@ +package co.worklytics.psoxy.rules.chatgpt; + +import co.worklytics.psoxy.rules.JavaRulesTestBaseCase; +import co.worklytics.psoxy.rules.RESTRules; +import lombok.Getter; + +import java.util.stream.Stream; + +@Getter +public class CodexEnterpriseAnalyticsTest extends JavaRulesTestBaseCase { + + final RESTRules rulesUnderTest = PrebuiltSanitizerRules.CODEX_ENTERPRISE_ANALYTICS; + + final RulesTestSpec rulesTestSpec = RulesTestSpec.builder() + .defaultScopeId("codex-enterprise-analytics") + .sourceKind("codex-enterprise-analytics") + .exampleApiResponsesDirectoryPath("example-api-responses/original/") + .exampleSanitizedApiResponsesPath("example-api-responses/sanitized/") + .checkUncompressedSSMLength(false) + .build(); + + @Override + public Stream getExamples() { + return Stream.of( + // /usage — page 1 (no cursor) + InvocationExample.of("https://api.chatgpt.com/v1/analytics/codex/workspaces/ws_abc123/usage?start_time=1780272000&end_time=1780358400", "usage.json"), + InvocationExample.of("https://api.chatgpt.com/v1/analytics/codex/workspaces/ws_abc123/usage?start_time=1780272000&end_time=1780358400&limit=100", "usage.json"), + // /usage — page 2+ (cursor pagination) + InvocationExample.of("https://api.chatgpt.com/v1/analytics/codex/workspaces/ws_abc123/usage?start_time=1780272000&end_time=1780358400&page=cursor_token_abc", "usage.json"), + InvocationExample.of("https://api.chatgpt.com/v1/analytics/codex/workspaces/ws_abc123/usage?start_time=1780272000&end_time=1780358400&page=cursor_token_abc&limit=100", "usage.json") + ); + } +}