feat(queries): add FinOps KPI query catalog#2166
Open
MSBrett wants to merge 5 commits into
Open
Conversation
microsoft-github-policy-service
Bot
added
the
Needs: Review 👀
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a FinOps Hub query catalog under src/queries/** to support KPI/scenario analytics (and downstream agent tooling) by introducing a curated set of KQL assets plus accompanying index and schema documentation updates.
Changes:
- Added a 37-query KQL catalog under
src/queries/catalog/covering common FinOps scenarios and FinOps Foundation KPIs. - Added/updated the query catalog index (
INDEX.md) and KPI mapping (KPI.md) for discoverability. - Updated the FinOps Hub database guide to reflect live Hub numeric types and expanded
Recommendations()schema guidance.
Reviewed changes
Copilot reviewed 23 out of 23 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| src/queries/INDEX.md | Query catalog landing page and scenario-to-query matrix (now includes KPI-focused entries). |
| src/queries/KPI.md | KPI-to-query mapping table referencing FinOps Foundation KPIs and corresponding catalog queries. |
| src/queries/finops-hub-database-guide.md | Hub schema documentation updates (notably numeric types and Recommendations() caveats). |
| src/queries/catalog/ai-cost-by-application.kql | Azure OpenAI cost allocation query (tags → showback/chargeback breakdown). |
| src/queries/catalog/ai-daily-trend.kql | Azure OpenAI daily token + cost trend query. |
| src/queries/catalog/ai-model-cost-comparison.kql | Azure OpenAI model cost efficiency comparison query. |
| src/queries/catalog/ai-token-usage-breakdown.kql | Azure OpenAI token usage breakdown by model/direction query. |
| src/queries/catalog/allocation-accuracy-index.kql | Allocation Accuracy Index (AAI) KPI query. |
| src/queries/catalog/anomaly-detection-rate.kql | Anomaly Detection Rate KPI query using time-series decomposition. |
| src/queries/catalog/anomaly-variance-total.kql | Total Unpredicted Variance KPI query for anomaly-flagged buckets. |
| src/queries/catalog/commitment-discount-utilization.kql | Commitment discount utilization scenario query. |
| src/queries/catalog/commitment-discount-waste.kql | Commitment discount waste KPI query. |
| src/queries/catalog/commitment-utilization-score.kql | Commitment utilization score KPI query. |
| src/queries/catalog/compute-cost-per-core.kql | Compute cost-per-core KPI query (hourly + effective-average variants). |
| src/queries/catalog/compute-spend-commitment-coverage.kql | Compute spend covered by commitments KPI query. |
| src/queries/catalog/cost-anomaly-detection.kql | Cost anomaly detection scenario query. |
| src/queries/catalog/cost-by-financial-hierarchy.kql | Financial hierarchy allocation scenario query. |
| src/queries/catalog/cost-by-region-trend.kql | Regional cost trend scenario query. |
| src/queries/catalog/cost-forecasting-model.kql | Cost forecasting scenario query. |
| src/queries/catalog/cost-optimization-index.kql | COIN KPI query based on recommendations savings opportunity vs cost. |
| src/queries/catalog/cost-per-gb-stored.kql | Storage cost-per-GB-month KPI query. |
| src/queries/catalog/cost-visibility-delay.kql | Cost visibility delay KPI query (ingestion vs charge-period end). |
| src/queries/catalog/costs-enriched-base.kql | Row-level enriched baseline query for drill-down diagnostics. |
| src/queries/catalog/data-update-frequency.kql | Data update frequency KPI query (ingestion cadence). |
| src/queries/catalog/macc-consumption-vs-commitment.kql | MACC consumption vs commitment KPI query. |
| src/queries/catalog/monthly-cost-change-percentage.kql | Month-over-month change percentage scenario query. |
| src/queries/catalog/monthly-cost-trend.kql | Monthly cost trend scenario query. |
| src/queries/catalog/percentage-unallocated-costs.kql | Unallocated costs KPI query. |
| src/queries/catalog/percentage-untagged-costs.kql | Untagged costs KPI query. |
| src/queries/catalog/quarterly-cost-by-resource-group.kql | Quarterly cost by resource group scenario query. |
| src/queries/catalog/reservation-recommendation-breakdown.kql | Reservation recommendation breakdown scenario query. |
| src/queries/catalog/savings-summary-report.kql | Savings summary / ESR scenario query. |
| src/queries/catalog/service-price-benchmarking.kql | Service price benchmarking scenario query. |
| src/queries/catalog/storage-tier-distribution.kql | Storage tier distribution KPI query. |
| src/queries/catalog/tagging-policy-compliance.kql | Tagging policy compliance KPI query. |
| src/queries/catalog/top-commitment-transactions.kql | Top commitment transactions scenario query. |
| src/queries/catalog/top-other-transactions.kql | Top non-usage/non-commitment transactions scenario query. |
| src/queries/catalog/top-resource-groups-by-cost.kql | Top resource groups by cost scenario query. |
| src/queries/catalog/top-resource-types-by-cost.kql | Top resource types by cost scenario query. |
| src/queries/catalog/top-services-by-cost.kql | Top services by cost scenario query. |
Comments suppressed due to low confidence (1)
src/queries/INDEX.md:63
- The References section includes the FinOps Foundation framework link twice ("FinOps Framework (FinOps Foundation)" and "FinOps Foundation" both point to https://www.finops.org/framework/). This looks like an accidental duplicate entry.
- [FinOps Framework (FinOps Foundation)](https://www.finops.org/framework/)
- [Implementing FinOps Guide (Microsoft Learn)](https://learn.microsoft.com/cloud-computing/finops/implementing-finops-guide)
- [Adopt FinOps on Azure (Microsoft Learn)](https://learn.microsoft.com/training/modules/adopt-finops-on-azure/)
- [FinOps hub Database Documentation](./finops-hub-database-guide.md)
- [FinOps Foundation](https://www.finops.org/framework/)
RolandKrummenacher
left a comment
RolandKrummenacher
left a comment
Collaborator
There was a problem hiding this comment.
Live validation of the KPI query catalog
I ran all 22 added/changed .kql files in src/queries/catalog/ against a live FinOps hub (ADX, ~35M Costs() rows, full calendar-month window, with commitment + recommendation data present). All executed read-only.
Result: 22/22 executed successfully — well-formed output, no semantic (SEM*) errors.
Highlights:
- All FOCUS columns,
x_enrichments, and themake-series/series_decompose_anomalies/mv-apply/materialize()patterns resolve against the live schema. - Cross-query consistency held: total
EffectiveCostreconciles across the allocation, anomaly, COIN, and savings queries; AAI + unallocated % = 100%; storage cost reconciles across the storage/anomaly queries; commitment-utilization potential = commitment-discount-waste total; anomaly per-category event counts sum to the all-services rollup. - The commitment KPIs (
commitment-discount-waste,commitment-utilization-score,commitment-discount-utilization,compute-spend-commitment-coverage,cost-optimization-index) returned real values on a commitment-active hub — these were schema-only / zero-coverage in the prior UAT hub.
Two non-blocking observations:
data-update-frequency.kqlgroups byBillingAccountIdwith case-sensitive==. Where the same account ID appears in mixed casing, this splits one logical account into multiple rows — considertolower(BillingAccountId).macc-consumption-vs-commitment.kql— the commitment denominator (x_MonetaryCommitmentfromTransactions()) was still empty (no MACC transaction rows on the test hub). The query's ownOPENnote already flags this; the consumption numerator populates correctly.
RolandKrummenacher
left a comment
RolandKrummenacher
left a comment
Collaborator
There was a problem hiding this comment.
Two line-specific follow-ups from the live-hub validation (see the summary review above for full results). Both non-blocking.
- Normalize BillingAccountId to lowercase before grouping in data-update-frequency, cost-visibility-delay, and macc-consumption-vs-commitment so mixed-case account IDs collapse to one logical account (extends Roland's live-hub finding). - Replace the specific ADX cluster hostname with the generic "FinOps Hub ADX cluster" wording across all `Last Tested` notes for catalog-wide consistency with the earlier compute-cost-per-core cleanup. - Drop the dollar sign from the cost-optimization-index `Last Tested` value and label it as the hub billing currency. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces part of #2111.
Scope:
src/queries/**.Review notes:
dev.memory://projects/finops-toolkit/pr-2111-split-plan.Verification:
git diff --check origin/dev..features/sre-kpi-query-catalog