265 of 268 aggregations declare no metric — the engine cannot compute a value, and every one returns an empty result

[rubenvdlinde]

265 of 268 aggregations are written in a vocabulary OpenRegister does not read

AggregationRunner::run() reads exactly nine keys off an aggregation spec.
Measured from source, not from docs:

$ grep -oE "spec\['[a-zA-Z]+'\]" lib/Service/Aggregation/AggregationRunner.php | sort -u
spec['field']  spec['filter']  spec['from']   spec['groupBy']  spec['join']
spec['metric'] spec['metrics'] spec['select'] spec['where']

metric / metrics is the only way the engine computes a value. A spec
without one returns "metric": "", "field": null and "groups": [] — an empty
result with HTTP 200, which every page renders as its own empty state and reads
as "no data yet".

Observed live on the rig, before and after removing the @self placeholder:

{"name":"byCostCenter","metric":"","field":null,"backend":"php-fallback",
 "truncated":false,"groups":[], ...}

This is #1216's first layer, unswept

#1216 had four causes
stacked on one page. The first was "the declaration spoke a grammar the engine
does not read" — it used source and a sum list. That was fixed for one
aggregation. It is still true of 265 others.

It also explains why nobody noticed #1255's @self.administrationId problem:
these aggregations were already returning nothing for a second, independent
reason, so removing the placeholder changes an empty result into a different
empty result.

Total aggregations declared: 268
Declaring metric or metrics (the only way the engine computes a value): 3
Cannot produce a value: 265

Those that CAN:

• CurrencyBalance.balanceByCurrency
• CommitmentLine.committedVsRealisedPerBudgetLine
• APInvoice.apAging

Keys the engine never reads, by frequency

key	occurrences	example
source	112	IBWinstOpgave.revenueExcludingVat (30-bookkeeping-ib-aangifte-zzp.json)
operations	92	IB47Record.ib47TotalsPerTaxYear (add-shillinq-detachering-payroll-administratie.json)
operation	73	Appointment.countByStatus (10-bookings-create-appointment.json)
sum	26	IBWinstOpgave.revenueExcludingVat (30-bookkeeping-ib-aangifte-zzp.json)
expression	19	StockMove.netQuantityForLocation (inventory-stock-movement-ledger.json)
type	15	CommitmentBudget.outstanding_commitments (bookkeeping-verplichtingenadministratie.json)
sourceSchema	14	GLLine.iv3DetailByTaakveldCategorie (shillinq_register.json)
label	12	ARInvoice.arAging (add-shillinq-bookkeeping-compliance.json)
register	12	ARInvoice.arAging (add-shillinq-bookkeeping-compliance.json)
schema	12	ARInvoice.arAging (add-shillinq-bookkeeping-compliance.json)
sourceRegister	10	Iv3Export.buckets (shillinq_register.json)
computedFields	7	ARInvoice.arAging (add-shillinq-bookkeeping-compliance.json)
params	6	GLTransaction.nivraBericht (add-shillinq-rekenkamer-audit-pack.json)
output	6	VpbBalansLink.vpbBalansFiltered (bookkeeping-vpb-corporate-tax-balans.json)
meta	5	GLTransaction.nivraBericht (add-shillinq-rekenkamer-audit-pack.json)
orderBy	5	APTransaction.agedPayablesDetail (bookkeeping-accounts-payable-core.json)
aggregate	5	APTransaction.agedPayablesSummary (bookkeeping-accounts-payable-core.json)
sources	5	ExpenseClaimEntry.settlementTotals (expense-reimbursement-or-passthrough.json)

What this costs

Every page reading one of these renders an empty state over live data. That is
not a hypothetical: it is exactly what #1216 was, and it took a live instance to
find, because every unit test, validator and gate was green throughout — a
test that asserts the declaration's own shape compares it to itself.

Fixing it

Each declaration needs translating into the engine's grammar with its semantics
preserved — source → from, sum: [x] → metric: "sum", field: "x",
operation → metric, and the operations list → metrics. Mechanical in
shape, but 265 of them, and each one's meaning has to be checked rather than
pattern-matched. It should be done in batches with a live instance to confirm
each batch actually returns rows.

Some are not translations at all and need redesign:

• TaxEstimate.ytdByCategory declares its filter as a SQL-like string;
• four SBRDocumentType aggregations use join.schema + join.condition (a SQL
  string) plus a check block — an entire invented sub-language.

Preventing more

tests/validate-registers.js now ratchets this: a new aggregation that declares
neither metric nor metrics pushes the count above its baseline and fails the
build. The baseline can only fall.

The alternative — implementing source/sum/operation as aliases in
OpenRegister — would make 265 declarations start working at once, with no
per-declaration check that the result is the intended one. That seems like the
more dangerous direction, but it is a real option and worth a decision rather
than an assumption.

[github-actions]

Thank you for opening this issue! It has been assigned to @rubenvdlinde for review.

What happens next:

1. A maintainer will review this issue and determine the appropriate action
2. If accepted, it may be converted into an OpenSpec change for structured development
3. You will be notified of any status changes

Triage labels:

• triage:accept-feature — accepted as a new feature
• triage:accept-bugfix — accepted as a bug fix
• triage:needs-info — more information needed
• triage:wontfix — will not be addressed
• triage:duplicate — duplicate of another issue

[rubenvdlinde]

46 of the 265 translated, and a second defect found underneath

PR #1263. operation: count|sum|avg on an aggregation over its own schema
maps faithfully to metric — count ignores field and counts rows either
way. That is 265 → 219 on the ratchet.

The part worth reading: renaming operation alone produces a WRONG NUMBER

AggregationQuery::normaliseGroupByFields() opens with

if (is_array($groupBy) === false) { return []; }

42 of the 46 declared "groupBy": "status" as a string, which is silently
dropped. So the obvious translation — rename operation, ship it — makes them
return one ungrouped total where the page asked for a breakdown.

Measured live, same aggregation at each stage:

stage	response
before	{"metric":"","field":"status","value":null}
operation → metric only	{"metric":"count","value":8} ← wrong
+ groupBy as an array	{"groups":[{"key":"active","value":8}]}

value: 8 is the grand total of all services, dressed as a per-status count.
That is worse than the empty result it replaced, and it is exactly why the
remaining 219 should not be bulk-translated: each needs a live check, not a
pattern match.

A zero-tolerance gate now rejects a string groupBy on any metric-bearing
aggregation, with a message that says the grouping is dropped rather than
rejected. 41 string groupBys remain on aggregations that have no metric —
latent rather than wrong, since those return nothing anyway. They will need
fixing at the same time as their metric.

Still open here

• 19 simple metrics whose source names a different schema. source
  does not map to from: from switches the runner into its cross-schema
  path, which needs a parent row. Redesign, not rename — and these are the
  GL/tax ones.
• ~200 using operations dicts with custom aliases (the engine emits
  sum_<field>, so every consumer changes too), expression SQL strings,
  and the SBRDocumentType check sub-language.

Reminder of the shortcut, and why I would still argue against it

Teaching OpenRegister to read source/sum/operation/operations would make
all 219 start returning data at once. The three-stage table above is the reason
not to: the failure mode here is not "no data", it is "a number that looks
right". One engine change would ship 219 of those in a single commit.

[rubenvdlinde]

Not all 219 matter equally — only 64 are actually consumed

• aggregations declared: 263
• referenced anywhere outside lib/Settings (src/, services, controllers, reporting, manifest): 67
• never referenced at all: 196
• consumed AND unable to compute a value: 64

A declaration nothing reads is not a user-visible defect — it is dead weight that
will bite whoever first builds a page on it. The 64 below are the ones a user
looks at today and sees an empty table.

The 64, grouped by declaring schema

• APTransaction — agedPayablesDetail, agedPayablesSummary, agedPayablesTimeline, vendorOpenApBalance
• ARInvoice — arAging
• Account — emuSaldoByQuarter
• AnalyticalDimension — segmentPnl, segmentPnlByCostObject, spentToDate
• BBVProgramme — complianceStatus, totalBudget, utilization, ytdSpend
• BalanceSheet — balanceSheetTotals
• BcfClaim — compensableVatBreakdown
• BtwAangifte — totalsByReturn
• CashflowForecastHorizon — bufferPolicyEvaluation
• ConsolidationPeriod — totalEliminationAmount, totalEliminationCount
• DepreciationSchedule — accumulatedDepreciationByAsset
• EMUReport — totaleAdjustments
• EuVatRate — rateByCountryCategoryDate
• GLLine — byAnalyticalDimension, byCostCenter, byCostCenterHierarchy, byProject, consolidatedTrialBalance, programmeBudgetVsActuals
• IBProfitAttribution — innovatieboxAdministratie
• IbAangifteExport — costs, profit, revenue
• IcpStatement — lines
• IcpSupply — icpLedgerByBuyerSupplyType
• IncassoKostenBerekening — bikStaffel, renteAccrual
• InnovatieboxElection — innovatieboxAdministratie
• InventoryReorderRule — lowStockDashboard, lowStockTrigger
• Iv3Export — buckets
• KORAnnualTurnover — korTurnoverByYear
• KorRegime — ytdRevenue
• OssReturn — linesByCountryRate
• Payroll — netAmount
• Programma — programmaRollup
• Programmabegroting — sluitendByBegroting
• Project — billedRevenue, costsIncurredToDate, profitAndLoss, segmentPnl
• QuarterlyTaxStatement — quarterlyTaxStatement
• RateSchedule — resolveRate
• ReconciliationMatch — candidateMatches
• SchatkistPosition — position
• SoUrenStaat — mededeling
• TrialBalance — trialBalanceTotals
• TrialBalanceLine — trialBalanceByAccountPeriod
• VATAuditRecord — vatByPeriod
• VatReturn — inputVat, rubrieken
• VpbBalansLink — orphanedVpbPligtigAccounts, vpbBalansFiltered
• ZzpDeduction — taxableProfit, ytdQualifyingHours

What this means for #1263

The 46 translated there are all in the unconsumed set. So that PR makes them
correct and installs two gates, but it moves nothing a user sees. Worth being
explicit about rather than letting "46 aggregations fixed" imply otherwise.

Suggested order

1. The 64 above, live-checked in small batches. They are also the hard ones
   — GL, VAT, tax, AP/AR ageing — which is exactly why they need a real instance
   per batch rather than a bulk pass.
2. The remaining ~155 unconsumed only when something is built on them, or
   delete the ones nobody intends to use. A declaration that has never been read
   is cheaper to remove than to translate.

The measurement script is inline in this issue's history; the counts here come
from matching each aggregation name across src/, lib/Service,
lib/Controller, lib/Reporting and the manifests, excluding lib/Settings
itself (where the declaration lives).

[rubenvdlinde]

Batch 1 done, and a classification that changes the shape of this issue

#1269 translates the segment-P&L group and verifies it against the rows.
Three of those seven translate; four cannot be translated at all. That
prompted a proper classification of all the consumed-and-broken aggregations,
and the result reframes the work.

The 62 consumed-and-broken, by what they actually need

bucket	count
translatable by renaming alone	1
custom aliases — declaration and every consumer	17
needs a capability the engine does not have	44

Only one of them — EuVatRate.rateByCountryCategoryDate — is a pure rename.
This is not, in the main, a repo of badly-spelled declarations.

Missing capabilities, ranked by how many need them

#	capability
11	groupBy a joined or wildcard field (AnalyticalDimension.parentCode, dimensions.*)
10	conditional metric — {"operation":"sum","condition":"GLLine.side = 'debit'"}
8	sourceSchema / sourceRegister
8	join.condition as a SQL string
5	computedFields — date buckets (agingBucket)
3	custom return projection
3	expression strings inside aggregate (incl. window.sum(...))
3	source as {register, schema}
3	top-level expression
2	buckets, one of them read from IAppConfig
1	recursive rollup (recursiveOver)

The one I would argue is worth building

A conditional metric. sum WHERE side = 'debit' is the debit/credit split,
and it is load-bearing in TrialBalance.trialBalanceTotals,
TrialBalanceLine.trialBalanceByAccountPeriod, BalanceSheet.balanceSheetTotals,
GLLine.consolidatedTrialBalance and both segmentPnl variants. Without it,
essentially no accounting report in this app can be declarative, which is what
ADR-031 asks for. It is also the most contained of the eleven: a condition on
a metrics entry, evaluated with the filter machinery that already exists.

groupBy on a joined field is the larger count but a much deeper change — it
means grouping the parent by a column that only exists after the join.

One more silent-wrong-answer, found while classifying

ARInvoice.arAging filters "lifecycleState": {"not-in": ["paid","written-off"]}.
The engine's operator is notIn, camelCase. not-in does not match it, and
an unrecognised operator falls through to default => true — so the filter
passes everything rather than excluding paid and written-off invoices. Once
that aggregation is given a metric, it will produce an AR ageing report that
silently includes settled invoices. Same family as the groupBy-as-a-string
trap in #1263: not empty, just wrong.

Suggested next step

Decide on the conditional metric before translating any more of the 64.
Roughly a third of them are waiting on it, and translating around it produces
declarations that have to be rewritten again once it lands.

[rubenvdlinde]

Conditional metrics shipped (openregister#2904) — and a correction

metrics[].condition (a filter object) plus metrics[].as are on
openregister development. Verified live on a real ledger:

GLLine.consolidatedTrialBalance
  1300  {"totalDebit": 12100, "totalCredit": 0}
  8000  {"totalDebit": 0,     "totalCredit": 10000}
  1510  {"totalDebit": 0,     "totalCredit": 2100}
  4400  {"totalDebit": 15000, "totalCredit": 6000}

Account 4400 carries rows on both sides — debit 5000 + 10000, credit
3000 + 3000 — so one group exercises both conditions. Every figure matches the
underlying rows. Had the conditions been dropped, both columns would read 21000.

The correction

I estimated this would unblock ~10 aggregations. It unblocks one.

My classifier flagged condition as the blocker but did not notice that
TrialBalance.trialBalanceTotals and BalanceSheet.balanceSheetTotals aggregate
field: "GLLine.amount" while being declared on TrialBalance and
BalanceSheet — those are cross-schema, which is a different capability
entirely. Both also want operation: "check" with an expression
(totalDebits = totalCredits) for a computed balanced flag.

Re-checked, precisely:

aggregation	remaining blocker
GLLine.consolidatedTrialBalance	none — translated
TrialBalance.trialBalanceTotals	cross-schema field + check/expression
BalanceSheet.balanceSheetTotals	cross-schema field + joined groupBy + check
TrialBalanceLine.trialBalanceByAccountPeriod	joined groupBy
AnalyticalDimension.segmentPnl / segmentPnlByCostObject	join.condition + joined groupBy
Project.segmentPnl	join.condition + joined groupBy

Conditional metrics remain a prerequisite for all seven — none of them can be
expressed without it — but six need a second capability on top.

What that implies for the ranking

The capability counts I posted earlier count each aggregation once per missing
capability, so they overstate what any single feature unlocks. The honest
reading is that most of the 62 need two or more capabilities, and the
next-largest single one is:

• a joined-field groupBy (grouping the parent by a column that only exists
  after the join) — 11 need it, and it is the gate on four of the seven above.

That is a materially deeper change than condition was: condition reused
applyFilter() wholesale, whereas grouping on a joined column means the group
key is not available until after the join is resolved.

Also shipped

Unknown filter operators now throw instead of matching every row
(default => true). APInvoice.apAging's not-in — the implemented spelling is
notIn — was quietly including settled invoices in an AR-ageing report; that now
fails loudly. The other unimplemented spellings (equals, not, between,
gteOrNull, notStartsWith, not_in) sit on aggregations that compute nothing
yet, so they will surface when someone gives them a metric.