Add metric tuning for custom metrics [feature branch] - #2446

Merged

akwasigroch merged 22 commits into

mainfrom

feat/metric-tuning-test-sets

Aug 26, 2026

Commits on Aug 26, 2026

feat(backend): add metric_id to test_set and test
Arkadiusz Kwasigroch
committed
5e3f44a
feat(backend): add metric tuning cases
Arkadiusz Kwasigroch
committed
42a22f9
feat(frontend): add experimental metric tuning tab
Arkadiusz Kwasigroch
committed
c3d5321
style(backend): format metric tuning router
Arkadiusz Kwasigroch
committed
d607ca4
feat(backend): store the whole case in the prompt content
Arkadiusz Kwasigroch
committed
56a881d
feat(backend): make the tuning case verdict optional
Arkadiusz Kwasigroch
committed
526f04e
feat(frontend): mark tuning cases with no verdict yet
Arkadiusz Kwasigroch
committed
039e856
docs(backend): drop the reserved metric seat claim
Arkadiusz Kwasigroch
committed
34261d1
Add tuning runs for custom metrics (#2470)

akwasigroch
authored and
Arkadiusz Kwasigroch
committed
a43fcb5
fix(backend): rebase metric_id migration onto current head
Arkadiusz Kwasigroch
committed
7b4cabf
fix(backend): resolve get_user from its crud module
Arkadiusz Kwasigroch
committed
de84c5e
refactor(backend): import test crud helpers directly
Arkadiusz Kwasigroch
committed
cbed65a
fix(backend): follow main's requirement rename
Arkadiusz Kwasigroch
committed
b9de69a
Review what the metric said instead of expecting a verdict (#2533)

akwasigroch
authored and
Arkadiusz Kwasigroch
committed
cd3b91b
Expire a tuning run claim that stops advancing (#2537)

akwasigroch
authored and
Arkadiusz Kwasigroch
committed
c3bf70a
Show the metric's agreement over its tuning cases (#2587)

akwasigroch
authored and
Arkadiusz Kwasigroch
committed
c08db48
Narrow the agreement read to the metadata it folds over (#2589)

akwasigroch
authored and
Arkadiusz Kwasigroch
committed
4facad6
Match the metric tuning tab to the rest of the app (#2598)

akwasigroch
authored and
Arkadiusz Kwasigroch
committed
33d2882
Improve a metric from the rejections its reviewers wrote (#2611)

akwasigroch
authored and
Arkadiusz Kwasigroch
committed
c198a61
fix(backend): rebase metric_id migration onto current head
Arkadiusz Kwasigroch
committed
599c18f
test(backend): patch launch_job after the jobs rename
Arkadiusz Kwasigroch
committed
d787b90
style(backend): reformat test set listing after rebase
Arkadiusz Kwasigroch
committed
17081dd

Navigation Menu

Uh oh!

Add metric tuning for custom metrics [feature branch] - #2446

Add metric tuning for custom metrics [feature branch]#2446
akwasigroch merged 22 commits into
mainfrom
feat/metric-tuning-test-sets

Commits on Aug 26, 2026

feat(backend): add metric_id to test_set and test

feat(backend): add metric tuning cases

feat(frontend): add experimental metric tuning tab

style(backend): format metric tuning router

feat(backend): store the whole case in the prompt content

feat(backend): make the tuning case verdict optional

feat(frontend): mark tuning cases with no verdict yet

docs(backend): drop the reserved metric seat claim

Add tuning runs for custom metrics (#2470)

fix(backend): rebase metric_id migration onto current head

fix(backend): resolve get_user from its crud module

refactor(backend): import test crud helpers directly

fix(backend): follow main's requirement rename

Review what the metric said instead of expecting a verdict (#2533)

Expire a tuning run claim that stops advancing (#2537)

Show the metric's agreement over its tuning cases (#2587)

Narrow the agreement read to the metadata it folds over (#2589)

Match the metric tuning tab to the rest of the app (#2598)

Improve a metric from the rejections its reviewers wrote (#2611)

fix(backend): rebase metric_id migration onto current head

test(backend): patch launch_job after the jobs rename

style(backend): reformat test set listing after rebase