-
Notifications
You must be signed in to change notification settings - Fork 16
Expand file tree
/
Copy pathcircuitrf.benchmark.runsettings
More file actions
44 lines (38 loc) · 2.05 KB
/
Copy pathcircuitrf.benchmark.runsettings
File metadata and controls
44 lines (38 loc) · 2.05 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
<?xml version="1.0" encoding="utf-8"?>
<!--
The Benchmark opt-in path (docs/sonnet-briefs/brief-test-default-fast.md, section 1). A command-line
`dotnet test` filter argument MERGES with (ANDs against) the project's own circuitrf.runsettings
filter in this SDK's VSTest version rather than overriding it, so filtering for Category=Benchmark
alone would resolve to the impossible AND of Category!=Benchmark and Category=Benchmark and match
nothing. Verified directly, not assumed. Passing a settings file on the command line DOES override
the project-level RunSettingsFilePath cleanly, so this file is the one-liner opt-in path instead:
dotnet test, with settings set to circuitrf.benchmark.runsettings
Runs the 500k-shape TIMED sweeps this project already treats as opt-in-only, unchanged in what they
measure.
-->
<RunSettings>
<RunConfiguration>
<TestCaseFilter>Category=Benchmark</TestCaseFilter>
</RunConfiguration>
<!--
Every run leaves a machine-readable record of WHICH tests failed, at a fixed path
(tests/<Project>/TestResults/last-run.trx), overwritten each run.
This exists because "which tests failed?" was repeatedly costing a SECOND full run to answer. The
console logger does print failures, but a long run's output is routinely piped through a grep, or
scrolled past, or truncated by whatever captured it — and a 1h25m benchmark tier that reports only
"Failed: 5" with no names is an hour and a half thrown away. A .trx is written by the test host
itself, so no amount of downstream filtering can lose it.
Cost is negligible: one XML file written once at the end of the run (a few MB for the ~8,600-test
Ui suite), no per-test overhead. TestResults/ is already gitignored.
Read it with `tools/test-failures.sh` (no argument = the most recently written run).
-->
<LoggerRunSettings>
<Loggers>
<Logger friendlyName="trx">
<Configuration>
<LogFileName>last-run.trx</LogFileName>
</Configuration>
</Logger>
</Loggers>
</LoggerRunSettings>
</RunSettings>