What we are asking for
| # |
Ask |
Where |
| 1 |
Read the frame rate the camera declares, and drop the hardcoded 14.95 |
Makefile:45, stream_density.py:41 |
| 2 |
Keep the measured per-stream rate and report it next to the target |
stream_density.py:936 |
| 3 |
Write the pass condition into the result in words |
run summary writer |
Ask 3 means the target, the rate a stream has to hold to pass, how many consecutive measurement
windows have to agree before the stream count changes, and how long a window is. Four numbers and
a sentence, all of them in hand at the point the result is written.
Acceptance criteria
- The per-stream target comes from the camera config's declared rate, with
TARGET_FPS still
able to override it.
- The result states the target, the pass mark and the measured rate as three separate numbers.
- The result states the pass rule, the window count and the window length in words.
- Every number in the result can be found in, or derived from, the raw log without reading source.
Context
Why 15, and what it does to the pass mark
Ask 1 gives a headline target of 15 taken from the camera's own declared rate, with a pass mark
of 14.25, which is 95% of it. That is the same arithmetic as today to within 0.05 fps, now a
stated rule instead of a constant. It also fixes a case nobody has hit yet: anyone benchmarking
a 30 fps camera today inherits a target derived from somebody else's 15 fps setup. TARGET_FPS
stays available for anyone who wants to override it.
What we found
A stream density result reports 14.95 fps. No camera achieved 14.95 fps. The raw log for the
same run contains 14.2025, and neither the summary nor the documentation explains how the two
relate, so a reader has to open the source to find out.
Three things combine to produce that:
| Value |
Set to |
File and line |
| Target frame rate |
14.95 |
loss-prevention Makefile:45 |
| Pass ratio |
0.95 |
performance-tools stream_density.py:34 |
| Resulting pass mark |
14.2025 |
14.95 × 0.95, the value repeating in the raw logs |
The achieved per-stream rate is computed for the pass decision at stream_density.py:936 and
then dropped before the result is written, so a run whose cameras held the pass mark is
recorded as 14.95.
Separately, every camera config already declares "fps": 15. The code that works out each
stream's target reads a different, optional key called targetFps and falls back to the global
14.95 when it is missing, which it is in every config that ships.
Why it matters
Two independent reviews of the ASC benchmark could not reconcile the reported number against the
logs the same run produced. One of them took two days. A partner who hits that gap will conclude
the number was chosen rather than measured, and we do not get to explain afterwards.
What the result would say
Stream density result
-----------------------------------------------
Streams sustained 12
Target per camera 15.00 fps (declared source rate)
Pass mark per camera 14.25 fps (95% of source rate)
Measured, slowest camera __.__ fps
Measurement window ___ s (after settle)
A camera passes when it sustains at least 95% of the
rate its source publishes. Every camera must pass for
the stream count to be accepted. The stream count is
only changed after two consecutive windows agree.
Field names are a suggestion. The window length is blank because it is not defined anywhere
today, which is a separate issue.
Related
#245 changes the statistic the pass decision is taken on. This issue changes what the result
reports about that decision. They touch the same summary code and are easier to do together than
one after the other.
#232 asks for per-stream input FPS sourced from camera_to_workload_*.json for heterogeneous
camera sets. This issue is the narrower case of the same mechanism: read the declared rate
rather than a constant. Whoever picks either one up should look at both.
Line references verified against performance-tools @ 144d3b6 and loss-prevention @ f865f5b, read 31 Aug 2026.
What we are asking for
Makefile:45,stream_density.py:41stream_density.py:936Ask 3 means the target, the rate a stream has to hold to pass, how many consecutive measurement
windows have to agree before the stream count changes, and how long a window is. Four numbers and
a sentence, all of them in hand at the point the result is written.
Acceptance criteria
TARGET_FPSstillable to override it.
Context
Why 15, and what it does to the pass mark
Ask 1 gives a headline target of 15 taken from the camera's own declared rate, with a pass mark
of 14.25, which is 95% of it. That is the same arithmetic as today to within 0.05 fps, now a
stated rule instead of a constant. It also fixes a case nobody has hit yet: anyone benchmarking
a 30 fps camera today inherits a target derived from somebody else's 15 fps setup.
TARGET_FPSstays available for anyone who wants to override it.
What we found
A stream density result reports 14.95 fps. No camera achieved 14.95 fps. The raw log for the
same run contains 14.2025, and neither the summary nor the documentation explains how the two
relate, so a reader has to open the source to find out.
Three things combine to produce that:
loss-preventionMakefile:45performance-toolsstream_density.py:34The achieved per-stream rate is computed for the pass decision at
stream_density.py:936andthen dropped before the result is written, so a run whose cameras held the pass mark is
recorded as 14.95.
Separately, every camera config already declares
"fps": 15. The code that works out eachstream's target reads a different, optional key called
targetFpsand falls back to the global14.95 when it is missing, which it is in every config that ships.
Why it matters
Two independent reviews of the ASC benchmark could not reconcile the reported number against the
logs the same run produced. One of them took two days. A partner who hits that gap will conclude
the number was chosen rather than measured, and we do not get to explain afterwards.
What the result would say
Field names are a suggestion. The window length is blank because it is not defined anywhere
today, which is a separate issue.
Related
#245 changes the statistic the pass decision is taken on. This issue changes what the result
reports about that decision. They touch the same summary code and are easier to do together than
one after the other.
#232 asks for per-stream input FPS sourced from
camera_to_workload_*.jsonfor heterogeneouscamera sets. This issue is the narrower case of the same mechanism: read the declared rate
rather than a constant. Whoever picks either one up should look at both.
Line references verified against
performance-tools@144d3b6andloss-prevention@f865f5b, read 31 Aug 2026.