Detection quality suffers with rotated cards

[HanClinto]

Still investigating, but detection quality seems to suffer immensely when a card hits a particular "magic" angle. Here is an animation showcasing the investigation and how a single corner behaves as the card is rotated by 1-degree increments:

The green crosshairs represent the peak argmax of the SimCC outputs, whereas the white crosshair represents what the library returns, because it's giving the average / soft argmax (which is averaging between the two peaks of possible corners that the detector is wrestling with).

So as the card rotates towards a fuzzy angle, the detector isn't sure if it should assign index 0 or index 1 to a corner, and it kinda' votes for both. One's higher than the other, but because we're using soft argmax instead of peak argmax, we are splitting the difference and guaranteeing we're getting a wrong answer.

This is a serious problem, and might be at the root of a LOT of our poor-quality detection issues. Stay tuned...

[HanClinto]

Looking at that animation, the important bit there is the two green graphs on the top / left sides -- that's how SimCC works, where there are two 1D arrays that show the best guess for that point's most-likely location on the X or Y axis. Notice how there's usually one peak, but occasionally a second peak appears as the network is kinda' guessing where the corner might be. When the second peak gets high, that's when the detection gets wonky, because we're taking the average of the two high peaks, rather than the max of the peaks.

I've been debugging on this issue for a while, and REALLY glad I was trying to write a tutorial on how SimCC works -- it let me catch this bug. A picture really is worth 1000 words!

[hanclintoclaw-pixel]

Root cause found and fixed in Cornelius 1.221 on Hugging Face.

The failure was not in the SimCC loss itself; it was in inference-time coordinate decoding. We were decoding each SimCC coordinate distribution with a soft expectation / soft-argmax. Around rotated-card tipping points, a corner axis can become multi-modal, and the expectation lands between the two peaks -- producing an implausible corner coordinate and skewing the quad inward.

The fix is to decode SimCC coordinates with peak argmax instead: choose the most likely bin for each coordinate. This is now exported in:

• Hugging Face model repo: https://huggingface.co/HanClinto/cornelius
• Corrected ONNX file: https://huggingface.co/HanClinto/cornelius/blob/main/cornelius-1.221.onnx
• model.onnx has also been updated to point to the corrected 1.221 model: https://huggingface.co/HanClinto/cornelius/blob/main/model.onnx

This release uses the epoch-221 weights and isolates the decoder fix before the later permutation-invariant-loss training continuation.

[hanclintoclaw-pixel]

SimCC decoder note from the Corndog / fastweb-single work: by switching the deployed SimCC head to absolute argmax, we may be giving up one of SimCC's useful advantages: sub-pixel coordinate recovery.

Current behavior in the ccg_card_id detector code is plain peak-bin decoding:

coords.append(logits.argmax(dim=-1).to(dtype=logits.dtype) / float(num_bins - 1))

That keeps the coordinate classification behavior clean and avoids the bad global-softargmax failure mode where probability mass between two plausible peaks averages into an implausible corner. But it also snaps each x/y coordinate to one of the SimCC bins, so with 384 bins at 384x384 input we are effectively limited to about one input pixel of decoder precision.

Suggested low-risk experiments before retraining anything:

1. Add decoder variants and evaluate them post-hoc on the same checkpoints:
   • plain argmax, current baseline
   • local soft-argmax around the winning bin, e.g. +/-2 or +/-3 bins
   • 3-point quadratic/parabolic peak offset from neighboring logits
   • possibly temperature-scaled local soft-argmax
2. Keep the refinement local to the winning peak, not global over all bins, so we preserve the main argmax advantage: no averaging between separate plausible peaks.
3. Compare normal IoU/collapse and the rotated-card stress test. The likely signal is small but meaningful IoU improvement without increased collapse/bad cases.
4. If local refinement helps, CollectorVision should probably expose/use that refined SimCC decode for fastweb-single / Cornelius-style SimCC detectors rather than hard integer-bin argmax.

This is especially relevant because the current fastweb-single checkpoint is close enough to Cornelius IoU that a decoder-only improvement might matter, and it would not require retraining or republishing weights if implemented client-side against the existing ONNX logits output.

[hanclintoclaw-pixel]

Local decoder-only probe results for the SimCC sub-pixel note. This was intentionally local-only / uncommitted, using Corndog epoch 23 (epoch_0023.pt) and original images resized at load time, not the fast pre-resized cache. The goal was A/B behavior on identical rows, not an official replacement for the train4070 full eval.

Normal probe: 4,092 original-image cases sampled across val_packopening, val_listos, test_listos, and test_clint.

decoder	mean IoU	p10 IoU	median IoU	collapse	bad < 0.5	mean movement vs argmax
argmax	0.947532	0.935701	0.974440	7	87	0.0000 px
local r1, temp 1	0.947703	0.935724	0.974483	7	86	0.0043 px
local r2, temp 1	0.947727	0.935748	0.974513	7	86	0.0127 px
local r2, temp 0.5	0.947511	0.935720	0.974587	7	87	0.0249 px
local r3, temp 1	0.947517	0.935712	0.974580	7	87	0.0250 px
parabolic 3-point	0.947990	0.936175	0.974797	7	86	0.2499 px

Rotation-style probe: 4,647 original-image/angle cases using the efficient angle set (-50..50 stress angles) across the same sampled buckets.

decoder	mean IoU	p10 IoU	median IoU	collapse	bad < 0.5	mean movement vs argmax
argmax	0.938479	0.919793	0.973636	5	125	0.0000 px
local r1, temp 1	0.938480	0.919802	0.973641	5	125	0.0045 px
local r2, temp 1	0.938139	0.919651	0.973682	5	127	0.0133 px
local r2, temp 0.5	0.938531	0.919837	0.973695	5	125	0.0260 px
local r3, temp 1	0.938534	0.919838	0.973697	5	125	0.0260 px
parabolic 3-point	0.938729	0.919910	0.973987	5	125	0.2498 px

Takeaways:

1. The effect is small, but real enough to keep testing: parabolic 3-point peak refinement was the most consistently positive variant in both normal and rotation probes.
2. Local soft-argmax is mostly neutral/slightly positive, but less reliable; local r2 temp 1 helped normal IoU but hurt the rotation subset slightly.
3. Collapse did not get worse for parabolic refinement, and bad-case count improved by one in the normal probe.
4. The movement scale is sensible: parabolic shifts coordinates by about 0.25 input pixels on average, which is exactly the kind of sub-bin correction we were hoping for.

Important implementation caveat: the currently published fastweb-single ONNX contract appears to export only corners, presence, and sharpness, not the raw SimCC logits. So this is not client-side switchable against the current ONNX artifact unless we either (a) bake parabolic refinement into the exported corners computation, or (b) publish/export logits or an auxiliary refined-corners output. For compatibility, baking the refinement into corners for a new testing artifact is probably the cleanest next experiment.