Skip to content

Commit 65f3fa6

Browse files
Document bottleneck-split measurement: CPU-bound, not I/O-bound
Fresh instrumented 50-card run against current code shows fetch is only ~13% of per-card cost; border/frame classification + pass-2 fallback dominate at ~65-72%. No separate fetch-thread pool exists currently - _compute_card's single worker pool handles fetch+OCR+ phash+fallback together. This favors a core-count resize or manifest mode over decoupling fetch threads. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016i9S7LQsCL3FGaih3ZTRBJ
1 parent 25eb8aa commit 65f3fa6

1 file changed

Lines changed: 44 additions & 0 deletions

File tree

docs/features/printing-tags.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1641,6 +1641,50 @@ vote exists), and a test proving the efficiency win itself (an absorbed member's
16411641
never passed to `run_ocr_for_card`, not just that it ends up with a vote). mypy clean.
16421642
`black`/`prettier` clean.
16431643

1644+
### Bottleneck split, current pipeline state (2026-07-16, throughput track item 2a)
1645+
1646+
Re-measured phase timing against the CURRENT code (post items 1/2a/3/4/4b) rather than trusting
1647+
item 3a's original breakdown, which predates dpi=250, crop tightening, bleed-first
1648+
classification, and clustering entirely. Real instrumented run, 50 selected candidates
1649+
(representative-only, post-clustering), against the live DB/API - not simulated:
1650+
1651+
| phase | mean/card | share (uncorrected) |
1652+
| ------------------------------------- | --------: | ------------------: |
1653+
| `fetch_card_image` | 0.450s | 13.4% |
1654+
| `classify_bleed_edge` | ~0.000s | ~0.0% |
1655+
| OCR (crop+preprocess+tesseract) | 0.478s | 14.3% |
1656+
| phash (hash+compare) | ~0.000s | ~0.0% |
1657+
| border/frame (`detect_illus_anchor`+) | 1.206s | 36.0% |
1658+
| pass-2 fallback | 1.218s | 36.3% |
1659+
1660+
**Measurement caveat, stated plainly**: this run called fallback unconditionally for every
1661+
representative (not gated on pass-1's real accept/reject outcome), so its 36.3% share is
1662+
inflated relative to real `run_pilot` behavior (item 3a's original sample: fallback fires
1663+
~70% of the time). Corrected estimate using that same 70% rate:
1664+
`0.450 + 0.478 + 1.206 + (1.218 × 0.7) ≈ 2.99s/card` sequential, for cards that reach full
1665+
compute (clustering representatives only).
1666+
1667+
**Bonus real data point from the same sample**: 13/50 selected cards (26%) were absorbed into
1668+
10 clusters by item 2a's dedup - a materially higher rate than assumed, though from one
1669+
50-card sample, not a claim about the full-catalog rate.
1670+
1671+
**The clear finding: this is CPU-bound, not I/O-bound.** `fetch_card_image` is ~13% of
1672+
per-card cost; `detect_illus_anchor`-plus-border-classification and pass-2 fallback together
1673+
are ~72% (uncorrected) / ~65% (corrected). This directly answers throughput track item 2a's own
1674+
question: **the "6-8 fetch threads, I/O-bound, no core needed" idea does not currently exist as
1675+
a mechanism** - `_compute_card`'s single `ThreadPoolExecutor(max_workers=workers)` runs fetch
1676+
AND OCR AND phash AND fallback all in the SAME worker, sized for CPU-bound work
1677+
(`DEFAULT_WORKERS=2`, matching this box's core count). Decoupling fetch into its own larger pool
1678+
would only ever attack the ~13% fetch share - a real potential improvement, but not the
1679+
dominant cost, and not built in this pass. This bottleneck split is the evidence that makes
1680+
manifest mode (item 2c) and a core-count resize (item 2b) the higher-leverage levers, not a
1681+
larger fetch pool.
1682+
1683+
**Current instance shape** (OCI instance-metadata endpoint, no auth needed - confirmed
1684+
`169.254.169.254/opc/v2/instance/`): `VM.Standard.A1.Flex`, **2 OCPUs, 12 GB RAM**,
1685+
`ca-montreal-1`. Matches `DEFAULT_WORKERS=2`'s own derivation (item 3d) exactly - this box has
1686+
never had spare cores for a bigger pool without a resize.
1687+
16441688
### No-match autopsy (2026-07-15, post-merge Hold #1 of the pre-scale program)
16451689

16461690
Classified all 176 OCR "parsed-but-no-match" cases from the pilot run

0 commit comments

Comments
 (0)