Specialized manga/manhwa panel segmentation + reading-order numbering, trained from scratch on low compute (a single consumer GPU, minutes not days).
Give it a page — it masks every panel and numbers them one, two, three… in reading order.
python scripts/predict.py page.png # manga (right-to-left)
python scripts/predict.py page.png --direction ltr # manhwa / webtoon
python scripts/predict.py page.png --masks-dir out/ --json panels.json
Outputs: an overlay PNG (tinted masks + order badges), optional per-panel binary
masks (panel_01.png, …) and a JSON of boxes in reading order.
Two deliberate splits keep this small and specialized:
- Learn only what must be learned. A 3.4M-param U-Net (GroupNorm/GELU conv stages, SDPA attention at the bottleneck for page-level layout context) predicts three channels: panel mask, instance markers (eroded panel cores), and panel boundaries. Instances come out via nearest-marker assignment restricted to the mask — no heavy detection framework, no pretrained backbone.
- Don't learn what geometry already knows. Reading order is a recursive
XY-cut (
panelforge/order.py): horizontal gaps split rows top-to-bottom, vertical gaps split right-to-left (manga) or left-to-right (manhwa), with escalating overlap tolerance for messy layouts. Two-page spreads usually order correctly (right page read fully first) — something naive row-sorting gets wrong. Zero training cost, and one flag retargets manga ↔ manhwa. Known limitation: 4-koma spreads whose gutters align across both pages can interleave the pages; a widest-gap-first cut is planned.
Post-processing rules (defaults): panels below mean-probability 0.9 are
dropped (--min-conf), and panel frames are straightened into polygons
while speech bubbles / figures breaking the frame keep their organic
segmentation border (--refine polygon).
Held-out-books panel F1@0.5: 0.863 raw / 0.898 with the confidence filter (1024-px inference). Early version — a larger multi-component model (panels + text/face/body/balloon masks) is in training.
Dataset: Manga109-derived (final_dataset/), 8210 train pages / 1920 val pages,
val books never seen in training. Targets are built on the fly from the
per-page instance maps.
python scripts/train_panels.py [add_steps] # base model; resume-aware cosine schedule
python scripts/train_panels_v3.py [add_steps] # scaled multi-component model (in progress)
512×512 random scaled crops, photometric jitter + hflip, bf16 autocast, AdamW. Loss = per-channel BCE + soft dice (boundary channel ×2). Eval = instance-level panel F1 at IoU 0.5/0.75 on whole pages.
MIT © DATGMAC