Four-point perspective transformation vs. crop border color

[fonic]

The four-point perspective transformation sometimes does more harm than good, especially when the receipt isn't cut precisely. In this example, the OCR mostly fails due to the transformation distorting the contents.

Would it be possible to modify the transformation to ignore those small imperfections?

Alternatively, would it be possible to crop the image based on color instead, e.g. use a black backdrop when scanning / taking a picture and then automatically crop that black backdrop?

Receipt:

Outline:

[bhimrazy]

Hi @fonic ,
Thanks for raising the issue — much appreciated!

Would it be possible for you to share another complete example where this issue affects OCR? It’d help us better understand the edge cases.

---

Would it be possible to modify the transformation to ignore those small imperfections?

I think that might be possible — for example, we could ensure the Y points are roughly aligned or use the max ends to straighten them. But we’d need to be careful that this doesn’t negatively impact other cases.

would you be interested in contributing to a potential fix?

[bhimrazy]

By the way, I’d love to hear how you’re using this project and how it’s been helpful for you.

[fonic]

Hi @bhimrazy,
thanks for getting back to me. I'm currently evaluating solutions for OCRing paper receipts, both commercial and open source. However, I'm not sure atm if tesseract is up actually to the task, OCR results seem to be unsatisfying even when using really good scans/images of perfectly printed receipts.

As for the issue described above, just use any receipt that does not have well-defined edges (or rip off small parts of the edges of any given receipt), and you'll notice how the four-point transformation distorts the contents, which then affects OCR.

I think that might be possible — for example, we could ensure the Y points are roughly aligned or use the max ends to straighten them. But we’d need to be careful that this doesn’t negatively impact other cases.

That might be a really good idea, some sort of straightening would probably do the trick.