Add Santali (sat) - first Ol Chiki script LSTM model

[psubhashish]

Adds the first Tesseract LSTM model for Santali (Ol Chiki script, ISO 639-3: sat).
No Santali model exists in tessdata, tessdata_best, or tessdata_contrib.

• Fine-tuned from Latin.traineddata (Ol Chiki is alphabetic, not abugida)
• Trained on balanced 1:1 synthetic + real scan data
• Fonts: Noto Sans Ol Chiki, GuruGomke, October
• Scan source: ᱜᱟᱱᱟᱝ ᱟᱨ ᱪᱤᱴᱷᱤ ᱥᱟᱠᱟᱢ (Gānāṁ ār ciṭhi sākām) (2008) by Hajam Baskey (under CC BY-SA 4.0)
• Synthetic text sources: OpenSpeaks (translations by R Ashwani Banjan Murmu, Fagu Baskey and Joy sagar Murmu) (CC BY SA 4.0), OpenSpeaks Community language documentation and archiving training - 1 (translation by Bodi Baski) (CC BY SA 4.0)
• Current accuracy: 59–75% CER on book scans (spaces stripped)
• Early release — accuracy improving with additional training data

Attribution:

• Trained by Subhashish Panigrahi (OpenSpeaks)
• Ramjit Tudu — source material and background on Santali OCR
• Scan training data from ᱜᱟᱱᱟᱝ ᱟᱨ ᱪᱤᱴᱷᱤ ᱥᱟᱠᱟᱢ (2008) by Hajam Baskey
• Indic OCR by RKVS Raman used for comparison benchmarking
• Fine-tuned from Latin.traineddata (Google, Apache 2.0)

[psubhashish]

Hi @zdenop @stweil: I wanted to check if I've submitted the Ol Chiki traineddata at the right repo or not.

I didn't see any LSTM-based model for Ol Chiki anywhere in Tesseract's official repositories (tessdata_best, tessdata, tessdata_contrib, etc.). The only one is in the Indic-OCR project. However, that's a pre-4.0 legacy model. Based on discussions with the Santali speakers, and with the help and guidance from some of them, I started this so that they can take over. Ramjit Tudu and Prasanta Hembram, both Santali speakers with whom I have also worked since 2018, have previously worked on Tesseract training. Both expressed their challenge given the lack of an existing model.

I used some of the scripts used in the Odia project, a Latin model to train, as mentioned earlier. Please let me know if there is anything I might have missed.

[stweil]

New models are normally added to https://github.com/tesseract-ocr/tessdata_contrib, but there is no need to create a new pull request there: we can move this one to a different repository any time.

It is more important to add missing parts. One of the main problems with tessdata is that there is no documentation which allows to reproduce (and potentially improve) the training of all models here. Therefore I want that new models are only added with a complete description of the training process and with any data which was used for the training.

[psubhashish]

New models are normally added to https://github.com/tesseract-ocr/tessdata_contrib, but there is no need to create a new pull request there: we can move this one to a different repository any time.

It is more important to add missing parts. One of the main problems with tessdata is that there is no documentation which allows to reproduce (and potentially improve) the training of all models here. Therefore I want that new models are only added with a complete description of the training process and with any data which was used for the training.

@stweil: Inshallah. Done! :)

[psubhashish]

Hi @stweil: quickly checking if there is anything that might be required to merge this. Thanks.