Type Stubs for subpackages don't work with basedpyright

[LyndonArmitage]

I am not fully versed with this projects ins and outs, but I am encountering an issue with the type stubs when using basedpyright.

Specifically, the type checker seems to default to using the __init__.py files as opposed to the .pyi files when importing submodules.

I noted that there was a change in how type stubs were generated in 0.23.1:

Type stubs are precise (#1928, #1997) — methods that returned Any now return real types; mypy --strict may surface previously-hidden errors. Stub layout also moved from tokenizers/<sub>/__init__.pyi to tokenizers/<sub>.pyi. This breaks the surface of some of the processors like RobertaProcessign's __init__ .

I think it's this stub layout that basedpyright is getting caught up on.

When I do something as simple as:

from tokenizers.models import BPE

It gets the type information from the generated __init__.py file, in this case:

# Generated content DO NOT EDIT

from .. import models

BPE = models.BPE
Model = models.Model
Unigram = models.Unigram
WordLevel = models.WordLevel
WordPiece = models.WordPiece

Rather than models.pyi. I was able to workaround this by copying models.pyi to my own type stubs renamed as __init__.pyi, but that isn't ideal.

Apologies, but I've not read much into PEP-561 and type stubbing in general so I don't know what the best solution to this is. Perhaps creating a stub __init__.pyi file?