Skip to content

NameError: name 'inspect_data' is not defined #1

Description

@kubbyfr

According to the ReadMe to train bpe tokenizer, an error happens:
vocab_size = 2043, target 2048
vocab_size = 2044, target 2048
vocab_size = 2045, target 2048
vocab_size = 2046, target 2048
vocab_size = 2047, target 2048
end traning
vocab size: 2048
starting encoding train text to token ids
start persisting train tokens ids
starting encoding valid text to token ids
start persisting valid tokens ids
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/lkhm/Desktop/ai/llm-from-scratch/llm/bpe_tokenizer.py", line 234, in
inspect_data()
^^^^^^^^^^^^
NameError: name 'inspect_data' is not defined

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions