cl100k

Here is 1 public repository matching this topic...

tiktoken in pure Python: byte-for-byte compatible BPE tokenizer for GPT-4/4o, no Rust extension, zero dependencies

tokenizer wasm openai pure-python gpt bpe pyodide llm tiktoken token-counting cl100k o200k

To associate your repository with the cl100k topic, visit your repo's landing page and select "manage topics."