FIX: Chunk large inputs to prevent tiktoken-rs stack overflow

[xfalcox]

Summary

• tiktoken-rs v0.9.1 uses fancy-regex for BPE tokenization, which can stack overflow on large inputs due to catastrophic backtracking (openai/tiktoken#245). The Rust code .unwrap()s the error, panics across the FFI boundary, and aborts the entire Ruby process — this is not rescuable from Ruby.
• A secondary issue is quadratic BPE merge time for long non-whitespace runs (openai/tiktoken#195), which causes the process to hang indefinitely.
• Python tiktoken fixed the backtracking in v0.8.0 with possessive quantifiers (openai/tiktoken#258), but tiktoken-rs 0.9.1 and tiktoken_ruby 0.0.15.1 have not ported the fix, and no newer versions are available.

The crash (from GDB / container logs):

tiktoken-rs-0.9.1/src/vendor_tiktoken.rs:267:33:
called `Result::unwrap()` on an `Err` value: RuntimeError(StackOverflow)

thread caused non-unwinding panic. aborting.

Fix

Adds a safe_encode method that chunks text larger than 50K characters at whitespace boundaries before passing to tiktoken. Since BPE token boundaries never span whitespace, chunked encoding produces identical results for normal text. For pathological inputs (e.g. 500K repeated characters with no whitespace), it completes in seconds instead of crashing or hanging.

OpenAiCl100kTokenizer inherits from OpenAiTokenizer and gets the fix automatically.

Test plan

• Verify normal text tokenization is unchanged (encode, decode, truncate, size, below_limit?)
• Verify large text with whitespace produces identical token output to direct encoding
• Verify pathological inputs (e.g. "^" * 500_000) complete without crashing
• Verify cl100k_base encoding also works via inheritance