leading spaces lost

[aiwojiagiegie]

When the text begins with spaces, the current Token encoding implementation ignores these spaces, causing the decoded text to lose its original format. For example:

• Enter text: " 助手。我很高兴能" (start with a space)
• Decoded result: "助手。我很高兴能" (leading spaces lost)

String text = " 助手。我很高兴能";
EncodingRegistry registry = Encodings.newDefaultEncodingRegistry();
ENCODING = registry.getEncoding(EncodingType.O200K_BASE);
EncodingResult encodingResult = ENCODING.encodeOrdinary(text, 1);
IntArrayList tokens = encodingResult.getTokens();
String decode = ENCODING.decode(tokens);
System.out.println(decode);