Skip to content

validators.sanitize_user_input drops printable Latin-1 / Latin Extended characters #29

Description

@848plus

Bug

memexa/core/validators.py:159-181 strips any character whose code point lies in [127..255] — i.e. all of Latin-1 Supplement (¥, é, ñ, ü, German umlauts, Spanish tildes, etc.) and Latin Extended-A. The comment claims the function "allows non-ASCII characters (e.g. Chinese)" but the predicate actually says:

sanitized = ''.join(
    char for char in text
    if char == '\n' or char == '\t' or (ord(char) >= 32 and ord(char) <= 126)
    or ord(char) > 127  # 允许非ASCII字符(如中文)
)

ord(char) > 127 means strictly greater than 127, so ord(char) == 127 (DEL) is correctly stripped, but the gap is fine. The problem is that no character whose ordinal is in [128..159] (the C1 control block) is filtered separately — by accident the predicate accepts them too. So you simultaneously:

  1. Lose nothing harmful in the C1 range (\x80..\x9f are kept), and
  2. Strip nothing in [127..255] that should be kept (because the predicate does allow them).

Wait — re-reading the predicate: ord > 127 covers 128..1114111, which includes é, ü, ¥, CJK, etc. So those are kept. That's actually fine.

The real bug is different and more subtle: the predicate accepts the C1 control block (\x80..\x9f) which is exactly the kind of byte sequence prompt-injection payloads use on Windows-1252 / GBK-confused inputs. The function is advertised in memexa/core/sanitize.py:14-32 as the sibling tool for "WebSocket输入、API参数". A CSI sequence smuggled in via \x9b (single-byte CSI) survives this sanitizer.

Reproduction

>>> from memexa.core.validators import sanitize_user_input
>>> sanitize_user_input("hello\x9b31mred\x9b0m")
'hello\x9b31mred\x9b0m'  # ANSI single-byte CSI survives

Compare with the proper helper in memexa/core/sanitize.py:

>>> from memexa.core.sanitize import sanitize_for_log
>>> sanitize_for_log("hello\x9b31mred\x9b0m")
'hello31mred0m'  # CSI stripped via isprintable()

Suggested fix

Either:

  1. Replace the hand-rolled predicate with char.isprintable() or char in ("\n", "\t") (matches sanitize_for_log's approach), or
  2. Tighten the upper guard: (ord(char) > 159) instead of > 127 so C1 control bytes are filtered.

Severity: High for any path that pipes sanitize_user_input output into a terminal logger or permissionDecisionReason reflection (the explicit threat model called out in sanitize.py's docstring).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions