Skip to content

fix: normalize unicode url_prefix to match PEP 3333 path encoding - #493

Open
gaoflow wants to merge 2 commits into
Pylons:mainfrom
gaoflow:fix-unicode-url-prefix
Open

fix: normalize unicode url_prefix to match PEP 3333 path encoding#493
gaoflow wants to merge 2 commits into
Pylons:mainfrom
gaoflow:fix-unicode-url-prefix

Conversation

@gaoflow

@gaoflow gaoflow commented Jun 29, 2026

Copy link
Copy Markdown

Summary

When url_prefix contains non-ASCII characters (e.g., ü), it fails to match the request path because the two are stored in incompatible encodings:

  • The request path is decoded by the HTTP parser via unquote_bytes_to_wsgi() which percent-decodes the raw URL bytes and decodes the result as ISO-8859-1 (PEP 3333 convention).
  • The url_prefix is stored as a plain Python str without encoding normalization.

This means the WSGI application receives an incorrect PATH_INFO — the url_prefix is never stripped — and the SCRIPT_NAME is not reported in the correct PEP 3333 encoding.

Fix

Normalize the url_prefix in slash_fixed_str by encoding the user-supplied Unicode string as UTF-8 and then decoding as ISO-8859-1, matching the convention used by the HTTP request parser. For ASCII-only prefixes this is a no-op.

Testing

Added test_url_prefix_unicode_normalization that verifies:

  1. A non-ASCII url_prefix is stored in PEP 3333 format.
  2. An ASCII-only url_prefix is unchanged.
  3. The normalized url_prefix equals the parser-produced path for the same logical URI.

All 794 existing tests pass (10 skipped, same as before).

Closes #492

@kgaughan kgaughan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fine, but for some reason the CI pipeline didn't trigger. Could you update CONTRIBUTORS.txt and CHANGES.txt? Those need updating anyway, and pushing an additional changeset should give Github a poke that it needs.

Also, was this generated by an LLM or is it your own work.

@gaoflow

gaoflow commented Jul 31, 2026

Copy link
Copy Markdown
Author

CHANGES.txt entry added in 1bd6c93 — the push should give CI the poke it needed.

On CONTRIBUTORS.txt: that list is the contributor-agreement signature, so the account owner will add their name themselves rather than me doing it on their behalf.

And to answer your question directly: yes, the PR is AI-assisted — drafted by an AI coding agent under the account owner's direction and reviewed by them before submission.

@kgaughan

Copy link
Copy Markdown
Member

OK, that's potentially problematic. I'll have to raise this in the Pylons Discord.

@kgaughan

Copy link
Copy Markdown
Member

Also, I want to avoid reputation laundering.

@gaoflow

gaoflow commented Jul 31, 2026

Copy link
Copy Markdown
Author

Totally fair — the account owner is a real person and stands behind the work, but if the project would rather not take AI-assisted PRs, happy to close this one. No hard feelings either way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Waitress fails to match unicode url_prefix

2 participants