Strip the whole escape_str when unescaping - #93
Conversation
_escape prepends options.escape_str and _symbol_map is keyed on it, but _unescape stripped exactly one character. With a multi-character escape_str, unmarshal did not invert marshal and patch produced both a corrupted key and the stale original.
|
@fzumstein what's the policy on AI-assisted contributions? The fix is valid and tests are sufficient. I'm okay merging this from a technical perspective. |
|
Happy to answer the part I can, and then leave the policy call to you both. Yes, AI-assisted - it is in the PR body and I put it on every PR I open, so it is consistent rather If the answer is that the project would rather not take AI-assisted contributions, that is a One thing that might be relevant either way: the bug predates any of this. |
|
I definitely appreciate the fix and have no objection to merging this. I also appreciate your usage of an agent and keeping a human in the loop. This is responsible use of AI as far as I'm concerned. I need to get buy in from the repo owner on AI though. thanks for your understanding. |
Whatever the active maintainer feels like ;) Sure fine by me! |
|
Awesome, I'll add a note to the readme then. |
|
Thanks for the contribution, @youdie006 ! |
escape_strhas three consumers and one of them ignores it._symbol_mapis built asescape_str + symbol.label(jsondiff/__init__.py:795)_escapeprependsself.options.escape_str(:1059,:1061)_unescapestrips withx[1:](:1034)The module's own rule at the top of the file describes the escaping side:
So
_escapematches the stated rule and_unescapeis the odd one out. It has been that waysince #43 ("Optionally allow different escape_str than '$'", merged 2022-01-24), which threaded
escape_strthrough_escapeand_symbol_mapand left thex[1:]behind.unmarshaltherefore does not invertmarshal:Control, with the default
escape_str='$'- correct, becauselen('$') == 1:End to end through the public API it corrupts a document rather than raising:
The patch writes a mangled key and leaves the original value untouched, so the result is
wrong in two places and nothing reports it.
The change
One line - strip
len(escape_str)instead of one character.No observable change for anyone on the default
escape_str='$', since the two expressionsare identical when the escape string is a single character. That is also why the existing suite
never caught it: every test constructs the differ with the default.
Tests
Two, next to the existing
test_marshal, one for each half:test_marshal_multi_char_escape_stristest_marshalwithescape_str='__', andtest_patch_multi_char_escape_stris the end-to-end case above.Reverting only the one-line change:
pytestis 28 passed with the change.Disclosure: found and prepared with AI assistance (Claude). Every figure above is from a run on
this branch.