Updating ultrajson#1
Open
WilliamBinquet wants to merge 801 commits into
Open
Conversation
Doing so allows for building on webassembly. Officially declareing support for WASM however will have to wait until pyodide/pytest-pyodide#59 is merged to facilitate testing WASM compatibility.
Committed via https://github.com/asottile/all-repos
Due to a bug in PyPy [1], PyObject_GetBuffer() fails to detect non C contiguous inputs. Allowing ujson.loads() to run on non-contiguous buffers such as a strided NumPy array would lead to mayhem. Approximate the intention of PyObject_GetBuffer() by explicitly checking that the input is within the set of common bytes-like objects known to be unconditionally C contiguous. - [1]: https://foss.heptapod.net/pypy/pypy/-/issues/3872
updates: - [github.com/asottile/pyupgrade: v2.38.2 → v3.3.1](asottile/pyupgrade@v2.38.2...v3.3.1) - [github.com/psf/black: 22.8.0 → 22.12.0](psf/black@22.8.0...22.12.0) - [github.com/PyCQA/isort: 5.10.1 → 5.11.4](PyCQA/isort@5.10.1...5.11.4) - [github.com/PyCQA/flake8: 5.0.4 → 6.0.0](PyCQA/flake8@5.0.4...6.0.0) - [github.com/pre-commit/pre-commit-hooks: v4.3.0 → v4.4.0](pre-commit/pre-commit-hooks@v4.3.0...v4.4.0)
[pre-commit.ci] pre-commit autoupdate
Update github-actions
Re: PyCQA/isort#2078 Committed via https://github.com/asottile/all-repos
The stdlib json module has supported these since Python 3.4. Since `enum.IntEnum` is an `int` subclass (and hence `PyLong_*` work as expected), ujson has supported them for a long time as well, but this was so far not tested.
Add test for int- and float-derived Enums
Both of these licenses require the copyright notice and the license text to be distributed in all copies. Instead of merely mentioning them, include the full text in LICENSE.txt Fixes #565; see that issue for further analysis.
…ed-repository_url Replace deprecated repository_url with repository-url
Update pypa/cibuildwheel action to v2.12.1
updates: - [github.com/psf/black: 22.12.0 → 23.3.0](psf/black@22.12.0...23.3.0) - [github.com/pre-commit/pygrep-hooks: v1.9.0 → v1.10.0](pre-commit/pygrep-hooks@v1.9.0...v1.10.0)
* Fix off by one errors in detecting end of string mid sequence * Add missing check for codepoints > max unicode * Add missing check for bad continuation bytes
GraalPy's str.decode() error spells utf-8 as utf_8. Also add testing on ubuntu+graalpy to CI
Similar to 169eaf3 but for `ujson.loads(b'"garbled utf-8 stuff"')`. Fixes: * Continuation bytes without preceding start byte being silently coerced into other characters. * SystemError when decoding a 4-byte character > MAX_UNICODE (caused by us not checking then PyUnicode_New() complaining about what we gave it). (#716) * Allowing continuation bytes with an invalid second bit (0b11xx_xxxx as opposed to 0b10xx_xxxx).
PyIter_Check() is used to skip straight to the container types when type-forking during serialisation. However, PyIter_Check() only returns true for generators – not arbitrary iterables – and since we don't allow generators to be used in place of lists, this shortcut is never used. (If it did work, then it would also break strings since they are iterable but not to be treated as containers.)
All strings are encoded with surrogatepass so they can always be converted to UTF-8. Only memory errors can reach *PyUnicodeToUTF8Raw() failed* code path.
decode_string() writes to an `escBuffer` (ds->escStart to ds->escEnd). `escBuffer` is initialised as a 256 x JSUINT32 array on the stack but can be replaced with a larger buffer malloc()ed on the heap. There is code to resize it up again but it's never used. The code is pretty unintelligible. This is what I think it does: * ds->start to ds->end represent the current position within JSON payload (start of a string excluding the leading ") and the end of payload respectively * ds->escHeap initialises to false (per parser – not reset if parsing a second string within one ujson.loads() call) * `escBuffer` (ds->escStart to ds->escEnd) is initialised as a 256 x JSUINT32 array * If ds->end - ds->start (remaining bytes to parse in whole JSON) is larger than escBuffer: - resize escBuffer to ds->end - ds->start using non-heap resize path - set ds->escHeap to true * If ds->end - ds->start (remaining bytes to parse in whole JSON) is later becomes larger than escBuffer again, using the *on the heap resize* path However, the number of remaining bytes to parse in a JSON payload will always decrease and escBuffer will never shrink so that last condition can never happen.
Buffer_EscapeStringUnvalidated() has no failure mode. Any code handling its return code is unreachable.
If given null as its buffer input, it'll create its own. But we never do.
…ools_scm] section (#746)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
17 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.