Issue #18 enhancements: O(1) peek + next_if, Rust-staple methods, repeat(1) fix#19
Merged
Conversation
…eat(1) fix - peek() now uses a single-item lookahead buffer (like Rust's Peekable) instead of stacking an itertools.tee layer per call, making repeated peeks O(1). Adaptors flush the buffered item back onto the underlying iterator before consuming it, so nothing is lost. The flush helper is a module-level function because ty 0.0.37 mis-infers generics at some call sites when it is a method or property. - next_if(predicate): consume the next item only if it satisfies the predicate, else leave it peekable and return None (Peekable::next_if). - New methods: sum(), prod(), dedup(), zip_longest(other, fillvalue=...), sorted_by(key, reverse=...). - repeat(1) no longer returns self; it behaves like every other n, returning a new Itr and leaving the original exhausted. Closes #18. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The peek buffer now lives in a private _Peekable iterator wrapper (mirroring Rust, where Peekable is its own adapter type) instead of on Itr itself. Since the buffered item is yielded by _Peekable.__next__, adaptors consuming self._it automatically see the full sequence and no flush accessor is needed - removing the module-level _flushed() helper that existed only because ty mis-infers generics when the accessor is a method or property (still broken in ty 0.0.56, but now irrelevant). Dev deps are ranges again instead of the ty==0.0.37 pin: ty>=0.0.56, ruff>=0.15.20, pytest>=9.1.1. All gates pass on the new versions. Also ignore .claude/ in .gitignore. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wrapping every Itr's source in _Peekable unconditionally put a Python-level __next__ frame between every stage of a pipeline, costing 2.5-7.5x on bulk operations vs the raw C iterator path. The buffer is now installed only on the first peek()/next_if() call, restoring parity with the pre-_Peekable performance for pipelines that never peek. For iterators that have been peeked, the new unpeek() method removes the buffer again (prepending any pending item via itertools.chain, so nothing is lost), recovering full iteration speed when no further lookahead is needed. _Peekable also gains __slots__. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Supersedes both the eager _Peekable wrapper and the lazy-wrap/unpeek variant, following the benchmark discussion on the PR: the premise of issue #18 item 2 is false in CPython, because itertools.tee() re-tees an already-teed iterator via its cheap __copy__ rather than nesting. Repeated peeks on the tee-based implementation are already flat and ~constant time (measured linear at ~0.22us/peek over 160k peeks), so the buffer only bought ~2x on rare peek-heavy loops while adding machinery, a public unpeek() API, and (in the eager form) a 2.5-7.5x penalty on ordinary pipelines. peek() is copy().next() again, next_if is built on peek + next with identical semantics, and unpeek()/_Peekable/_MISSING are removed. All other #18 enhancements are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the remaining suggestions from #18 (item 1,
value_countsviaCounter, was already done in #17), plus anext_ifmethod.peek (#18 item 2): measured, and left tee-based
The suggested lookahead buffer was implemented two ways (an always-on
_Peekablewrapper, then a lazy variant withunpeek()) and benchmarked — and then removed, because the premise of the suggestion turns out to be false in CPython:itertools.tee()re-tees an already-teed iterator via its cheap__copy__rather than nesting, so repeatedpeek()calls are already flat and ~constant time (measured linear at ~0.22 µs/peek over 160k peeks). The buffer only bought ~2x on rare peek-heavy loops, while the always-on form cost 2.5–7.5x on ordinary pipelines (every item paid a Python-level__next__instead of staying on the C iterator fast path) and the lazy form kept the machinery plus a publicunpeek()API for marginal benefit. See the commit history for both variants and the revert rationale.next_if
next_if(predicate)consumes and returns the next item only if it satisfies the predicate; otherwise the item stays in place (a subsequentpeek/nextsees it) andNoneis returned (alsoNoneon exhaustion) — mirroring Rust'sPeekable::next_if. It is built onpeek+nextand needs no buffer.Rust-trait staples (#18 item 3)
sum()/prod()— terminal aggregations (0 / 1 on empty input)dedup()— lazy removal of adjacent duplicates; works on infinite iterators and unhashable itemszip_longest(other, fillvalue=...)— likezipbut pads the shorter inputsorted_by(key, reverse=...)— eager stable sort by keyrepeat(1) aliasing (#18 item 4)
Breaking:
repeat(1)now returns a newItrand leaves the original exhausted, consistent with every othern(previously it returnedself).Dev tooling
Dev dependencies are unpinned back to ranges:
ty>=0.0.56,ruff>=0.15.20,pytest>=9.1.1.All quality gates pass: ruff check/format, ty, pytest (181 tests, 100% coverage). README, apidoc and relnotes updated.
Closes #18.
🤖 Generated with Claude Code