@@ -544,20 +544,45 @@ unique_ints: stream[int] = (
544544assert list (unique_ints) == [0 , 1 ]
545545```
546546
547- ## vs ` builtins.map/filter `
547+ ## performance
548548
549549There is zero overhead during iteration compared to ` builtins.map ` and ` builtins.filter ` :
550550
551551``` python
552552odd_int_chars = stream(range (N)).filter(lambda n : n % 2 ).map(str )
553553```
554554
555- ` iter(odd_int_chars) ` visits the operations lineage and returns exactly this iterator :
555+ ` iter(odd_int_chars) ` visits the operation lineage and returns exactly:
556556
557557``` python
558558map (str , filter (lambda n : n % 2 , range (N)))
559559```
560560
561+ Throughput of each operation compared to ` builtins.map ` :
562+
563+ | operation| times slower than ` builtins.map(lambda _: _, range(N)) ` |
564+ | --| --|
565+ | ` stream(range(N)).map(lambda _: _) ` | 1.00x|
566+ | ` stream(range(N)).filter(lambda _: _) ` | 1.05x|
567+ | ` stream(range(N)).skip(N) ` | 1.9x|
568+ | ` stream(range(N)).do(lambda _: _) ` | 2.0x|
569+ | ` stream(range(N)).catch(ValueError) ` | 2.0x|
570+ | ` stream(range(N)).group(5) ` | 2.4x|
571+ | ` stream(range(N)).take(N) ` | 3.0x|
572+ | ` stream(range(N)).observe('ints', do=bool) ` | 4.2x|
573+ | ` stream(range(N)).observe('ints', do=bool, every=N) ` | 4.4x|
574+ | ` stream(range(N)).observe('ints', do=bool, every=timedelta(...)) ` | 5.5x|
575+ | ` stream(range(N)).throttle(N, per=timedelta(...)) ` | 5.8x|
576+ | ` stream(range(N)).group(by=bool, up_to=5) ` | 7.3x|
577+ | ` stream((i,) for i in range(N)).flatten() ` | 17x|
578+ | ` stream(range(N)).buffer(N) ` | 50x|
579+ | ` stream(range(N)).map(lambda _: _, concurrency=2) ` | 330x|
580+ | ` stream(range(N)).map(lambda _: _, concurrency=2, as_completed=True) ` | 340x|
581+ | ` stream(range(N)).group(within=timedelta(...)) ` | 350x|
582+ | ` stream((i,) for i in range(N)).flatten(concurrency=2) ` | 850x|
583+
584+ (source: ` pytest -s tests/benchmark.py ` )
585+
561586## e.g. ETL via [ ` dlt ` ] ( https://github.com/dlt-hub/dlt )
562587
563588A ` stream ` is an expressive way to declare a ` dlt.resource ` :
0 commit comments