Skip to content

Commit 35f936e

Browse files
Boris PeyriguereBoris Peyriguere
authored andcommitted
Bound streamed metadata scans to configured split size
1 parent 23eefc0 commit 35f936e

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

  • complexity/generative/detection

complexity/generative/detection/data.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -252,7 +252,7 @@ def _stream(self, *, metadata_only: bool = False):
252252
if metadata_only:
253253
stream = stream.cast_column(self.image_column, HFImage(decode=False))
254254
stream = stream.remove_columns(self.image_column)
255-
return stream
255+
return stream.take(self.total_examples)
256256
stream = stream.shuffle(
257257
seed=self.seed + self.epoch,
258258
buffer_size=self.shuffle_buffer,

0 commit comments

Comments
 (0)