Skip to content

feat: add darkness filtering capability to dataloader#150

Open
maharajamihir wants to merge 3 commits into
mainfrom
feat/darkness-filter
Open

feat: add darkness filtering capability to dataloader#150
maharajamihir wants to merge 3 commits into
mainfrom
feat/darkness-filter

Conversation

@maharajamihir

Copy link
Copy Markdown
  • Introduced a DarknessFilter class to filter out sequences with images that are too dark based on a specified threshold.
  • Added a darkness_threshold parameter to Args and the get_dataloader function to support this feature.
  • DarknessFilter is optional. A threshold value greater than 0.0 activates it.

- Introduced a DarknessFilter class to filter out sequences with images that are too dark based on a specified threshold.
- Added a darkness_threshold parameter to Args and the get_dataloader function to support this feature.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds darkness filtering capability to the dataloader to filter out video sequences containing images that are too dark based on a configurable threshold.

  • Introduced a DarknessFilter class that computes average brightness of image sequences and filters them based on a threshold
  • Added darkness_threshold parameter to the Args class and get_dataloader function with a default value of 0.0
  • Integrated the darkness filter into the data processing pipeline between episode processing and batching

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
utils/dataloader.py Implements DarknessFilter class and adds darkness filtering to the dataloader pipeline
train_dynamics.py Adds darkness_threshold parameter to configuration and passes it to the dataloader

Comment thread utils/dataloader.py
darkness_threshold=darkness_threshold
),
grain.transforms.Batch(batch_size=per_process_batch_size, drop_remainder=True),
]

Copilot AI Aug 12, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DarknessFilter is always instantiated even when darkness_threshold is 0.0 (disabled). Consider conditionally adding the filter only when darkness_threshold > 0.0 to avoid unnecessary processing overhead.

Suggested change
]
]
if darkness_threshold > 0.0:
operations.append(
DarknessFilter(
darkness_threshold=darkness_threshold
)
)
operations.append(
grain.transforms.Batch(batch_size=per_process_batch_size, drop_remainder=True)
)

Copilot uses AI. Check for mistakes.
Comment thread utils/dataloader.py
element_greyscale = np.dot(element[...,:3], [0.2989, 0.5870, 0.1140])
average_brightness = np.mean(element_greyscale)
if average_brightness < self.darkness_threshold:
print(

Copilot AI Aug 12, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using print() statements for logging in production code is not recommended. Consider using a proper logging framework like Python's logging module to allow for better control over log levels and output destinations.

Suggested change
print(
logging.info(

Copilot uses AI. Check for mistakes.
Comment thread utils/dataloader.py Outdated
True if the sequence is not too dark, False otherwise.
"""
# Convert the RGB image to grayscale using numpy
element_greyscale = np.dot(element[...,:3], [0.2989, 0.5870, 0.1140])

Copilot AI Aug 12, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RGB to grayscale conversion coefficients [0.2989, 0.5870, 0.1140] are magic numbers. Consider defining these as named constants (e.g., RGB_TO_GRAYSCALE_WEIGHTS) to improve code readability and maintainability.

Suggested change
element_greyscale = np.dot(element[...,:3], [0.2989, 0.5870, 0.1140])
element_greyscale = np.dot(element[...,:3], RGB_TO_GRAYSCALE_WEIGHTS)

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, this is a magic number

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants