Params File - #17
Open
thomasrgray wants to merge 2177 commits into
Open
Conversation
Owner
|
@abuzarmahmood @thomasrgray Are you guys planning to merge this to master and is there something you need me to do? Right now, your branch has conflicts with the master here - maybe try merging the master from here onto your branch and resolving any conflicts? |
…://github.com/katzlabbrandeis/blech_clust into add-avg-intra-car-similarity-warning-653
…p column to string type - Convert CAR_group column to string type after loading to handle float values (NaN) - Also convert when overwriting with original_CAR_group to maintain consistency - Fixes TypeError: bad operand type for unary ~: 'float' at line 491
for more information, see https://pre-commit.ci
Replace confusing double-indexing pattern (emg_bool[none_bool]) with direct indexing using electrode_ind from the filtered electrode_layout_frame. This ties the correlation matrix indexing directly to the electrode layout frame, avoiding indexing issues with emg and none CAR groups. Addresses comment on PR #669 about adequate indexing of corr-mat. Co-authored-by: Ona <no-reply@ona.com>
- Add print statement showing which attributes are created when stacking is done - Add print statement showing which attributes are created when stacking is not done - Addresses issue #725 by providing clearer information about available attributes - Includes test to verify correct attribute names are displayed Co-authored-by: openhands <openhands@all-hands.dev>
- Include array shapes in the generated attributes print statements - For stacking case: shows shapes of all 6 generated attributes - For non-stacking case: shows shapes of firing_list arrays and time_vector - Updated test to verify shape information is included - Provides users with complete information about data structure dimensions Co-authored-by: openhands <openhands@all-hands.dev>
- Improved readability by putting each generated attribute on its own line - Added proper indentation for visual hierarchy - Updated test to verify multi-line format with indentation - Enhanced user experience with clearer, more organized output Co-authored-by: openhands <openhands@all-hands.dev>
for more information, see https://pre-commit.ci
…rates-print-statements Update print statements in get_firing_rates to show generated attributes
…corr-mat-indexing
- Check if 'original_CAR_group' exists and use it if present. - Display message indicating the usage of original CAR groups. - Ensure 'CAR_group' values are converted to lowercase and whitespace is trimmed.
…_averages - Calculate `num_groups` dynamically using the `CAR_group` column in `electrode_layout_frame`. - Simplify logic for assigning predicted cluster numbers to `CAR_group` names. - Add an assertion to ensure the number of predictions matches the number of electrodes. - Refine logic for warning on average intra-CAR similarity by directly accessing the threshold parameter. - Use more precise criteria for triggering a similarity warning.
for more information, see https://pre-commit.ci
Centralizes test data configuration in test_config.json: - Test data directory location - Dataset names and Google Drive IDs Updates download_test_data.sh, prefect_pipeline.py, and s3_utils.py to read from the config file instead of hardcoded values. Fixes #624 Co-authored-by: Ona <no-reply@ona.com>
for more information, see https://pre-commit.ci
- Relocate argument parsing into conditional blocks for isolated testing. - Implement xarray to save the correlation matrix, retaining full metadata. - Ensure accurate execution log generation for better traceability. - Clean up redundant and misplaced code elements to improve readability.
…bbrandeis/blech_clust into fix-car-corr-mat-indexing
for more information, see https://pre-commit.ci
- Added `xarray` import to enable advanced manipulation of data arrays. - Transitioned correlation matrix format from `.npy` to `.nc` using `xarray` for improved indexing and selection capabilities. - Enhanced code clarity by using `xarray` for direct channel selection. - Ensured datatype consistency by converting CAR_group columns to strings for uniformity. - Improved error handling by raising runtime errors during intra-CAR similarity checks for better debugging information. Context: Addressed issues in corr_mat handling with `xarray` indexing.
…bbrandeis/blech_clust into fix-car-corr-mat-indexing
for more information, see https://pre-commit.ci
…e NaN values - Convert 'CAR_group' and 'original_CAR_group' columns to string type. - Fill NaN values in 'original_CAR_group' with 'none' to ensure consistent data representation. - Address issue where 'none' was incorrectly saved as a string representation. - Remove unnecessary empty line for cleaner code formatting.
…bbrandeis/blech_clust into fix-car-corr-mat-indexing
for more information, see https://pre-commit.ci
Move pipeline check initialization and logging to the correct section within the non-testing conditional block. This ensures that pipeline checks are correctly initialized in production environments.
Fix correlation matrix indexing to use electrode_ind directly
Calculate channel correlations after common average reference and generate comparison plots in QA_output: - Heatmaps showing correlation matrices before/after CAR with difference - Paired points plot showing correlation changes per channel pair - Histogram of correlation changes with summary statistics Closes #727 Co-authored-by: Ona <no-reply@ona.com>
for more information, see https://pre-commit.ci
- Changed the comment to clarify that noise waveforms are discarded from GMM fitting instead of training.
…abbrandeis/blech_clust into 644b-allow-dig-ins-MAR_abu
- Updated the format for electrode arrays to use formatted strings for clarity. - Added print statements to log the completion of LFP extraction for each electrode and digital input for better tracking of the process. - Removed unused variable `dig_in_channels` in two functions to clean up the code.
for more information, see https://pre-commit.ci
- Added support for recursive node removal in `remove_node` function. - Updated STFT processing to handle multiple datasets, specifically for different tastes. - Refined the extraction of LFP data and improved printing for debug clarity. - Modularized STFT calculations for amplitude and phase across multiple datasets. - Introduced error handling and assertion for STFT computation consistency. - Improved data writing logic to the HDF5 file by using dig-in names for organization.
…abbrandeis/blech_clust into 644b-allow-dig-ins-MAR_abu
for more information, see https://pre-commit.ci
- Introduced `method_kwargs` to define test parameters for specific methods. - Extended `test_ephys_data` to run methods with specified kwargs where applicable. - Added checks and iteration over parameters to ensure comprehensive method testing.
- Introduced `load_stft_data` function to encapsulate the logic for loading STFT data from HDF5. - Reduced redundancy by calling the new function instead of duplicating code. - Added checks and warnings for missing STFT data nodes within the HDF5 file. - Ensured recalculation flags are set appropriately if data nodes are missing.
- Introduced a new section in README.md for community and support, directing users to the Discourse forum and GitHub issues for different needs. - Provided guidance on where users should address questions and feature requests. style(ephys): improve code readability with better formatting - Reformatted print statements and list comprehensions for better readability in `ephys_data.py`. - Ensured consistent indentation and line breaks to enhance code clarity.
…abbrandeis/blech_clust into 644b-allow-dig-ins-MAR_abu
for more information, see https://pre-commit.ci
Co-authored-by: aider (claude-sonnet-4-5) <aider@aider.chat>
Co-authored-by: aider (claude-sonnet-4-5) <aider@aider.chat>
Co-authored-by: aider (claude-sonnet-4-5) <aider@aider.chat>
…abbrandeis/blech_clust into 644b-allow-dig-ins-MAR_abu
for more information, see https://pre-commit.ci
Co-authored-by: aider (claude-sonnet-4-5) <aider@aider.chat>
…abbrandeis/blech_clust into 644b-allow-dig-ins-MAR_abu
644b allow dig ins MAR abu
This update addresses issue #792 by documenting that 'none' and 'emg' are privileged/hardcoded CAR group names: - 'none': Channels marked with CAR group 'none' are excluded from common average reference (CAR) processing and will not be analyzed - 'emg' (or any CAR group containing 'emg'): Channels with CAR groups containing 'emg' are treated as EMG channels and processed separately for EMG analysis Changes: - docs/tutorials.md: Added 'Reserved CAR Group Names' section - docs/reference/core-pipeline.md: Added CAR groups section to blech_common_avg_reference.py documentation - blech_exp_info.py: Improved prompt message to clarify reserved names Co-authored-by: openhands <openhands@all-hands.dev>
- Added new documentation file: docs/resources/intan_headstage_pinouts.md - This document provides reference information for Intan headstage electrode connector pinouts, which is useful when creating the electrode_layout_frame in blech_exp_info.py - Added links to the new documentation from docs/index.md and docs/tutorials.md - References the official Intan Technologies documentation at intantech.com Fixes #793
…r-group-names docs: clarify reserved CAR group names 'none' and 'emg' (fixes #792)
…-images Include Intan headstage pinout images in documentation
…stage-pinouts-docs Add Intan headstage pinouts documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Params file can be more descriptively named