Enhance video processing and improve extrinsics detection fallback - #285
Merged
Conversation
-if zip in video field, pull out video, rewrite video to database, put zip as a result -if an extra camera is in recording that was not calibrated, ignore it
[wip] New video download
calcExtrinsics relied solely on cv2.findChessboardCorners with CALIB_CB_ADAPTIVE_THRESH, which can fail on otherwise-valid frames (certain contrast/exposure profiles), causing "checkerboard not detected by at least one camera" even when the board is clearly visible. Add the SB detector as a fallback that only runs when the primary detector fails. SBWithMeta corners are already subpixel-accurate, so cornerSubPix is skipped in that path. Uses ACCURACY|LARGER (no EXHAUSTIVE) to keep the fallback fast and avoid OpenCL-related crashes. No behavior change when the primary detector succeeds. Co-authored-by: Cursor <cursoragent@cursor.com>
Add findChessboardCornersSBWithMeta fallback for extrinsics detection
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces comprehensive support for handling LiDAR video data and associated depth information throughout the data processing pipeline. The main changes include robust downloading, extraction, compression, and uploading of LiDAR depth data and camera intrinsics, as well as improved API interactions for video and result uploads. Additionally, a new local trial processing script is added for sandbox testing.
LiDAR Data Handling and Depth Compression:
camera_matrix.csv, and compressing depth data for database upload using zstandard. This includes_compressDepthForDB,_compressLidarDepthFiles, and related helpers inutils.py. [1] [2] [3]Camera Intrinsics and Calibration:
camera_matrix.csv) when present, ensuring accurate calibration for LiDAR cameras. [1] [2] [3]API and S3 Upload Improvements:
uploadFileToS3function for both videos and results, ensuring consistent and robust S3 uploads and updating API references accordingly. [1] [2] [3]Result Management Enhancements:
New Local Trial Processing Script:
sandbox_app_local.py, a standalone script for locally polling and processing trials, supporting both single-trial and continuous modes with robust logging and error handling.These changes collectively enable seamless handling of LiDAR data in the pipeline, from download to upload, while improving maintainability and reliability.
Most important changes:
LiDAR Data Handling and Compression:
utils.py). [1] [2] [3]Camera Intrinsics and Calibration:
camera_matrix.csv) when available, ensuring correct calibration for LiDAR cameras (main.py). [1] [2] [3]API and S3 Upload Improvements:
utils.py). [1] [2] [3]Result Management:
utils.py).Local Trial Processing:
sandbox_app_local.pyfor local polling and processing of trials, with command-line options and robust error handling.