Bug Description
In compute_in_hand.py and compute_to_hand.py, if some checkerboard images fail detection, the script may pair later image observations with the wrong robot poses from poses.txt.
This can produce an incorrect hand-eye calibration result even when most collected data is valid.
Root Cause
The script only appends image points when checkerboard detection succeeds:
if ret:
obj_points.append(objp)
img_points.append(corners2)But later it uses:
N = len(img_points)
for i in range(int(N)):
R_tool.append(tool_pose[0:3,4*i:4*i+3])
t_tool.append(tool_pose[0:3,4*i+3])This means it takes the first N robot poses, instead of the robot poses corresponding to the successfully detected image ids.
Example
Suppose a dataset has 17 images, but 11.jpg and 12.jpg fail checkerboard detection.
Detected images:
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 15, 16, 17
The current code pairs them like this:
1.jpg -> pose line 1
2.jpg -> pose line 2
...
10.jpg -> pose line 10
13.jpg -> pose line 11 # wrong
14.jpg -> pose line 12 # wrong
15.jpg -> pose line 13 # wrong
16.jpg -> pose line 14 # wrong
17.jpg -> pose line 15 # wrong
The correct behavior should be:
13.jpg -> pose line 13
14.jpg -> pose line 14
15.jpg -> pose line 15
16.jpg -> pose line 16
17.jpg -> pose line 17
Impact
This causes cv2.calibrateHandEye() to receive mismatched camera poses and robot end-effector poses, producing a wrong hand-eye transform.
In my test case, the incorrect result caused fixed-board position deviation of about 104.9 mm. After matching N.jpg with line N in poses.txt, the deviation was reduced to about 1.5 mm.
Suggested Fix
Track the numeric image id for every successfully detected checkerboard image, then use that id to select the corresponding robot pose from poses.txt.
For example:
valid_image_ids.append(i)
...
pose_index = image_id - 1
R_tool.append(tool_pose[0:3, 4*pose_index:4*pose_index+3])
t_tool.append(tool_pose[0:3, 4*pose_index+3])
I have prepared a patch that applies this fix to both:
compute_in_hand.py
compute_to_hand.py
Bug Description
In
compute_in_hand.pyandcompute_to_hand.py, if some checkerboard images fail detection, the script may pair later image observations with the wrong robot poses fromposes.txt.This can produce an incorrect hand-eye calibration result even when most collected data is valid.
Root Cause
The script only appends image points when checkerboard detection succeeds:
But later it uses:
This means it takes the first N robot poses, instead of the robot poses corresponding to the successfully detected image ids.
Example
Suppose a dataset has 17 images, but 11.jpg and 12.jpg fail checkerboard detection.
Detected images:
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 15, 16, 17
The current code pairs them like this:
1.jpg -> pose line 1
2.jpg -> pose line 2
...
10.jpg -> pose line 10
13.jpg -> pose line 11 # wrong
14.jpg -> pose line 12 # wrong
15.jpg -> pose line 13 # wrong
16.jpg -> pose line 14 # wrong
17.jpg -> pose line 15 # wrong
The correct behavior should be:
13.jpg -> pose line 13
14.jpg -> pose line 14
15.jpg -> pose line 15
16.jpg -> pose line 16
17.jpg -> pose line 17
Impact
This causes cv2.calibrateHandEye() to receive mismatched camera poses and robot end-effector poses, producing a wrong hand-eye transform.
In my test case, the incorrect result caused fixed-board position deviation of about 104.9 mm. After matching N.jpg with line N in poses.txt, the deviation was reduced to about 1.5 mm.
Suggested Fix
Track the numeric image id for every successfully detected checkerboard image, then use that id to select the corresponding robot pose from poses.txt.
For example:
I have prepared a patch that applies this fix to both:
compute_in_hand.pycompute_to_hand.py