# Stretch Calibration: detect\_aruco\_markers error

**URL:** <https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826>\
**Category:** Ask\
**Created:** [October 11, 2023, 2:23am UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826 "2023-10-11T02:23:00Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![arjung](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/arjung/32/410_2.png) [@arjung](https://forum.hello-robot.com/u/arjung)\
**Post date:** [October 11, 2023, 2:23am UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/1 "2023-10-11T02:23:00Z")

</div>

Hi!

I’ve been following the stretch calibration instructions [here](https://github.com/hello-robot/stretch_ros/blob/master/stretch_calibration/README.md) and on the third step (when collecting new observations) I get the following error (and warning, which may or may not be relevant):

```auto
[WARN] [1696990073.768834877]: Hardware Notification:Depth stream start failure,1.69699e+12,Error,Hardware Error
[ERROR] [1696990073.857499]: bad callback: <bound method Subscriber.callback of <message_filters.Subscriber object at 0x7f98419b0430>>
Traceback (most recent call last):
  File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
    cb(msg)
  File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/ __init__.py", line 76, in callback
    self.signalMessage(msg)
  File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/ __init__.py", line 58, in signalMessage
    cb(*(msg + args))
  File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/ __init__.py", line 240, in add
    self.signalMessage(*msgs)
  File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/ __init__.py", line 58, in signalMessage
    cb(*(msg + args))
  File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/detect_aruco_markers", line 624, in image_callback
    self.aruco_marker_collection.update(self.rgb_image, self.camera_info, self.depth_image, self.rgb_image_timestamp)
  File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/detect_aruco_markers", line 551, in update
    self.aruco_corners, self.aruco_ids, aruco_rejected_image_points = aruco.detectMarkers(self.gray_image,
AttributeError: module 'cv2.aruco' has no attribute 'detectMarkers'

```

In attempt to fix this, I tried replacing my existing ROS1 Noetic workspace (using the instructions [here](https://github.com/hello-robot/stretch_install/blob/master/docs/ros_workspace.md)), but this did not seem to help.

Any ideas for what might be going on? Any help will be much appreciated, thanks in advance!

---

<div class="post-metadata">

**Author:** ![bshah](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/bshah/32/55_2.png) [@bshah](https://forum.hello-robot.com/u/bshah)\
**Post date:** [October 12, 2023, 4:29am UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/2 "2023-10-12T04:29:30Z")

</div>

Hey @arjung, Stretch Calibration uses OpenCV for detection of the Aruco markers on Stretch’s body, and with the latest version of OpenCV, the Aruco detection API has changed. Sylvia Wang (at University of Washington) wrote a fix that adopts the new API. We just [merged it here](https://github.com/hello-robot/stretch_ros/pull/100).

Would you try replacing your Noetic workspace again? The new changes should resolve the error you’re seeing with Stretch Calibration.

---

<div class="post-metadata">

**Author:** ![arjung](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/arjung/32/410_2.png) [@arjung](https://forum.hello-robot.com/u/arjung)\
**Post date:** [October 12, 2023, 8:10pm UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/3 "2023-10-12T20:10:09Z")

</div>

Hey @bshah, thanks so much for your response! This fixed the error!

I was able to run the stretch calibration commands and I’m getting a calibration error of 0.056, which exceeds the expected threshold (\< 0.05) – do you have any ideas for how this calibration error can be reduced?

Furthermore, what initially prompted the need for calibration was the following observation. When the camera is commanded to “look ahead” (as described [here](https://docs.hello-robot.com/0.2/stretch-tutorials/stretch_body/tutorial_stretch_body_api/#using-the-head-class)) and the robot base is aligned with the tiles (which are aligned with the cabinets), I get the following image:

 ![Screenshot 2023-10-12 at 2.34.46 PM](https://canada1.discourse-cdn.com/flex030/uploads/hello_robot2/original/1X/9e4803880e3a8c5748cb48c66f446099aa57c9ed.jpeg)

As evident, the line connecting the ground to the cabinets _does not_ appear to be horizontal, suggesting that there is a camera calibration issue.

Now, when I rotate the yaw of the camera 8 degrees to the left, I get the following image:

 ![look_ahead_rotated](https://canada1.discourse-cdn.com/flex030/uploads/hello_robot2/original/1X/63ddc2baf31d8db2da40fef4c5df4db02bd39d20.jpeg)

Here, while the line connecting the ground to the cabinets _does_ appear to be horizontal, the ceiling is not very horizontal. It seems like there may be a non-zero amount of rotation in one of the other axes, even though only the yaw was changed after the camera was put into “look ahead” mode.

I’m hoping that reducing the calibration error and using the updated urdf may help resolve this – what do you think?

Thanks once again for all your help!

---

<div class="post-metadata">

**Author:** ![bshah](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/bshah/32/55_2.png) [@bshah](https://forum.hello-robot.com/u/bshah)\
**Post date:** [October 12, 2023, 8:49pm UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/4 "2023-10-12T20:49:56Z")

</div>

Hi @arjung,

I suspect calibration would not help here. Calibration solves the problem of matching where the robot “thinks” its body is to where it sees its body, but it doesn’t calibrate with respect to the world (e.g. leveling its view with the world).

I think there are two explanations for the skew you are seeing in those images.

1. Likely, those images are the distorted images that the Realsense libraries return by default. You can [undistort](https://docs.opencv.org/4.5.2/d9/d0c/group__calib3d.html#ga69f2545a8b62a6b0fc2ee060dc30559d) the RGB image using [OpenCV](https://docs.opencv.org/4.5.2/dc/dbb/tutorial_py_calibration.html) and the camera model reported in the /camera/color/camera\_info topic.
2. The camera in the head sees the world with a slight “roll”. This could happen if Stretch’s mast were tilted slightly with respect to its mobile base, if the camera were tilted slightly with respect to the pan/tilt assembly, if the floor of the environment wasn’t completely flat (the mobile base tilted with respect to the world), or some combination of those. Estimating the tilt contributed by each of these sources is tricky, so if my use-case required a level view of the world, I would correct for the roll by rotating the image. You could do this via guess & check, or use the IMU in the camera. Gravity will point downwards on the accelerometer, allowing you to estimate the amount of roll the camera experiences.
  - ![Screenshot from 2023-10-12 16-33-55](https://canada1.discourse-cdn.com/flex030/uploads/hello_robot2/original/1X/b37b901d7d91a552a29c77181207a971a6478921.png)

---

<div class="post-metadata">

**Author:** ![arjung](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/arjung/32/410_2.png) [@arjung](https://forum.hello-robot.com/u/arjung)\
**Post date:** [October 12, 2023, 9:24pm UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/5 "2023-10-12T21:24:57Z")

</div>

Hey @bshah, thanks very much for the pointers! I will try out some of these ideas.

It still seems to me that I should resolve the calibration issue, whether or not that is the source of the skew in my images. Do you have any suggestions for how I can go about doing this?

Thanks for your help!

---

<div class="post-metadata">

**Author:** ![bshah](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/bshah/32/55_2.png) [@bshah](https://forum.hello-robot.com/u/bshah)\
**Post date:** [October 14, 2023, 2:28am UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/6 "2023-10-14T02:28:55Z")

</div>

Hey @arjung, at the moment, the main suggestion I can give you is to try the calibration procedure in a controlled environment. If you have multiple Stretchs, make sure your robot doesn’t accidentally detect the Aruco markers off another robot. Try to ensure the room is evenly lit, and doesn’t have harsh overhead lighting. If your calibration error remains high in the controlled environment, we can schedule a support call to determine if there’s another cause of error. In the future, we’d like to provide better debugging tools to help Stretch self-diagnose issues during the calibration procedure.

---

<div class="post-metadata">

**Author:** ![arjung](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/arjung/32/410_2.png) [@arjung](https://forum.hello-robot.com/u/arjung)\
**Post date:** [October 17, 2023, 7:49pm UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/7 "2023-10-17T19:49:52Z")

</div>

Hey @bshah, thanks so much for your help! Unfortunately, I’m still seeing a calibration error \> 0.05, even when we re-do the calibration in different rooms / lighting conditions. Would it be possible to schedule a support call to determine if there’s another cause of error sometime this week?

---

<div class="post-metadata">

**Author:** ![bshah](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/bshah/32/55_2.png) [@bshah](https://forum.hello-robot.com/u/bshah)\
**Post date:** [October 18, 2023, 4:22am UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/8 "2023-10-18T04:22:31Z")

</div>

Yes, that sounds like the best way to get to the bottom of it. Could you reach out to [support@hello-robot.com](mailto:support@hello-robot.com), and we can schedule a meeting.

---

<div class="post-metadata">

**Author:** ![arjung](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/arjung/32/410_2.png) [@arjung](https://forum.hello-robot.com/u/arjung)\
**Post date:** [October 20, 2023, 5:25am UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/9 "2023-10-20T05:25:24Z")

</div>

Hey @bshah, I reached out to [support@hello-robot.com](mailto:support@hello-robot.com) right after your response and have yet to hear back. Would it be possible for you to check if my email was received? Sorry to bother you once again about this.

---

<div class="post-metadata">

**Author:** ![arjung](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/arjung/32/410_2.png) [@arjung](https://forum.hello-robot.com/u/arjung)\
**Post date:** [October 27, 2023, 3:32am UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/10 "2023-10-27T03:32:10Z")

</div>

Hi @bshah, the calibration error is finally \< 0.03 (0.021)! It turned out that the rooms I had tested calibration in were indeed too brightly lit with overhead lights. Re-doing the calibration procedure in a darker room with no overhead lights resolved the issue! Thanks once again for all your help!

---

<div class="post-metadata">

**Author:** ![bshah](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/bshah/32/55_2.png) [@bshah](https://forum.hello-robot.com/u/bshah)\
**Post date:** [October 27, 2023, 5:02pm UTC](https://forum.hello-robot.com/t/stretch-calibration-detect-aruco-markers-error/826/11 "2023-10-27T17:02:30Z")

</div>

Really glad to hear that @arjung! Thanks for letting me know.

Also for anyone else who is reading this thread in the future, another issue we discovered over video call was running the “[Checking the current calibration](https://github.com/hello-robot/stretch_ros/tree/noetic/stretch_calibration#checking-the-current-calibration-with-new-observations)” instructions, instead of starting a new calibration. Which explains why the final error was always around 0.05, even in different environments. So make sure you’re running the instructions for “[Creating a new calibration](https://github.com/hello-robot/stretch_ros/tree/noetic/stretch_calibration#calibrate-the-stretch-re1)” instead of checking how good your existing one is.