Working with ArUco Markers

Knowledge Base
1

Hello!

I’ve been working with the detect_aruco_markers node from stretch_core in the stretch_ros repository and have been noticing some discrepancies between the detected position of a given set of Aruco tags and their actual position. See the attached image for an example of what this looks like.

aruco_misaligned2_cropped_anno
aruco_misaligned2_cropped_anno534×1047 227 KB

I created this image by simply using Rviz to display the MarkerArrays published by detect_aruco_markers to the /aruco/marker_array and /aruco/axes topics, as well as the PointCloud output by the depth camera. I added some annotations to hopefully make the issue a little more clear.

The tags circled in green are those where the generated marker array and the tag are aligned (it’s a little hard to tell but the detected axes are visible). All of those tags are stickers that came with the robot, either on its body or in the bag of extras.

The tags boxed in blue (came in the box of extras with the robot) or red (printed on paper) are those where there is poor alignment between the detected and actual position of the tag.

I’m having trouble identifying what may be causing the issue since there doesn’t seem to be a unifying characteristic between the tags that are affected and those that are not. Any insight you may have would be much appreciated. Thank you!

2

Hello @kpputhuveetil!

This is an excellent question. It’s great that you’re working with ArUco markers that aren’t on the robot’s body. I don’t think we’ve provided clear documentation for this, so I’m going to take this opportunity to both answer your question and provide additional context.

The ArUco Marker Dictionary

The most likely cause of your issue is stretch_marker_dict.yaml, which uses YAML to define a dictionary that holds information about ArUco markers.

If detect_aruco_markers doesn’t find an entry in stretch_marker_dict.yaml for a particular ArUco marker ID number, it uses the default entry. For example, most robots have shipped with the following default entry

  'default':
    'length_mm': 24
    'use_rgb_only': False
    'name': 'unknown'
    'link': None

and the following entry for the ArUco marker on the top of the wrist

  '133':
    'length_mm': 23.5
    'use_rgb_only': False
    'name': 'wrist_top'
    'link': 'link_aruco_top_wrist'

.

It’s good practice to add an entry to stretch_marker_dict.yaml for each ArUco marker you use.

  • The dictionary key for each entry is the ArUco marker’s ID number or default. For example, the entry shown above for the ArUco marker on the top of the wrist assumes that the marker’s ID number is 133.
  • A mismatch between length_mm and your ArUco markers is probably the source of your pose estimation errors. The next section provides more details.
  • If use_rgb_only is True, detect_aruco_markers will ignore depth images from the Intel RealSense D435i depth camera when estimating the pose of the marker and will instead only use RGB images from the D435i.
  • name is used for the text string of the ArUco marker’s ROS Marker in the ROS MarkerArray Message published by the detect_aruco_markers ROS node.
  • link is currently used by stretch_calibration. It is the name of the link associated with a body-mounted ArUco marker in the robot’s URDF.

Carefully Measure Custom ArUco Markers

The length_mm value used by detect_aruco_markers is important for estimating the pose of an ArUco marker. If the actual width and height of the marker do not match this value, then pose estimation will be poor.

In practice, you should measure any marker by hand. For example, at Hello Robot we’ve printed out 180mm markers in the past only to later learn that they’re actually 179mm markers. Similar discrepancies with early wrist markers resulted in poor calibration.

1280×720 1.21 MB

1280×720 1.33 MB

Official Guide to Creating New ArUco Markers

At Hello Robot, we’ve used the following guide when generating new ArUco markers.

We generate ArUco markers using a 6x6 bit grid (36 bits) with 250 unique codes. This corresponds with DICT_6X6_250 defined in OpenCV. We generate markers using this online ArUco marker generator by setting the Dictionary entry to 6x6 and then setting the Marker ID and Marker size, mm as appropriate for the specific application. As described above, we measure the actual marker we use by hand prior to adding an entry for it to stretch_marker_dict.yaml.

We select marker ID numbers using the following ranges.

  • 0 - 99 : reserved for users
  • 100 - 249 : reserved for official use by Hello Robot Inc.
    • 100 - 199 : reserved for robots with distinct sets of body-mounted markers
      • Allows different robots near each other to use distinct sets of body-mounted markers to avoid confusion. This could be valuable for various uses of body-mounted markers, including calibration, visual servoing, visual motion capture, and multi-robot tasks.
      • 5 markers per robot = 2 on the mobile base + 2 on the wrist + 1 on the shoulder
      • 20 distinct sets = 100 available ID numbers / 5 ID numbers per robot
    • 200 - 249 : reserved for official accessories
      • 245 for the prototype docking station
      • 246-249 for large floor markers

When coming up with this guide, we expected the following:

  • Body-mounted accessories with the same ID numbers mounted to different robots could be disambiguated using the expected range of 3D locations of the ArUco markers on the calibrated body.
  • Accessories in the environment with the same ID numbers could be disambiguated using a map or nearby observable features of the environment.

I hope you find this helpful!

Best wishes,
Charlie

Charlie Kemp, PhD
co-founder & CTO
Hello Robot Inc.
http://charliekemp.com

1 Like
3

Thank you so much Dr. Kemp! This is extremely helpful!

You were right - stretch_marker_dict.yaml was indeed the cause of the problem. When I asked the question, I actually had been editing the yaml file to include the additional tags. However, I guess I wasn’t careful enough in ensuring that the tabs/spaces in the file were correct and as a result, some of the new entries were ignored and the default settings were applied. Fixing the formatting in the file has resolved the problem.

Thank you again for your detailed reply. I’ll definitely be keeping this additional context in mind for further work with the ArUco markers.

1 Like