Wrist-Mounted Depth Camera

Ask
1

Hi, we recently installed the wrist-mounted depth camera from the upgrade kit onto our Stretch RE2, and observed that the depth image is extremely noisy (with roughly half the depth image being holes):

rgb_depth_fig
rgb_depth_fig1294×1458 222 KB

Any idea why this is so noisy, or what can be done to obtain a cleaner depth image?

Thanks!

2

Hi @arjung, there’s typically two circumstances that cause noisy depth images for the D405:

  1. The camera’s exposure time is too fast for the available light in the room. This can be solved by tweaking the camera’s exposure setting and/or providing more light in the room.
  2. The camera doesn’t see enough visual disparity to infer a clean depth image. This can be solved by adding more visual clutter to the scene seen by the camera (e.g. adding a poster to a blank wall. You want to avoid repeating patterns).

I’ve added two flags to our stretch_realsense_visualizer.py CLI, the --d405 flag to visualize the D405 wrist camera instead of the D435if head camera, and the --exposure flag, which sets the exposure for the camera. The exposure flag is “auto” by default, but I’ve noticed that the D405’s autoexposure behavior suffers in poor lighting. The valid range of input for the flag is [0, 165000], where 0 is a very fast exposure time, and 165000 is a very slow exposure time. Note that setting a slow exposure time does impact the frame rate you get from the camera.

For example, here’s the depth imagery I see in poorly lit room:

and if I run it with the exposure flag:

$ stretch_realsense_visualizer.py --d405 --exposure medium

lastly, here’s what it looks like with more overhead lighting

There’s a Github PR (D405 in Realsense visualizer CLI by hello-binit · Pull Request #310 · hello-robot/stretch_body · GitHub) for the changes that add the two flags I’ve mentioned above. Feel free to leave some feedback if you have any.

1 Like
3

Hi @bshah, thanks so much for such a prompt and detailed response! I will try these things and get back to you. Really appreciate your continuous support!

1 Like