Robot Crashes when Attempting to Stream from Two Cameras Simultaneously

Ask
1

Hello!

We are currently trying to simultaneously stream from both the robot’s head camera and an external Realsense camera (D435i) that is plugged into the robot, and are running into some problems.

To do this, we first roslaunch the external camera via roslaunch realsense2_camera rs_camera.launch camera:=camera_2 serial_no:=<serial number>, at a resolution of 1280x720 and at 6FPS. When we then attempt to roslaunch the head camera using d435i_high_resolution.launch, the system becomes unresponsive and we have to restart the robot.

Each camera can be streamed individually, its when we try to stream from both that we run into this problem. We have tried simultaneous streaming from the same cameras using the same parameters on a different Stretch robot in the lab without issue. Any guidance you may have for resolving this issue would be greatly appreciated!

Thanks!

2

Hi @kpputhuveetil, I’ve got a few suggestions for debugging what’s happening here. First off, whereas the head Realsense camera is internally connected to a USB3 port, the external Realsense camera has only USB2 port options on Stretch RE1 (there are USB3 ports in the trunk if using Stretch 2). This can saturate the USB bus and cause the computer to become unresponsive. Try reducing the resolution and framerate of both camera when roslaunching to see whether the system still freezes. Monitor the CPU usage and temperature to determine if the CPU cannot keep up with the cameras’ streams. Additionally, quality and length of the USB cable connecting the external Realsense to Stretch can impact the performance of the system. I recommend trying a few well-known high quality cables (e.g. Realsense cameras typically ship with a high quality blue USB C cable).

In the past, we have seen Stretch’s onboard computer freeze a few seconds after starting the ROS Realsense drivers. This occurred on systems running Ubuntu 18.04 and a version of librealsense2 that uses the RSUSB backend. We wrote the installation scripts for Ubuntu 20.04 to ensure that the more stable version of librealsense2 was used. Upgrading your robot to 20.04 by following this upgrade guide may help here.

3

Hello Binit,

Thank you for your earlier response and troubleshooting tips! Due to our fixed framerate and resolution requirements for the secondary camera, we ended up streaming the external camera from a computer and then sending the relevant information to Stretch at runtime, which worked well. However, we have started seeing that the robot’s computer will occasionally become unresponsive when launching even just its internal camera by itself (for example, when running the stretch_ros demos). The robot will freeze for extended periods and eventually requires a full restart. The robot is dual-booted with both Ubuntu 18.04 and Ubuntu 20.04 and we’ve noticed this issue on both operating systems. I appreciate any guidance you may have for troubleshooting this!

Thanks!

4

Hi @kpputhuveetil, thanks for letting me know. I’d like to check what version of librealsense2 is being used. Would you login into the Ubuntu 20.04 partition, and run the following command, and report back the output.

sudo apt list | grep realsense

Additionally, capture the robot ID using:

echo $HELLO_FLEET_ID
5

Hey Binit!

Here is the output of sudo apt list | grep realsense from both partitions (I personally use the Ubuntu 18.04 partition, the other is used by others in our group but I am willing to switch if necessary):

Ubuntu 18.04

Summary

kpputhuveetil@stretch-re1-1082:~$ sudo apt list | grep realsense
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

librealsense2/bionic,now 2.53.1-0~realsense0.8249 amd64 [installed,automatic]
librealsense2-dbg/bionic,now 2.53.1-0~realsense0.8249 amd64 [installed]
librealsense2-dev/bionic,now 2.53.1-0~realsense0.8249 amd64 [installed]
librealsense2-dkms/bionic,now 1.3.18-0ubuntu1 all [installed]
librealsense2-gl/bionic,now 2.53.1-0~realsense0.8249 amd64 [installed,automatic]
librealsense2-gl-dbg/bionic 2.53.1-0~realsense0.8249 amd64
librealsense2-gl-dev/bionic 2.53.1-0~realsense0.8249 amd64
librealsense2-net/bionic,now 2.53.1-0~realsense0.8249 amd64 [installed,automatic]
librealsense2-net-dbg/bionic 2.53.1-0~realsense0.8249 amd64
librealsense2-net-dev/bionic 2.53.1-0~realsense0.8249 amd64
librealsense2-udev-rules/bionic,now 2.53.1-0~realsense0.8249 amd64 [installed,automatic]
librealsense2-utils/bionic,now 2.53.1-0~realsense0.8249 amd64 [installed]
ros-melodic-librealsense2/bionic,now 2.50.0-1bionic.20211115.141014 amd64 [installed,automatic]
ros-melodic-librealsense2-dbgsym/bionic 2.50.0-1bionic.20211115.141014 amd64
ros-melodic-realsense2-camera/bionic,now 2.3.2-1bionic.20221025.193216 amd64 [installed]
ros-melodic-realsense2-camera-dbgsym/bionic 2.3.2-1bionic.20221025.193216 amd64
ros-melodic-realsense2-description/bionic,now 2.3.2-1bionic.20221025.185246 amd64 [installed]

Ubuntu 20.04

Summary

stretch-1082@stretch1082-NUC8i5BEH:~$ sudo apt list | grep realsense
[sudo] password for stretch-1082:

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

librealsense2-dbg/focal,now 2.50.0-0~realsense0.6128 amd64 [installed]
librealsense2-dev/focal,now 2.50.0-0~realsense0.6128 amd64 [installed]
librealsense2-dkms/focal,now 1.3.18-0ubuntu1 all [installed]
librealsense2-gl-dbg/focal 2.50.0-0~realsense0.6128 amd64
librealsense2-gl-dev/focal 2.50.0-0~realsense0.6128 amd64
librealsense2-gl/focal,now 2.50.0-0~realsense0.6128 amd64 [installed,automatic]
librealsense2-net-dbg/focal 2.50.0-0~realsense0.6128 amd64
librealsense2-net-dev/focal 2.50.0-0~realsense0.6128 amd64
librealsense2-net/focal,now 2.50.0-0~realsense0.6128 amd64 [installed,automatic]
librealsense2-udev-rules/focal,now 2.50.0-0~realsense0.6128 amd64 [installed,automatic]
librealsense2-utils/focal,now 2.50.0-0~realsense0.6128 amd64 [installed]
librealsense2/focal,now 2.50.0-0~realsense0.6128 amd64 [installed,automatic]
ros-foxy-librealsense2-dbgsym/focal 2.50.0-2focal.20220223.160142 amd64
ros-foxy-librealsense2/focal 2.50.0-2focal.20220223.160142 amd64
ros-foxy-realsense-msgs-dbgsym/focal 2.0.8-2focal.20220209.142543 amd64
ros-foxy-realsense-msgs/focal 2.0.8-2focal.20220209.142543 amd64
ros-foxy-realsense2-camera-dbgsym/focal 4.0.4-1focal.20220323.000248 amd64
ros-foxy-realsense2-camera-msgs-dbgsym/focal 4.0.4-1focal.20220322.234811 amd64
ros-foxy-realsense2-camera-msgs/focal 4.0.4-1focal.20220322.234811 amd64
ros-foxy-realsense2-camera/focal 4.0.4-1focal.20220323.000248 amd64
ros-foxy-realsense2-description/focal 4.0.4-1focal.20220323.000251 amd64
ros-galactic-librealsense2-dbgsym/focal 2.50.0-2focal.20220222.210235 amd64
ros-galactic-librealsense2/focal,now 2.50.0-2focal.20220222.210235 amd64 [installed,automatic]
ros-galactic-realsense2-camera-dbgsym/focal 4.0.4-1focal.20220430.140243 amd64
ros-galactic-realsense2-camera-msgs-dbgsym/focal 4.0.4-1focal.20220430.123523 amd64
ros-galactic-realsense2-camera-msgs/focal,now 4.0.4-1focal.20220430.123523 amd64 [installed,automatic]
ros-galactic-realsense2-camera/focal,now 4.0.4-1focal.20220430.140243 amd64 [installed]
ros-galactic-realsense2-description/focal,now 4.0.4-1focal.20220430.134808 amd64 [installed]
ros-noetic-librealsense2-dbgsym/focal 2.50.0-1focal.20211115.134455 amd64
ros-noetic-librealsense2/focal,now 2.50.0-1focal.20211115.134455 amd64 [installed,automatic]
ros-noetic-realsense2-camera-dbgsym/focal 2.3.2-1focal.20220519.100731 amd64
ros-noetic-realsense2-camera/focal,now 2.3.2-1focal.20220519.100731 amd64 [installed]
ros-noetic-realsense2-description/focal,now 2.3.2-1focal.20220212.161322 amd64 [installed]

The robot ID is stretch-re1-1082

Thanks!

6

Great, thanks! The unresponsiveness is related to the ros-*-librealsense2 packages. On Ubuntu 18.04, ros-melodic-librealsense2 should be uninstalled, and on Ubuntu 20.04, ros-noetic-librealsense2 and ros-galactic-librealsense2 should be uninstalled. Additionally, there’s a few other steps to setup the Realsense drivers.

We have install scripts that correctly configure the Realsense drivers on a new robot install, so the easiest solution would be to follow the upgrade guide to perform a fresh 18.04/20.04 robot install. Or I can provide steps for you to correct the Realsense setup on your existing Ubuntu 18.04/20.04.