Telepresence and web_interface using Stretch RE2

Ask
1

Hello community,

I am currently trying to use the Teleop kit and the stretch_fisheye_web_interface.

The current problem that I am running into is that those installation instructions are for RE1 and run melodic while I have a RE2 running noetic.

I tried going into the web_interface_installation_sh file and updating all the installation lines that would install melodic to noetic, but I am still running into problems (mainly missing files)
I also tried using these instructions

as well as these instructions

to no success.

recent errors:

RLException: while processing /home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/launch/imu_filter.launch:
Invalid roslaunch XML syntax: [Errno 2] No such file or directory: ‘/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/launch/imu_filter.launch’
The traceback for the exception was written to the log file

bash: ./launch/gripper_camera.launch: No such file or directory
[ERROR] [1683158228.037435552]: Cannot identify ‘/dev/hello-navigation-camera’: 2, No such file or directory
[gripper_camera-2] process has died [pid 14768, exit code 1, cmd /opt/ros/noetic/lib/usb_cam/usb_cam_node /usb_cam/image_raw:=/gripper_camera/image_raw __name:=gripper_camera __log:=/home/hello-robot/.ros/log/35279706-ea0e-11ed-a5d9-6b132f8583dd/gripper_camera-2.log].
log file: /home/hello-robot/.ros/log/35279706-ea0e-11ed-a5d9-6b132f8583dd/gripper_camera-2*.log
[navigation_camera-4] process has died [pid 14770, exit code 1, cmd /opt/ros/noetic/lib/usb_cam/usb_cam_node /usb_cam/image_raw:=/navigation_camera/image_raw __name:=navigation_camera __log:=/home/hello-robot/.ros/log/35279706-ea0e-11ed-a5d9-6b132f8583dd/navigation_camera-4.log].
log file: /home/hello-robot/.ros/log/35279706-ea0e-11ed-a5d9-6b132f8583dd/navigation_camera-4*.log

2

Hi @Rafael_MM, I’ll let others chime in on the hcrlab/stretch_web_interface and hello-robot/stretch_fisheye_web_interface repos since I’m not very familiar with them. But I think I may know what’s causing the errors you’ve appended at the bottom of your post. The Cannot identify '/dev/hello-navigation-camera' error likely results from an incorrect UDEV rule(s) for the teleop cameras, which prevents the usb_cam ROS node from connecting to one or both of the cameras. Let’s start by running ll /dev/hello* to see which UDEV rules have triggered. Ideally, we would see an entry for both /dev/hello-navigation-camera and /dev/hello-gripper-camera.

3

Hi @Rafael_MM! Sorry about the difficulty setting up the interface. We need to update the README.

First lets get everything installed for the stretch_web_interface:

  • Create a workspace with the hcrlab fork of stretch_ros (on noetic-devel) and stretch_web_interface
  • Run rosdep install --from-paths . --ignore-src -y -r from the workspace src directory to get all the package dependencies
  • Run web_interface_installation.sh to get extra dependencies (you are correct in updating the packages to noetic)
  • Run npm install in the stretch_web_interface directory
  • Browser features like the camera and microphone access require that the page be running in an SSL context, so we need certificates in order to serve the interface and enable SSL for the rosburdge websocket (see the launch files). We are going to use mkcert to manage a set of certificates accross the development machines and Stretch. Download the mkcert pre-built binaries and install mkcert:
curl -JLO "https://dl.filippo.io/mkcert/latest?for=linux/amd64"
chmod +x mkcert-v*-linux-amd64
sudo cp mkcert-v*-linux-amd64 /usr/local/bin/mkcert
sudo apt-get install libnss3-tools
roscd stretch_web_interface/certificates && CAROOT=`pwd` mkcert --install
  • To make your own certificates for your robot run (this example uses our robot slinky): mkcert slinky.hcrlab.cs.washington.edu slinky.local slinky.dev localhost 127.0.0.1 0.0.0.0 ::1
  • Then copy the Certificate Authority that mkcert created (rootCA-key.pem and rootCA.pem ) on the robot to ~/.local/share/mkcert

Next we want to get the fisheye cameras working. Binit is correct that they cannot be identified because the udev rules are likely incorrect.

  • Run sudo dmesg | grep usb to get information about currently plugged in devices. If you unplug and replug one of the fisheye cameras before running the command its information will be displayed at the bottom. You should see something like New USB device found, idVendor=1d6c, idProduct=0103, bcdDevice= 0.10.
  • Update the idVendor and idProduct in /etc/udev/rules.d/88-hello-navigation-camera.rules and /etc/udev/rules.d/89-hello-gripper-camera.rules
  • Unplug and replug both cameras after updating the udev rules and reload the udev rules: sudo udevadm control --reload-rules
  • Run ll /dev/hello* and check for both /dev/hello-navigation-camera and /dev/hello-gripper-camera
  • If you don’t see them then try unplugging/replugging the cameras a couple times and/or restarting the robot
  • You might need to swap the kernels: KERNELS=="1-1.3.*" changed to KERNELS=="1-1.2.*" and KERNELS=="1-1.2.*" changed to KERNELS=="1-1.3.*"
  • Every time you change the udev rules you will have to reload the rules and unplug and replug the cameras to see if the symlinks appear

If you can see the symlinks you can try running the interface. Run the following commands on the robot in separate terminals:

roslaunch stretch_web_interface web_interface.launch
./start_web_server_and_robot_browser.sh

Enter your robot’s hostname or IP address on google chrome and you should see the interface.

Hope this helps, please let me know if you run into any issues!

1 Like
4

Hey Vinitha,

I have a problem with setting up the Kernels. The Kernel name that was on my device was set to “Video*” for both gripper and navigation, I tried using 1-1.2. and 1-1.3, but the cameras don’t get set up as dev/hello…

I was able to find the IdVender and IdProduct and that has been switch, but I am still unable to get the cameras to be display

5

You want to keep KERNEL=video* and add KERNELS

6

Still can’t get it to appear (I added some pictures just to make sure I am doing this correctly) as /dev/hello-navigation-camera and /dev/hello-gripper-camera , and I have tried it with both /etc/udev/rules.d/88-hello-navigation-camera.rulesand/etc/udev/rules.d/89-hello-gripper-camera.rules`

Troubleshooting:
I first disconnected (one at a time) both fish-eye cameras to verified what USB device they were.

After that I reconnected each camera and type sudo dmesg | grep to get the information of each device once they were connected

One thing I did notice was that the serial numbers were different so I did try changing them so they matched, but was not able to get the cameras working by doing so.

7

It’s supposed to be KERNEL==video* (not Video*). From you screenshots it looks like the kernels for the navigation camera is 1-2.1.1 and 1-4.1.3.1 for the gripper camera. Add KERNELS==1-2.1* to the navigation camera udev rules and KERNELS==1-4.1* to the gripper camera udev rules.

Some more helpful information: udev - ArchWiki

Hope this helps!

8

Hey @vinitha,

Thanks for the help after changing those suggested things as well as matching the serial# I was able to get Stretch to register both gripper and navigation cameras.

I am now running into a different problem. When I execute

roslaunch stretch_web_interface web_interface.launch

I get this error

FileNotFoundError: [Errno 2] No such file or directory: '/home/hello-robot/catkin_ws/src/stretch_ros/stretch_description/urdf/stretch.urdf'

Which after looking into the files it does seem like stretch.urdf is missing from the repo.
[update: I was able to reinstall my urdf files by reassigning them]

new error now

RLException: while processing /home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/launch/imu_filter.launch:
Invalid roslaunch XML syntax: [Errno 2] No such file or directory: '/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/launch/imu_filter.launch'
The traceback for the exception was written to the log file

This files DOES seem to be missing since other repos do include it.
[update: after making a launch file of imu_filter.launch i am now getting

base_local_planner_params.yaml

that is missing. It seems like a couple of files are missing from this repo GitHub - hcrlab/stretch_ros: ROS-related code for the Stretch RE1 mobile manipulator from Hello Robot Inc.
compare to the hello-robot repo GitHub - hello-robot/stretch_ros: ROS-related code for the Stretch RE1 mobile manipulator from Hello Robot Inc.
]

I am able to run

roscd stretch_web_interface/bash_scripts/
./start_web_server_and_robot_browser.sh

which gives me this after connecting to the web interface using username: 01 Password: xXTgfdH8

successful connection to socket.io
data.user =
{
  _id: -,(removed for sharing purposes) 
  username: 'o1',
  name: 'operator 1',
  role: 'operator',
  approved: true,
  date: 2017-10-17T22:30:43.733Z,
  __v: 0
}
connection authorized!
new socket.io connection
socket.handshake = 
{
  headers: {
    host: '-', (removed for sharing purposes) 
    connection: 'keep-alive',
    'sec-ch-ua': '"Chromium";v="113", "Not-A.Brand";v="24"',
    accept: '*/*',
    'sec-ch-ua-mobile': '?0',
    'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36',
    'sec-ch-ua-platform': '"Linux"',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-mode': 'cors',
    'sec-fetch-dest': 'empty',
    referer: 'https://-/operator/', (removed for sharing purposes) 
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US,en;q=0.9',
    cookie: 'connect.sid=-
  },(removed for sharing purposes) 
  time: 'Mon May 15 2023 17:56:05 GMT-0700 (Pacific Daylight Time)',
  address: '::-'(removed for sharing purposes) ,
  xdomain: false,
  secure: true,
  issued: 1684198565583,
  url: '/socket.io/?EIO=4&transport=polling&t=OWY3gR1',
  query: [Object: null prototype] {
    EIO: '4',
    transport: 'polling',
    t: 'OWY3gR1'
  },
  auth: {}
}
AN OPERATOR HAS CONNECTED
available_robots =
Set(0) {}
operator has requested the available robots
available_robots = Set(0) {}

Which allows me to access the interface, but tells me that there is no robot connected.

9

Hi @Rafael_MM

Sorry for the late response! Could you try using hello robot’s stretch_ros in the dev/noetic branch. To get stretch.urdf run ./xacro_to_urdf.sh in the stretch_description folder.

The reason the there’s no available robots after running ./start_web_server_and_robot_browser.sh is because you haven’t been able to successfully launch web_interface.launch. Hopefully this fixes the error and you should see r1 in the list of available robots.

1 Like
10

Hi Rafael,

I also got FileNotFoundError for stretch.urdf. I was wondering how did you reassign them.

11

Problem resolved. I copied the file from another user account to mine.