Use Gamepad controller to teleop robot remotely when connected to same wifi

Hii- I have a use case where i want to operate the robot remotely situated at a different location using gamepad controller and view the camera feed on the screen remotely.

  • Stretch3 has a web browser to where we can view the camera feed and control the robot by clicking on the buttons in the screen. Is it possible that I could connect the gamepad to my laptop remotely and control the robot from different location through it.
  • Any resources to do so would be helpful.
1 Like

Hi @vipul, Yes you should be able to control the robot using the Gamepad controller from a remote computer in the network using the newly added feature Gamepad Mode to the ROS2 stretch_driver.

You can activate Gamepad Mode by calling the /switch_to_gamepad_mode service when the stretch driver is running.

ros2 service call /switch_to_gamepad_mode std_srvs/srv/Trigger

In this mode, stretch driver listens to /gamepad_joy topic for a valid Joy-type message. The Joy message formats and helper functions are provided in the

You can run the node on your remote computer in the network with the provided gamepad USB dongle plugged in, and control the robot using the ROS2 drivers.

Here are the steps I have summarised for you to use Gamepad Mode with the Stretch Web Teleop interface running so you can see the camera streams in the browser:

  • First on the robot, start the Web interface
# Robot side
cd ~/ament_ws/src/stretch_web_teleop/
  • Install Stretch Body python package in the remote computer
# remote computer
pip3 install hello-robot-stretch-body
# remote computer
cd <your-ros2-work-space>/src/
git clone
cd ../
colcon build --packages-select hello_helpers stretch_core
  • Plug in the Gamepad Dongle to remote computer
  • Activate gamepad mode and start node in the remote PC
# remote computer
ros2 service call /switch_to_gamepad_mode std_srvs/srv/Trigger
ros2 run stretch_core remote_gamepad

Here are some more helpful info on using the Gamepad Mode

Published topics

  • /stretch_gamepad_state: Gamepad Joystick states
  • /is_gamepad_dongle: Check if the gamepad dongle is present in the robot

Joy Message Reference Index Table


Index Gamepad Button Name Stretch motion Description Float Data range Default
0 left_stick_x Base Rotational speed -1 to +1 0
1 left_stick_y Base Linear speed -1 to +1 0
2 right_stick_x Arm Linear speed -1 to +1 0
3 right_stick_y Lift Linear speed -1 to +1 0
4 left_trigger_pulled Precision mode (if v>0.9) 0 to +1 0
5 right_trigger_pulled Fast Navigation mode (if v>0.9 and lift,arm is in stow pos) 0 to +1 0


Index Gamepad Button Name Stretch motion Description Integer Data holding Default
0 left_stick_button_pressed Un-Assigned 0 or 1 0
1 right_stick_button_pressed Un-Assigned 0 or 1 0
2 bottom_button_pressed (A) Gripper Close 0 or 1 0
3 right_button_pressed (B) Gripper Open 0 or 1 0
4 left_button_pressed (X) Switch between Dex/Head 0 or 1 0
5 top_button_pressed (Y) Stow robot 0 or 1 0
6 left_shoulder_button_pressed Wrist_Yaw CCW rotate 0 or 1 0
7 right_shoulder_button_pressed Wrist_Yaw CW rotate 0 or 1 0
8 select_button_pressed (back) Long-press Shutdown 0 or 1 0
9 start_button_pressed (start) Home robot if not calibrated 0 or 1 0
10 bottom_pad_pressed (down) Wrist_Pitch / Head_Tilt CW rotate 0 or 1 0
11 top_pad_pressed (up) Wrist_Pitch / Head_Tilt CCW rotate 0 or 1 0
12 left_pad_pressed (left) Wrist_Roll / Head_Pan CW rotate 0 or 1 0
13 right_pad_pressed (right) Wrist_Roll / Head_Pan CCW rotate 0 or 1 0

Thanks @Mohamed_Fazil that did work, with this setup I can teleop the robot with gamepad on remote computer. Is there a way to get the camera feed as well on the remote computer while running the teleop through gamepad?

@vipul Glad you are able to do remote gamepad teleop now. For viewing the camera streams, I assumed you had launched the Stretch Driver node by launching the web interface pointed out in the first step. You can use the browser to see all the camera feeds through the Web Interface UI using the URL printed while you launched the interface and keep controlling the robot using the gamepad in gamepad mode from remote computer.
Note: Currently, the motion buttons in the Web Interface UI don’t work when you’re in gamepad mode.

@Mohamed_Fazil Thank you! That worked out completely fine. Also I was wondering if the stretch can send camera views to a VR headset and can be controlled from there ? Any resources would be helpful.

@vipul We don’t have any work from us to do a VR teleop of the robot. But we can provide you with resources from our robot side software to enable you to build your own VR teleop project if you are interested. For example, we have this Pythonic Stretch Dex Teleop demo, which allows you to do 6 DOF End-effector control and you can go way up into integrating it into a VR.