Hii- I have a use case where i want to operate the robot remotely situated at a different location using gamepad controller and view the camera feed on the screen remotely.
Hi @vipul, Yes you should be able to control the robot using the Gamepad controller from a remote computer in the network using the newly added feature Gamepad Mode to the ROS2 stretch_driver.
You can activate Gamepad Mode by calling the /switch_to_gamepad_mode service when the stretch driver is running.
ros2 service call /switch_to_gamepad_mode std_srvs/srv/Trigger
In this mode, stretch driver listens to /gamepad_joy topic for a valid Joy-type message. The Joy message formats and helper functions are provided in the gamepad_conversion.py
You can run the remote_gamepad.py node on your remote computer in the network with the provided gamepad USB dongle plugged in, and control the robot using the ROS2 drivers.
Here are the steps I have summarised for you to use Gamepad Mode with the Stretch Web Teleop interface running so you can see the camera streams in the browser:
# Robot side
cd ~/ament_ws/src/stretch_web_teleop/
./launch_interface.sh
# remote computer
pip3 install hello-robot-stretch-body
# remote computer
cd <your-ros2-work-space>/src/
git clone https://github.com/hello-robot/stretch_ros2
cd ../
colcon build --packages-select hello_helpers stretch_core
# remote computer
ros2 service call /switch_to_gamepad_mode std_srvs/srv/Trigger
ros2 run stretch_core remote_gamepad
Here are some more helpful info on using the Gamepad Mode
/stretch_gamepad_state: Gamepad Joystick states/is_gamepad_dongle: Check if the gamepad dongle is present in the robot| Index | Gamepad Button Name | Stretch motion Description | Float Data range | Default |
|---|---|---|---|---|
| 0 | left_stick_x | Base Rotational speed | -1 to +1 | 0 |
| 1 | left_stick_y | Base Linear speed | -1 to +1 | 0 |
| 2 | right_stick_x | Arm Linear speed | -1 to +1 | 0 |
| 3 | right_stick_y | Lift Linear speed | -1 to +1 | 0 |
| 4 | left_trigger_pulled | Precision mode (if v>0.9) | 0 to +1 | 0 |
| 5 | right_trigger_pulled | Fast Navigation mode (if v>0.9 and lift,arm is in stow pos) | 0 to +1 | 0 |
| Index | Gamepad Button Name | Stretch motion Description | Integer Data holding | Default |
|---|---|---|---|---|
| 0 | left_stick_button_pressed | Un-Assigned | 0 or 1 | 0 |
| 1 | right_stick_button_pressed | Un-Assigned | 0 or 1 | 0 |
| 2 | bottom_button_pressed (A) | Gripper Close | 0 or 1 | 0 |
| 3 | right_button_pressed (B) | Gripper Open | 0 or 1 | 0 |
| 4 | left_button_pressed (X) | Switch between Dex/Head | 0 or 1 | 0 |
| 5 | top_button_pressed (Y) | Stow robot | 0 or 1 | 0 |
| 6 | left_shoulder_button_pressed | Wrist_Yaw CCW rotate | 0 or 1 | 0 |
| 7 | right_shoulder_button_pressed | Wrist_Yaw CW rotate | 0 or 1 | 0 |
| 8 | select_button_pressed (back) | Long-press Shutdown | 0 or 1 | 0 |
| 9 | start_button_pressed (start) | Home robot if not calibrated | 0 or 1 | 0 |
| 10 | bottom_pad_pressed (down) | Wrist_Pitch / Head_Tilt CW rotate | 0 or 1 | 0 |
| 11 | top_pad_pressed (up) | Wrist_Pitch / Head_Tilt CCW rotate | 0 or 1 | 0 |
| 12 | left_pad_pressed (left) | Wrist_Roll / Head_Pan CW rotate | 0 or 1 | 0 |
| 13 | right_pad_pressed (right) | Wrist_Roll / Head_Pan CCW rotate | 0 or 1 | 0 |
Thanks @Mohamed_Fazil that did work, with this setup I can teleop the robot with gamepad on remote computer. Is there a way to get the camera feed as well on the remote computer while running the teleop through gamepad?
@vipul Glad you are able to do remote gamepad teleop now. For viewing the camera streams, I assumed you had launched the Stretch Driver node by launching the web interface pointed out in the first step. You can use the browser to see all the camera feeds through the Web Interface UI using the URL printed while you launched the interface and keep controlling the robot using the gamepad in gamepad mode from remote computer.
Note: Currently, the motion buttons in the Web Interface UI donât work when youâre in gamepad mode.
@Mohamed_Fazil Thank you! That worked out completely fine. Also I was wondering if the stretch can send camera views to a VR headset and can be controlled from there ? Any resources would be helpful.
@vipul We donât have any work from us to do a VR teleop of the robot. But we can provide you with resources from our robot side software to enable you to build your own VR teleop project if you are interested. For example, we have this Pythonic Stretch Dex Teleop demo, which allows you to do 6 DOF End-effector control and you can go way up into integrating it into a VR.