# Use Gamepad controller to teleop robot remotely when connected to same wifi

**URL:** https://forum.hello-robot.com/t/use-gamepad-controller-to-teleop-robot-remotely-when-connected-to-same-wifi/1005
**Category:** Knowledge Base
**Created:** [June 6, 2024, 5:53pm UTC](https://forum.hello-robot.com/t/use-gamepad-controller-to-teleop-robot-remotely-when-connected-to-same-wifi/1005 "2024-06-06T17:53:17Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![vipul](https://avatars.discourse-cdn.com/v4/letter/v/cdc98d/32.png) [@vipul](https://forum.hello-robot.com/u/vipul)
#### Post date: [June 6, 2024, 5:53pm UTC](https://forum.hello-robot.com/t/use-gamepad-controller-to-teleop-robot-remotely-when-connected-to-same-wifi/1005/1 "2024-06-06T17:53:17Z")

</div>

Hii- I have a use case where i want to operate the robot remotely situated at a different location using gamepad controller and view the camera feed on the screen remotely.

- Stretch3 has a web browser to where we can view the camera feed and control the robot by clicking on the buttons in the screen. Is it possible that I could connect the gamepad to my laptop remotely and control the robot from different location through it.
- Any resources to do so would be helpful.

---

<div class="post-metadata">

### Author: ![Mohamed\_Fazil](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/mohamed_fazil/32/261_2.png) [@Mohamed\_Fazil](https://forum.hello-robot.com/u/Mohamed_Fazil)
#### Post date: [June 6, 2024, 8:57pm UTC](https://forum.hello-robot.com/t/use-gamepad-controller-to-teleop-robot-remotely-when-connected-to-same-wifi/1005/2 "2024-06-06T20:57:22Z")

</div>

Hi @vipul, Yes you should be able to control the robot using the Gamepad controller from a remote computer in the network using the newly added feature **Gamepad Mode** to the ROS2 stretch\_driver.

> <https://github.com/hello-robot/stretch_ros2/pull/87>
>
> \## This PR introduces the new \`gamepad\` mode to the stretch driver
> Gamepad mode… enables the provided gamepad with stretch to control the robot's motions with ros2 Stretch Driver. If the gamepad USB dongle is plugged out, the robot will stop making any motions in this mode and can be plugged back in real-time.
> Activate Gamepad Mode by calling the \`/switch\_to\_gamepad\_mode\` service when the stretch driver is running.
> 
> Alternatively, in this mode, stretch driver also listens to \`/gamepad\_joy\` topic for a valid \[Joy-type\](http://docs.ros.org/en/melodic/api/sensor\_msgs/html/msg/Joy.html) message from a remote gamepad to control stretch remotely in the network. The Joy message formats and helper functions are provided in the \[gamepad\_conversion.py\](https://github.com/hello-robot/stretch\_ros2/blob/cdb24f976baa71de2c532827d7737626d37a9c56/hello\_helpers/src/hello\_helpers/gamepad\_conversion.py)
> 
> Added the \[remote\_gamepad.py\](https://github.com/hello-robot/stretch\_ros2/blob/feature/gamepad\_mode/stretch\_core/stretch\_core/remote\_gamepad.py) which allows a user to plug-in the provided gamepad USB dongle to a remote workstation in network and teleop the robot remotely through ROS2.
> \### Remote Gamepad Teleop
> \- First on the robot side start Stretch Driver in gamepad mode
> \`\`\`
> ros2 launch stretch\_core stretch\_driver.launch.py mode:=gamepad
> \`\`\`
> \- Install Strech Core and Stretch Body package in the remote PC
> \- Plug in the Gamepad Dongle to remote PC
> \- Start \[remote\_gamepad.py\](https://github.com/hello-robot/stretch\_ros2/blob/feature/gamepad\_mode/stretch\_core/stretch\_core/remote\_gamepad.py) node in the remote PC
> \`\`\`
> ros2 run stretch\_core remote\_gameoad.py
> \`\`\`
> 
> \### Published topics
> \- \`/stretch\_gamepad\_state\`: Gamepad Joystick states
> \- \`/is\_gamepad\_dongle\`: Check if the gamepad dongle is present in the robot
> 
> \### Joy Message Reference Index Table 
> \#### Axes
> 
> Index | Gamepad Button Name | Stretch motion Description | Float Data range | Default
> \-- | -- | -- | -- | --
> 0 | left\_stick\_x | Base Rotational speed | -1 to +1 | 0
> 1 | left\_stick\_y | Base Linear speed | -1 to +1 | 0
> 2 | right\_stick\_x | Arm Linear speed | -1 to +1 | 0
> 3 | right\_stick\_y | Lift Linear speed | -1 to +1 | 0
> 4 | left\_trigger\_pulled | Precision mode (if v\>0.9) | 0 to +1 | 0
> 5 | right\_trigger\_pulled | Fast Navigation mode (if v\>0.9 and lift,arm is in stow pos) | 0 to +1 | 0
> 
> \#### Buttons
> 
> Index | Gamepad Button Name | Stretch motion Description | Integer Data holding | Default
> \-- | -- | -- | -- | --
> 0 | left\_stick\_button\_pressed | Un-Assigned | 0 or 1 | 0
> 1 | right\_stick\_button\_pressed | Un-Assigned | 0 or 1 | 0
> 2 | bottom\_button\_pressed (A) | Gripper Close | 0 or 1 | 0
> 3 | right\_button\_pressed (B) | Gripper Open | 0 or 1 | 0
> 4 | left\_button\_pressed (X) | Switch between Dex/Head | 0 or 1 | 0
> 5 | top\_button\_pressed (Y) | Stow robot | 0 or 1 | 0
> 6 | left\_shoulder\_button\_pressed | Wrist\_Yaw CCW rotate | 0 or 1 | 0
> 7 | right\_shoulder\_button\_pressed | Wrist\_Yaw CW rotate | 0 or 1 | 0
> 8 | select\_button\_pressed (back) | Long-press Shutdown | 0 or 1 | 0
> 9 | start\_button\_pressed (start) | Home robot if not calibrated | 0 or 1 | 0
> 10 | bottom\_pad\_pressed (down) | Wrist\_Pitch / Head\_Tilt CW rotate | 0 or 1 | 0
> 11 | top\_pad\_pressed (up) | Wrist\_Pitch / Head\_Tilt CCW rotate | 0 or 1 | 0
> 12 | left\_pad\_pressed (left) | Wrist\_Roll / Head\_Pan CW rotate | 0 or 1 | 0
> 13 | right\_pad\_pressed (right) | Wrist\_Roll / Head\_Pan CCW rotate | 0 or 1 | 0
> 
> \</div\>\<br /\>

You can activate Gamepad Mode by calling the `/switch_to_gamepad_mode` service when the stretch driver is running.

```auto
ros2 service call /switch_to_gamepad_mode std_srvs/srv/Trigger

```

In this mode, stretch driver listens to `/gamepad_joy` topic for a valid [Joy-type](http://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/Joy.html) message. The Joy message formats and helper functions are provided in the [gamepad\_conversion.py](https://github.com/hello-robot/stretch_ros2/blob/cdb24f976baa71de2c532827d7737626d37a9c56/hello_helpers/src/hello_helpers/gamepad_conversion.py)

You can run the [remote\_gamepad.py](https://github.com/hello-robot/stretch_ros2/blob/feature/gamepad_mode/stretch_core/stretch_core/remote_gamepad.py) node on your remote computer in the network with the provided gamepad USB dongle plugged in, and control the robot using the ROS2 drivers.

Here are the steps I have summarised for you to use Gamepad Mode with the Stretch Web Teleop interface running so you can see the camera streams in the browser:

- First on the robot, start the Web interface

```auto
# Robot side
cd ~/ament_ws/src/stretch_web_teleop/
./launch_interface.sh

```

- Install Stretch Body python package in the remote computer

```auto
# remote computer
pip3 install hello-robot-stretch-body

```

- Install [hello\_helpers](https://github.com/hello-robot/stretch_ros2/tree/humble/hello_helpers) and [stretch\_core](https://github.com/hello-robot/stretch_ros2/tree/humble/stretch_core) ROS2 packages from stretch\_ros2 in remote computer

```auto
# remote computer
cd <your-ros2-work-space>/src/
git clone https://github.com/hello-robot/stretch_ros2
cd ../
colcon build --packages-select hello_helpers stretch_core

```

- Plug in the Gamepad Dongle to remote computer
- Activate gamepad mode and start [remote\_gamepad.py](https://github.com/hello-robot/stretch_ros2/blob/feature/gamepad_mode/stretch_core/stretch_core/remote_gamepad.py) node in the remote PC

```auto
# remote computer
ros2 service call /switch_to_gamepad_mode std_srvs/srv/Trigger
ros2 run stretch_core remote_gamepad

```

Here are some more helpful info on using the Gamepad Mode

#### Published topics

- `/stretch_gamepad_state`: Gamepad Joystick states
- `/is_gamepad_dongle`: Check if the gamepad dongle is present in the robot

#### Joy Message Reference Index Table

#### Axes

| Index | Gamepad Button Name | Stretch motion Description | Float Data range | Default |
| --- | --- | --- | --- | --- |
| 0 | left\_stick\_x | Base Rotational speed | -1 to +1 | 0 |
| 1 | left\_stick\_y | Base Linear speed | -1 to +1 | 0 |
| 2 | right\_stick\_x | Arm Linear speed | -1 to +1 | 0 |
| 3 | right\_stick\_y | Lift Linear speed | -1 to +1 | 0 |
| 4 | left\_trigger\_pulled | Precision mode (if v\>0.9) | 0 to +1 | 0 |
| 5 | right\_trigger\_pulled | Fast Navigation mode (if v\>0.9 and lift,arm is in stow pos) | 0 to +1 | 0 |

#### Buttons

| Index | Gamepad Button Name | Stretch motion Description | Integer Data holding | Default |
| --- | --- | --- | --- | --- |
| 0 | left\_stick\_button\_pressed | Un-Assigned | 0 or 1 | 0 |
| 1 | right\_stick\_button\_pressed | Un-Assigned | 0 or 1 | 0 |
| 2 | bottom\_button\_pressed (A) | Gripper Close | 0 or 1 | 0 |
| 3 | right\_button\_pressed (B) | Gripper Open | 0 or 1 | 0 |
| 4 | left\_button\_pressed (X) | Switch between Dex/Head | 0 or 1 | 0 |
| 5 | top\_button\_pressed (Y) | Stow robot | 0 or 1 | 0 |
| 6 | left\_shoulder\_button\_pressed | Wrist\_Yaw CCW rotate | 0 or 1 | 0 |
| 7 | right\_shoulder\_button\_pressed | Wrist\_Yaw CW rotate | 0 or 1 | 0 |
| 8 | select\_button\_pressed (back) | Long-press Shutdown | 0 or 1 | 0 |
| 9 | start\_button\_pressed (start) | Home robot if not calibrated | 0 or 1 | 0 |
| 10 | bottom\_pad\_pressed (down) | Wrist\_Pitch / Head\_Tilt CW rotate | 0 or 1 | 0 |
| 11 | top\_pad\_pressed (up) | Wrist\_Pitch / Head\_Tilt CCW rotate | 0 or 1 | 0 |
| 12 | left\_pad\_pressed (left) | Wrist\_Roll / Head\_Pan CW rotate | 0 or 1 | 0 |
| 13 | right\_pad\_pressed (right) | Wrist\_Roll / Head\_Pan CCW rotate | 0 or 1 | 0 |

---

<div class="post-metadata">

### Author: ![vipul](https://avatars.discourse-cdn.com/v4/letter/v/cdc98d/32.png) [@vipul](https://forum.hello-robot.com/u/vipul)
#### Post date: [June 7, 2024, 2:09am UTC](https://forum.hello-robot.com/t/use-gamepad-controller-to-teleop-robot-remotely-when-connected-to-same-wifi/1005/3 "2024-06-07T02:09:13Z")

</div>

Thanks @Mohamed_Fazil that did work, with this setup I can teleop the robot with gamepad on remote computer. Is there a way to get the camera feed as well on the remote computer while running the teleop through gamepad?

---

<div class="post-metadata">

### Author: ![Mohamed\_Fazil](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/mohamed_fazil/32/261_2.png) [@Mohamed\_Fazil](https://forum.hello-robot.com/u/Mohamed_Fazil)
#### Post date: [June 7, 2024, 5:16am UTC](https://forum.hello-robot.com/t/use-gamepad-controller-to-teleop-robot-remotely-when-connected-to-same-wifi/1005/4 "2024-06-07T05:16:45Z")

</div>

@vipul Glad you are able to do remote gamepad teleop now. For viewing the camera streams, I assumed you had launched the Stretch Driver node by launching the web interface pointed out in the first step. You can use the browser to see all the camera feeds through the Web Interface UI using the URL printed while you launched the interface and keep controlling the robot using the gamepad in gamepad mode from remote computer.  
Note: Currently, the motion buttons in the Web Interface UI don’t work when you’re in gamepad mode.

---

<div class="post-metadata">

### Author: ![vipul](https://avatars.discourse-cdn.com/v4/letter/v/cdc98d/32.png) [@vipul](https://forum.hello-robot.com/u/vipul)
#### Post date: [June 7, 2024, 4:20pm UTC](https://forum.hello-robot.com/t/use-gamepad-controller-to-teleop-robot-remotely-when-connected-to-same-wifi/1005/5 "2024-06-07T16:20:34Z")

</div>

@Mohamed_Fazil Thank you! That worked out completely fine. Also I was wondering if the stretch can send camera views to a VR headset and can be controlled from there ? Any resources would be helpful.

---

<div class="post-metadata">

### Author: ![Mohamed\_Fazil](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/mohamed_fazil/32/261_2.png) [@Mohamed\_Fazil](https://forum.hello-robot.com/u/Mohamed_Fazil)
#### Post date: [June 7, 2024, 9:06pm UTC](https://forum.hello-robot.com/t/use-gamepad-controller-to-teleop-robot-remotely-when-connected-to-same-wifi/1005/6 "2024-06-07T21:06:06Z")

</div>

@vipul We don’t have any work from us to do a VR teleop of the robot. But we can provide you with resources from our robot side software to enable you to build your own VR teleop project if you are interested. For example, we have this Pythonic Stretch Dex Teleop demo, which allows you to do 6 DOF End-effector control and you can go way up into integrating it into a VR.

> **[GitHub - hello-robot/stretch\_dex\_teleop: Dexterous teleoperation for the...](https://github.com/hello-robot/stretch_dex_teleop/?tab=readme-ov-file#dex-teleop-for-stretch)**
>
> Dexterous teleoperation for the Stretch mobile manipulators from Hello Robot Inc. - hello-robot/stretch\_dex\_teleop