How do I add custom Mujoco environments to docker install of stretch_ros2/stretch_simulation

Ask
1

Hi,

I am getting started in using the docker installation of the stretch mujoco simulation and am wondering if I can upload a custom mujoco scene or even just a plain scene where the robot starts at the origin on a a basic floor. I have looked through the stretch_mujoco_driver.launch file and it seems like choosing a robo casa scene is the default. Is there a step I am missing or is there a workaround to this, specifically in the docker installation version?

Thanks!

2

Hi @jimmy, thanks for your post!

The StretchMujocoSimulator class accepts scene_xml_path and model parameters (stretch_mujoco/stretch_mujoco/stretch_mujoco_simulator.py at main · hello-robot/stretch_mujoco · GitHub ), you can pass either the path to your xml file, or a loaded MjModel instance and it will use it!

I changed the first few lines of the example in the README to show this usage:

from stretch_mujoco import StretchMujocoSimulator

if __name__ == "__main__":
    sim = StretchMujocoSimulator(scene_xml_path="path/to/custom_scene_file")
    sim.start(headless=False) # This will open a Mujoco-Viewer window

For getting these changes into the Docker image, it would depend on your setup; you could use devcontainer in vscode or something similar to access the code on the container directly, or mount your custom scene as a volume and use it within your container!

2 Likes
3

@shehab thank you for the reply! Mounting the volume makes the most sense.

A few follow ups:

  1. Does this also work for the ros launch file stretch_mujoco_driver.launch.py in: /ament_ws/src/stretch_ros2/stretch_simulation/launch ?

  2. I found that including the --network=host as a startup setting prevents ros messages from making it out of the docker environment and to another terminal on the same computer (might just be an issue for me but figured I’d include this in case anyone else runs into it).

  3. What I’m ultimately trying to do is test out a controller in simulation that sends joint commands as a ros message, so I thought this docker install might be the easiest approach to immediately get started. Based on your response, I’m starting to think it might be easier to write my own simple ros wrapper around the stretch mujoco sim library to just start up a simple simulation and receive joint commands from my controller. What are your thoughts, would that be an easier approach?

Thanks so much!

4

Hi @jimmy,

  1. We did not expose scene_xml_path as a ros2 parameter, sorry, you might have to add it for now. It would take us some time to get to this. Thank you!
  2. Interesting, I’ve not run into this before. I wonder if it’s a DDS quirk. Please feel free to open an issue on the stretch_ros2 repo and we can look into it!
  3. You could run your control code on a separate node and send ros2 messages to stretch_simulation. I’m not sure if there is any benefit to building another wrapper, but please feel free to do whatever is easiest for you! If there is something we can add to support your use-case, please let us know here or by opening an issue on stretch_ros2 or stretch_mujoco.

Best wishes,
Shehab

2 Likes
5

Hi @shehab thank you so much for the responses

A couple last questions for now:

  1. Is the sim function “sim.add_world_frame((0.1,0,0), (0,0,0))" to add reference frames to the scene accessible as a ros2 parameter or service call?

  2. In the stretch_mujoco python library, the documentation mentions that the StretchMujocoSimulator() class also exposes the mjModel and mjData. I’ve been trying to access the mjModel and mjData using the basic example on the github readme page but haven’t had any luck in finding those attributes in my StretchMujocoSimulator() object. What is the best way to access those variables?

Thanks!

6

Hi Jimmy,

Thank you for your posts!

  1. stretch_mujoco_driver (stretch_ros2/stretch_simulation/stretch_mujoco_driver/stretch_mujoco_driver.py at humble · hello-robot/stretch_ros2 · GitHub) does not expose a service like this yet. You would need to add it for your code for now, sorry, it would take some time for us to add it, but we plan to!
  2. I see this line in the README, unfortunately, it’s a bit confusing; the StretchMujocoSimulator is the client, it abstracts control of the server (stretch_mujoco/stretch_mujoco/mujoco_server.py at main · hello-robot/stretch_mujoco · GitHub), and does not have access to mjdata and mjmodel. Data between the server and client is handled by MujocoServerProxies via inter-process communication (IPC), you can add your own data to this dict to ferry custom data over (stretch_mujoco/stretch_mujoco/mujoco_server.py at main · hello-robot/stretch_mujoco · GitHub).

Best wishes,
Shehab

1 Like
7

@shehab

One last question, I promise :smiling_face_with_tear:

Is there a simple way to clear (or update) the frames added by the add_world_frame() method in the python library? Or is this something I would also have to make a custom change for?

Thank you so much, you have been very helpful

1 Like
8

Hey @jimmy,

They’re all good questions! Please feel free to keep asking! I took notes to add your suggestions in, sorry it might be a while until we do though. Feel free to push a PR for anything that you think might be useful to others as well. Thanks!

The client does not have a way to modify or remove them. For some background, mujoco does not allow users to add objects at runtime. The world frame objects are added to the special user_scn object in the viewer (Python - MuJoCo Documentation).
I am not sure if there is a way to delete an object that has been added. As a hack, you can change its size to be really small or set the alpha of the rgba value to 0. You can store the geom ID to a dict when the geom is created (here: stretch_mujoco/stretch_mujoco/mujoco_server_passive.py at main · hello-robot/stretch_mujoco · GitHub ) to modify later.
I have not tried modifying them before, hopefully you don’t run into any issues!

Best,
Shehab

1 Like