Race Condition When Running Two Stretches

Hi! We are trying to send commands to two stretches at the same time so that they can turn to the object, lift the arm, and grab the item. However, when we tried this portion of the functions:

It shows the error “Got a result when we were already in the DONE state”. Our hypothesis is that the functions send_goal() and wait_for_result() might be accessing the same variables, causing a race condition when one robot finished execution (done = True) but the other hasn’t. Is there a way to resolve this? Thank you!

Hi @JannaLam, just to confirm that I understand, is it the case that you’re sending both Stretchs the same command with a single node? This might explain the error; if both Stretch received the same command, they would both report Done when they completed the command. For your use-case, would it make sense to remap each Stretch’s topics (e.g. /stretch1/stretch_controller/follow_joint_trajectory and /stretch2/stretch_controller/follow_joint_trajectory), so you can send commands to each Stretch without mixing them up.

We did not directly subscribe to these topics. Instead, we use the built-in function send_goal(), etc. If we add namespace for each of the Stretch’s trajectory topic, should we also revise the send_goal function? Thanks!

Hi, we tried adding “stretch1” manually before everywhere “/stretch_controller/follow_joint_trajectory” is used on the robots, and also adding a remap in stretch_driver.launch, but then when running our launch file there is no response and no error message, so looks like there is some sort of miscommunication - could you please provide more suggestions on this? Thanks:)

I’d recommend using the rqt_graph tool to visualize the topics and the connections between them. You can use the ROS graph to identify which edge is missing, which should help you identify the reason for the no response error message.

1 Like

Just an update :slight_smile:
We are able to have a master node controlling several stretches at a time to have them doing several tasks (same or different) at the same time! We made some revisions in the built-in hello_misc.py so that each robot can recognize which result should be sent to which robot, and we can also apply multi-threading rn. Thanks again for your help!

That’s great to hear @JannaLam! If you’re interested in merging your solution back upstream, let me know!