I have tried to run the stretch_ros demos, but I am not been successful. For example, whenever I try to run the grasp object demos, I get the error below. How do I fix it?
Hi @sesem378, welcome to the forum! Could you describe your robot and include the full roslaunch output?
- Stretch RE1 or 2
- Serial no.
- What tool in on the robot?
- Which version of Ubuntu is installed?
- Which version of ROS is being used?
We have the Stretch 2 robot (Serial No. 2020) equipped with the default gripper. Our system setup includes Ubuntu 20.04 with ROS Noetic.
Thank you @sesem378 for the details. I don’t see any errors in the roslaunch output. Could you describe what’s happening when you run the grasp_object code? The demos are intended to serve as a starting point for writing autonomous code, but they aren’t robust to every situation, so there could be a few reasons for why it didn’t achieve the grasp.
I suppressed the warnings that I showed in the previous picture. But when I run it, nothing happens.
Hey @sesem378, gotcha. It seems like we’ll need to debug what the node is seeing to determine why nothing is happening. First off, I’d ask that you confirm that your set-up closely matches the nominal set-up shown in the grasping video: a small flat table set adjacent to the robot with only 1 object on it, with clear lighting and no wall behind the table. Second, when the grasp object demo runs, debug images are logged to the ~/stretch_user/debug directory on your robot. The images logged there can help us understand what the node is seeing. For example, in the image below:
The three debug images show what the robot sees, the 2 in ~/stretch_user/debug/grasp_object/ show a overhead RGB and depth image of the table with the water bottle on it. The image in ~/stretch_user/debug/get_grasp_target/ shows what the grasp_object node thinks. Green is what FUNMAP thinks the surface is, red is the points above the surface (i.e. the object to grasp), and the white lines w/black outline is the major and minor axis of an ellipse estimated to represent the object to grasp (grasp_object attempts to grasp the object around the minor axis).
Hi @sesem378, looking at the original warning “Could not get base to fixed frame transform […] when looking up transform from frame [base_link] to frame [odom]”, I’d like to see what’s happening there. As you may know, base_link is the robot’s first frame and every other frame on the robot is defined with respect to that link. The base_link is rigidly attached to the mobile base, in between the two wheels. The odom frame, as defined by REP 105, is fixed in the world and gives you an noisy estimate of where the robot has moved in the world. It’s the laser_scan_matcher_node’s job to publish the base_link to odom transform by looking at the 2D lidar and estimating how Stretch has moved. In grasp_object.launch, lines 29-31 include that node.
Could you run the following command to see if TF2 knows about a transform between base_link and odom:
rosrun tf tf_echo base_link odom
On my robot, the output looks like:
Output
$ rosrun tf tf_echo base_link odom
At time 1689029640.313
- Translation: [0.004, 0.001, 0.000]
- Rotation: in Quaternion [0.000, 0.000, -0.002, 1.000]
in RPY (radian) [0.000, 0.000, -0.004]
in RPY (degree) [0.000, 0.000, -0.248]
At time 1689029641.038
- Translation: [0.004, 0.001, 0.000]
- Rotation: in Quaternion [0.000, 0.000, -0.002, 1.000]
in RPY (radian) [0.000, -0.000, -0.005]
in RPY (degree) [0.000, -0.000, -0.269]
At time 1689029641.995
- Translation: [0.005, 0.001, 0.000]
- Rotation: in Quaternion [0.000, 0.000, -0.003, 1.000]
in RPY (radian) [0.000, -0.000, -0.005]
in RPY (degree) [0.000, -0.000, -0.307]
Additionally, let’s verify that the laser_scan_matcher_node is alive using:
rosnode list
The output should be:
Output
$ rosnode list
/aggregator_node
/camera/realsense2_camera
/camera/realsense2_camera_manager
/configure_d435i
/d435i_accel_correction
/funmap
/grasp_object
/joint_state_publisher
/laser_filter
/laser_scan_matcher_node
/lidar_node
/robot_state_publisher
/rosout
/rviz_1689025409739790621
/stretch_driver
@bshah If it’s any help, that warning only started coming when I performed a fresh robot-level install
Thanks @sesem378, your output from rosrun tf tf_echo base_link odom indicates that not only is the odom frame missing, but also every frame is missing. For example, here’s what happens if I ask tf_echo for a fake link:
$ rosrun tf tf_echo fake_link link_mast
Failure at 1689201685.466587395
Exception thrown:"fake_link" passed to lookupTransform argument target_frame does not exist.
The current list of frames is:
Frame camera_accel_frame exists with parent camera_link.
Frame camera_link exists with parent camera_bottom_screw_frame.
Frame camera_accel_optical_frame exists with parent camera_accel_frame.
Frame camera_color_frame exists with parent camera_link.
Frame camera_color_optical_frame exists with parent camera_color_frame.
Frame camera_depth_frame exists with parent camera_link.
Frame camera_depth_optical_frame exists with parent camera_depth_frame.
Frame camera_gyro_frame exists with parent camera_link.
Frame camera_gyro_optical_frame exists with parent camera_gyro_frame.
Frame camera_infra1_frame exists with parent camera_link.
Frame camera_infra1_optical_frame exists with parent camera_infra1_frame.
Frame camera_infra2_frame exists with parent camera_link.
Frame camera_infra2_optical_frame exists with parent camera_infra2_frame.
Frame camera_bottom_screw_frame exists with parent link_head_tilt.
Frame link_head_tilt exists with parent link_head_pan.
Frame caster_link exists with parent base_link.
Frame link_arm_l4 exists with parent link_lift.
Frame link_lift exists with parent link_mast.
Frame link_aruco_inner_wrist exists with parent link_arm_l0.
Frame link_arm_l0 exists with parent link_arm_l1.
Frame link_aruco_left_base exists with parent base_link.
Frame link_aruco_right_base exists with parent base_link.
Frame link_aruco_shoulder exists with parent link_lift.
Frame link_aruco_top_wrist exists with parent link_arm_l0.
Frame base_imu exists with parent base_link.
Frame link_grasp_center exists with parent link_gripper.
Frame link_gripper exists with parent link_wrist_yaw.
Frame link_wrist_yaw exists with parent link_arm_l0.
Frame link_gripper_fingertip_left exists with parent link_gripper_finger_left.
Frame link_gripper_finger_left exists with parent link_gripper.
Frame link_gripper_fingertip_right exists with parent link_gripper_finger_right.
Frame link_gripper_finger_right exists with parent link_gripper.
Frame link_head exists with parent link_mast.
Frame link_mast exists with parent base_link.
Frame laser exists with parent base_link.
Frame respeaker_base exists with parent link_mast.
Frame link_arm_l1 exists with parent link_arm_l2.
Frame link_arm_l2 exists with parent link_arm_l3.
Frame link_arm_l3 exists with parent link_arm_l4.
Frame link_head_pan exists with parent link_head.
Frame link_left_wheel exists with parent base_link.
Frame link_right_wheel exists with parent base_link.
Notice that where my output lists the current frames (Frame <child_link> exists with parent <parent_link>), your output has an empty list. I would normally suspect a networking issue to explain why the frames are missing, however, a robot with a fresh robot-level install has defaults that prevent networking errors. Would you run the following four commands?
$ roslaunch stretch_demos grasp_object.launch #first launch the grasp_object demo as per normal
# in a separate terminal
$ rosnode list # lists the active nodes
$ rostopic list # lists the active topics
$ echo $ROS_MASTER_URI # the address for the ROS master server
I may have run the rosrun tf tf_echo base_link odom because I get a different output now. I get the same output that you showed, however the demo is still not working.
The rosrun tf tf_echo fake_link link_mast also produce the same thing as your output
echo $ROS_MASTER_URI produces http://localhost:11311
Hi @sesem378, glad to hear there wasn’t a networking error. Your outputs look alright. Just to double check, you’re aware that you have to press the " key to trigger the grasp object demo, right? As per the instructions, run roslaunch stretch_demos grasp_object.launch first, and then once the keyboard menu appears, press the key with ‘ and “ on it while in the terminal to initiate a grasp attempt.
Wow! Thank you so much for your help, and I apologize for my stupidity 
Anyway, Stretch is doing something. I have tried to pick up multiple things but it consistently fails to grasp the intended targets.
Haha! Don’t worry, it’s not stupid, it’s a very common mistake and I should have asked about that first!
Now that Stretch is doing something, the debug images I previously mentioned should be available. If you post them here, I can help you interpret them.