Hi! I’m wondering if the hello robot team/ROS has anything written that we can use for object detection, either for while the robot is stationary or for when it is moving (although I also understand that object detection/avoidance is part of stretch_navigation already as the robot will avoid obstacles while navigating?)
Hi @Robot_Lover, if you are looking for classification labels of objects in the environment, Stretch ships with stretch_deep_perception, a ROS package with neural networks that can detect and classify objects that the Realsense camera sees. You can find more information in the README.
If you are looking for object detection for mapping and path planning, the ROS standard is the navigation stack, which is what stretch_navigation is built upon. Additionally, we have developed stretch_funmap, which utilizes the Realsense camera and other sensors on Stretch for object detection, navigation, and to form a depth map of its environment. Please review the README for instructions on how to use the package.
Hi Binit,
Thanks for the response! I was trying out stretch_funmap but I ran into a few errors regarding navigation (the step post head-scan)
First, when I press the key that contains \ and |, I get the followiing error
/usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:3118: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py:85: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
median_floor_height_m = nan
[ERROR] [1621829821.082892]: Error processing request: Expected Ptr<cv::UMat> for argument 'img'
['Traceback (most recent call last):\n', ' File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 632, in _handle_request\n response = convert_return_to_response(self.handler(request), self.response_class)\n', ' File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/nodes/funmap", line 503, in trigger_drive_to_scan_service_callback\n display_on=False)\n', ' File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/src/stretch_funmap/navigation_planning.py", line 305, in select_next_scan_location\n display_on=display_on, verbose=True )\n', ' File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/src/stretch_funmap/navigation_planning.py", line 742, in distance_map_simple\n draw_robot_footprint_rectangle(robot_x_pix, robot_y_pix, robot_ang_rad, m_per_pix, footprint_test_image)\n', ' File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/src/stretch_funmap/navigation_planning.py", line 583, in draw_robot_footprint_rectangle\n cv2.fillConvexPoly(image, poly_points, value)\n', "TypeError: Expected Ptr<cv::UMat> for argument 'img'\n"]
Traceback (most recent call last):
File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/keyboard_teleop", line 408, in <module>
node.main()
File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/keyboard_teleop", line 383, in main
command = self.keys.get_command(self)
File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/keyboard_teleop", line 127, in get_command
trigger_result = node.trigger_drive_to_scan_service(trigger_request)
File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 442, in __call__
return self.call(*args, **kwds)
File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 522, in call
responses = transport.receive_once()
File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 735, in receive_once
p.read_messages(b, msg_queue, sock)
File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 360, in read_messages
self._read_ok_byte(b, sock)
File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 343, in _read_ok_byte
raise ServiceException("service [%s] responded with an error: %s"%(self.resolved_name, str))
rospy.service.ServiceException: service [/funmap/trigger_drive_to_scan] responded with an error: error processing request: Expected Ptr<cv::UMat> for argument 'img'
[keyboard_teleop-14] process has died [pid 21635, exit code 1, cmd /home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/keyboard_teleop --mapping_on __name:=keyboard_teleop __log:=/home/hello-robot/.ros/log/953d761c-bc46-11eb-b37f-94e6f7b7f4ad/keyboard_teleop-14.log].
log file: /home/hello-robot/.ros/log/953d761c-bc46-11eb-b37f-94e6f7b7f4ad/keyboard_teleop-14*.log
[ WARN] [1621829823.075188501]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead.
[ WARN] [1621829823.915207350]: No stream match for pointcloud chosen texture Process - Color
[ WARN] [1621829825.114444227]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead.
[ WARN] [1621829827.114701276]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead.
[ WARN] [1621829829.115639899]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead.
Then, when I try using navigation with the publish point feature on Rviz, I get this error
transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
reach_width_pix = 26.6666666667
kernel_width = 27
num_angles = 1600
pix_per_m = 166.666666667
max_arm_travel_m = 0.5
No valid base pose candidate found.
[ERROR] [1621830188.289121]: No valid base pose found for reaching the target.
[ERROR] [1621830188.290540]: bad callback: <bound method FunmapNode.reach_to_click_callback of <__main__.FunmapNode instance at 0x7fba7dd36e10>>
Traceback (most recent call last):
File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/nodes/funmap", line 692, in reach_to_click_callback
success, message = self.navigate_to_map_pixel(robot_reach_xya_pix[:2],
TypeError: 'NoneType' object has no attribute '__getitem__'
I tried publishing to points close and far from the robot, as well as points in all different color ranges, but nothing seemed to work.
Finally, when I used 2d navigation in rviz, I get this error
[INFO] [1621830252.017203]: Set D435i to Default mode
[INFO] [1621830252.019269]: trigger_result = success: True
message: "Default mode enabled."
[INFO] [1621830252.036754]: /stretch_driver joint_traj action: New trajectory received with joint_names = ['joint_head_tilt', 'joint_head_pan']
[INFO] [1621830252.094665]: /stretch_driver joint_traj action: Achieved all target points.
median_floor_height_m = nan
[ERROR] [1621830252.143274]: Failed to create distance map and traversable mask.
[ WARN] [1621830253.436278187]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead.
[ WARN] [1621830255.474910707]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead.
I would greatly appreciate response on any of these so we could try and get stretch_funmap working past this step. For reference, this is another message that keeps getting displayed on the terminal continuously post head-scan, in case this is helpful.
23/05 21:27:30,291 WARNING [139626522580736] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
[ WARN] [1621830451.280697814]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead.
[ WARN] [1621830453.281077018]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead.
Hi @Robot_Lover, thanks for the logs. Trying out the same steps you’ve run through, I don’t see the same errors that appear in the logs. It would be helpful to run through these steps over a video call to debug the issue live. I have sent you a message to set up a meeting.
Hi @bshah. I am also facing the same issues while trying out stretch_funmap. Could you please let me know how I can solve this?
Thanks!
Hi @AswinGururaj, welcome to the forum! Would you mind starting a new post and including the error logs you’re seeing?
1 Like