# Object detection with the stretch robot **URL:** \ **Category:** Ask\ **Created:** [May 18, 2021, 7:41pm UTC](https://forum.hello-robot.com/t/object-detection-with-the-stretch-robot/171 "2021-05-18T19:41:40Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Robot\_Lover](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@Robot\_Lover](https://forum.hello-robot.com/u/Robot_Lover)\ **Post date:** [May 18, 2021, 7:41pm UTC](https://forum.hello-robot.com/t/object-detection-with-the-stretch-robot/171/1 "2021-05-18T19:41:40Z")
Hi! I’m wondering if the hello robot team/ROS has anything written that we can use for object detection, either for while the robot is stationary or for when it is moving (although I also understand that object detection/avoidance is part of stretch\_navigation already as the robot will avoid obstacles while navigating?) ---
**Author:** ![bshah](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/bshah/32/55_2.png) [@bshah](https://forum.hello-robot.com/u/bshah)\ **Post date:** [May 21, 2021, 4:41am UTC](https://forum.hello-robot.com/t/object-detection-with-the-stretch-robot/171/2 "2021-05-21T04:41:31Z")
Hi @Robot_Lover, if you are looking for classification labels of objects in the environment, Stretch ships with [stretch\_deep\_perception](https://github.com/hello-robot/stretch_ros/tree/master/stretch_deep_perception), a ROS package with neural networks that can detect and classify objects that the Realsense camera sees. You can find more information in the [README](https://github.com/hello-robot/stretch_ros/blob/master/stretch_deep_perception/README.md). If you are looking for object detection for mapping and path planning, the ROS standard is the navigation stack, which is what stretch\_navigation is built upon. Additionally, we have developed [stretch\_funmap](https://github.com/hello-robot/stretch_ros/tree/master/stretch_funmap), which utilizes the Realsense camera and other sensors on Stretch for object detection, navigation, and to form a depth map of its environment. Please review the [README](https://github.com/hello-robot/stretch_ros/blob/master/stretch_funmap/README.md) for instructions on how to use the package. ---
**Author:** ![Robot\_Lover](https://avatars.discourse-cdn.com/v4/letter/r/e9a140/32.png) [@Robot\_Lover](https://forum.hello-robot.com/u/Robot_Lover)\ **Post date:** [May 24, 2021, 5:02am UTC](https://forum.hello-robot.com/t/object-detection-with-the-stretch-robot/171/3 "2021-05-24T05:02:51Z")
Hi Binit, Thanks for the response! I was trying out stretch\_funmap but I ran into a few errors regarding navigation (the step post head-scan) First, when I press the key that contains \ and |, I get the followiing error ```auto /usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:3118: RuntimeWarning: Mean of empty slice. out=out, **kwargs) /usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py:85: RuntimeWarning: invalid value encountered in double_scalars ret = ret.dtype.type(ret / rcount) median_floor_height_m = nan [ERROR] [1621829821.082892]: Error processing request: Expected Ptr for argument 'img' ['Traceback (most recent call last):\n', ' File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 632, in _handle_request\n response = convert_return_to_response(self.handler(request), self.response_class)\n', ' File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/nodes/funmap", line 503, in trigger_drive_to_scan_service_callback\n display_on=False)\n', ' File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/src/stretch_funmap/navigation_planning.py", line 305, in select_next_scan_location\n display_on=display_on, verbose=True )\n', ' File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/src/stretch_funmap/navigation_planning.py", line 742, in distance_map_simple\n draw_robot_footprint_rectangle(robot_x_pix, robot_y_pix, robot_ang_rad, m_per_pix, footprint_test_image)\n', ' File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/src/stretch_funmap/navigation_planning.py", line 583, in draw_robot_footprint_rectangle\n cv2.fillConvexPoly(image, poly_points, value)\n', "TypeError: Expected Ptr for argument 'img'\n"] Traceback (most recent call last): File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/keyboard_teleop", line 408, in node.main() File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/keyboard_teleop", line 383, in main command = self.keys.get_command(self) File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/keyboard_teleop", line 127, in get_command trigger_result = node.trigger_drive_to_scan_service(trigger_request) File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 442, in __call__ return self.call(*args, **kwds) File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 522, in call responses = transport.receive_once() File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 735, in receive_once p.read_messages(b, msg_queue, sock) File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 360, in read_messages self._read_ok_byte(b, sock) File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 343, in _read_ok_byte raise ServiceException("service [%s] responded with an error: %s"%(self.resolved_name, str)) rospy.service.ServiceException: service [/funmap/trigger_drive_to_scan] responded with an error: error processing request: Expected Ptr for argument 'img' [keyboard_teleop-14] process has died [pid 21635, exit code 1, cmd /home/hello-robot/catkin_ws/src/stretch_ros/stretch_core/nodes/keyboard_teleop --mapping_on __name:=keyboard_teleop__ log:=/home/hello-robot/.ros/log/953d761c-bc46-11eb-b37f-94e6f7b7f4ad/keyboard_teleop-14.log]. log file: /home/hello-robot/.ros/log/953d761c-bc46-11eb-b37f-94e6f7b7f4ad/keyboard_teleop-14*.log [WARN] [1621829823.075188501]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead. [WARN] [1621829823.915207350]: No stream match for pointcloud chosen texture Process - Color [WARN] [1621829825.114444227]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead. [WARN] [1621829827.114701276]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead. [WARN] [1621829829.115639899]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead. ``` Then, when I try using navigation with the publish point feature on Rviz, I get this error ```auto transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 reach_width_pix = 26.6666666667 kernel_width = 27 num_angles = 1600 pix_per_m = 166.666666667 max_arm_travel_m = 0.5 No valid base pose candidate found. [ERROR] [1621830188.289121]: No valid base pose found for reaching the target. [ERROR] [1621830188.290540]: bad callback: > Traceback (most recent call last): File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callback cb(msg) File "/home/hello-robot/catkin_ws/src/stretch_ros/stretch_funmap/nodes/funmap", line 692, in reach_to_click_callback success, message = self.navigate_to_map_pixel(robot_reach_xya_pix[:2], TypeError: 'NoneType' object has no attribute ' __getitem__' ``` I tried publishing to points close and far from the robot, as well as points in all different color ranges, but nothing seemed to work. Finally, when I used 2d navigation in rviz, I get this error ```auto [INFO] [1621830252.017203]: Set D435i to Default mode [INFO] [1621830252.019269]: trigger_result = success: True message: "Default mode enabled." [INFO] [1621830252.036754]: /stretch_driver joint_traj action: New trajectory received with joint_names = ['joint_head_tilt', 'joint_head_pan'] [INFO] [1621830252.094665]: /stretch_driver joint_traj action: Achieved all target points. median_floor_height_m = nan [ERROR] [1621830252.143274]: Failed to create distance map and traversable mask. [WARN] [1621830253.436278187]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead. [WARN] [1621830255.474910707]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead. ``` I would greatly appreciate response on any of these so we could try and get stretch\_funmap working past this step. For reference, this is another message that keeps getting displayed on the terminal continuously post head-scan, in case this is helpful. ```auto 23/05 21:27:30,291 WARNING [139626522580736] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11 [WARN] [1621830451.280697814]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead. [WARN] [1621830453.281077018]: Transform from imu_mobile_base to base_link was unavailable for the time requested. Using latest instead. ``` ---
**Author:** ![bshah](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/bshah/32/55_2.png) [@bshah](https://forum.hello-robot.com/u/bshah)\ **Post date:** [May 26, 2021, 4:37am UTC](https://forum.hello-robot.com/t/object-detection-with-the-stretch-robot/171/4 "2021-05-26T04:37:24Z")
Hi @Robot_Lover, thanks for the logs. Trying out the same steps you’ve run through, I don’t see the same errors that appear in the logs. It would be helpful to run through these steps over a video call to debug the issue live. I have sent you a message to set up a meeting. ---
**Author:** ![AswinGururaj](https://avatars.discourse-cdn.com/v4/letter/a/f6c823/32.png) [@AswinGururaj](https://forum.hello-robot.com/u/AswinGururaj)\ **Post date:** [September 30, 2021, 10:46pm UTC](https://forum.hello-robot.com/t/object-detection-with-the-stretch-robot/171/5 "2021-09-30T22:46:28Z")
Hi @bshah. I am also facing the same issues while trying out stretch\_funmap. Could you please let me know how I can solve this? Thanks! ---
**Author:** ![bshah](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.hello-robot.com/bshah/32/55_2.png) [@bshah](https://forum.hello-robot.com/u/bshah)\ **Post date:** [October 1, 2021, 6:18am UTC](https://forum.hello-robot.com/t/object-detection-with-the-stretch-robot/171/6 "2021-10-01T06:18:26Z")
Hi @AswinGururaj, welcome to the forum! Would you mind starting a [new post](https://forum.hello-robot.com/c/ask/5) and including the error logs you’re seeing?