The position or the shape of the object

Knowledge Base
1

Hello,

Does the shape or position of an object affect the execution of a task? For instance, when running the command python -m stretch.app.ai_pickup --use_llm to pick up a book, I notice that the gripper fails to grasp the book if it is lying flat on the table; it is only able to pick it up if the book is standing upright. How, then, can I control the movement of the gripper?

2

Hi @Mallak_Alqaisi,

Yes, the shape and position of an object can significantly affect the success of a grasp. When the book is lying flat on the table, the gripper has very limited space to get underneath it and fewer good grasp points, which makes it difficult to pick up. When the book is upright, the robot can more easily detect edges and perform a stable side grasp.

Regarding control of the gripper: when using
python -m stretch.app.ai_pickup --use_llm,
the grasping motion is automatically determined by the perception and grasp planning system, so it is not directly user-controlled.

However, you can influence the behavior in a few ways:

  • Place the object near the edge of the table or slightly tilted to improve accessibility
  • Reposition the robot to approach from a different angle
  • Try running the command multiple times, as small perception differences can affect the result

If you need more direct control over the gripper motion (for example, specifying the approach direction or grasp strategy), this would require using teleoperation tools or lower-level control APIs rather than ai_pickup.

Best,
Jason

3

Can you help me of how can i do that by using APIs ?

4

Yes, absolutely. If you want direct control over the gripper motion, you would need to use a lower-level interface instead of python -m stretch.app.ai_pickup --use_llm, since that command handles the grasp automatically.

A good option is to use the ROS 2 APIs here:

  1. Motion commands in ROS 2
    This lets you manually command individual joints:
    https://docs.hello-robot.com/0.3/ros2/jogging/
  2. Follow Joint Trajectory
    This is useful if you want to script a full grasp sequence step by step, for example: