Any thoughts on this from deepmind?
How difficult would it be to get this running on a stretch?
1 Like
It’s really impressive work! I’d love to see more language or language + behavior models used to make planning for/interacting with Stretch better. With RT-1 and SayCan, they published checkpoints, their code, and their dataset. I imagine these would serve as a good starting point if you wanted to reproduce their work. AFAIK, they haven’t released the same with RT-2 yet.
@cpaxton has done a bit of this, so he might have some advice on how to get started. He’s one of the authors of the HomeRobot, a library that makes creating/running agents on Stretch easier.
1 Like
RT-1/2 are predicting end effector positions so in principle it could work, I think the big obstacle will be whether there is enough Stretch data to fully adapt to running it on the robot.
Would be interesting to look into though!