Local Neural Descriptor Fields:
Locally Conditioned Object Representations for Manipulation

Ethan Chun, Yilun Du, Anthony Simeonov, Tomás Lozano Peréz, Leslie Pack Kaelbling


A robot operating in a household environment will see a wide range of different objects. In this paper, we present a method to generalize object manipulation skills, acquired from a limited number of demonstrations, to novel objects of new categories of shapes. Our approach, Local Neural Descriptor Fields (L-NDF), utilizes neural descriptors defined on the local geometry of the object to effectively transfer manipulation demonstrations to novel objects at test time, leveraging the shared local geometry of novel objects. We illustrate the efficacy of our approach in manipulating novel objects in novel poses in both simulation as well as in the real world.

Thanks for Reading!!!