3D-DLP Learns Self-Supervised Object-Centric 3D Scene Representations

Understanding and representing complex 3D scenes is a fundamental challenge in AI, particularly for robotics. This research introduces 3D-DLP (3D Deep Latent Particles), a self-supervised object-centric representation learning model. It is designed to decompose scene-level RGB-D or voxel observations into a collection of 3D latent particles. Each of these particles encodes disentangled attributes, such as 3D keypoint position, bounding box dimensions, and appearance features, effectively representing a distinct entity within the scene. The model learns interpretable per-particle segmentation maps through an end-to-end self-supervised reconstruction objective, meaning it learns without explicit human labels for objects. Demonstrations on both simulated and real-world datasets confirm that the learned latent space is highly interpretable and controllable. By manipulating the positions of these particles and then decoding them, novel scene configurations can be generated. Furthermore, leveraging these compact 3D latent particles for downstream robotic manipulation tasks significantly improves performance compared to baseline methods that either lack explicit 3D information or rely on memory-intensive dense 3D inputs without object-centric structure.