SE3-Nets: Learning Rigid Body Motion using Deep Neural Networks
We introduce SE3-Nets, which are deep neural networks designed to model and learn rigid body motion from raw point cloud data. Based only on sequences of depth images along with action vectors and point wise data associations, SE3-Nets learn to segment effected object parts and predict their motion resulting from the applied force. Rather than learning point wise flow vectors, SE3-Nets predict SE3 transformations for different parts of the scene. Using simulated depth data of a table top scene and a robot manipulator, we show that the structure underlying SE3-Nets enables them to generate a far more consistent prediction of object motion than traditional flow based networks. Additional experiments with a depth camera observing a Baxter robot pushing objects on a table show that SE3-Nets also work well on real data.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Locally-symplectic neural networks for learning volume-preserving dynamics
We propose locally-symplectic neural networks LocSympNets for learning the flow of phase volume-preserving dynamics. The construction of LocSympNets stems from the theorem of the local Hamiltonian description of the dive…
RRV: A Spatiotemporal Descriptor for Rigid Body Motion Recognition
Motion behaviors of a rigid body can be characterized by a 6-dimensional motion trajectory, which contains position vectors of a reference point on the rigid body and rotations of this rigid body over time. This paper de…
DescriptivePositionDistributed Collision-Free Motion Coordination on a Sphere: A Conic Control Barrier Function Approach
This letter studies a distributed collision avoidance control problem for a group of rigid bodies on a sphere. A rigid body network, consisting of multiple rigid bodies constrained to a spherical surface and an interconn…
Collision AvoidanceDistributed OptimizationMulti-body Non-rigid Structure-from-Motion
Conventional structure-from-motion (SFM) research is primarily concerned with the 3D reconstruction of a single, rigidly moving object seen by a static camera, or a static and rigid scene observed by a moving camera --in…
3D ReconstructionClusteringMultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization
We present MultiBodySync, a novel, end-to-end trainable multi-body motion segmentation and rigid registration framework for multiple input 3D point clouds. The two non-trivial challenges posed by this multi-scan multibod…
Motion EstimationMotion SegmentationSegmentation