Scene Understanding for Autonomous Manipulation with Deep Learning
Over the past few years, deep learning techniques have achieved tremendous success in many visual understanding tasks such as object detection, image segmentation, and caption generation. Despite this thriving in computer vision and natural language processing, deep learning has not yet shown significant impact in robotics. Due to the gap between theory and application, there are many challenges when applying the results of deep learning to the real robotic systems. In this study, our long-term goal is to bridge the gap between computer vision and robotics by developing visual methods that can be used in real robots. In particular, this work tackles two fundamental visual problems for autonomous robotic manipulation: affordance detection and fine-grained action understanding. Theoretically, we propose different deep architectures to further improves the state of the art in each problem. Empirically, we show that the outcomes of our proposed methods can be applied in real robots and allow them to perform useful manipulation tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Action UnderstandingAffordance DetectionCaption GenerationDeep LearningImage Segmentationobject-detectionObject DetectionScene UnderstandingSemantic SegmentationSimilar Papers 제목 키워드 기반
INTENTION: Inferring Tendencies of Humanoid Robot Motion Through Interactive Intuition and Grounded VLM
Traditional control and planning for robotic manipulation heavily rely on precise physical models and predefined action sequences. While effective in structured environments, such approaches often fail in real-world scen…
Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?
Task specification is at the core of programming autonomous robots. A low-effort modality for task specification is critical for engagement of non-expert end-users and ultimate adoption of personalized robot agents. A wi…
Robot ManipulationScene UnderstandingState EstimationArticulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation
A kitchen assistant needs to operate human-scale objects, such as cabinets and ovens, in unmapped environments with dynamic obstacles. Autonomous interactions in such environments require integrating dexterous manipulati…
ObjectGeneralizable Humanoid Manipulation with 3D Diffusion Policies
Humanoid robots capable of autonomous operation in diverse environments have long been a goal for roboticists. However, autonomous manipulation by humanoid robots has largely been restricted to one specific scene, primar…
Camera CalibrationPoint Cloud SegmentationMulti-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving
Efficient data utilization is crucial for advancing 3D scene understanding in autonomous driving, where reliance on heavily human-annotated LiDAR point clouds challenges fully supervised methods. Addressing this, our stu…
Autonomous DrivingLIDAR Semantic SegmentationScene UnderstandingSemantic Segmentation