On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline
In this paper, we examine the effectiveness of pre-training for visuo-motor control tasks. We revisit a simple Learning-from-Scratch (LfS) baseline that incorporates data augmentation and a shallow ConvNet, and find that this baseline is surprisingly competitive with recent approaches (PVR, MVP, R3M) that leverage frozen visual representations trained on large-scale vision datasets -- across a variety of algorithms, task domains, and metrics in simulation and on a real robot. Our results demonstrate that these methods are hindered by a significant domain gap between the pre-training datasets and current benchmarks for visuo-motor control, which is alleviated by finetuning. Based on our findings, we provide recommendations for future research in pre-training for control and hope that our simple yet strong baseline will aid in accurately benchmarking progress in this area.
Code (1)
Tasks
BenchmarkingData AugmentationSimilar Papers 제목 키워드 기반
Introspective Visuomotor Control: Exploiting Uncertainty in Deep Visuomotor Control for Failure Recovery
End-to-end visuomotor control is emerging as a compelling solution for robot manipulation tasks. However, imitation learning-based visuomotor control approaches tend to suffer from a common limitation, lacking the abilit…
Imitation LearningRobot ManipulationAdversarial Feature Training for Generalizable Robotic Visuomotor Control
Deep reinforcement learning (RL) has enabled training action-selection policies, end-to-end, by learning a function which maps image pixels to action outputs. However, it's application to visuomotor robotic policy traini…
Deep Reinforcement LearningReinforcement LearningReinforcement Learning (RL)Transfer LearningAdapting Pretrained ViTs with Convolution Injector for Visuo-Motor Control
Vision Transformers (ViT), when paired with large-scale pretraining, have shown remarkable performance across various computer vision tasks, primarily due to their weak inductive bias. However, while such weak inductive …
Inductive BiasEfficient Training of Generalizable Visuomotor Policies via Control-Aware Augmentation
Improving generalization is one key challenge in embodied AI, where obtaining large-scale datasets across diverse scenarios is costly. Traditional weak augmentations, such as cropping and flipping, are insufficient for i…
Data AugmentationReinforcement Learning (RL)Robot ManipulationUnified Visuomotor Targets: Supervising VLAs Beyond Physical Actions
VLA models are trained to predict robot actions from visual and language observations. This is a natural choice, but it creates a mismatch: VLMs encode rich, high-level representations of scenes and goals, while robot ac…