Learning to Drive from Simulation without Real World Labels
Simulation can be a powerful tool for understanding machine learning systems and designing methods to solve real-world problems. Training and evaluating methods purely in simulation is often "doomed to succeed" at the desired task in a simulated environment, but the resulting models are incapable of operation in the real world. Here we present and evaluate a method for transferring a vision-based lane following driving policy from simulation to operation on a rural road without any real-world labels. Our approach leverages recent advances in image-to-image translation to achieve domain transfer while jointly learning a single-camera control policy from simulation control labels. We assess the driving performance of this method using both open-loop regression metrics, and closed-loop performance operating an autonomous vehicle on rural and urban roads.
Code (0)
등록된 구현이 없습니다.
Tasks
Image-to-Image TranslationTranslationSimilar Papers 제목 키워드 기반
Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
Driven by steady progress in deep generative modeling, simulation-based inference (SBI) has emerged as the workhorse for inferring the parameters of stochastic simulators. However, recent work has demonstrated that model…
Information-Preserving Domain Transfer with Unlabeled Data in Misspecified Simulation-Based Inference
Simulation-based inference (SBI) provides amortized Bayesian parameter inference from simulator-generated data without requiring explicit likelihood evaluation. Its reliability can degrade under model misspecification, w…
Boosting Multi-View Stereo with Depth Foundation Model in the Absence of Real-World Labels
Learning-based Multi-View Stereo (MVS) methods have made remarkable progress in recent years. However, how to effectively train the network without using real-world labels remains a challenging problem. In this paper, dr…
SimSort: A Data-Driven Framework for Spike Sorting by Large-Scale Electrophysiology Simulation
Spike sorting is an essential process in neural recording, which identifies and separates electrical signals from individual neurons recorded by electrodes in the brain, enabling researchers to study how specific neurons…
Spike SortingZero-shot GeneralizationLearning to Drive from a World Model
Most self-driving systems rely on hand-coded perception outputs and engineered driving rules. Learning directly from human driving data with an end-to-end method can allow for a training architecture that is simpler and …
model