Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challenges
Decision-making is a dynamic process requiring perception, memory, and reasoning to make choices and find optimal policies. Traditional approaches to decision-making suffer from sample efficiency and generalization, while large-scale self-supervised pretraining has enabled fast adaptation with fine-tuning or few-shot learning in language and vision. We thus argue to integrate knowledge acquired from generic large-scale self-supervised pretraining into downstream decision-making problems. We propose Pretrain-Then-Adapt pipeline and survey recent work on data collection, pretraining objectives and adaptation strategies for decision-making pretraining and downstream inference. Finally, we identify critical challenges and future directions for developing decision foundation model with the help of generic and flexible self-supervised pretraining.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingFew-Shot LearningSimilar Papers 제목 키워드 기반
Position: Foundation Agents as the Paradigm Shift for Decision Making
Decision making demands intricate interplay between perception, memory, and reasoning to discern optimal policies. Conventional approaches to decision making face challenges related to low sample efficiency and poor gene…
Decision MakingPositionMTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining
Foundation models have reshaped the landscape of Remote Sensing (RS) by enhancing various image interpretation tasks. Pretraining is an active research topic, encompassing supervised and self-supervised learning methods …
Aerial Scene ClassificationBuilding change detection for remote sensing imagesChange DetectionChange detection for remote sensing images+13VESSA: Video-based objEct-centric Self-Supervised Adaptation for Visual Foundation Models
Foundation models have advanced computer vision by enabling strong performance across diverse tasks through large-scale pretraining and supervised fine-tuning. However, they may underperform in domains with distribution …
Self-Supervised LearningA Controlled Study of Self-Supervised Image and Video Pretraining under Limited Resources
Visual foundation models are a cornerstone of image and video understanding but typically require large amounts of data and computation. The current scale required for pretraining visual foundation models may be unsustai…
Self-Supervised LearningRepresentation LearningImage ClassificationPose EstimationStanding on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation
Pretraining for electroencephalogram (EEG) foundation models has predominantly relied on self-supervised masked reconstruction, a paradigm largely adapted from and inspired by the success of vision and language foundatio…