MixMAS: A Framework for Sampling-Based Mixer Architecture Search for Multimodal Fusion and Learning
Choosing a suitable deep learning architecture for multimodal data fusion is a challenging task, as it requires the effective integration and processing of diverse data types, each with distinct structures and characteristics. In this paper, we introduce MixMAS, a novel framework for sampling-based mixer architecture search tailored to multimodal learning. Our approach automatically selects the optimal MLP-based architecture for a given multimodal machine learning (MML) task. Specifically, MixMAS utilizes a sampling-based micro-benchmarking strategy to explore various combinations of modality-specific encoders, fusion functions, and fusion networks, systematically identifying the architecture that best meets the task's performance metrics.
Code (1)
Tasks
BenchmarkingSimilar Papers 제목 키워드 기반
MixMask: Revisiting Masking Strategy for Siamese ConvNets
The recent progress in self-supervised learning has successfully combined Masked Image Modeling (MIM) with Siamese Networks, harnessing the strengths of both methodologies. Nonetheless, certain challenges persist when in…
object-detectionObject DetectionSelf-Supervised LearningMixerFlow: MLP-Mixer meets Normalising Flows
Normalising flows are generative models that transform a complex density into a simpler density through the use of bijective transformations enabling both density estimation and data generation from a single model. %Howe…
Density EstimationKolmogorov-Arnold NetworksNormalising FlowsJustDense: Just using Dense instead of Sequence Mixer for Time Series analysis
Sequence and channel mixers, the core mechanism in sequence models, have become the de facto standard in time series analysis (TSA). However, recent studies have questioned the necessity of complex sequence mixers, such …
Time Series AnalysisTSMixer: An All-MLP Architecture for Time Series Forecasting
Real-world time-series datasets are often multivariate with complex dynamics. To capture this complexity, high capacity architectures like recurrent- or attention-based sequential deep learning models have become popular…
AllDeep LearningTime SeriesTime Series Analysis+1Boosting Adversarial Transferability of MLP-Mixer
The security of models based on new architectures such as MLP-Mixer and ViTs needs to be studied urgently. However, most of the current researches are mainly aimed at the adversarial attack against ViTs, and there is sti…
Adversarial Attack