Scaling Law Hypothesis for Multimodal Model
We propose a scaling law hypothesis for multimodal models processing text, audio, images, and video within a shared token and embedding space. Our framework predicts model performance based on modality-specific compression and tokenization efficiency, extending established scaling laws from text-based decoder models to mixed-modality systems. We explore whether leveraging more training data in multiple modalities can reduce the size of the multimodal model, enabling efficient deployment on resource-constrained devices.
Code (0)
등록된 구현이 없습니다.
Tasks
DecodermodelSimilar Papers 제목 키워드 기반
From Zipf's Law to Neural Scaling through Heaps' Law and Hilberg's Hypothesis
We inspect the deductive connection between the neural scaling law and Zipf's law -- two statements discussed in machine learning and quantitative linguistics. The neural scaling law describes how the cross entropy rate …
Will Pre-Training Ever End? A First Step Toward Next-Generation Foundation MLLMs via Self-Improving Systematic Cognition
Recent progress in (multimodal) large language models ((M)LLMs) has shifted focus from pre-training to inference-time compute scaling and post-training optimization, driven by concerns over limited high-quality real-worl…
Caption GenerationImage CaptioningMultimodal ReasoningSelf-LearningMultiple power laws and scaling relation in exploratory locomotion of the snail Tegula nigerrima
One of goals in soft robotics is to achive spontaneous behavior like real organisms. To gain a clue to achieve this, we examined the long (16-hour) spontaneous exploratory locomotion of snails. The active forager snail, …
RelationThe Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm
The rapid proliferation of Vision-Language Models (VLMs) is often framed as enabling unified multimodal knowledge discovery but rests on an under-examined assumption: that current VLMs faithfully synthesise multimodal da…
Multimodal ReasoningVeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct
Scaling reinforcement learning for visual mathematical reasoning requires more than generating harder questions: as data volume grows, the reward labels themselves must remain reliable. Yet existing data pipelines scale …
Reinforcement LearningMathematical Reasoning