Have Multimodal Large Language Models (MLLMs) Really Learned to Tell the Time on Analog Clocks?

2025-05-16 · Tairan Fu, Miguel González, Javier Conde, Elena Merino-Gómez, Pedro Reviriego

Multimodal Large Language Models which can answer complex questions on an image struggle to tell the time on analog clocks. This is probably due to the lack of images with clocks at different times in their training set. In this work we explore this issue with one of the latest MLLMs: GPT-4.1 to understand why MLLMs fail to tell the time and whether fine-tuning can solve the problem. The results show how models are making progress in reading the time on analog clocks. But have they really learned to do it, or have they only learned patterns in their training datasets? In this work we put the models to the test with different clocks to illustrate the limitations of MLLMs to abstract and generalize.

📄 PDF Abstract BibTeX arXiv:2505.10862

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Adam 설명 없음

Multi-Head Attention 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Cross-Modal Projection in Multimodal LLMs Doesn't Really Project Visual Attributes to Textual Space

2024-02-26 · Gaurav Verma, MinJe Choi, Kartik Sharma, Jamelle Watson-Daniels 외

Multimodal large language models (MLLMs) like LLaVA and GPT-4(V) enable general-purpose conversations about images with the language modality. As off-the-shelf MLLMs may have limited capabilities on images from domains l…

Language Modeling Language Modelling Large Language Model

Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?

2025-02-17 · Zichen Wen, Yifeng Gao, Weijia Li, Conghui He 외

Multimodal large language models (MLLMs) have shown remarkable performance for cross-modal understanding and generation, yet still suffer from severe inference costs. Recently, abundant works have been proposed to solve …

Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs

2026-02-09 · Siqu Ou, Tianrui Wan, Zhiyuan Zhao, Junyu Gao 외 arxiv

While chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks, existing approaches largely rely on long textual reasoning trajectories and provide l…

Reinforcement Learning Visual Reasoning

Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?

2026-07-14 · Kaiwen Zheng, Junchen Fu, Wenhao Deng, Hu Han 외 arxiv

Recent advances in multimodal large language models (MLLMs) have significantly improved the performance of multimodal emotion recognition (MER) and enabled interpretable description generation by jointly modeling video, …

Multimodal Emotion Recognition Knowledge Distillation

AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?

2024-12-03 · Kaixiong Gong, Kaituo Feng, Bohao Li, Yibing Wang 외

Recently, multimodal large language models (MLLMs), such as GPT-4o, Gemini 1.5 Pro, and Reka Core, have expanded their capabilities to include vision and audio modalities. While these models demonstrate impressive perfor…

Multiple-choice