MMR total
1개 벤치마크 · 논문 12편 · 이 태스크의 논문 보기 →
Benchmarks
MRR-Benchmark
Most implemented
Visual Instruction Tuning
GPT-4 Technical Report
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
Papers
MMR: Evaluating Reading Ability of Large Multimodal Models
Large multimodal models (LMMs) have demonstrated impressive capabilities in understanding various types of image, including text-rich images. Most existing text-rich image benchmarks are simple extraction-based question …
Font RecognitionMMR totalOptical Character Recognition (OCR)Question Answering+2Claude 3.5 Sonnet Model Card Addendum
This addendum to our Claude 3 Model Card describes Claude 3.5 Sonnet, a new model which outperforms our previous most capable model, Claude 3 Opus, while operating faster and at a lower cost. Claude 3.5 Sonnet offers i…
Code GenerationMMR totalmodelMulti-task Language Understanding+2GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding
Animal ethology is an crucial aspect of animal research, and animal behavior labeling is the foundation for studying animal behavior. This process typically involves labeling video clips with behavioral semantic tags, a …
Activity RecognitionMMR totalSemantic correspondenceVideo Understanding+1What matters when building vision-language models?
The growing interest in vision-language models (VLMs) has been driven by improvements in large language models and vision transformers. Despite the abundance of literature on this subject, we observe that critical decisi…
1 Image, 2*2 StitchingImage RetrievalLong-Context UnderstandingMMR totalPhi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtr…
Language ModelingLanguage ModellingMathMMLU+1InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
The exponential growth of large language models (LLMs) has opened up numerous possibilities for multimodal AGI systems. However, the progress in vision and vision-language foundation models, which are also critical eleme…
Image RetrievalImage-to-Text RetrievalLanguage ModellingLarge Language Model+11