Video-based Generative Performance Benchmarking 벤치마크

Video-based Generative Performance Benchmarking on VideoInstruct

69개 결과 · ⬇ CSV · JSON

mean

Rank	Model	mean	Correctness of Information	Detail Orientation	Contextual Understanding	Temporal Understanding	Consistency	Paper	Code	Year
1	PPLLaVA-7B-dpo	3.73	3.85	3.56	4.21	3.21	3.81	PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance	farewellthree/ppllava	2024
2	VLM-RLAIF	3.49	3.63	3.25	4	3.23	3.32	Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback	yonseivnl/vlm-rlaif	2024
3	TS-LLaVA-34B	3.38	–	–	–	–	–	TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models	tingyu215/ts-llava	2024
4	PLLaVA-34B	3.32	3.60	3.20	3.90	2.67	3.25	PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning	magic-research/PLLaVA	2024
4	PPLLaVA-7B	3.32	3.32	3.20	3.88	3.0	3.20	PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance	farewellthree/ppllava	2024
4	SlowFast-LLaVA-34B	3.32	–	–	–	–	–	SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models	apple/ml-slowfast-llava	2024
7	VideoGPT+	3.28	3.27	3.18	3.74	2.83	3.39	VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding	mbzuai-oryx/videogpt-plus	2024
8	IG-VLM-GPT4v	3.17	3.40	2.80	3.61	2.89	3.13	An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM	imagegridworth/IG-VLM	2024
9	ST-LLM-7B	3.15	3.23	3.05	3.74	2.93	2.81	ST-LLM: Large Language Models Are Effective Temporal Learners	TencentARC/ST-LLM	2024
10	VideoChat2_HD_mistral	3.10	3.40	2.91	3.72	2.65	2.84	MVBench: A Comprehensive Multi-modal Video Understanding Benchmark	opengvlab/ask-anything · magic-research/PLLaVA · bytedance/tarsier	2023
11	CAT-7B	3.07	3.08	2.95	3.49	2.81	2.89	CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios	rikeilong/bay-cat	2024
12	LITA-13B	3.04	2.94	2.98	3.43	2.68	3.19	LITA: Language Instructed Temporal-Localization Assistant	nvlabs/lita	2024
13	LLaMA-VID-13B (2 Token)	2.99	3.07	3.05	3.60	2.58	2.63	LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models	lastmile-ai/aiconfig · dvlab-research/llama-vid	2023
13	Chat-UniVi	2.99	2.89	2.91	3.46	2.39	2.81	Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding	pku-yuangroup/chat-univi · skyworkai/moh · skyworkai/moe-plus-plus · +1	2023
15	VideoChat2	2.98	3.02	2.88	3.51	2.66	2.81	MVBench: A Comprehensive Multi-modal Video Understanding Benchmark	opengvlab/ask-anything · magic-research/PLLaVA · bytedance/tarsier	2023
16	LLaMA-VID-7B (2 Token)	2.89	2.96	3.00	3.53	2.46	2.51	LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models	lastmile-ai/aiconfig · dvlab-research/llama-vid	2023
17	VTimeLLM	2.85	2.78	3.10	3.40	2.49	2.47	VTimeLLM: Empower LLM to Grasp Video Moments	huangb23/vtimellm	2023
18	BT-Adapter	2.69	2.68	2.69	3.27	2.34	2.46	BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning	farewellthree/BT-Adapter	2023
19	BT-Adapter (zero-shot)	2.46	2.16	2.46	2.89	2.13	2.2	BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning	farewellthree/BT-Adapter	2023
20	Video-ChatGPT	2.38	2.4	2.52	2.62	1.98	2.37	Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models	mbzuai-oryx/video-chatgpt · qiujihao19/artemis	2023