paper-with-me

홈 › Papers

Exploring Multi-Modal Data with Tool-Augmented LLM Agents for Precise Causal Discovery

2024-12-18 · ChengAo Shen, Zhengzhang Chen, Dongsheng Luo, Dongkuan Xu, Haifeng Chen, Jingchao Ni

Causal discovery is an imperative foundation for decision-making across domains, such as smart health, AI for drug discovery and AIOps. Traditional statistical causal discovery methods, while well-established, predominantly rely on observational data and often overlook the semantic cues inherent in cause-and-effect relationships. The advent of Large Language Models (LLMs) has ushered in an affordable way of leveraging the semantic cues for knowledge-driven causal discovery, but the development of LLMs for causal discovery lags behind other areas, particularly in the exploration of multi-modal data. To bridge the gap, we introduce MATMCD, a multi-agent system powered by tool-augmented LLMs. MATMCD has two key agents: a Data Augmentation agent that retrieves and processes modality-augmented data, and a Causal Constraint agent that integrates multi-modal data for knowledge-driven reasoning. The proposed design of the inner-workings ensures successful cooperation of the agents. Our empirical study across seven datasets suggests the significant potential of multi-modality enhanced causal discovery.

📄 PDF Abstract BibTeX arXiv:2412.13667

Code (1)

D2I-Group/matmcd 공식 구현

Tasks

Causal DiscoveryCausal Inference

Similar Papers 제목 키워드 기반

Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting

2025-02-06 · Siru Zhong, Weilin Ruan, Ming Jin, Huan Li 외

Recent advancements in time series forecasting have explored augmenting models with text or vision modalities to improve accuracy. While text provides contextual understanding, it often lacks fine-grained temporal detail…

Time SeriesTime Series Forecasting

MM-StanceDet: Retrieval-Augmented Multi-modal Multi-agent Stance Detection

2026-04-30 · Weihai Lu, Zhejun Zhao, Yanshu Li, Huan He arxiv

Multimodal Stance Detection (MSD) is crucial for understanding public discourse, yet effectively fusing text and image, especially with conflicting signals, remains challenging. Existing methods often face difficulties w…

Stance Detection

AgroTools: A Benchmark for Tool-Augmented Multimodal Agents in Agriculture

2026-05-21 · Zi Ye, Yibin Wen, Xiaoya Fan, Xinyu Zhang 외 arxiv

Agricultural decision-making increasingly requires multimodal systems that can transform visual observations into reliable, executable actions. However, existing agricultural multimodal benchmarks mainly evaluate final-a…

TAMA: Tool-Augmented Multimodal Agent for Procedural Activity Understanding

2025-09-30 · Kimihiro Hasegawa, Wiradee Imrattanatrai, Masaki Asada, Ken Fukuda 외 arxiv

Procedural activity assistants potentially support humans in a variety of settings, from our daily lives, e.g., cooking or assembling flat-pack furniture, to professional situations, e.g., manufacturing or biological exp…

Multimodal Reasoning

CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detection

2022-04-01 · CVPR 2022 1 · Yanan Zhang, Jiaxin Chen, Di Huang

In autonomous driving, LiDAR point-clouds and RGB images are two major data modalities with complementary cues for 3D object detection. However, it is quite difficult to sufficiently use them, due to large inter-modal di…

3D Object DetectionAutonomous DrivingContrastive LearningData Augmentation+3