paper-with-me

홈 › Papers

DIO: Refining Mutual Information and Causal Chain to Enhance Machine Abstract Reasoning Ability

2025-08-21 · Ruizhuo Song, Beiming Yuan arxiv

Despite deep learning's broad success, its abstract-reasoning bottleneck persists. We tackle Raven's Progressive Matrices (RPM), the benchmark for pattern, reasoning and problem-solving intelligence. We model the full causal chain image $\rightarrow$ attributes $\rightarrow$ progressive patterns $\rightarrow$ consistency $\rightarrow$ answer and build the baseline DIO. Yet DIO's mutual-information lower-bound objective does not embed human logic: the bound is loose and statistic-based, ignoring causal subject-object links. We therefore present three refinements. 1) Brando introduces trainable negative options to tighten the variational bound. 2) WORLD replaces generation with a Gaussian-mixture feature model that supplies infinite, weighted negatives, further tightening the bound. 3) DIEGO adds metadata supervision to rectify the "attributes $\rightarrow$ patterns" semantic gap, aligning representations with human rules. These upgrades substantially boost discriminative RPM accuracy and, for the first time, let DIO generate valid answers in open-ended RPM. The work provides causal-driven design guidelines, objective-refinement strategies and cross-modal insights for abstract-reasoning research.

📄 PDF Abstract BibTeX arXiv:2508.15387

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

A Refining Underlying Information Framework for Monaural Speech Enhancement

2023-12-18 · Rui Cao, Tianrui Wang, Meng Ge, Longbiao Wang 외

Supervised speech enhancement has gained significantly from recent advancements in neural networks, especially due to their ability to non-linearly fit the diverse representations of target speech, such as waveform or sp…

Speech Enhancement

Redefining the Shortest Path Problem Formulation of the Linear Non-Gaussian Acyclic Model: Pairwise Likelihood Ratios, Prior Knowledge, and Path Enumeration

2024-04-18 · Hans Jarett J. Ong, Brian Godwin S. Lim, Renzo Roel P. Tan, Kazushi Ikeda

Effective causal discovery is essential for learning the causal graph from observational data. The linear non-Gaussian acyclic model (LiNGAM) operates under the assumption of a linear data generating process with non-Gau…

Causal Discovery

Dynamic Information Transfer in Stochastic Biochemical Networks

2022-08-08 · Anne-Lena Moor, Christoph Zechner

We develop numerical and analytical approaches to calculate mutual information between complete paths of two molecular components embedded into a larger reaction network. In particular, we focus on a continuous-time Mark…

StableMask: Refining Causal Masking in Decoder-only Transformer

2024-02-07 · Qingyu Yin, Xuzheng He, Xiang Zhuang, Yu Zhao 외

The decoder-only Transformer architecture with causal masking and relative position encoding (RPE) has become the de facto choice in language modeling. Despite its exceptional performance across various tasks, we have id…

DecoderLanguage ModelingLanguage ModellingPosition

Characterizing Dependence of Samples along the Langevin Dynamics and Algorithms via Contraction of $Φ$-Mutual Information

2024-02-26 · Jiaming Liang, Siddharth Mitra, Andre Wibisono

The mixing time of a Markov chain determines how fast the iterates of the Markov chain converge to the stationary distribution; however, it does not control the dependencies between samples along the Markov chain. In thi…