PixT3: Pixel-based Table-To-Text Generation

2023-11-16 · Iñigo Alonso, Eneko Agirre, Mirella Lapata

Table-to-text generation involves generating appropriate textual descriptions given structured tabular data. It has attracted increasing attention in recent years thanks to the popularity of neural network models and the availability of large-scale datasets. A common feature across existing methods is their treatment of the input as a string, i.e., by employing linearization techniques that do not always preserve information in the table, are verbose, and lack space efficiency. We propose to rethink data-to-text generation as a visual recognition task, removing the need for rendering the input in a string format. We present PixT3, a multimodal table-to-text model that overcomes the challenges of linearization and input size limitations encountered by existing models. PixT3 is trained with a new self-supervised learning objective to reinforce table structure awareness and is applicable to open-ended and controlled generation settings. Experiments on the ToTTo and Logic2Text benchmarks show that PixT3 is competitive and, in some settings, superior to generators that operate solely on text.

📄 PDF Abstract BibTeX arXiv:2311.09808

Code (1)

alonsoapp/pixt3 공식 구현

Tasks

Data-to-Text Generation Self-Supervised Learning Table-to-Text Generation Text Generation

Alignment Drift in Multimodal LLMs: A Two-Phase, Longitudinal Evaluation of Harm Across Eight Model Releases

2026-02-04 · Casey Ford, Madison Van Doren, Emily Dix arxiv

Multimodal large language models (MLLMs) are increasingly deployed in real-world systems, yet their safety under adversarial prompting remains underexplored. We present a two-phase evaluation of MLLM harmlessness using a…

Pixtral 12B

2024-10-09 · Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout 외

We introduce Pixtral-12B, a 12--billion-parameter multimodal language model. Pixtral-12B is trained to understand both natural images and documents, achieving leading performance on various multimodal benchmarks, surpass…

Language Modeling Language Modelling

Tracing Copied Pixels and Regularizing Patch Affinity in Copy Detection

2026-02-19 · Yichen Lu, Siwei Nie, Minlong Lu, Xudong Yang 외 arxiv

Image Copy Detection (ICD) aims to identify manipulated content between image pairs through robust feature representation learning. While self-supervised learning (SSL) has advanced ICD systems, existing view-level contr…

Self-Supervised Learning Representation Learning

PiXTime: A Model for Federated Time Series Forecasting with Heterogeneous Data across Nodes

2026-01-09 · Yiming Zhou, Jiahao Wang, Mingyue Cheng, Hao Wang 외 arxiv

While collaborative forecasting on distributed time series is highly desirable, directly pooling localized datasets is often impractical due to data sharing constraints. Federated learning offers a promising alternative,…

Time Series Forecasting Federated Learning

PixTrack: Precise 6DoF Object Pose Tracking using NeRF Templates and Feature-metric Alignment

2022-09-08 · Prajwal Chidananda, Saurabh Nair, Douglas Lee, Adrian Kaehler

We present PixTrack, a vision based object pose tracking framework using novel view synthesis and deep feature-metric alignment. We follow an SfM-based relocalization paradigm where we use a Neural Radiance Field to cano…

CPU Multi-Object Tracking NeRF Novel View Synthesis+3