paper-with-me

홈 › Papers

Interpreting Models by Allowing to Ask

2018-11-13 · Sungmin Kang, David Keetae Park, Jaehyuk Chang, Jaegul Choo

Questions convey information about the questioner, namely what one does not know. In this paper, we propose a novel approach to allow a learning agent to ask what it considers as tricky to predict, in the course of producing a final output. By analyzing when and what it asks, we can make our model more transparent and interpretable. We first develop this idea to propose a general framework of deep neural networks that can ask questions, which we call asking networks. A specific architecture and training process for an asking network is proposed for the task of colorization, which is an exemplar one-to-many task and thus a task where asking questions is helpful in performing the task accurately. Our results show that the model learns to generate meaningful questions, asks difficult questions first, and utilizes the provided hint more efficiently than baseline models. We conclude that the proposed asking framework makes the learning agent reveal its weaknesses, which poses a promising new direction in developing interpretable and interactive models.

📄 PDF Abstract BibTeX arXiv:1811.05106

Code (0)

등록된 구현이 없습니다.

Tasks

Colorization

Similar Papers 제목 키워드 기반

Interpreting and learning voice commands with a Large Language Model for a robot system

2024-07-31 · Stanislau Stankevich, Wojciech Dudek

Robots are increasingly common in industry and daily life, such as in nursing homes where they can assist staff. A key challenge is developing intuitive interfaces for easy communication. The use of Large Language Models…

Decision MakingLanguage ModelingLanguage ModellingLarge Language Model

Learning Invariances for Interpretability using Supervised VAE

2020-07-15 · An-phi Nguyen, María Rodríguez Martínez

We propose to learn model invariances as a means of interpreting a model. This is motivated by a reverse engineering principle. If we understand a problem, we may introduce inductive biases in our model in the form of in…

Penzai + Treescope: A Toolkit for Interpreting, Visualizing, and Editing Models As Data

2024-08-01 · Daniel D. Johnson

Much of today's machine learning research involves interpreting, modifying or visualizing models after they are trained. I present Penzai, a neural network library designed to simplify model manipulation by representing …

Activation Scaling for Steering and Interpreting Language Models

2024-10-07 · Niklas Stoehr, Kevin Du, Vésteinn Snæbjarnarson, Robert West 외

Given the prompt "Rome is in", can we steer a language model to flip its prediction of an incorrect token "France" to a correct token "Italy" by only multiplying a few relevant activation vectors with scalars? We argue t…

Language ModelingLanguage Modelling

What do the metrics mean? A critical analysis of the use of Automated Evaluation Metrics in Interpreting

2026-01-09 · Jonathan Downie, Joss Moorkens arxiv

With the growth of interpreting technologies, from remote interpreting and Computer-Aided Interpreting to automated speech translation and interpreting avatars, there is now a high demand for ways to quickly and efficien…