홈 › Papers

Building Optimal Neural Architectures using Interpretable Knowledge

2024-03-20 · CVPR 2024 1 · Keith G. Mills, Fred X. Han, Mohammad Salameh, Shengyao Lu, Chunhua Zhou, Jiao He, Fengyu Sun, Di Niu

Neural Architecture Search is a costly practice. The fact that a search space can span a vast number of design choices with each architecture evaluation taking nontrivial overhead makes it hard for an algorithm to sufficiently explore candidate networks. In this paper, we propose AutoBuild, a scheme which learns to align the latent embeddings of operations and architecture modules with the ground-truth performance of the architectures they appear in. By doing so, AutoBuild is capable of assigning interpretable importance scores to architecture modules, such as individual operation features and larger macro operation sequences such that high-performance neural networks can be constructed without any need for search. Through experiments performed on state-of-the-art image classification, segmentation, and Stable Diffusion models, we show that by mining a relatively small set of evaluated architectures, AutoBuild can learn to build high-quality architectures directly or help to reduce search space to focus on relevant areas, finding better architectures that outperform both the original labeled ones and ones found by search baselines. Code available at https://github.com/Ascend-Research/AutoBuild

📄 PDF Abstract BibTeX arXiv:2403.13293

Code (1)

ascend-research/autobuild 공식 구현 pytorch

Tasks

image-classification Image Classification Neural Architecture Search

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Focus 설명 없음

ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

Learning unfolded networks with a cyclic group structure

2022-11-16 · Emmanouil Theodosis, Demba Ba

Deep neural networks lack straightforward ways to incorporate domain knowledge and are notoriously considered black boxes. Prior works attempted to inject domain knowledge into architectures implicitly through data augme…

Data Augmentation Rotated MNIST

Localizing Knowledge in Diffusion Transformers

2025-05-24 · Arman Zarei, Samyadeep Basu, Keivan Rezaei, Zihao Lin 외

Understanding how knowledge is distributed across the layers of generative models is crucial for improving interpretability, controllability, and adaptation. While prior work has explored knowledge localization in UNet-b…

Model Editing

Building a Knowledge Graph from Natural Language Definitions for Interpretable Text Entailment Recognition

2018-05-01 · LREC 2018 5 · Vivian Silva, Andr{\'e} Freitas, H, Siegfried schuh

Information Retrieval Open Information Extraction Question Answering

Combining Transformers with Natural Language Explanations

2021-09-02 · Federico Ruggeri, Marco Lippi, Paolo Torroni

Many NLP applications require models to be interpretable. However, many successful neural architectures, including transformers, still lack effective interpretation methods. A possible solution could rely on building exp…

Argument Mining

LCEN: A Novel Feature Selection Algorithm for Nonlinear, Interpretable Machine Learning Models

2024-02-27 · Pedro Seber, Richard D. Braatz

Interpretable architectures can have advantages over black-box architectures, and interpretability is essential for the application of machine learning in critical settings, such as aviation or medicine. However, the sim…

feature selection Interpretable Machine Learning