Coherent and Consistent Relational Transfer Learning with Autoencoders
Human defined concepts are inherently transferable, but it is not clear under what conditions they can be modelled effectively by non-symbolic artificial learners. This paper argues that for a transferable concept to be learned, the system of relations that define it must be coherent across domains. This is to say that the learned concept-specific relations ought to be consistent with respect to a theory that constrains their semantics and that such consistency must extend beyond the representations encountered in the source domain. To demonstrate this, we first present formal definitions for consistency and coherence, and a proposed Dynamic Comparator relation-decoder model designed around these principles. We then perform a proposed Partial Relation Transfer learning task on a novel data set, using a neural-symbolic autoencoder architecture that combines sub-symbolic representations with modular relation-decoders. By comparing against several existing relation-decoder models, our experiments show that relation-decoders which maintain consistency over unobserved regions of representational space retain coherence across domains, whilst achieving better transfer learning performance.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderRelationTransfer LearningSimilar Papers 제목 키워드 기반
A Universal Vibe? Finding and Controlling Language-Agnostic Informal Register with SAEs
While multilingual language models successfully transfer factual and syntactic knowledge across languages, it remains unclear whether they process culture-specific pragmatic registers, such as slang, as isolated language…
Learning Autoencoders with Relational Regularization
A new algorithmic framework is proposed for learning autoencoders of data distributions. We minimize the discrepancy between the model and target distributions, with a \emph{relational regularization} on the learnable la…
MULTI-VIEW LEARNINGCycle-Consistent Adversarial Autoencoders for Unsupervised Text Style Transfer
Unsupervised text style transfer is full of challenges due to the lack of parallel data and difficulties in content preservation. In this paper, we propose a novel neural approach to unsupervised text style transfer, whi…
Style TransferText Style TransferUnsupervised Text Style TransferEducating Text Autoencoders: Latent Representation Guidance via Denoising
Generative autoencoders offer a promising approach for controllable text generation by leveraging their latent sentence representations. However, current models struggle to maintain coherent latent spaces required to per…
DenoisingSentenceStyle TransferText Generation+1Attribution-Guided Distillation of Matryoshka Sparse Autoencoders
Sparse autoencoders (SAEs) aim to disentangle model activations into monosemantic, human-interpretable features. In practice, learned features are often redundant and vary across training runs and sparsity levels, which …