I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures
Modularity is a paradigm of machine translation with the potential of bringing forth models that are large at training time and small during inference. Within this field of study, modular approaches, and in particular attention bridges, have been argued to improve the generalization capabilities of models by fostering language-independent representations. In the present paper, we study whether modularity affects translation quality; as well as how well modular architectures generalize across different evaluation scenarios. For a given computational budget, we find non-modular architectures to be always comparable or preferable to all modular designs we study.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
How Far Can LLMs Emulate Human Behavior?: A Strategic Analysis via the Buy-and-Sell Negotiation Game
With the rapid advancement of Large Language Models (LLMs), recent studies have drawn attention to their potential for handling not only simple question-answer tasks but also more complex conversational abilities and per…
Demand Prediction Using Machine Learning Methods and Stacked Generalization
Supply and demand are two fundamental concepts of sellers and customers. Predicting demand accurately is critical for organizations in order to be able to make plans. In this paper, we propose a new approach for demand p…
BIG-bench Machine LearningEnsemble LearningSeller-buyer networks in NFT art are driven by preferential ties
Non-Fungible Tokens (NFTs) have recently surged to mainstream attention by allowing the exchange of digital assets via blockchains. NFTs have also been adopted by artists to sell digital art. One of the promises of NFTs …
Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Space
Recent advances in operator learning have produced two distinct approaches for solving partial differential equations (PDEs): attention-based methods offering point-level adaptability but lacking spectral constraints, an…
Computational EfficiencyOperator learningPoseLLM: Enhancing Language-Guided Human Pose Estimation with MLP Alignment
Human pose estimation traditionally relies on architectures that encode keypoint priors, limiting their generalization to novel poses or unseen keypoints. Recent language-guided approaches like LocLLM reformulate keypoin…
Large Language ModelPose EstimationZero-shot Generalization