Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning
We use Bayesian optimization to learn curricula for word representation learning, optimizing performance on downstream tasks that depend on the learned representations as features. The curricula are modeled by a linear ranking function which is the scalar product of a learned weight vector and an engineered feature vector that characterizes the different aspects of the complexity of each instance in the training corpus. We show that learning the curriculum improves performance on a variety of downstream tasks over random orders and in comparison to the natural corpus order.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationRepresentation LearningSimilar Papers 제목 키워드 기반
Improving Environment Robustness of Deep Reinforcement Learning Approaches for Autonomous Racing Using Bayesian Optimization-based Curriculum Learning
Deep reinforcement learning (RL) approaches have been broadly applied to a large number of robotics tasks, such as robot manipulation and autonomous driving. However, an open problem in deep RL is learning policies that …
Autonomous DrivingAutonomous RacingBayesian OptimizationDeep Reinforcement Learning+4Dynamic curriculum learning via data parameters for noise robust keyword spotting
We propose dynamic curriculum learning via data parameters for noise robust keyword spotting. Data parameter learning has recently been introduced for image processing, where weight parameters, so-called data parameters,…
Keyword SpottingManifold Bandits: Bayesian Curriculum Learning over the Latent Geometry of Large Language Models
Reinforcement learning (RL) is a central approach for improving reasoning capabilities in large language models (LLMs), where training efficiency depends critically on how problems are sampled during optimization. Existi…
Reinforcement LearningLearning to select data for transfer learning with Bayesian Optimization
Domain similarity measures can be used to gauge adaptability and select suitable data for transfer learning, but existing approaches define ad hoc measures that are deemed suitable for respective tasks. Inspired by work …
Bayesian OptimizationDiversityPart-Of-Speech TaggingSentiment Analysis+1Understanding Curriculum Learning in Policy Optimization for Online Combinatorial Optimization
Over the recent years, reinforcement learning (RL) starts to show promising results in tackling combinatorial optimization (CO) problems, in particular when coupled with curriculum learning to facilitate training. Despit…
Combinatorial OptimizationReinforcement Learning (RL)