Efficient Transfer Learning Schemes for Personalized Language Modeling using Recurrent Neural Network
In this paper, we propose an efficient transfer leaning methods for training a personalized language model using a recurrent neural network with long short-term memory architecture. With our proposed fast transfer learning schemes, a general language model is updated to a personalized language model with a small amount of user data and a limited computing resource. These methods are especially useful for a mobile device environment while the data is prevented from transferring out of the device for privacy purposes. Through experiments on dialogue data in a drama, it is verified that our transfer learning methods have successfully generated the personalized language model, whose output is more similar to the personal language style in both qualitative and quantitative aspects.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingTransfer LearningSimilar Papers 제목 키워드 기반
PROPS: Probabilistic personalization of black-box sequence models
We present PROPS, a lightweight transfer learning mechanism for sequential data. PROPS learns probabilistic perturbations around the predictions of one or more arbitrarily complex, pre-trained black box models (such as r…
ArticlesLanguage ModelingLanguage ModellingTransfer LearningPersonalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer
Personalized federated learning (FL) aims to train model(s) that can perform well for individual clients that are highly data and system heterogeneous. Most work in personalized FL, however, assumes using the same model …
Federated LearningPersonalized Federated LearningTransfer LearningNormalizing the Normalizers: Comparing and Extending Network Normalization Schemes
Normalization techniques have only recently begun to be exploited in supervised learning tasks. Batch normalization exploits mini-batch statistics to normalize the activations. This was shown to speed up training and res…
image-classificationImage ClassificationLanguage ModelingLanguage Modelling+1Personalized Federated Learning over non-IID Data for Indoor Localization
Localization and tracking of objects using data-driven methods is a popular topic due to the complexity in characterizing the physics of wireless channel propagation models. In these modeling approaches, data needs to be…
Federated LearningIndoor LocalizationPersonalized Federated LearningShort-Term Forecasting of Thermostatic and Residential Loads Using Long Short-Term Memory Recurrent Neural Networks
Internet of Things (IoT) devices in smart grids enable intelligent energy management for grid managers and personalized energy services for consumers. Investigating a smart grid with IoT devices requires a simulation fra…
energy managementLoad ForecastingManagement