Efficient Sampled Softmax for Tensorflow
This short paper discusses an efficient implementation of \emph{sampled softmax loss} for Tensorflow. The speedup over the default implementation is achieved due to simplification of the graph for the forward and backward passes.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On the Effectiveness of Sampled Softmax Loss for Item Recommendation
The learning objective plays a fundamental role to build a recommender system. Most methods routinely adopt either pointwise or pairwise loss to train the model parameters, while rarely pay attention to softmax loss due …
Contrastive LearningRecommendation SystemsSelf-Supervised LearningSoftmax Optimizations for Intel Xeon Processor-based Platforms
Softmax is popular normalization method used in machine learning. Deep learning solutions like Transformer or BERT use the softmax function intensively, so it is worthwhile to optimize its performance. This article prese…
BIG-bench Machine LearningDeep LearningSampled Softmax with Random Fourier Features
The computational cost of training with softmax cross entropy loss grows linearly with the number of classes. For the settings where a large number of classes are involved, a common method to speed up training is to samp…
TensorBNN: Bayesian Inference for Neural Networks using Tensorflow
TensorBNN is a new package based on TensorFlow that implements Bayesian inference for modern neural network models. The posterior density of neural network model parameters is represented as a point cloud sampled using H…
Bayesian InferenceGPUMemory-Efficient Training of RNN-Transducer with Sampled Softmax
RNN-Transducer has been one of promising architectures for end-to-end automatic speech recognition. Although RNN-Transducer has many advantages including its strong accuracy and streaming-friendly property, its high memo…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition