Benchmarking down-scaled (not so large) pre-trained language models
Code (1)
Tasks
BenchmarkingSimilar Papers 제목 키워드 기반
Benchmarking down-scaled (not so large) pre-trained language models
Large Transformer-based language models are pre-trained on corpora of varying sizes, for a different number of steps and with different batch sizes. At the same time, more fundamental components, such as the pre-training…
BenchmarkingWeight subcloning: direct initialization of transformers using larger pretrained ones
Training large transformer models from scratch for a target task requires lots of data and is computationally demanding. The usual practice of transfer learning overcomes this challenge by initializing the model with wei…
image-classificationImage ClassificationTransfer LearningScaleDet: A Scalable Multi-Dataset Object Detector
Multi-dataset training provides a viable solution for exploiting heterogeneous large-scale datasets without extra annotation cost. In this work, we propose a scalable multi-dataset detector (ScaleDet) that can scale up i…
Objectobject-detectionObject DetectionDownscaled Representation Matters: Improving Image Rescaling with Collaborative Downscaled Images
Deep networks have achieved great success in image rescaling (IR) task that seeks to learn the optimal downscaled representations, i.e., low-resolution (LR) images, to reconstruct the original high-resolution (HR) images…
Image ReconstructionImage RescalingSuper-ResolutionBlind Super-Resolution Kernel Estimation using an Internal-GAN
Super resolution (SR) methods typically assume that the low-resolution (LR) image was downscaled from the unknown high-resolution (HR) image by a fixed 'ideal' downscaling kernel (e.g. Bicubic downscaling). However, this…
Blind Super-ResolutionSuper-Resolution