Finetuning Is a Surprisingly Effective Domain Adaptation Baseline in Handwriting Recognition
In many machine learning tasks, a large general dataset and a small specialized dataset are available. In such situations, various domain adaptation methods can be used to adapt a general model to the target dataset. We show that in the case of neural networks trained for handwriting recognition using CTC, simple finetuning with data augmentation works surprisingly well in such scenarios and that it is resistant to overfitting even for very small target domain datasets. We evaluated the behavior of finetuning with respect to augmentation, training data size, and quality of the pre-trained network, both in writer-dependent and writer-independent settings. On a large real-world dataset, finetuning provided an average relative CER improvement of 25 % with 16 text lines for new writers and 50 % for 256 text lines.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationDomain AdaptationHandwriting RecognitionSimilar Papers 제목 키워드 기반
Unsupervised Domain Adaptation within Deep Foundation Latent Spaces
The vision transformer-based foundation models, such as ViT or Dino-V2, are aimed at solving problems with little or no finetuning of features. Using a setting of prototypical networks, we analyse to what extent such fou…
Decision MakingDomain AdaptationUnsupervised Domain AdaptationOn Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline
In this paper, we examine the effectiveness of pre-training for visuo-motor control tasks. We revisit a simple Learning-from-Scratch (LfS) baseline that incorporates data augmentation and a shallow ConvNet, and find that…
BenchmarkingData AugmentationRethinking Unsupervised Domain Adaptation for Semantic Segmentation
Unsupervised domain adaptation (UDA) adapts a model trained on one domain (called source) to a novel domain (called target) using only unlabeled data. Due to its high annotation cost, researchers have developed many UDA …
Domain AdaptationSemantic SegmentationUnsupervised Domain AdaptationPowering Finetuning in Few-Shot Learning: Domain-Agnostic Bias Reduction with Selected Sampling
In recent works, utilizing a deep network trained on meta-training set serves as a strong baseline in few-shot learning. In this paper, we move forward to refine novel-class features by finetuning a trained deep network.…
Few-Shot LearningHow Far Can Pretrained LLMs Go in Symbolic Music? Controlled Comparisons of Supervised and Preference-based Adaptation
Music often shares notable parallels with language, motivating the use of pretrained large language models (LLMs) for symbolic music understanding and generation. Despite growing interest, the practical effectiveness of …
Domain Adaptation