Choose What You Need: Disentangled Representation Learning for Scene Text Recognition Removal and Editing
Scene text images contain not only style information (font background) but also content information (character texture). Different scene text tasks need different information but previous representation learning methods use tightly coupled features for all tasks resulting in sub-optimal performance. We propose a Disentangled Representation Learning framework (DARLING) aimed at disentangling these two types of features for improved adaptability in better addressing various downstream tasks (choose what you really need). Specifically we synthesize a dataset of image pairs with identical style but different content. Based on the dataset we decouple the two types of features by the supervision design. Clearly we directly split the visual representation into style and content features the content features are supervised by a text recognition loss while an alignment loss aligns the style features in the image pairs. Then style features are employed in reconstructing the counterpart image via an image decoder with a prompt that indicates the counterpart's content. Such an operation effectively decouples the features based on their distinctive properties. To the best of our knowledge this is the first time in the field of scene text that disentangles the inherent properties of the text images. Our method achieves state-of-the-art performance in Scene Text Recognition Removal and Editing.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderRepresentation LearningScene Text RecognitionSimilar Papers 제목 키워드 기반
Choose What You Need: Disentangled Representation Learning for Scene Text Recognition, Removal and Editing
Scene text images contain not only style information (font, background) but also content information (character, texture). Different scene text tasks need different information, but previous representation learning metho…
DecoderRepresentation LearningScene Text RecognitionForecasting Future Videos from Novel Views via Disentangled 3D Scene Representation
Video extrapolation in space and time (VEST) enables viewers to forecast a 3D scene into the future and view it from novel viewpoints. Recent methods propose to learn an entangled representation, aiming to model layered …
Motion ForecastingNovel View SynthesisImage Generation and Translation with Disentangled Representations
Generative models have made significant progress in the tasks of modeling complex data distributions such as natural images. The introduction of Generative Adversarial Networks (GANs) and auto-encoders lead to the possib…
Conditional Image GenerationFace GenerationImage GenerationImage-to-Image Translation+2Linear Disentangled Representations and Unsupervised Action Estimation
Disentangled representation learning has seen a surge in interest over recent times, generally focusing on new models which optimise one of many disparate disentanglement metrics. Symmetry Based Disentangled Representati…
DisentanglementRepresentation LearningDEVIAS: Learning Disentangled Video Representations of Action and Scene
Video recognition models often learn scene-biased action representation due to the spurious correlation between actions and scenes in the training data. Such models show poor performance when the test data consists of vi…
Action RecognitionDecoderDisentanglementTemporal Action Localization+2