Even the Simplest Baseline Needs Careful Re-investigation: A Case Study on XML-CNN
The power and the potential of deep learning models attract many researchers to design advanced and sophisticated architectures. Nevertheless, the progress is sometimes unreal due to various possible reasons. In this work, through an astonishing example we argue that more efforts should be paid to ensure the progress in developing a new deep learning method. For a highly influential multi-label text classification method XML-CNN, we show that the superior performance claimed in the original paper was mainly due to some unbelievable coincidences. We re-examine XML-CNN and make a re-implementation which reveals some contradictory findings to the claims in the original paper. Our study suggests suitable baselines for multi-label text classification tasks and confirms that the progress on a new architecture cannot be confidently justified without a cautious investigation.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningMulti Label Text ClassificationMulti-Label Text Classificationtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
From Priest to Doctor: Domain Adaptaion for Low-Resource Neural Machine Translation
Many of the world's languages have insufficient data to train high-performing general neural machine translation (NMT) models, let alone domain-specific models, and often the only available parallel data are small amount…
Domain AdaptationLow Resource Neural Machine TranslationLow-Resource Neural Machine TranslationLow Resource NMT+2Majority-of-Three: The Simplest Optimal Learner?
Developing an optimal PAC learning algorithm in the realizable setting, where empirical risk minimization (ERM) is suboptimal, was a major open problem in learning theory for decades. The problem was finally resolved by …
Learning TheoryPAC learningFast and Furious Learning in Zero-Sum Games: Vanishing Regret with Non-Vanishing Step Sizes
We show for the first time, to our knowledge, that it is possible to reconcile in online learning in zero-sum games two seemingly contradictory objectives: vanishing time-average regret and non-vanishing step sizes. This…
Machine Learning with Multi-Site Imaging Data: An Empirical Study on the Impact of Scanner Effects
This is an empirical study to investigate the impact of scanner effects when using machine learning on multi-site neuroimaging data. We utilize structural T1-weighted brain MRI obtained from two different studies, Cam-CA…
BIG-bench Machine LearningAssessing the Limits of In-Context Learning beyond Functions using Partially Ordered Relation
Generating rational and generally accurate responses to tasks, often accompanied by example demonstrations, highlights Large Language Model's (LLM's) remarkable In-Context Learning (ICL) capabilities without requiring up…
In-Context LearningRelation