Using the Naive Bayes as a discriminative classifier
For classification tasks, probabilistic models can be categorized into two disjoint classes: generative or discriminative. It depends on the posterior probability computation of the label $x$ given the observation $y$, $p(x | y)$. On the one hand, generative classifiers, like the Naive Bayes or the Hidden Markov Model (HMM), need the computation of the joint probability p(x,y), before using the Bayes rule to compute $p(x | y)$. On the other hand, discriminative classifiers compute $p(x | y)$ directly, regardless of the observations' law. They are intensively used nowadays, with models as Logistic Regression, Conditional Random Fields (CRF), and Artificial Neural Networks. However, the recent Entropic Forward-Backward algorithm shows that the HMM, considered as a generative model, can also match the discriminative one's definition. This example leads to question if it is the case for other generative models. In this paper, we show that the Naive Bayes classifier can also match the discriminative classifier definition, so it can be used in either a generative or a discriminative way. Moreover, this observation also discusses the notion of Generative-Discriminative pairs, linking, for example, Naive Bayes and Logistic Regression, or HMM and CRF. Related to this point, we show that the Logistic Regression can be viewed as a particular case of the Naive Bayes used in a discriminative way.
Code (0)
등록된 구현이 없습니다.
Tasks
regressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Semi-Supervised Adaptive Discriminative Discretization Method Improving Discrimination Power of Regularized Naive Bayes
Recently, many improved naive Bayes methods have been developed with enhanced discrimination capabilities. Among them, regularized naive Bayes (RNB) produces excellent performance by balancing the discrimination power an…
Discriminative classification with generative features: bridging Naive Bayes and logistic regression
We introduce Smart Bayes, a new classification framework that bridges generative and discriminative modeling by integrating likelihood-ratio-based generative features into a logistic-regression-style discriminative class…
Deriving discriminative classifiers from generative models
We deal with Bayesian generative and discriminative classifiers. Given a model distribution $p(x, y)$, with the observation $y$ and the target $x$, one computes generative classifiers by firstly considering $p(x, y)$ and…
Revisiting Discriminative vs. Generative Classifiers: Theory and Implications
A large-scale deep model pre-trained on massive labeled or unlabeled data transfers well to downstream tasks. Linear evaluation freezes parameters in the pre-trained model and trains a linear classifier separately, which…
Few-Shot LearningImage ClassificationLinear evaluationregressionAre Generative Classifiers More Robust to Adversarial Attacks?
There is a rising interest in studying the robustness of deep neural network classifiers against adversaries, with both advanced attack and defence techniques being actively developed. However, most recent work focuses o…
Adversarial DefenseAdversarial Robustness